Addressing Systems Engineering Challenges Through Collaborative Research

Transcription

1 Addressing Systems Engineering Challenges Through Collaborative Research June 2008 Dr. Donna H. Rhodes Massachusetts Institute of Technology

2 Field of Systems Engineering seari.mit.edu 2008 Massachusetts Institute of Technology 2

3 What is Systems Engineering? SYSTEMS ENGINEERING (Traditional) Systems engineering is the process of selecting and synthesizing the application of the appropriate scientific and technical knowledge in order to translate system requirements into system design. (Chase) SYSTEMS ENGINEERING (Advanced) Systems engineering is a branch of engineering that concentrates on design and application of the whole as distinct from the parts looking at the problem in its entirety, taking into account all the facets and variables and relating the social to the technical aspects. (Ramo) seari.mit.edu 2008 Massachusetts Institute of Technology 3

4 What is Systems Engineering? Systems Engineering is an interdisciplinary approach and means to enable the realization of successful systems. Systems Engineering integrates all the disciplines and specialty groups into a team effort forming a structured development process that proceeds from concept to production to operation. Systems Engineering considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs. International Council on Systems Engineering seari.mit.edu 2008 Massachusetts Institute of Technology 4

5 Where is Systems Engineering Needed? Products, Systems, Systems-of-Systems, Services seari.mit.edu 2008 Massachusetts Institute of Technology 5

6 Systems engineering is evolving as a broader and more multifaceted field, as the problems and challenges of this century are solved better by systems approaches, rather than through application of technology alone. (photo credit: INCOSE) Systems engineering is essential to successfully design, develop, and sustain the highly complex systems of the 21st century. seari.mit.edu 2008 Massachusetts Institute of Technology 6

7 Changing Face of Systems Engineering TRADITIONAL SE Transformation of customer requirements to design Requirements clearly specified, frozen early Emphasis on minimizing changes Design to meet well specified set of requirements Performance objectives specified at project start Focus on reliability, maintainability, and availability ADVANCED SE Effective transformation of stakeholder needs to fielded (and sustainable) solution Focus on product families and systems-of-systems Complex interdependencies of system and enterprise Growing importance of systems architecting Designing to accommodate change Emphasis on expanded set of ilities and designing in robustness, flexibility, adaptability in concept phase seari.mit.edu 2008 Massachusetts Institute of Technology 7

8 Motivations for Research in Advanced Systems Engineering seari.mit.edu 2008 Massachusetts Institute of Technology 8

9 Findings: DSB/AFSAB Report on Acquisition of National Security Space Programs May 2003 Cost has replaced mission success as the primary driver in managing space development programs Unrealistic estimates lead to unrealistic budgets and unexecutable programs Undisciplined definition and uncontrolled growth in system requirements increase cost and schedule delays Government capabilities to lead and manage the acquisition process have seriously eroded Industry has failed to implement proven practices on some programs seari.mit.edu 2008 Massachusetts Institute of Technology 9

10 Critical Need for Systems Engineering for Robustness In a 2004 workshop, Dr. Marvin Sambur, (then) Assistant Secretary of the AF for Acquisition, noted that average program is 36% overrun according to recent studies -- which disrupts the overall portfolio of programs. The primary reason cited in studies of problem programs state the number one reason for programs going off track is systems engineering. Systems Engineering for robustness means developing systems/system-of-systems that are: Capable of adapting to changes in mission and requirements Expandable/scalable Designed to accommodate growth in capability Able to reliably function given changes in threats and environment Effectively/affordably sustainable over their lifecycle Easily modified to leverage new technologies Reference: Rhodes, D., Workshop Report Air Force/LAI Workshop on Systems Engineering for Robustness, July 2004, seari.mit.edu 2008 Massachusetts Institute of Technology 10

11 Mr Yuri Bakhvalov, First Deputy Director General of the Khrunichev Space Centre on behalf of the Russian State Commission officially confirmed that the launch of CryoSat ended in a failure due to an anomaly in the launch sequence. missing command from the onboard flight control system. Today s Failures Exhibit Global Engineering Complexities October CryoSat Mission lost due to launch failure This loss means that Europe and the worldwide scientific community will not be able to rely on such data from the CryoSat mission and will not be able to improve their knowledge of ice, especially sea ice and its impact on climate change. Will this event have an impact on ESA s relationship with Russia? Space has always been a risky business. Failures can happen on each side. From this end I do not expect any impact on relations with Russia. I wish to underline that in this particular case we, ESA, were customers to Eurockot, the launch service provider, which is a joint venture between EADS Space Transportation (Germany) and Krunichev (Russia). seari.mit.edu 2008 Massachusetts Institute of Technology 11

12 Systems Engineering Continues to Be Cited as a Source of Problems DOD IG: Lack of systems engineering imperils missile system Published on Mar. 20, 2006 A lack of systems engineering plans could derail a $30 billion effort to field an integrated Ballistic Missile Defense System (BMDS), the Defense Department s inspector general said in a report released earlier this month. The Missile Defense Agency (MDA) has not completed a systems engineering plan or developed a sustainment plan for BMDS, jeopardizing the development of an integrated BMDS, the DOD IG said. The report emphasizes that DOD must practice strong systems engineering to effectively sustain weapons systems. That begins with design and development. seari.mit.edu 2008 Massachusetts Institute of Technology 12

13 Evolution of Practice of Systems Engineering Over the past five or six decades, the discipline known as Systems Engineering has evolved. At one time, many years ago, development of a capability was relatively simple to orchestrate. The design and development of parts, engineering calculations, assembly, and testing was conducted by a small number of people. Those days are long gone. Teams of people, sometimes numbering in the thousands are involved in the development of systems; and, what was previously only a development practice has evolved to become a science and engineering discipline. Saunders, T., et al, System-of-Systems Engineering for Air Force Capability Development: Executive Summary and Annotated Brief, AF SAB TR , 2005 seari.mit.edu 2008 Massachusetts Institute of Technology 13

14 Contemporary Systems Engineering Systems of systems Extended enterprises Network-centric paradigm Delivering value to society Sustainability of systems Design for flexibility Managing uncertainty Predictability of systems Spiral capable processes Model-based engineering This requires a broader field of study for future systems leaders and enabling changes in education and research and more seari.mit.edu 2008 Massachusetts Institute of Technology 14

15 Engineering Systems as the Context Field for Systems Engineering seari.mit.edu 2008 Massachusetts Institute of Technology 15

16 MIT Engineering Systems Division MIT is tackling the large-scale engineering challenges of the 21st century through a new organization. The Engineering Systems Division (ESD) creates and shares interdisciplinary knowledge about complex engineering systems through initiatives in education, research, and industry partnerships. Cross-cutting academic unit including engineering, management, social sciences Broadens engineering practice to include context of challenges as well as consequences of technological advancement Dual mission: (1) evolve engineering systems as new field of study and (2) transform engineering education and practice Council of 40+ universities is collaborating on this goal ( seari.mit.edu 2008 Massachusetts Institute of Technology 16

17 SYSTEMS ENGINEERING (Traditional) ES versus SE What Is the Difference? Systems engineering is the process of selecting and synthesizing the application of the appropriate scientific and technical knowledge in order to translate system requirements into system design. (Chase) SYSTEMS ENGINEERING (Advanced) Systems engineering is a branch of engineering that concentrates on design and application of the whole as distinct from the parts looking at the problem in its entirety, taking into account all the facets and variables and relating the social to the technical aspects. (Ramo) ENGINEERING SYSTEMS A field of study taking an integrative holistic view of large-scale, complex, technologically-enabled systems with significant enterprise level interactions and socio-technical interfaces. seari.mit.edu 2008 Massachusetts Institute of Technology 17

18 Engineering Systems as a Field of Study Economics, Statistics Systems Theory Operations Research /Systems Analysis System Architecture & Eng /Product Development ENGINEERING SYSTEMS Engineering Management Technology & Policy Organizational Theory Political Economy seari.mit.edu 2008 Massachusetts Institute of Technology 18

19 Engineering Systems Requires Four Perspectives 1. A very broad interdisciplinary perspective, embracing technology, policy, management science, and social science. 2. An intensified incorporation of system properties (such as sustainability, safety and flexibility) in the design process. Note that these are lifecycle properties rather than first use properties. These properties, often called ilities emphasize important intellectual considerations associated with long term use of engineering systems. 3. Enterprise perspective, acknowledging interconnectedness of product system with enterprise system that develops and sustains it. This involves understanding, architecting and developing organizational structures, policy system, processes, knowledgebase, and enabling technologies as part of the overall engineering system. 4. A complex synthesis of stakeholder perspectives, of which there may be conflicting and competing needs which must be resolved to serve the highest order system (system-of-system) need. seari.mit.edu 2008 Massachusetts Institute of Technology 19

20 Research Landscape A research landscape is the overall mental model under which research is formulated, performed, and transitioned to practice 1. Provides context for the research agenda, methods, and specific projects 2. Determines a community of interest for research impact 3. Opportunities for/constraints on funding sources and sponsors 4. Significantly influences research outcomes and impact Engineering systems is a field of study taking an integrative holistic view of large-scale, complex technologically enabled systems with significant enterprise level interactions and socio-technical interfaces seari.mit.edu 2008 Massachusetts Institute of Technology 20

21 Impact of Engineering Systems on Systems Engineering ES can provide a broader landscape (context field) for SE ES brings together a more diverse set of researchers and scholars ES establishes a larger footprint in the university, driving a strong research focus and investment The Engineering Systems Division provides the research venue for a new initiative on advanced systems engineering seari.mit.edu 2008 Massachusetts Institute of Technology 21

22 MIT ESD Systems Engineering Advancement Research Initiative (SEAri) seari.mit.edu 2008 Massachusetts Institute of Technology 22

23 Systems Engineering Advancement Research Initiative (SEAri) Mission Advance the theories, methods, and effective practice of systems engineering applied to complex sociotechnical systems through collaborative research Current Sponsors: US Air Force Office of Scientific Research, Singapore DSO, US Air Force, MIT Portugal Program, Draper Laboratory, Lean Advancement Initiative, US Government Agency 3 Cambridge Center NE20 388/343 seari.mit.edu 2008 Massachusetts Institute of Technology 23

24 Advancing SE Traditional Systems Engineering Advanced Systems Engineering Purpose System Architecture System Interoperability System ilities Acquisition and Management Anticipation of Needs Cost Development of single system to meet stakeholder requirements and defined performance System architecture established early in lifecycle; remains relatively stable Defines and implements specific interface requirements to integrate components in system Reliability, Maintainability, Availability are typical ilities Centralized acquisition and management of the system Concept phase activity to determine system needs Single or homogenous stakeholder group with stable cost/funding profile and similar measures of success Evolving new system of systems capability by leveraging synergies of legacy systems Dynamic adaptation of architecture as needs change Component systems can operate independently of SoS in a useful manner Protocols and standards essential to enable interoperable systems Enhanced emphasis on ilities such as Flexibility, Adaptability, Composeability SoS component systems separately acquired, and continue to be managed and operated as independent systems Intense concept phase analysis followed by continuous anticipation, aided by ongoing experimentation Multiple heterogeneous stakeholder groups with divergent cost goals and measures of success seari.mit.edu 2008 Massachusetts Institute of Technology 24

25 SEAri Structure Structured with four interacting clusters that undertake research in a portfolio of five topics: 1. Socio-Technical Decision Making 2. Designing for Value Robustness 3. Systems Engineering Economics 4. Systems Engineering in the Enterprise 5. Systems Engineering Strategic Guidance seari.mit.edu 2008 Massachusetts Institute of Technology 25

26 Research Cluster-Portfolio Mapping V-STARS R-STARS SE-Field SE-Synthesis Socio-Tech Decision Making X X X X Designing for Value Robustness X SE Economics X SE in the Enterprise X SE Strategic Guidance X seari.mit.edu 2008 Massachusetts Institute of Technology 26

27 Sponsor Engagement Models 1. Classical basic research sponsors Targeted topic toward broad scientific goals 2. Innovation grant sponsors Higher risk/higher payoff research 3. Contract research sponsors Toward solving sponsor problem 4. Consortium sponsors Pooled funds for shared research benefits 5. Deep engagement partnerships Symbiotic relationship seari.mit.edu 2008 Massachusetts Institute of Technology 27

28 Underlying Research Structure and Research Portfolio Prescriptive methods seek to advance state of the practice based on sound principles and theories, as grounded in real limitations and constraints Normative research: identify principles and theories -- should be Descriptive research: observe practice and identify limits/constraints RESEARCH PORTFOLIO 1. Socio-Technical Decision Making 2. Designing for Value Robustness 3. Systems Engineering Economics 4. Systems Engineering in the Enterprise 5. Systems Engineering Strategic Guidance seari.mit.edu 2008 Massachusetts Institute of Technology 28

29 Portfolio Projects Mapped to Research Structure Tradespace Exploration Method Validated in Real World Cases Codified Successful Practices based on Empirical Studies Guidance based on Research Synthesis Exploratory Studies of SE Practice seari.mit.edu 2008 Massachusetts Institute of Technology 29

30 Research Portfolio (1) SOCIO-TECHNICAL DECISION MAKING This area of research is concerned with the context of socio-technical systems. Based on a multi-disciplinary approach, decision making techniques are developed through the exploration of: Studies of decision processes and effectiveness of techniques Constructs for representing socio-technical systems to perform impact analysis Decision strategies for system of systems Visualization of complex trade spaces and saliency of information Understanding and mitigating cognitive biases in decision processes While organizational theorists have well developed theories of how organizations function and make decisions, this understanding needs to be integrated into the design phase in a quantifiable way.then it will be the case that apriori the effect of the enterprise organization on the engineering system will be predicted rather than being a surprise Hastings, MIT ESD Symposium, 2004 seari.mit.edu 2008 Massachusetts Institute of Technology 30

31 Formal Modeling of System of Systems Objective To validate through a formal model key heuristics related to the design and management of systems of systems (SoS) Method Synthesis of disparate case literature on SoS into core practitioner recommended practices Agent-based model that captures the essential features of an SoS vs. systems in general Anticipated Contributions Conditions under which various SoS practices are effective New potential heuristics focused on constituent behavior Network technologies have transformed the nature of systems RV $ = V + Af(SOSV) V SOSV kg kg kg SOS Value RV $ = V + Cg(SOSV) V $ $ $ SOSV Constituents struggle to manage participation in multiple SoS over time and across contexts SOSV RV $ = V + Bh(SOSV) V Time A B Faster Operations Tempo A A Centralized, integrated uni-function systems (2) (1) Flexible, multi-function Systems of systems SoS value is generated via interaction between constituent systems Participation impact constituents Contexts C C Globally Networked Environment A A D D Nirav B. Shah, Aero/Astro PhD Candidate, August 2008 (expected) seari.mit.edu 2008 Massachusetts Institute of Technology 31

32 Objective Demonstrate a proposed methodology for identifying areas of interest, or hotspots, in a system which may provide opportunities to embed flexibility Method Surveys and Interviews Construct Engineering Systems Matrix (ESM) Change Propagation Analysis & Sensitivity DSMs to identify hotspots Anticipated Contributions Demonstration of analysis using ESM Two case studies: detailed ESM and high-level ESM Recommendations for methodology improvements A Methodology for Identification of System Hotspots for Embedded Flexibility Jennifer Wilds, Aero/Astro and Technology & Policy Program SM, September 2008 seari.mit.edu 2008 Massachusetts Institute of Technology 32

33 Integrating Unmanned Aircraft into the National Airspace System: An Application of Value-Focused Thinking and Enterprise Architecting Objective Create a viable AF/FAA airspace integration enterprise Method Value-Focused Thinking Enterprise Architecting Design for Changeability Anticipated Contributions An applied analysis of valuefocused thinking to a real-world challenge using enterprise architecting principles to generate alternative approaches to maximize effectiveness/efficiency Objective State Architecting Enterprise Value As-Is Effort Luke Cropsey, ESD & Sloan School of Management SDM, September 2008 seari.mit.edu 2008 Massachusetts Institute of Technology 33

34 Research Portfolio (2) DESIGNING for VALUE ROBUSTNESS This area of research seeks to develop methods for concept exploration, architecting and design using a dynamic perspective for the purpose of realizing systems, products, and services that deliver sustained value to stakeholders in a changing world. Methods for dynamic multi-attribute trade space exploration Architecting principles and strategies for designing survivable systems Quantification of the changeability of a system design Techniques for the consideration of unarticulated and latent stakeholder value Taxonomy for enabling stakeholder dialogue on itlies Value robustness is the ability of a system to continue to deliver stakeholder value in the face of changing contexts and needs. Architecting value robust systems requires new methods for exploring the concept tradespace, as well as for decision making. Also needed are architecting principles and strategies, an approach for the quantification of changeability, and an improved ability for architects and analysts to classify value for purposes of dialogue and implementation Ross and Rhodes, 2008 seari.mit.edu 2008 Massachusetts Institute of Technology 34

35 Dynamic Multi-Attribute Tradespace Exploration Method Traditional trade studies insufficient for comprehensive conceptual design Tradespace exploration adds computer-based parametric models and simulations enabling comparison of hundreds/ thousands of architectures Can be applied to static case, but higher benefit through dynamic exploration Design transition rules can be applied to consider if and how to transition from one design to another Dynamic Tradespace Point designs in a tradespace can be linked as a network via transition rules to assess changeability seari.mit.edu 2008 Massachusetts Institute of Technology 35

36 Dynamic Multi-Attribute Tradespace Exploration Method Implications for Systems Engineering Practice 1. Ability to explore many design options and prevent too early focus on single point design 2. Enables quantitative assessment of factors such as variability in technical performance and cost, and impacts in markets 3. Suitable to multiple domains and demonstrated to improve decision making Vision: designers will have an enhanced ability to consider concept alternative in a rigorous way, not only for present situation but also in considering futures where needs and contexts have shifted seari.mit.edu 2008 Massachusetts Institute of Technology 36

37 Based on a networked tradespace, Ross (2006) defines filtered outdegree as quantification of subjective changeability Outdegree of a design is the number of transition paths from the design to possible future state designs Imposing a filter on the outdegree determines the only viable transition paths Changeability differs across decision makers based on thresholds for acceptable transition cost Quantification of Changeability of System Designs State 1 State 2 Filtered Outdegree A 1 2 Cost Cost Cost Cost Filtered Outdegree A B C A measure of changeability of a design as related to a decision maker's subjective threshold for acceptable cost of change seari.mit.edu 2008 Massachusetts Institute of Technology 37

38 What Strategies Can be Used to Achieve Value Robustness? New Context Drivers External Constraints Design Technologies Value expectations RESEARCH SUGGESTS TWO STRATEGIES FOR VALUE ROBUSTNESS 1. Passive Choose clever designs that remain high value Quantifiable: Pareto Trace number 2. Active Choose changeable designs that can deliver high value when needed Quantifiable: Filtered Outdegree Value robust designs can deliver value in spite of inevitable context change Dr. Adam M. Ross, PhD 2006, seari.mit.edu 2008 Massachusetts Institute of Technology 38 0Utility Utility Time Active Cost Passive T 1 T 2 Epoch 1 Epoch 2 S 1,b State 1 S 1,e S 2,b State 2 S 2,e DV 2 DV 1 DV 2 =DV 1 U Cost

39 Objective Application of MATE to a transportation engineering system Refine MATE so that it accounts for one or several of the following issues(*) found in transportation systems: different cost types, different stakeholder types, inheritance Method Economic analysis Case studies on transportation systems Anticipated Contributions Increase space and transp. professionals awareness of domain biases to support cross-domain knowledge-sharing Provide framework to space system designers to consider issues (*) not typically made explicit Possibly provide MATE framework applicable to transportation design and planning Concept factors Space Transportation Understanding of concept Inheritance Control over system Mainly physical Not adressed, soft inheritance plays important role Dispersed or central, decision makers negotiate and codesign system Not an issue Julia Nickel, ESD SM, June 2009 (expected) Physical and operational Common issue, addressed Central or dispersed at various degrees, codesign not always possible Common issue, addressed Compensation for losers Types of cost Monetary Multiple types of cost (monetary, environmental, other) seari.mit.edu 2008 Massachusetts Institute of Technology 39

40 Quantifying Flexibility in the Operationally Responsive Space Paradigm Objective Apply Dynamic Multi-Attribute Tradespace Exploration on ORS case study to test flexibility evaluation. Method Identify ORS paradigm utility curves Develop models and transition rules for flexibility Run simulations Anticipated Contributions Provide a useful flexibility metric for incorporating into ility trades Example Increasing Utility Curve Utility Prior Performance Baseline Attribute Example Decreasing Utility Curve Past performance as baseline for attribute expectations Utility Prior Performance Baseline Attribute Value of flexibility emerges from an uncertain future M 1 Optimal Design for M 1 M 5 Designs with 0< U(M) < 1 Utility = 0 M x Uncertain future mission M 4 M 3 Acceptable Transition path M 2 Optimal design has much more utility Mission with one design of U(M)>0 Lauren Viscito, Aero/Astro SM, June 2009 (Expected) seari.mit.edu 2008 Massachusetts Institute of Technology 40

41 Tradespace Exploration of Systems of Systems (SoS) Objective Method Apply Dynamic Tradespace Exploration method to SoS Identify challenges arising in SoS design and analysis of alternatives Extend Dynamic Multi-Attribute Tradespace Exploration method to address SoS-specific considerations Anticipated Contributions Validate Dynamic MATE as a framework for SoS tradespace exploration Provide a framework for testing SoS design heuristics suggested in literature Value Local and Global SoS Stakeholder Groups component system local stakeholders SoS component system local stakeholders global stakeholders component system local stakeholders Characterizing Time-Dependent SoS Value Delivery and Design Options SoS A SoS B SoS C SoS D... SoS A SoS B SoS N Switching Cost... Switching Costs Chattopadhyay, D., Ross, A.M., and Rhodes, D.H., A Framework for Tradespace Exploration of Systems of Systems, 6th Conference on Systems Engineering Research, Los Angeles, CA, April Time component systems Dynamic Strategy To Maintain SoS Value Delivery Debarati Chattopadhyay, Aero/Astro SM, June 2009 (expected) seari.mit.edu 2008 Massachusetts Institute of Technology 41

42 Architecting for Survivability Objective Method To develop and test a methodology for the conceptual design of survivable aerospace systems Multi-Attribute Tradespace Exploration Epoch-Era Analysis Anticipated Contributions Dynamic, value-centric conceptualization of survivability Set of general design principles for survivability Extensions of dynamic tradespace exploration to accommodate hostile and natural disturbances Evaluation of performance, cost, and survivability of alternative operationallyresponsive space capabilities Definition of Survivability Ability of a system to minimize the impact of a finite disturbance on value delivery through either (I) the reduction of the likelihood or magnitude of a disturbance or (II) the satisfaction of a minimally acceptable level of value delivery during and after a finite disturbance V(t) value original state V e emergency value threshold disturbance degradation disturbance duration T d Epoch 1a Epoch 2 Type II Survivability Epoch: Time period with a fixed context; characterized by static constraints, design concepts, available technologies, and articulated attributes (Ross 2006) Type I Survivability recovery T r permitted recovery time Epoch 1b Design Principles of Survivability 1.1 prevention 2.1 hardness 1.2 mobility 1.3 concealment 1.5 preemption 1.4 deterrence 1.6 avoidance 2.2 redundancy 2.3 margin 2.4 heterogeneity 2.5 distribution 2.6 failure mode reduction 2.7 fail-safe 2.8 evolution 2.9 containment 2.10 replacement 2.11 repair active passive V x required value threshold time Matthew Richards, ESD PhD Candidate, June 2009 (Expected) seari.mit.edu 2008 Massachusetts Institute of Technology 42

43 Epoch-Era Analysis for Evaluating System Timelines in Uncertain Futures Objective Apply Epoch-Era Analysis to large space system of a US Government Agency Method Enumerate future needs and contexts (incl. technology, policy, etc.) Develop models Run simulations Anticipated Contributions Validate dynamic analysis technique for evaluating system performance under large number of future contexts and needs Develop metric for representing dynamic system success across changing futures: utopia trajectory Changing Futures Impact on System Epoch Dynamic Strategies for Systems Two aspects to an Epoch: 1. Needs (expectations) 2. Context (constraints, etc.) Adam M. Ross, PhD and Donna H. Rhodes, PhD Government Agency Sponsored Project seari.mit.edu 2008 Massachusetts Institute of Technology 43

44 Research Portfolio (3) SYSTEMS ENGINEERING ECONOMICS This research area aims at developing a new paradigm that encompasses an economics view of systems engineering to achieve measurable and predictable outcomes while delivering value to stakeholders. Measurement of productivity and quantifying SE ROI Advanced methods for reuse, cost modeling, and risk modeling Application of real options in systems and enterprises Leading indicators for systems engineering effectiveness In a 2004 Air Force/MIT workshop, Dr. Marvin Sambur, (then) Assistant Secretary of the AF for Acquisition, noted that the average program is 36% overrun according to recent studies disrupting the overall portfolio of programs seari.mit.edu 2008 Massachusetts Institute of Technology 44

45 Models, Measures, and Leading Indicators for Project Success Through Better Execution of Systems Engineering Cost and schedule modeling Project Risk Assessment Person Months Confidence (Cumulative Probability) Risk (= Prob. That Actual Person Months Will Exceed Indicated, X-Axis, Figure) 100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% 100% 95% 90% 85% 80% 75% Person Months 70% Risk 65% 60% 55% 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Cumulative Probability of Person Months Person Months Person Months Leading Indicators for Performance Systems Engineering ROI seari.mit.edu 2008 Massachusetts Institute of Technology 45

46 Research Portfolio (4) SYSTEMS ENGINEERING in the ENTERPRISE This research area involves empirical studies and case based research for the purpose of understanding how to achieve more effective systems engineering practice in context of the nature of the system being developed, external context, and the characteristics of the associated enterprise. Engineering systems thinking in individuals and teams Collaborative, distributed systems engineering practices Social contexts of enterprise systems engineering Alignment of enterprise culture and processes Socio-technical systems studies and models The understanding of the organizational and technical interactions in our systems, emphatically including the human beings who are a part of them, is the present-day frontier of both engineering education and practice. Dr. Michael D. Griffin, Administrator, NASA, 2007 Boeing Lecture, Purdue University seari.mit.edu 2008 Massachusetts Institute of Technology 46

47 Collaborative Distributed Systems Engineering Utter (2007) performed empirical case studies to identify successful practices and lessons learned Social and technical factors studied: collaboration scenarios, tools, knowledge and decision management, culture, motivations, others Can not be achieved without first overcoming possible barriers and issues Preliminary set of success factors identified Success Factor: Invest in Up-front Planning Activities Spending more time on the front- end activities and gaining team consensus shortens the implementation cycle. It avoids pitfalls as related to team mistrust, conflict, and mistakes that surface during implementation. seari.mit.edu 2008 Massachusetts Institute of Technology 47

48 It is not enough to understand systems thinking in individuals but also how it emerges in groups and enterprises Collaborative Systems Thinking Lamb (2008) performing empirical studies - focus on interaction of process and culture Research seeks to identify promising patterns that can lead to larger cross-cutting studies Pilot interviews have provided insights to inform the study Factors in Collaborative Systems Thinking: These traits are not necessarily of one individual but emerge through interactions of a group of individuals as influenced by culture, team norms, environment, and processes seari.mit.edu 2008 Massachusetts Institute of Technology 48

49 Objective Method How can real options be used for holistic decision making and architecting of enterprises under uncertainty? Model enterprise views Identify potential mechanisms and types of options that encompass enterprise views Use real options valuation toolbox Anticipated Contributions Framework for systematic exploration and valuation of mechanisms and types of real options that deal with enterprise level uncertainties, enabled through holistic modeling of enterprise dependencies. An Integrated Approach to Real Options Analysis in Socio-Technical Enterprises Enterprise views Enterprise views Tsoline Mikaelian, Aero/Astro PhD Candidate, June 2009 (expected) seari.mit.edu 2008 Massachusetts Institute of Technology 49

50 Research Portfolio (5) SYSTEMS ENGINEERING STRATEGIC GUIDANCE This research area involves synthesis of theory with empirical and case based research for the purpose of developing prescriptive strategic guidance to inform the development of policies and procedures for systems engineering in practice. Systems Engineering research guidelines Participation in focus groups and pilot-phase reviews Position papers on proposed policies Recommendations for integrating SE research into curriculum Identification of SE research gaps and opportunities The full impact of systems engineering research can only be achieved through synthesis of research outcomes seari.mit.edu 2008 Massachusetts Institute of Technology 50

51 Influencing Proposed Policy As a first step toward a prescriptive model, the Department of Defense (DoD) is developing a Guide to System of Systems Engineering (SoSE) based on best present state knowledge The guide provides 16 DoD technical and management processes to help sponsors, program managers, and chief engineers address the unique considerations for DoD SoS SEAri participated as an academic review body in the pilot phase of the development of the guide A recent position paper by SEAri researchers included recommendations for how a normative, descriptive and prescriptive framework can be used to contribute to evolving SoSE guidance Valerdi, R., Ross, A., and Rhodes, D., A Framework for Evolving System of Systems Engineering, Crosstalk: The Journal of Defense Software Engineering, 2007 seari.mit.edu 2008 Massachusetts Institute of Technology 51

52 Collaborative Research Imperatives and Example Projects seari.mit.edu 2008 Massachusetts Institute of Technology 52

53 Imperative Engineering research while still dependent upon individual contributors must evolve to be more synergistic Our society is faced with large scale problems demanding a multi-faceted and interdisciplinary systems approach Requires researchers from diverse disciplines to collaboratively work on problems using shared data sets and aligning around harmonized research threads Need to understand how to synthesize individual research efforts, with good mechanisms for research succession planning and transition of research to practice We strive for research leading to sustainable engineering systems meeting broad societal needs we are challenged by current policies, funding approach, and traditional university/research stovepipes seari.mit.edu 2008 Massachusetts Institute of Technology 53

54 Imperative Engineering education and research must be a collaborative endeavor of government, industry, and academia Complex engineering research can not take place solely in a laboratory within university walls but rather real world enterprises must be our learning laboratories Expanded view of who an educator is -- faculty, researchers, practitioners, policy makers, peers Additionally, we need more cross cutting experiences for educators and practitioners alike Faculty have a very urgent need for case studies for use in the classroom without practitioner involvement these will lack depth to have educational impact Engineering education and research can not be just a cooperation; must be a true collaboration seari.mit.edu 2008 Massachusetts Institute of Technology 54

55 AF/DOD SE Revitalization Policies + AF/LAI Workshop on Systems Engineering June 2004 Systems Engineering Leading Indicators Initiative Collaboration of LAI, INCOSE, SEAri, PSM SE LI Working Group + With SSCI and PSM BETA Guide to SE Leading Indicators (December 2005) The leading indicators project is an excellent example of how academic, government, and industry experts can work together to perform collaborative research that has real impact on engineering practice INDUSTRIAL ENGINEER MAGAZINE March 2007 Pilot Programs (several companies) Masters Thesis (1 case study) Validation Survey (>100 responses/ one corporation) Practical Software & Systems Measurement Workshops (1) July 2005 (2) July 2007 (3) July 2008 SE LI Working Group + With SSCI and PSM V. 1.0 Guide to SE Leading Indicators June 2007 Applications IBM Rational Method Composer RUP Measurement Plug-in Knowledge Exchange Event Tutorial on SE Leading Indicators (many companies) (1) January 2007 (2) November 2007 seari.mit.edu 2008 Massachusetts Institute of Technology 55

56 Draper /MIT Research Dynamic Tradespace Exploration Applied to System of Systems Extending Research to Enhance Practice New research launched in July 2007 (first Draper project with MIT ESD) to extend work of Ross (2006) University research project coupled to related in-house IR&D project Leverage geographic co-location for highly interactive research engagement Mutual benefit Enhance Draper capabilities and processes Further validate and extend MIT methodology Collaborative learning seari.mit.edu 2008 Massachusetts Institute of Technology 56

57 Knowledge Sharing seari.mit.edu 2008 Massachusetts Institute of Technology 57

58 Access to Research Websites are important mechanisms for sharing knowledge and also emerging as powerful collaboration venues seari.mit.edu 2008 Massachusetts Institute of Technology 58

59 Sharing Research Outcomes 2007 SEAri Research Summit October 16 MIT Faculty Club SEARI Research Bulletin Published at End of Each Semester 2008 SEAri Research Summit October 21 seari.mit.edu 2008 Massachusetts Institute of Technology 59

60 Summary seari.mit.edu 2008 Massachusetts Institute of Technology 60

61 SEAri Seeks To Impact Theory, Methods, And Practice MIT Engineering Systems Division (ESD) provides an interdisciplinary research venue Strategic collaboration with other MIT education and research centers (e.g., LAI, SDM) Hybrid research model for collaboration Single sponsor research projects Consortium research Realization of research goals is predicated on deep collaboration with industry and government seari.mit.edu 2008 Massachusetts Institute of Technology 61

62 Additional References ESD Website ESD Research Centers ESD Working Papers ESD Symposium Monographs and Papers Lean Advancement Initiative Refer to websites for additional information and working papers related to systems engineering at MIT seari.mit.edu 2008 Massachusetts Institute of Technology 62

63 QUESTIONS?