Addressing Systems Engineering Challenges Through Collaborative Research

Transcription

1 Addressing Systems Engineering Challenges Through Collaborative Research October 2007 Dr. Donna H. Rhodes Massachusetts Institute of Technology

2 Field of Systems Engineering Massachusetts Institute of Technology 2

3 What is Systems Engineering? SYSTEMS ENGINEERING (Traditional) Systems engineering is the process of selecting and synthesizing the application of the appropriate scientific and technical knowledge in order to translate system requirements into system design. (Chase) Massachusetts Institute of Technology 3

4 What is Systems Engineering? SYSTEMS ENGINEERING (Advanced) Systems engineering is a branch of engineering that concentrates on design and application of the whole as distinct from the parts looking at the problem in its entirety, taking into account all the facets and variables and relating the social to the technical aspects. (Ramo) Massachusetts Institute of Technology 4

5 Changing Face of Systems Engineering TRADITIONAL SE Transformation of customer requirements to design Requirements clearly specified, frozen early Emphasis on minimizing changes Design to meet well specified set of requirements Performance objectives specified at project start Focus on reliability, maintainability, and availability ADVANCED SE Effective transformation of stakeholder needs to fielded (and sustainable) solution Focus on product families and systems-of-systems Complex interdependencies of system and enterprise Growing importance of systems architecting Designing to accommodate change Emphasis on expanded set of ilities and designing in robustness, flexibility, adaptability in concept phase Massachusetts Institute of Technology 5

6 What is Systems Engineering? Systems Engineering is an interdisciplinary approach and means to enable the realization of successful systems. Systems Engineering integrates all the disciplines and specialty groups into a team effort forming a structured development process that proceeds from concept to production to operation. Systems Engineering considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs. International Council on Systems Engineering Massachusetts Institute of Technology 6

7 Motivations for Research in Advanced Systems Engineering Massachusetts Institute of Technology 7

8 Findings: DSB/AFSAB Report on Acquisition of National Security Space Programs May 2003 Cost has replaced mission success as the primary driver in managing space development programs Unrealistic estimates lead to unrealistic budgets and unexecutable programs Undisciplined definition and uncontrolled growth in system requirements increase cost and schedule delays Government capabilities to lead and manage the acquisition process have seriously eroded Industry has failed to implement proven practices on some programs Massachusetts Institute of Technology 8

9 Critical Need for Systems Engineering for Robustness In a 2004 workshop, Dr. Marvin Sambur, (then) Assistant Secretary of the AF for Acquisition, noted that average program is 36% overrun according to recent studies -- which disrupts the overall portfolio of programs. The primary reason cited in studies of problem programs state the number one reason for programs going off track is systems engineering. Systems Engineering for robustness means developing systems/system-of-systems that are: Capable of adapting to changes in mission and requirements Expandable/scalable Designed to accommodate growth in capability Able to reliably function given changes in threats and environment Effectively/affordably sustainable over their lifecycle Easily modified to leverage new technologies Reference: Rhodes, D., Workshop Report Air Force/LAI Workshop on Systems Engineering for Robustness, July 2004, Massachusetts Institute of Technology 9

10 Mr Yuri Bakhvalov, First Deputy Director General of the Khrunichev Space Centre on behalf of the Russian State Commission officially confirmed that the launch of CryoSat ended in a failure due to an anomaly in the launch sequence. missing command from the onboard flight control system. Today s Failures Exhibit Global Engineering Complexities October CryoSat Mission lost due to launch failure This loss means that Europe and the worldwide scientific community will not be able to rely on such data from the CryoSat mission and will not be able to improve their knowledge of ice, especially sea ice and its impact on climate change. Will this event have an impact on ESA s relationship with Russia? Space has always been a risky business. Failures can happen on each side. From this end I do not expect any impact on relations with Russia. I wish to underline that in this particular case we, ESA, were customers to Eurockot, the launch service provider, which is a joint venture between EADS Space Transportation (Germany) and Krunichev (Russia) Massachusetts Institute of Technology 10

11 Systems Engineering Continues to Be Cited as a Source of Problems DOD IG: Lack of systems engineering imperils missile system Published on Mar. 20, 2006 A lack of systems engineering plans could derail a $30 billion effort to field an integrated Ballistic Missile Defense System (BMDS), the Defense Department s inspector general said in a report released earlier this month. The Missile Defense Agency (MDA) has not completed a systems engineering plan or developed a sustainment plan for BMDS, jeopardizing the development of an integrated BMDS, the DOD IG said. The report emphasizes that DOD must practice strong systems engineering to effectively sustain weapons systems. That begins with design and development Massachusetts Institute of Technology 11

12 Evolution of Practice of Systems Engineering Over the past five or six decades, the discipline known as Systems Engineering has evolved. At one time, many years ago, development of a capability was relatively simple to orchestrate. The design and development of parts, engineering calculations, assembly, and testing was conducted by a small number of people. Those days are long gone. Teams of people, sometimes numbering in the thousands are involved in the development of systems; and, what was previously only a development practice has evolved to become a science and engineering discipline. Saunders, T., et al, System-of-Systems Engineering for Air Force Capability Development: Executive Summary and Annotated Brief, AF SAB TR , Massachusetts Institute of Technology 12

13 Contemporary Systems Engineering Systems of systems Extended enterprises Network-centric paradigm Delivering value to society Sustainability of systems Design for flexibility Managing uncertainty Predictability of systems Spiral capable processes Model-based engineering This requires a broader field of study for future systems leaders and enabling changes in education and research and more Massachusetts Institute of Technology 13

14 MIT Venue for Systems Education and Research Massachusetts Institute of Technology 14

15 MIT Engineering Systems Division as Intellectual Home for Systems Research MIT is tackling the large-scale engineering challenges of the 21st century through a new organization. The Engineering Systems Division (ESD) creates and shares interdisciplinary knowledge about complex engineering systems through initiatives in education, research, and industry partnerships. Cross-cutting academic unit including engineering, management, social sciences Broadens engineering practice to include context of challenges as well as consequences of technological advancement Dual mission: (1) evolve engineering systems as new field of study and (2) transform engineering education and practice Council of 40+ universities is collaborating on this goal ( Massachusetts Institute of Technology 15

16 SYSTEMS ENGINEERING (Traditional) ES versus SE What Is the Difference? Systems engineering is the process of selecting and synthesizing the application of the appropriate scientific and technical knowledge in order to translate system requirements into system design. (Chase) SYSTEMS ENGINEERING (Advanced) Systems engineering is a branch of engineering that concentrates on design and application of the whole as distinct from the parts looking at the problem in its entirety, taking into account all the facets and variables and relating the social to the technical aspects. (Ramo) ENGINEERING SYSTEMS A field of study taking an integrative holistic view of large-scale, complex, technologically-enabled systems with significant enterprise level interactions and socio-technical interfaces Massachusetts Institute of Technology 16

17 Engineering Systems as a Field of Study Economics, Statistics Systems Theory Operations Research /Systems Analysis System Architecture & Eng /Product Development ENGINEERING SYSTEMS Engineering Management Technology & Policy Organizational Theory Political Economy Massachusetts Institute of Technology 17

18 Engineering Systems Requires Four Perspectives 1. A very broad interdisciplinary perspective, embracing technology, policy, management science, and social science. 2. An intensified incorporation of system properties (such as sustainability, safety and flexibility) in the design process. Note that these are lifecycle properties rather than first use properties. These properties, often called ilities emphasize important intellectual considerations associated with long term use of engineering systems. 3. Enterprise perspective, acknowledging interconnectedness of product system with enterprise system that develops and sustains it. This involves understanding, architecting and developing organizational structures, policy system, processes, knowledgebase, and enabling technologies as part of the overall engineering system. 4. A complex synthesis of stakeholder perspectives, of which there may be conflicting and competing needs which must be resolved to serve the highest order system (system-of-system) need Massachusetts Institute of Technology 18

19 Impact of Engineering Systems on Systems Engineering ES can provide a broader landscape (context field) for SE ES brings together a more diverse set of researchers and scholars ES establishes a larger footprint in the university, driving a strong research focus and investment The Engineering Systems Division provides the research venue for a new initiative on advanced systems engineering Massachusetts Institute of Technology 19

20 MIT Research Initiative in Advanced Systems Engineering Massachusetts Institute of Technology 20

21 Systems Engineering Advancement Research Initiative (SEAri) Mission Advance the theories, methods, and effective practice of systems engineering applied to complex sociotechnical systems through collaborative research Current Sponsors: US Air Force Office of Scientific Research, Singapore Defense Sciences Office, US Air Force, Aerospace Corporation, MITRE Corporation, NASA, MIT Portugal Program, Draper Laboratory, Lean Aerospace Initiative 3 Cambridge Center NE20 388/ Massachusetts Institute of Technology 21

22 Traditional Systems Engineering Advanced Systems Engineering Purpose System Architecture System Interoperability System ilities Acquisition and Management Anticipation of Needs Cost Development of single system to meet stakeholder requirements and defined performance System architecture established early in lifecycle; remains relatively stable Defines and implements specific interface requirements to integrate components in system Reliability, Maintainability, Availability are typical ilities Centralized acquisition and management of the system Concept phase activity to determine system needs Single or homogenous stakeholder group with stable cost/funding profile and similar measures of success Evolving new system of systems capability by leveraging synergies of legacy systems Dynamic adaptation of architecture as needs change Component systems can operate independently of SoS in a useful manner Protocols and standards essential to enable interoperable systems Enhanced emphasis on ilities such as Flexibility, Adaptability, Composeability SoS component systems separately acquired, and continue to be managed and operated as independent systems Intense concept phase analysis followed by continuous anticipation, aided by ongoing experimentation Multiple heterogeneous stakeholder groups with divergent cost goals and measures of success Massachusetts Institute of Technology 22

23 Structured with four interacting clusters that undertake research in a portfolio of five topics: 1. Socio-Technical Decision Making 2. Designing for Value Robustness 3. Systems Engineering Economics 4. Systems Engineering in the Enterprise 5. Systems Engineering Strategic Guidance Products and Services Normative research Prescriptive research System Design V-STARS Value-driven V- STARS Descriptive research Military and Security e.g., Dynamic MATE Design for Value Robustness Tradespace Value-driven visualizations Workshops, Tutorials, White papers, Conferences, Journals Commercial Systems SE- Synthesis e.g., SE Strategy- Education SE Policy generating SE-Field Research SE-Field Empirically relevant e.g., Empirically-relevant Leading indicators Collaborative distributed SE Enablers to sys thinking Enterprises R-STARS Resource-effective R-STARS e.g., Uncertainty assessment & management Resource-effective usage Cost estimating Knowledge Deployment Systems of Systems Applications Theory Theory-based based Practice Practice-based based Massachusetts Institute of Technology 23

24 Research Cluster-Portfolio Mapping V-STARS R-STARS SE-Field SE-Synthesis Socio-Tech Decision Making X X X X Designing for Value Robustness X SE Economics X SE in the Enterprise X SE Strategic Guidance X Massachusetts Institute of Technology 24

25 Research Portfolio (1) SOCIO-TECHNICAL DECISION MAKING This area of research is concerned with the context of socio-technical systems. Based on a multi-disciplinary approach, decision making techniques are developed through the exploration of: Studies of decision processes and effectiveness of techniques Constructs for representing socio-technical systems to perform impact analysis Decision strategies for coupling in system of systems Visualization of complex trade spaces and saliency of information Understanding and mitigating cognitive biases in decision processes Massachusetts Institute of Technology 25

26 Mi cr o How Can Socio-technical Systems be Represented for Analysis and Screening for Real Options? Lighter Power Supplies UA V SYSTEM DRIVERS DSM Engineering System Matrix (ESM) Lighter Targeting System Battlefield Communications Future AF Ground Operator Vulnerability to change? high low STAKEHOLDERS DSM OBJECTIVES DSM SYSTEM BOUNDARY... FUNCTIONS DSM.... OBJECTS DSM.... ACTIVITIES DSM Dr. Jason Bartolomei, PhD Massachusetts Institute of Technology 26

27 Uncertainty Management Real Options in Enterprise Architecture Tsoline Mikaelian, Aero/Astro PhD 2009 What enterprise representation/models can be used to identify potential real option investment opportunities? How can you quantify the value of real options in enterprises to enable the selection of an options portfolio in enterprise decision making? Benefit Calculation: Prescriptive Analysis (Hot/Cold Spot Analysis) Wing Connector Tail Connector Interchangeable Battery Module Benefit (utils,$) Cost Calculation: Measure of Uncertainty/Volatility High p High p High Low pbenefit High pcost High Low Benefit Cost High Low Benefit Benefit Low Cost Low phigh Low Cost p Low Low pcost Low High pcost High Low Benefit Benefit High Low Cost Benefit Low Cost High Benefit Cost (effort,$) Uncertainty/Volatility Measure: Engineering Systems Matrix for Real Options Analysis Jennifer Wilds, SM Aero/Astro and TPP 2008 How can the Engineering Systems Matrix (ESM) be applied for understanding real options in complex systems? Sensitivity Analysis Network Analysis Forecast Massachusetts Institute of Technology 27

28 When Should Systems Use Tight or Loose Coupling? Tight Coupling Loose Coupling No Coupling Loosely coupling is approach to designing interfaces across constituent systems to reduce the interdependencies across constituent systems Seeks to increase flexibility in adding constituent systems, replacing constituent systems, changing operations within constituent systems and re-architecting the SOS A way to manage tension between global and local value in SOS design Nirav Shah, PhD Candidate, Massachusetts Institute of Technology 28

29 Research Portfolio (2) DESIGNING for VALUE ROBUSTNESS This area of research seeks to develop methods for concept exploration, architecting and design using a dynamic perspective for the purpose of realizing systems, products, and services that deliver sustained value to stakeholders in a changing world. Methods for dynamic multi-attribute trade space exploration Architecting principles and strategies for designing survivable systems Quantification of the changeability of a system design Techniques for the consideration of unarticulated and latent stakeholder value Taxonomy for enabling stakeholder dialogue on ilities Massachusetts Institute of Technology 29

30 How Can Dynamic Tradespaces be Explored? Utility Tradespace Networks Utility Transition rules Cost Tradespace designs = nodes Applied transition rules = arcs Cost Cost Transition rules are mechanisms to change one design into another The more outgoing arcs, the more potential change mechanisms Dr. Adam M. Ross, PhD 2006, adamross@mit.edu Massachusetts Institute of Technology 30

31 How Can Changeability of a Design be Measured? objective Outdegree # outgoing arcs from a given node subjective Filtered Outdegree # outgoing arcs from design at acceptable cost (measure of changeability) OD(<C) OD(< Ĉ ) R K+1 R K R K+1 OD K <Ĉ >Ĉ >Ĉ OD( Ĉ ) Outdegree Ĉ Subjective Filter Cost C Filtered outdegree is a measure of the apparent changeability of a design Dr. Adam M. Ross, PhD 2006, adamross@mit.edu Massachusetts Institute of Technology 31

32 What Strategies Can be Used to Achieve Value Robustness? New Context Drivers External Constraints Design Technologies Value expectations RESEARCH SUGGESTS TWO STRATEGIES FOR VALUE ROBUSTNESS 1. Passive Choose clever designs that remain high value Quantifiable: Pareto Trace number 2. Active Choose changeable designs that can deliver high value when needed Quantifiable: Filtered Outdegree Value robust designs can deliver value in spite of inevitable context change Dr. Adam M. Ross, PhD 2006, Massachusetts Institute of Technology 32 0Utility Utility Time Active Cost Passive T 1 T 2 Epoch 1 Epoch 2 S 1,b State 1 S 1,e S 2,b State 2 S 2,e DV 2 DV 1 DV 2 =DV 1 U Cost

33 What are the Principles for Architecting for Survivability? The interdependence of large-scale, distributed systems has grown since the advent of modern telecommunications V(t) value original state disturbance Epoch: Time period during with a with fixed a context; fixed characterized context; characterized by static by static constraints, design concepts, available technologies, and articulated available technologies, attributes (Ross and 2006) articulated attributes (Ross 2006) recovered state Engineering systems are increasingly at risk from disturbances that rapidly propagate through networks, damage critical infrastructure, and undermine system-ofsystems V e emergency value threshold Epoch 1a Epoch 2 recovery T r permitted recovery time Epoch 1b V x expected value threshold time Matt Richards, PhD Student, Massachusetts Institute of Technology 33

34 Research Portfolio (3) SYSTEMS ENGINEERING ECONOMICS This research area aims at developing a new paradigm that encompasses an economics view of systems engineering to achieve measurable and predictable outcomes while delivering value to stakeholders. Measurement of productivity and quantifying SE ROI Advanced methods for reuse, cost modeling, and risk modeling Application of real options in systems and enterprises Leading indicators for systems engineering effectiveness Massachusetts Institute of Technology 34

35 How does resource allocation and transferal affect local and global SoS value? RV $ = V A+ f(sosv V (SOSV) $ $ $ SOSV SOSV kg kg kg SOS Value RV = V + h(sosv) $ B V $ C V RV = V + g(sosv) SOSV Nirav Shah, PhD Candidate, Massachusetts Institute of Technology 35

36 Models, Measures, and Leading Indicators for Project Success Through Better Execution of Systems Engineering Cost and schedule modeling Project Risk Assessment Person Months Confidence (Cumulative Probability) Risk (= Prob. That Actual Person Months Will Exceed Indicated, X-Axis, Figure) 100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% 100% 95% 90% 85% 80% 75% Person Months 70% Risk 65% 60% 55% 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Cumulative Probability of Person Months Person Months Person Months Leading Indicators for Performance Systems Engineering ROI Massachusetts Institute of Technology 36

37 Research Portfolio (4) SYSTEMS ENGINEERING in the ENTERPRISE This research area involves empirical studies and case based research for the purpose of understanding how to achieve more effective systems engineering practice in context of the nature of the system being developed, external context, and the characteristics of the associated enterprise. Engineering systems thinking in individuals and teams Collaborative, distributed systems engineering practices Social contexts of enterprise systems engineering Alignment of enterprise culture and processes Socio-technical systems studies and models Massachusetts Institute of Technology 37

38 The understanding of the organizational and technical interactions in our systems, emphatically including the human beings who are a part of them, is the present-day frontier of both engineering education and practice. Dr. Michael D. Griffin, Administrator, NASA Boeing Lecture, Purdue University 28 March Massachusetts Institute of Technology 38

39 Enabling Systems Thinking to Accelerate the Development of Senior Systems Engineers Even though systems thinking definitions diverge, there is consensus on primary mechanisms that enable or obstruct systems thinking development in engineers Dr. Heidi Davidz, PhD 2006 Consensus on primary mechanisms that enable or obstruct systems thinking development in engineers 1. Experiential learning 2. Individual characteristics 3. Supportive environment Massachusetts Institute of Technology 39

40 Collaborative Systems Thinking Aligning Culture and Standardized Process Examines the development of systems thinking within teams of engineers. Emphasis placed on the role of standard process and its interactions with organizational culture. Research motivated by desire to better understand systems thinking at the team level within engineering. Focuses on the role of standardized process, its artifacts and associated tools, in enabling or promoting team level systems thinking termed collaborative systems thinking. How do culture and process enable collaborative systems thinking? Culture Collaborative Systems Thinking How do engineering processes interact with culture? Standardized Process Caroline Twomey Lamb, PhD Student, Massachusetts Institute of Technology 40

41 Collaborative Distributed Systems Engineering in the Aerospace Industry Develop heuristics for successful CDSE resulting from case studies Recommendations to overcome barriers to successful CDSE Recommendations for future work in this area Social Factors: Local/Company Culture Differences Career Development Advancement Distributed Teamwork Communications Working w/ IT and tools Project Management CDSE Technical Factors: SE Process/Architecture Distributed Decision Making Tools/Information Technology Knowledge Management Cost & Schedule Product Impact Darlene Utter, S.M Massachusetts Institute of Technology 41

42 Research Portfolio (5) SYSTEMS ENGINEERING STRATEGIC GUIDANCE This research area involves synthesis of theory with empirical and case based research for the purpose of developing prescriptive strategic guidance to inform the development of policies and procedures for systems engineering in practice. Systems Engineering research guidelines Participation in focus groups and pilot-phase reviews Position papers on proposed policies Recommendations for integrating SE research into education curriculum Identification of SE research gaps and opportunities Massachusetts Institute of Technology 42

43 MIT-Portugal Engineering Systems Education The Portuguese Government, through the Ministry of Science, Technology and Higher Education, is engaged in a long-term collaboration with MIT focusing on basic research and education Project is part of the anchor program within the overall effort, of which one objective is: in anticipation of a future workforce that will have to "think differently," create educational material on engineering systems that can be taught in Portuguese schools and that can be incorporated in the educational initiatives underway in the separate focus areas SEAri undertook a project in summer 2007 to develop education materials in the area of architecting and design decision making, based on its research in this area Modular set of teaching materials developed, along with several recommended options for the future packaging and integration of the material into existing curricula and courses Massachusetts Institute of Technology 43

44 Influencing Proposed Policy As a first step toward a prescriptive model, the Department of Defense (DoD) is developing a Guide to System of Systems Engineering (SoSE) based on best present state knowledge The guide provides 16 DoD technical and management processes to help sponsors, program managers, and chief engineers address the unique considerations for DoD SoS SEAri participated as an academic review body in the pilot phase of the development of the guide A recent position paper by SEAri researchers included recommendations for how a normative, descriptive and prescriptive framework can be used to contribute to evolving SoSE guidance Valerdi, R., Ross, A., and Rhodes, D., A Framework for Evolving System of Systems Engineering, Crosstalk: The Journal of Defense Software Engineering, Massachusetts Institute of Technology 44

45 Collaborative Research Imperatives and Example Projects Massachusetts Institute of Technology 45

46 Imperative Engineering research while still dependent upon individual contributors must evolve to be more synergistic Our society is faced with large scale problems demanding a multi-faceted and interdisciplinary systems approach Requires researchers from diverse disciplines to collaboratively work on problems using shared data sets and aligning around harmonized research threads Need to understand how to synthesize individual research efforts, with good mechanisms for research succession planning and transition of research to practice We strive for research leading to sustainable engineering systems meeting broad societal needs we are challenged by current policies, funding approach, and traditional university/research stovepipes Massachusetts Institute of Technology 46

47 Imperative Engineering education and research must be a collaborative endeavor of government, industry, and academia Complex engineering research can not take place solely in a laboratory within university walls but rather real world enterprises must be our learning laboratories Expanded view of who an educator is -- faculty, researchers, practitioners, policy makers, peers Additionally, we need more cross cutting experiences for educators and practitioners alike Faculty have a very urgent need for case studies for use in the classroom without practitioner involvement these will lack depth to have educational impact Engineering education and research can not be just a cooperation; must be a true collaboration Massachusetts Institute of Technology 47

48 SE Leading Indicators Project Creating New Knowledge AF/DOD SE Revitalization Policies + AF/LAI Workshop on Systems Engineering June 2004 SE LI Working Group + With SSCI and PSM BETA Guide to SE Leading Indicators (December 2005) Project is an example of the power of collaborative research Pilot Programs (several companies) Masters Thesis (1 case study) Validation Survey (1 company) SE LI Working Group + Knowledge Exchange Event With SSCI and PSM Tutorial on SE Leading Indicators (many companies) (1) January 2007 V. 1.0 Guide to SE (2) November 2007 Leading Indicators June Massachusetts Institute of Technology 48

49 MITRE/MIT Research on Social Contexts of Systems Engineering Leverage Diversity of Research Team Develop social science capabilities and products complementing MITRE s technical capabilities in order to meet the challenges of Systems Engineering at the Enterprise level Transform practical field experience of MITRE site staff into social-scientific understanding that is usefully transferable Leverage experience and approaches from MIT partners Technical Approach Case Studies Workshops 2 nd Round of Case Studies Communicate Lessons Learned Massachusetts Institute of Technology 49

50 Draper /MIT Research Dynamic Tradespace Exploration Applied to System of Systems Extending Research to Enhance Practice New research launched in July 2007 (first Draper project with MIT ESD) to extend work of Ross (2006) University research project coupled to related in-house IR&D project Leverage geographic co-location for highly interactive research engagement Mutual benefit Enhance Draper capabilities and processes Further validate and extend MIT methodology Collaborative learning Massachusetts Institute of Technology 50

51 Summary Massachusetts Institute of Technology 51

52 SEAri Seeks To Impact Theory, Methods, And Practice MIT Engineering Systems Division (ESD) provides an interdisciplinary research venue Strategic collaboration with other MIT education and research centers (e.g., LAI, SDM) Hybrid research model for collaboration Single sponsor research projects Consortium research Realization of research goals is predicated on deep collaboration with industry and government Massachusetts Institute of Technology 52

53 Access to Research Navigation Home About People Research Related Courses Documents Events Sponsors Community Contact Internal Purpose Web portal for sharing research within SEAri, MIT, and systems community seari.mit.edu Massachusetts Institute of Technology 53

54 Sharing Research Outcomes 2007 SEAri Research Summit October 16 MIT Faculty Club SEARI Research Bulletin Published at End of Each Semester Massachusetts Institute of Technology 54

55 Additional References ESD Website ESD Research Centers ESD Working Papers ESD Symposium Monographs and Papers Lean Aerospace Initiative Refer to websites for additional information and working papers related to systems engineering at MIT Massachusetts Institute of Technology 55

56 QUESTIONS