Socio-Technical Decision Making and Designing for Value Robustness

Transcription

1 RESEARCH PROFILE Socio-Technical Decision Making and Designing for Value Robustness October 21, 28 Dr. Adam M. Ross Massachusetts Institute of Technology

2 Portfolio RESEARCH PORTFOLIO 1. Socio-Technical Decision Making 2. Designing for Value Robustness 3. Systems Engineering in the Enterprise 4. Systems Engineering Economics 5. Systems Engineering Strategic Guidance seari.mit.edu 28 Massachusetts Institute of Technology 2

3 Research Portfolio (1) SOCIO-TECHNICAL DECISION MAKING This research area seeks to develop multi-disciplinary representations, analysis methods, and techniques for improving decision making for socio-technical systems. Examples include: Studies of decision processes and effectiveness of techniques Constructs for representing socio-technical systems for impact analysis on costs, benefits, and uncertainties Effective visualization of complex tradespaces nderstanding and mitigating cognitive biases in decision processes Developing dynamic system strategies (e.g. timing technology investments and execution of system change options) Methods for representing distribution of costs and benefits to multiple stakeholders of socio-technical systems Representations, analysis methods, and techniques for improving socio-technical decision making seari.mit.edu 28 Massachusetts Institute of Technology 3

4 The Scope of pfront Decisions Conceptual Design is a high leverage phase in system development Key Phase Activities Needs Captured Design ~66% Resources Scoped In Situ vs. Top-side sounder After Fabrycky and Blanchard 1991 Concept(s) Selected seari.mit.edu 28 Massachusetts Institute of Technology 4

5 The Scope of pfront Decisions Conceptual Design is a high leverage phase in system development Key Phase Activities Needs Captured Design ~66% Resources Scoped Reliance upon BOGGSAT could have large consequences How can we make better decisions? In Situ vs. Top-side sounder After Fabrycky and Blanchard 1991 Concept(s) Selected seari.mit.edu 28 Massachusetts Institute of Technology 5

6 Three keys to good upfront decisions Structured program selection process Choosing the programs that are right for the organization s stakeholders Classical systems engineering Determining stakeholder needs, generating concept of operations, and deriving requirements Conceptual design practices Finding the right form to maximize stakeholder value over the product (or product family) lifetime Good system decisions must include both socio as well as technical components seari.mit.edu 28 Massachusetts Institute of Technology 6

7 Flexibility Representations tility = Optimal Design for M 1 M 1 Designs with < (M) < 1 ncertain future M x mission M 2 Optimal design has much more utility Acceptable Transition path Mission with one design of (M)> Managing ncertainty in Socio-Technical Enterprises using a Real Options Framework Tsoline Mikaelian, Aero/Astro PhD 29 What enterprise representation/models can be used to identify potential real option investment opportunities? How can real options be used for holistic decision making and architecting of socio-technical enterprises under uncertainty? Metrics for Flexibility in the Operationally Responsive Space Paradigm Lauren Viscito, Aero/Astro SM 29 Can a flexibility metric be used for explicit trades in conceptual space system design? M 5 M M 4 3 seari.mit.edu 28 Massachusetts Institute of Technology 7

8 Visualization Constructs for Tradespace Exploration 1 Survivability Tradespace - no filtering (n=256) design utility (dimensionless) median time-weighted utility loss (dimensionless) threshold availability (5th percentile).1 utility (dimensionless) tility Trajectory - DV(1137) time (years) Mapping to Survivability Definition V(t) Threshold average time-weighted average utility (dimensionless) cost ($M) 62 threshold availability - 5th percentile (filtered) cost ($M) 127 number specifies baseline design vector no avoidance, no servicing no avoidance, servicing avoidance, no servicing avoidance, servicing servicing response avoidance response shielding response Richards, M.G., Ross, A.M., Shah, N.B., and Hastings, D.E., Metrics for Evaluating Survivability in Dynamic Multi-Attribute Tradespace Exploration, AIAA Space 28, San Diego, CA, September 28. seari.mit.edu 28 Massachusetts Institute of Technology 8

9 Research Portfolio (2) DESIGNING for VALE ROBSTNESS This research area seeks to develop methods for concept exploration, architecting and design using a dynamic perspective for the purpose of realizing systems, products, and services that deliver sustained value to stakeholders in a changing world. Examples include: Methods for and applications of dynamic Multi-Attribute Tradespace Exploration Architecting principles and strategies for designing survivable systems Architecting strategies and quantitative tradespace exploration of systems of systems Quantification of the changeability of system designs Techniques for the consideration of unarticulated stakeholder and latent system value Taxonomy for enabling stakeholder dialogue on ilities Representations, analysis methods, and techniques for designing systems for success in dynamic contexts seari.mit.edu 28 Massachusetts Institute of Technology 9

10 Meeting Customer Needs Goal of design is to create value (profits, usefulness, voice of the customer, etc ) Requirements capture a mapping of needs to specifications to guide design seari.mit.edu 28 Massachusetts Institute of Technology 1

11 Deploying a Valuable System seari.mit.edu 28 Massachusetts Institute of Technology 11

12 Deploying a Valuable System Contexts change seari.mit.edu 28 Massachusetts Institute of Technology 12

13 Meeting Customer Needs (cont.) Goal of design is to create value (profits, usefulness, voice of the customer, etc ) Requirements People change capture their minds a mapping of needs to specifications To continue to to deliver guide value, designthe systems may need to pursue changeability or versatility seari.mit.edu 28 Massachusetts Institute of Technology 13

14 Selecting Best Designs Best design? Doesn t look good anymore! Benefit 1 A B C D E Time Benefit 2 A F C B D E Cost Cost As uncertainty resolves, new contexts reveal new cost-benefit trades How can a program select best designs in an uncertain and changing context? Designing for Value Robustness directly addresses this challenge seari.mit.edu 28 Massachusetts Institute of Technology 14

15 Washington, DC in June 24 LAI/AF Systems Engineering for Robustness Workshop According to Dr. Marvin Sambur, Systems Engineering for Robustness means developing systems that are Capable of adapting to changes in mission and requirements Expandable/scalable, and designed to accommodate growth in capability Able to reliably function given changes in threats and environment Effectively/affordably sustainable over their lifecycle Developed using products designed for use in various platforms and systems Easily modified to leverage new technologies Robustness scope expanded beyond classical robustness Experts questioned What does it mean? How can it be measured/analyzed? Who is going to pay for it? How can designers account for this new robustness? *Adapted from Ross, A., Rhodes, D., and Hastings, D., Defining System Changeability: Reconciling Flexibility, Adaptability, Scalability, and Robustness for Maintaining System Lifecycle Value, INCOSE Int l Symposium 27, San Diego, CA, June 27 seari.mit.edu 28 Massachusetts Institute of Technology 15

16 Six Areas of Research How do stakeholders perceive value? How can stakeholders have a dialogue on value? How can value robustness be quantified? How can value robust systems be identified? How can we architect for value robustness? What about systems of systems? (1) Attribute Class Spectrum (2) Change Taxonomy (3) Metrics for Value Robustness (4) Tradespace Exploration Method (5) Architecting for Ilities (6) SoS Tradespace Exploration A.M Ross and D.H. Rhodes, Architecting Systems for Value Robustness: Research Motivations and Progress, 2 nd Annual IEEE Systems Conference, Montreal, CA, April 4-5, 28 **BEST PAPER AWARD** seari.mit.edu 28 Massachusetts Institute of Technology 16

17 Attribute Class Spectrum Articulated, narticulated and Latent Value Research focuses on an approach for ensuring designers account for unarticulated as well as articulated value perceptions, by intentionally building latent system value attributes according to the ease by which a system can display them Implications for Systems Engineering Practice 1. Better decisions by improving the practice through more rigorous constructs that characterize system attributes and their costs 2. Ability to more effectively explore unarticulated stakeholder and latent system value can uncover essential needs and desires early in the process 3. Observation during experimentation or early use of how stakeholders leverage latent value can be an important source of innovation A.M Ross and D.H. Rhodes, sing Attribute Classes to ncover Latent Value during Conceptual Systems Design, 2 nd Annual IEEE Systems Conference, Montreal, CA, April 4-5, 28 seari.mit.edu 28 Massachusetts Institute of Technology 17

18 Change Taxonomy Flexibility, Adaptability, Scalability, Modifiability, etc. Research focuses on developing a rigorous, consistent taxonomy for specifying, evaluating, and validating temporal system properties, sometimes called the new ilities Implications for Systems Engineering Practice 1. Remove ambiguity and provide quantitative description of ilities to improve acquisition and development 2. Potential to lead to the normative specification of the ilities as a basis for prescriptive guidance 3. Taxonomy provides a common lexicon for stakeholder dialogue A.M Ross and D.H. Rhodes, Defining Changeability: Reconciling Flexibility, Adaptability, Scalability, Modifiability, and Robustness for Maintaining Lifecycle Value, Systems Engineering, Vol. 11, No. 3, Fall 28, pp M.G. Richards, D.E. Hastings, D.H. Rhodes, and A.L. Weigel, Defining Survivability for Engineering Systems, 5 th Conference on Systems Engineering Research, Hoboken, NJ, March 27 seari.mit.edu 28 Massachusetts Institute of Technology 18

19 Metrics for Value Robustness Versatility or Changeability for Maintaining Value Research focuses on developing rigorous, quantitative metrics for evaluating and comparing, on a common basis, the ability of alternative systems to maintain value delivery over time Implications for Systems Engineering Practice 1. Construct for quantitatively assessing changeability of candidate designs in a tradespace 2. Provides designers with analytic construct for making design decisions 3. Contributes to composing repeatable and verifiable requirements for changeability State 1 State 2 Filtered Outdegree 1 2 A Cost Cost Cost Cost A B C A.M Ross and D.H. Rhodes, Defining Changeability: Reconciling Flexibility, Adaptability, Scalability, Modifiability, and Robustness for Maintaining Lifecycle Value, Systems Engineering, Vol. 11, No. 3, Fall 28, pp N.B. Shah, L. Viscito, J.M. Wilds, A.M. Ross, and D.E. Hastings, Quantifying Flexibility for Architecting Changeable Systems, 6 th Conference on Systems Engineering Research, Los Angeles, CA, April 28 seari.mit.edu 28 Massachusetts Institute of Technology 19

20 Dynamic Multi-Attribute Tradespace Exploration Value-driven Conceptual Design for Evolving Systems Research focuses on developing value-drive method for exploring relationship between dynamic value space and design space of many system alternatives on a common basis across time Implications for Systems Engineering Practice 1. Ability to explore many design options and prevent too early focus on single point design 2. Enables quantitative assessment of factors such as variability in technical performance and cost, and impacts in markets 3. Suitable to multiple domains and demonstrated to improve design decision making Tradespace exploration uses computer-based models to compare thousands of alternatives Avoids limits of local point solutions Maps decision maker preference structure to potential design space A.M. Ross and D.H. Rhodes, The Tradespace Exploration Paradigm, INCOSE International Symposium 25, Rochester, NY, July 25 A.M. Ross, Managing narticulated Value: Changeability in Dynamic Multi-Attribute Tradespace Exploration, PhD Dissertation, MIT, June 26 seari.mit.edu 28 Massachusetts Institute of Technology 2

21 Research focuses on developing rigorous, empirically supported design principles for guiding design toward better performance in temporal system properties, such as survivability Implications for Systems Engineering Practice 1. Validated design principles provide more rigorous guidance than classical heuristics 2. Provides a explicit mapping of design principles to timing of context events Architecting for ilities Design Principles for Dynamic System Properties V(t) 4 V x (Ball 23) T r V e Epoch 1a Epoch 2 Epoch 1b 1.1 prevention 2.1 hardness 1.2 mobility 1.3 concealment 1.5 preemption 1.4 deterrence 1.6 avoidance 2.2 redundancy 2.3 margin 2.4 heterogeneity 2.5 distribution 2.6 failure mode reduction 2.7 fail-safe 2.8 evolution 2.9 containment 2.1 replacement 2.11 repair time original modified new M.G. Richards, A.M. Ross, D.E. Hastings, and D.H. Rhodes, Design Principles for Survivable System Architecture, 1 st Annual IEEE Systems Conference, Honolulu, HI, April 27 M.G. Richards, A.M. Ross, D.E. Hastings, and D.H. Rhodes, Two Empirical Tests of Design Principles for Survivable System Architecture, INCOSE International Symposium 28, trecht, the Netherlands, June 28, **BEST PAPER AWARD** seari.mit.edu 28 Massachusetts Institute of Technology 21

22 SoS Tradespace Exploration Determining SoS Components and Interfaces Research targeted at providing a more rigorous method for system of systems engineering, which requires continuous tradespace exploration as constituent systems enter and exit the system Implications for Systems Engineering Practice 1. Identified unique considerations for exploring SoS tradespaces versus traditional systems 2. Methods for negotiating across multiple stakeholder value propositions becomes central to successful SoS development 3. Reinforces the importance of proper interface design as essential to SoS value delivery Legacy Systems Aircraft Satellite AV Time-Varying Available Component Sets New Legacy New Legacy Systems Systems Systems Systems New Systems Switching Cost... SoS A SoS B SoS N T 1 T 2 T N D. Chattopadhyay, A.M. Ross and D.H. Rhodes, A Framework for Tradespace Exploration of Systems of Systems, 6 th Conference on Systems Engineering Research, Los Angeles, CA, April 28 R. Valerdi, A.M. Ross, and D.H. Rhodes, "A Framework for Evolving System of Systems Engineering," CrossTalk--The Journal of Defense Software Engineering, October 27 seari.mit.edu 28 Massachusetts Institute of Technology 22 Time component systems

23 Distributed Decision Making in Systems of Systems Extended from Schneeweiss (23) Distributed Decision Making System of System research has tended to focus on technical interfaces of constituent systems. Proper design of both technical and non-technical interfaces is essential for creating value-enhancing and stable SoS. Taking a value-centric approach reveals the importance of distributed decision making in SoS and mechanisms for influencing or affecting these decisions to create value robust SoS. Nirav Shah, PhD Candidate, 29 seari.mit.edu 28 Massachusetts Institute of Technology 23

24 Tradespace Exploration Coupled with Value-driven Design Many system designs can be compared through tradespace exploration: Models and simulations determine attribute performance of many designs (1s to 1s or more) 1. Elicit Value with attributes and utility 2. Generate Concepts using design variables and cost model insights 3. Develop models/sims to assess designs in terms of cost and utility DESIGN VARIABLES: Design trade parameters Orbital Parameters Apogee Altitude (km) Perigee Altitude (km) Orbit Inclination (deg) Spacecraft Parameters Antenna Gain Communication Architecture Propulsion Type Power Type Total Delta V Cost, tility ATTRIBTES: Design decision metrics Data Lifespan (yrs) Equatorial Time (hrs/day) Latency (hrs) Latitude Diversity (deg) Sample Altitude (km) Assessment of cost and utility of large space of possible system designs sing value metrics focuses analysis on most important system aspects seari.mit.edu 28 Massachusetts Institute of Technology 24

25 Spacetug vs CX-OLEV Example Real Systems XTOS vs Streak Total Lifecycle Cost ($M22) XTOS (22 study) Streak (Oct 25 launch) Wet Mass kg Dry Mass kg Propellant kg Equipment kg DV m/s tility Cost Electric Cruiser (22 study) *** CX-OLEV (29 launch) 14 67* 73* 213* 159**.69 13* Wet Mass kg Lifetime (yrs) Orbit LV tility Modified tility** Cost $M Instruments Minotaur Three (?) Ion gauge and atomic oxygen sensor seari.mit.edu 28 Massachusetts Institute of Technology a-296p -> 96 Minotaur *.59 75***

26 Tradespace Analysis: Selecting best designs Classic best design New best design tility 1 A B C D E tility 2 A B C D E Cost Time Cost If the best design changes over time, how does one select the best design? seari.mit.edu 28 Massachusetts Institute of Technology 26

27 Tradespace Networks tility tility Transition rules Cost Cost Tradespace designs = nodes Applied transition rules = arcs Cost Transition rules are mechanisms to change one design into another The more outgoing arcs, the more potential change mechanisms seari.mit.edu 28 Massachusetts Institute of Technology 27

28 Tradespace Networks tility Rule R1: Plane Change R2: Apogee Burn R3: Perigee Burn R4: Plane Tug R5: Apogee Tug R6: Perigee Tug R7: Space Refuel R8: Add Sat Example: X-TOS Transition Rules Cost Change all orbit, V Description Increase/decrease inclination, decrease V Increase/decrease apogee, decrease V Transition rules Increase/decrease perigee, decrease V Increase V, requires refuelable tility Increase/decrease inclination, requires tugable Increase/decrease apogee, requires tugable Increase/decrease perigee, requires tugable Change agent origin Internal (Adaptable) Internal (Adaptable) Internal (Adaptable) External (Flexible) External (Flexible) External (Flexible) External (Flexible) External (Flexible) Cost Tradespace designs = nodes Applied transition rules = arcs Cost Transition rules are mechanisms to change one design into another The more outgoing arcs, the more potential change mechanisms seari.mit.edu 28 Massachusetts Institute of Technology 28

29 Tradespace Networks: Changing designs over time Classic best design New best design tility 1 A B C D E tility 2 A B C D E Cost Time Cost Select changeable designs that can approximate best designs in new contexts seari.mit.edu 28 Massachusetts Institute of Technology 29

30 sing Epochs to Represent Contexts and Expectations Attributes (performance, expectations) System Expectation 1 Expectation 1 Context 1 Context 2 Expectation 2 Context 2 Expectation 3 NEW NEED METRIC Context 3 Expectation 4 Context 4 Time (epochs) Two aspects to an Epoch: 1. Needs (expectations) 2. Context (constraints including resources, technology, etc.) Needs: + Context: Epoch 1 Epoch 2 Epoch 3 Epoch 4 Epoch 5 Example system: Serviceable satellite System BOL Value degradation New Context: new value function (objective fcn) Service to Major failure upgrade Major failure T 1 T 2 T n System EOL Epoch 1 Epoch 2 Epoch n System timeline with serviceability -enabled paths allow continued value delivery seari.mit.edu 28 Massachusetts Institute of Technology 3 Value outage: S 1,b S 1,e S 2,b S 2,e S n,b S Servicing time n,e Service to restore Same system, but perceived value decrease Service to restore

31 Epoch-Era Analysis: Epochs Epoch Time period with a fixed context and needs; characterized by static constraints, design concepts, available technologies, and articulated attributes (Ross 26) Define Epochs T i Epoch i T i Epoch i T i Epoch i T j Epoch T j j Epoch T j j Epoch j Potential Contexts Potential Needs Construct Eras Epoch Series Dynamic Strategies Discretization of change timeline into short run and long run enables analysis Allows for rigorous consideration of many possible futures seari.mit.edu 28 Massachusetts Institute of Technology 31

32 Epoch-Era Analysis: Eras Era System life with varying contexts and needs, formed as an ordered set of epochs; characterized by varying constraints, design concepts, available technologies, and articulated attributes Define Epochs T i Epoch i T i Epoch i T i Epoch i T j Epoch T j j Epoch T j j Epoch j Potential Contexts Potential Needs Construct Eras Epoch Series Dynamic Strategies Discretization of change timeline into short run and long run enables analysis Allows for analysis of system varying performance over possible futures seari.mit.edu 28 Massachusetts Institute of Technology 32

33 Passive Value Robustness as Pareto Trace across Epochs Tradespace Networks in the System Era System Era T 1 T 2 T 3 T n Epoch 1 Epoch 2 Epoch 3 Epoch n S 1,b S 1,e S 2,b S 2,e S 3,b S 3,e S n,b S n,e Time 1 Change Tradespace (N=81), Path: 81-->1, Goal til:.97 1 Change Tradespace (notional), Goal til:.97.9 Changeability Quantified as Filtered Outdegree N k R k OD k Total Delta C Total Transition Time Multiple metrics and analytic techniques available across system timeline seari.mit.edu 28 Massachusetts Institute of Technology 33

34 Era Paths Reveal System Evolution Strategies System Era T 1 T 2 T 3 T n Epoch 1 Epoch 2 Epoch 3 Epoch n S 1,b S 1,e S 2,b S 2,e S 3,b S 3,e S n,b S n,e Time Active value robustness strategy: Maintain given level of value through Context changes Epoch 63 Epoch 171 Epoch 193 Epoch 22 Epoch yrs 4 yrs 1 yr 3 yrs 3 yrs Temporal strategy can be developed across networked tradespaces seari.mit.edu 28 Massachusetts Institute of Technology 34

35 Achieving Value Robustness Research suggests two strategies for Value Robustness Active Passive New Context Drivers External Constraints Design Technologies Value Expectations tility T 1 T 2 Epoch 1 Epoch 2 1. Passive Choose versatile designs that remain high value Quantifiable: Pareto Trace number 2. Active Choose changeable designs that can deliver high value when needed Quantifiable: Filtered Outdegree tility Time S 1,b State 1 S 1,e S 2,b State 2 S 2,e DV 2 DV 1 DV 2 =DV 1 Cost Value robust designs can deliver value in spite of inevitable context change Cost seari.mit.edu 28 Massachusetts Institute of Technology 35

36 Designing for Value Robustness Mindshift: recognize dynamic contexts and fallacy of static preferences the inevitability of change Two primary strategies: Matching changeable systems to changing needs leads to sustained system success Creating versatile systems with latent value leads to sustained system success Methods for increasing Changeability Increase number of paths (change mechanisms) Lower cost or increase acceptability threshold (alter apparent changeability) Changeability can be used as an explicit and consistent metric for designing systems Methods for increasing Versatility Increase number of displayed fundamental or combinatorial system attributes Decrease cost for displaying or hiding attributes Designed for changeability or versatility, systems will be empowered to become value robust, delivering value in spite of context and preference changes seari.mit.edu 28 Massachusetts Institute of Technology 36

37 Summary Socio-Technical Decision Making Designing for Value Robustness Next segment Research Report: Matthew Richards, Design for Survivability Research Report: Tsoline Mikaelian, Real Options in Enterprise Architecture seari.mit.edu 28 Massachusetts Institute of Technology 37

38 QESTIONS

39 Agenda 9: Welcome and Introductions Dr. Donna Rhodes, SEAri Director 9:3 SEAri Overview of the SEAri Research Program Dr. Donna Rhodes, Research Director 1: Research Profile: Socio-Technical Decision Making and Designing for Value Robustness 1:45 Break Dr. Adam Ross, Research Scientist 11: Research Report: Designing Systems for Survivability Matt Richards, Doctoral Research Assistant 11:3 Research Report: Real Options in Enterprise Architecture Tsoline Mikaelian, Doctoral Research Assistant noon LNCH 1: Research Profile Systems Engineering in the Enterprise Dr. Donna Rhodes, Principal Research Scientist 1:3 Research Report: Leveraging Organizational Culture, Standard Process, and Team Norms to Enable Collaborative Systems Thinking 2: Stretch Break Caroline Lamb, Doctoral Research Assistant 2:1 Research Profile: Systems Engineering Economics Dr. Ricardo Valerdi, Research Associate 2:5 Research Poster Session with Refreshments SEAri Research Assistants 4:15 Participant Feedback and Recommendations for Research SEAri Leadership 4:55 Closing Remarks Dr. Donna Rhodes 5: Adjourn seari.mit.edu 28 Massachusetts Institute of Technology 39

40 Future Research Directions Socio-Technical Decision Making Decision criteria for valuing ilities : when to seek active vs. passive Value Robustness Refined quantitative metrics Advanced dynamic tradespace visualization experiments and methods development ATSV and MATE with Penn State? Field study assessing current senior decision making practices Application of descriptive temporal utility theories to elucidate changing preferences on system designs Can engineers predict or anticipate certain classes of preference change? Assessment of current acquisition policies for research deployment opportunities Designing for Value Robustness Application of Dynamic MATE to nonaerospace system Transportation with MIT-Portugal Program Application of Dynamic MATE to Systems of Systems Aerospace SoS Categorization of architectural approaches for increasing changeability Towards a changeable design toolkit Refinement of fundamental attribute set, validated through case studies In depth application of Epoch-Era Analysis to case study(s) Large datasets generated this summer Application of industrial economics to market classes and the value of changeability Compare/contrast various types of markets, i.e. Aerospace, consumer electronics, etc. seari.mit.edu 28 Massachusetts Institute of Technology 4