Contributing toward a Prescriptive Theory of Ilities RT-113 Foundations

Transcription

1 Contributing toward a Prescriptive Theory of Ilities RT-113 Foundations Dr. Adam M. Ross, MIT 1 st Annual SERC Technical Review March 19, 2014 University of Southern California Los Angeles, CA ASTR 2014 March 19,

2 Objectives What are you trying to do? Develop a novel solution to the ility definition wars How is it done today? People assert definitions (examples soon follow) What s new in your approach Using a semantic basis with derived ility term labels Who cares? What positive difference will it make? Researchers and practitioners should care. This could help to resolve ambiguities in meaning for ilities, as well as result in ability to create verifiable ility requirements for systems One of the key objectives for this work is to stimulate a new conversation on a theory of ilities ASTR 2014 March 19,

3 Approach What are the key technical ideas? Ilities are members of one or more semantic fields Each of these fields can be described by a basis (i.e. set of distinct dimensions with allowable levels) A given ility concept can be described by a position (or subspace) within the span of the basis What are the key technical activities? Iterated theoretical and practical derivation of basis dimensions with allowed levels Application of basis to map ility term labels Application of basis to generate ility statements Validation of basis through formalization (e.g. with UVA) and usability (e.g. alignment with ility concept semantics and ility statement use cases) ASTR 2014 March 19,

4 Progress and Plans What s been done to date? Identification of at least three semantic fields Derivation of a 10D, 14D, and now 20D basis Iterated mapping of ility label terms to different versions of basis Preliminary formalization with UVA New refinement of 20D basis with several proposed use cases What comes next? Formalization of 20D basis Refinement of proposed use cases Broader ility term label mapping to 20D basis Revisit of comparison to broader ility research Comparison with other RT-113 ility work Some usability testing ASTR 2014 March 19,

5 Goals for the Research Wouldn t it be great to have this on your desk? ASTR 2014 March 19,

6 A Linguistic Approach to Ilities Linguistic meaning vs. speaker meaning Literal vs. nonliteral Rep. John Mica called on the agency to "reform" and "become...a thinking, risk-based, flexible agency that analyzes risks, sets security standards and audits security performance. ULA says the network augments 'more robust and flexible execution of Command and Control, Communications Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) To Axe, there is a "clear" need for the K-MAX because operations in the country are "highly dependent on flexible, reliable and secure logistics Defense Secretary Panetta: "The US joint force will be smaller and it will be leaner. But it will be more agile, more flexible, ready to deploy quickly, innovative and technologically advanced." the Defense Department and the Office of the Director of National Intelligence pledged to foster an industrial base that is 'robust, competitive, flexible, healthy, and delivers reliable space capabilities on time and on budget.'" Linguistics is the scientific study of human natural language, including semantics Semantics is the study of meaning and is a promising area for clarifying the ilities Meaning arises from interplay of use (i.e. speech) and prescription (i.e. dictionaries) Technical and political leaders are using ilities so we need to understand them well enough to ensure systems predictably display these properties Semantic field a group of words with related meanings, for example kinship terms or color terms Akmajian et al 2001, p. 587 Quotes from AIAA Daily Launch, 20 Jul Feb 2012 ASTR 2014 March 19,

7 Semantic Challenges for Ilities Fundamental ambiguity in terms Many of these terms are use colloquially and therefore inherit meaning Polysemy The property of [a term] having multiple meanings that are semantically related o Flexibility (able to be changed) and flexibility (able to satisfy multiple needs) Synonymy The property of multiple terms having similar meaning o Flexibility (able to be changed) and changeability (able to be changed or change itself) Problem partly stems from considering one ility at a time e.g. flexibility: Saleh, J. H., Mark, G.T., and Jordan, N.C. (2009). "Flexibility: a multi-disciplinary literature review and a research agenda for designing flexible engineering systems." Journal of Engineering Design. Nilchiani, R. (2005). Measuring Space Systems Flexibility: A Comprehensive Six-element Framework. PhD in Aeronautics and Astronautics, MIT. De Neufville, R. and Scholtes, S. (2011). Flexibility in Engineering Design, MIT Press: Cambridge MA. Some work done on sets e.g. changeability: Fricke and Schulz (2005). Design for Changeability (DfC): Principles to Enable Changes in Systems Throughout their Entire Lifecycle. Systems Engineering Ross, Rhodes and Hastings (2008). Defining Changeability: Reconciling Flexibility, Adaptability, Scalability, Modifiability, and Robustness for Maintaining System Lifecycle Value. Systems Engineering If challenge can be addressed by looking at sets of ilities, how do we select members of the set? ASTR 2014 March 19,

8 Ility Set Potential Sources (1) ASTR 2014 March 19,

9 Ility Set Potential Sources (2) ASTR 2014 March 19,

10 Ility Set Potential Sources (3) RT-46 Proposed Top Level List Resilience Dependability Affordability Flexibility Robustness Protection Mission Effectiveness Resource Utilization Composability Modifiability, Tailorability, Adaptability Reliability, Availability, Maintainability, Survivability Security, Safety Speed, Physical Capability, Cyber Capability, Accuracy, Impact, Endurability, Maneuverability, Usability, Scalability, Versatility Cost, Duration, Key Personnel, Other Scarce Resources, Manufacturability, Sustainability Interoperability, Openness, Service-Orientation ASTR 2014 March 19,

11 Descriptive Approach: Ilities Mentioned in Literature Frequency of ilities mentioned in journal articles and Google hits (de Weck, Roos, Magee 2011, p. 67) Frequency of ilities mentioned in literature across time (de Weck, Roos, Magee 2011, p. 69) This work shows the frequency of written usage of various ilities in a snapshot, and as a function of time (Q: Does frequency correlate with importance or relevance?) The challenge is that these results do not capture relationships amongst ilities or reasons behind the frequencies de Weck, O.L., Ross, A.M., and Rhodes, D.H., "Investigating Relationships and Semantic Sets amongst System Lifecycle Properties (Ilities)," 3rd International Conference on Engineering Systems, TU Delft, the Netherlands, June ASTR 2014 March 19,

12 Descriptive Approach: Derivation of Ilities Relationships Ility term co-occurrence in the literature with implied dependence (de Weck, Roos, Magee 2011, p. 83) This work shows the co-occurrence of written usage of various ilities in a snapshot, (Q: What is the nature of the co-occurrence? Complementary? Substitution? +/-?) These results are a good first step for proposing deeper inquiry into the nature of the relationships amongst ilities, but why this particular list of ilities? de Weck, O.L., Ross, A.M., and Rhodes, D.H., "Investigating Relationships and Semantic Sets amongst System Lifecycle Properties (Ilities)," 3rd International Conference on Engineering Systems, TU Delft, the Netherlands, June ASTR 2014 March 19,

13 Prescriptive Approach: Fricke and Schulz (2005) Design for Changeability (DfC): Principles to Enable Changes in Systems Throughout their Entire Lifecycle Systems Engineering, Vol. 8, No. 4, This work is based on the authors PhD research and experiences in German Product Development (e.g. BMW) Explicit ilities: changeability = {adaptability, robustness, agility, flexibility}, architecture principles = {simplicity, independence, modularity, integrability, autonomy, scalability, non-hierarchy, decentralization, redundancy Implicit ilities: evolvability, Related, but distinct: platforming ASTR 2014 March 19,

14 Prescriptive Approach: Ross, Rhodes and Hastings (2008) Defining Changeability: Reconciling Flexibility, Adaptability, Scalability, Modifiability, and Robustness for Maintaining System Lifecycle Value Systems Engineering, Vol. 11, No. 3, This work attempted to build upon Fricke and Schulz, ESD, and others to create a more rigorous basis for specifying and quantifying several ilities Explicit ilities: changeability = {adaptability, flexibility, scalability, modifiability, robustness} Verifiable changeability statement ASTR 2014 March 19,

15 Means-Ends Hierarchy from Prescriptive Definitions Using SEAri definitions, groups constructed means-ends hierarchies from a given set of ilities Lack of consensus and emergent depth criterion suggested more than means-ends relationships exist among ilities de Weck, O.L., Ross, A.M., and Rhodes, D.H., "Investigating Relationships and Semantic Sets amongst System Lifecycle Properties (Ilities)," 3rd International Conference on Engineering Systems, TU Delft, the Netherlands, June ASTR 2014 March 19,

16 Many Student Theses on Ilities Beesemyer, J.C., Empirically Characterizing Evolvability and Changeability in Engineering Systems, SM, Aero/Astro, MIT, June Fitzgerald, M.E., Managing Uncertainty in Systems with a Valuation Approach for Strategic Changeability, SM, Aero/Astro, MIT, June Friedel, A., Investigating the Management of Uncertainty in Product Platform Lifecycles, Dipl., TUM, January Fulcoly, D.O., A Normative Approach to Designing for Evolvability: Methods and Metrics for Considering Evolvability in Systems Engineering, SM, Aero/Astro, MIT, June Koo, C.K.K., Investigating Army Systems and Systems of Systems for Value Robustness, SM, SDM, MIT, February Mekdeci, B., Managing the Impact of Change through Survivability and Pliability to Achieve Viable Systems of Systems, PhD, ESD, MIT, February Nilchiani, R.N., Measuring the Value of Space Systems Flexibility: A Comprehensive Six-element Framework, PhD, Aero/Astro, MIT, September Richards, M.G., Multi-Attribute Tradespace Exploration for Survivability, PhD, ESD, MIT, June Richards, M.G., On-Orbit Serviceability of Space System Architectures, SMx2, Aero/Astro and TPP, MIT, June Roark, III, H.H., Value Centric Approach to Target System Modularization Using Multi-Attribute Tradespace Exploration and Network Measures of Component Modularity, SM, SDM, MIT, June Roberts, C.J., Architecting Evolutionary Strategies using Spiral Development for Space Based Radar, SM, TPP, MIT, June Ross, A.M., Managing Unarticulated Value: Changeability in Multi-Attribute Tradespace Exploration, PhD, ESD, MIT, June 2006 Saleh, J.H., Weaving Time into System Architecture: New Perspectives on Flexibility, Spacecraft Design Lifetime, and On-orbit Servicing, PhD, Aero/Astro, MIT, June Shah, N.B., Modularity as an Enabler for Evolutionary Acquisition, SM, Aero/Astro, MIT, June Viscito, L., Quantifying Flexibility in the Operationally Responsive Space Paradigm, SM, Aero/Astro, MIT, June Wilds, J.M., A Methodology for Identifying Flexible Design Opportunities, SM, TPP and Aero/Astro, MIT, September Have led to the following observation: There are at least three types of ilities 1. Change-related 2. Architecture-related 3. New ability related These are just some of the theses from our group explicitly addressing ilities their literature reviews uncovered many, many more theses outside of MIT, in addition to MIT ESD s recent theses Our working hypothesis* is that architecture-related ilities are enablers for change-related ilities For now, we will focus on change-related ilities * this is supported by our research on design principles (e.g. for survivability, evolvability, etc.) ASTR 2014 March 19,

17 Theoretical Framework for the System Value Design Problem Beesemyer, J.C., Empirically Characterizing Evolvability and Changeability in Engineering Systems, Master of Science Thesis, Aeronautics and Astronautics, MIT, June ASTR 2014 March 19,

18 10-D Semantic Basis Drawing analogy from linear algebra, a basis describes a spanning set that defines a space Can we decompose the change-related semantic field into distinct basis vectors? This would enable direct representation of how ilities are related to each other It would also help to avoid definition wars one ility at a time Dimensions can be used to differentiate between considered ilities Basis can be used to generate change statements for description or prescription ASTR 2014 March 19,

19 Generalizing the Change-related Statement: A Prescriptive Basis (From Ross, Rhodes, and Hastings 2008) Prescriptive Semantic Basis for Change-related Ilities In response to cause in context, desire agent to make some change in system that is valuable Cause Context System Agent Change Valuable (choose one) Why Where What What When Who What What What What When When For What For What Cause Context Entity Aspect Phase Agent Param Change Type Effect (Scale) Effect (Amount) Potential States Timing Span Resources Benefit perturbation specificity abstraction aspect LC phase executes param type level set target range reaction duration cost utility disturbance circumstantial architecture form pre-ops internal level bigger more one sooner shorter more more shift general design function ops external set smaller less few later longer less less none any system operations inter-lc either any not-same not-same many always same same same any any any any none same same any any any any any any any any category basis for specifying change-related ilities ASTR 2014 March 19,

20 Using the Basis to Map Ility Labels Early Application ASTR 2014 March 19,

21 Example Insights Using the Basis: Versatility Discussion in system aspect lead to clarification on types of versatility Different flavors of versatility The ability of a system to satisfy diverse needs for the system without having to change form (measure of latent value). Can accomplish this two different ways Change in function equates to functional versatility Change in operations equates to operational versatility Possibly where there are many regulations of operations are dynamic (rules of engagement) ASTR 2014 March 19,

22 Example Insights Using the Basis: Substitutability Discovered as the counterpart to versatility in the ility framework Framework allows clarification and exploration of system properties Substitutability then is using multiple forms to accomplish the same function with same operations Example would be components in a desktop computer (cd drives, hard drives, monitors, keyboards) ASTR 2014 March 19,

23 Definition Re-Examination Basis was used to audit definitions to uncover implicit assumptions ASTR 2014 March 19,

24 14-D Prescriptive Semantic Basis *key change: addition of impetus and outcome rather than just outcome (triggered by verbose versatility in 10D) ASTR 2014 March 19,

25 Reconciling Challenges in 10-D Version: 14-D Semantic Basis ASTR 2014 March 19,

26 Ilities as Responses to Uncertainties Uncertainties Responses Perturbations and limitations impact value Changes and resistances maintain value Perturbation Type Value Sustainment Outcome Parameter System Parameter Suppose we want to maintain value (i.e. no-change in outcome parameter value) There are four high level ility responses Shift Disturbance VALUE No-change Change Robustness Survivability No-change Versatility/ Insensitivity Changeability Change Having a basis allows us to quickly derive responses ASTR 2014 March 19,

27 Work with UVA In addition to iterating internally, we ve been working with UVA on refining the basis UVA formalized the basis in Coq by specifying: Abstract syntax Pretty printing A type assignment function that assigns zero or more ility term labels to a given change statement Feedback from UVA revealed some implicit assumptions in the basis Also triggered realization of different use cases for the basis that should be explicated ASTR 2014 March 19,

28 Latest Version of Basis (20D) The semantic basis would be used differently in different use cases Full basis: When trying to write a very specific requirement statement (should not occur until AFTER analysis to determine what should be done) Subset of basis: Early in the design phase, leave out the valuable categories (these are subjective, depend on outside factors) If one is trying to avoid fixating on a solution-centric approach, leave out change mechanism (allow engineers to propose own alternatives) ASTR 2014 March 19,

29 Different Use Cases of the Basis (1) Full basis: 20 columns (19 columns) When to use: before engineering design/analysis has determined the best mechanism for achieving the change via impetus to achieve outcome. (14 columns) When to use: to be OUTCOME oriented (i.e., focused on the effects ) as well as ensuring the change is valuable relative to defined dimensions. ASTR 2014 March 19,

30 Different Use Cases of the Basis (2) Full basis: 20 columns (10 columns) (11 columns) When to use: early in design in order to not over specify the change mechanism (allow engineers to propose/evaluate alternatives), or impetus (i.e. this is OUTCOME oriented). Note: Leaving out valuable part of statement supports exploration. Later, when implications of ility statement are better understood, one can specify (differently across stakeholders, if desired) subjective thresholds on what makes the change valuable. When to use: if there is a constraint to make use of an existing/inherited mechanism, for example. Note: This version is OUTCOME oriented, leaving open the valuable specification, but leaves in the mechanism category to constrain how the change should occur. ASTR 2014 March 19,

31 ASTR 2014 March 19,

32 Toward a Theory of Ilities What are the semantic fields that span the general set of ilities? e.g. change-type, architecture-type, new ability-type Basis Generated ility labels Derived ility hierarchies We do not want more definitions, but rather, unambiguous, verifiable, standardized representations of desired system properties Ultimate Goal: develop the basis/bases to be a prescriptive instrument(s) for spanning the semantic fields whose union encompass all ilities ASTR 2014 March 19,

33 Next Steps & Research Questions High level feedback Does this approach make sense? Applicability to Semantic Fields Does this basis only apply to change-type semantic field? What are the members of this field? What other semantic fields may exist? Can a different basis be used for each semantic field? Refinement of basis What are appropriate basis categories? What are appropriate choices within a category? Refinement of ility labels Are there consensus patterns in matching SEAri ilities to basis? Are there consensus patterns for given ility terms without provided definitions? How do other definitions for ility labels map to basis? Ultimately we do not want more definitions, but rather, unambiguous, verifiable, standardized representations of desired system properties Prescriptive use Can someone use the basis to generate change statements, which will automatically label with the appropriate ilities? How useful is the change statement for supporting verifiable requirements? Ultimate Goal: develop the basis/bases to be a prescriptive instrument(s) for spanning the semantic fields whose union encompass all ilities ASTR 2014 March 19,

34 References Akmajian, A., Demers, R.A., Farmer, A.K., and Harnish, R.M. (2001), Linguistics. MIT Press: Cambridge, MA. Boegh, J., A New Standard for Quality Requirements, IEEE Software, March/April 2008, pp Boehm, B., Port, D, and Al Said, M., Avoiding the Model-Clash Spiderweb, IEEE Computer, November 2000, pp Boehm, B., Huang, L., Jain, A., and Madachy, R., "The Nature of Information System Dependability: A Stakeholder/Value Approach," USC-CSSE TR USC-CSE ; Buschmann, F., Ameller, D., Ayala, C., Cabot, J., and Franch, X., Architecture Quality Revisited, IEEE Software, July/August 2012, pp de Neufville, R. and Scholtes, S. (2011), Flexibility in Engineering Design, MIT Press: Cambridge MA. de Weck, Roos, and Magee (2011), Life-cycle Properties of Engineering Systems, in Engineering Systems, MIT Press: Cambridge, MA. de Weck, O.L., Ross, A.M., and Rhodes, D.H., "Investigating Relationships and Semantic Sets amongst System Lifecycle Properties (Ilities)," 3rd International Conference on Engineering Systems, TU Delft, the Netherlands, June Fricke, E. and Schulz, A.P., (2005), Design for Changeability (DfC): Principles to Enable Changes in Systems Throughout their Entire Lifecycle, Systems Engineering, Vol. 8, No. 4, pp ISO/IEC :2001, Software Engineering Product Quality Part 1: Quality Model, Int'l Organization for Standardization, ISO/IEC 25030:2007, Software Engineering Software Product Quality Requirements and Evaluation (SQuaRE) Quality Requirements,Int'l Organization for Standardization, Nilchiani, R. (2005), Measuring Space Systems Flexibility: A Comprehensive Six-element Framework, PhD in Aeronautics and Astronautics, MIT, Cambridge MA. Ross, A.M. (2006), Managing Unarticulated Value: Changeability in Multi-Attribute Tradespace Exploration, PhD in Engineering Systems, MIT, Cambridge, MA. Ross, A.M., Rhodes, D.H., and Hastings, D.E. (2008), Defining Changeability: Reconciling Flexibility, Adaptability, Scalability, Modifiability, and Robustness for Maintaining Lifecycle Value, Systems Engineering, Vol. 11, No. 3, pp Ross, A.M., Beesemyer, J.C., and Rhodes, D.H. (2011), A Prescriptive Semantic Basis for System Lifecycle Properties. SEAri Working Paper Series, WP , pp (last accessed on 20 February 2014) Saleh, J. H., Mark, G.T., Jordan, N.C., (2009), "Flexibility: a multi-disciplinary literature review and a research agenda for designing flexible engineering systems," Journal of Engineering Design, Vol. 20, No. 3. pp ASTR 2014 March 19,