Prospects for an Engineering Discipline of Software

Transcription

1 Prospects for an Engineering Discipine of Software Mary Shaw, Carnegie Me//on University Software engineering is not yet a true engineering discipine, but it has the potentia to become one. Oder engineering fieds suggest the character soft ware engineering m&ht have. T he term softwxe engineering was coined in 1968 as a statement of aspiration - a sort of raying cry. That year, the North Atantic Treaty Organization convened aworkshop by that name to assess the state and prospects of software production. Capturing the imagination of software deveopers, the NATO phrase software engineering achieved popuarity during the 1970s. It now refers to a coection of management processes, soft- ware tooing, and design activities for software deveopment. The resuting practice, however, differs significanty from the practice of oder forms of engineering. What is engmwing? Software engineering is a abe appied to a set of current practices for deveopment. But using the word engineering to describe this activity takes considerabe ibertywith the common use of that term. The more customary usage refers to the discipined appication of scientific knowedge to resove conficting constraints and requirements for probems of immediate, practica significance. Definitions of engineering abound. Athough detais differ, they share some common causes: Creating co.tt-effective soutions Engineering is not just about soving prob ems; it is about soving probems with economica use of a resources, incuding money.. to practica probems Engineering deas with practica probems whose soutions matter to peope outside the engineering domain-the customers. by appying scimtifc knowedge.. Engineering soves probems in a particuar way: by appying science, mathematics, and design anaysis.. to buiding things Engineering emphasizes the soutions, which are usuay tangibe artifacts.. in the service of mankind. Engineering not ony serves the immediate November /90/ 100/0015/$ EEE 15

2 customer, but it aso deveops technoogy and expertise that wi support the society. tions for software designs are not adequate for the task of both recording and Engineering reies on codifying scien- communicating designs, so they fai to tific knowedge about a technoogica provide a suitabe representation for such probem domain in a form that is directy usefu to the practitioner, thereby providing answers for questions that commony handbooks. Software in most appication domains is treated more often as origina than rouoccur in practice. Engineers of ordinary tine - certainy more so than woud be taent can then appy this knowedge to sove probems far faster than they otherwise coud. In this way, engineering shares prior soutions rather than reying aways on virtuoso probem soving. Engineering practice enabes ordinary practitioners so they can create sophistinecessary if we captured and organized what we aready know. One path to increased productivity is identifying appications that coud be routine and deveop ing appropriate support. The current focus on reuse emphasizes capturing and organizing existing knowcated systems that work - unspectac- edge of a particuar kind: knowedge exuary, perhaps, but reiaby. The history of pressed in the form of code. Indeed, sub deveopment is marked by both successes routine ibraries - especiay of system and faiures. The successes have often cas and genera-purpose mathematica been virtuoso performances or the resut of diigence and hard work. The faiures have often refected poor understanding of the probem to be soved, mismatch of soution to probem, or inadequate fo- Given our track record, there are fundamenta ow-through from design to impementa- probems with the tion. Some faied by never working, others use ofthe term by overrunning cost and schedue budgets. %o#vware engneefi * In current software practice, knowedge about techniques that work is not shared effectivey with ater projects, nor is there a arge body of deveopment knowedge routines - have been a stape of programorganized for ready reference. Computer ming for decades. But this knowedge canscience has contributed some reevant not be usefu if programmers do not know theory, but practice proceeds argey independenty of this organized knowedge. Given this track record, there are fundaabout it or are not encouraged to use it. Furthermore, ibrary components require more care in design, impementation, and menta probems with the use of the term documentation than simiar components software engineer. that are simpy embedded in systems. Practitioners recognize the need for mechanisms to share experience with good designs. This cry from the widerness appeared on the Software Engineering News Group, a moderated eectronic maiing ist: In Chem E, when I needed to design a heat exchanger, I used a set of references that tod me what the constants were... and the standard design equations.... In genera, uness I, or someone ese in Routine and innovative design. Engineering design tasks are of severa kinds. One of the most significant distinctions separates routine from innovative design. Routine design invoves soving famiiar probems, reusing arge portions of prior soutions. Innovative design, on the other hand, invoves finding nove soutions to unfamiiar probems. Origina designs are much more rarey needed than routine designs, so the atter is the bread and butter of engineering. Most engineering discipines capture, organize, and share design knowedge to make routine design simper. Handbooks and manuas are often the carriers of this organized information. But current nota- 16 my [software-] engineering group, has read or remembers and makes known a soution to a past probem, I m doomed to recreate the soution.... I guess... the critica difference is the abiity to put together itte pieces of the probem that are reativey we known, without having to gen- erate a custom soution for every appication.. I want to make it cear that I am aware of agorithm and code ibraries, but they are incompete soutions to what I am describing. (There is no Perry s Handbook for Software Engineering.) This former chemica engineer is com- paining that software acks the institutionaized mechanisms of a mature engineering discipine for recording and disseminating demonstraby good designs and ways to choose among design aternatives. (Perry s Chemica Engineering Handbook pubished by McGraw-Hi, is the standard design handbook for chemica engineering; it is about four inches thick and printed in tiny type on 8.5 x 11 tissue paper.) Mode for the evoution of an engineering discipine. Historicay, engineering has emerged from ad hoc practice in two stages: First, management and production techniques enabe routine production. Later, the probems of routine pro duction stimuate the deveopment of a supporting science; the mature science eventuay merges with estabished practice to yied professiona engineering practice. Figure 1 shows this mode. The expoitation of a technoogy begins with craftsmanship: A set of probems must be soved, and they get soved any which way. They are soved by taented amateurs and by virtuosos, but no distinct professiona cass is dedicated to prob ems of this kind. Intuition and brute force are the primary movers in design and construction. Progress is haphazard, particuary before the advent of good communication; thus, soutions are invented and reinvented. The transmission of knowedge between craftsmen is sow, in part because of underdeveoped com- munications, but aso because the taented amateurs often do not recognize any specia need to communicate. Nevertheess, ad hoc practice eventuay moves into the fokore. This craft stage of deveopment sees extravagant use of avaiabe materias. Construction or manufacture is often for persona or oca use or for barter, but there is itte or no argescae production in anticipation of resae. Community barn raisings are an exampe IEEE Software

3 Production Science \ Professiona engineering Commercia of this stage; so is software written by appication experts for their own ends. At some point, the product of the technoogy becomes widey accepted and demand exceeds suppy. At that point, attempts are made to define the resources necessary for systematic commercia marufacture and to marsha the expertise for expoiting these resources. Capita is needed in advance to buy raw materias, so financia skis become important, and the operating scae increases over time. As commercia practice fourishes, skied practitioners are required for continuity and for consistency of effort. They are trained pragmaticay in estabished procedures. Management may not know why these procedureswork, but they know the procedures do work and how to teach peope to execute them. The procedures are refined, but the refinement is driven pragmaticay: A modification is tried to see if it works, then incorporated in standard procedure if it does. Economic considerations ead to concerns over the efficiency of procedures and the use of materias. Peope begin to expore ways for production faciities to expoit the technoogy base; economic issues often point out probems in commercia practice. Management strategies for controing deveopment fit at this point of the mode. The probems of current practice often stimuate the deveopment of a corresponding science. There is frequenty a strong, productive interaction between commercia practice and the emerging science. At some point, the science becomes sufficienty mature to be a significant contributor to the commercia practice. This marks the emergence of engineering practice in the sense thatwe know it today - sufficient scientific basis to enabe a core of educated professionas so they can appy the theory to anaysis of probems and synthesis of soutions. For most discipines, this emergence occurred in the 18th and eary 19th centuries as the common interests in basic phys ica understandings of natura science and engineering graduay drew together. The reduction of many empirica engineering techniques to a more scientific basis was essentia to further engineering progress. And this iaison stimuated fur- November 1990 Craft / Virtuosos and taented amateurs intuition and brute force. Haphazard progress Casua transmission 9 Extravagant use of avaiabe materias Manufacture for use rather than sae Skied craftsmen Estabished procedure Pragmatic refinement Training in mechanics Economic concern for cost and suppy of materias Manufacture for sae Educated professionas Anaysis and theory Progress reies on science Educated professiona cass Enabing new appications through anaysis Market segmentation by product variety Figure 1. Evoution of an engineering discipine. The ower ines track the technoogy, and the upper ines show how the entry of production skis and scientific knowedge contribute new capabiity to the engineering practice. ther advances in natura science. An important and mutuay stimuating tie-up between natura and engineering science, a deveopment [that] had been discouraged for centuries by the ong-dominant infuence of eary Greek thought, was at ong ast consummated, wrote historian James Kip Finch. The emergence of an engineering discipine ets technoogica deveopment pass imits previousy imposed by reying on intuition; progress frequenty becomes dependent on science as a forcing function. A scientific basis is needed to drive anaysis, which enabes new appications and even market segmentation via product variety. Attempts are made to gain enough contro over design to target specific products on demand. Thus, engineering emerges from the commercia expoitation that suppants craft. Modern engineering reies criticay on adding scientific foundations to craft and commerciaization. Expoiting technoogy depends not ony on scientific engineering but aso on management and the marshaing of resources. Engineering and science support each other: Engineering generates good probems for science, and science, after finding good probems in the needs of practice, returns workabe soutions. Science is often not driven by the immediate needs of engineering; however, good scientific probems often foow from an understanding of the prob ems that the engineering side of the fied is coping with. The engineering practice of software has recenty come under criticism for acking a scientific basis. The usua curricuum has been attacked for negecting mathematics and engineering science. Athough current software practice does not match the usua expectations of an engineering discipine, the mode described here suggests that vigorous pursuit of ap picabe science and the reduction of that science to practice can ead to a sound engineering discipine of software. Exampes from traditiona engineering. Two exampes make this mode concrete: the evoution of engineering discipines as demonstrated by civi and chemica engineering. The comparison of the two is aso iuminating, because they have very different basic organizations. Civi engineting: a basis in theory. Originay socaed to distinguish it from miitary engineering, civi engineering incuded a of civiian engineering unti the midde of the 19th century. A divergence of interests ed engineers speciaizing in other technoogies to break away, and today civi engineers are the technica experts of the construction industry. They are concerned primariy with arge-scae, capita-intensive construction efforts, ike buidings, bridges, dams, tunnes, canas, highways, rairoads, pubic water suppies, and sanitation. As a rue, civi-engineering efforts invove we-defined task groups that use appropriate toos and technoo 17

4 1700: statics 1700: strength of materiats Fire Production 2. Evoution of civi engineering. gies to execute we-aid pans. Athough arge civi structures have been buit since before recorded history, ony in the ast few centuries has their design and construction been based on theoretica understanding rather than on intuition and accumuated experience. Neither the artisans of the Midde Ages nor of the ancient word showed any signs of the deiberate quantitative appication of mathematics to determine the dimensions and shapes that characterizes modern civi engineering. But even without forma understanding, they documented pragmatic rues for recurring eements. Practica buiders had highy deveoped intuitions about statics and reied on a few empirica rues. The scientific revoution of the Renais sance ed to serious attempts by Gaieo Gaiei, Fiippo Bruneeschi, and others to expain structures and why they worked. Over a period of about 200 years, there were attempts to expain the composition of forces and bending of a beam. However, progress was sowed for a ong time by probems in formuating basic notions ike force, particuary the idea that gravity coud be treated as just another force ike a the others. Unti the basic concepts were sorted out, it was not possibe to do a proper anaysis of the probem of combining forces (using vector addition) that we now teach to freshmen, nor was it possibe to dea with strengths of materias. Around 1700, Pierre Varignon and Isaac Newton deveoped the theory of statics to expain the composition of forces and Chares Augustin de Couomb and Louis Marie Henri Navier expained bending with the theory of strength of materias. These now provide the basis for civi engineering. By the midde of the 18th century, civi engineers were tabuating properties of materias. First century: ffomans : properties of matefas. fu ana&& of brhjge The mid-18th century aso saw the first attempts to appy exact science to practica buiding. Pope Benedict ordered an anaysis of St. Peter s dome in 1742 and 1743 to determine the cause of cracks and propose repairs; the anaysis was based on the principe of virtua dispacement and was carried out precisey (athough the mode is now known to fai to account propery for easticity). By 1850, it was pos sibe for Robert Stephenson s Britannia Tubuar Bridge over the Menai Strait between Waes and Engand to be subjected to a forma structura anaysis. Thus, even after the basic theories were in hand, it took another 150 years before the theory was rich enough and mature enough to have direct utiity at the scae of a bridge design. Civi engineering is thus rooted in two scientific theories, corresponding to two cassica probems. One probem is the composition of forces: finding the resutant force when mutipe forces are combined. The other is the probem of bending: determining the forces within a beam supported at one end and weighted at the other. Two theories, statics and strength of materias, sove these probems; both were deveoped around Modern civi engineering is the appication of these the* ries to the probem of constructing buidings. for neary two centuries, civi engineering has undergone an irresistibe transition from a traditiona craft, concerned with tangibe fashioning, towards an a, stract science, based on mathematica cacuation. Every new resut of research in structura anaysis and technoogy of materias signified a more rationa design, more economic dimensions, or entirey new structura possibiities. There were no apparent imitations to the possibiities of anaytica approach; there were no appar- ent probems in buiding construction [that] coud not be soved by cacuation, wrote Hans Straub in his history of civi engineering.4 You can date the transition from craft to commercia practice to the Romans extensive transportation system of the first century. The underying science emerged about 1700, and it matured to successfu appication to practice sometime between the mid-18th century and the mid-19th century. Figure 2 paces civi engineering s significant events on my mode of engineering evoution. Chemica mginem ng: a basis in practice. Chemica engineering is a very different kind of engineering than civi engineering. This discipine is rooted in empirica observations rather than in a scientific theory. It is concerned with practica probems of chemica manufacture; its scope covers the industria-scae production of chemica goods: sovents, pharmaceuticas, synthetic fibers, rubber, paper, dyes, fertiizers, petroeum products, cooking ois, and so on. Athough chemis try provides the specification and design of the basic reactions, the chemica engineer is responsibe for scaing the reactions up from aboratory scae to factory scae. As a resut, chemica engineering depends as heaviy on mechanica engineering as on chemistry. Unti the ate 18th century, chemica production was argey a cottage industry. The first chemica produced at industria scae was akai, which was required for the manufacture of gass, soap, and texties. The first economica industria process for akai emerged in 1789, we before the atomic theory of chemistry expained the underying chemistry. By the mid-19th century, industria production of dozens of chemicas had turned the British Midands into a chemica-manufacturing district. Laws were passed to contro the resuting poution, and poution-contro inspectors, caed akai inspectors, monitored pant compiance. One of these akai inspectors, G.E. Davis, worked in the Manchester area in the ate 1880s. He reaized that, athough the pants he was inspecting manufactured dozens of different kinds of chemicas, there were not dozens of different 18 IEEE Software

5 1800: atomic fheorv procedures invoved. He identified a coection of functiona operations that took pace in those processing pants and were used in the manufacture of different chemicas. He gave a series of ectures in 1887 at the Manchester Technica Schoo. The ideas in those ectures were imported to the US by the Massachusetts Institute of Technoogy in the atter part of the century and form the basis of chemica engineering as it is practiced today. This structure is caed unit o@rutions; the term was coined in 1915 by Arthur D. Litte. The fundamenta probems of chemica engineering are the quantitative contro of arge masses of materia in reaction and the design of cost-effective industria-scae processes for chemica reactions. The unit-operations mode asserts that industria chemica-manufacturing processes can be resoved into a reativey few units, each of which has a definite function and each of which is used repeatedy in different kinds of processes. The unit operations are steps ike fitration and carification, heat exchange, distiation, screening, magnetic separation, and fotation. The basis of chemica engineering is thus a pragmaticay determined coection of very high-eve functions that adequatey and appropriatey describe the processes to be carried out. Chemica engineering as a science... is not a composite of chemistry and mechanica and civi engineering, but a science of itsef, the basis of which is those unit operations [that] in their proper sequence and coordination constitute a chemica process as conducted on the industria scae. These operations.. are not the subject matter of chemistry as such nor of mechanica engineering. Their treatment is in the quantitative way, with proper exposition of the aws controing them and of the materias and equipment concerned in them, the American Institute of Chemica Engineers Committee on Education wrote in This is a very different kind of structure from that of civi engineering. It is a pragmatic, empirica structure - not a theoretica one. You can date the transition from craft to commercia practice to the introduction of the LeBanc process for akai in The science emerged with the British Figure 3. Evoution of chemica engineering. chemist John Daton s atomic theory in the eary 19th century, and it matured to successfu merger with arge-scae mechanica processes in the 1890s. Figure 3 paces chemica engineering s significant events on my mode. Softwaretechnoogy Where does software stand as an engineering discipine? For software, the probem is appropriatey an engineering probem: creating cost-effective soutions to practica probems, buiding things in the service of mankind. Information processing as an economic force. The US computer business - incuding computers, peripheras, packaged software, and communications - was about $150 biion in 1989 and is pro jetted to be more than $230 biion by The packaged-software component is projected to grow from $23.7 biion to $37.5 biion in this period, according to the Data Anaysis Group s fourth-quarter 1989 forecasts. Services, incuding systems integration and in-house deveopment, are not incuded in these figures. Wordwide, software saes amounted to about $65 biion in This does not incude the vaue of in-house deveop ment, which is a much arger activity. Word figures are hard to estimate, but the cost of in-house software in the US aone may be in the range of $150 biion to $200 biion.6 It is not cear how much modification after reease (so-caed maintenance ) is incuded in this figure. Thus, software is coming to dominate the cost of information processing. The economic presence of information processing aso makes itsef known through the actua and opportunity costs of systems that do not work. Exampes of costy system faiures abound. Less obvi : first industria akai process : unit operations ous are the costs of computing that is not even tried: deveopment backogs so arge that they discourage new requests, gigabytes of unprocessed raw data from sateites and space probes, and so on. Despite very rea (and substantia) successes, the itany of mismatches of cost, schedue, and expectations is a famiiar one. Growing roe of software in critica ap pications. The US Nationa Academy of Engineering recenty seected the 10 greatest engineering achievements of the ast 25 years? Of the 10, three are informatics achievements: communications and informationgathering sateites, the microprocessor, and fiber-optic communication. Two more are direct appications of computers: computer-aided design and manufacturing and the computerized axia tomography scan. And most of the rest are computer-intensive: the Moon anding, advanced composite materias, the jumbo jet, asers, and the appication of genetic engineering to produce new pharmaceuticas and crops. The conduct of science is increasingy driven by computationa paradigms standing on equa footing with theoretica and experimenta paradigms. Both scientiiic and engineering discipines require very sophisticated computing. The demands are often stated in terms of raw processing power - an exafop (1018) processor with teraword memory, a petabyte ( 10i5) of storage, as one artice put it* - but the supercomputing community is increasingy recognizing deveopment, not mere raw processing, as a critica botteneck. Because of software s pervasive pres ence, the appropriate objective for its deveopers shoud be the effective deivery of computationa capabiity to rea users in forms that match their needs. The dis November

6 Tabe 1. Significant shifts in research attention. Attribute 1960 f 5 years: programming any-which-way 1970 f 5 years: programming-in-the sma 1980 * 5 years: programming-in-the-arge Characteristic probems Data issues Sma programs Representing structure and symboic information Agorithms and programming Data structures and types Interfaces, management system structures Long-ived databases, symboic as we as numeric Contro issues Eementary understanding of Programs execute once and Program assembies execute contro fows terminate continuay Specification issues State space Mnemonics, precise use of prose State not we understood apart from contro Simpe input/output specifications Sma, simpe state space Systemswith compex specifications Large, structured state space Management focus None Individua effort Team efforts, system ifetime maintenance Toos, methods Assembers, core dumps Programming anguage, compiers, inkers, oaders Environments, integrated toos, documents tinction between a system s computa- agorithms and data structures did not tiona component and the appication it emerge unti 1967, when Donad Knuth serves is often very soft - the deveop showed the utiity of thinking about them ment of effective software now often re- in isoation from the particuar programs quires substantia appication expertise. that happened to impement them. A simiar shift in attitudes about specifications took pace at about the same time, when Robert Foyd showed how attaching ogica formuas to programs aows forma reasoning about the programs. Thus, the ate 1960s saw a shift from crafting Maturity of deveopment techniques. Our deveopment abiities have certainy improved over the 40 or so years of pro gramming experience. Progress has been both quaitative and quantitative. Moreover, it has taken different forms in the words of research and practice. One of the most famiiar characterizations of this progress has been the shift from programming-in-the-sma to pro gramming-in-the-arge. It is aso usefu to ook at a shift that took pace 10 years before that, from programming-any-whichway to programming-in-the-sma. Tabe 1 summarizes these shifts, both ofwhich describe the focus of attention of the software research community. Before the mid-1960s programming was substantiay ad hoc; itwas a significant accompishment to get a program to run at a. Compex software systems were created - some performed very we - but their construction was either highy empirica or a virtuoso activity. To make pro grams inteigibe, we used mnemonics, we tried to be precise about writing comments, and we wrote prose specifications. Our emphasis was on sma programs, which was a we coud hande predictaby. We did come to understand that computers are symboic information proces sots, not just number crunchers - a significant insight. But the abstractions of monoithic programs to an emphasis on agorithms and data structures. But the programs in question were sti simpe programs that execute once and then terminate. You can view the shift that took pace in the mid-1970s from programming-in-thesma to programming-in-the-arge in much the same terms. Research attention turned to compex systems whose speciications were concerned not ony with the functiona reations of the inputs and outputs, but aso with performance, reiabiity, and the states through which the systern passed. This ed to a shift in emphasis to interfaces and managing the programming process. In addition, the data of compex systems often outives the programs and may be more vauabe, so we earned that we now have to worry about integrity and consis tency of databases. Many of our programs (for exampe, the teephone switching sys- ment methodoogies tem or a computer operating system) shoud not terminate; these systems require a different sort of reasoning than do programs that take input, compute, pro duce output, and terminate. In systems that run indefinitey, the sequence of systern states is often much more important than the (possiby undesirabe) termination condition. The toos and techniques that accompanied the shift from programming-anywhich-way to programming-in-the-sma provided first steps toward systematic, routine deveopment of sma programs; they aso seeded the deveopment of a science that has matured ony in the ast decade. The toos and techniques that accompanied the shift from programming-in-the-sma to programming-in-thearge were argey geared to supporting groups of programmers working together in ordery ways and to giving management a view into production processes. This directy supports the commercia practice of deveopment. Practica deveopment proceeded to arge compex systems much faster than the research community did. For exampe, the Sage missie-defense system of the 1950s and the Sabre airine-reservation system of the 1960s were successfu interactive systems on a scae that far exceeded the maturity of the science. They appear to have been deveoped by exceent engineers who understood the requirements we and appied design and deveopment methods from other (ike eectrica) engineering discipines. Modern deveop are management procedures intended to guide arge numbers of deveopers through simiar discipines. The term software engineering was introduced in 1968 to name a conference 20 IEEE Software

7 convened by NATO to discuss probems of software production. Despite the abe, most of the discussion deat with the chaenge of progressing from the craft stage to the commercia stage of practice. In 1976, Barry Boehm proposed the definition of the term as the practica appication of scientific knowedge in the design and construction of computer programs and the associated documentation required to deveop, operate, and maintain them. O This definition is consistent with traditiona definitions of engineering, athough Boehm noted the shortage of scientific knowedge to appy. Unfortunatey, the term is now most often used to refer to ife-cyce modes, routine methodoogies, cost-estimation techniques, documentation frameworks, configuration-management toos, quaityassurance techniques, and other techniques for standardizing production activities. These technoogies are characteristic of the commercia stage of evoution - software management woud be a much more appropriate term.!scieniific basis for engineering practice. Engineering practice emerges from commercia practice by expoiting the resuts of a companion science. The scientific resuts must be mature and rich enough to mode practica probems. They must aso be organized in a form that is usefu to practitioners. Computer science has a few modes and theories that are ready to sup port practice, but the packaging of these resuts for operationa use is acking. Fmre A Cyce of how good software modes deveop as a resut of the InteractIon Detween science and engineering. be abstracted from individua programs, and their essentia properties were described and anayzed. The 1970s saw substantia progress in supporting theories, incuding performance anaysis and correctness. Concurrenty, the programming impications of these abstractions were expored; ab stractdata-type research deat with such is sues as: Specifications: abstract modes and agebraic axioms. Software structure: bunding representation with agorithms. Language issues: modues, scope, and user-defined types. Information hiding: protecting the integrity of information not in the specifi- cation. Maturity of supporting science. Despite the Integrity constraints: invariants of data criticism sometimes made by software structures. producers that computer science is irree- Composition rues: decarations. vant to practica software, good modes and theories huvebeen deveoped in areas that have had enough time for the thee ries to mature. In the eary 196Os, agorithms and data structures were simpy created as part of Both sound theory and anguage sup port were avaiabe by the eary 1980s and routine good practice now depends on this support. Compier construction is another good exampe. In 1960, simpy writing a comeach program. Some fokore grew up pier at a was a major achievement; it is about good ways to do certain sorts of things, and it was transmitted informay. By the mid-1960% good programmers not cear thatwe reay understood what a higher eve anguage was. Forma syntax was first used systematicay for AgodO, shared the intuition that if you get the and toos for processing it automaticay data structures right, the rest of the pro gram is much simper. In the ate 196Os, agorithms and data structures began to (then caed compier compiers, but now caed parser generators) were first deveoped in the mid-1960s and made practica in the 1970s. Aso in the 197Os, we started deveoping theories of semantics and types, and the 1980s have brought significant progress toward the automation of compier construction. Both of these exampes have roots in the probems of the 1960s and became genuiney practica in the 1980s. It takes a good 20 years from the time that work starts on a theory unti it provides serious assistance to routine practice. Deveopment periods of comparabe ength have aso preceded the widespread use of systematic methods and technoogies ike structured pro gramming, Smatak, and Unix, as Sam Redwine and coeagues have shown. But the whoe fied of computing is ony about 40 years od, and many theories are emerging in the research pipeine. Interaction between science and engineering. The deveopment of good modes within the software domain foows this pattern: We engineers begin by soving probems anyway we can. After some time, we distinguish in those ad hoc soutions things that usuay work and things that do not usuay work. The ones that do work enter the fokore: Peope te each other about them informay. As the fokore becomes more and more systematic, we codify it as written heuristics and rues of procedure. Eventuay, that codification becomes crisp enough to support modes and theories, together with the associated mathematics. These can then hep improve practice, November

8 15-70: agoritms, data structures \ Figure 5. Evoution of software engineering. and experience from that practice can sharpen the theories. Furthermore, the improvement in practice et us think about harder probems -which we first sove ad hoc, then find heuristics for, eventuay deveop new modes and theories for, and so on. Figure 4 iustrates this cyce. The modes and theories do not have to be fuy feshed out for this process to assist practice: The initia codification of fokore may be usefu in and of itsef. This progression is iustrated in the use of machine anguage for contro fow in the 1960s. In the ate 1950s and the eary 196Os, we did not have crisp notions about what an iteration or a conditiona was, so we aid down specia-purpose code, buiding each structure individuay out of test and branch instructions. Eventuay, a sma set of patterns emerged as generay usefu, generay easy to get right, and generay at east as good as the aternatives. Designers of higher eve anguages expicity identified the most usefu ones and codified them by producing specia-purpose syntax. A forma resut about the competeness of the structured constructs provided additiona reassurance. Now, amost nobody beieves that new kinds of oops shoud be invented as a routine practice. A few kinds of iterations and a few kinds of conditionas are captured in the anguages. They are taught as contro concepts that go with the anguage; pee pe use them routiney, without concern for the underying machine code. Further experience ed to verifiabe forma specifications of these statements semantics and of the programs that used them. Experience with the formaization in turn refined the statements supported in programming anguages. In this way, ad hoc practice entered a period of fokore and eventuay matured to have conventiona syntax and semantic theories that s software - deveopment methodoogies expain it. Where is software? Where, then, does current software practice ie on the path to engineering? It is sti in some cases craft and in some cases commercia practice. A science is beginning to contribute resuts, and, for isoated exampes, you can argue that professiona engineering is taking pace. (Figure 5 shows where software practice fits on my mode.) That is not, however, the common case. There are good grounds to expect that there wi eventuay be an engineering discipine of software. Its nature wi be technica, and itwi be based in computer science. Athough we have not yet matured to that state, it is an achievabe goa. The next tasks for the software profes sion are to pick an appropriate mix of shortterm, pragmatic, possibe purey empirica contributions that hep stabiize commercia practice and to invest in ong-term efforts to deveop and make avaiabe basic scientific contributions. T he profession must take ive basic steps on its path to becoming a true engineering discipine: Understand the nature of expertise. Proficiency in any fied requires not ony higher order reasoning skis but aso a arge store of facts together with a certain amount of context about their impications and appropriate use. Studies have demonstrated this across a wide range of probem domains, incuding medica diagnosis, physics, chess, financia anaysis, architecture, scientific research, poicy decision making, and others, as Herbert Simon described in the paper Human Experts and Knowedge-Based Systems presented at the 1987 IFIP Working Group 10.1 Workshop on Concepts and Characteristics of Knowedge-Based Systems. An expert in a fied must know about 50,000 chunks of information, where a chunk is any custer of knowedge sufficientyfamiiar that it can be remembered rather than derived. Furthermore, in do mains where there are fu-time profes sionas, it takes no ess than 10 years for a wordcass expert to achieve that eve of proficiency. Thus, fuency in a domain requires content and context as we as skis. In the case of natura-anguage fuency, E.D. Hirsch has argued that abstract skis have driven out content; students are expected (unreaisticay) to earn genera skis from a few typica exampes rather than by a piing up of information ; and inteectua and socia skis are supposed to deveop naturay without regard to the specific content.12 However, Hirsch wrote, specific information is important at a stages. Not ony are the specific facts important in their own right, but they serve as carriers of shared cuture and shared vaues. A software engineer s expertise incudes facts about computer science in genera, software design eements, programming idioms, representations, and specific knowedge about the program of current interest. In addition, it requires ski with toos: the anguage, environment, and support software with which this program is impemented. Hirsch provided a ist of some 5,000 words and concepts that represent the information actuay possessed by iterate Americans. The ist goes beyond simpe vocabuary to enumerate objects, concepts, tites, and phrases that impicity invoke cutura context beyond their dictio nary definitions. Whether or not you agree in detai with its composition, the ist and accompanying argument demonstrate the need for connotations as we as denotations of the vocabuary. Simiary, a programmer needs to know not ony a programming anguage but aso the system cas supported by the environment, the genera-purpose ibraries, the appication-specific ibraries, and how to combine invocations of these definitions effectivey. The programmer must IEEE Software

9 Cost distributions Tabe 2. for the thee ways to get a piece of information. Method Infrastructure cost Initia-earning cost Cost of use in practice be famiiar with the goba definitions of the program of current interest and the rues about their use. In addition, a deveoper of appication software must understand appication-area issues. Simpy put, the engineering of software woud be better supported ifwe knew better what specific content a software engineer shoud know. We coud organize the teaching of this materia so usefu subsets are earned first, foowed by progressivey more sophisticated subsets. We coud aso deveop standard reference materias as carriers of the content. Recognize different ways to get information. Given that a arge body of knowedge is important to a working profes siona, we as a discipine must ask how software engineers shoud acquire the knowedge, either as students or as working professionas. Generay speaking, there are three ways to get a piece of information you need: You can remember it, you can ook it up, or you can derive it. These have different distributions of costs, as Tabe 2 shows. Memorization requires a reativey arge initia investment in earning the materia, which is then avaiabe for instant use. Reference materias require a arge investment by the profession for deveoping both the organization and the content; each student must then earn how to use the reference materias and then do so as a working professiona. Deriving information may invove ad hoc creation from scratch, it may invove instantiation of a forma mode, or it may invove inferring meaning from other avaiabe information. To the extent that forma modes are avaiabe, their formuation requires a substantia initia investment. Students first earn the modes, then appy them in practice. Because each new appication requires the mode to be appied anew, the cost in use may be very high.13 Each professiona s aocation of effort among these aternatives is driven by what he has aready earned, by habits deveoped during that education, and by the reference materias avaiabe. Today, generai-purpose reference materia for Software is scarce, athough documentation for specific computer systems, anguages, November 1990 Memory Low High Low Reference High Low Medium Derivation Medium-high Medium High and appications may be extensive. Even when documentation is avaiabe, however, it may be underused because it is poory indexed or because deveopers have earned to prefer fresh derivation to use of existing soutions. The same is true of subroutine ibraries. Simpy put, software engineering requires investment in the infrastructure cost - in creating the materias required to organize information, especiay reference materia for practitioners. Encourage routine practice. Good engineering practice for routine design depends on the engineer s command of factua knowedge and design skis and on the quaity of reference materias avaiabe. It aso depends on the incentives and vaues associated with innovation. Unfortunatey, computer-science education has prepared deveopers with a background that emphasizes fresh creation amost excusivey. Students earn to work aone and to deveop programs from scratch. They are rarey asked to understand software systems they have not written. However, just as natura-anguage fuency requires instant recognition of a core vocabuary, programming fuency shoud require an extensive vocabuary of definitions that the programmer can use famiiary, without repeated recourse to documentation. Fred Brooks has argued that one of the great hopes for software engineering is the cutivation of great designers.14 Indeed, innovative designs require great designers. But great designers are rare, and most designs need not be innovative. Systematic presentation of design fragments and techniques that are known to work can enabe designers of ordinary taent to produce effective resuts for a wide range of more routine probems by using prior resuts (buying or growing, in Brooks s terms) instead of aways buiding from scratch. It is unreasonabe to expect a designer or deveoper to take advantage of scientific theories or experience if the neces sary information is not readiy avaiabe. Scientific resuts need to be recast in operationa form; the important information from experience must be extracted from exampes. The content shoud incude design eements, components, interfaces, interchange representations, and ago rithms. A conceptua structure must be deveoped so the information can be found when it is needed. These facts must be augmented with anaysis techniques or guideines to support seection of aternatives that best match the probem at hand. A few exampes of we-organized reference materias aready exist. For exampe, the summary fowchart of Wiiam Martin s sorting survey captured in one page the information a designer needed to choose among the then-current sorting techniques. Wiiam Cody and Wiiam Waite s manua for impementing eementary mathematica functions16 gives for each function the basic strategy and specia considerations needed to adapt that strategy to various hardware architectures. Athough engineering has traditionay reied on handbooks pubished in book form, a software engineers handbook must be on ine and interactive. No other aternative aows for rapid distribution of updates at the rate this fied changes, and no other aternative has the potentia for smooth integration with on-ine design toos. The on-ine incarnation wi require soutions to a variety of eectronic-pubishing probems, incuding distribution, vaidation, organization and search, and coection and distribution of royaties. Simpy put, software engineering woud benefit from a shift of emphasis in which both reference materias and case studies of exempary software designs are incorporated in the curricuum. The discipine must find ways to reward preparation of materia for reference use and the deveopment of good case studies. Expect professiona speciaizations. As software practice matures toward engineering, the body of substantive technica knowedge required of a designer or deveoper continues to grow. In some areas, it has ong since grown arge enough to 23

10 require speciaization - for exampe, database administration was ong ago sep arated from the corresponding programming. But systems programming has been resistant to expicit recognition of profes siona speciaties. In the coming decade, we can expect to see speciaization of two kinds: interna speciaization as the technica content in the core of software grows deeper and externa speciaization with an increased range of appications that require both substantive appication knowedge and substantive computing knowedge. Interna speciaties are aready starting to be recognizabe for communications, reiabiity, rea-time programming, scientific computing, and graphics, among otbers. Because these speciaties rey criticay on mastery of a substantia body of computer science, they may be most appropriatey organized as postbaccaaureate education. Externa speciaization is becoming common, but the required dua expertise is usuay acquired informay (and often incompetey). Computationa speciaizations in various discipines can be sup ported via joint programs invoving both computer science and the appication department; this is being done at some universities. Simpy put, software engineering wi require expicit recognition of speciaties. Educationa opportunities shoud be pro vided to support them. However, this shoud not be done at the cost of a soid foundation in computer science and, in the case of externa speciaization, in the appication discipine. Improve the couping between science and commercia practice. Good science is often based on probems underying the probems of production. This shoud be as true for computer science as for any other discipine. Good science depends on strong interactions between researchers and practitioners. However, cutura differences, ack of access to arge, compex systems, and the sheer difficuty of understanding those systems have interfered with the communication that sup ports these interactions. Simiary, the adoption of resuts from the research community has been impeded by poor understanding of how to turn a research resut into a usefu eement of a production environment. Some companies and universities are aready deveoping cooperative programs to bridge this gap, but the ogistics are often daunting. Simpy put, an engineering basis for software wi evove faster if constructive interaction between research and production communities can be nurtured. 9 Acknowedgments This artice benefited from comments by Aen Newe, Norm Gibbs, Frank Friedman, Tom Lane, and the other authors of artices in this specia issue. Most important, Edon Shaw fostered my appreciation for engineering. Without his support, this work woud not have been possibe, so 1 dedicate this artice to his memory. This work was supported by the US Defense Dept. and a grant from Mobay Corp. References 1. J.K. Finch, Engineering and Western Ciuitization, McGraw-Hi, New York, E.W. Dijkstra, On the Cruety of Reay Teaching Computing Science, &mm. Am Dec. 1989, pp. 1,3981, D.L. Parnas, Education for Computing Professionas, Cum@&r,Jan. 1990, pp H. Straub, A History of Civi En&wing An OutZinefiom Ancknt to Modern Times, MIT Press, Cambridge, Mass., F.J. van Antwerpen, The Origins of Chemica Engineering, in H&my of ChC ca Engineering, W.F. Furter, ed., American Chemica Society, Washington, D.C., 1980, pp Computer Science andtechnoogy Board, Nationa Research Counci, k&@ng the US Computer Industry Competitive, Nationa Academy Press, Washington, D.C., Nationa Academy of Engineering, En@- neering and the Advancaat of Human Wefare: 10 OutstandingAchkwm.ents , Nationa Academy Press, Washington, D.C., E. L&n, Grand Chaenges to Computa tiona Science, Comm. ACM, Dec. 1989, pp. 1,4561, So&we Engineering: Repotion a Conference Spon.sozd by &NATO Science Committee, Garmisch, t&many, 1968, P. Naur and B. Rande, eds., Scientific Affairs Div., NATO, Brusses, B.W. Boehm, Software Engineering, Z&EETmns. Computers, Dec. 1976, pp. 1,226 1, S.T. Redwine et a., DOD-Reated Software Technoogy Requirements, Practices, and Prospects for the Future, Tech. Report P- 1788, Inst. Defense Anayses, Aexandria, Va., E.D. Hirsch,Jr., CuUuraLiterugc W&Every American Needs to Know, Houghton Miffin, Boston, M. Shaw, D. Giuse, and R. Reddy, What a Software Engineer Needs to Know I: Vo cabuary, tech. report CMU/SEI89-TR- 30, Carnegie Meon Univ., Pittsburgh, Aug F.P. Brooks, Jr., No Siver Buet: Essence and Accidents of Software Engineering, Znfonatkm Procesring86, pp. 1,069-, W.A. Martin, Sorting, ACM Computing Survqrs, Dec. 1971, pp WJ. Cody, Jr., and W.M. Waite, Software Manua f&r the Eementary Function$ Prentice-Ha, Engewood Ciffs, NJ., Mary Shaw is a professor of computer science at Carnegie Meon University, where she has been on the facuty since From 1984 to 1987, she was chief scientist at the Software Engineering Institute, with which she sti has a joint appointment. Her primary research in- terests are programming systems and software engineering, particuary abstraction techniques and anguage toos for deveoping and evauating software. Shaw received a BA in mathematics from Rice University and a PhD in computer science from Carnegie Meon University. She is a member of the IEEE Computer Society, ACM, New York Academy of Sciences, and Sigma Xi. She aso serves on the Nationa Research Counci s Computer Science and Technoogy Board, IEEE Technica Committee on Software Engineering, and IFIP Working Group 2.4 (System Impementation Languages). Address questions about this artice to the author at Computer Science Dept., Garnegie Meon University, Pittsburgh, PA ; Internet shaw@cs.cmu.edu. 24 IEEE Software