ARVIKA - Augmented Reality for Development, Production and Service

Transcription

1 ARVIKA - Augmented Reality for Development, Production and Service Dipl.-Ing. Wolfgang Friedrich Siemens AG, Automation and Drives Advanced Technologies and Standards Gleiwitzerstr. 555, Nuremberg, Germany Abstract Augmented Reality (AR) is a form of human-machine interaction where information is presented in the field of view of an individual e.g. through a head mounted display, thus augmenting his or her perception of reality. This occurs in a context-sensitive manner, i.e. in accordance with and derived from the observed object, such as a part, tool or machine, or his or her location. In this way, the real-world field of view of a skilled worker, technician or design engineer is augmented with superimposed notes to present information that is relevant to this individual. ARVIKA develops this technology or applications in the fields of design, production, and service in the automotive and aerospace industries, for power and processing plants and for machine tools and production machinery. This technology offers special dimensions for mid-sized businesses that can leverage an improved diagnostic and maintenance competence to increase their flexibility and efficiency, thereby strengthening their global competitive position. ARVIKA /1/ is primarily designed to implement an Augmented Reality system for mobile use in industrial applications. This article presents the results that have been achieved after a project duration of almost four years: A basic system for Augmented Reality systems was developed based on a specially designed software architecture. About 30 application-specific prototypes have been created on this basis and have been evaluated in usability tests. The solutions that can be demonstrated today indicate very good, worldwide acknowledged results of the project in terms of the base technologies, such as Augmented Reality, interaction as well as AR-oriented information delivery and workflow, but they also show the challenges for major further developments. These further developments need to focus on AR authoring tools and AR-compatible information structures, tracking methods that doesn t require to change the environment and wearable and personalizable AR devices that make it possible to use AR not only in niche applications but also in mainstream industrial applications. ARVIKA is the world s largest project for the use of Augmented Reality in industrial application and, thanks to numerous inventions, has been exemplary in the protection of intellectual property for the German scientific community and the industry.

2 1 Project Overview and Vision 1.1 General situation and project overview Augmented Reality is an innovative form of human-machine interaction where information is presented in the field of view of a user e. g. through a head mounted display. This occurs in a context-sensitive manner, i.e. in accordance with and derived from the observed object, such as a part or an assembly environment. In this case, Augmented Reality replaces the traditional assembly manual and provides additional updated information that are relevant to the process, such as pressure, temperature, rpm, etc. In addition to this situation-sensitive interaction, the use of computers that can be worn on the body enables AR applications that require a high degree of mobility as well as process, measuring or simulation data to support the workflow. Up to now, AR has only been a subject of individual research projects and a small number of application-specific industrial projects on a global scale /5/ /6/. The current state of the art and the available appliances in 2003 only permit a niche-oriented application of the technology. However, AR enables a new, innovative form of human-machine interaction that not only places the individual in the center of the industrial workflow but also offers a high potential for process and quality improvements in production and process workflows. While Virtual Reality, especially in the development phases of a product, supports the design and improvement of products without any real environment, Augmented Reality focuses on the real product and the real environment and augments this reality in a situation-sensitive manner with information right attached to the object that can enable or facilitate the design, manufacture or maintenance of an industrial product. This is the reason why the German ministry of Education and Research (Bundesministerium für Bildung und Forschung, BMBF) sponsors the ARVIKA pilot project to support the development of Augmented Reality technologies that assist workflows in design, production, and maintenance operations for complex technical products and plants, both in a user-centric and application-driven manner. Figure 1: ARVIKA project structure The primary application-specific topics of ARVIKA (see Figure 1) aim at testing Augmented Reality in the development, production and service of a product in the field or the service of the machines and systems that are required for production. More specifically, the planned scope of the project will focus on automotive and aircraft equipment in the field of design, on

3 automotive and aircraft assembly in the field of production, and on power and process plants, the areas of automotive and aircraft as well as the required production/manufacturing machine tools and machines in the field of service. This will cover the major industrial applications for AR, eliminate duplicate development efforts and achieve the necessary application depth to verify this innovative technology. These main topics are embedded in a user-centric system design that accompanies the project in all phases using ergonomic methods with respect to ergonomic principles, Usability Engineering and system-ergonomic investigations. The basis for all development work in the various fields of application are the Augmented Reality Base technologies that support high-end / power applications in design in the same way as the low-end use of the technology as a belt-mounted appliance by a skilled worker in real-world production or service environments. They provide the AR modules, the components for information provision and the AR-compatible types of interaction. 1.2 Visions and results The vision of the project is the provision of user-centric stationary and mobile AR systems for industrial applications. The consortium partners have documented and published their visions, goals, and the status of the development in numerous publications, status meetings and at several ARVIKA forums in more than 30 application-oriented projects (as an example see: Several AR application visions and their results are explained in the examples below. Field of application: Design The comparison of test and calculation of the crash test scenario (Figure 2, VW) assists the calculation engineer with AR techniques in structural simulation. Figure 2: Example from the field of application "Design" Following the crash test, AR technology enables the immediate comparison of calculated results with the actual deformations of the crashed vehicle. The data are mostly graphic information (finite elements method grid) that is linked with results such as stress, strain, internal energy and can be enhanced with textual information. Figure 2 illustrates the vision as well as the actual results that were achieved in the project. The overlay of the simulation data on the crashed door immediate shows the deviations from the deformed door. Other scenarios cover the Economic layout design in the aircraft cockpit (DC, EADS), the Flow visualization of aircraft passenger seats (DC, Airbus) and the Layout design and

4 improvement of prototyping in automotive manufacturing (Audi, BMW). Selected applications are supported by a miniaturized, personalized infrared tracking system (A.R.T.). Some of the mentioned examples will be put into productive service upon completion of the project, i.e. they will be integrated into the development processes of the participating industrial partners. Field of application: Production/Assembly Two key issues were developed and tested in the Production/Assembly field of application: the assembly of industrial products and the creation and commissioning support of production equipment by means of AR technology. An AR assembly process that has already become a classic was developed and tested for the production of cable harnesses for the Eurofighter. In this process, up to 60 different, highly complex cable harnesses are produced on cable boards with the dimensions of 1 x 6 m (Figure 3, EADS). The assembly technician retrieves the routing schedule from the workshop order management system in incremental steps according to his relative position to his location in front of the cable board. Via an AR headset (visual augmenting, recording of head movements, speech input/output), he is provided with the starting point as well as the routing direction of the cable. He can request situation-specific information via speech control. The evaluation with a large number of test persons based on the available AR headset yielded the result that the use of AR in this area of production is considered promising after a further optimization phase. Figure 3: Example from the field of application "Production" Other scenarios include AR-assisted quality assurance along the production line (VW), Assembly of fresh water system and Connection of an electrical distribution panel (Airbus), and Assembly optimization in one-off and small batch production (WZL/RWTH- Aachen). The building and commissioning support of production equipment by means of AR technology was impressively demonstrated in scenarios such as Assembly of new production equipment (Ford) and AR-assisted factory and plant planning (VW) and tested with commission engineers, also at separate geographic locations. The scenario AR-assisted factory and plant planning (Figure 4, VW) shows very impressively how limitations of virtual planning techniques are solved with AR. For example, the required computing power for comprehensive plant models, differences between virtual and real halls, and virtual, non-acquired production equipment are no longer important. Using markers on the floor, the production equipment to be planned (robots, conveyors, manual stations) are positioned and optimally arranged in the real space. It is also possible to place

5 workers in this mixed real/virtual environment and take them into account in the workflow. This permits the inclusion of operators in the planning process using a realistic representation. Field of application: Service Figure 4: AR-assisted factory and plant planning AR will play a dominant role in service applications but will also be very exacting in terms of mobility, ruggedness and in an industrial environment, etc. (Figure 5). The application scenario Troubleshooting and service on production machines is designed to use Augmented Reality technologies to assist service personnel and end users in troubleshooting, commissioning, maintenance, and repair in the field or through direct interaction with the service center. In the early stage of the project, this was documented by an impressive vision video /7/ for internal and external project partners. The video is considered the benchmark for AR applications and used by equipment manufacturers worldwide. Figure 5: Example from the field of application "Service" The service case on a machine tool that was developed and tested in the project covers (as shown in the vision video) the AR-assisted troubleshooting autonomous and by means of a Remote Expert, the AR-assisted repair and the archiving of the gathered knowledge for future use worldwide. The scenario uses the linking of documents for selective information retrieval, the use of information units for augmenting, a symptom database based on service reports and the remote augmenting of the field of view of the on-site technician by the hotline agent. In addition to the static information from documents, drawings and service reports, real-time data from the machine controller are also displayed within the field of view of the service technician. This exemplary scenario was processed, using a changing focus, by the

6 participating machine tool manufacturers (Index, Hüller-Hille, Gühring, Ex-Cell-0 and DS- Technologie). Especially in the Service field of application, this scenario was considerably extended over the duration of the project, and new scenarios from the aircraft and automotive industry were developed and tested. In the area of service and maintenance of power and process plants (Framatome ANP), the functions of localization, logging, noting and annotation for inspection and maintenance were further developed. The new scenario for the extension of an existing piping system was developed. Using AR technology, it was checked whether a new pipe would fit into the space that was already fitted with units while avoiding collisions and meeting the necessary installation requirements. Due to the harsh environment, Tablet-PCs with cameras were used as augmentation platforms. In the field of application Military Aircraft, a maintenance assistant demonstrator for the NH90 NATO helicopter was built (EADS). Using AR, it is possible to visualize installations that are vital for airworthiness behind a damaged external skin, thus enabling an informed decision for safe take-off. For the automotive service case, the scenarios of Replacement of the Valvetronic servo motor on the 8-cylinder engine as well as Troubleshooting and repair of the sliding/lifting roof module of the BMW Series 7 were developed and tested (BMW). The scenarios were evaluated with more than 30 test persons. The positive response led to the planning of a field test on an international scale, which is conducted by BMW on the basis of the ARVIKA software. 1.3 Implementation and supplementary conditions The implementation of these visions for the fields of application Design, Production and Service required an integrated approach. Therefore: ARVIKA is Augmented Reality with object identification and information attachment on the identified object. ARVIKA is Information Delivery for AR applications as a function of the working context; this includes extracts from documents in different formats but especially also process data of the machine/plant as well as data from legacy systems (SAP, PDM,...). ARVIKA is Interaction, i.e. ARVIKA considers different input/output devices, speech input and output for hands-free operations and designs new user interfaces for mobile IT-assisted work using Webpads and head mounted displays. It is also assumed for ARVIKA that AR is introduced in a real-world system and process environment and becomes a part of it. This makes ARVIKA the only project that embeds the application-oriented research of AR into an integrated approach and tests it with end users. The applications are embedded in the respective industry-specific IT environments as well as system and process environments.

7 2 The Solution: An Integrated AR System The architecture that has been designed in the ARVIKA project offers the highest possible openness toward extensions, customization etc. and supports the latest IT advances. This not only makes it possible to implement future applications but also to provide AR researchers around the world with a platform to include new algorithms for tracking, calibration etc., thus meeting the requirements of additional applications. The AR Base technologies support stationary applications using high-end graphics systems in design labs as well as mobile low-end systems in the form of belt-worn appliances to be used by skilled workers in a real production or service environment. The following topics have been identified as focal points for the implementation of the technologies: Tracking is a key technology for AR. The position and orientation of the user head or possibly the eye must be accurately recorded by the tracking in order to overlay the virtual information with the proper orientation with respect to the real environment in the user s field of view. Information provision enables the user to make use of the data from the IT world or the processes according to the situation. Any AR application that aims at improving the industrial workflow must provide an intelligent AR-oriented information system. The interaction in an AR application is characterized by the different input/output devices, especially speech input and output that support hands-free operations. 2.1 Architecture The architecture of the Base technologies is component-based. This allows the provision of the system basis and the implementation of applications according to the building block principle. In this process, two major areas have emerged: stationary systems for high-end solutions and mobile solutions in Web-based environment (Figure 6). Components such as AR browser, tracking and device integration can be used in both fields of application. Stationary system Mobile system Application Internet Browser Web Server Device Interface Tracking Device Interface Tracking Arvika- Base Services AR- Browser AR- Browser Application Platform (IRIX, Linux, Windows) Platform (Windows) Platform (Windows/Linux) Figure 6: Stationary vs. mobile solution The Base ARVIKA services (Figure 7) are implemented by client- and server-side components. Web integration is implemented with a Servlet interface where the output via Internet Explorer is based on Java Server Pages. The runtime environment is the Tomcat

8 Servlet Engine Apache combined with Java 2 Standard Edition. Integration with other Web servers (Microsoft IIS, Apache,...) is possible. The Context Manager stores all information that describe the situation or context of a user (HW/SW profile, user profile, current activity etc.). The basic components are largely decoupled through its event-based communications. The InfoService enables the transparent access to information in a wireless and mobile environment. The InfoBroker assists the user with a situation-sensitive delivery of information that has previously been described in the product model in XML format. A consistent data interface supports the connection of legacy systems for the delivery of process or product data. The WorkflowEngine guides the user through predefined assembly or maintenance steps, offering AR assistance and documentation for every step. A specific XML dialect was defined for this purpose. NetCollaboration enables the integration of remote experts, e. g. a hotline, into AR-assisted work situations, which makes it possible to look over the local user s shoulder and to augment the user s field of view with expert instructions. The User Interface Configuration customizes the contents and interactions depending on the input/output devices (HMD or Webpad) and the current context of the user. Workflow- Editor AR-Editor Informationsspace XML Workflows XML AR-Scenes Internet Browser Web Server VRML Tracking Intersense Tracking (hybride) Device Interface (IDEAL) Legacy- System Legacy- System Collabo- NetCollarativeAboration Localization Video Server AR- Browser Tracking (marker based) Tracking (markerless) InfoService Workflow Engine Net Collaboration InfoBroker Context-Manager Annotation System Application extension User Interface- Configuration PDF- Viewer CAD- Viewer Platform (Windows) Platform (Windows/Linux + Apache Tomcat) Figure 7: ARVIKA Architecture for mobile applications The AR Browser is the core system for the interaction and visualization in an augmented environment. Three-dimensional data are loaded and processed in real time. A device interface enables the use of different AR-related input and output devices /2/. An interaction component makes it possible to assign a behavior to individual superimposed objects, such as movements, color changes or selection modes, and to link it to corresponding interaction mechanisms. The AR Browser also provides components that allow the superimposition of the presentation with real images (optical see-through or video see-through). The AR Browser was implemented both as a standalone application (IRIX, Windows and Linux for high-end applications) and as an ActiveX component (Windows for low-end stationary and mobile applications) and can thus be integrated into other applications (e. g. in other Internet browsers). In addition, the AR Browser is linked to a workflow module that provides the user e. g. with step-by-step maintenance or troubleshooting instructions.

9 The AR Browser also provides for video-based tracking. A library of image processing routines allows the identification of markers within the image as well as the recovery of the required tracking data. The methods that have been developed in the ARVIKA project are robust, precise and offer the necessary performance for mobile AR applications. With a 500 MHz wearable computer, a frame rate of 10 images per second can be obtained. A method for the calibration of see-through head-mounted displays has been integrated and is available for evaluation. The tracking method that has been implemented in ARVIKA relies on marker-based tracking. Since it is not practical in numerous industrial applications to apply markers to the product or its environment, the implementation of the so-called markerless tracking has been initiated. In this process, so-called reference images are recorded in a first engineering step, which will be compared to the camera video stream at runtime. The algorithm attempts to correlate the camera image with one of the reference images within the color frequency space. Rotation and minor scaling are detected in the process. This means that the algorithm is not yet suitable to replace the marker-based tracking because the 3 rd dimension is missing. 2.2 Stationary Augmented Reality systems High-performance stationary systems are used for the scenarios in the Design field of application. This is characterized by extreme requirements in terms of the quality of the presentation, the accuracy of the tracking methods and the performance that can currently only be met by the use of high-end equipment (computer hardware, tracking methods, etc.). This is also the reason for the use of an existing VR system that has already been optimized for this purpose. Accordingly, a version of the ARVIKA system was implemented with the first prototype that meets these requirements. In the course of the project, significant improvements of the AR browser developed in ARVIKA permitted the implementation of IRIX-dominated applications. To validate the usability for AR applications, various tracking systems that were available in the market were evaluated for their positioning accuracy. The comparison was based on a real model under experimental conditions. The results were used to optimize existing systems or led to in-house developments. For example, a miniaturized, personalized infrared tracking system was developed in the context of the project. The miniaturized camera is produced in two variants: a lightweight version suitable for inside/outside tracking and a version for stationary use. Since none of the known tracking methods is capable of delivering comprehensive and satisfactory solutions, work on hybrid tracking was started but not yet completed. The augmentation was implemented using optical see-through displays. The tests with the first prototype yielded the result that the currently available displays do not yet meet the requirements. This is particularly true for the parameters visible field of view, transparency (optical see-through) and resolution as well as for the indispensable wearing comfort. 2.3 Mobile Augmented Reality systems A scenario like the one presented below can already be demonstrated using the prototype of the AR system and were subjected to an on-site evaluation. In the case of a malfunction, the AR system provides the service technician with component and situation-related assistance from the information system. The cause of the fault can then be isolated located based on the error description and finally detected with the help of AR. To

10 correct the problem, the workflow guides the service technician step by step through the maintenance instructions in the info module for this component. If required, the service technician can be supported in finding and correcting the fault by an expert in the service center by means of augmentable video and audio communications. Mobile applications require an adequately high, work-related frame rate per second that often exceeds the capabilities of the currently available mobile computers. The information display is generally limited to circles, arrows, and short texts. However, the information to be delivered must meet high requirements. In the case of a service call e. g. on a production machine, the entire documentation of a machine plus current process data must be available. These data are centrally stored in an AR-compatible format on one or more servers so that they can be retrieved by a mobile device over a network. The currently available information structures are not suitable for AR-assisted applications. For the application-specific prototypes of mobile AR systems, the ARVIKA project developed an information system that consolidates and links the information items that are currently available but have been created independently. This enables the service technician to navigate within a machine-dependent product structure and access documents for specific components. The information for a machine component with respect to operation, programming, maintenance, failures, etc. is assessed with a defined XML structure and organized in selfcontained units (info modules). The info modules consist of small, independent information items such as warnings, specifications, or individual operations. The creation of workflows is supported by editors. All of these information items can be displayed in a situation-specific manner as superimposed images in the mobile user devices, Webpads or mobile computers that are equipped with HMDs. 3 User-Centric Design and Documented Experiences 3.1 User-centric system design Ergonomic methods are used in order to advance the user-centric and user-driven development of Augmented Reality technologies. This ensures that the developed AR systems meet the requirements of users and work processes, that the user interfaces are ergonomically designed and that AR functionality enables improvements of the work organization. To begin with, the customer or user requirements, respectively, in terms of the assistance of work processes by AR systems are assessed in the analysis phase (Figure 8). By using a scenario-based method, it is possible to assess typical activities in the fields of application along the value chain and prioritize them with regard to potential user- and task-oriented improvements. Another major focus of the user-centric system design was the design and development of AR systems from the perspective of hardware and software ergonomics. In addition to basic research dealing with issues such as the minimum required display size of information (text or graphics) in head mounted displays, usability tests and system-ergonomic research could be used to create hypotheses for task-oriented steps for AR systems. In additional steps, the prototype AR systems were evaluated with regard to the critical scenarios. Commercially available AR products and AR technologies such as visualization

11 components, tracking systems or interaction tools are being examined in the respective field of application from an information technology point of view. Figure 8: User-centric system design 3.2 Ergonomic principles To overlay the real world with additional virtual information, Head Mounted Displays (HMDs) play a key role. In the early stage of the project, the investigation focused on the readability/perception of the virtual information in the use of such displays as well as other characteristics of the viewing process, including human error. In addition, comprehensive measurements were conducted on seven commercial HMDs to determine various mechanical and optical characteristics /8/. The evaluation of another HMD wearing experiment with 19 test persons provided additional findings about the origin of discomfort effects. It was found that the lid closure frequency is also greatly reduced when HMDs are used, similar to work at display terminals. This is probably one of the causes of the expressed eye troubles. In addition, a relationship was found between the use of head worn systems and headaches. By using the Zeiss system as base for head worn appliances it was possible to reduce the discomfort by up to 23%. The problems of people wearing glasses with HMDs were analyzed. Calculations were made and possible solutions developed. As special focus was put on the issue of using continuous bifocal glasses with HMDs. In addition, a series of experiments were conducted to determine the influence of tracking on information processing. Such an influence is not statistically relevant, i.e. the perception of a tracked information (that is overlaid on the real environment) occurs just as fast as with the provision of untracked information. Another issue in an augmented environment is the maximum degree of coverage of the real world, i.e. how much of the real world may be covered without causing the risk that safetyrelated details of the environment are overlooked during the work (work safety issues). Within the investigated overlay areas, no significant change in the quality of information perception (processing accuracy) could be detected. With respect to the processing quantity, a significant improvement of the speed of information perception (processing time) could even be recorded up to a certain degree of overlay. However, it is suspected that this occurred in an area where a restriction of the view of the real world did cause performance impairments, but where such impairments were more than compensated by the faster recognition of the virtual

12 information. Based on these findings, no major impairment of the perception of real-world information is assumed in the area that has been defined for AR for an overlay of the real world with virtual information (50% maximum according to Milligram's virtuality-reality continuum). The degree of overlay should be significantly lower in rapidly changing environments. In environments that are close to production, a coverage degree of about 15% was considered optimal. A large number of commercially available Head Mounted Displays were tested and evaluated in the ARVIKA project. The products that are available in the market today meet neither the technical nor the ergonomic requirements. None of the systems met the requirement of system-capability, i.e. the integration of projection, visual information input and speech input and output. Only one system tolerated visual defects. The requirements that were determined in the project were discussed with several global HMD manufacturers. According to the manufacturers, AR HMDs based on the ARVIKA specifications are being developed. 3.3 Usability Engineering Design rules for user interfaces and the interaction of AR systems, Webpads, and HMDs have been documented in an AR Style Guide (Figure 9) and can guide application developers in the implementation of application-specific prototypes. The Style Guide for AR Systems is the first of its kind worldwide. Figure 9: Examples of user interfaces for Webpad and HMD The Style Guide describes a complete set of dialog elements for mobile AR-assisting information systems. The dialog elements include basic elements such as text input, numbering schemes and complex elements such as registers, menus, tables, etc. The dialog elements are described for different platforms but are all based on a consistent concept. In this manner, the Style Guide not only supports the application developers, but also promote a seamless, context-sensitive change of equipment by the end user. 3.4 System-ergonomic studies In the field of system ergonomics, a Remote Expert system was investigated. This system with augmented and tracked pointing capabilities enables a hotline agent to provide the assembly worker with superimposed position and additional information in the field. In addition to the Remote Expert system, two alternative variants for assistance via a hotline were evaluated (hands-free phone and videoconferencing system). The AR Remote Expert system in its current form competes with traditional videoconferencing systems and cannot yet be considered fit for everyday use. However, while the videoconferencing system performs better in an assembly task, the AR Remote Expert system supports the necessary hands-free operations, the provision of additional information in the national language and

13 online marking capabilities. Additional studies are being conducted, e.g. for service work on a machine toll. In addition to time and quality, the perceived strain as well as utility- and application-driven criteria are evaluated (such as dialog design to DIN EN ISO : Fitness for task at hand, self-description capability, controllability, compliance with expectations). Figure 10: Impressions from the system-ergonomic studies The test persons were selected from the user population that represented the target audience for subsequent use in order to ensure that the experience-based knowledge of the testers was the same as with the later users. During the usability tests, the test persons were required to use the prototypes to execute real-world work tasks in order to allow the use of observation and questionnaire methods (video recording, thinking aloud, structured interviews, etc.) to evaluate the systems. In early stages of the project, innovative integration concepts that have not yet been implemented were included in the tests, using methods like Wizard of Oz. Systematic evaluations are very difficult because a wide variety of fields of application were studied, so that the obtained results could not be compared or were only partially comparable. The following are statements from several studies: Efficiency increase in non-routine work, elimination of working steps for the retrieval of information, browser metaphor is understood, high acceptance of speed navigation, cabling on HMD problematic, AR Flat Panel would be an alternative, significant time advantage compared to existing troubleshooting and technical information system, no detectable difference in strain, etc. The pictures in Figure 10 reflect some impressions from ongoing evaluations of applicationoriented prototypes. 3.5 Cost-effectiveness and market In the empirical research that was jointly conducted with the users, the ergonomic researchers and the industry attempted to make statements about the cost-effectiveness of an AR system. This included the consideration of cost and value criteria. Cost-related criteria include: initial costs of hardware and software, training costs, cost of AR-compatible data preparation, recurring costs such as transmission costs, license fees, engineering costs, system maintenance etc. To determine the value of AR, the following was identified: higher machine availability, lower travel expenses, reduction of service times, new selling points, selling of highly qualified services, quality improvement of the provided information, synergies through data sharing for documentation and training, more efficient knowledge transfer through assembly reports, improvement of the quality of the performed work. This list illustrates the difficulty of obtaining quantifiable data of cost-effectiveness at this stage of the project. The

14 available material does not yet permit a statement of cost savings vs. the investment in an AR system. Forecasts for an AR-related market environment estimate a volume of US$ 13 2 with respect to AR equipment and of US$ 30 with respect to AR-based services by 2010 /3/. However, this must be seen under the supplementary conditions with respect to the availability of the following components /4/: Wearables with tracking components as of 2007 Displays wearable like glasses as of 2008 Continuous speech recognition as of 2009 Further segment-specific market research is conducted by the industry partners and is subject to strict confidentiality. The socio-economic impact of the AR technology on the society and the work process (individual, work process, qualification, dependency, globalization) were discussed with labor representatives in the second project phase. From the perspective of the labor representatives, it was unusual that a R&D project searches the contact in this early phase. An essential statement was to ensure that the involved staff is involved in the design of the new technology in an early stage through training and testing. This must be enforced upon the completion of the project. 4 Summary and Outlook In compliance with the project schedule, the first application-oriented prototypes have been created based on the AR basic system and have been evaluated in usability tests. The solutions that can already be demonstrated today testify to a good progress of the project in terms of the Base technologies such as Augmented Reality (marker-based tracking), interaction and AR-oriented delivery of information and workflow, but they also show the challenges for further development. According to approved scientists, ARVIKA is the only AR project worldwide that pursues the integral approach, in which a single project directly relates the research and development of AR with the provision of information and the new forms of human interaction with the environment. The architecture designed in ARVIKA is the basis for numerous prototypes in applications for the design, manufacture and service of products. This architecture was defined and the applications were implemented on the basis of comprehensive end user interviews. The ARVIKA project has made great progress, also for the international AR community. Through numerous registrations of inventions, research and industry partners protected the achieved advantage for the German industry. The tracking method on systems used in the industrial environment have demonstrated that they can be used for well-defined applications. These uses permit the instrumentation of the environment of the augmented object or the object itself. The selected architecture not only enables the retrieval of the desired information, but also adapts this information to the user s devices, such as AR-based Webpads or head mounted displays. The designed user interfaces integrate speech as an essential mode of interaction.

15 Nevertheless, the existing results leave the following issues open: New methods for object recognition or object localization must provide solutions that do not require an obvious instrumentation of the environment. The eye metaphor is and remains the challenge. The existing information about products such as cars and aircraft, machine, systems and equipment are not adapted to the use by AR, or only to a very limited degree. ARVIKA suggests initial solutions for the addressing of information in a manner that is suitable for end users. The creation of AR systems has not yet outgrown the stage of engineering-intensive lab solutions. Science and industry must enforce the design of AR-assisting authoring systems. Equipment manufacturers have not yet pursued a systematic approach to combine the eye as a reception sensor with an AR-compatible unit for the presentation of information. ARVIKA has demonstrated this to equipment manufacturers on the basis of practical application scenarios. Despite this additional requirement of application- and use-oriented research, ARVIKA has demonstrated that AR is not only feasible for industrial applications, but also offers significant advantages. Augmented Reality is no longer utopia, but ARVIKA has made AR a practical vision! Let s continue our work to make it happen!

16 The Consortium An interdisciplinary consortium that represents different segments of the industry ensured the feasibility of the ambitious goals both from a scientific perspective and with respect to the determination of requirements and testing in industrial applications. In the second half of the project, Zeiss was accepted into the project to cover the ergonomic requirements for ARcompatible HMDs, and BMW to provide additional industrial scenarios. *) Companies from the automotive and aerospace industry, such as Airbus Deutschland, EADS, DaimlerChrysler, VW, AUDI, Ford and BMW Mid-sized companies in the tool and machine tool segment, such as DS Technologie, Hüller-Hille, Gühring, Index, Ex- Cell-O and users from the power plant and process industry such as Framatome ANP Specialist providers of real-time tracking solutions, VR, user interfaces and glasses such as A.R.T., VRCom, UIDesign, and Zeiss For IT technologies: Fhg-IGD, ZGDV, and TUM, for applications: the Laboratory for Machine Tools and Production Engineering (WZL) and the Institute of Industrial Engineering and Ergonomics (IAW) of the RWTH Aachen, and Siemens as a system integrator and head of the consortium AACHEN This article documents a part of the results of the excellent cooperation of the ARVIKA consortium partners, which were committed to pursue the goal of Augmented Reality for industrial applications over a period of four years. The project experienced a very good support by the project sponsor and the BMBF. We want to thank everyone involved! 1 References: /1/ /2/ Presentations on the overall topic Tracking, Sensing and Gestures (Müller, FhG-IGD), HCI conference, Saarbrücken 9/2001 /3/ Source: IDC 1999, VDMA 1999, Siemens 2000 /4/ Source: Delphi Studie 10/2000 /5/ Study on Software Architectures for Augmented Reality Systems, Prof. Brügge et. al., TUM, 11/2002 /6/ Report on Comparison of international AR activities, Fraunhofer-IGD, 3/2002 /7/ Vision video Advanced e-production, Business and Service, Siemens A&D Nuremberg 2001 /8/ Ergonomics of Head Mounted Displays in Proceedings of the 6 th International Conference on Work With Display Units, 5/2002