One Display for a Cockpit Interactive Solution: The Technology Challenges A. Xalas, N. Sgouros, P. Kouros, J. Ellinas Department of Electronic Computer Systems, Technological Educational Institute of Piraeus, 250 P. Ralli & Thivon, 12244 Egaleo, Greece One Display for a Cockpit Interactive Solution (ODICIS) is an FP7 European project for Aeronautics that aspires in developing a single display cockpit associated with adequate means of interaction and Human System Interface (HSI). The project will improve the operational safety and efficiency and reduce the aircraft development cost as addresses three current major aeronautics needs: the system architecture flexibility, the useful surface optimisation and the information continuity. Such a large seamless display surface may be created by multiple projectors that project tiled images. This involves establishing advanced architectures with graphics generators able to process any image format correcting display curvature. The input device that will be used to control the cockpit may be a multi-touch screen which will provide more sophisticated operations like multi-point-and-click, drag-anddrop, magnify-shrink of displayed objects or complicated application specific gestures, that they can be used to manipulate virtual digital 2D and 2.5D terrain maps, virtual pointing devices like virtual keyboards or virtual FCU. Additionally, multi-touch is compatible with collaborative works: the captain and the first officer can manipulate simultaneously the same surface. HSI becomes more reliable and effective if it is combined with haptics technologies that allow getting tactile feedback. Haptics can be applied to reproduce buttons and switches with texture and tactile sense. Different technologies exist, from electromagnetic ones to piezoelectric ones. The realization of this feature greatly increases the perception of the pilot for most of the procedures where tactile sensation is mandatory. Keywords: ODICIS, Single-Touch Screen, Multi-Touch Screen, Haptics, Tactile feedback. 1. Introduction After World War II the cockpit layouts of civil air transports became more advanced and complicated following the technology advances and the emerge of more modern airplanes. Figures 1 and 2 show the advances of cockpit designs during the last four decades. Concorde disposed a too abundant information display, A380 integrated more functional characteristics to a unified area while A350 and B787, which are currently developed by Thales and Rockwell-Collins, will probably involve the current trends as: Increasing the size of displays while reducing their number. Tendency to paperless cockpits. The Electronic Flight Bag (EFB) is one step towards that facility. It is a general purpose computing platform
intended to reduce, or replace, paper-based reference material including the Aircraft Operating Manual, Aircrew Operating Manual, and Navigational Charts. Improving system availability and fault tolerance. Reducing the number of dedicated input media/output devices and processing platforms. Optimizing the usage of the available input media/output devices. Figure 1: View of Concorde cockpit (1969) and A380 cockpit (2004). The ODICIS consortium (Thales, Diehl Aerospace, Alenia, Alitalia, Optinvent, IMEC, Denmark Technical University, University of Malta and TEI of Piraeus) will meet these major challenges through the development of a single large display cockpit [1], [2]. The main targeted requirements for such a display are the following: Need of a large surface, non-standard form factor display with high information content density and superior image quality compared to today s display cockpit. Need of a display that meets Avionics specifications in term of safety, ergonomics and cockpit environment. Need of a display that offers better features in terms of flexibility for showing user-defined cockpit configurations and user interactivity. Figure 2: View of Boeing B787 cockpit (2007). Need of a less expensive maintenance: quick and possible for each line replaceable unit, whilst maintaining the dispatch rate. 2
Fig. 3 provides an artistic view of a single display cockpit, which will be characterized by system architecture flexibility, useful surface optimization and information continuity. Figure 3: Artistic view of a single display cockpit. 2. Cockpit display system Today s cockpit display systems can either provide a large aggregate resolution through multiple individual screens or a significant but limited resolution in a single screen. ODICIS consortium proposes to design a single avionics high resolution display for regional or business jets. Such a display should provide performances similar to existing cockpit displays in terms of optical transmission, viewing angles for instance, but it should also appear as a really seamless display without any borders and the same contrast and luminance everywhere. It should also be in adequacy with bigger markets: aerospace is now relying on technologies developed there and compliant with performances and obsolescence issues. Some today s applications require large displays, like tiled displays for gaming or screen walls for show rooms and command centre and use projection technology with multiple projector units. These are usually rearprojection systems based on DMD (Digital Micro-mirror Device) projection technology and consist of LED-based projectors tiled to provide large diagonal display (42 for NEC system). The current projection technologies are DMD and LCoS (Liquid Crystal on Silicon). Digital Light Processing (DLP) systems use either a DMD or LCoS technology to create a picture [3], [4]. Basically, DMD contains millions of microscopic mirrors that reflect light from a lamp. Each mirror creates one pixel of the final projected image. The mirrors flip back and forth between their "on" and "off" positions very rapidly. When mirrors are on, they point toward a projection lens and create pixels which become brighter as the mirror stays longer in the on position. When mirrors are off, black pixels are created. A colour wheel spins between the lamp and the DMD adding red, green and blue light to the picture. The viewer's eyes combine these colours to create the finished image, as Fig. 4 illustrates. LCoS uses a very similar idea, but instead of tiny mirrors that turn on and off, it uses liquid crystals to control the amount of reflected light. A liquid crystal is a substance that is in mesomorphic state, that is, it is not exactly a liquid or a solid. Its molecules usually hold their shape, like a solid, but they 3
can also move around, like a liquid. Most LCDs use twisted nematic (TN) crystals which straighten out with the application of an electrical charge. When placed between two polarized panels, the twisted crystals guide the path of light. By changing the direction of the light, the crystals allow or prevent its passage through the second panel, as Fig. 5 shows. Figure 4: DLP rear projection system using DMD technology. Figure 5: Transmission of light in an LCD display system. Fig. 6 illustrates the operation of the LCoS projection technology. The process includes a high-intensity lamp, a series of mirrors and microdevices arranged into a cube, a prism and a projection lens. Figure 6: LCoS projection system operation and LCoS chips. 4
3. Interaction with the cockpit display system In a single display cockpit, touch-screens may be the type of input devices to control the cockpit with an efficient and robust way. The cockpits of the French fighter Rafale, or the USA fighter JSF, are equipped with touchscreens. A touch-screen solution may lead to a full renovation of the controls, which can be embedded on it, in order to meet functionality and usability requirements. Multi-touch screens may offer a number of user defined gestures, like drag and drop, objects transposition, rotation, magnification or shrink. They can also be used to manipulate virtual digital 2D and 2.5D terrain maps, virtual pointing devices like virtual keyboards or virtual FCU. Additionally, multi-touch is compatible with collaborative works: the captain and the first officer can manipulate simultaneously the same surface. Resistive, capacitive, infrared, frustrated total internal reflection (FTIR), sound acoustic wave technologies may be employed, as they are compatible with any projection system display. In conjunction with a multi-touch screen, haptics is the technology that may be used to enhance the performance of the display providing a tactile feedback sense. It can be applied to reproduce buttons and switches with texture and tactile sense. The most common form of feedback is the vibrotactile, where a number of actuators can simulate vibrations to a specific position of the screen. Thus, a number of different modalities, like multifunction knobs (or rotators), could be used to group functions together. The differentiation between the functions is made thanks to different tactile feedback sensations. Fig. 7 illustrates the principle of operation of touchscreen with haptics, whereas Fig. 8 shows the hardware behind a haptics application [5]. A haptics implementation incorporates the control board, actuators, and a touch-screen carrier. The actuators are optimized for generating high forces with small displacements. The number and size of the actuators depend on the touch-screen s size, weight, and implementation. Figure 7: Haptics operation on a multi-touch-screen. 5
Figure 8: Hardware of a haptics application. 4. Conclusions ODICIS is an FP7 European Research project that is challenged to develop a single display cockpit employing all the recent advances of technologies associated with display systems and interactivity, while meeting operational efficiency and safety. The development of such a display presupposes an extensive research on projection technologies, the operation of the optical engine, the adoption of the proper touch technology and the suitable tactile feedback system. All the technologies that will be employed must conform to the avionics standards as far as operation, functionality and safety is concerned. Acknowledgements This paper was realised within the European Research Program ODICIS funded under the Seventh Framework Program Theme Transports (including aeronautics) (ACP8-GA-2009-233605). The authors would like to acknowledge all the partners of this project for their contribution. References [1] Thales Avionics, ODICIS: Description of Work, March 2009. [2] http://www.odicis.org, ODICIS project overview presentation. [3] T. V. Wilson, Ryan Johnson, How DLP Sets Work, HowStuffWorks.com, 9 August 2005. [4] T. V. Wilson, "How LCoS Works", HowStuffWorks.com, 12 January 2006. [5] Immersion, HAPTICS- Improving the Mobile User Experience through touch, White paper, 2007. 6