Field-of-View Enhancement for NADS Non- Standard Applications

Transcription

1 Field-of-View Enhancement for NADS Non- Standard Applications Yefei He, Chris Schwarz, Jeff Gordon, Shawn Allen, Tim Hanna* National Advanced Driving Simulator 2401 Oakdale Blvd Iowa City, IA *Olin College Olin Way Needham, MA Abstract The National Advanced Driving Simulator was designed as a very high fidelity device targeted for passenger cars and trucks. The use of the NADS to simulate a different kind of vehicle would be classified as a non-standard application, in light of its charter. Two recent examples of non-standard NADS applications are agricultural and construction vehicle simulation. In both of these cases, an extended vertical field-of-view (FOV) was required to achieve the desired level of realism. Fortunately, a solution was conceived that could address the need of both of these applications. Two additional projectors were mounted near the ceiling of the dome oriented downward. Two new screens were constructed and mounted to the floor of the dome in the locations where extra FOV was required. This paper describes the design and implementation of the FOV enhancement for the NADS. Constraints on space, combined with minimum required image size made the design challenging. Additionally, channel configuration for the additional display screens had to be computed for the image generator. Details on the constructions of low cost custom screens are given, as are the specifications of the selected projectors. The procedure for configuring the additional channels is described. Finally, the application of the FOV enhancement to agricultural and construction vehicle simulation is reported.

2 Introduction The National Advanced Driving Simulator (NADS) at the University of Iowa is a very high fidelity simulator that provides a convincing driving experience through its immersive virtual environment that combines audio, visual, motion, and haptic cues. All activities take place inside the 7.3-m (24-ft) dome where a full or partial cab of an actual vehicle is placed in the center of the floor, surrounded by floor-to-ceiling curved projection screens providing 360 horizontal field of view (FOV) and 40 vertical FOV, with a slight downward elevation of about -1.2 in the front. Such configuration is sufficient for reproducing the full view that a driver observes while sitting inside a regular passenger vehicle, due to the location of car windows and the inherent view obstruction caused by the front hood. In addition, during regular driving, the driver s eyes usually focus at some distances ahead, therefore a near-field view of the ground area immediately surrounding the vehicle is not critical. Many studies carried out at NADS use regular passenger vehicle cabs, where the existing display environment is sufficient. On the other hand, the NADS system is capable of simulating non-standard vehicles with cab layouts that are very different from passenger vehicles, and with driving tasks that are best simulated with a non-conventional field of view, as well. Two NADS applications in particular called for enhancement to the field of view: simulation of a Caterpillar 980G medium wheel loader, and simulation of a John Deere 7920 tractor. Both vehicles have cabs with front and side windows that extend to the cab floor to provide very good near-field front and side visibility necessary to operate the vehicles with the requisite precision for their routine tasks. For the wheel loader, due to the articulated steering, the front assembly including the bucket, lift arms and front wheels, can sweep an angle of about 40 degrees horizontally. The front assembly is not part of the cab, and the existing display environment cannot cover it at all, leaving a large gap between the view of the front displays and the cab, making it difficult for very common wheel loader tasks such as lane following, lane changing, obstacle avoidance, and truck loading. Similarly, the gap between the front projection screen and the tractor cab, although partially filled by the hood which is part of the cab, still leaves the front wheels and the ground underneath absent from the virtual environment, which are fully visible in an actual tractor and are used as a critical visual cue for the drivers, thus hampering tasks that require precise steering control. In order to overcome the limited vertical field of view of the existing NADS display environment and make it viable for non-traditional applications such as the wheel loader and the tractor, a plan for the enhancement of the near-field vertical field of view was developed by using additional projectors and ground-level projection screens. Emphasis was placed on minimizing costs while working within the NADS image generator capacity and channel configuration. Because cab changes are frequent and necessary, ease of installation, removal and flexible configuration were important considerations. The solution also needed to fulfill somewhat different enhanced FOV requirements of the

3 wheel loader and tractor cabs. The plan was implemented and put to use in several studies, and the solution proved to be effective. Design The NADS-1 dome visual subsystem consists of a computer image generator (IG) based on the Quantum3D Independence architecture. This architecture uses individual renderers for different visual channels. The existing 360 surround view is made up from eight channels, with images projected from eight ceiling-mounted Barco SIM-6 Ultra LCD projectors. However, the IG hardware has additional renderers as spares that are not for use within the NADS-1 dome. Therefore, it is feasible to utilize the spare renderers and configure them to render additional channels that enhance the field of view. The hardware required to complete the enhanced configuration are additional projectors and screens, and mounting devices inside the dome that can sustain motion stresses. Screen design Ideal layouts for field-of-view enhancement Based on the geometry of the dome, the dimensions of the cabs, and the area of visual coverage to be expanded that the applications demand, two ideal layouts for near-field ground view expansion were conceived for the wheel loader and the tractor respectively. The maximum number of additional channels was limited to four, due to the number of spare renderers available from the IG. When issues of cost, ease of installation and removal, and flexibility were taken into consideration, a two-projector solution was selected, and a screen aspect ratio of 4 3 was chosen over Moreover, in order to maximize the screen sizes, it was decided that the front vibration actuators of the wheel loader cab would be removed and the aluminum frame that secures the cab to the NADS- 1 dome floor would be modified. No such modifications were necessary for the tractor cab. Figure 1. John Deere 7920 tractor ideal FOV enhancement

4 Figure 2. Caterpillar 980G Medium Wheel Loader ideal FOV enhancement Figure 1 and Figure 2 show the top-down and side views of the ideal design for the fieldof-view enhancement for the tractor and wheel loader respectively. For the wheel loader, the additional channels, when combined with the existing channels, provide complete coverage of the front assembly when the articulation angle is small, and provide very good coverage even when the articulation angle reaches its limit of +/-40 degrees. For the tractor, the additional channels provide complete coverage of the front wheels regardless of their steering angles. The gap between the main front channels and the ground view channels shown in the top-down views is in fact invisible from the driver s eye point due to the elevation of the ground view screens, as illustrated by the side views. Actual sizing and shape Figures 1 and 2 represent the ideal layouts for the FOV enhancement, but in reality, some modifications to the shape and size of the screens were needed. To determine the size of the viewing area it was necessary to establish the distance from the center of the dome to the base of the wall and the height of the observer. The distance from the center of the dome to the base of the front wall was taken to be 115 inches. The height of the driver was found by measuring from the ground to the seat level and then adding the assumed height of the driver. For the purpose of the design the height to the seat of the tractor cab was 17 inches and 14 inches for the wheel loader cab; the height of the driver was assumed to be 54 inches for both cabs. Each cab would also be 11 inches off the ground. Once this was done a simple trigonometric relationship could be set up to determine the length of the screen. With the trig diagram in Figure 3, the distance from the center of the dome to where the observer s field of view met with the bottom front wall of the dome was calculated to be 96 inches. This measurement would become the radius of the arc sweep which formed the basic shape of the screens as can be seen in the right of Figure 3. To get the actual distance from the screen to the front of the dome, the distance from the center of the screen to the front end of the tractor cab was subtracted from the 96 inches. This yielded a screen length of 60 inches. The curved shape of the screens was intended to allow them to cover the full field of view from the front of the cab up to the dome s

5 existing field of view, while minimizing overlap. In addition, the screens were designed in three sections so that the center could be removed, allowing the screens to fit on either side of the tractor body. For the wheel loader cab, the combination of the three screens is intended to provide full coverage of the front field of view. This meant that a minimum number of pieces were needed in order to provide full coverage for the two different cab designs, reducing the number of elements to fabricate and the amount of effort required to switch cabs. Figure 3. Left, trig diagram used to calculate screen dimensions; right, diagram of screen Screen fabrication The fabrication design of the screens took into account both ease of construction and durability. The construction of the screens was a skeleton frame which would support a ¼ inch plywood facing covered with stretched muslin and coated with commercial screen paint. For durability, all three frames were made from welded steel angle iron. The steel reduces warping of the plywood frame thus distorting projected images. The plywood for the frame was cut to the shape of screen and attached to the frame by screwing the plywood to 1 3" boards through predrilled holes in the steel frame. Muslin canvas was then stretched over the frames and coated with a mixture of latex paint, water and wood glue to help bond it to the plywood. This treatment helped to provide a smooth, uniform surface upon which to apply the screen paint. When dried, the screen paint was painted on the screens. Projector selection The design of the screen required that the projectors needed to produce images with an aspect ratio of 4 3. Also, the cost of each projector was confined to about $2000. Other factors included ability for image morphing and color adjustments to match the main channels, weight, and sustainability to motion. At this price range, the candidates were presentation projectors and vertical keystone adjustment was the only image morphing available. Image size, resolution, and brightness also needed to be examined. Image size The image magnification m of a projection system is approximated by D m f

6 Where D is the throw distance, and f is the lens focal length ([4]). Therefore, the diagonal image size S is K S = mk D f Where K is the diagonal size of the display element of the projector. Due to constraints from the dome geometry, the maximum throw distance from the mounted projector to the screen was only around 100 inches. In order to achieve the designed image size of 74" 56", or a diagonal of 93", the value of K/f needed to be sufficiently large. Among the candidates, only the Casio XJ-360 DLP projector, with a 0.7" display element and a zoom lens of focal length from 20.0 to 40.0mm, met the requirement. Image resolution Each of the eight main channels of the NADS has a field of view of including 3 on both sides for edge blending. The front channels have a pixel resolution of , while the side channels have a resolution of , and the rear even less ([1]). In terms of pixels per arc minute, the resolutions are and , respectively. The wheel loader set up had an image size of 74" 56" per channel. Using the position information from Figure 3, the FOV was calculated as approximately The resolution of the Casio projector s display element is , making it pixels per arc minute. For the tractor set up, the image size was 60" 45" with an FOV of 38 30, resulting in a resolution of Thus, the ground view channels did not have as good resolution as the front channels, but were comparable to the side and rear. Image brightness The luminance or brightness B of a projected image can be calculated by F B = g S Where F is the light output of the projector, S is the area of the projected image, and g is the gain of the screen ([2], [4]). The Barco projectors have an on-axis light output of 3000 lumens (lm), and the image size is 12.5ft 8.6ft, resulting in a brightness of (27.9g) footlamberts (fl). For the Casio projector, the on-axis light output is 2200 lumens with the zoom ratio set at 1, when the relative aperture N 0 is 2. In the worst case scenario when the zoom ratio is 2 to achieve maximum image size, the relative aperture N 1 is 2.8, and the light output becomes 2 2 N 0 2 F1 = F0 = lm ([4]) N The ground view image size for the wheel loader setup was 6.17ft 4.67ft, so the image brightness would be no worse than (38.9g) fl. The gain of the main channel screen is fairly similar to that of the ground view screens, which is in the range of 1.8 on axis. Therefore, the brightness of the ground view images would be more than sufficient. Visual channel configuration Mantis, the rendering software that the Quantum3D IG uses, comes with a visual channel configuration utility tool ([3]). To set up a channel, the size, position and orientation of the display need to be specified. The distance for the main channel displays

7 has been previously set to 0.5 database units (dbu), corresponding to the 12-ft radius of the NADS dome. The size and position values of the ground view channel displays would then also be converted to dbu s. In the tractor cab setup, the two projectors were rotated +90 and -90, respectively. This had to be reflected in the roll values of the displays. Figure 4 shows the Mantis GUI for channel configuration and the layout of all displays, including the 8 main displays, the ground view displays, and a bird s eye view screen. Figure 4. Ground view channel configuration in the Mantis software. Left, MWL, right, tractor. Installation Screen installation Before the installation of the screens in the dome the frame corners were reinforced and legs were added to elevate the screens. The reinforcement of the corners was done by gluing wood braces into each corner. The legs were short lengths of aluminum angle with a small aluminum angle foot for attachment to the dome floor. The legs were mounted to the screens with bolts that pass through the wood and steel frame. The small aluminum angle foot on each leg was bolted directly to the dome floor. The dome floor was drilled and tapped for the mounting of these feet. Only the side screens had legs added. The center screen was supported by its direct attachment to the side screens by bolts that passed through the wood and steel frame. Projector installation The differing fields of view between the wheel loader and tractor cabs necessitated two different projector mounting techniques. The wheel loader cab used the center screen and required an image at the front of the cab that was wider than it was deep. The tractor cab did not use the center screen and required an image on both sides of the hood that was deeper than it was wide. For the wheel loader, the two projectors were mounted in the same plane, pointing straight down with no rotation about its vertical axis. They were installed in the dome by the use of a ball and socket mounting system sold under the trade name RAM Mount. This mounting system was supplemented by metal strapping to

8 further secure the projectors. This assembly was then attached to the existing projector framing. For the tractor, the two projectors were mounted in two parallel planes, pointing straight down and rotated 90 about its vertical axis.they were installed in the dome by mounting them to two aluminum plates, which were then attached to the existing projector framing. Figure 5 shows the two different set ups, and a mounted projector in the MWL set up with the RAM Mount and metal strapping visible. Figure 5. Ground view set up with MWL (left, screens, center, a projector); right, with the tractor. Image adjustments After the ground view screens and the projectors were installed, the two new visual channels were configured using the Quantum3D Mantis software. Due to the small throw distance, the zoom ratio of the projectors was set to 2. The image size was measured at 63" 47" for the tractor set up, slightly exceeding the designed size, but only 66.5" 48.5" for the MWL, smaller than designed. There were also some keystone effects because the projectors were not pointed exactly perpendicularly towards the screens. The vertical keystone adjustment function on the projectors could only partially overcome the effects. The actual size and location of the images were measured, compensated for the keystone effects, and then used for the configuration. Several iterations of further adjustments were made after visual inspection from the designed driver s eye point, which turned out to be 7 inches behind the center of dome for the MWL. Default color balance of the Casio projectors was quite different from the Barco ones. Some adjustments were made but they still did not match perfectly. Image brightness was also reduced. Results As mentioned, the installation and final ground view images somewhat deviated from the design. However, drivers feedbacks from the studies that utilized the FOV enhancement were positive. Imperfections in areas such as parallax error, color and brightness matching were not commented upon by the drivers. Drivers were able to perform tasks that rely on near-field views in the real world without complaints. Prior to the FOV enhancement, drivers in similar studies had complained about a lack of continuation of visual flow from the front view to the cab. Figure 6 shows clearly how the continuous visual flow was preserved by the added ground view channels. A point of concern had been how rigid the projector and screen mounts would be, and how they would stand up to the rigorous motion that the NADS-1 dome is subject to, in particular during the wheel loader tasks. Results showed the motion seemingly had no ill effect on the projectors, and there was no visible vibration on the ground view images

9 during simulation. However, gradual drifting took place, and the visual channel configuration had to be tweaked between simulator runs to compensate for it. Also, each time the ground view projectors and screens were reinstalled after a cab change, the size and location of the images changed slightly, requiring adjustments in channel layout. Figure 6. Comparison of forward views from inside the actual cabs vs. the enhanced simulator cabs. Top row, Caterpillar 980G MWL, actual vs. simulated; bottom row, John Deere 7920 tractor. Due to differences in drivers heights and seat position settings, variations in the actual eye point location took place. Drivers also moved their heads when driving, further changing the eye point. The channel configuration was done to a set eye point location, which would cause parallax errors when the actual eye point moved away. However the drivers evidently did not pay attention to this issue as no one complained about it. Conclusions The field-of-view enhancement increased the range of applications for the NADS simulator, making it suitable for tasks requiring near-field ground views. The drivers feedbacks in related studies confirmed the effectiveness of the enhancement. There were issues when the design was implemented, however they did not diminish the benefits. In the future, the sturdiness of the mounting can be improved, and a systematic way of compensating for eye point variations is desirable. Future designs of driving simulators should take into consideration whether to include near-field views from the start. References [1] Allen, S. (2006). NADS Visual Subsystem. Technical Report. Iowa City, IA. [2] Calvert, J. B. (2003, August 14). Illumination. Retrieved March 10, 2006, from Dr. James B. Calvert's web site: [3] Quantum3D, Inc. Mantis Users Guide for Version 2.0. San Jose, CA.

10 [4] Ray, S. F. (1988). Applied Photographic Optics: Imaging Systems for Photography, Film and Video. London & Boston: Focal Press.