Test-bed for Unified Perception & Decision Architecture

Transcription

1 Test-bed for Unified Perception & Decision Architecture Luca Bombini, Stefano Cattani, Pietro Cerri, Rean Isabella Fedriga, Mirko Felisa, and Pier Paolo Porta Abstract This paper presents the test-bed that will be developed for a Unified Perception & Decision Architecture (UPDA). Due to the increasing demand of ADAS systems to be mounted on cars, it is more and more important to develop a unified architecture that can communicate and share information between these systems. This is the aim of an ERC-founded project and to develop and test such architecture a car has been set up with many different sensors. 1 Introduction VisLab is undertaking highly innovative research within its ERC-founded European project, whose topic is the development of an open standard for the perception and decision subsystems of intelligent vehicles. Currently, many commercial vehicles include sophisticated control devices like ABS, ESP, and others. These control equipments have been independently developed by car manufacturers and suppliers. Generally, they also act independently, and are singularly tuned. Nevertheless, new methods to improve overall performance are currently under development, exploiting communication and cooperation of these devices: the recently introduced Unified Chassis Control (UCC) is an example. The deployment of the UCC in the mass market, requires to adapt and rethink all control subsystems to provide communication, data fusion, and an overall tuning: namely to integrate all of them together. From the car manufacturers and suppliers point of view, the introduction of the UCC requires the redesign of each single block (ABS, ESP,...) meaning an additional financial effort, besides the obvious delay in reaching the market. Had a complete UCC architecture been defined well in advance with respect to the development of each single block, its implementation would have been straightforward, less costly, and would have reached the market earlier. Fig. 1. Test-bed vehicle for the UPDA architecture Perception and decision modules are in an earlier development stage than control ones: the advanced driver assistance systems that are currently available on the market are only basic ones (Lane Departure Warning, Blind Spot Monitoring, Automatic Cruise Control, Collision Mitigation), independently developed by different car manufacturers and suppliers. The state of the art of advanced driver assistance systems, in fact, has not yet defined a complete architecture that would allow to fuse together all these basic blocks and benefit from their integration. The availability of such architecture would allow to define a standard module interface so that the following research efforts could be more focused in providing modular systems, already engineered to fit into this architecture. In order to develop these concepts, VisLab is setting up a vehicle prototype (three more are expected for 2009) integrating various sensing technologies. A new architecture (UPDA, Unified Perception & Decision Architecture)

2 is meant to take as input all the data coming from the different perception modules, to fuse them, and to build a more sophisticated world model to be used in the following decision level. In doing this, a standard interface will be defined for each perception module, allowing different providers to integrate their own perception system into this standard architecture, thus boosting competition. The prototype vehicle will be used for testing both UPDA and ADAS applications. The car integrates 10 cameras, 4 laserscanners of 2 different brands, a laser, a GPS unit connected to an Inertial Measurement Unit, 4 PCs; the whole car has been set up for drive by wire, and can be controlled via CAN messages. In the following, a detailed description of the overall system aims is listed: Crossings assistance [ref A]: lateral perception is important when facing an intersection, and as junctions may have very different layouts, the lateral system must be able to perceive traffic coming from different angular directions. During a traffic merging manoeuvre, the vehicle is allowed to pull into traffic only when oncoming traffic leaves a gap of a sufficient time interval (usually at least 10 seconds). Hence the vehicle, regardless of the junction layout, needs to perceive the oncoming traffic as well as estimate cars speed from long distance. Overtaking assistance [ref A]: when driving on a road with multiple lanes, the lane change manoeuvre is a common task in order to overtake slower traffic. Two rear cameras installed acquire images of the road behind and on the vehicle side. The cameras are installed so that they can frame the area close to the vehicle and extend their vision over the horizon. This system can overcome some LIDAR limitations, like its inability to provide useful results when moving along dusty roads, where the clouds produced by the vehicle itself negatively affect the laser beams. This is a problem mainly of the rear LIDAR since dust clouds are produced by the vehicle in motion; sometimes it affects also the front ones especially during sudden stops when dust clouds overtake the car. Despite this problem, LIDAR data have been used to refine the detected obstacles position measurement, thus reducing the errors introduced by vehicle pitching, since distance estimation is performed by a monocular system. Obstacle & Pedestrian detection [ref A]: the sensing of obstacles up to a distance of 50m is of great importance when driving in a urban environment since other vehicles traveling in either direction or fixed and moving obstacles may be on the vehicle's trajectory. Stereoscopic systems offer great advantages since they allow to reconstruct the 3D environment and therefore help to detect anything above the road surface. Lane & Stop Line detection, Lane keeping [ref A]: navigation in a urban environment has to be very precise. The vehicle must be able to detect obstacles and lane markings with high confidence on the detection probability and position accuracy; moreover the system must detect stop lines, to allow a precise positioning of the vehicle at intersections, and lane markings in sharp curves. The perception system must therefore cover a distance range useful for detections at driving at medium-to-high speeds. The vehicle includes two stereo systems (one on the front and one at the back) which provide precise sensing in its close proximity. Parking assistance [ref C e D]: the system aims at supporting the driver during parking manoeuvres. Road Signs detection [ref B]: the vehicle is able to detect and classify a large variety of road signs. This functionality may warn drivers and supplies additional environmental information to other on-board systems such as ACC. Stop-and-Go & ACC [ref B]: this functionality is to be used for speed adjustment in respect of the preceding vehicle. Stop-and-Go is for queue-driving, when the vehicle speed is lower than 15km/h in a urban-like environment, whereas ACC operates at higher speeds, like on highways. Once engaged, gas and brake control are handled by the system until some driver s action on the pedals disables the automatic speed control of the car. 2 Hardware Architecture The vehicles is equipped with a variety of devices, including sensors for world perception, navigation and control, systems, localizations devices, computers, displays, batteries, wiring. A lot of effort was spent to have a high level

3 of integration: from the outside the devices are barely visible, while in the inside all necessary controls and displays are integrated into the dash board, headrest and armrest. Figure 2 shows the overall equipment schema. The acronyms used are the following: Cameras: DLR and DLL = DragonFly2 Lateral Right and Left FRR and FRL = FireFly2 Rear Right and Left DWR and DWL = DragonFly2 Wide (baseline) Right and Left DGNR and DGNL = DragonFly2 Grey (b/w) Narrow (baseline) Right and Left DBR and DBL = DragonFly2 Back Right and Left Laser and Laserscanners: HFR and HFL = Hella Front Right and Left HIDIS = Hella Idis IL = IBEO Lux HB = Hella Back Other Devices: 2 Point Grey DragonFly2 [7] colour cameras with 1024x768 resolution and 2.5mm optics are placed near the end of the hood, above the tires at both ends of the bumper. Computers: 2 Point Grey DragonFly2 [7] colour cameras with 1024x768 resolution and 2.5mm optics are placed near the end of the hood, above the tires at both ends of the bumper. Console and Displays: 2 Point Grey DragonFly2 [7] colour cameras with 1024x768 resolution and 2.5mm optics are placed near the end of the hood, above the tires at both ends of the bumper. Fig. 2. Complete scheme of vehicle s hardware components

4 2.1 Perception and Localization Devices The UDPA architecture implemented on this vehicle receives, as input, information coming form laserscanners, cameras, GPS antenna, inertial navigation system and vehicle state. All these information are gathered to create, and continuously update, a perception map of the environment where the car is called to move, on the basis of which the implemented driver assistance actions are evaluated and taken. In this paragraph we will go through details about the perception and localization devices Cameras The equipment described in this section is addressed to supply the vehicle with an overall vision of the road status around it in terms of pedestrians, generic obstacles, overtaking cars in order to safeguard its safety and that of the other road users. Fig. 3. Cameras set-up The camera devices are the following: 2 Point Grey DragonFly2 [7] colour cameras with 1024x768 resolution and 2.5mm optics are placed near the end of the hood, above the tires at both ends of the bumper. 2 Point Grey FireFly2 [7] colour cameras with 1024x768 resolution and 6mm optics are installed in both wing mirrors lodging, besides the respective mirrors. 2 Point Grey DragonFly2 colour cameras and 2 Point Grey DragonFly2 b/w cameras, all with 1024x768 resolution and 6.4mm optics, are installed in the main frontal area, behind the windscreen. 2 Point Grey DragonFly2 colour cameras with 1024x768 resolution and mm optics are placed at the back of the vehicle, between the rear bumper and the number plate. The front lateral cameras (Point Grey DragonFly2) are employed for the following aims: Crossing assistance: the very wide field of view makes these laserscanner particularly useful in this task, allowing a variety of intersection patterns faceable (e.g. private to public road exits); The front cameras (Point Grey DragonFly2) are employed for the following aims:

5 Obstacle & Pedestrian detection: here is especially enforced the IBEO Lux s long detection range (200m), to compensate the (relatively) short range of cameras and beam scanner, that, on the other hands, provides 3D information of the detected items; Lane & Stop Line detection, Lane keeping [ref A]: navigation in a urban environment has to be The wing mirror cameras (Point Grey FireFly2) are employed for the following aims: Overtaking assistance [ref A]: when driving on a road with multiple lanes, the lane change The rear cameras (Point Grey DragonFly2) are employed for the following aims: Parking assistance [ref A]: when driving on a road with multiple lanes, the lane change Laser and Laserscanners Lasers and scanning lasers (laserscanners) are typically very precise in measuring distances, and very robust with respect of light condition. At the same time they are very dependant on the vehicle attitude (roll, pitch) and provide very few information useful to classify the object causing a certain reflection at a certain time. Hence, they are the ideal complementary for vision based sensors (i.e. cameras), that are normally less precise in distance estimation but very powerful in providing colour/texture information. For an in-depth examination of laser-camera fusion, see [1], [2]. Fig. 4. Laser and laserscanners set-up The vehicle is equipped with a total of 5 laser based devices (see schema Fig. 4). The higher speeds are normally reached when driving forward, hence a larger number of sensors has been planned on the front: 4 are on the front and 1 is on the back. Let discuss them separately. The laser devices are the following: 3 Hokuyo UTM-30LX [3] single layer laserscanners, mounted on the left and right corner. They have a 270 degrees and 30 meters scanning range. The angular resolution is 0.25 degrees and the scanning frequency is 25Hz;

6 1 IBEO Lux [4], a 4 layer laserscanner with 100 degrees and 200 meters range, mounted on the centre front. The angular resolution is 4cm, the scanning frequency is 12.5Hz and the vertical field of view is 3.4 degrees; 1 Hella MultiBeam [5] infrared laser. This is not a scanning device, but a multi beam laser with 16 beams, each one of 1 degree horizontal and 3 degree vertical field of view. It cat be seen a very low resolution range camera [6]. The front side laser/laserscanners (IBEO and Hella) are employed for the following aims: Obstacle & Pedestrian detection: here is especially enforced the IBEO Lux s long detection range (200m), to compensate the (relatively) short range of cameras and beam scanner, that, on the other hands, provides 3D information of the detected items; Stop-and-Go & ACC: while IBEO sensor is a really wide porpoise device, the Hella range sensor is specifically though (and limited) for this scope, due to the ability of providing 3D information even at very short distances (out of the stereo field of view); The corner side laserscanners (Hokuyo) are employed for the following aims: Crossing assistance: the very wide field of view makes these laserscanner particularly useful in this task, allowing a variety of intersection patterns faceable (e.g. private to public road exits); Overtaking assistance: again, the wide field of view if enforced to detect overtaking vehicles in the blind spot; Parking assistance: free space detection when moving inside parking lots, looking for an available and suitable parking spot, can be made by motion stereo with the monocular cameras available on the vehicles sides, however the reinforcement in distances estimation given by the laserscanner is very valuable; The rear side laserscanner (Hokuyo) is employed for the following aim: Parking assistance: the backup manoeuvre normally occurs at reduced speed, typically during parking, hence the main support asked by a driver is obstacle presence warning, like pedestrians, other vehicles, poles, etc. For this reason on the back side only one Hokuyo UTM-30LX laserscanner has been mounted, placed on the centre Other Devices Besides the previous perception devices, in order to understand the vehicle s position in world coordinates and its basic motion information (in terms of type, rate and direction), two other devices are installed: a Racelogic VBOX II SX GPS Global Positioning System and a Three Axis Racelogic VBOX II IMU Inertial Measurement Unit both with 20Hz update rate (see Fig. 2). 2.2 Processing Devices The processing of all data acquired by the previous perception equipment is exploited by four computers, all placed in the vehicle s boot. Three of such computers were especially assembled and employ two AmericanInc A and a Voom2 motherboards whereas the fourth one is a dspace Micro AutoBox (see Fig. 5). The dspace computer is dedicated to manage the low-level devices like ESP, EPS and ACC systems, interfacing itself with the car s switchboard and making it work.

7 Fig. 5. Computers set-up 2.3 Console and Displays In order to get a visual feedback of the overall system behaviour and let the driver interact with it, five displays are installed in the vehicle s passenger compartment. One touchscreen monitor 15 EASYPANEL EP150AD33-15W- U-DM, used for debug purposes, is installed on the dashboard in front of passenger s seat. A touchscreen monitor 7 EASYPANEL EP150AD33-15W-U-DM is placed on the command panel above radio car s lodging and two others EASYPANEL EP070AV28EP DM in passengers headrests. Last monitor 7 Hardstone MM070 is installed in the rear-view mirror (see Fig. 6). Fig. 6. Interface set-up 3 Project s objectives The project is aimed at the development of autonomous vehicles and supervised driving systems with the ultimate goal of defining a common open architecture which will be proposed as a standard to the automotive sector. Besides providing clear advantages on safety for road users, the availability of an open architecture will encourage and make possible the sharing of knowledge between public and private research communities (academic and automotive industry) and thus speed up the design of a standard platform for future vehicles. Further research steps will be eased -and therefore made more effective- thanks to the common and open architectural layer proposed. The project is divided into the following two main milestones: 1. the development of fully autonomous vehicles and,

8 2. extension towards driving assistance systems, namely systems able to supervise a driver and to intervene when necessary. Figure??? shows the evolution of driving assistance systems leading to autonomous driving. The first 4 steps have a human being as vehicle main leader: starting from a set of independent warning systems (step 2) like lane departure warning, to independent active systems (step 3) like adaptive cruise control, collision avoidance, and finally a unified perception and decision architecture (step 4) devoted to perform an active cooperative driving. From step 5 onward the vehicle leader changes from the human being to the electronic pilot increasing the sensing capabilities. First with autonomous driving and then (step 6) with supervised driving, in which the human instructs and directs the manoeuvre while the main control is owned by the electronic pilot. Human contribution, in this case, is treated like one of the other sensing devices, and thus overridable. This project will reach step 3 and step 4, using the new concept architecture to make all the systems cooperate together. The first milestone will be a demonstration of fully autonomous vehicles able to cope with real scenarios and not only with controlled environments. This requires to develop both an extended perception system able to build an accurate world model and a sophisticated decision system. This part will be based on the work already developed by VisLab for other projects (like ARGO [1p] and the DARPA Challenges [2p]) and its experience as a primary player in this field. In the second stage, leading to the second milestone, a perception module able to analyze the driver s intentions as well as a Human Machine Interface will be added in order to enable driving assistance features. As mentioned, this requires to extend both the perception and decision functionalities in order to integrate new inputs. During both stages, the logical architecture of the vehicle (i) autonomous system and (ii) supervisory system will be designed, tested, and validated thanks to intermediate tests on its completeness, feasibility, and scalability by means of the test bed described. References [AMAA Reference Heading] For the references please use [ ] to number your references serially, then press tab and continue typing the refernces.] [1] Details at [7] Specification and details at [a] A. Broggi, A. Cappalunga, S. Cattani, and P. Zani, ``Lateral Vehicles Detection Using Monocular High Resolution Cameras on TerraMax'' in Procs. IEEE Intelligent Vehicles Symposium 2008, Eindhoven, Netherlands, Jun [b] ref B: Real time road signs recognition (Marelli, IV07) ref C: 3D parking assistant system ref D: Evaluation of a Vision-Based Parking Assistance System

9 [1] Alberto Broggi, Stefano Cattani, Pier Paolo Porta, and Paolo Zani, A Laserscanner- Vision Fusion System Implemented on the TerraMax Autonomous Vehicle, In International Conference on Intelligent Robots and Systems (IROS06), Beijing, China, October [2] Massimo Bertozzi, Luca Bombini, Pietro Cerri, Paolo Medici, Pier Claudio Antonello, and Maurizio Miglietta, Obstacle Detection and Classification fusing Radar and Vision, In Procs. IEEE Intelligent Vehicles Symposium 2008, pages , Eindhoven, Netherlands, June [3] Specification and details at [4] Specification and details at [5] Specification and details at [6] S.B. Gokturk, H. Yalcin, C. Bamji, A Time-of-Flight Depth Sensor System Description, Issues, and Solutions, In Proc. IEEE Workshop Real-Time 3D Sensors, 2004 [1p] [2p] Yi-Liang Chen, Venkataraman Sundareswaran, Craig Anderson, Alberto Broggi, Paolo Grisleri, Pier Paolo Porta, Paolo Zani, and John Beck, TerraMax: Team Oshkosh Urban Robot, Journal of Field Robotics, 25(10): , October Alberto Broggi, Massimo Bertozzi, Alessandra Fascioli, and Gianni Conte, Automatic Vehicle Guidance: the Experience of the ARGO Vehicle. World Scientific, Singapore, April Luca Bombini, Stefano Cattani, Pietro Cerri, Rean Isabella Fedriga, Mirko Felisa, and Pier Paolo Porta VisLab Dipartimento di Ingegneria dell Informazione Università degli Studi di Parma Italy {bombini,cattani,cerri,fedriga,felisa,porta}@vislab.it Keywords: ADAS, Autonomous vehicle, Unified Perception and Decision Architecture. [AMAA Keywords]