Following Dirt Roads at Night-Time

Transcription

1 Following Dirt Roads at Night-Time Sensors and Features for Lane Recognition and Tracking Sebastian F. X. Bayerl Thorsten Luettel Hans-Joachim Wuensche Autonomous Systems Technology (TAS) Department of Aerospace Engineering University of the Bundeswehr Munich

2 Motivation Recognition of ego lane is prerequisite for many ADAS Camera based methods usually Most methods valid for well marked roads at night Color Color gradient Little work done for unmarked rural roads at night Color Color gradient PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 2

3 Hardware Stock color camera Color of surface Integration time: 30ms Integration time: 100ms PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 3

4 Hardware Stock color camera Color Night Vision (CNV) camera Color of Surface Integration time: 50ms PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 4

5 Hardware Stock color camera Color Night Vision (CNV) camera Near Infrared (NIR) camera Reflectivity Paved road Forest road PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 5

6 Hardware Stock color camera Color Night Vision (CNV) camera Near Infrared (NIR) camera Far Infrared (FIR) camera Temperature (12.5 C C) PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 6

7 Hardware Velodyne LiDAR 3D measurements NIR reflectivity PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 7

8 Fusion and Accumulation Fusion and accumulation into a Local Terrain Map Multiple layers Heights Slopes Obstacles NIR Reflectivity Color Temperature update step 1. Update robot position 2. Update Velodyne 3. Update camera layers PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 8

9 Features Color Features Gradient at lane boundary Saturation channel Ratio of green color channel Color with obstacles (red) Color gradient Green color ratio g / ( r+g+b ) Color saturation PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 9

10 Features Temperature Transitions at lane boundary Temperature back projection 15 C 17 C Surface temperature Temperature gradient Temperature back projection PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 10

11 Features 3D / LiDAR Obstacle probability Heights Slopes cross section of a hill cross section of a valley Obstacles (red) Heights PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 11

12 True positive rate Features - Evaluation positive and negative samples for each feature Different road scenes: paved, unpaved, dirt, forest, Different seasons: summer, winter, Different weather conditions: sun, rain, snow Different day times! Receiver Operating Characteristic (ROC) Color gradient Color gradient False positive rate PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 12

13 True positive rate True positive rate Features - Evaluation Color Color gradient Color green ratio False positive rate False positive rate Degradation of color from day to night! PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 13

14 Features - Evaluation Temperature E.g. temperature gradient Temperature more informative without illumination! PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 14

15 True positive rate Features - Evaluation Thermal limitations ROC of temperature back projection False positive rate road covered by leaves FIR Camera CNV Camera PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 15

16 Features - Evaluation 3D/LiDAR e.g. obstacle probability PDF ROC No dependency to illumination No stand alone feature PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 16

17 True positive rate Features - Evaluation Benefit of night sensors: Classifier with full feature capability (C A ) Classifier with reduced feature capability (C B ) False positive rate PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 17

18 Tracking Geometry Clothoid(s) for modelling road net w d Ψ x lane = d Ψ c0 c1 w T x cross = p x p y x brach1 x brach2 T x branch = Ψ c0 c1 w T PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 18

19 Tracking Particle Filter Correction step: Project particles into Local Terrain Map Calculate mean feature values F f for all particles (state vector x p ) Naive Bayes Classification result as particle weight w p F 1,, F n = f=1,,n p f (F f x p ) State Vector and Covariance from weighted mean Prediction: model road as static object moving with inverse robot motion PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 19

20 Movie PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 20

21 Thank you for your attention! Questions? PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 21

22 Motivation Sensors Hardware Fusion and Accumulation Features Road Features Evaluation Perception Particle Filter Limitations Movie PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 22

23 Hardware Robot: Mucar PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 23

24 Features Color Features Edges at lane boundary Saturation channel Ratio of green color channel Color with obstacles (red) Color edge intensity Green color ratio Color saturation PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 24

25 Limitation Limitations 1. Sun scene (14.5 C 17.0 C) 2. Rain scene (14.5 C 16.0 C) 3. Forest scene (14.5 C 17.0 C) 4. Forest scene (15.0 C 17.0 C) PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 25

26 Limitation Limitations 5. Foggy winter scene (10.0 C 10.5 C) 6. Snow scene (3.0 C 4.0 C) 7. Winter scene (8.0 C 10.0 C) PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 26

27 Tracking Thermal limitations ROC of thermal edge direction Set of features provides robustness: At least one significant feature necessary PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 27

28 Tracking Geometry Clothoid(s) for modelling road net w d Ψ x lane = d Ψ c0 c1 w T x cross = p x p y x brach1 x brach2 T x branch = Ψ c0 c1 w T Using rough information of road map to switch between road and crossroad Distance to crossroad Direction of outgoing branch PPNIV Sebastian Bayerl: Following Dirt Roads at Night Time 28