SAFE CYCLING FOR ELDERLY: "SEE" BACHELOR GRADUATION PROJECT

Transcription

1 07/07/2017 SAFE CYCLING FOR ELDERLY: "SEE" BACHELOR GRADUATION PROJECT Bachelor thesis by Lukas Bos Supervisor from the University of Twente: dr. ir. Oresti Banos Supervisor from Indes B.V.: ir. Maurice Tak Critical observer: dr. ir. Bert-Jan van Beijnum

2

3 CONTENTS 1 Introduction Background Motivation Problem statement Research questions State of the Art Research on accidents among elderly cyclists Applications for bicycles Principles Sensors LiDAR Radar Sonar Camera Time-Of-Flight-Camera Roadmaps Smartphones Discussion Design Requirements Overview of the concept Software Development Selection of sensors Garmin LiDAR Lite V Innosent IPS-265 Radar HB-100 Radar Handling the output of the radars Selection of platform Electronics LiDAR circuit Radar circuit 1: HB Radar circuit 2: IPS Software Communication Visualization Smartphone Results Testing the sensors Static sensor testing Dynamic sensor testing Testing a smartphone Sensor accuracy Lidar Lite V IPS HB Tablet results

4 6 Discussion Elderly cyclists Sensors Tablet results Recommendations Conclusion Appendix Code for the prototypes Datasheets for the sensors IPS-265 sketch HB100 sketch HB100 test results IPS-265 test results Lidar Lite V3 test results

5 CHAPTER 1 INTRODUCTION 1.1 Background The bicycle is a widely used vehicle. Twenty one million bicycles, including electric bicycles, are sold every year in Europe [1]. A bicycle is a useful vehicle that can be used by all age groups. However, it appears that elderly cyclists have a higher risk of getting injured in traffic [2]. In figure 1.1 it can be seen that elderly people have a very large risk of getting involved in a cycling accident as compared to other age groups. In this research the technical possibilities of helping elderly cyclists in traffic will be explored. Currently there are a lot of new developments in the automotive car industry. The new technologies emerging from this industry might also be suitable to be used on bicycles instead of cars. These sensors can for example be used to detect other road users or the environment. A lot of elderly people are already using a smartphone, and it is expected that the amount of elderly using a smartphone will increase in the future [3]. Because a smartphone offers relatively much processing power, it is interesting to find out if smartphones can be used to provide the data processing which is needed by the other parts of the system. Figure 1.1: Accident risk of cyclists in the Netherlands by age group, per billion (109) kilometers, in 1983 and There were 25,231 fatal and nonfatal accidents involving cyclists aged nine years or older in that period [4]. Indes This research will be conducted at the company Indes, located in Enschede, the Netherlands. Indes is operational in the field of surgery robotics and e-mobility. Indes aims to create reliable products and puts a lot of time and effort in perfecting their products to be very useful, safe, secure and also beautifully designed. 5

6 1.2 Motivation In their daily life, elderly people do have more physical problems, such as pain in their knees or trouble turning their head, and they also have an increased reaction time. These physical disadvantages make it more difficult for elderly to keep up with the fast pace of traffic [5]. Elderly cyclists are more cautious in traffic and they sometimes evade possibly dangerous situations [6]. But still a large part of reported bicycle-accidents involve one or more elderly cyclists. Cycling is a healthy activity and these health benefits already outweigh the risks [7]. Also elderly people should be able to profit from these health benefits. 1.3 Problem statement Cyclists aged 75 years or older have a greatly increased risk of dying or being hospitalized due to a bicycle accident [8]. It appeared from interviews [6] that elderly cyclists are aware of their vulnerability in traffic, and that they are not confident in traffic. They indicate to avoid dangerous or crowded situations while cycling. They are also afraid of, for example, falling and breaking their hip. Also the amount of severe injuries is generally increasing with age [8]. Both the amount of traffic accidents among elderly cyclists and the severity of these accidents indicate that traffic accidents among elderly people are a large problem, and a solution for this problem needs to be found. To aid these people in traffic it is necessary to find out what the common traffic accidents among elderly cyclists are, and also how these accidents are caused. Once that is known it is possible to find a fitting solution to this problem. 1.4 Research questions The goal of this research is to find the underlying problems elderly are experiencing in traffic and to find a technical solution to make cycling for elderly people more safe. An extra focus has been given to find out if the bicycle could detect other road-users in the area. Additionally, research will be done to find out if it is possible to integrate a smartphone in this technical solution, for example if a smartphone can be used to process sensor data and/or show useful information for the user on the screen. The question that has been used for this research is the following: How could the safety of elderly cyclists be increased by using road-user-detection technology?" To guide the research in answering the main research question four subquestions have been formulated. 1. What are dangerous situations for elderly cyclists?" 2. What limitations do elderly cyclists have in traffic?" 3. What sensors are able to detect road users in traffic?" 4. Can a smartphone be used to detect road users in traffic?" 6

7 CHAPTER 2 STATE OF THE ART A lot of road-user-detection technology is currently being developed for (automotive) vehicles, for example cars, trucks and even trains. However, there are a lot of other technologies that can be used to make cycling safer, and also there are already some systems available to improve the safety of cyclists. In this chapter some results from earlier research is documented to find out what the common accidents among elderly cyclists are. Also multiple existing technologies and systems to detect road-users and obstacles will be discussed. Also some technical principles behind these sensors or systems are described. 7

8 2.1 Research on accidents among elderly cyclists There are multiple kinds of traffic accidents among elderly cyclists that occur relatively frequently. In figure 2.1 these kinds of accidents are shown. Unfortunately most of the accidents are of an unknown type. It appears that a lot of traffic accidents among elderly cyclist are because of the cyclist being influenced by alcohol. After that the most common cause for traffic accidents among elderly are objects. However, rails, curbs and animals can also be shared under the name obstacles. From that point of view, obstacles are the major cause for traffic accidents among elderly cyclists. Other research shows another dangerous situation for elderly cyclists. This situation happens on two-way cycle paths [5]. Most accidents on cycle paths happen when traffic is also allowed to come from an unexpected direction, which is for example the cause on two-way cycle paths. Unexpected traffic coming up from behind and accidents where the cyclist is in the blind spot of a car or truck are also situations where a lot of accidents among elderly cyclists occur [9] [10]. Figure 2.1: Classification of single sided cyclist accidents in GIDAS based on accident description [single case analysis] [8] Limitations of elderly cyclists Elderly cyclists usually indicate that they have trouble with keeping up with traffic. Also they cannot turn their head very well because their neck is more stiff as compared to younger people. Elderly cyclists also have more trouble with hearing and seeing [9] [5]. Also, Elderly people have pain in their knees while cycling. Because of this they are often using electrical bicycles. Unfortunately, these electrical bicycles allow elderly cyclists to move at higher speeds, which could bring them into dangerous situations. 8

9 2.2 Applications for bicycles There are already some commercial systems available to help cyclists and elderly cyclists in traffic. The systems described in this chapter are created with the purpose of increasing the insight of the cyclist in traffic. But there are also applications currently in development. In this section these existing or emerging technologies have been listed. TNO smart bike [11] TNO has created a smart bike. Already a lot of relevant research has been used in this bicycle, and within this project it is specifically tried to improve the insight of the user in traffic without being to intrusive. This bicycle has a MobilEye camera sensor on the rear of the bike, and a radar attached to the front of the bicycle (for radar, also see chapter 2.4). The TNO bike originated from a research also specifically focused on the safety of elderly people in traffic. Garmin Varia Radar [12] The Garmin Varia Radar consists of two parts. The main part is a rear light that get s brighter when traffic is coming up from behind. In this way the visibility of the cyclist is improved. On the steer an indicator for the user can be placed. The indicator visualizes using LED s how close the traffic coming up from behind is. This functionality is meant to increase the awareness of the cyclist about what is happening on the rear of the bicycle. The system is using a long-range radar. Garmin has mainly focused this project on racing bicycles, which means that it is not mainly developed for elderly people. Cycle Alert [13] The Cycle Alert is a system especially created for bicycles and trucks. A lot of accidents between cyclists and trucks happen because cyclists are in the blind spot of the truck. The blind spots of a truck (or other vehicle) are the places where the driver can not see the other road users. Cycle Alert uses RFID technology to solve this problem. Using detectors and both the bike and the truck, the truck-driver gets warned when there are cyclists in the blind spot. Cycle Alert is a solution to a very specific problem and it requires both the cyclists and the truckdrivers to have the system installed. LaneSight [14] The LaneSight is a small device that can be put to the back of a bicycle. LaneSight is currently in development. It is designed to detect traffic coming up from behind. It detects traffic coming up from behind using a camera and a sonar sensor. When LaneSight detects a dangerous situation (that is when an object get s in range of the sonar) it also starts recording and saving the data. In that way the cyclist can look back to the video of a particular dangerous situation. From this point of view the LaneSight application can actually be seen as a smart dashcam instead of an assistant for the cyclist, since this system does not warn the user. Byxee [15] Byxee uses artificial intelligence algorithms to recognize the possible risks, and is designed to identify obstacles and other road users in front of the cyclist. Byxee is an IndieGogo project, which means that this project is set up using crowd-funding. However, the desired financial goal of Byxee is never reached. Still it appears that the team behind Byxee is continuing the development of Byxee. But there are currently no signs of a working prototype. 9

10 SeeMe [16] SeeMe Cycle Warning is a stationary warning system used to warn other traffic about the presence of cyclists. This system is using inductive loops or heat sensing and activates some lights (for example above traffic signs) to warn other road users that there are cyclists nearby. TurnSafe [17] TurnSafe is a range of products from the company Vision-techniques. These products are meant to improve the safety of cyclists in the blind spots of cars. One of these products is a camera that aids drivers with detecting cyclists in the blind spot of the car. When an obstacle (e.g. a cyclist) is detected in the blind spot of the car the driver gets an audible warning. Another product in the TurnSafe package are sonar sensors that can be connected on the side of the car or truck to better detect obstacles in the blind spot. 2.3 Principles Within the car industry a lot research has been done on specific distance sensors. The concept is to measure the distance in all directions to create an image of the surroundings, and/or to measure the velocity and direction of objects near the user. In the next section multiple sensor technologies are discussed, which can be used for achieving these concepts. There are multiple principles that apply to these sensors, which are important for their function in traffic. In this section these principles are explained. Beam divergence An important concept for interfacing distance sensors is the beam divergence. In the datasheets the beam divergence is often described according to an "azimuth" angle and an "elevation" angle. The Azimuth angle is an horizontal angle measured clockwise from the "north", and the elevation is the vertical angle measured from the "horizon". Datasheets of distance sensors often note both the azimuth and the elevation angle, which can be interpreted as respectively the horizontal and vertical spread of the "beam". A visualization of these angles is shown in figure 2.2. Time-Of-Flight principle Sonar, LiDAR and 3D camera s can use the Time-Of-Flight (TOF) principle to measure distances. The TOF principle uses the velocity of light (LiDAR, 3D camera) or sound (Sonar) to detect distances. At first a signal will be transmitted by the sensor. That sensor will reflect and the reflection of the signal will be received by sensor again. The time it took for the signal to return is used to calculate the distance to the measured object. This can be achieved by a timer, but another approach would be to compare the phase of the signals. Then from the phase-shift the distance can be calculated. The time of flight using a timer can be calculated using formula 2.1: t = 2 D/c (2.1) Where t is the time between the transmission and receival of the signal, D is the distance to the object which the signal is pointed to and c = the speed of light ( m/s). Doppler shift Simple radar sensors (CW radars) output a sinusoid with a certain frequency, where the frequency indicates the velocity of the detected object. The relation between distance and the frequency of the radar output is shown in formula 2.2. f = v (2 F/c) (2.2) 10

11 Figure 2.2: Visual representation of Azimuth and Elevation angles Where f is the output frequency, v is the velocity of the measured object, F is the radar carrier frequency (approximately Hz) and c is the speed of light ( m/s). This means that when the radar carrier frequency is known, it is possible to derive a velocity from the output frequency of the radar. Frequency Modulated Carrier Wave (FMCW) radar A major setback of a standard CW radar is it s inability to measure distances. Where CW radar only uses the Doppler effect for velocities, an FMCW radar also uses frequency modulation to determine the distance to the measured object. The FMCW radar uses Doppler shift for measuring the velocity of the object, but beside measuring the phase shift of the signal these sensors also measure the frequency of the signal. According to the moment a certain frequency is retrieved it is known when that frequency is sent and when it is received. Using the time between the transmission and receival of that frequency the distance to the measured object can be calculated. FMCW radars are more complex and more expensive than standard CW radars, but the ability to measure both distance and velocity is a major advantage over other types of radar, and other sensors in general. Sample rate When detecting road users in traffic some constraints and some assumptions need to be made. In this research some low-quality sensors will be used to detect road-users and obstacles from a bike. However, the price and availability of the sensors results in some constraints for the final test result. The sample rate of a sensor means amount of measurements the sensor can do within one second. A higher sample rate results in a higher precision, but more precision would also demand more processing power if the data needs to be handled by, for example, an algorithm. 11

12 2.4 Sensors There are already multiple sensor technologies available that can be used to detect obstacles, road users and the environment in traffic. In table 2.1 an overview is given of a variety of these automotive sensors available today and their respective properties. The bottom row indicates the overall scores of each sensor technology, from which can be seen that Radar and LiDAR are promising technologies. In this section these sensor technologies and their respective properties will be discussed in more detail. Short range Radar Long range Radar LiDAR Sonar Video Camera 3D-camera Range Measurement <2m Range Measurement 2-30m n.a Range Measurement m n.a Angle Measurement <10 deg Angle Measurement >30 deg Angular Resolution Direct velocity information Operation in rain Operation in fog or snow Operation if dirt on Sensor Night vision n.a. n.a. n.a. n.a. 2 3 Total Table 2.1: Typical strengths and weaknesses of automotive sensors available today [18]. (1: impossible, 2: only possible with large additional effort, 3: possible, but drawbacks to be expected, 4: Good performance, 5: Ideally suited) LiDAR LiDAR is the abbreviation for Light Detection and Ranging. LiDAR is an emerging technology which is currently developed for automotive vehicles. The LiDAR sensor transmits one or multiple infra-red lasers and detects the reflection of these lasers. By keeping track of the time it took before the laser returned it can calculate, using the speed of light, the distance of that which the laser reflected on. This method is also called the Time-Of-Flight (TOF). By using lasers the LiDAR works both during night and day, and due to the specific wavelength ( nm) of the infra-red laser a LiDAR does not have interference from light in the surroundings [19]. However, LiDAR sensors are functioning significantly worse in rainy or foggy weather, since the beam of the laser refracts through water. Also with dirt or smoke a LiDAR can give incorrect measurement values [18]. Almost all companies working on automotive vehicles are currently testing their systems using a LiDAR. The only exception on this is the company Tesla, who refuses to use LiDAR because of the high costs of this technology [20]. However, the expectation for the near future is that the costs of LiDAR will decrease dramatically [21]. Currently LiDAR technology is too expensive to be implemented on bicycles, but for the near future it might be cheap enough to be put on bicycles also. Unlike Radar, LiDAR is unable to measure the speed of an object directly. The speed of a tracked object can however be obtained by comparing different measurements and differentiating these results [22]. This is possible since the LiDAR is highly accurate in it s measurements. The angular accuracy of the LiDAR is generally 0.1 degree [23] Radar Radar (Radio Detection and Ranging) is a well-known technology used for various applications. Simple radar sensors are currently available for only a few dollars, but there are also very accurate high-range radars available. Radar works by transmitting and receiving high frequency (about 77 GHz) microwaves, and here the Time-Of-Flight method can be used to obtain a distance. However, with these microwaves also the Doppler effect occurs, which makes it possible 12

13 to detect the velocity of the measured object [22]. Because of this feature only one measurement is required to obtain the location and velocity of a measured object. Also, since Radar is able to detect movement it is also able to detect objects in bad weather conditions, such as fog or snow, and also quite well in rain. Also dirt will not affect the measurement of the Radar. This is because of the usage of microwaves instead of infra-red light, which is used by LiDAR systems. The difference between microwaves and infra-red light, which are both electromagnetic radiations, is that the wavelength of microwaves (about 4mm) is longer than the wavelength of infra-red ( nm). Larger wavelengths have the ability to penetrate through larger objects, for example dirt on a sensor or a raindrop. There are both short-range and long-range radar systems available. Short-range radars generally are used for close object detection, for example to aid car drivers with parking their car. These sensors typically have a field-of-view up to 30 degrees, and have a range between 20 and 30 meters. Long-range radars often have a field-of-view up to only 10 degrees, but can measure up to 250 meters. The angular accuracy of a radar is typically between 0.5 and 5 degrees, yielding lateral uncertainties from 87 cm to 4 m at 50 m [22]. The properties of the Radar in bad weather conditions, and the use of the Doppler effect are a large advantage of Radar compared to the LiDAR. However, the radar is much less accurate than the LiDAR Sonar Sonar (Sound Detection and Ranging) is mainly used in submarines, but it also has applications by the company Tesla for sensing in traffic. The sensing methods used by Tesla are for shortrange object detection, since sonar waves (ultrasound waves) cannot travel far in air. The technical behavior of a Sonar is much alike the behavior of a radar. Sonar also uses the Time-Of-Flight principle to calculate the distance to a certain object, and using the Doppler effect the velocity of the object can also be calculated. Because Sonar does not measure that far in air, it is best to use the sonar for short-range sensing. Compared to the long Range radar or LiDAR systems sonar does not have an advantage over radar or LiDAR. However, a sonar might be well used in combination with other long-range sensors, and can function as a warning trigger for very close obstacles around the (elderly) cyclist Camera Sensing the environment using a camera has some particular advantages and disadvantages as compared to distance sensors as LiDAR, radar and sonar. Detecting objects and obstacles using a camera [22] requires a lot of image processing, for which a strong processing unit is needed. Also a camera has a large influence from (sun)light. When a light shining into the camera, the images obtained by the camera are too bright and the camera is not able to detect the objects. Also the vision of the camera is heavily influenced by fog, snow and dirt. However, the camera is also able to generate very clear images of the surroundings, which can be an advantage of LiDAR or Radar. The velocity of objects need to determined by comparing successive images. But for the detection of objects alone there is already a lot of processing needed. The processing needs of the camera images are probably the main concern of the use of a camera. But with current developments in Artificial Intelligence, Deep learning and new processing units this problem might be taken care of in the future. Another advantage of the camera is that the images can be directly presented to the user on a screen without even processing the data. The view of the camera could be presented to the cyclist immediately. When the user has a screen and the camera on the back of the bicycle this system can show the user immediately what is happening behind the cyclist Time-Of-Flight-Camera Time-Of-Flight camera s, or 3D camera s capture an entire scene with a single light pulse (unlike a LiDAR). The TOF-camera is able to measure the distance for each pixel by comparing the 13

14 phase of modulated infrared light. When compared to video camera s the resolution of the TOF camera is much lower, about 200 X 200 pixels. The distance can be measured to a maximum of 10 meters at an accuracy of approximately 1cm. Also, TOF camera s can reach very high frame rates, going up to 200 fps. However, TOF camera s are still very expensive and these sensors are heavily influenced by sunlight [22], which is a large disadvantage of using this sensor. 2.5 Roadmaps A simple, low-tech solution to increase the mobile awareness and insight in traffic for a cyclist is to use a roadmap. A roadmap shows all roads and surroundings of the area. Digital roadmaps have the ability to update this information to increase the correctness of the data, and to show additional information to the user. A roadmap, however, does not help the cyclist get information about his immediate surroundings, such as other road-users around him. Paper roadmaps Roadmaps on paper have the advantage of clarity and trustworthiness. However, within a few years these are outdated. Besides that, most of these roadmaps are not easy to use on a bike and they are hard to understand, since they do not show the current location. Digital roadmaps (navigational systems) Digital roadmaps have the advantage that they can be updated, and also have the ability of displaying the current location of the user. An advanced digital roadmap or navigational system can give the user insight in traffic jams, traffic accidents or other problems on the road. An example of a digital roadmap is Google Maps, which also has a directions functionality. There are also many bike-computers available that can show directions to the cyclist to guide them to the destination. 2.6 Smartphones From a research in 2014 it appeared that, at that time, 63.3 percent of the people aged 50 years or older were in possession of a smartphone [3]. The expectation is that the amount of elderly people using a smartphone will increase over time. The smartphone is a relatively strong computing device which a lot of people have in their pocket nowadays. Smartphones do have a lot potential since they are applicable in many ways. They contain a lot of different sensors, such as an accelerometer, a compass, GPS and a camera. This combination of sensors can be used to help people participating in traffic. There are already some applications for smartphones in traffic available, which are listed in this section. Flitsmeister Flitsmeister is an application warning road users for traffic jams and speed limits. This app works by crowdsourcing all data, which means that users can indicate whether there is a speed limit control, and the respective location. Flitsmeister strongly builds upon the concept of crowdsourcing, meaning that it can only work properly when a lot of people are using it. Navigation on smartphones There are many navigation apps available in smartphones to aid road users in finding their way. However, there are no apparent navigation applications available that provide the user with live information about their direct environment. There are applications providing data about traffic jams or changes on the road, but no information is given about vehicles around the user of the application. 14

15 PsycleSafe [24] PsycleSafe is a canceled Kickstarter project. The concept of PsycleSafe was to put a phoneholder to the back of a bicycle. In that holder an iphone could be placed, running a certain application. The camera of the iphone would be used to detect traffic coming up from behind. A small LED indicator connected to the steer of the bicycle should warn the cyclist about this traffic. PsycleSafe has claimed that they already have a working version of the application, however, the project is cancelled by the organizer of the project. The reason for this cancellation is unknown. 2.7 Discussion Already multiple systems exist for aiding cyclists in traffic. These systems do not focus specifically on elderly cyclists, but on cyclists in general. The Garmin Varia Radar, LaneSight and PsycleSafe focus only on traffic coming up from behind, but while traffic coming up from behind is a common cause of accidents among elderly, it is not as significant as obstacles. The other existing solutions stated in this chapter are mainly focusing on the blind spot. Only Byxee appears to be doing obstacle detection in front of the user, but a working prototype has not been demonstrated yet. From earlier research it can be said that dangerous situations from elderly cyclists are mainly obstacles. Also two-way cycle paths (where other road-users come from the side, from the point-of-view from the cyclist), vehicles taking over quickly and getting in the blind spot of a car or truck are large causes for traffic accidents. With the sensing methods described in this chapter it might be possible to aid elderly in some of these situations. It appears that there are a lot of different methods of sensing a (traffic) environment. Standalone sensors can work with a high accuracy and, when programmed well, these sensors can be relied on. However, these sensors, especially LiDAR and TOF camera s are still quite expensive and developing for these sensors is a tedious task. This is because the data from a LiDAR or TOF camera needs to be processed and interpreted before it can be shown towards the cyclist. Doing this processing requires multiple algorithms to interpret the data well. When looking to table 2.1 it appears that a combination of LiDAR and Radar would cancel out each others disadvantages very well, except for the high cost of the LiDAR. For this combination it would be best to wait a few years until more research on these topics has been done, and when more development tools are available. Mobile phones are an uprising force. The processing power of smartphones is still significantly lower than desktop computers or servers. With the increasing speeds of mobile networks (4G, LTE). It appears that all systems have their own advantages and disadvantages. The camera of the smartphone can also be used to assist an elderly cyclist in traffic. Since the camera is able to get a detailed view of its surroundings. Setbacks of the camera are the large processing requirements and the influence of weather conditions. Smartphones do however have good processing units. For both ios and Android the library OpenCV [25] is well-known and it appears to be the standard for image processing in general. The live detection of obstacles using a smartphone is probably still a difficult task to do and it will probably contain a lot of errors, but it might become possible in the future. A combination of some of these systems might return the best result. Furthermore it would depend on the pricing of the bicycle. For a small price, for example using a smartphone app, it will be possible to create a useful app that can, when the weather is right, help the user a little bit by detecting the surroundings using the smartphone camera. However, with a higher investment the system could be heavily improved by using LiDAR, radar or camera, or a combination of one of those. Since a lot of people already have adopted the smartphone this would be a very interesting field to explore. 15

16 16

17 CHAPTER 3 DESIGN Based on the findings in the State-of-the-art it appears to be possible to use detection sensors to increase the sight of a cyclist in traffic. In this chapter a design for such a system is made. The prototype created by this research is a proof-of-concept, which means that the system is a prototype designed for the purpose of showing that something can, or cannot be done. This particular proof-of-concept is made to show whether it is possible or not to detect obstacles and road-users in traffic. 3.1 Requirements For designing a system for aiding elderly cyclists there have to be requirements. These requirements are constructed using a MoSCoW analysis. The system MUST: Be able to detect the presence of other road users. Be able to detect the presence of obstacles on the road Function properly on a smartphone. Be able to show the data about the surroundings to the user quickly enough such that the traffic situation has not been significantly changed in the meanwhile. Be able to show to the cyclist that a situation can be dangerous. The system SHOULD: Be able to detect the location of other road users. Be able to detect the velocity of other road users. Be able to detect the direction of other road users. The system COULD: Be low-cost. Warn the cyclist using audio. Have automatic detection of dangerous situations. Track an object to estimate how the situations will unfold. Have a wireless connection between the smartphone and sensors. Be powered using the battery from an e-bike as power source. Be able to operate on the power of the smartphone only. The system WILL NOT: Have control over the movements of the bicycle. Warn other road-users about the behaviour of the cyclist. Warn the cyclist about his own behaviour (for example when the cyclist breaks a traffic rule. 3.2 Overview of the concept The requirements lead to an overview of the final system, which is shown in 3.1. This system will detect the surroundings using sensors. Using a microcontroller this data is sent towards a smartphone, where the user can see the sensor data on the screen. This sensor data will 17

18 be shown in a graphical way, such as a live-updating plot. In this case the user can see the immediate sensor data in an insightful way. The working of this system works in multiple steps. At first, there is something happening in the surroundings, for example a car coming up. The sensor detects this and this detection causes a deviation in the output signal of the sensor. Secondly, the microcontroller reads the output signal of the sensor and interprets this signal. The signal is converted to a velocity, direction, or distance, and measurements that appear to be incorrect (for example velocities over 200 km/h) are filtered. This data is send digitally towards the smartphone, which can show the data to the user. Figure 3.1: Full concept diagram of the designed system Based on the requirements and the State-of-the-Art analysis it is decided to test both LiDAR and radar. This is mainly because of the good overall quality of the LiDAR and radar, which is clearly seen in figure 2.1. These sensor technologies usually have a good sample rate and good accuracy, and those features are needed to detect other road users or obstacles in traffic. Multiple prototypes of this system will be built to test whether the requirements can be fulfilled using a LiDAR or radar. The block-diagram for the radar circuit can be seen in figure 3.2, and the circuit for the LiDAR can be seen in figure 3.3. A major difference between the LiDAR and radar system is that the LiDAR measures distance and angle, and the radar measures velocity and direction. This means that the data is shown differently towards the user for these sensors. Still both systems can work with a microcontroller in the middle. 18

19 Figure 3.2: Block diagram of a radar sensor connected to microcontroller Figure 3.3: Block diagram of Lidar connected to the microcontroller 3.3 Software For this application also software for both the microcontroller and laptop/smartphone needs to be written. For this research it is necessary to make the system work on a smartphone, but for the initial tests it would be easier to also make the system work on a laptop. Because of this a mobile platform and a programming language needs to be found that is easily portable to a desktop/laptop environment. The software on the microcontroller does not need to do much. A microcontroller is usually not very powerful in calculations, and it is best if the micro- 19

20 controller is focused on getting the data from the sensors and sending it towards the computer or smartphone. The data can then be interpreted by the computer or smartphone which shows the relevant data to the cyclist. For this research it is also useful if the software contains an export functionality such that the data of the sensors can be retrieved for further research. A schematic overview of the software for the computer/smartphone is given in figure 3.4. When the system shows the data to the cyclists it is not necessary to run multiple difficult algorithms on the data before showing it. A simple radar view that can indicate the location and distance of an object might already be enough to notify the user that something is coming up. Also, for this research it would be enough, since this project is about finding out if and how road-users can be detected using road-user-detection technology. Figure 3.4: Overview of the classes in software 20

21 CHAPTER 4 DEVELOPMENT Based on the proposed concept in the previous chapter, this chapter describes the building of prototypes that will be used to test if it is possible for the chosen sensors to detect obstacles or other road-users in traffic. 4.1 Selection of sensors In the state of the art the behavior of multiple sensor types was analyzed. Using the requirements in chapter 3 a selection has been made for sensors that appear to be suitable for this project, and these sensors and their characteristics are shown in table 4.1. Important factors are: High sample rate, large range, low cost and high accuracy. A high sample rate is important to detect the object or road user quickly enough. When the system could, for example, take only 1 measurement per second it is too slow too properly detect an obstacle. Depending on the technology a different sample rate is needed. For Li- DAR the sample rate must be at least several hundred measurements per second. For a Radar this is generally not an issue, since these work with the Doppler effect which immediately and constantly influences the output signal. A large range is important to detect obstacles at large distances. Then the obstacle can be detected long before it is near the cyclist. It is important for the final product to keep an eye on the costs of the product. Some sensors can be very expensive, but a bicycle that is much more expensive because of this sensor would probably not sell very well. For this research also more expensive sensors could be tried when it is expected that the prices for these sensors will drop in the future. A high accuracy is needed for the reliability of the sensors. Incorrect measurements are problematic for this system, which means that it should be tried to use sensors that are very accurate to reduce the incorrect measurements. Beam divergence is not necessarily important for the quality of the sensor, but it is an important feature when interfacing and using the sensor, and is important to keep in mind for later research. From the selected sensors in table 4.1, three sensors have been selected to be tested further. These three sensors are the Lidar Lite V3, the HB100 Radar and the IPS-265 Radar. A special case within the selection of sensors is the Innosent IVS-465. This appears to be a very good all-round sensor, capable of both measuring speed and distance towards a certain object. This is possible because the IVS-465 is a FMCW radar. However, this sensor is not fabricated anymore by the manufacturer, and for this research no other alternative sensor for the IVS-465 has been found. In advance of this research it can already be said that FMCW radars might be very good to be tested in the future. 21

22 Table 4.1: Sensors that appear interesting for this research Sensor Producer Technology Range (m) Sample rate (Hz) Carrier wave frequency Azimuth Elevation Cost ( ) Supply voltage Velocities HB100 AgilSense CW Radar 0-20 m 2 khz 10,520-10,530 GHz V unknown Lidar Lite V3 Garmin LiDAR 0,05-40 m Hz n.a. 0,5 0, V n.a. IPM-165 Innosent CW Radar 0-15 m Hz 24,050-24,250 GHz V 1 - ca. 200 km/h IVS 465 Innosent FMCW Radar 0-30 m unknown 24,050-24,250 GHz V unknown IPS-265 Innosent CW Radar unknown unknown 24,050-24,250 GHz V 1 - ca. 200 km/h K-LD2 RFBeam CW Radar 0-25 m 1,28-2,5 khz 24,050-24,250 GHz V km/h 22

23 4.1.1 Garmin LiDAR Lite V3 The LiDAR Lite V3 is with 150,- relatively expensive in comparison to the two radar sensors, however, the technical specifications for the Lidar Lite V3 are very promising, since it has a high sample rate combined with a high accuracy. This can be very useful to detect other traffic users. A setback of the LiDAR Lite V3 is that the beam divergence is very small, since the emitted signal is a laser beam. To increase the beam divergence of this sensor it will be installed on top of a servo, which will rotate the LiDAR horizontally. The measured point distance can then be combined with the angle of the servo. The known data then consists of an angle and a distance towards the measured object. According to the datasheet, the Lidar Lite V3 is a very accurate sensor, with a very small beam divergence. The expectation is that the Lidar Lite V3 is very well capable of detecting even small objects in a relatively large range around the user. Figure 4.1: The Garmin LiDAR Lite V3" LiDAR sensor Innosent IPS-265 Radar The Innosent IPS-265 Radar is a radar developed by the company Innosent. This is a dualchannel radar, which means that it sends and receives two radar signals instead of one. This has the large advantage that from the signals of the IPS-265 Radar, a direction can be derived by comparing the phase-shift between the two IF-signals. The IPS-265 is not able to measure the distance towards the measured object; it can only measure the velocity. This is because this radar does not support frequency modulated signals (FMCW) which is needed for distance measurement. Since the Innosent IPS-265 is built for small automotive applications the expectation is that the sensor can be quite reliable in measuring velocities and direction. Some setbacks for the IPS-265 radar are the difficult interfacing and calculating the direction from these signals HB-100 Radar The HB-100 sensor does not appear to be very promising: The low price is an indication that the quality of the sensor is probably not very high. Also the range of this sensor is not shown on the datasheet. In this research it will be tried to find out the range of the HB-100. This sensor is mainly included in this research due to it s low price. The output signal of HB-100 requires a lot of filtering and processing before it can be interpreted correctly. 23

24 Figure 4.2: The Innosent IPS-265" Radar sensor Figure 4.3: The HB100" Radar sensor Handling the output of the radars Both the HB-100 and the IPS-265 output an oscillating signal between 4,7 and 5,3 volt. The measurement of the sensor affect the frequency of this output signal, which means that, for an Arduino to successfully read the signal, an amplification circuit needs to be made. Also, it is useful if the frequencies of the oscillating signal can be amplified and that the other frequencies are filtered. In this way the measurements may become more accurate and a lot of noise can be filtered out. 4.2 Selection of platform The initial tests will be done using a laptop. This is because development can be done on a laptop and some quick code-adjustments can easily be made when something appears to be wrong. Also the data acquisition is easier on a laptop, since the sensor data can be exported to the.csv file format. Besides, a laptop is able to provide more electrical current for the system to work, and it is expected to cause less problems with the powering of the servo and LiDAR together. Using a smartphone in this phase would thus only slow down the testing process. However, in a later phase it would be necessary to also run the application on a smartphone, and it is efficient to re-use the same code. Multiple programming languages and platforms have been considered and only a few remained: C, C++ using OpenFrameworks, or Java. After having done some small tests using C and C++ (using OpenFrameworks), the choice for Java and a Windows tablet has been made. For C and C++ there were no working Serial-library available, which meant that it would be too much work to implement. Android remains to be a good option, since Java is natively supported on Android and also a serial library is available. This means that the Java code can run on both macos, Windows and Android, which would be very efficient in the workflow of this research. Still some code for the visualization of the data needs 24

25 to be rewritten, and therefore it would be more efficient to test directly on a windows tablet. Detection using external sensors Another solution might be to use a smartphone as processing unit to handle and show the data from a LiDAR/sonar/radar/camera sensor. This can be done using a serial communication over the USB port of a smartphone. Several USB-serial libraries are available for both Android and ios. However, for ios it is necessary to join the Apple MFI program [26] to create applications using a physical connection. It is also possible to interface the external sensors using bluetooth. Also for ios this is possible without the MFI program. But, when interfacing external sensors the bandwidth of the sensor must be taken into account, since the amount of data that can be transmitted over bluetooth is limited as compared to the amount of data than can be transmitted over USB. This will require further research when a specific external sensor is known. For a USB serial connection using Android two serial libraries have been found. 4.3 Electronics To interface the sensors some electronic circuits had to be created. For the Lidar Lite V3 only a basic circuit, which is indicated on the datasheet, is needed. For the HB100 and the IPS265 an amplification and filtering circuit needed to be built. At Indes a self-fabricated microcontroller is used for the products. However, this is a specific piece of hardware which is not directly developed for prototyping purposes. It appears that this board would require relatively much additional programming and additions to the hardware. But since this research is about a proof-of-concept, an Arduino Uno Rev3 board will suffice. The communication between the microcontroller and the Arduino will be done using USB. For the final product a wireless connection, such as Bluetooth, may be preferred over USB. However, a wireless connection between the smartphone and the Arduino is considered out-of-scope for this research. The choice to use Arduino and Java is based mainly on the experience of the researcher; which are both environments that he is comfortable to work with LiDAR circuit The Lidar Lite V3 is connected to the Arduino in such a way that it can be interfaced by the I2C protocol. The LidarLiteV3 Library is used to obtain the measured distances. This library is created by the manufacturer of the Lidar Lite V3 and is considered trustworthy. In this research it will not be checked if there might be a better or more accurate way to interface this sensor, and it is assumed that the manufacturer already found a very decent solution to this. Because an 680 nf capacitor was not available, this capacitor is replaced by three 220 nf capacitors in parallel, having a total capacitance of 660 nf. The small difference of 20 nf is not measured with since this is a very small difference. To rotate the LiDAR it is connected on top of a servo motor using two screws. The servo that has been used is the Batan S1123. The schematic of the LiDAR circuit is based on the datasheet from the Lidar Lite V3, but it also includes the servo. The schematic is shown in figure Radar circuit 1: HB100 The output of the HB100 is indicated by the IF port, which is a sine wave which oscillates between 4.7 and 5.3 Volt. However, this signal is unreadable for the Arduino, since the Arduino can only read voltages up to 5.0 Volt. Besides, it is desired to use the zero crossing method to determine the frequency of the signal. Because of this the signal needs to be shifted towards an average of 2.5V, and it is desired that the frequencies that belong to the radar measurements are amplified. The circuit for the HB100 is found in figure 4.5. A visual representation of this scheme is shown in the appendix in figure 7.2. This circuit adds an offset of -2.5 Volts and amplifies the frequency of the signal. The result is a circuit that outputs a signal that oscillates between 0 and 3 Volts.The change between 0 25

26 Figure 4.4: Fritzing sketch of the I2C connection circuit for the Lidar Lite V3 and the accompanying servo. In the prototype the Lidar is connected on top of the servo. and 3 Volt is large enough to be read by the Arduino, since the frequency of the signal will be measured using a method called zero crossing. The output of the circuit will be connected to a digital pin on the Arduino. The digital input of the Arduino only reads the values HIGH (voltages higher than 2.5 V) or LOW (voltages below 2.5 V). To make the arduino interpret the signal correctly, the library FreqPeriod is used. This library is able to retrieve a velocity from the frequency in the signal. 26

27 27 Figure 4.5: Schematic of the amplification/filtering circuit for the HB-100

28 4.3.3 Radar circuit 2: IPS-265 The output of the IPS-265 is comparable with the output of the HB100. The IPS-265 also has an output which oscillates between 4.7 and 5.3 Volts. However, the IPS-265 uses two radar signals and has two of these output signals. The circuit used for the IPS-265 is very much alike the circuit for the HB100: for both the outputs of the IPS-265 the same specific amplifier circuit has been used. The output of the first amplifier circuit will be connected to pin 7 (to be interpreted by the FreqPeriod library) and the Analog0 pin. The output of the second amplifier will be connected to the Analog1 pin of the Arduino. The signals entering the analog pins of the Arduino are used for measuring the direction of the detected object. The circuit is shown in figure 4.6 and, for a more visual approach, figure 7.1 in the Appendix. A plot of both output signals of the IPS-265 is shown in figure 4.7. The reason why only one of the radar outputs can be interpreted by the Arduino has to do with the need for the internal timer of the Arduino. There is only one timer available to measure the small frequencies, which means that the frequency of only one radar signal can be retrieved. This means that the two signals from the IPS-265 cannot be processed both. Figure 4.6: Schematic of the amplification/filtering circuit for the IPS-265 In figure 4.7 it can be seen that sometimes the signal oscillates between 0 and 3 Volts, but sometimes it is a noisy signal with a much smaller voltage difference. Also the frequency measurements do not measure a frequency at those points. The noisy small signals appear 28

29 when there is not a moving object detected. When a moving object is in range of the sensor the signal immediately starts to become a frequency modulated signal between 0 and 3 Volt. Figure 4.7: The raw output of the two ips265 outputs The IPS-265 radar module returns two radar signals since it is a stereo -radar, which can be interpreted as being two radars in one. From these two signals it is possible to measure a direction. however, for an Arduino this is difficult. Since there are some tedious algorithms that need to be used for good results. However, an algorithm to simply detect if an object is coming towards the sensor, or moving away from the sensor is much more feasible. By subtracting one signal from the other a direction can be determined. In figure 4.8 the output of the difference between the signals is shown, while moving a hand towards and from the sensor. Figure 4.8: The resulting signal when subtracting signal 1 from signal 0 During the development of the electronic circuit for the radars not all required resistors were available. Some resistors have been replaced by other resistors in series. In some cases there is a small deviation in the resistor values. A detailed overview of these measurements are shown in table

30 Component Required value to datasheet Used components Measured value* Unit Resistor 8k2 2 x 4k7 = 9k4 9,39k Ω Resistor 10k 2 x 4k7 = 9k4 9,39k Ω Resistor 12k 2 x 4k7 = 9k4 9,39k Ω Resistor 100k 100k 99,85k Ω Resistor 330k 220k + 100k = 320k 321,7k Ω Resistor 1m 1m 1,01m Ω Capacitor 4u7 4u7 4u4 F Capacitor 2n2 2n2 2n33 F Table 4.2: Values of the components required and used in the electronic circuits for the radar 4.4 Software The software is built using Eclipse Neon.3 (4.6.3) in the Java 1.8 programming language. In this research this application has run on an Apple macbook Pro Late-2013 running macos The software used for Android is written in C, using Arduino Communication To let the microcontroller communicate with the java application a serial connection is opened between the devices. The baud rate of the serial port is Baud. A simple communication protocol has been built to make sure the data could be easily transmitted and received by both the Arduino and the java application. This protocol consists on sending for each piece of data a newline with a capital letter (here called identifier ), and an integer. When the microcontroller sends the data that an object is detected at an angle of 70 degrees at a distance of 10 meters, which is typical data for the LiDAR circuit, it transmits the following data over the serial connection: 1 \ l s t s e t { numbers=none } D1000, A70 For the radar circuit a velocity and, if the radar supports it, a direction is sent. In the following example the data is shown for an object that is detected at a velocity of 3,9 km/h which is coming towards the sensor. \ l s t s e t { numbers=none } 2 V390, R1 When the radar does not support direction, it won t send any data with the identifier D. The java application handles every serial line and executes certain code according to the identifier. An overview of the identifiers and value interpretations is given in table 4.3. Identifier Meaning Unit of additional Integer Technology D Distance cm LiDAR A Angle Degree Servo V Velocity 0.1 km/h Radar R Direction boolean Radar* Table 4.3: Communication protocol between Arduino and Java application This is a very easy and understandable communication protocol, which is is also in itself very easy to read Visualization When the data from the microcontroller is received, it is also important to show this data on the screen of the device. A simple screen for the LiDAR has been created which plots the angles 30

31 and distances on the screen. A picture of this visualization is shown in figure 4.9. This picture shows the visualization on a laptop, but this can also be shown on a smartphone. The length of the black lines indicate the distance in that specific angle. This means that when someone is standing 1 meter in front of the sensor, a white gap will be seen, since at that place the black lines are very short (indicating 1 meter). When the LiDAR does not measure anything means that there is no object. Then the sensor returns 0 as value, but this plot then show the maximum range, which is 40 meters. Figure 4.9: Visualization of the LiDAR prototype The visualization for the radar system is more straightforward. This visualization, which can be seen in figure 4.10, shows a velocity and a direction in text on the screen. For a final product this would not be sufficient, since this is not clear for an average elderly cyclists. However, according to the requirements this is sufficient for this research. 31

32 Figure 4.10: Visualization of the radar prototypes. For the HB100 the direction data is not shown. 4.5 Smartphone To test if a smartphone is capable of showing the sensor measurements towards cyclist it is necessary to connect the sensors to a smartphone or tablet. This is done using a USB connection (A micro-usb to USB-A converter has been used). Tablets running Windows are capable of running Java applications, and are therefore able to immediately run the laptop-version of the initial sensor-testing application. For this research initially an windows tablet will be used. The tablet that will be used for this research is the Dell Venue 8 Pro. It is expected that the performance of a tablet running a java application on Windows would be worse as compared to native code running on a Unix-based device, such as an Android or ios device. However, it is assumed that when the application would work using Java on windows, it would work even better on Android or ios devices. This being said, when the performance of the windows tablet appears to be insufficient, a part of the code will be rewritten to run natively on a high-end Android smartphone to further test if it would be possible to detect road users in traffic. 32

33 CHAPTER 5 RESULTS In this chapter it is described how the developed setup is performing. For multiple dangerous scenario s in traffic for elderly cyclists, a test has been designed in which the sensor needs to detect a person. For all tests and all sensors it is tried to detect a person. Detecting cars and bicycles, or small poles would be very interesting but due to time-constraints this will not be taken into account in this research. 5.1 Testing the sensors To answer the research questions it is very important that the three chosen sensors will be tested. In this research the capability of the sensors to measure obstacles around them is tested. This means that range of the sensors will be tested, along with their beam divergence. The tests have been divided into thee categories: Static sensor testing, dynamic sensor testing, and smartphone testing Static sensor testing The first tests will take place when the sensor is not moving. The sensor will be put on a height of 80cm, which is representative for the height of the place of a bicycle frame where such a system could be connected. A picture of this sensor setup is shown in figure 5.1. All sensors will be put on top of the box and they will all be tested under the same circumstances. Figure 5.1: Picture of the radar setup for static testing 33

34 Static test 1: The middle of the beam The first test is that the person will walk in straight in front of the sensor; 30 meters away from the sensor and then 30 meters back. This should be the test where the radar sensors would detect the person very well. This is because of the specified beam divergence of the sensor. As an illustration the beam divergence information from the datasheet of the HB100 is shown in figure 5.2. The second test is that the person will do exactly the same as in the first test, but then running instead of walking. A diagram of this test is shown in figure 5.3. This test is representative for detecting other road-users on the same lane as the cyclist. Walking and running from the sensor here indicates another road-user driving away from the sensor, and by walking and running back towards the sensor gives a clear indication of what the data will look like if a head-on collision occurs. Static test 2: 5 meter to the side of the beam The third and fourth test are comparable to the first and second test. The only difference here is the route of the person that will be detected. The person now walks 5 meters on the side of the sensor, 30 meters front and back. According to figure 5.2 it is expected that the sensor will pick up the movement as good as for the first static test, since the azimuth angle for the HB100 is very wide. (For the IPS-265 this also holds, since it has a similar azimuth angle, this can be seen in the Appendix). This test can be seen as detecting other road-users that drive on the side, such as cars on the road to the side of the cyclist. It is important to compare the data of this test with the first test, where a head-on collision occurs. There should be a difference in the data, otherwise it cannot be detected if someone just drives on the line beside you, or is driving straightly towards you. A visualization of the test setup is given in figure 5.4. Figure 5.2: The azimuth and elevation angle from HB100" Radar sensor, according to the datasheet of the sensor (see appendix) 34

35 Figure 5.3: Top-view schematic of static sensor test setup 1. The numbers indicate sequential actions in the same test. Figure 5.4: Top-view schematic of static sensor test setup 2. The numbers indicate sequential actions in the same test. 35

36 5.1.2 Dynamic sensor testing The second series of testing takes place when the sensor is moving. Since the sensors will eventually be used on a bicycle which is also moving. This will be achieved by putting the sensors on top of a small cart that can drive around. This chart will be moved and steered by a person. This person will walk a specified distance, which depends on the test. Besides, this person will keep track of the time it takes to walk this distance. From this data an average speed can be calculated. The chart used for this research is shown in figure 5.5. In this figure it can be seen that the sensors are connected to the front of the chart. The chart is a little more high as compared to the box used in the static tests, which means that the sensors are now placed at a height of 120 cm. Figure 5.5: A picture of the chart which is used to move the sensors for the dynamic tests. The expectation for the dynamic tests is that because the sensor is moving, the data is probably more difficult to interpret, and there might be more noise. Dynamic test 1: Road-user coming up from the front One of the things the system should detect are road users or obstacles coming up from the front. In this test setup the cart and the person walk towards each other. They start with a distance of 30 meters between them and at a certain point they walk past each other at a short distance. A schematic of this test is shown in figure 5.6. Dynamic test 2: Being overtaken Elderly state that they do not feel safe when they are suddenly overtaken. Therefore it would be useful to let the system detect other traffic users from behind. In this test the cart is driven backwards, starting at 5 meters in front of the person that needs to be detected. The cart will be driven backwards in the person will walk faster than the cart is moving, which results in the person overtaking the cart. A schematic of this test is shown in figure

37 Figure 5.6: Top-view schematic of dynamic sensor test setup 1. The numbers indicate sequential actions in the same test. Figure 5.7: Top-view schematic of dynamic sensor test setup 2. The numbers indicate sequential actions in the same test. 37

38 Dynamic test 3: Road-user coming up from the side In this test the cart will move 10 meters to the front, while a person will move 10 meters from the right, till both cart and person end at approximately the same location. This is representative as a collision from the side, for example when another road user is coming from the right. This corresponds with one of the more dangerous situations for elderly in traffic, for example on two-way cycle paths or blind spots. Although this is a situation in which the cyclist will probably see the other road-user coming, it is still good that the system can detect such situations. A schematic of this test is shown in figure 5.8. Figure 5.8: Top-view schematic of dynamic sensor test setup 3. The numbers indicate sequential actions in the same test Testing a smartphone In this research it will also be tested if the prototype can run on a smartphone. In this research the Dell Venue 8 Pro tablet is used to run the system. Because in this prototype a USBconnection with the tablet is used, it might be possible that the tablet can power the electronics for the sensors. To test the effect of the prototype on the tablet there will be looked to a few things. At first there will be checked if the tablet provides enough power. When that works there can be looked at battery drainage. If the tablet is not able to provide enough power then it can be concluded that the prototype draws too much power for the tablet alone, and the circuit will be connected with an extra power source. Secondly there will be looked at the CPU load (the amount of processing which is needed by the application) and if the program is running smoothly on the tablet. When the program appears to be requiring too much processing from the CPU this is probably noticeable in the overall working of the tablet. Also the program itself should run smoothly on the tablet. For clarification a flow-chart diagram of this test is shown in figure

39 Figure 5.9: Flow chart of how the tablet performance will be tested. When the tablet itself can power the prototype, then the battery drainage will be measured. This is done by charging the battery of the tablet to 100 percent, and then turning on the prototype for so long till the battery of the tablet is empty, or until the moment the tablet does not provide enough power anymore to power the prototype. This scenario is possible because the power of a full battery is higher as compared to a low battery. The time until such a situation occurs will be measured. While doing this test it the conditions are adapted to fit the purpose of the final product: A screen that displays information from the sensors. This means that the screen is always on, with a brightness of 50 percent. Other applications besides the task manager are closed. To test the influence of the system on the tablet, another test with the tablet will be conducted where the screen is also on 50 percent brightness, but without powering the prototype, and without running the application. In the end the time until the battery is empty can be compared. 5.2 Sensor accuracy According to the stationary test setups an indication can be made about how well the sensors have performed. All sensors have been tested according to the described test setups and the output of the sensors is captured Lidar Lite V3 Static test results In the static tests the Lidar Lite V3 performed good. When first looking at the data it appears that the output data from the LiDAR is difficult to read, but when understanding this data it becomes understandable what is going on. In figure 5.10 a lot of different measurement points are shown. The constant U-shape at the bottom of the graph is the wall on the right of the sensor. This is an U-shape because the sensor itself is rotating on a servo. In figure 5.11 the view of the sensor can be seen. From this picture it can be derived what the measured points in the lidar chart are: the measured points at meters are the bush in front of the tree, and the measured points at 28 meters are a part of the bushes on the left of the picture. This measurement repeats, but one thing changes: the location of the person. This explains the diagonal lines in the graph. The Lidar Lite V3 proves to be very accurate until a distance of 20 meters. After 20 meters the diagonal lines are not (completely) visible anymore, meaning that the person is not detected after that. The measurement of a distance can go up to 40 meters, which resonates with the datasheet. Unfortunately it is not possible to detect a person at such a large range. Also, the environment around the Lidar Lite V3 is static, which means that nothing of it moves. In theory 39

40 this environment can be filtered out, such that only the moving points remain. However, every time the Lidar is rotated forth an back, the results are a little different. (Look for example to the points at 40 meters, which are constantly a little bit different). Because of these small inaccuracies it might be difficult to filter out the environment. Figure 5.10: Walking 30 meters from and to the Lidar Lite V3. The movement pattern is visible in the diagonal lines Figure 5.11: The view of the sensor in the static test setup 40

41 Dynamic test results In the dynamic results the problem of the surroundings become much worse. The LiDAR detects the distance to all objects, which means also the static objects. In the static tests the wall was clearly visible in the graph. In the dynamic tests this is also the case, however, the signal resulting from measuring the wall is constantly different when the sensor is moving. In some cases this leads to unreadable data, as can be seen in figure While doing this test the chart was not moving precisely parallel to the wall and that can be seen very clearly, since the u-shapes (which are usually walls) are changing of position in the chart. Besides this there is no pattern to be found of a person walking straight in front of the sensor. In this environment the Lidar Lite V3 is not usable to detect persons coming up from the front. The same phenomenon has occurred in the test where the sensor detects a person coming up from the back. The corresponding chart can be found in the Appendix, figure Figure 5.12: Results of the Lidar Lite V3 in the first dynamic test where a person and the sensor on a chart walk towards each other But the Lidar Lite V3 suddenly performs very well in the third dynamic test, where a person comes up from the side. The test was performed at a distance of 5 meters instead of 10 meters the surroundings did not allow for a larger distance. In figure 5.13 it is clearly visible that the object at 5 meters is colliding. Also the measured distance appears to be very correct. Figure 5.13: Results of the Lidar Lite V3 in the third dynamic test where a person comes from the side IPS-265 The IPS-265 is able to measure both velocity and direction. In the charts for the IPS-265 this direction is calculated with the velocity. When the direction of the movement is towards the sensor, the velocity will be multiplied by -1, which in the charts is thus shown as negative velocities. These negative velocities mean a velocity towards the sensor, while positive velocities mean a velocity moving away from the sensor. 41

42 Static test results The results of the first static test (walking) can be seen in figure While walking away from the sensor it has measured velocities between 0.3 and 5.6 km/h, but after only a few seconds it has stopped detecting the user (at 12 seconds). From the video of this test it appears that this is at approximately 5 meters from the sensor, which means that the IPS-265 practically has a range of only 5 meters. This is also seen when walking back, since the sensor detects the user again at 54 seconds, which is very late, and also indicating a range of 5 meters. This is much too short for a detection system on a bicycle. At 59 seconds there is a final measurement which is probably something moving while turning of the system. There will be some more elaboration on that in the next results. Also while running (appendix, figure 7.11) the measurements are short and at the way back the person is almost not detected. Besides, the measured velocities are much lower than running speed. However, the IPS-265 is also capable of measuring direction, and that system appear to work quite well. In figure 5.14 it can be seen that the direction while walking from the sensor are perfect, and the measurements while walking back towards the sensor are also very good. From the first static test it can be seen that while walking and running in front of the IPS-265 results in poor velocity measurements and a bad range, but the direction of the object is tracked well. Unfortunately the system almost does not detect the obstacle, meaning that the direction measurements are not that useful. Figure 5.14: Velocity graph of the first static test for the IPS-265 Dynamic test results In the dynamic tests the IPS-265 performed even worse. The velocity measurements and the direction measurements are not very reliable. Because the sensor itself is moving it picks up a lot of different directions, sometimes switching rapidly between directions and velocities. This can be clearly seen in the first dynamic test, which is shown in figure Here it is very unclear that something is moving towards the sensor. Also there are some velocities measured over 20 km/h, which is incorrect. 42

43 Figure 5.15: Direction graph of the first dynamic test of the IPS-265 (person coming and sensor moving towards each other) An interesting thing to note for the IPS-265 is the direction measurement in the second dynamic test. This can be seen in figure 5.16, while focusing on the direction of the measurements (i.e. whether the velocities are positive or negative). The sensor constantly detects something moving towards the sensor, and exactly at the moment where the person passes the sensor it detects that the person is moving away from the sensor again. This corresponds with the timing and the sensor picked this up very well. Unfortunately, after that the sensor started detecting movement towards the sensor again, while only the sensor itself was moving backward. The expectation was that it would detect objects moving away from the sensor instead. In the third dynamic test the same phenomenon occurs (see appendix, figure When the person is walking towards the sensor this is detected very well, but when only the sensor is moving and there is no other moving object, these direction values cannot be relied on. 43

44 Figure 5.16: Direction graph of the second dynamic test of the IPS-265 (person coming and sensor moving towards each other) 44

45 5.2.3 HB100 The HB100 has better performance on velocity and range as compared to the IPS-265. There are more measurements and an object can be detected at longer distances. Static test results The expectation for the HB100 was low, since it is only able to detect velocities and cannot be used for distance measurement or even direction measurement. However, it appears that the HB100 performs better at measuring velocities than the IPS-265. In figure 5.17 the same phenomenon as in the IPS-265 can be seen. However, when this image is compared to the image of the IPS-265 (figure 5.14) it appears that the range of the HB100 is larger than the range of the IPS-265, and also more data points are shown. Figure 5.17: Chart of the test results of the HB100 on the first static test, while walking The static measurement on the side of the signal of the HB100 are, unfortunately, not very good. Detecting a road-user in another lane would be problematic since there are almost no measurements, such as seen in figure

46 Figure 5.18: Chart of the test results of the HB100 on the second static test, while walking Dynamic test results The data from the dynamic tests of the HB100 is very unreadable. Which can for example be seen in figure There are multiple strange velocities (some over 500 km/h) picked up by the sensor. It would require further research to find out what is happening at the points below 50 km/h. It might be possible that some useful points may be found then. Figure 5.19: data of the first dynamic test of the HB100 (sensor and person moving towards each other) 46

47 5.3 Tablet results In this research also the tablet has been tested. When connecting the servo and Lidar Lite V3 to the system it appears that the tablet could provide enough power to run the prototype. This means that the application could be started and the tablet would run for a few hours until the battery was empty. Occasionally the battery level and time have been noted, which resulted in the data shown in figure In this figure can be seen that the time in which the battery is empty is almost 5 hours shorter when the LiDAR prototype is attached. After about 270 minutes, which is 4.5 hours, the battery was completely empty. 4.5 hours is long enough, since most people do not cycle that long. But most people do use their smartphone a whole day and for multiple occasions, and in that case it is not desirable for them to have such a batteryconsuming system on their bicycle. Figure 5.20: Battery level over time of the system connected to the Lidar Lite V3 and servo The CPU pressure of the application was constantly between 37 and 38 percent of it s total capacity. This is a CPU load comparable with browsing the web, and is not extremely high. This also explains why the application is running very smoothly: the tablet can handle it easily. This would of course be a larger problem when difficult algorithms are needed. But for now such algorithms are not necessary and from these results it appears that a smartphone is very well capable of displaying the sensor data. This also means that, when an obstacle/road-user can be detected in sensor data, that the smartphone can show this data. 47

48 48

49 CHAPTER 6 DISCUSSION From the results in the previous chapter some conclusions can be made. This research was about increasing the safety of elderly people by using road-user-detection technology. For this the current situation of elderly cyclist has been researched, and based on those results a proposal for a system has been developed and tested. In this chapter the research questions posed in the beginning of this report are being answered. For clarification the research (sub)questions are the following: 1. What are dangerous situations for elderly cyclists?" 2. What limitations do elderly cyclists have in traffic?" 3. What sensors are able to detect road users in traffic?" 4. Can a smartphone be used to detect road users in traffic?" 6.1 Elderly cyclists In the introduction it was stated that a lot of elderly cyclist suffer from accidents in traffic, because they are more vulnerable and it is difficult for these people to keep up with the fast pace of traffic. In a literature search for the state-of-the-art it is found that obstacles, two-way cycle paths, vehicles taking over quickly and blind spots are the dangerous situations. A technical solution might fit on some of these situations to aid cyclists in their problems in traffic. Also, a technical solution might help elderly cyclist with their general limitations, which are decreased sight, decreased hearing and more trouble keeping up with everything that happens in traffic. 6.2 Sensors Lidar Lite V3 From the three tested sensors the Lidar Lite V3 appears to be the most promising sensor to be used on a bicycle, due to the high range and high sample rate. The Lidar Lite V3 is apparently very capable of detecting and measuring other road users. But still this sensor is not easy to use for the desired application, since it needs a servo to increase the detection range, and the data is difficult to interpret. Although the maximum range is 40 meters, the measurements at 40 meters are not very trustworthy. Most measurements are starting to get useful at around 30 meters. This is still a very good distance for obstacle detection on bicycles. When the sensor itself is also moving (when the bicycle moves for example), the data becomes extremely difficult to interpret and some smart computer algorithms are needed. Besides that, it is difficult and relatively inefficient to calculate the speed from the LiDAR data. Lastly, there is a lot of surroundings that need to be filtered out by some algorithm, and it appears that even the Lidar Lite V3, with it s high accuracy, is not accurate enough for that. Another setback of the Lidar Lite V3 is that it needs to be put on top of a servo to be used as a detection system on bicycles. Probably (elderly) cyclists do not want to use a system with clearly moving parts. Also, a servo system is very vulnerable for the rough environments a bicycle usually can be in. As a note: the whole construction of a LiDAR upon a servo appears to be quite fragile. 49

50 IPS-265 The IPS-265 behaved poorly and especially the range and velocity measurements where much lower than expected. Because of this the IPS-265 in itself is not suitable to detect road-users or objects in traffic. However, the direction measurements from the IPS-265 were surprisingly good and a direction measurement on a bicycle can be a very nice addition to the sensor. Another sensor with direction measurement might do the trick, but not the IPS-265. HB100 The HB100 is tested mainly because the very low pricing of the sensor. With the low price of only 5 dollars this sensor performed very well compared to the IPS-265, returning more velocity measurements and having a larger range of about 10 meters. Still the HB100 is, just as the IPS-265, not in itself fit for detecting road-users or obstacles. This is mainly because it is not able to measure direction, and the direction measurements as seen at the IPS-265 appears to be very useful to detect if some other road-user is coming up. Also the dynamic measurements from the HB100 proved that this sensor is not useful for the purpose of detecting road users or obstacles in traffic. Detecting road users in traffic The answer to the third research question according to the tests with the Lidar Lite V3, the IPS-265 and the HB100 is that only the Lidar Lite V3 can really be used to detect road users in traffic. Still it is not recommended to use this sensor, since it is fragile due to the fact that it moves on a servo. However, the LiDAR technology is very promising and for future research for a system described in this project it is recommended to look into other LiDAR sensors to improve this system. 6.3 Tablet results The tablet used in this research showed that is was perfectly capable of running the complete system for 4 hours straight. This is a relatively long period of time, since most people don t cycle for such distances. However, at the end of the day it might become problematic when only a small amount of battery is left in the tablet or smartphone. However, in general the tablet is capable of retrieving and displaying the sensor data, making it useful in an assistance-system for elderly cyclists. The tablet also had enough processing power to run the application, and according to the CPU there is room for even more processing, for example running algorithms to make the system even smarter. The results from the tests with the tablet indicate that it is definitely possible to use a smartphone for the system designed in this research. 6.4 Recommendations The dangerous situations for elderly cyclists in traffic allow a technical system to aid these people in traffic. Sensors on a bicycle are able to detect obstacles and other road users around the cyclist, so this is a good concept that has potential for the future. Unfortunately, for now this system is not yet feasible, or at least, it is not feasible in the standard price range for bicycles, since high-end radars and LiDARs are still expensive. Because the radar and LiDAR systems both have completely different workings the sensors from the tests in this research could be combined. However, this gives even more data making all signals even more difficult to process. Still a combination of Radar and LiDAR might give a good overview of the distance towards an object, and the velocity and direction of that object. It would then be important to use a better radar to compare for the range, since the range of the radars was significantly shorter than the range of the Lidar Lite V3. Besides, it might be good to do more research about the very promising FMCW radars, which have not been tested in this research. Especially a high-end FMCW radar might give very good results. 50

51 For the future it appears that LiDAR is a promising solution. However, a system that functions as an assistant on a bicycle requires more data of the surroundings. A good solution might be to investigate these possibilities in a high-end LiDAR. These high-end LiDARs are currently in production, but these sensors are still very expensive. For the future it is expected that these prices will drop significantly, since these sensors will be developed for the automotive industry. This makes these sensors interesting for an application on bicycles. But, with more processing the usage of a smartphone might become increasingly difficult. For retrieving and showing the raw data a smartphone is suitable, but for the use of heavy algorithms the smartphone has not been tested. 6.5 Conclusion In this project a proposal and a proof-of-concept is made for a system that can aid elderly cyclists in traffic using road-user-detection technology. This system works for multiple dangerous situations that often occur with elderly cyclists. This means that it is possible to use road-userdetection technology to detect other road users on a bicycle, and it is also possible to detect obstacles, which are also a major cause for traffic accidents among elderly cyclists. The data from the sensors could be used to aid the user in traffic. Using some low priced sensors from different types, it appears that using a LiDAR a person or obstacle can be detected at a distance of approximately 30 meters. However, this data is difficult to read and there is a lot of noise that needs to be filtered out. A high-end LiDAR might give more overview in the data and has a higher precision. A radar can be used to determine the direction and velocity of an object, but the accuracy and range in low price ranges are not reliable. As with the LiDAR it is better to use a high-end FMCW radar to detect other road-users and obstacles in traffic. Detecting around a cyclist in traffic is a very difficult task, and in traffic a lot of different situations occur and these situations are often happening very fast. Creating a system that can warn a cyclist in dangerous traffic situations is feasible. This can be said since the automotive industry is currently making large progress. For bicycles it is still a little early to start using these technologies, since the price range on a bicycle is significantly lower. In the upcoming years the automotive industry will develop further and while the current low-price sensors are not yet good enough for this application, in a few years they might. For now it is best for elderly cyclists to keep their eyes on the road, so maybe they can experience this system fully functional in the future. 51

52 52

53 REFERENCES [1] CONEBI, European bicycle market 2016 edition, [2] J. Vanparijs, L. I. Panis, R. Meeusen, and B. de Geus, Exposure measurement in bicycle safety analysis: A review of the literature, Accident Analysis and Prevention, vol. 84, pp. 9 19, [Online]. Available: S [3] J. Choudrie, S. Pheeraphuttharangkoon, E. Zamani, and G. Giaglis, Investigating the adoption and use of smartphones in the uk: a silver-surfers perspective, [4] W. Maring and I. Van Schagen, Age dependence of attitudes and knowledge in cyclists, Accident Analysis and Prevention, vol. 22, no. 2, pp , [5] C. Engbers, F. Westerhuis, R. Dubbeldam, and D. de Waard, How do older adults experience interaction at different traffic situations? a questionnaire study, [6] F. Westerhuis, C. Engbers, R. Dubbeldam, and D. de Waard, What do older cyclists experience? an identification study of perceived difficulties in everyday cycling interactions, [7] J. J. de Hartog, H. Boogaard, H. Nijland, and G. Hoek, Do the health benefits of cycling outweigh the risks? Environmental Health Perspectives, vol. 118, no. 8, pp , [Online]. Available: [8] S. de Hair, C. Engbers, R. dubbeldam, T. Zeegers, and H. Liers, A better understanding of single cycle accidents of elderly cyclists, Berichte der Bundesanstalt fuer Strassenwesen. Unterreihe Fahrzeugtechnik, no. 102, p. 11, [9] C. Engbers, F. Westerhuis, R. Dubbeldam, and D. de Waard, How do older adults experience interaction with other road users? a systematic literature research, [10] C. Engbers, A. boelhouwer, F. Westerhuis, R. Dubbeldam, and D. de Waard, Zien en gezien worden. rapportage focusgroepen intelligent verlichtingssysteem en persoonlijke adviseur, [11] TNO, Veilig en bewust op de fiets 2 realisatie van een veiligheidsbevorderend feedforwardsysteem voor op de fiets, TNO, Tech. Rep., [12] [Online]. Available: [13] [Online]. Available: [14] [Online]. Available: [15] [Online]. Available: Byxee.com [16] [Online]. Available: AE-cykelvarning [17] [Online]. Available: [18] R. Rasshofer and K. Gresser, Automotive radar and lidar systems for next generation driver assistance functions, Advances in Radio Science, vol. 3, no. B. 4, pp , [19] J. Shackleton, B. VanVoorst, and J. Hesch, Tracking people with a 360-degree lidar, in Proceedings of the th IEEE International Conference on Advanced Video and Signal Based Surveillance, ser. AVSS 10. Washington, DC, USA: IEEE Computer Society, 2010, pp [Online]. Available: 53

54 [20] [Online]. Available: tesla-again-spurns-lidar-betting-instead-on-radar [21] [Online]. Available: velodyne-announces-breakthrough-in-solid-state-lidar-design [22] D. Gohring, M. Wang, M. Schnurmacher, and T. Ganjineh, Radar/lidar sensor fusion for car-following on highways, The 5th International Conference on Automation, Robotics and Applications, pp , [23] F. de Ponte Müller, Survey on ranging sensors and cooperative techniques for relative positioning of vehicles, Sensors, vol. 17, no. 2, p. 271, [24] [Online]. Available: psyclesafe-bicycle-safety-warning-system/comments [25] [Online]. Available: [26] [Online]. Available: 54

55 CHAPTER 7 APPENDIX 7.1 Code for the prototypes The code for the several prototypes can be found on GitHub. The java application can be found under the following link. There are four branches, each containing a version of the program where only the configuration options are different. One branch for the LiDAR on laptop, one for the LiDAR on the tablet, one for Radar on laptop and one for radar on the tablet. The arduino code for this application can be found under the following link: For testing purposes and the plots in the development chapter the GraphWriter Arduino sketch has been used Datasheets for the sensors AgilSense HB100 Innosent IPS rev1.3.pdf Garmin Lidar Lite V3 pdf 55

56 7.3 IPS-265 sketch Figure 7.1: Fritzing sketch of the amplification/filtering circuit for the IPS

57 7.4 HB100 sketch Figure 7.2: Fritzing sketch of the amplification/filtering circuit for the HB HB100 test results Figure 7.3: Results of the static test 1 (walking) for the HB100 57

58 Figure 7.4: Results of the static test 1 (running) for the HB100 Figure 7.5: Results of the static test 2 (walking) for the HB100 Figure 7.6: Results of the static test 2 (running) for the HB100 58

59 Figure 7.7: Results of the dynamic test 1 for the HB100 Figure 7.8: Results of the dynamic test 2 for the HB100 Figure 7.9: Results of the dynamic test 3 for the HB100 59

60 7.6 IPS-265 test results Figure 7.10: Velocity results of the static test 1 (walking) for the IPS-265 Figure 7.11: Velocity results of the static test 1 (running) for the IPS-265 Figure 7.12: Velocity results of the static test 2 (walking) for the IPS

61 Figure 7.13: Velocity results of the static test 2 (running) for the IPS-265 Figure 7.14: Velocity results of the dynamic test 1 for the IPS-265 Figure 7.15: Velocity results of the dynamic test 2 for the IPS

62 Figure 7.16: Velocity results of the dynamic test 3 for the IPS Lidar Lite V3 test results Figure 7.17: Distance results of static test 1 (walking) for the Lidar Lite V3 Figure 7.18: Distance results of static test 1 (running) for the Lidar Lite V3 62

63 Figure 7.19: Distance results of static test 2 (walking) for the Lidar Lite V3 Figure 7.20: Distance results of static test 2 (running) for the Lidar Lite V3 Figure 7.21: Distance results of dynamic test 1 for the Lidar Lite V3 Figure 7.22: Distance results of dynamic test 1 for the Lidar Lite V3 63

64 Figure 7.23: Distance results of dynamic test 1 for the Lidar Lite V3 64