Multi-Sensor Integration and Fusion using PSoC

Transcription

1 Multi-Sensor Integration and Fusion using PSoC M.S. FINAL PROJECT REPORT Submitted by Student Name Master of Science in Electrical and Computer Engineering The Ohio State University, Columbus Under the Guidance of Dr. Lisa Fiorentini Assistant Professor, Clinical Department of Electrical and Computer Engineering The Ohio State University, Columbus i

2 TABLE OF CONTENTS ABSTRACT... 1 INTRODUCTION... 2 PROGRAMMABLE SYSTEM-ON-CHIP (PSoC)... 2 MULTI-SENSOR SYSTEM ARCHITECTURE... 5 ENCODERS... 7 TEMPERATURE SENSOR... 8 ACCELEROMETER MAGNETOMETER GPS REAL TIME CLOCK (RTC) DATA LOGGING IN F-RAM COMMUNICATION WITH RASPBERRY PI LIDAR REFERENCES ii

3 LIST OF TABLES Table 1: List of Hardware Components Used... 5 Table 2: Encoder Color Code Description... 6 Table 3: PSoC 4200M Pin Connections... 6 Table 4: Sensor Characters, Table 5: Special Characters LIST OF FIGURES Figure 1: High Level Block Diagram of PSoC 4200M... 3 Figure 2: CY8CKIT-044 PSoC 4 M-Series Pioneer Kit... 3 Figure 3: TopDesign Schematics of PSoC Creator Project... 4 Figure 4: Design Wide Resources of PSoC Creator Project... 5 Figure 5: A and B Channel Encoder Output Waveforms... 7 Figure 6: Quadrature Decoder Component Configuration... 7 Figure 7: TMP102 Internal Register Structure... 8 Figure 8: I2C Component Configuration... 9 Figure 9: I2C Timing Diagram to Read data from TMP Figure 10: I2C Timing Diagram for ADXL Figure 11: Internal Schematic diagram of HMC5883L Figure 12: Raw NMEA Sentences transmitted by the GPS Figure 13: RTC Component Configuration Figure 14: Timing Diagram to write data into F-RAM Figure 15: Timing Diagram to read data from F-RAM Figure 16: Bridge Control Panel to read data from F-RAM Figure 17: UART Component Configuration Figure 18: Example communication packets between Raspberry Pi and PSoC Figure 19: PSoC Firmware Flowchart for communication and unparsing Figure 20: LIDAR Lite Block Diagram Figure 21: PSoC 4 I2C Component Configuration for LIDAR iii

4 ABSTRACT Multi-Sensor Integration and Fusion subsystem is a part of the multidisciplinary research project Unmanned Ground Vehicle (UGV) and Aerial (UAV) Vehicle Swarms. In order for autonomous navigation, path planning and target identification of the autonomous vehicles, various sensors measurements are required. A PSoC 4 (Programmable System-on-Chip) from Cypress Semiconductors was used to interface various sensors. The PSoC 4 is based on ARM Cortex M0 architecture along with the integration of programmable analog and digital blocks. The PSoC 4 acts as a coprocessor collecting all the sensor data in real time and communicating it to Raspberry Pi which performs high level controls based on the sensor measurements. The sensor data is also logged in an external memory, a 256 K byte F-RAM available on the PSoC 4 Development Kit. A specific communication scheme was also developed on top of UART protocol between the PSoC 4 and Raspberry Pi to transfer specific sensor readings requested by Raspberry Pi. Multiple sensors like Motor Encoders, IMU, GPS, Temperature Sensor and LIDAR were successfully interfaced with PSoC 4. The Motor Encoders consists of hall effect sensors which produces a pulse every time the motor rotates, the counters in PSoC 4 are used to count these pulses to determine the rpm of the motor and hence the wheel. The IMU consists of a combination of Accelerometer and Magnetometer to measure the linear acceleration and orientation of the robot with respect to the Earth s Magnetic field essentially a compass pointing towards the north direction. The temperature sensor is used to measure the ambient temperature. GPS gives the position of the robot in terms of longitude and latitude. In addition to the position, the speed of the robot can also be measured and the UTC time from the GPS is used to lock the RTC time of PSoC 4 for the time stamp of the sensor readings. LIDAR is used to detect obstacles in front of the robot. 1

5 INTRODUCTION Multi-Sensor integration refers to the process of using of multiple sensors to obtain more accurate and reliable information regarding the system and its environment. Multi-Sensor fusion refers to the process of combining the information from various sensors and representing it in a common format understandable to the main processor which will take decisions based on the sensor data. The acquisition of sensor data must be done in real-time in order to make the decisions at the right time. The time at which the each sensor data is acquired must also be included in the data format to be sent to the main processor. In addition to sending the sensor data to the main processor, it must also be logged in an external memory. In this project, the main processor is a Raspberry Pi 2 and a PSoC 4200M performs the multisensor integration and fusion in real-time and sends the fused sensor data to Raspberry Pi. The sensors currently integrated with the PSoC 4200M are, 1. Motor Encoders 2. Temperature Sensor 3. Accelerometer 4. Magnetometer 5. GPS 6. LIDAR PROGRAMMABLE SYSTEM-ON-CHIP (PSoC) The PSoC 4200M device from Cypress Semiconductors is a mixed signal system on chip based on 32-bit Arm Cortex M0 architecture with programmable analog and digital blocks as depicted in the high level block diagram in Figure 1. Multiple sensors, both analog as well as digital sensors can be interfaced with PSoC 4200M to acquire sensor data in real-time. The programmable analog and digital blocks operate independently along with the CPU of the PSoC [1]. This enables acquisition of data from multiple sensors, formatting of sensor data and communication with the main processor in parallel. The PSoC 4200M device also contains a Real Time Clock (RTC) which is used to time stamp the sensor data. 2

6 PSoC 4200M Sensor Inputs Programmable Digital Blocks ARM Cortex M0 Sensor Data to Raspberry Pi Programmable Analog Blocks RTC Figure 1: High Level Block Diagram of PSoC 4200M CY8CKIT-044 PSoC 4 M-Series Pioneer Kit shown in Figure 2 features a PSoC 4200M device which is used in this project to interface with the various sensors and the Raspberry Pi. This development kit also contains an on-board programmer and debugger and therefore no additional hardware is required to program and debug the PSoC 4200M device [2]. Figure 2: CY8CKIT-044 PSoC 4 M-Series Pioneer Kit 3

7 The firmware for the PSoC 4200M device was developed using the PSoC Creator IDE (Version 3.2 SP1). The PSoC Creator contain PSoC components which are virtual ICs which users can drag and drop into a design and configure them to meet the application requirements. Every PSoC component comes with its own set of API libraries [3]. PSoC Creator is free to use and can be downloaded from the following link, Figure 3 shows the top design schematics of the PSoC Creator developed for Multi-Sensor Integration and Fusion project. Figure 3: TopDesign Schematics of PSoC Creator Project Figure 4 shows the design wide resources of the PSoC Creator developed for Multi-Sensor Integration and Fusion project. 4

8 Figure 4: Design Wide Resources of PSoC Creator Project MULTI-SENSOR SYSTEM ARCHITECTURE The Multi-Sensor system architecture with the interconnections between the multiple sensors and PSoC 4200M development kit is shown in figure 5. The Temperature sensor and the Accelerometer both use I2C interface and are connected in the same I2C bus with the PSoC 4200M. The PSoC 4200M identifies the respective sensor with its unique I2C address. Table 1 lists all the hardware components used in the Multi-Sensor system architecture along with their web links. The TMP102, ADXL345 and HMC5883L sensor breakout boards from SparkFun are used in this project. Table 1: List of Hardware Components Used Component Description Web link CY8CKIT-044 PSoC 4200M Development Kit Raspberry Pi 2 Main Processor Encoders 48 CPR Quadrature Encoder TMP102 Digital Temperature Sensor ADXL345 3-Axis Accelerometer HMC5883L 3-Axis Magnetometer GPS Ultimate GPS module LIDAR LIDAR Lite 5

9 Table 2 describes the color code functionality of connecting wires in the encoder which is obtained from Pololu Robots and Electronics. Table 3 gives the pin connection details between the different components and PSoC 4200M in the Multi-Sensor system architecture. Table 2: Encoder Color Code Description Color Red Black Green Function Motor power (connects to one motor terminal) Motor power (connects to the other motor terminal) Encoder GND Blue Encoder VCC ( V) Yellow White Encoder A Output Encoder B Output Table 3: PSoC 4200M Pin Connections Component Pin PSoC 4200M Pin Encoder 1 Encoder 2 Temperature Sensor Accelerometer Magnetometer GPS LIDAR Raspberry Pi 2 Encoder A Output P2.0 Encoder B Output P2.1 Encoder A Output P2.2 Encoder B Output P2.3 SDA P4.1 SCL P4.0 SDA P4.1 SCL P4.0 SDA P4.1 SCL P4.0 UART TX UART RX P1.1 (UART RX) P1.0 (UART TX) SDA P6.1 SCL P6.0 GPIO 14 (UART TX) GPIO 15 (UART RX) P3.0 (UART RX) P3.1 (UART TX) 6

10 ENCODERS An encoder is an electromechanical device used to measure the position and speed of a motor shaft. The motors used in the robot contain a two-channel Hall-effect sensor encoder which outputs square waves corresponding to the rpm (rotations per minute) of the motor shaft [5]. The outputs of the two channels are 90 degrees out of phase which refers to a quadrature encoder as shown in Figure 5. Figure 5: A and B Channel Encoder Output Waveforms PSoC Creator offers a quadrature decoder component as shown in Figure 6 which is used to acquire data from the two channel encoders. The encoding mode in the component is set to 4x which offers highest resolution by counting both the rising and falling edges of both the square waves to provide the count value output [4a]. Figure 6: Quadrature Decoder Component Configuration 7

11 The Counter_X_ReadCounter() API is used to read the current counter value from the quadrature encoder. The difference between the encoder values is computed by reading the encoder values at two different times with a given time interval which is then formatted and sent to the Raspberry Pi. TEMPERATURE SENSOR The temperature sensor used in this project is TMP102, which is a digital temperature sensor with I2C interface. The TMP102 has a resolution of C, and accuracy of 0.5 C over the temperature range of -25 C to +85 C [6]. Figure 7 shows the internal register configuration of the temperature sensor. To read the current temperature from the sensor, the pointer register must first be initialized to point to the address of the temperature register. After pointing to the temperature register, data can be read from the temperature register. Two bytes of data must be read from the temperature register which corresponds to the MSB and LSB of the temperature reading. Figure 7: TMP102 Internal Register Structure The PSoC 4200M communicates with the TMP102 sensor via I2C interface. The PSoC 4200M is configured as an I2C master with 100 Kbps data rate as shown in Figure 8. To communicate with the slave I2C sensor, the master should know the I2C address of the slave. For TMP102 sensor, the 7-bit I2C address is 0x48 which is provided in the datasheet of the sensor. The I2C timing diagram to read data from TMP102 is shown in Figure 9. 8

12 Figure 8: I2C Component Configuration Figure 9: I2C Timing Diagram to Read data from TMP102 9

13 Following is the sequence of steps followed in the firmware to get the temperature reading: 1. Initialize the 8-bit pointer register to 00 to point to the temperature register using the API, I2C_I2CMasterWriteBuf 2. Read two bytes of data from temperature register using the API, I2C_I2CMasterReadBuf [4b] 3. Combine the MSB and LSB of the temperature reading 4. Shift left the combined data by 4 bits and multiply by to get the current temperature 5. Compute the difference between two temperature readings in a given time interval which is then formatted and sent to the Raspberry Pi ACCELEROMETER The accelerometer used in this project is ADXL345, which is a 3-axis MEMS accelerometer with 13-bit resolution and measurement at up to +/-16 g. Digital output data is formatted as 16-bit twos complement and is accessible through either a SPI (3- or 4-wire) or I2C digital interface. The ADXL345 automatically modulates its power consumption in proportion to its output data rate which can be configured. Registers 0x32 to 0x37 inside the ADXL345 holds the output data for each axis. Two 8-bit registers hold the data for one axis. Register 0x32 and Register 0x33 hold the output data for the x-axis, Register 0x34 and Register 0x35 hold the output data for the y-axis, and Register 0x36 and Register 0x37 hold the output data for the z-axis [7]. The PSoC 4200M communicates with the ADXL345 sensor via I2C interface. The accelerometer is connected to the same I2C bus as the temperature sensor. For ADXL345 sensor, the 7-bit I2C address is 0x53 which is provided in the datasheet of the sensor. The I2C timing diagram for ADXL345 is shown in Figure 10. Figure 10: I2C Timing Diagram for ADXL345 Following is the sequence of steps followed in the firmware to get the accelerometer reading: 1. Initialize the accelerometer by first going into standby mode 10

14 2. Configure the accelerometer in full resolution, 100 Hz data rate, stream mode and measurement mode 3. Read data from two registers corresponding to each axis 4. Combine the data from the two registers for each axis to get the current reading for the respective axis 5. Compute the difference between two accelerometer readings in a given time interval which is then formatted and sent to the Raspberry Pi MAGNETOMETER The magnetometer used in this project is HMC5883L, which is a 3-axis magnetometer from Honeywell. The HMC5883L contains a high-resolution HMC118X series magneto-resistive sensors plus an integrated application specific processor for amplification, automatic degaussing strap drivers, offset cancellation, and a 12-bit ADC that enables 1 to 2 compass heading accuracy. Digital output data is formatted as 16-bit twos complement and is accessible through e I2C digital interface. Registers 0x03 to 0x08 inside the HMC5883L holds the output data for each axis. Two 8-bit registers hold the data for one axis. Register 0x03 and Register 0x04 hold the output data for the x-axis, Register 0x07 and Register 0x08 hold the output data for the y-axis, and Register 0x05 and Register 0x06 hold the output data for the z-axis. The PSoC 4200M communicates with the HMC5883L sensor via I2C interface. The magnetometer is connected to the same I2C bus as the temperature sensor and the accelerometer. For HMC5883L sensor, the 7-bit I2C address is 0x1E which is provided in the datasheet of the sensor [8]. Figure 11 shows the internal schematics of the HMC5883L magnetometer sensor along with the example connection with the I2C Master which in this case is a PSoC 4200M. 11

15 Figure 11: Internal Schematic diagram of HMC5883L Following is the sequence of steps followed in the firmware to get the magnetometer reading: 1. Initialize the magnetometer by first going into standby mode 2. Configure the magnetometer in full resolution, 100 Hz data rate and continuous measurement mode 3. Read data from two registers corresponding to each axis 4. Combine the data from the two registers for each axis to get the current reading for the respective axis 5. Compute the difference between two magnetometer readings in a given time interval which is then formatted and sent to the Raspberry Pi The magnetometer readings are not tilt compensated, however since the accelerometer reading is also available to the main processor which is a Raspberry Pi, it can perform tilt compensation using both the magnetometer and accelerometer reading to perform tilt compensation and calculate the true reading when the magnetometer is not lying flat which is usually the case in off terrain environment. 12

16 GPS The GPS module used in the project is the Ultimate GPS module from Adafruit which is a breakout board with the MTK3339 chipset, which is a high-quality GPS module that can track up to 22 satellites on 66 channels, has an excellent high-sensitivity receiver (-165 db), and a built in antenna. The GPS module also has built in data logging ability and can be powered using a CR1220 coin cell to keep the RTC running. The GPS module also contains an LED which blinks at about 1Hz while it's searching for satellites and blinks once every 15 seconds when a fix is found to conserve power [9]. The GPS module transmits the received messaged through UART interface. The default baud rate of the GPS module is 9600 bps. GPS modules start transmitting data as soon as they are powered on and try to get a 'fix' (location verification). The data transmitted by them is the raw GPS "NMEA sentence" output which contains multiple different kinds of NMEA sentences [10]. The two NMEA sentences used in this project are the $GPRMC (Global Positioning Recommended Minimum Coordinates) and the $GPGGA sentences. These two provide the time, date, latitude, longitude, altitude, estimated land speed, and fix type. Fix type indicates whether the GPS has locked onto the satellite data and received enough data to determine the location (2D fix) or location + altitude (3D fix). The PSoC 4200M communicates with the GPS module via UART interface which is configured at 9600 baud rate [4c]. The PSoC firmware contains a function called GetGPSMessage() which extracts the required information such as longitude, latitude, altitude, speed an UTC time. The UTC time can also be used to initialize the RTC of PSoC 4200M since RTC resets the time every time power is turned off to the PSoC. Figure 12 shows an example of the raw GPS NMEA sentences transmitted by the GPS module. 13

17 Figure 12: Raw NMEA Sentences transmitted by the GPS Following is the sequence of steps followed in the firmware to get the GPS readings: 1. Start the UART component which is configured at 9600 baud rate 2. Store the raw NMEA sentences in a buffer using UART_G_GetChar() API until one complete set of NMEA sentences are received 3. Check whether a fix is obtained from the received NMEA sentences 4. If a fix is obtained, call the GetGPSMessage() function 5. The GetGPSMessage() function first extracts the $GPRMC and the $GPGGA sentences from the raw NMEA sentences 6. The GetGPSMessage() function then extracts the longitude, latitude, altitude, speed and UTC time from $GPRMC and $GPGGA sentences REAL TIME CLOCK (RTC) The data acquired from all the sensors is formatted into a common format along with a time stamp at which the sensor data was acquired. The RTC component in PSoC Creator as shown in Figure 13 is used obtain the current time. The RTC_GetHours, RTC_GetMinutes, RTC_GetSecond APIs are used to get the hours, minutes and seconds respectively from the time value passed from the RTC_GetTime() API [4d]. The time in hours: minutes: seconds is then added to the sensor data which is then sent to the Raspberry Pi. 14

18 Figure 13: RTC Component Configuration DATA LOGGING IN F-RAM The CY8CKIT-044 also provides onboard memory storage via Cypress s non-volatile F-RAM device of 1 Mb capacity [11]. The F-RAM is connected to the I2C interface of the PSoC 4200M device with a 7-bit I2C address of 0x50. It is used for data logging in this project. The combined sensor data from all the sensors along with the timestamp is stored in the F-RAM. Figures 14 and 15 show the timing diagram for write and read operations using F-RAM. Figure 14: Timing Diagram to write data into F-RAM 15

19 Figure 15: Timing Diagram to read data from F-RAM The data stored in the F-RAM can later be read using the Bridge Control Panel software available along with the installation of PSoC Creator [2]. Connect the Kit to PC using USB cable and in the Bridge Control Panel, select KitProg and once it connects read data from the F-RAM using the command, w r 50 x x x x x x x x x x x x x x x p as shown in Figure 16. The above command reads 15 bytes of data starting from the memory location with address 00. To read data from a different memory location, specify the two byte address after w 50 followed by the rest of the command. Figure 16: Bridge Control Panel to read data from F-RAM 16

20 COMMUNICATION WITH RASPBERRY PI The PSoC 4200M communicates with the Raspberry Pi via UART interface. The UART component in PSoC 4200M is configured at baud rate as shown in Figure 17. Customized handshake control is implemented while communicating with the Raspberry Pi. Figure 17: UART Component Configuration All the sensor data and timestamp information are in hexadecimal representation which are converted to ASCII characters using sprintf function to send the data over UART. Note that the acquisition of sensor data, formatting and data logging continues irrespective of the handshake control status i.e., the PSoC 4200M will not be idle until it receives a start from the Raspberry Pi. It will continue to perform other functions in parallel. Once the Raspberry Pi receives the sensor data, it decodes the data format and makes decisions based on the individual sensor data. 17

21 The following tables give the characters chosen for different sensors and also for additional details like timestamp and sensor data logged for a given time duration, Table 4: Sensor Characters # Sensor Sensor Reading Character Example data 1 Encoders Left Encoder EL 2489 Right Encoder ER Accelerometer X-Axis AX 212 Y-Axis AY -163 Z-Axis AZ 12 3 Magnetometer X-Axis MX 253 Y-Axis MY 26 Z-Axis MZ GPS Latitude GX N Longitude GY W Altitude GZ Speed GS Time GT 12:35:19 5 Temperature Sensor Temperature TE Time Stamp - TS 00:01:30 Table 5: Special Characters # Additional Readings Character 1 Timestamp TS 2 All Sensors AL 3 Previous Sensor Data PR XX 05* 4 End Character!! * XX Respective Sensor Character followed by the time duration in seconds 18

22 Example: Raspberry Pi sends the command ELERGXGY!! to request the Encoder and GPS data from the PSoC and the PSoC responds with the respective sensor data along with timestamp as shown in Figure 18. EL ER GX GY!! Raspberry Pi EL 1234 ER 1234 GX N GY W TS 00:01:30!! PSoC Figure 18: Example communication packets between Raspberry Pi and PSoC UART Settings for communication between Raspberry Pi and PSoC: Baud rate: bps (bits per second) Data bits: 8 bits Parity: None Stop bits: 1 Following is the representation of each of the sensor reading: Encoder Data: ± Number of ticks in 100ms (example: 2489, -1672) Accelerometer (units in G-forces (g)) and Magnetometer (units in micro Tesla): ±Reading for each axis (example: 212, -163) GPS Example Readings Latitude: N (Latitude 40 degrees minutes North) Longitude: W (Longitude 082 degrees minutes East) Altitude: (Meters, above mean sea level) Speed: (Speed over the ground in knots) Time: 12:35:19 (UTC time) Temperature: ± reading in degree Celsius (example: 23.24, -2.58) Time Stamp: Hours:Minutes:Seconds (example: 00:01:30) 19

23 Figure 19 shows the flowchart for the communication and unparsing firmware functions. Figure 19: PSoC Firmware Flowchart for communication and unparsing 20

24 LIDAR LIDAR is an optical distance measurement technology that uses laser to determine the distance from targets. LIDAR is useful because of its high accuracy over long ranges. The module used in this project is the LIDAR Lite sensor from PulsedLight [12]. It is a compact, low cost, and low power proximity sensor with a range of up to 40 meters with an accuracy of ±2.5cm. It can be interfaced with a microcontroller through either the I2C or PWM interfaces. I2C is used in this project as the internally processed distance measurements can be read directly from registers in the LIDAR lite module, and this reduces the work of the microcontroller. In principle, the LIDAR lite measures distance based on the precise measurement of the time delay between the transmission of a laser signal and its reception. The high accuracy is achieved by the digitization and averaging of two signals a reference signal emitted by the transmitter before distance measurement, and a received signal reflected from the target. The time delay between these signals is estimated through an accurate correlation algorithm. This time delay is then translated to a distance measurement based in the known speed of light. All signal processing is dine internally on the Signal Processing Core, and the measured distance values on internal registers which can be accessed through the I2C interface as shown in Figure 20. The default slave address for the LIDAR lite is 0x62. Figure 20: LIDAR Lite Block Diagram 21

25 Figure 21 shows the I2C PSoC Creator component configuration for interfacing with LIDAR. Figure 21: PSoC 4 I2C Component Configuration for LIDAR Following is the sequence of steps followed in the firmware to get the distance reading: 1. Write the value 0x04 to register 0x00 to initiate a DC stabilization cycle, signal acquisition and data processing. 2. Wait until an ACK is received. The unit responds with a NACK to read or write commands with a NACK when it is busy processing. 3. Initiate a 2 byte read starting at register 0x8f and store the received bytes separately. These are the upper and lower bytes of the distance in centimeters. 4. Combine the upper and lower bytes to get the measured distance. Future work: The LIDAR lite module does not include a motor that can allow the sensor to scan a wide area. Hence, it will be useful to mount the LIDAR lite on a rotating platform attached to a servo motor to enable the sensor to scan either a 180º forward view or an entire 360º view to create a 2D map of all the obstacles around the vehicle based on its current position. This data can be further utilized to perform Simultaneous Localization and Mapping. 22

26 REFERENCES 1. PSoC 4200M Datasheet 2. User Guide of CY8CKIT PSoC Creator Quick Start Guide 4. PSoC Creator Component Datasheets a. Quadrature Decoder b. I2C c. UART d. RTC 5. Encoder Specifications 6. TMP102 Datasheet 7. ADXL345 Datasheet 8. HMC5883L Datasheet cdn.sparkfun.com/datasheets/sensors/magneto/hmc5883l-fds.pdf 9. Adafruit Ultimate GPS learn.adafruit.com/adafruit-ultimate-gps/overview 10. GPS - NMEA sentence information aprs.gids.nl/nmea/ 11. F-RAM Datasheet LIDAR lite v1 Operating Manual github.com/pulsedlight3d/lidar-lite-documentation/blob/master/docs/lidar-lite-v1- docs.pdf 23