Development of Control Architectures for Multi-Robot Agricultural Field Production Systems

Transcription

1 Development of Control Architectures for Multi-Robot Agricultural Field Production Systems Santosh K. Pitla, Ph.D. Assistant Professor Department of Biological Systems Engineering University of Nebraska-Lincoln IEEE RAS Agricultural Robotics & Automation Technical Committee Webinar #013 December

2 Overview Education and Research Background Control Architectures Individual Robot Control Architecture (IRCA) Multi Robot System Control Architecture (MRSCA) Autonomous Vehicle Platform (AVP) Development Validation of Control Architectures Post-Doctoral Research Future Work at UNL Q&A 2

3 Education and Research Background 2000 to 2004 BS, Mechanical Engineering, Osmania University, Hyderabad, India Aug 2004 to May MS, Mechanical Engineering and Biosystems and Agricultural Engineering University of Kentucky, Lexington, KY Thesis Title: Development of an Electro-Mechanical System to Identify and Map Soil Compaction March 2007 to April 2012 Research Engineer, Machine Systems and Automation, BAE, University of Kentucky Jan 2008 to Jan Ph.D. Program May 2012 to Present, Post-Doc, The Ohio State University OCT 2013 to Present, Assistant Professor and Advanced Machinery Engineer, University of Nebraska-Lincoln 3

4 Control Architectures Un-manned agricultural machines Road blocks Cost Safety Intelligence Control architecture work is underway (Brooks, 1986; Arkin, 1990; Yavuz and Bradshaw, 2002; Blackmore et al., 2002; Torrie et al., 2002; and Mott et al., 2009) Individual Robot Control Architecture Multi-Robot System Control Architecture Weeding Robot (Madsen and Jakobsen, 2001) Autonomous Harvester (Pilarski et al., 2002) Citrus Fruit Harvesting Robot (Hannan et al., 2004) Agricultural Machines Control Architectures Mato Grosso,Brazil Agricultural Robots 4

5 Paradigm Shift (Big Vs. Small)

6 Paradigm Shift (Big Vs. Small) Spray material Application Variation (Luck and Pitla, 2010) 6

7 Individual Robot Control Architecture (IRCA) Intelligence Individual Robot Control Architecture (IRCA) 7

8 IRCA (continued) Sensing Layer (SL) Sensor Stack Array of sensors that aid the robot in learning about the unknown environment Wireless Communication Module (WCM) Processes the information obtained wirelessly from remote or off-machine entities The SL receives the environment data obtained from the sensor stack (on-machine) and the WCM (off-machine) and passes the information to the BL for further processing and filtering

9 IRCA (continued) Behavior Layer (BL) Deliberative Behavior High level decision making processes that require planning and algorithm execution Reactive Behavior Low level processes that do not require considerable computation but are crucial for safety and operability By-Pass of Control The switching of the control command generation source in response to the changing environment

10 Individual Robot Control Architecture (IRCA) FSM I FSM IV FSM II FSM III Desired Control Commands 10

11 IRCA (continued) FSML Simulation (MATLAB) Input signals created using Signal Builder (MATLAB) corresponding to the five scenarios Inputs Scenario I Scenario II Scenario III Scenario IV Scenario IV dob (m) TuF Five scenarios for FSM simulation SA d (deg) ±0.125 ±2.5 - ±0.125 ±0.125 SA r (deg) - - ± Eflag ReF TrF

12 IRCA (continued) FSML simulation Parallel States SIMULINK model created for FSML simulation Mutually Exclusive States FSM I Internal States Cruise Trigger conditions TuF~=1 && dob>=4 && Eflag ~=1 FSML created using the StateFlowChart tool Slow Safe Speed TuF==1 && dob>=4 TuF>=0 && dob<4 && dob>=2 Dead dob<2 Eflag==1 FSM II Navigate TuF>=0 && dob>=4 Safe Navigate FSM III Lower (1) Raise (0) FSM IV ON (1) OFF (0) TuF>=0 && dob<4 ReF~=1 && Eflag~=1&&TrF~=1&& dob>=4 ReF==1 Eflag==1 TrF==1 dob<4 ReF~=1&&Eflag~=1&&TrF~= 1&&dob>=4 ReF==1 Eflag==1 TrF==1 d ob<4 12

13 IRCA (continued) FSM Simulation Results and Discussion Active states of FSML (I to IV) in the default state Actives states in Scenario I Actives states in Scenario III 13

14 IRCA (continued) FSM Simulation Results and Discussion 14 FSM simulation outputs (desired control commands) Active states of FSML (I to IV) in Scenario I Scenarios Simulation Active Internal States Desired Control time (s) (FSM I to IV) Commands Scenario I 0-10 Cruise, Navigate, Lower, ON 4 km/h, ±0.125 o, 1, 1 Scenario II Slow, Navigate, Lower, ON 2 km/h, ± 2.5 o, 1, 1 Scenario III Safe Speed, Safe Navigate, Raise, OFF 1 km/h, ± 5 o, 0, 0 Scenario IV Cruise, Navigate, Lower, ON 4 km/h, ±0.125 o, 1, 1 Scenario V Dead, Navigate, Raise, OFF 0 km/h, ±0.125 o, 0, 0

15 Multi-Robot System Control Architecture (MRSCA) Control Variables: m = Mode (values: 0,1,2) r = Role (values: 0,1,2) c = Communication (values: 0,1) Coordination Strategy (No Cooperation) Control Variables (m, r, c) Stand Alone Behavior (0,0,0) No Cooperation Mode Stand Alone Role Transmit MRS planting in unique work zones (WZ I to WZ III) No Cooperation 15

16 MRSCA (continued) Coordination Strategy (Modest Cooperation) Control Variables (m, r, c) Leader (Baler) (1,1,0) Follower (Bale Spear) (1,2, 1) MRS performing baling and retrieval operations Modest Cooperation 16

17 MRSCA (continued) Coordination Strategy (Absolute Cooperation) Control Variables (m, r, c) Leader (Combine) (2,1, 0) Follower (GC-I, GC-II) (2,2, 1) MRS performing Harvesting Operation Absolute Cooperation 17

18 MRSCA (continued) 18

19 MRSCA (continued) Global Information Module Coordination Strategy Control Variables (m, r, c) No Cooperation (0,0,0) Modest Cooperation (1,1,0), (1,2, 1) Absolute Cooperation (2,1, 0), (2,2, 1) 19

20 MRSCA (continued) Hierarchy of the FSMs in GIM. GIM Role Coordination Wireless Communication Follower StandAlone Leader No Cooperation Modest Cooperation Absolute Cooperation Tx Rx Wait Unload Go To/ Retrieve Follow/ Load Unload Parallel Finite States Internal Finite States Level I Internal Finite States Level II 20

21 MRSCA (continued) Active States during Simulation in Stand Alone Mode Status Flag Definition High Low SA Raised when role of the robot is Stand Alone 1 0 L1 Raised when the robot is performing a leader role with the task Wait is active 1 0 L2 Raised when the robot is performing a leader role with the Unload task active 1 0 F1 Raised when the robot is performing a follower role and the task Goto is active 1 0 F2 Raised when the robot is performing a follower role and the task Follow/Load is active 1 0 F3 Raised when the robot is performing a follower role and the task Unload is active

22 MRSCA (Continued) StatMsgL.mat To File StatMsgF.mat To File1 m r c BL Inputs-IRCA (Leader) m r c BL SA L1 L2 F1 F2 F3 Global Information Module Baler (Leader) SA L1 L2 F1 F2 F3 Scope Leader m r c BD FE BW Inputs-IRCA (Follower) m r c BD FE BW SA L1 L2 F1 F2 BLf F3 Global Information Module Bale Retriever (Follower) SAf L1f L2f F1f F2f F3f Scope Follower BL External Wireless Input Generic Flags Definition High Low BL, BL f Raised when the bale is ready to be retrieved 1 0 BD Raised when the Bale Retriever is closer to the hay bale to be retrieved 1 0 FE Raised when the Bale Retriever is close to the field edge for dropping of the bale 1 0 BW Raised when the bale is loaded on the Bale Retriever

23 MRSCA (Continued) Modest Cooperation (Baling Bale Retrieving) Active states of the Baler (Leader) (a) (b) (c) Active tasks a) Go To; b) Load; and c) Unload of the Bale Retriever during the execution of the bale retrieving task 23

24 MRSCA (Continued) Absolute Cooperation (Harvesting) m r c PS Inputs (IRCA) m r c PS SYNCL2 SYNCL1 SA L1 L2 F1 F2 F3 Global Information Module Leader SA L1 L2 F1 F2 F3 StatMsgL.mat To File Scope Leader m r c od PS FE InputsF1 (IRCA) m SA r L1 L2 c F1 od F2 PS F3 Sf1 FE SYNCF1 Sf1 Global Information Module Follower1 SAf1 L1f1 L2f1 F1f1 F2f1 F3f1 StatMsgF1.mat To File1 Scope Follower1 StatMsgF2.mat External Wireless Input (Follower 1) External Wireless Input (Follower 2) m r c od PS FE InputsF2 (IRCA ) m SA r L1 L2 c F1 od F2 PS F3 FE SYNCF2 Sf2 Global Information Module Follower2 SAf2 L1f2 L2f2 F1f2 F2f2 F3f2 Sf2 To File2 Scope Follower2 Flags Definition High Low PS Raised when the Grain Carts are full with grain or when the grain is available in the Combine for unloading 1 0 OD Raised when the Grain Cart is at a desired bearing (heading and location) relative to the Combine 1 0 SYNCF1/S Raised when Grain Cart I wants to synchronize with YNCL1 the Combine for the transfer of grain 1 0 SYNCF2/S Raised when Grain Cart II wants to synchronize with YNCL2 the Combine for the transfer of grain

25 MRSCA (Continued) 25

26 MRSCA (Continued) 26

27 MRSCA (Continued) 27

28 Autonomous Vehicle Platform (AVP) Development Differential Drive Motor Flexible Coupling AVP framework: (a) solid model of basic frame, (b) fabricated AVP frame with mechanical components (a) (b) (c) (a) Drive motor mounting and roller chain drive to differential at rear axle (b) 24 VDC steering actuator, (c) steering actuator and linkage mounted on the front axle of the AVP Infra-Red Sensor Orthogonal Steel Plate Oscillating DC Motor (a) (b) (c) (d) Ground speed sensor: (a) schematic and wiring diagram and (b) actual mounting location and configuration. Infra-red sensor array construction: a) solid model, b) SHARP GP2Y0A700K NIR sensor, c) OEM 212 series oscillating motor drive, d) assembled sensor array mounted on the AVP 28

29 AVP Development (continued) Terminator CAN Bus IX F Module IX R Module Terminator MC Plus+1 microcontroller (Sauer Danfoss, MN) System Controller Speed Controller Steering Controller IR F Controller AVP distributed controller network topology IR R Controller Leaf Light HS (Kvaser, Sweden) CAN to RS232 gateway Task Computer for the AVP (Eee PC 1000, ASUSTeK Computer, Inc., Peitou, Taipei, Taiwan, R.O.C) 9XTend PKG (B&B Electronics Manufacturing Co., Ottawa, IL) radio frequency modem Trimble AgGPS 132 (Trimble Navigation Ltd., Sunnyvale, CA) GPS engine and antenna 29

30 FWD Bkwd Left Right Stop Stop AVP Development (continued) 24 V Key Switch Manual Auto Signal Auto Signal Manual V+ M+ Speed Driver V- S + - M- 50 A Relay Fuse Block 30 A 5 A 5 A V+ M+ Steering Driver S + - V- M- 10 A CAN PWR 4.4 KΩ 4.4 KΩ Wireless module (b) Speed Controller Steering Controller System Controller Inputs (a) AVP PDP (a) schematic of PDP, and (b) completed PDP installed on AVP Task Computer VB.Net User Interface RS232 RS 232 GPS RS 232 RF Modem CAN to Serial Converter Terminator CAN Bus Terminator System Controller Speed Controller Steering Controller IRF Controller IRL Controller Drive Motor Steering Actuator IR Sensor Front Component Map of the AVP IR Sensor Rear GUIDE software used for programming the Micro Controllers 30

31 AVP Development (continued) Multi-Robot System (MRS) created by replicating the AVP Completely electric - 24 VDC Operates in Manual and Automatic modes On-board rechargers for Lead-Acid Batteries CAN based distributed controller network Ability to override the automatic mode via manual commands in the event of emergency Three way AVP safety 1) IR sensory arrays, 2) onboard emergency stops, and 3) wireless remote stop. Task computer, GPS and wireless communications for automatic operation. 31

32 Validation of IRCA Deliberative and Reactive Behavior 40 AB Line Robot Path Obstacle 35 B 30 Northing (m) A Easting (m) AVP tracking of AB line with obstacle in its path Speed States of the AVP 32

33 Validation of MRSCA Standalone Behavior Automatic tracking of AB lines by three AVPs simulating planters. 33

34 Validation of MRSCA (continued) Modest Cooperation C Baler Path Bale Retriever Path F 35 Field Edge Northing (m) B Bale drop off location A D Easting (m) Motion paths of Baler and Bale Retriever AVPs 34

35 Validation of MRSCA (continued) Absolute Cooperation Paths of Harvester, GC1 and GC2 AVPs during TS3. 35

36 Post-Doctoral Research Next Gen Autonomous Plat Form (AVP II) 36

37 Post-Doctoral Research Controller Area Network (CAN) Data Acquisition from Field Machinery ISO (Source: Data Acquisition from the ISO Diagnostic Port (Tractor: MX340) Anhydrous Applicator 16 row planter 12 row planter Sprayer 37

38 Post-Doctoral Research CAN Data Acquisition Decoded GPS CAN Data Screenshot of Vector CANalyzer Interface (Data collection from CASE IH MX340 Tractor) Time ID Data length D0 D1 D2 D3 D4 D5 D6 D FEF31Cx Rx d 8 E2 C F0 ED AE 4B GPS CAN Message 38

39 Post-Doctoral Research CAN Data Acquisition Work Period Turning Dwell Period 39

40 Future Work at UNL Machine Automation and Agricultural Robotics for Row Crop and Bio Energy Production Moving Biomass Bales from the field to On-Farm Bio Mass Processing Facility Spreading the by-product (fertilizer) in the field Soil Sampling, selective spraying, Weed Mapping, Crop Health MRS Planting or Spraying UAV (Source: Hawk.com) AR Drone 2.0 ( 40

41 Thank You!! Questions? 42