2018 Citrus Circuits Electronic Scouting System

Transcription

1 2018 Citrus Circuits Electronic Scouting System 1678 App Programming Team: August 1, 2018

2 Contents 1 Introduction History System Overview Training Resources Foundations System Components Technical Details Collection Scout Super Scout Pit Scout Processing Server Visualization Android Viewer ios Viewer Logistics Development Process and Schedule Calculations Low Level Average (Standard) Average (Last Four Matches) High Level timd.teleopexchangeability timd.teleopscaleability timd.teleopallianceswitchability timd.teleopopponentswitchability R-Score winchance predictedrps predictedseed Conclusion Review Overall Infrastructure Appendix Additional Resources Collected Data Point Reference Scout Super Scout Pit Scout Calculated Data Point Reference Team Team In Match Data (TIMD)

3 Chapter 1 Introduction 1.1 History Citrus Circuits first used a custom-built electronic scouting system in 2013, and has used such a system every year since. Every year, we have released a whitepaper with the details of that year s scouting system. For past years, please reference the following whitepapers: 2013 White Paper: White Paper: White Paper: White Paper: White Paper: System Overview Training Resources ios Resources CC-Login-System Android Resources Python Resources Foundations Firebase Realtime Database Firebase database is a data storage and syncing software that is intended to be reliable and easy to implement. It is responsible for the storage of all data (other than photos) in our system, and automatically syncs data across the various devices. It allows apps to continue working (although possibly with stale data) even without a good internet connection, because a query to the database on a given device will be answered with the most up-to-date data available to that device (even if it was from an hour ago, the last time the device was connected to the internet). Poor cell service and no reliable public wifi is a condition that is present at a significant number of FRC competition venues. Firebase Storage Firebase Storage is responsible for the transfer and storage of all of the robot photos. Unlike the Realtime Database, firebase storage works on a more classic query system. It does not automatically sync photos, and a query will error if the client device is not connected to the internet System Components The basic flow of the scouting system can be broken up into three different sections: Collection, Processing, and Visualization. Within these three sections are seven different component programs and applications. 2

4 Collection The largest of the three sections of the scouting system, Collection is devoted to entering and uploading all of the data that is used by the other systems. In our system, Collection is made up of three applications: Scout Android: Collects objective data about the robot s performance on the field. Example data points from this year are the number of ground cube intakes, type of climb, etc. Between 1 and 3 scouts collect data on each team in each qualification match. Super Scout Android: Collects more subjective data on comparative robot performance on the field, such as robot speed, agility, and defense. Generally used by a team member who is able to effectively rank the three robots playing on their assigned alliance from best to worst in each category. 1 Super Scout collects data on each alliance in each qualification match. Pit Scout ios: Collects images and robot design data in the pits. Example data points from this year include robot photos, programming language, ramp time, etc. Generally used by someone with good social skills. Processing Immediately following Collection is Processing. Processing is the smallest of the three sections, consisting of only one application: the server. It relies on Collection for the raw data that it condenses down to meaningful calculated metrics that can be used for strategy. The calculations used this year are covered later in the Calculations section. Server Python: Performs the calculations that extract metrics of robot performance from the raw data. Also allows the export of the calculated data points to an.csv file used in the strategy meeting (discussed later). Visualization The goal of the Visualization section is to allow the strategy and drive teams to quickly visualize both raw and calculated data. Viewers (Android and ios) Allows for the quick viewing of relevant scouting data for the strategist and drive coach. Features include the ability to graph calculated and raw data points over a robot s matches, viewing of a robot s photos, and the sorting of robots in order of nearly any datapoint (ex. by first pick rank, second pick rank). Scouting Spreadsheet Google Sheets: Imports the raw.csv files outputted from the server, with all scouting data. Uses lookup functions to populate pre-formatted sheets and graphics with pertinent data for pick list creation. 3

5 Figure 1.1: 2018 Scouting System Outline 4

6 Chapter 2 Technical Details 2.1 Collection Scout Feature List Simple and efficient interface for data entry Ability to follow pre-set scouting schedule Ability to resend match data Ability to read QR Code for scout assignments Collecting data about both Autonomous and teleoperated (and end game) robot performance Generate QR Code of entire compressed match data (a) Pre-Match Screen (b) Autonomous Screen (c) teleoperated Screen Figure 2.1: Scout User Interface 5

7 Details The Scout app is designed to be as consistent and simple as possible. Data is sent to Firebase, an online database, by each scout who uses a mobile QR Scan app to upload scout data. The scout assignment varies as each scout s precision ranking changes over the course of the competition. The mobile QR Scan app also shows a QR code generated from a datapoint in Firebase that represents their rank. The Scout table has a camera feature that scans the QR code on the QR app as part of the dynamic scout assignment process. In circumstances where there is a problem with the dynamic scout assignment system, a backup system is ready at hand where the scouting tablet automatically pulls its robot assignment from a pre-downloaded schedule file using its unique scout tablet ID. Data entry occurs through an interface consisting solely of buttons, counters (a plus button and a minus button with a number between them), and pop-up dialogs for more specific data input. We have found that in the highpressure environment of scouting live matches, this interface of buttons often provides greater data accuracy than a more complex UI. In addition, our data transfer system was designed to not require any internet capability from the scout tablets. Even by using the cell data of our scouts personal mobile devices, data usage turned out to be less than a megabyte of data for the entirety of the competition. Nevertheless, the collection of timing data such as recording the start and success time of cubes scored on the scale and switch is minimized due to its relative inaccuracy amongst the scouts. 6

8 2.1.2 Super Scout Feature List Simple interface for data entry Note-taking interface for recording additional observations requested during the event by the strategy team Qualitative ranking of an alliance Ability to override match data Ability to resend match data Backup system in external storage to prevent data loss (a) Pre-Match Screen (b) Auto Screen (c) Tele Ranking Screen (d) Final Datapoint Screen Figure 2.2: Super Scout User Interface Details The Super Scout app is responsible for subjectively ranking the quality of robots. Scouts give exact, quantitative data, such as number and type of intake, number of cubes placed, and type of robot climb while super scouts rank robots on data like speed, agility, defense etc. The only quantitative data the super scout collects is the final match data: score, number of foul points awarded, etc. Unlike the Scout, the Super Scout automatically obtains a match schedule from the firebase prior to the beginning of the competition. The Super Scout can also override this automated progression in case a match needs to be replayed or any other deviation from the schedule occurs. Unlike the scout, each super scout enters data on an entire alliance, rather than a single robot. Data is collected throughout the match. Similar to the Scout app, the interface is made up almost entirely of counters so that data entry is made as simple as possible. Evaluation is done on an ordinal ranking system of 0 to 4. A 1 in defense, for example, means that the team in question played no real defense and merely played defense by driving around the field, so the scouter could not evaluate the defensive ability of the team. A team that does not show up to the match receives a 0 in every category. A 4 in defense, for example, means that the robot in question clearly 7

9 played the best defense on the field. Most of the time, the robots will simply be given a rank 1, 2, or 3, where 1 is the worst on the alliance, and 3 is the best, in any given category. Each piece of data is individualized and relative to the other robots (e.g. only one robot on each alliance could get a 3 in speed). If the super scout feels it is necessary or is requested by the strategy team to note something about a robot that does not fit into any of the categories they are scoring, they can tap on the team number to open a dialog where they can enter notes on that team. After all data is entered, it is automatically written to Firebase and the app returns to the Pre-Match Screen, in preparation for the next match. From the Pre-Match Screen, data from previous matches can also be resubmitted to Firebase. The Super Scout tablets are the only tablets with a prepaid data plan. 8

10 2.1.3 Pit Scout Feature List Short list of data points for rapid collection Local data storage when offline Missing data list Ability to select default team image Ability to share the data in JSON format Works across multiple devices (a) Team List Screen (b) Data Entry Screen (c) Data Filter Screen Figure 2.3: Pit Scout User Interface Details Unlike the Scout and Super Scout apps, the pit scout app is designed to collect relatively few data points. Because of this, data entry is significantly simplified. There is a single data entry screen where the user can enter in data points, such as a robot s available weight, robot dimensions, wheel diameter, programming language, drivetrain, etc. The user can also add and delete pictures, view all the uploaded pictures, and set a selected image to be viewed in the viewer apps. All data is automatically uploaded when entered, and the UI is marked red if there is no value. Navigation between teams is done through three views: the Team List View, the Filter View, and the Nil Values View. Any of these views can bring you to the data entry view of a specific team. The Team List view lists the team in numeric order. To indicate which teams have been scouted, a long press on a team adds a check to their name and moves them to the bottom of the team list. In addition, the status of image uploading is displayed on the Team List Screen. Teams with images that are still uploading are highlighted in red, while teams with completely uploaded images are written in green text. If a team has not had photos uploaded, the text is the black. If a team is missing from the database, the pit scouter can add a new team by hitting the Add Team button. 9

11 The Filter View is used to search for specific data points or data values. The Drop-down menu feature is prevalent for the usage of displaying, different data points, various data point values, and highlighting teams with specific missing data points. Missing data can be checked through the Nils view where teams are shown with their corresponding missing data points. 10

12 2.2 Processing Server Feature List Parallel processing of data for accelerated calculations Caching to improve processing speed Export of calculated data to.csv file for strategy meetings Automated Slack crash reporting to maximize uptime during competition Asynchronous data writing for accelerated uploading Scout performance review algorithm to maximize data accuracy (a) Team Calculations (b) Calculation Process (c) Match Calculations Figure 2.4: Server Calculations Details The Server is responsible for all data processing in our scouting system. Data is sent from scout tablets via a QR code to Firebase. The server then downloads the data from Firebase, decompresses it, and then consolidates the data. During most matches, we have 18 scouts, resulting in 3 sets of identical data per robot. Our server looks for differences in these data sets and calculates which data to use based on majority rule and our scout precision ranking system. From there, data for teams and matches is updated based on the new data. The Server is also responsible for sending assignments to scout tablets. Before competition, after the match schedule is published, we send files to our scout tablets with pre-determined assignments. Our normal system uses our scout precision rank (SPR) to pair each robot with one of the 6 most accurate scouts, one of the 6 least accurate scouts, and one of the 6 middle-ranked scouts. The file we send contains an assignment based on the accuracy ranking (1-18). The accuracy ranking is sent every scout rotation (about 5 matches) through a QR code. In case this system fails, we also send a second file with assignments based on scout ID. Each tablet has a unique scout ID (1-18). The file contains an assignment based on scout ID, randomly distributed. When scouts send data after each match, they also send diagnostics to ensure scouts are using the correct assignment mode (either the QR code or the secondary backup file). This is to ensure that there is a correct distribution of scouts. The Server also checks for missing data and notifies our lead scouts through Slack. 11

13 2.3 Visualization Android Viewer Feature List Live data sync updates after every new match has been played and scouted Graphing data across matches for visualization of team performance trends Ranking of teams by data point for flexible data analysis Access to individual TIMD raw data points Starring teams and matches Match summary screen for quick evaluation of matches Quick access to team schedule Live pick-list editor Show predicted team seeding based on scout data calculation Displays last four match averages for calculated scouting data points for each team (a) Navigation Drawer (b) Match Schedule (c) Match Summary (d) Team Calculated Data (e) Team Raw Data (f) Graphing (g) Seeding (h) Super Scout Data (i) Highlight Feature (j) Settings Figure 2.5: Android Viewer User Interface 12

14 Details As part of Visualization, the Android Viewer is one of the most feature-rich parts of the scouting system. Built to meet the needs of all members of the drive and strategy teams, the Android Viewer is designed for a flexible variety of uses. One of our newest additions to the Android Viewer in the season was the live editable pick-list which allows strategists to make changes to our overall alliance pick-list. The core of the application is the navigation drawer, which allows the user to easily access many common data views, such as our team s schedule, starred matches, the first and second pick lists, the current seeding, the predicted seeding, and the live editable pick-list. Navigation through the app is intended to also be very flexible, allowing the user to follow their train of thought. For example, tapping a match on the schedule brings the user to that match s summary screen. Tapping a team on that screen brings the user to the data screen for that team. The user can then easily look at that team s matches, from which they can view the summary screen of a match, from which they can view another team. To help with the long chains of activities this creates, long pressing the back button returns the user to the application s root activity. A service runs constantly in the background in order to ensure that the latest data is displayed. With Firebase s live data sync capability, the Android Viewer live updates all data after every match has been played and scouted. There are times when data update seems to be an issue, so there is a built in refresh capability on the main screen where the user can manually update the viewer. Four main features are designed to ensure flexibility in data analysis: graphing, raw data viewing, rankings, and sorting. The user is able to graph almost every data point over matches by tapping on the data point. For example, to identify any trends in the consistency of a team s cube scoring ability in the scale, the user can tap on the Avg. Cubes in Scale Tele for a team. This will create a bar graph of the team s number of cubes scored on the scale per match. In order to examine what happened in each individual match, the user can tap on the bar for that match to bring up all of the raw collected data and notes from that match. In order to compare data points between teams, the user can long press a data point to rank all of the teams by that data point. For example, to identify the most consistent auto run robots in the competition, the user can long press the Auto Run Percent Success for a team, calling up a list of all of the teams ranked by their auto run success record. Further sorting other than by rank for a team s datapoint can be done with a dropdown sorting menu, allowing the user to sort teams by team number, rank, First Pick Ability, etc. The combination of these four features enable data to be easily analyzed on a variety of scales, from an individual TIMD to the whole competition. 13

15 2.3.2 ios Viewer Feature List Optional background image downloading for immediate loading at competition Caching of image data for lower network usage Graphing data across matches for visualization of team performance trends Ranking teams by any data point for flexible data analysis Starring teams and matches to prevent missed matches-to-watch Match summary screen for quick evaluation of matches Quick access to team schedule (a) Match Schedule (b) Match Summary (c) Graphing Capabilities (d) First Pick Rankings (e) Team Data Screen (f) Team Match Schedule (g) Searchable Lists (h) Settings Figure 2.6: ios Viewer User Interface 14

16 Details Due to the similarities of the Android and ios Viewers, only the differences between the two will be noted. Unless stated, the ios Viewer shares all of the features of the Android Viewer. Instead of a navigation drawer, the ios Viewer handles root view navigation through a navigation bar at the bottom of the screen. This is done due to the ios Application Design s lower reliance on navigation drawers. The ios Viewer also has the option to turn off automatic image downloading and instead download images when the user requests them. While this conserves cell data and hard drive space, it does require the user to wait for the image to be downloaded. The ios Viewer also allows the user to examine the non-selected images for a team, while the Android Viewer only allows the user to view the selected image. The ios Viewer includes a settings page similar to that of the Android Viewer called the Options Screen for customizing the UI of your app inside of a use cycle. It has capabilities for simplifying schedule UI, changing text size, and even changing what data is displayed where on some views. It also allows you to link your Citrus Circuits Slack account, sending notifications for your starred matches when on your Slack as well as with push notifications. Finally, the ios Viewer does not show match number and data values on the graphs unless the bar is tapped, and does not allow viewing of raw Team In Match Data. 15

17 2.4 Logistics Development Process and Schedule As is often the case, Citrus Circuits worked on scouting throughout the year, including during offseason. The year was roughly divided into 5 sections: offseason (May 2-January 6), early build season (January 7-January 31), late build season (February 1-19), competition season (February 28-April 8), and championships (April 17-April 22). Scheduling and activities during each section can be seen in the Google documents below: Ori0sk5iO-Zookrcu8ft167Bmx3ldY/edit?usp=sharing docs.google.com/document/d/1y2ylsj-hisozhoejiuvu5ibpboywnvffnhkybi3dzhi/edit?usp=sharing com/document/d/1z0pyji4odngzmei6ycjkwq1zmiwom7lmdggywvo1o44/edit?usp=sharing 16

18 Chapter 3 Calculations 3.1 Low Level Average (Standard) Data Points: disabledpercentage, incapacitatedpercentage, dysfunctionalpercentage, avgspeed, avgdefense, avgagility, avgdrivingability, avgclimbtime, avgnumrobotslifted, avgnumcubesplacedauto, avgnumcubesplacedtele, avgnumallianceplatformintakeauto, avgnumallianceplatformintaketele, avgnumopponentplatformintaketele, avgnumcubesfumbledauto, avgnumcubesfumbledtele, avgnumelevatedpyramidintakeauto, avgnumelevatedpyramidintaketele, avgnumgroundpyramidintakeauto, avgnumgroundpyramidintaketele, avgnumgroundintaketele, avgnumgroundportalintaketele, avgnumhumanportalintaketele, avgnumexchangeinputtele, avgnumreturnintaketele, avgnumgooddecisions, avgnumbaddecisions, avgcubesspilledauto, avgcubesspilledtele, avgcubesplacedinscaleauto, avgcubesplacedinscaletele, avgallianceswitchcubesauto, avgallianceswitchcubestele, avgopponentswitchcubestele, avgscaletimeauto, avgscaletimetele, avgtimetoownallianceswitchauto, avgtimetoownscaleauto, avgallianceswitchtimeauto, avgallianceswitchtimetele, avgopponentswitchtimetele The average of a data point across a team s played TIMDs. Boolean values of True and False are treated as 1 and 0 respectively Average (Last Four Matches) Data Points: lfmavgnumrobotslifted, lfmavgnumcubesplacedauto, lfmavgnumcubesplacedtele, lfmavgnumallianceplatformintakeauto, lfmavgnumallianceplatformintaketele, lfmavgnumopponentplatformintaketele, lfmavgnumcubesfumbledauto, lfmavgnumcubesfumbledtele, lfmavgnumelevatedpyramidintakeauto, lfmavgnumelevatedpyramidintaketele, lfmavgnumgroundpyramidintakeauto, lfmavgnumgroundpyramidintaketele, lfmavgnumgroundintaketele, lfmavgnumgroundportalintaketele, lfmavgnumhumanportalintaketele, lfmavgnumexchangeinputtele, lfmavgnumreturnintaketele, lfmavgnumgooddecisions, lfmavgnumbaddecisions, lfmavgcubesspilledauto, lfmavgcubesspilledtele, lfmavgcubesplacedinscaleauto, lfmavgcubesplacedinscaletele, lfmavgallianceswitchcubesauto, lfmavgallianceswitchcubestele, lfmavgopponentswitchcubestele, lfmavgscaletimeauto, lfmavgscaletimetele, lfmavgallianceswitchtimeauto, lfmavgallianceswitchtimetele, lfmavgopponentswitchtimetele, lfmmaxscalecubes, lfmmaxexchangecubes, lfmautorunpercentage, lfmincapacitatedpercentage, lfmdisabledpercentage, lfmdysfunctionalpercentage, lfmtotalcubesplaced, lfmsoloclimbpercentage, lfmassistedclimbpercentage, lfmactiveliftclimbpercentage, lfmactivenoclimbliftclimbpercentage, lfmactiveassistclimbpercentage, lfmavgclimbtime, lfmavgspeed, lfmavgdefense, lfmavgagility, lfmavgdrivingability The average of a data point across a team s four most recent TIMDs. Boolean values of True and False are treated as 1 and 0 respectively. 3.2 High Level timd.teleopexchangeability timd.teleopexchangeability tea(t) for team T is defined as tea(t ) = (ae(t ) 5) + ((AεP lp (A)) 30) where ae(t ) refers to the avgexchangeinput for team T, P refers to a set of team T s alliances in their completed matches, and lp(a) refers to the levitateprobability for alliance A. 17

19 3.2.2 timd.teleopscaleability timd.teleopscaleability tsa(t) for team T is defined as tsa(t ) = asc(t ) ppsc where asc(t ) refers to the average number of scale cubes placed by team T, and ppsc is a constant defined as the average number of points earned per scale cube in the entire competition timd.teleopallianceswitchability timd.teleopallianceswitchability tasa(t) for team T is defined as tasa(t ) = aasc(t ) ppasc where aasc(t ) refers to the average number of alliance switch cubes placed by team T, and ppasc is a constant defined as the average number of points earned per alliance switch cube in the entire competition timd.teleopopponentswitchability timd.teleopopponentswitchability tosa(t) for team T is defined as tosa(t ) = aosc(t ) pposc where aosc(t ) refers to the average number of opponent switch cubes placed by team T, and pposc is a constant defined as the average number of points lost by the opposing alliance per opponent switch cube in the entire competition R-Score Data Points: RScoreSpeed, RScoreDefense, RScoreAgility, RScoreDrivingAbility As can be seen from the graph, the ranking system used by the scouts creates a distribution of scores with many teams ranked near the middle. Figure 3.1: Super Ranking Average Distribution In order to properly spread out the teams, an r-score is used. The r-score of a super ranking average is zs(t ) = µ(t ) µ comp σ comp T C µ(t ) C where µ(t ) is the super ranking average for team T, µ comp = competition, and σ comp is the standard deviation of the µ(t ) s across C. where C is the set of all teams in the 18

20 3.2.6 winchance In order to determine the win chance of alliance A facing opposing alliance O, Welch s t-test 1 is used. This test is expressed in the formula t = X 1 + X 2 s 2 1 N 1 + s2 2 N 2 where X 1 is the mean of the first sample, s 1 is the standard deviation of the first sample, N 1 is the size of the first sample, X2 is the mean of the second sample, s 2 is the standard deviation of the second sample, and N 2 is the size of the second sample. This t is then converted to a win probability wc(a) using the cumulative distribution function 2 for a t-distribution T (t ν). In this case, X1 is S p (A) for alliance A, s 1 is S p σ (A) for alliance A, and N 1 is the average number of completed TIMDs nct (A) of the teams on alliance A. Similarly, X2 = S p (O), s 2 = S p σ (A), and N 2 = nct (O). The function wc(a) comes out to wc(a, O) = T (t ν) t is the t-value generated by the Welch s test and ν is the degrees of freedom approximated by the Welch-Satterthwaite equation 3 : ν ( s 2 1 N 1 + s2 2 N 2 ) 2 s 4 1 N + s ν1 N2 2 ν2 where ν 1 = N 1 1 (the degrees of freedom for the first variance) and ν 2 = N 2 1 (the degrees of freedom for the second variance) predictedrps The predictedrps prp (A) for alliance A is defined as predictedseed prp (A) = 2 wc(a) + bc(a) + rc(a) predictedseed is calculated by sorting teams by their predicted total number of RPs prs total (T ). In order to maximize the accuracy of the prediction, we use the actual number of ranking points nrp (M) generated by played matches and reserve the predicted number of ranking points prp (M) to unplayed matches. prs total (T ) = M P nrp (M) + M U prp (M) where P is the set of team T s played matches and U is the set of team T s unplayed matches. In the case of ties, the second tiebreaker, cumulative park and climb points, is used. Similar logic is used in generating this predicted number of cumulative park and climb points, with the actual number of points generated being used for played matches and the predicted number of points being used for unplayed matches. Using this method, the predicted number of park and climb points pap total is calculated using the formula pap total (T ) = M P nap (M) + M U pap (M) where nap (M) is the sum of points gained from climbing or parking for match M and pap (M) = T A aa(t ) is the predicted number of climb and park points for alliance A in match M. 1 t-test 2 distribution function 3 equation 19

21 Chapter 4 Conclusion 4.1 Review Overall The Citrus Circuits 2018 Scouting System was the most effective yet. Stable, fast, and easy to use, it allowed for flexible and powerful strategic analysis that was crucial in helping us to reach the Einstein field for the sixth year in a row Infrastructure Firebase Firebase was a core part of the scouting system. The simplicity of integrating Firebase into applications helped us accomplish more development and testing throughout build season. The automated syncing was highly reliable even when network connectivity was poor. Our only major issue was the free plan running out of download space during Championships, forcing us to temporarily migrate to a different account. Photo/Media Storage Firebase storage was initially used in the season to store photos from pit-scouting, but due to logistic issues we switched to using Smugmug. 20

22 Chapter 5 Appendix 5.1 Additional Resources Simbots Seminar Series: Scouting and Match Strategy Seminar by the renowned mentor of Team 1114 Karthik Kanagasabapathy. Covers various methods of scouting, development of match strategy, and the foundations of mathematical analysis techniques in FRC. Overview and Analysis of FIRST Stats Comprehensize overview of mathematical analysis of FRC and FTC teams. Covers concepts ranging from simple analyses such as OPR, DPR, and CCWM to more complex methods such as WMPR, EPR, and MMSE techniques. Some knowledge of linear algebra required. Citrus Circuits Fall Workshops A variety of workshops/seminars put on by students and mentors on Team For those interested in scouting and strategy, we recommend the Strategy and Game Analysis and Scouting Development workshops. Team 2834 Scouting Database Explanation Simple introduction the concept of OPR. Great for beginners and those without much knowledge of linear algebra. The Blue Alliance Excellent website for viewing data on FRC. Provides an API for easily pulling data from the site to use in custom calculations Collected Data Point Reference Scout timd.scoutname: The initials or name of the scout who scouted the match. Used for scout evaluations, robot assignment, and tracking down the source of incorrect data. timd.allianceswitchattemptauto: Data on cube placement in the alliance switch, including stack layer, begin time, success/fail time, prior ownership status, during the autonomous period. timd.allianceswitchattempttele: Data on cube placement in the alliance switch, including stack layer, begin time, success/fail time, prior ownership status, during the teleoperated period. timd.opponentswitchattempttele: Data on cube placement in the opponent switch, including stack layer, begin time, success/fail time, prior ownership status, during the teleoperated period. timd.scaleattemptauto: Data on cube placement on the Scale, including stack layer, including begin time, success/fail time, prior ownership status, during the autonomous period. timd.scaleattemptauto: Data on cube placement on the Scale, including stack layer, including begin time, success/fail time, prior ownership status, during the teleoperated period. timd.climb: Data on a robot s climb during the END GAME period, including type of climb, total time to succeed or fail, and, if applicable, the number of robots lifted. 21

23 timd.allianceplatformintakeauto: The position(s) of cube(s) a robot intakes from the line-up of cubes in the region between the alliance switch and the alliance platform zone during the autonomous period. timd.allianceplatformintaketele: The position(s) of cube(s) a robot intakes from the line-up of cubes in the region between the alliance switch and the alliance platform zone during the teleoperated period. timd.opponentplatformintaketele: The position(s) of cube(s) a robot intakes from the line-up of cubes in the region between the opponent switch and the opponent platform zone during the teleoperated period. timd.numcubesfumbledauto: The number of cubes unintentionally lost to the open field or another robot from a position of a controlled ownership by a robot during the autonomous period. timd.numcubesfumbledtele: The number of cubes unintentionally lost to the open field or another robot from a position of a controlled ownership by a robot during the teleoperated period. timd.numexchangeinput: The number of cubes inserted into the Exchange Zone during the teleoperated period. timd.numgroundintaketele: The number of cubes a robot intakes from the field ground during the teleoperated period. timd.numgroundportalintaketele: The number of cubes a robot intakes in which the cube was given by the human player in the Portal Zone and dropped in field ground position. timd.numhumanportalintaketele: The number of cubes a robot intakes in which the cube was directly transitioned from the Portal to the robot and NOT from field ground position. timd.numgroundpyramidintakeauto: The number of ground-level positioned cubes a robot intakes from the Pyramid Zone during the autonomous period. timd.numgroundpyramidintaketele: The number of ground-level positioned cubes a robot intakes from the Pyramid Zone during the teleoperated period. timd.numelevatedpyramidintakeauto: The number of elevated-level positioned cubes a robot intakes from the Pyramid Zone during the autonomous period. timd.numelevatedpyramidintaketele: The number of ground-level positioned cubes a robot intakes from the Pyramid Zone during the teleoperated period. timd.numreturnintake: The number of cubes a robot intakes from the Return slot of the Exchange Zone. timd.numspilledcubesauto: The number of cubes knocked off the Scale or Switch, not including a failed attempt of placing a cube in the robots current possession in the Scale or Switch during the autonomous period. timd.numspilledcubestele: The number of cubes knocked off the Scale or Switch, not including a failed attempt of placing a cube in the robots current possession in the Scale or Switch during the teleoperated period. timd.staringpostition: The autonomous starting position of a robot specified as either Left, Center, or Right. timd.didmakeautorun: period. Whether or not the robot successfully crossed the auto line during the autonomous timd.didpark: Whether or not the robot successfully parked by the end of the END GAME period. timd.didgetincapacitated: Whether or not the robot died during the match. timd.didgetdisabled: Whether or not the robot started the match dead or was absent from the match. 22

24 5.2.2 Super Scout timd.rankspeed: A qualitative measure of the robot s speed. Robots are ranked from 1 (Worst) to 3 (Best) of the robots on their alliance. The best robot on the field is given a 4 instead of a 3. If a robot does not take any action that allows the measurement of this characteristic, a 2 is entered. Refer to the ordinal ranking system mentioned in the Super Scout App section under Technical Details. timd.rankdefense: A qualitative measure of the robot s ability to play defense, measured by the ordinal ranking system. timd.rankagility: A qualitative measure of the robot s ability to navigate the field quickly and effectively, measured by the ordinal ranking system. timd.numgooddecisions: A count of the number of good decisions made by a robot during the match subjectively justified by the Super Scout. timd.numbaddecisions: A count of the number of bad decisions made by a robot during the match subjectively justified by the Super Scout. timd.supernotes: Any notes that the super scouts believes need to be inputted about the robot. reddidautoquest: Whether or not the red alliance successfully completed the Auto Quest. reddidfaceboss: Whether or not the red alliance successfully faced the boss during the END GAME period. redcubesforpowerup: The number of cubes in the red alliance vault at the time of activation. redcubesinvaultfinal: The number of cubes left standing in each power vault of the red alliance at the end of the match. bluedidautoquest: Whether or not the blue alliance successfully completed the Auto Quest. bluedidfaceboss: Whether or not the blule alliance successfully faced the boss during the END GAME period. bluecubesforpowerup: The number of cubes in the blue alliance vault at the time of activation. bluecubesinvaultfinal: The number of cubes left standing in each power vault of the blue alliance at the end of the match. redscore: The score of the red alliance in the match. Only inputted by the red super scout. bluescore: The score of the blue alliance in the match. Only inputted by the blue super scout. foulpointsgainedred: The number of points the red alliance gained due to fouls by the blue alliance. Only inputted by the red super scout. foulpointsgainedblue: The number of points the blue alliance gained due to fouls by the red alliance. Only inputted by the blue super scout Pit Scout pitimagekeys: Keys specifying what images have been uploaded. pitallimageurls: The URLs of all stored images of the robot. pitcancheesecake: Whether or not the robot would be able to accommodate a cheesecake built by our team to improve its performance. pitdrivetrain: The type of drive train the robot uses. pitprogramminglanguage: The programming language used to program the robot. pitselectedimage: The image of the robot that the viewers should display. pitavailableweight: The difference between the current weight of the robot and the weight limit. pitsealsnotes: The notes taken by the Citrus Circuits SEALs team. 23

25 pitclimbertype: The type of climbing mechanism of the robot. pitrobotdimensions: The dimensions of the robot. pithascamera: Whether or not the robot possesses a camera. pitwheeldiameter: The wheel diameter of the robot. 5.3 Calculated Data Point Reference Team Overall Data firstpickability: Score prediction calculation to determine the strength of the robot as a first pick. secondpickability: Score prediction calculation to determine the strength of the robot as a second pick. nummatchesplayed: The number of matches a robot has played. actualseed: Seed calculated from collected data. Pulled from TBA by default, but overridden if TBA of FIRST s reporting is down. actualnumrps: Actual number of ranking points per match in the robot s matches calculated from collected data. Pulled from TBA by default, but overridden if TBA or FIRST s reporting is down. predictedseed: Predicted seed at the end of qualification matches. Calculated using predictednumrps. predictednumrps: Predicted number of ranking points per match at the end of qualification matches. totalnumrps: The total cumulated number of RPs of a robot across matches. avgnumcubesplacedauto: Average of timd.numcubesplacedauto across matches. avgnumrobotslifted: Average of timd.numrobotslifted across matches. avgnumcubesplacedtele: Average of timd.numcubesplacedtele across matches. avgnumallianceplatformintakeauto: Average of timd.numallianceplatformintakeauto across matches. avgnumallianceplatformintaketele: Average of timd.numallianceplatformintaketele across matches. avgnumopponentplatformintaketele: Average of timd.numopponentplatformintaketele across matches. avgnumcubesfumbledauto: Average of timd.numcubesfumbledauto across matches. avgnumcubesfumbledtele: Average of timd.numcubesfumbledtele across matches. avgnumelevatedpyramidintakeauto: Average of timd.numelevatedpyramidintakeauto across matches. avgnumelevatedpyramidintaketele: Average of timd.numelevatedpyramidintaketele across matches. avgnumgroundpyramidintakeauto: Average of timd.numgroundpyramidintakeauto across matches. avgnumgroundpyramidintaketele: Average of timd.numgroundpyramidintaketele across matches. avgnumgroundintaketele: Average of timd.numgroundintaketele across matches. avgnumgroundportalintaketele: Average of timd.numgroundportalintaketele across matches. avgnumhumanportalintaketele: Average of timd.numnumhumanportalintaketele across matches. avgnumexchangeinputtele: Average of timd.numexchangeinputtele across matches. avgnumreturnintaketele: Average of timd.numreturnintaketele across matches. avgnumgooddecisions: Average of timd.numgooddecisions across matches. 24

26 avgnumbaddecisions: Average of timd.numbaddecisions across matches. avgcubesspilledauto: Average of timd.numcubesspilledauto across matches. avgcubesspilledtele: Average of timd.numcubesspilledtele across matches. avgtotalcubesplaced: Average of timd.totalcubesplaced across matches. avgcubesplacedinscaleauto: Average of timd.numscalesuccessauto across matches. avgcubesplacedinscaletele: Average of timd.numscalesuccesstele across matches. avgallianceswitchcubesauto: Average of timd.numallianceswitchsuccessauto across matches. avgallianceswitchcubestele: Average of timd.numallianceswitchsuccesstele across matches avgopponentswitchcubestele: Average of timd.numopponentswitchsuccess across matches. avgscalecubesby100s: Average of timd.numcubesscaleat100s across matches. avgscalecubesby110s: Average of timd.numcubesscaleat110s across matches. avgclimbtime: Average of timd.climbtime across matches. avgscaletimeauto: Average time for a robot to successfully place one cube on the scale during the autonomous period. avgscaletimetele: Average time for a robot to successfully place one cube on the scale during the teleoperated period. avgallianceswitchtimeauto: Average time for a robot to successfully place one cube in the alliance switch during the autonomous period. avgallianceswitchtimetele: Average time for a robot to successfully place one cube in the alliance switch during the teleoperated period. avgopponentswitchtimetele: Average time for a robot to successfully place one cube in the opponent switch during the teleoperated period. avgallvaulttime: Average time it takes for a robot to fill up the vault. Derived from vaulttimescore and avgnumexchangeinputtele numsuccessfulclimbs: The number of successful climbs across matches. climbpercentage: Climb success percentage of a robot across matches. predictedclimb: Predicted climb success percentage of a robot across matches. soloclimbpercentage: Solo climb success percentage where a solo climb is defined as a robot successfully climbing independently during the END GAME period. activeassistclimbpercentage: Active-Assist climb success percentage where an active-assist climb is defined as a robot successfully climbing by using an active mechanism and given passive aid by another robot. activeliftclimbpercentage: Active-Lift climb success percentage where an active-lift climb is defined as a robot successfully climbing independently while giving aid to another robot s climb by using a passive mechanism such as a ramp. assistedclimbpercentage: Assisted climb success percentage where an assisted climb is defined as a robot successfully climbing while being totally dependent on an active or passive mechanism of another robot for aid. parkpercentage: Park success percentage. autorunpercentage: Success percentage of a robot in crossing the auto line during the autonomous period. allianceswitchsuccesspercentageauto: Success percentage of a robot in placing one cube in the alliance switch during the autonomous period. 25

27 allianceswitchsuccesspercentagetele: Success percentage of a robot in placing one cube in the alliance switch during the teleoperated period. opponentswitchsuccesspercentagetele: switch during the teleoperated period. Success percentage of a robot in placing one cube in the opponent scalesuccesspercentageauto: autonomous period. scalesuccesspercentagetele: teleoperated period. Success percentage of a robot in placing one cube on the scale during the Success percentage of a robot in placing one cube on the scale during the allianceswitchfailpercentageauto: during the autonomous period. allianceswitchfailpercentagetele: during the teleoperated period. opponentswitchfailpercentagetele: during the teleoperated period. Fail percentage of a robot in placing one cube in the alliance switch Fail percentage of a robot in placing one cube in the alliance switch Fail percentage of a robot in placing one cube in the opponent switch scalefailpercentageauto: Fail percentage of a robot in placing one cube on the scale during the autonomous period. scalefailpercentagetele: Fail percentage of a robot in placing one cube on the scale during the teleoperated period. canscorebothswitchsidesauto: Whether or not a robot can successfully score on the alliance switch under the circumstance of random alliance color configurations. didthreeexchangeinputpercentage: Exchange Zone. The percentage of times a robot successfully inputted three cubes in the percentsuccessoppositeswitchsideauto: Success percentage of robot in successfully placing a cube in the opposite side of the alliance switch diagonal to the robot s initial starting position during the autonomous period. cangroundintake: Whether or not a robot can intake a cube from the ground. teleopexchangeability: A weighted numeric measurement of a robot s ability to input cubes in the Exchange Zone. teleopscaleability: A weighted numeric measurement of a robot s ability to place cubes successfully on the scale. teleopallianceswitchability: A weighted numeric measurement of a robot s ability to place cubes successfully in the alliance switch. teleopallianceswitchability: A weighted numeric measurement of a robot s ability to place cubes successfully in the opponent s switch. vaulttimescore: The total time it took/ or would have taken for the robot to completely fill the vault during its Exchange Zone input cycle. maxscalecubes: The maximum number of cubes placed on the scale during a match across all matches. maxexchangecubes: The maximum number of cubes delivered through the Exchange Zone during a match across all matches. avgagility: Average of timd.rankagility across matches. avgspeed: Average of timd.rankspeed across matches. avgdefense: Average of timd.rankdefense across matches. 26

28 avgdrivingability: A weighted average of the R scores of the subjective data points collected by the super scout: RScoreAgility, RScoreSpeed, RScoreDefense. RScoreAgility: The number of standard deviations that the robot s avgagility is away from the competition mean. RScoreSpeed: The number of standard deviations that the robot s avgspeed is away from the competition mean. RScoreDefense: The number of standard deviations that the robot s avgdefense is away from the competition mean. RScoreDrivingAbility: The number of standard deviations that the robot s avgdrivingability is away from the competition mean. dysfunctionalpercentage: The percentage of the robot s matches in which the robot is dead or absent. The sum of the robot s disabledpercentage and its incapacitatedpercentage. disabledpercentage: The percentage of the robot s matches in which the robot begins the match dead or is absent. Average of timd.didstartdisabled across matches. incapacitatedpercentage: The percentage of the robot s matches in which the robot dies during the match. Average of timd.didbecomeincapacitated across matches. Last Four Matches lfmavgnumrobotslifted: avgnumrobotslifted, only taking into account the robot s last four matches. lfmavgnumcubesplacedauto: avgnumcubesplacedauto, only taking into account the robot s last four matches. lfmavgnumcubesplacedtele: avgnumcubesplacedtele, only taking into account the robot s last four matches. lfmavgnumallianceplatformintakeauto: robot s last four matches. lfmavgnumallianceplatformintaketele: robot s last four matches. lfmavgnumopponentplatformintaketele: robot s last four matches. avgnumallianceplatformintakeauto, only taking into account the avgnumallianceplatformintaketele, only taking into account the avgnumopponentplatformintaketele, only taking into account the lfmavgnumcubesfumbledauto: avgnumcubesfumbledauto, only taking into account the robot s last four matches. lfmavgnumcubesfumbledtele: avgnumcubesfumbledtele, only taking into account the robot s last four matches. lfmavgnumelevatedpyramidintakeauto: avgnumelevatedpyramidintakeauto, only taking into account the robot s last four matches. lfmavgnumelevatedpyramidintaketele: avgnumelevatedpyramidintaketele, only taking into account the robot s last four matches. lfmavgnumgroundpyramidintakeauto: last four matches. lfmavgnumgroundpyramidintaketele: last four matches. avgnumgroundpyramidintakeauto, only taking into account the robot s avgnumgroundpyramidintaketele, only taking into account the robot s lfmavgnumgroundintaketele: avgnumgroundintaketele, only taking into account the robot s last four matches. lfmavgnumgroundportalintaketele: avgnumgroundportalintaketele, only taking into account the robot s last four matches. lfmavgnumhumanportalintaketele: four matches. avgnumhumanportalintaketele, only taking into account the robot s last lfmavgnumexchangeinputtele: avgnumexchangeinputtele, only taking into account the robot s last four matches. 27