Measuring and Improving Scout Accuracy

Scouts are instruments#

Every scout has an error rate. Top scouting teams measure it rather than hoping. The goal is not to shame scouts but to find systematic bias and fix it through training and weighting.

Ground truth options#

You need something to compare a scout's record against:

• TBA score_breakdown gives the alliance totals (auto coral count, teleop coral, algae, barge points) for REEFSCAPE. If your three scouts' summed coral matches TBA's alliance coral, the trio was accurate; persistent gaps localize to a station.
• Double-scouting: during practice or a slow event, put two scouts on the same robot. Their disagreement is a direct accuracy signal.
• Match video review: re-scout a handful of matches from the TBA/FIRST video stream as a gold standard.

A simple accuracy metric#

For a scout over N matches, compute mean absolute error on a key metric versus ground truth. Example for L4 coral:

scout_error = average( |scout_L4 - truth_L4| ) over their matches

Also track bias (signed mean, not absolute): a scout averaging +1.5 on L4 is consistently over-counting and can be coached or even corrected with an offset.

Weighting unreliable scouts#

When you must use data from scouts of varying skill, weight their contributions by inverse error so accurate scouts dominate. In practice, weighting schemes can give a proven scout several times the influence of an unreliable one. The cleaner approach for most teams: identify the bottom scouts and either retrain them or move them to lower-stakes tasks (pit scouting, photography) before elims.

Training that actually moves the number#

• Calibration sessions: scout the same archived REEFSCAPE match as a group, then compare everyone's counts to the known result. Disagreements become teaching moments.
• One job per scout: accuracy rises sharply when a scout tracks one robot and ideally one task category, instead of everything at once.
• Rotation with overlap: when scouts swap out, overlap one match so nobody is cold, and keep names on every record (the QRScout scoutName field with formResetBehavior: preserve) so you can trace error back to a person.

Close the loop#

Accuracy work is only valuable if it changes assignments. After day one of an event, rank scouts by error, retrain or reassign the weakest, and put your most accurate scouts on the robots most likely to be alliance captains or top picks. The data you most need to be right is on the best robots, so staff those stations with your best instruments.