RoboCup Data Collection and Evaluation

A crucial part of an empirical scientific discipline is a set of methods and practices for evaluation and comparison of proposed models and solutions. RoboCup provides a unique common test scenario for robotics, but its potential is not used to the fullest by far. Conducting games costs a tremendous amount of effort. The scientific outcome, however, is quite limited and often not very conclusive. In most cases only the final score of the games provides slim feedback about the performance of a team. In this project we develop a toolbox (a set of tools) to support collection, organization and analysis of large amounts of RoboCup specific data enabling detailed analysis to promote data driven research and development in RoboCup.

Crucial components of the proposed toolbox are:
  • Automatic recording of game videos synchronized with team communication and game control data;
  • Infrastructure for recording of log files on each robot during the game;
  • Synchronization of local (logs recorded by the robots) and global (game videos) information sources;
  • (Semi) automatic mining in log files;
  • Tool for manual annotation of events in videos;
  • Augmentation of videos with meta information (detected robots).