Name
Abstract | ||
|---|---|---|
The reproducibility and objective assessment of a robot's manipulation ability is a crucial topic in robotics. The definition of testing protocols and the automation of such test procedures is challenging. Most evaluation protocols in benchmarks and competitions include not completely standardized interactions of a human/referee and are furthermore influenced by partially subjective criteria across different research areas. A promising and more objective solution is to automate benchmarking protocols for a range of reproducible manipulation tasks. We present an approach that is simplified to the essential elements, in which most of the process and evaluation is automated. The proposed system has low price and effort requirements and allows researchers to compare their robots without placing them side by side. To allow a wide range of robots to be benchmarked all parts will be available open source. In our system, an object is positioned in front of the benchmarked robot by a mobile minirobot that localizes itself on a table according to the individual task's predefined positions. By automating most of the process we ensure that the object locations tested as well as the timings are reproducible and well documented. This approach fosters the objective comparability of robot performances. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/ICARSC.2018.8374190 | 2018 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) |
Keywords | Field | DocType |
testing protocols,test procedures,evaluation protocols,partially subjective criteria,different research areas,benchmarking protocols,reproducible manipulation tasks,benchmarked robot,individual task,object locations,objective comparability,robot performances,automated pick-and-place benchmarking system,robot manipulation ability | Testing protocols,Software engineering,Computer science,Automation,SMT placement equipment,Artificial intelligence,Robot,Comparability,Benchmarking,Robotics,Test procedures | Conference |
ISSN | ISBN | Citations |
2573-9360 | 978-1-5386-5222-0 | 0 |
PageRank | References | Authors |
0.34 | 0 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Nils Neumann | 1 | 0 | 0.34 |
| Sebastian Meyer Zu Borgsen | 2 | 18 | 4.01 |
| Phillip Lücking | 3 | 0 | 0.68 |
| Sven Wachsmuth | 4 | 267 | 43.83 |