One research interest of the Autonomous Robotics and Human-Machine Systems group at the Max Planck Institute for Biological Cybernetics is the human-in-the-loop bilateral control of a swarm of robots. The human operator may be presented with multiple feedback cues, e.g., visual, vestibular, haptic, auditive or even olfactory stimuli. The goal is to find a general methodology on how to best control multiple robots in a remote environment.

Software frameworks simulating a virtual environment are indispensable for this research work. Being able to access all information of an experiment setup is important to gain insights that may prove difficult or impossible to gain otherwise. Furthermore, hardware-in-the-loop and human-in-the-loop scenarios depend on software that is capable of performing calculations in real-time. A process running in simulation must proceed with the same pace as the process in reality.

Software suites capable of simulating multiple robots already exist and are widely used in research. Nonetheless, finding software that meets all requirements, i.e., real-time execution, realistic, exchangeable visual and physical representation, extendable software architecture and full control over inherent information of all simulated robots is a difficult task.

SwarmSimX (SSX) is capable of simulating both the visual and physical properties of robots acting in a user-defined environment in real-time. The software architecture allows for an easy exchange of the underlying libraries used for simulation. Shared modules may be loaded at runtime, extending the simulation with new functionality, without the need of recompiling the whole simulation environment.

Software architecture

The simulation environment can be divided into three main parts. Those parts are the visual representation, physical representation and the creation part. The following figure gives an overview of the elements used in the simulation and how they are related. In the nomenclature of SSX, the visual representation is managed by the RenderEngine. RenderNodes are used to define individual elements of the RenderEngine. In a symmetric way the physical representation is managed by the PhysicEngine and the individual parts are calledPhysicNode. Both, Physic- and RenderNodes can be connected to form tree structures. Child nodes are defined (i.e., represented by position, orientation, mass, etc.) w.r.t. to the parent node to which they are connected.

 The visual and physical representations are related to each other, as the physical representation is used to perform the actual simulation and the visual displays the behaviour of the objects being simulated. This relationship is expressed in the parallel structure of the design layout. At every timestep of the simulation the execution of logical modules is triggered. These modules can perform any kind of calculation and are used to extend the simulation with custom logic or custom functionality. They are called Driver and Sensor.

Drivers represent the behaviour of objects within the simulation environment. E.g. a Driver can represent the attitude controller of a Unmanned Arial Vehicle (UAV) flying in simulation that receives desired angles and adds forces/torques to the physical object being controlled. Another example is the logical module of an automated door that opens whenever an object moves within a certain range and closes after 5 s with no sensor input.

Sensors are used whenever information of objects within the simulation or the simulation itself needs to be retrieved. E.g. a sensor could emulate the expected values of a real sensor. This way a control algorithm implemented for a real microcontroller can be used with this sensor output, without modifying the code or the algorithm of the microcontroller code. Also omnipotent sensors can be implemented that measure the state of all simulated robots in the simulation. This kind of sensor may not exist in reality, but may prove helpful while developing new algorithms.

All those parts are stored together into a single object called Artifact. An Artifact may consist of multiple RenderNodes and PhysicNodes. Sensors and a Driver are not mandatory. It is possible to simulate several different Artifacts simultaneously. Artifacts state the main concept of SwarmSimX. Everything that can be placed in the simulation environment is defined as an Artifact.

Media

Flight of a simulated quadrotor following a given trajectory indicated by a red ball. In the first part of the video additional debug information (e.g., the reference frame of quadrotor) is visible.

References

Johannes Lächele, Development of a Real-Time Simulation Environment for multiple Robot Systems, diploma thesis, Eberhard Karls Universität Tübingen, October 2011.

Citation

If you want to refer to my work, please use the following citation.

@INCOLLECTION{Lachele2012,
author = {Johannes L{\"a}chele and Antonio Franchi and {Heinrich H.} B{\"u}lthoff and {Paolo Robuffo} Giordano},
title = {SwarmSimX: Real-time simulation environment for multi-robot systems},
booktitle = {Simulation, modeling, and programming for autonomous robots},
publisher = {Springer},
year = {2012},
pages = {375--387},
url = {http://laechele.eu/swarmsimx/}
}