-
Notifications
You must be signed in to change notification settings - Fork 6
/
Copy pathmain.tex
557 lines (473 loc) · 30.7 KB
/
main.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
\documentclass{llncs}
% UTF8 support
\usepackage[utf8x]{inputenc}
\usepackage{url}
\usepackage{graphicx}
\usepackage{float}
\usepackage{paralist}
\usepackage[draft]{fixme}
\usepackage{todonotes}
\graphicspath{{figs/}}
\newcommand{\eg}{{\textit{e.g.~}}}
\newcommand{\etal}{{\textit{et al.~}}}
\newcommand{\ie}{{\textit{i.e.~}}}
\begin{document}
\title{Simulation and HRI\\Recent Perspectives with the MORSE Simulator}
\author{
Séverin Lemaignan\inst{1} \and
Marc Hanheide\inst{2} \and
Michael Karg\inst{3} \and
Harmish Khambhaita\inst{4} \and \\
Lars Kunze\inst{5} \and
Florian Lier\inst{6} \and
Ingo Lütkebohle\inst{7} \and
Grégoire Milliez\inst{4}
}
\authorrunning{Séverin Lemaignan et al.}
\institute{CHILI Lab, EPFL, Lausanne, Switzerland
\and Centre for Autonomous Systems, University of Lincoln, United Kingdom
\and IAS, Technische Universität München, Germany
\and LAAS/CNRS, Université de Toulouse, France
\and Intelligent Robotics Lab, University of Birmingham, United Kingdom
\and CITEC, Bielefeld University, Germany
\and Machine Learning and Robotics Lab, Universität Stuttgart, Germany}
\maketitle
\begin{abstract}
Simulation in robotics is often a love-hate relationship: while simulators do
save us a lot of time and effort compared to regular deployment of complex
software architectures on complex hardware, simulators are also known to evade
many of the real issues that robots need to manage when they enter
the real world. Because humans are the paragon of dynamic, unpredictable,
complex, real world entities, simulation of human-robot interactions may look
condemn to fail, or, in the best case, to be mostly useless. This collective
article reports on five independent applications of the MORSE simulator in the
field of human-robot interaction: It appears that simulation is already useful,
if not essential, to successfully carry out research in the field of HRI, and
sometimes in scenarios we do not anticipate.
\end{abstract}
\section{Introduction}
The use of simulators for human-robot interaction (HRI) encompasses a variety of
use-cases, from prototyping through evaluation to anticipatory simulation at
runtime. It however suffers from a specific integration problem: Simulation in
HRI requires to model robots in all their complexity \emph{plus} a mean of
representing and interacting with human agents. We therefore believe that an
important stepping stone for a wider use of simulation in HRI is the
availability of an integrated, easy-to-use framework that can encompass all
currently important use-cases, and that provides an integration interface for
developers \emph{and} end-users of HRI simulation. In particular, we feel that
it must be both easy to install and use, and offer adequate domain abstractions
to facilitate development and integration. This paper presents how recent work
using the \emph{Modular OpenRobots Simulation Engine}~\cite{morse_simpar_2012}
(MORSE, figure~\ref{fig|morse-hri}) attempts to address this challenge.
We will first review the range of current use-cases for simulation in HRI, then
introduce MORSE with a focus on its HRI specific features, and finally
demonstrate and discuss MORSE's versatility through several case studies. The
case studies also illustrate the collective nature of this article: We report on
contributions and experiences in human-robot interaction simulation from five
unrelated projects, conducted by different people in different organizations,
only sharing the MORSE simulator as common simulation platform.
%The sections~\ref{sc:assessment} to~\ref{sc:ci}
%present each of these projects, and try to highlight both the positive
%outcomes of deploying simulation environments for HRI, and the pitfalls and more
%fundamental issues that simulation of human-robot interaction still faces.
\subsection*{HRI and simulation}
\begin{figure}[t]
\centering
\includegraphics[width=0.7\linewidth]{morse_pr2.jpg}
\caption{Simulation and HRI: A PR2 and a human avatar in MORSE.}
\label{fig|morse-hri}
\end{figure}
\subsubsection*{Applications of simulation in HRI}
\begin{inparaenum}[\itshape 1\upshape)]
In the HRI literature, several distinct goals for the use of simulation can be
discerned. Without claim to completeness, we categorize them into \item prototyping,
\item human modeling, \item interactive simulation, and \item anticipatory simulation.
\end{inparaenum}
The most well-known use-case is probably \emph{prototyping}: The use of a
simulator to reconstruct and run experiments in a simulated target situation
prior to real-world evaluation. Apart from convenience, reasons to do so include
simulation of unsafe situations (\eg navigation in narrow
spaces~\cite{sisbot2007human,kidokoro2013will} or
crowds~\cite{henry2010learning}), and exploration of edge cases (\eg humans not
paying attention~\cite{knepper2012pedestrian,guzzi2013human}).
\emph{Human modeling} is one way of realizing human agents in simulation.
\cite{garrell2010model} present for example a pedestrian model which has been
evaluated against a large-scale database of recorded human movements. When
detailed motion or other actions (such as speaking) is required, cognitive
models have often been used~\cite{sun2006cognition}, and recently also in
HRI~\cite{trafton2013act}. It is probably safe to say that such models are
still far from general, but already quite useful for specific situations.
A problem with these approaches is the significant up-front effort required for
modeling. Therefore, some research has explored the use of game engines in what
we call \emph{interactive simulation}: a real human controls a simulated human
avatar interactively. While not fully automated, it allows reliable capture of
interaction data for later analysis. This has been used for a long time in
tele-operation settings~\cite{wang2005validating} and also more recently for
so-called ``crowd-sourcing'' work~\cite{breazeal2013crowdsourcing}.
A very different use of simulation is \emph{anticipatory simulation}. Here,
a simulator is used \emph{at runtime}, to
be able to quickly explore the likely consequences of robotic actions. This
uses simulation engines mainly to support spatial computations, \eg
to compute social metrics such as walking comfort~\cite{kidokoro2013will} or
proxemics~\cite{hoffman2010effects}. The general goal is to enable the system to
choose an action based on these metrics.
We believe it is clear that these use-cases benefit from each other. Particularly
prototyping requires models, which could be manually specified, learned from
real-world data, or learned interactively through the simulator.
\subsubsection*{Simulators for HRI}
Softwares used for HRI simulation are currently fairly diverse, and can be
distinguished by their use-cases. Prototyping work often uses
``standard'' robot simulators such as USARSim~\cite{lewis2007usarsim}
(commonly used for rescue robotics applications but also beyond), or
Gazebo~\cite{Koenig2004} (though Gazebo's human agent support
is currently limited) and MORSE~\cite{morse_simpar_2012}.
In contrast, work in models or more advanced use-cases such as interactive or
anticipatory simulation currently uses custom software -- this is true for all
of the papers cited in the previous sections at least. The pedestrian modeling
community seems to share some tools, \eg the work by Treuille et
al~\cite{treuille2006continuum} is known to have been re-used, but it has no
connection to robot simulators.
Both standard robot simulators and pedestrian simulators use fairly coarse
human models. In contrast, work in the Embodied Virtual Agent (EVA) community
usually provides higher-level functionality, such as simulated emotion dynamics, behavior
generation based on action primitives, conversational dialogue systems, and up
to cognitive simulations. However, integrating these into a coherent system with
an acceptable interface remains challenging~\cite{gratch2002creating}.
As stated before, we think that the integration of these diverse functionalities
into standard robot simulators would be an ideal next step, making specialized
tools available to a much wider audience, and thus likely also identifying new
avenues for improvements.
\subsection*{HRI and the MORSE simulator}
All five projects that are presented in this article rely on the
domain-independent MORSE simulator as simulation platform. MORSE is an
open-source tool developed for academic robotic research with contributions from
over 15 institutions worldwide. It extends the Blender \emph{Game Engine}, a 3D
engine which features shader-based 3D rendering and physics simulation ({\sc
Bullet} physics engine). This allows for semi-realistic simulation of complex
environments. The MORSE components (sensors and actuators) exchange data with
the robotics software via middleware bindings (\emph{Software In The Loop}
architecture). Four middlewares designed for robotics are currently supported,
including ROS and YARP, as well as a generic socket-based protocol. This design
aims at providing a seamless experience when switching back and forth between
the simulator and the physical robot. Standard robotic platforms, actuators and
sensors (more than 50 components) are provided and enable fast creation of
simulation scenarios, while custom components and behaviors can be added via
simple Python scripts.
%Two MORSE features stand out. First, MORSE has been primarily designed with a
%command-line interface, and only features a minimal (and fully optional)
%graphical user interface. This makes MORSE mainly targeted to an academic
%audience, where efficiency and lightness prevail. Simulation scenes are
%actually short Python programs, thus well suited for sharing and versioning.
%This also eases the integration of the simulator into larger development
%workflows, and MORSE is successfully integrated into several continuous
%integration systems (Travis, Jenkins).
MORSE also introduces a concept of \emph{abstraction levels}: sensors and actuators
may expose several levels of abstraction, corresponding to different level of
realism. For instance, users may choose if the odometry sensor returns only a
curvilinear distance, a $dX, dY, dZ$ differential vector, or the absolute
position of the robot (integrated odometry). This allows users that are testing
low-level components to do so, while users working at higher abstraction
levels (typically in HRI) do not have to run full robotic software stacks (and
thus, benefit from a lighter environment) and can work in a more deterministic
environment. This feature can be finely controlled, on a per-component basis.
For HRI applications, MORSE provides a human avatar that can be fully controlled
(displacement, gaze, grasping of objects, interactions with the environment like
turning lights on, opening drawers, doors...) from a first-person-shooter perspective.
This enables the researcher to quickly setup and test human-robot interactions
with a tele-operated human model, hence with realistic human behaviors. As
presented in~\cite{lemaignan2012morse}, the human avatar can be controlled using
a Kinect-like device. The same avatar can also be programmatically controlled
by external scripts, like any robot in MORSE. With standard MORSE actuators like
the \emph{waypoint} actuator, the researcher can for instance pre-define paths
that the human avatar will follow in a simulated environment.
\section{HRI Simulation : Five Scenarios}
To illustrate how simulation can support research in HRI, we present in the next
sections five case-studies. The first three scenarios, \emph{Situation
Assessment for HRI and Simulated Feedback}, \emph{An Expectations Framework for
Domestic Robot Assistants} and \emph{Preliminary Testing of Human-Aware
Navigation Planner} illustrate the typically use-case for simulation: rather
complex virtual environments are created where human presence plays a
central role, and HRI algorithms are tested in a convenient and repeatable way.
Note that, while we introduce here \emph{simulation-only} scenarios, they all
are test-cases of experiments that have been conducted on real robots:
simulation is used here to support real-world deployments.
The fourth scenario, \emph{Data Acquisition through Automatic Scene Generation}
shows how simulation is used as an alternative source of input to train robots to
behave in human environments, and the last scenario, \emph{Automated Execution
of Prototype HRI Experiments}, presents how the simulator can be used to provide
automatic testing of human-aware behaviors, fully integrated to the software
development workflow. Each of the presentations follow the same structure: we
first introduce the scenario, we then highlight how simulation has been leveraged
and its benefits, and we finally mention some of the shortcomings of the tool.
\subsection{Situation Assessment for HRI and Simulated Feedback}
\label{sc:assessment}
When studying human-robot interaction, understanding the environment in which
agents will interact is a key issue. In this first application, MORSE provides a
virtual environment that we use to harness situation assessment algorithms
(performed by a software called SPARK ~\cite{Milliez2014}, for SPAtial
Reasoning and Knowledge) that also include human-centered \emph{perspective
taking}. The robot updates its knowledge in SPARK using its own position, human
position and objects seen through abstracted, symbolic cameras provided by MORSE
(so-called \emph{semantic} cameras). In this particular scenario
(figure~\ref{fig|spark}), the human is sitting in a couch and ask the robot to
bring specific objects that may be in another room (Pick-Place-Carry task).
\begin{figure}[t]
\centering
\includegraphics[width=0.7\linewidth]{morsespark.png}
\caption{On the left side, the MORSE environment ; on the right side, the same
environment, as perceived by the robot in the SPARK situation assessment
module.}
\label{fig|spark}
\end{figure}
\emph{Benefits of the simulation} The direct benefits of relying on MORSE for
the development of the situation assessment algorithms is the low-cost of
deployment (manual testing on physical systems is labour-intensive), as well as
the repeatability of the experimental conditions, important to assess the
algorithmic improvements. Also, relying on MORSE effectively supports
collaboration between the partners involved in this project (MaRDi
project\footnote{\url{http://mardi.metz.supelec.fr}}): our partners are also
using MORSE simulation to test their software and collect data with the same
environment in their laboratory, where they focus on dialog processing. They can
train their dialog system using MORSE feedback to test the robot behaviors~\cite{simparmardi2014}.
\subsection{An Expectations Framework for Domestic Robot Assistants}
\label{sc:expectations}
In this scenario, an apartment is simulated in which a domestic service robot is
living together with a person (figure~\ref{fig|apartment}). A PR2 robot is
controlled via ROS and the {\sc Cram} reactive plan language, which is used on
several other real robots. The robots' duty is to observe the person performing
different activities and detect unexpected situations based on the validation of
different types of expectations~\cite{Karg2013}. The detection of such
unexpected behavior can help future domestic service robots to better assess
situations and adapt their actions to human behavior.
\emph{Benefits of the simulation} The use of the MORSE simulator enabled us to
set up a large testbed by reusing the real robot control layer via the ROS
middleware and easy-to-generate unpredictable human behaviors using the human
component of MORSE. Setting up of such an apartment in a real-world setting,
together with a suitable sensor setup and a reliable robot control, would not
only introduce huge costs but would also be a time-consuming task which can
distract researchers from their actual research focus. The use of the simulated
scenario enables us to gain many insights into the problem domain in a scenario
that would not been possible within our project, while the algorithms were
eventually validated on the real robot, inside a smaller real-world environment.
\begin{figure}[t]
\centering
\includegraphics[width=0.7\linewidth]{morse_apartment.png}
\caption{A simulated apartment with a domestic service robot and a person.}
\label{fig|apartment}
\end{figure}
The human component of MORSE enabled us to test and validate our approach
dynamically in a variety of situations. Since it can be controlled in real-time
like in a 3D computer game while at the same time, a robot can be simulated by
state-of-the-art components, it is possible to generate a multitude of
situations on which the robot has to react. This greatly supported our project
to gain insights about our approach, detect weak points and make improvements.
\subsection{Preliminary Testing of Human-Aware Navigation Planner}
\label{sc:navigation}
To evaluate the improvements in the human-aware navigation planner developed at
LAAS we carried out a user study. An experiment was set up,
where a robot encounters a human crossing its path (at $90^{\circ }$ angle to
each other) while the robot is moving forward to its navigation goal. For
preliminary testing of the planning algorithm, our lab area was simulated in
MORSE. This simulated environment was extensively used
to review the algorithm before it was deployed on the PR2 robot to carry out
real-world experiments (whose results have been published
in~\cite{ThibaultKruse2014}).
\emph{Benefits of the simulation} Development of human-robot interaction
algorithms often require iterative process of prototyping, testing and
reviewing. Setting up and experiment and testing of robot navigation algorithms
especially for large environments involving humans is time consuming and is
subject to availability of lab resources while working in a shared lab between
different groups of researchers. Full support of the PR2 robot model among
others, availability of a human model, and a convenient way of setting up
experiment environment using Blender software were the most prominent features
for choosing MORSE as the simulation environment for these experiments. Since
MORSE already provides ROS bindings for the PR2 robot and human pose it requires
minimal effort to switch between real-world and simulated environments.
As a consortium member in the EU project
SPENCER\footnote{\url{http://www.spencer.eu}}, we plan to develop novel
algorithms for robot navigation in large populated environments, \eg airports.
In the future we plan to use MORSE to simulate such large environment with
multiple human models. This will certainly push the limits of simulation for HRI
and hopefully provide new benchmarks.
\subsection{Data Acquisition through Automatic Scene Generation}
\label{sc:generation}
This fourth study proposes a different perspective on the role of simulation in
HRI: simulating credible human environments to train systems to appropriately
react to them: autonomous mobile robots that are to help and assist people in
care homes, households, and at other workplaces have to understand how human
activities affect the dynamics of objects in the environment. That is, robots
need to know, when, where and how people manipulate objects and how they arrange
and structure them in space. In the context of the STRANDS
project\footnote{\url{http://www.strands-project.eu}} we aim for robots that
understand the long-term, spatio-temporal relationships of objects and
activities of people. In the scenario described here, we looked in
particular at learning qualitative spatial relations of objects on office desks.
As an accurate classification and pose estimation of objects on real-world
office desks is still a challenging and difficult task for current robot
perception systems we acquired a data set of object arrangements using the MORSE
simulator. For this, we first bootstrapped an object statistics from manually
labeled images of real office desks, and secondly, automatically generated a set
of physically possible desktop scenes
(figure~\ref{fig:simulated-desktop-scenes}). Based on the generated data we
learned relational models of object arrangements on desks. The learnt models
enabled a robot to predict the position of an object given a landmark. We
employed these models to effectively guide a simulated and a real robot in
object search tasks and evaluated its performance~\cite{kunze14indirect}.
\emph{Benefits of the simulation} First, the automatic scene generation (made
easy by the use of Python to ``program'' the simulation scenes) and annotation
of object arrangements in simulation is useful for the acquisition of large
amounts of data over short periods of time. The generated data enabled us to
design, implement and to evaluate our methods for predicting object locations
before having a real-world data set in place. Secondly, the generation of object
arrangements can increase the variability of scenes in human-robot experiments
in general. Given the dynamics of objects in the real world it is important not
to oversimplify human-robot experiments in simulated environments but make them
as realistic as possible (in a controlled way). Finally, in future work, we plan
to use the generated desktop scenes in web-applications to crowdsource Natural
Language descriptions of object arrangements and commands for robots from
Internet users.
\begin{figure}[t]
\centering
\includegraphics[width=.7\columnwidth]{figs/scenes.png}
\caption{Automatically generated scenes of office desks.}
\label{fig:simulated-desktop-scenes}
\end{figure}
\subsection{Automated Execution of Prototype HRI Experiments}
\label{sc:ci}
In human-robot interaction studies, robots often indicate behavioral variability
that may influence the experiment's final outcome. However, manual testing on
physical systems is usually the only way to prevent this, but remains
labour-intensive. To tackle this issue, we introduced \emph{early automated
prototype testing}~\cite{2645922} that consists of: a simulation environment, a
software framework for automated bootstrapping of prototype systems, execution
verification of system components, automated result assessment of experiments
and a Continuous Integration (CI) server to centralize experiment execution. In
our setup we bootstrap and execute a simulated prototype system on a CI server
and assess the results in each run. In this particular scenario, a robot must
report the location of a virtual human in a domestic environment. Both the
robot and the human are moving in the scene and meet in front of a table.
The goal of this simulation setup is to incrementally decrease the level of
abstraction until a satisfactory/sufficient degree of ``realism'' to make an
assumption about real world behavior, is reached --- in an integrated and
continuous approach. In order to achieve this goal, we make use of two essential
MORSE features: \textit{a)} the human avatar that can be steered (set waypoints)
interactively via middleware and \textit{b)} a \emph{semantic} camera that
extracts the location of a specific entity in the simulation environment. The
semantic camera is attached to the robot. If the human enters the robot's field
of view, the location is reported and sent via middleware. After each CI run,
the recorded movement trajectory of the human avatar is assessed (plotted) and
archived. We have explicitly chosen to simplify the extraction of the location
of the human to acquire a ground truth in the first iterations of the
simulation. As an example, a system component (running outside of the simulation)
that is intended to classify whether there is a human in front of a robot, by fusing
multiple sensory inputs, can be evaluated based on this ground truth. Subsequently,
we are able to exchange/add diverse virtual sensors, \ie add a simulated laser
scanner to build a person hypothesis for instance, thus gradually develop,
assess and implement more complex scenarios.
\emph{Benefits of the simulation} First of all, the interactive (remotely
controllable) human avatar is useful to include a dynamic, yet not too
realistic, human component in this setup. Secondly, the level of abstraction of different
sensors, \ie semantic camera versus virtual laser scanner enables us to
gradually raise the level of complexity/realism and test different algorithms
based on abstract and almost realistic sensor inputs. Lastly, the chance to
deploy MORSE in a Continuous Integration environment, \ie automatically run
simulation scripts, generates an additional benefit.
\section{Discussion: Towards Unification}
While the five scenarios that we present here implement different use-cases,
they actually cover similar approaches, while relying on the same simulator:
study~\ref{sc:assessment} shows how MORSE can be thought as a computation
engine, \ref{sc:expectations} exploits the human agent in a computer game style,
\ref{sc:navigation} uses MORSE for assessing and tuning the performance of
algorithms, and scenario~\ref{sc:generation}, while somewhat unique, still share
similarities with Garrell et al., in that a model for object positions is
trained on real-world data. Finally, the use-case presented in~\ref{sc:ci}
proposes a different approach, with a focus on continuous testing, and can
arguably be seen as the natural progression of using simulators for evaluation,
extended here to cover HRI scenarios.
From this perspective, one may consider that the experiments recently conducted
in the MORSE community around the simulation of HRI applications constitute the
first steps towards building an unifying platform for HRI simulation, with two
additional features: its \emph{programmability} (simulation scenarios are Python
scripts) and its concept of \emph{abstraction levels} that provides an effective
way to focus simulation on a particular problem by hiding irrelevant simulation
artifacts.
These diverse use-cases support the idea that simulation is not only actually
useful as a support tool for development of human-robot applications, but also
\emph{enables} new research techniques in HRI. \emph{Continuous Integration}
illustrates this point: while HRI experiments are considered as notoriously
difficult to deploy, test and repeat, we show here how a simulator may enable
automated testing of more and more complex scenarios, including long-term
interaction.
Several issues are also raised and must be clearly stated. In its current state,
the MORSE simulator provides only an incomplete model of the environment.
Sounds/speech models are incomplete, and human models do not yet provide good
enough accuracy, both at the level of the user interface (some actions can not
be done with the interactive avatar), and at the simulation level (poor/missing
walking cycles for instance). Finally, the overall convenience of MORSE for HRI
could be improved, for instance by providing more assets (furnitures, objects)
related to human environments. These issues, that are mostly technical and could
be addressed at the software level, show that simulators dedicated to HRI
application still need to mature. In that regard, the next section presents some
of the directions that are currently researched.
\subsection*{The next steps}
Several noteworthy developments related to HRI are currently shaping up in the
MORSE community. We outline below some of them, that suggest new applications
that we believe are relevant to HRI research.
A first line of investigation relates to the procedural generation of a variety
of realistic human models. {\sc MakeHuman} is such an open-source tool that
generates anatomically, kinetically and visually realistic human models. This
software has a tight integration with Blender, and MORSE is soon to provide as
well seamless integration with {\sc MakeHuman} models. This will bring a wide
range of characters to feed the simulations, and extend testing environments
with gender/size/age/skin color variances.
Besides being able to control a human avatar in simulation programmatically and
deterministically, the possibility to automatically generate believable and
realistic crowd behaviors is being explored. In this context, the objective is
to adopt in MORSE technologies previously developed for computer games to generate
trajectories that control the MORSE avatars. Based on the idea of \emph{social
forces}, the work of~\cite{Szymanezyk2012crowd} is to be
adapted to provide believable and realistic movement of several humans within
MORSE. This
implementation would provide a more realistic and dynamic environment to study
human-robot spatial interaction and to provide a testbed for human-aware motion
planning, to give two exemplary use-cases.
Another line of investigation looks at \emph{embedding} the researcher into the
robotic simulation. The purpose of such efforts is to provide a life-like
immersive simulation environment that would allow at the same time ecologically
valid human behaviors and repeatable, lightweight interaction settings.
In~\cite{lemaignan2012morse}, we presented how a human agent could interact with
a virtual robot through a deictic interface based on a Kinect. Two distinct
projects are currently looking into extending this approach, one (at Bielefeld
University) aiming at integrating emerging Virtual Reality devices (like Occulus
Rift) with MORSE, the other one (MarDI project) developing a virtual reality
cave, that include 360° projections and spatialized sound.
Also often suggested, the \emph{on-line} deployment of HRI simulations could
efficiently support large scale HRI studies. The simulator and specific
interaction scenarios would be embedded in a dedicated webpage and users would
control a human avatar from their webbrowsers. This would potentially enable
collection of large behavioral datasets. While MORSE development in that
direction has yet to start, Breazeal et al. presented an initial attempt in that
direction in~\cite{breazeal2013crowdsourcing} and the Gazebo simulator features
a limited WebGL client that act as a proof-of-concept of on-line robotic
simulation.
%Faster-than-realtime simulation, a recent feature of MORSE (currently at testing
%stage), is a last development that can have significant applications for
%HRI\fixme{finish that}
\subsection*{Conclusion}
These examples and ideas hopefully give a picture of the lively landscape of the
``Simulation for HRI'' community, that has built itself around the MORSE
simulator. In the introduction, we mentioned how simulation in HRI had to
address in parallel constraints stemming from \emph{robotic simulation} and
\emph{virtual agent simulation}, while remaining a lightweight, easy-to-use
tool. We are certainly not yet there, much remains to be imagined, refined and
achieved. Yet MORSE is already deployed in several institutions as a platform
that efficiently supports research in human-robot interaction. As an open-source
project, we strive for new use-cases and ideas, and warmly welcome researchers
that would like to join the effort.
\section*{Acknowledgment}
This research has received funding from the European
Union (FP7/2007-2013) under grant agreements
FP7-600623 (STRANDS) and FP7-600877 (SPENCER).
\bibliographystyle{abbrv}
\bibliography{main}
\end{document}