Motivated by the increasing number of robots that share a space with people, SPENCER will break new ground for cognitive systems in human environments. While there is an increasing trend towards socially aware robots, previous research is still limited in its capacity to perceive, learn and model human social behavior and to use this knowledge to plan appropriate actions in real-time from mobile platforms. SPENCER will systematically address these problems and advance the fields of

SPENCER will extend previous research by considering humans to be more than dynamic objects but as individuals with different attributes, social relations, social hierarchies, and social rules. A fundamental objective of the project is to demonstrate that for robots among humans, social compliance and task-efficiency is not a trade-off but can go hand in hand.

By addressing these problems jointly and in a multi-disciplinary project team, the SPENCER consortium will exploit synergies that will enable us to better understand human-robot relationships in populated environments and to better design effective cognitive systems that operate robustly and safely among humans.

Demonstration Scenario and Applications

The project is also motivated by actual challenges in the aviation industry. KLM, the end-user in the consortium, considers the technologies developed in SPENCER as highly relevant for the area of transfer passenger services. As with other hub airports, up to 80% of passenger traffic at their home base Schiphol in Amsterdam is due to transfer passengers whose efficient handling is a significant operational challenge. An important bottleneck are transfer passengers that have to go through the Schengen passport control in order to catch a connecting flight to Schengen countries. Every day, people miss their connections due to delayed arrivals, short disembark-embark times, wayfinding problems, language and alphabet barriers, or other reasons. Thus, the SPENCER consortium will deploy a robotic demonstrator for smart passengers flow management whose tasks include guidance of short-transfer time passengers from their gate of arrival to the priority lane of the Schengen barrier and mobile information provision. The deployment is an excellent benchmark of the research developed in SPENCER given the demanding nature of airports as highly populated environments and has a large exploitation potential within and beyond the aviation industry.

The SPENCER consortium will integrate the developed technologies onto a robot platform whose task consist in picking up short-transfer time passenger groups at their gate of arrival, identifying them with an on-board boarding pass reader, guiding them to the Schengen barrier and instructing them to use the priority track. Additionally, the platform will be equipped with a KLM information kiosk and provide services to passengers in need of help.

The project is also expected to have an impact on our society by laying the foundation for new studies about robots in everyday life. The resulting technologies and insights will inform the work of organizations, local governments, robot developers and researchers in cognitive systems and social science allowing them to take advantage of these new capabilities in understanding human group behaviors with relevance to such diverse social challenges as international travel, emergency response, eduation and healthcare.

Work Plan

The work plan of SPENCER is organized into the following eight work packages:


Description and Tasks



Requirement Analysis, Platform Specification and Design

Task 1.1: Requirement Analysis
Task 1.2: Hardware Specification, Design and Subcontracting
Task 1.3: Robot Platform
Task 1.4: Software Specification of the Robot Architecture

BlueBotics SA (BLUE)


Far-Range Perception: People and Object Analysis

Task 2.1: People Detection and Tracking
Task 2.2: On-Line Object Learning
Task 2.3: Group Detection and Tracking
Task 2.4: SLAM and Socially Annotated Mapping

Technische Universität
München (TUM)


Close-Range Perception: Human Attribute Analysis

Task 3.1: Human Attribute Classification
Task 3.2: Rough Posture Estimation
Task 3.3: Head Pose Estimation
Task 3.4: Upper-Body Analysis

Rheinisch-Westfälische Technische
Hochschule Aachen (RWTH)


Group-Level Analysis and User Studies

Task 4.1: Behavior Evaluation Through User Studies
Task 4.2: Social Activity Detection
Task 4.3: Social Relation Analysis
Task 4.4: Spokesperson Detection

Universiteit Twente (UT)


Behavior Learning and Planning

Task 5.1: Learning Simple Normative Behaviors
Task 5.2: Motion Planning Under Social Constraints
Task 5.3: Learning Complex Normative Behaviors
Task 5.4: On-Line Behavior Adaptation
Task 5.5: Task Planning and Supervision

Freiburg (ALU-FR)


System Integration, Deployment, and Evaluation

Task 6.1: System Integration
Task 6.2: Data Collection and Annotation
Task 6.3: Final Deployment
Task 6.4: Iterative and Final Evaluation

Centre National de la Recherche
Scientifique (CNRS)


Dissemination and Exploitation

Task 7.1: Dissemination
Task 7.2: Exploitation

Örebro University (ORU)



Task 8.1: Administrative and Financial Management
Task 8.2: Project Status Monitoring

Freiburg (ALU-FR)