…zur Esri UC nach San Diego. In diesem Jahr wurde Aleksandr Shirokov vom Institut für Geoinformatik (ifgi) an der Westfälischen Wilhelms-Universität Münster als “Young Scholar” ausgezeichnet. Im Folgenden könnt ihr seinen Beitrag lesen. Natürlich wird Aleksandr auch über seine Erfahrungen auf der UC im Juni/Juli 2016 berichten.
Gesture-based interactions with Esri Story Maps in the context of public places
Developed by: Aleksandr Shirokov
M.Sc. (Master of Science) Geoinformatics
Institute for Geoinformatics, Münster
Demonstration of the project:
https://www.youtube.com/watch?v=c03MvuaTROs
https://www.youtube.com/watch?v=U130YNoYYPM
Main Idea
The idea of this project was to develop a gesture-based application which extends Esri Story Maps by allowing users to choose any available story map app and get full control over the way a story is presented, using hand gestures. Potential users are able to perform simple actions, such as pressing buttons, swiping and scrolling pages, as well as more sophisticated geo-related tasks (panning, zooming, changing base map and a certain layer)
Motivation
Several years ago, Esri Inc. launched a framework for web applications called ‘Story Maps’, which aims to inform, engage and inspire people with any story involving maps, location and geography. Even though they are simple, yet powerful tools, there is a great chance that a potential user would consider a certain story as just one of billions of other webpages. However, it is important to remember that storytelling is not just a process of putting and exploring content on a web page, but is a more involved experience, requiring audience, participation and an engaging atmosphere.
In today’s “era of touch”, museums and galleries mostly use either static or touch displays to add the element of interactivity. However, display size constraints and a limited number of people simultaneously involved in an interaction do not deliver the required atmosphere and, consequently, lead to a less optimal storytelling experience. Gestures offer a solution for these problems. Gestures are natural, intuitive, and an easy-to-learn method of interaction. In addition, gesture-based applications do not require a touch area, as the content for the interaction can be displayed on any static display or directly projected on any surface. At the same time, such applications greatly increase the number of people which are simultaneously involved in storytelling process.
Project milestones
Story maps are essentially web applications built using one of the available, predefined templates. Every story map is intended to share not only data, photos, videos and sounds, but also various kinds of spatial information. While simple actions such as clicking or swiping pages can be performed in an intuitive way using hand gestures, slightly complex geo-related task can potentially be more complicated for novel users.
That is why the development was divided into two main steps:
Step 1. In order to understand what kind of gestures are better perceived and used by people, two types of the interface for map interaction have been developed using Kinect V2, a motion sensor, developed by Microsoft, which allows extracting the precise position of each body joint (hands, shoulders, legs, head etc.) in a three-dimensional coordinate space, relatively to the position of the device.
Both versions allow users to perform basic GIS operations, such as panning, zooming and changing the base map and operational layers, but in two conceptually different ways. The first version aims to simulate actions which resemble our everyday activities, while the second version simulates touch screen gestures that people perform while using smartphones or tablets.
The application was available for visitors of the Museum of Communication in Frankfurt in the period from April to August 2015. Consequently, a large-scale user study was conducted, involved more than 100 visitors of the museum.
Step 2. Taking into account the valuable results received from the conducted user study, the final application was developed at the Institute for Geoinformatics, extending every available Story Map template (Spyglass, Playlist, Map Tour, Map Journal etc.) by applying most the appropriate set of gestures for a specific template.
For instance, the ‘Map Journal’ template usually requires a lot of swiping and less precise navigation, while the ‘Spyglass’ template, in contrary, requires a lot of precise movements in order to control ‘spy glass’ object and less swiping between pages.
(‘Map Journal’ – left; ‘Spy Glass’ – right)
Conclusion
The developed application is mainly intended to bring the storytelling experience to a new level, using combining powerful story-creation tools, such as Esri Story Maps, but also to create simple, fun and engaging ways of interactions. Moreover, the usage of the developed applications is not restricted to the museum and art galleries only, but it can be used in variety of different ways, for instance, for education purposes. Seeing a specific map, picture or even complete story in a textbook is one thing. Yet, reaching out and navigating the map, and, more importantly, being engaged in a storytelling process with your bare hands is a completely different experience. Moreover, the developed applications will encourage students to create and share their own stories, while utilizing Esri Software, learning basic geoscience concepts and new ways of human-computer interaction.
I would like to thank Esri Deutschland for selecting this project as a winner this year as well as the Institute for Geoinformatics of the University of Münster, where I have a unique opportunity to work on great projects and ideas along with truly brilliant people from all over the world.
