Schemes for Engineers in Research and Development

Global Research Awards: Profiles

Howell Istance – De Montfort University

Gaze and Gaming: enabling people with motor disabilities to take part in online games and communities

Background

The project began in Autumn 2007 as collaboration between the University of Tampere in Finland and De Montfort University in UK. Howell Istance has spent nearly a year working on the project in the Tampere Unit for Computer-Human Interaction (TAUCHI) at the University of Tampere supported by a Global Research Award. Other sponsors of the project include the Institute of Creative Technologies (IOCT) at De Montfort University, the Academy of Finland, and the European Commission’s Communication by Gaze Interaction (COGAIN) FP6 Network of Excellence.

Introduction

The popularity of Massively Multi-player Online Games (MMOGs) has increased enormously in recent years. World of Warcraft, probably the most popular fantasy role playing game, has 11 million monthly subscribers. Second Life is a virtual community rather than a game where people can participate in a wide range of activities, both social and serious. In one seven day period in March 2009, half a million ‘residents’ logged into Second Life. People with severe motor disabilities can derive much enjoyment from playing these games and taking part in virtual communities. Participation can be challenging and fun, it gives opportunities for social interaction, and the extent of the player’s disability need not be apparent to other players. For some groups of people, eye gaze offers the only input modality with the potential for sufficiently high bandwidth to support the range of time-critical interaction tasks required to play. There has been some work on how eye gaze can be integrated with other input modalities, such as keyboard and mouse, for games playing by able-bodied users, but there has been no work to our knowledge on how far gaze can be used as the only input modality to play MMOGs.

This project is studying the best ways to use eye gaze for the variety of tasks encountered during game play, and how these techniques can be integrated into a single user interface that can be adapted to the needs of individual users playing different games.

Project Activities

There are two main strands of work, the development and testing of specific gaze-based interaction techniques for game playing, and the development of a theory to choose the most appropriate sets of interaction techniques based on an analysis of the tasks involved in playing a particular game.

The overall approach is to use patterns of gaze behaviour to generate keyboard and mouse events, so that the underlying game or program treats them as if they had come from the actual keyboard and mouse devices. As the program that generates these events is independent from the game itself, we can, with certain limitations, drive any game that responds to keyboard and mouse events by gaze input. World of Warcraft and Second Life have been used so far as the main test applications.

We have developed different groups of gaze interaction techniques suitable for different aspects of game playing.

• Gaze gestures: It is important to be able to activate commands while looking in the centre of the screen during periods of game play, such as fighting with other characters. A system of gaze gestures has been developed to enable this and is currently being tested. An example is shown in Figure 1.

Fig. 2. Keypad to support player to player communication.

• Visual keyboard: we use a system of predictive text, similar to the T9 system found on mobile phones to support generating the types of short messages with abbreviations common in game play. The keypad has large keys to make gaze selection easier, a small total area to lessen obscuring the game in progress, and is semi-transparent to allow the player to observe the situation in the game while they create the message. This is shown in Figure 2.

• Locomotion modes: part of MMORPG and other game play is being able to move quickly around in the world while being free to look around while moving. We have developed several different approaches to this. The most promising of these causes keystrokes events to be continually sent to the game when the play looks in different regions of the screen. These regions are not visible to the players during use. We have also overlaid a heat map showing the most common places people looked at during navigation through the world in the game. The sizes of the regions have been adjusted accordingly to avoid accidental operation. These are shown in Figure 3.

  

  Fig. 3. The locomotion mode showing the screen zones and heat map visualization of gaze positions. The ‘W’,’S’, ‘A’ and ‘D’ keys control avatar motion in many games.

• Toolglasses and magnifiers: These techniques involve dropping a transparent panel or ‘glass’ over an object in the world or game. Different commands can be given by ‘clicking through’ the glass at the object beneath. A general problem with gaze control is inaccuracy when the pointer is being controlled by gaze. One version of this technique is a simple ‘magnifying glass’ which enlarges the area covered, making it easier to hit small targets with a gaze-controlled pointer (Figure 4).

  

  Fig. 4. The magnifier glass can be dropped by dwell to a location where a close-up manipulation is needed. The green bar on the left shows the device is in left click mode.

• Fast mode-switching: it is important to be able to use gaze behaviour to mean different things and to generate different mouse and keyboard events, for example: a left mouse click, a right mouse click, start a left mouse drag, make no events. We have developed a means of defining different modes of gaze behaviour interpretation and switching between these using glances off-screen.

The mapping theory part of the project involves studying periods of game play using our own event logging software and eye analysis software to identify significant patterns of tasks and of events. Work is in progress to define mappings between this task data and candidate interaction techniques. This should enable a theory to be built which describes or predicts the suitability of different gaze interaction techniques for different tasks. The ultimate purpose of this is to allow automatic user interface building when a motor-impaired player buys a new game. This will take into account the particular game, the nature of the players motor impairment, their preferences, and the type of eye tracking equipment they have.

Software Release

The first version of software will be released at the end of May 2009 and will be free to download. It is intended that this will run with a variety of commercial eye trackers. We will engage communities of gamers with motor impairments to test the software, which will be actively promoted by the COGAIN Association.

Project Contacts

Howell Istance
Dept. Informatics
De Montfort University, UK
hoi@dmu.ac.uk (after May 2009)
howell.istance@cs.uta.fi (until May 2009)

Steve Vickers
Dept. Informatics
De Montfort University, UK
svickers@dmu.ac.uk

Aulikki Hyrskykari
Dept. Computer Sciences
University of Tampere, FI
ah@cs.uta.fi

Project Publications

1. Bates, R., Istance, H., & Vickers, S. (2008). Gaze Interaction with Virtual On-Line Communities: Levelling the Playing Field for Disabled Users. Proceedings of the 4th Cambridge Workshop on Universal Access and Assistive Technology; CWUAAT 2008.

2. Istance, H., Bates, R., Hyrskykari, A., and Vickers, S. Snap clutch, a moded approach to solving the Midas touch problem. Proc. ETRA '08, ACM, New York, NY, 221-228.

3. Vickers, S., Istance, H., Hyrskykari, A., Ali, N., & Bates, R. (2008). Keeping an eye on the game: eye-gaze interaction with Massively Multiplayer Online Games and virtual communities for motor impaired users. Proceedings of the 7th International conference on disability, virtual reality and associated technologies; ICDVRAT 2008

4. Vickers, S., Bates, R., & Istance, H. (2008). Gazing into a Second Life: Gaze-Driven Adventures, Control Barriers, and the Need for Disability Privacy in an Online Virtual World. Proceedings of the 7th International conference on disability, virtual reality and associated technologies; ICDVRAT 2008.

5. Istance, H., Hyrskykari, A., Vickers, S., and Ali, N. User Performance of Gaze-based Interaction with On-line Virtual Communities. Proc. COGAIN Conference 2008, 28-32.

6. Istance H., Vickers S., Hyrskykari A. (2009) Gaze-Based Interaction with Massively Multiplayer On-line Games, Human Factors in Computing Systems, Proceedings of CHI'09 Conference, Boston, MA, 4-9th April 2009

7. Istance H., Hyrskykari A.,Vickers S., Chaves, T. (2009) For Your Eyes Only: Controlling 3D Online Games by EyeGaze, Proceedings of 12th IFIP conference on Human-Computer Interaction: INTERACT 2009, Uppsala, Sweden, 24-28th August 2009.

Howell Istance
Visiting Professor at TAUCHI
Department of Computer Sciences
University of Tampere, FINLAND

Skip to page footer

Schemes for engineers in research and development Research Exchanges with China and India Distinguished Visiting Fellowship Scheme Research chairs Research Chair in Emerging Technologies Senior research fellowships Leverhulme Trust Senior Research Fellowships MOD Research Fellowship Research Fellowships Newton International Fellowships Engineering Enterprise Fellowships