Classification of Multi Modal Input Data

Idea : present a classification of multi modal input events. Discuss the various properties of these classes and describe possible transformations between them. Explain why this information can help developers to implement better multi modal interfaces with less effort.

Classes could be

• continous time data - typically sampled at regular or irregular intervals
• command data - event occurance signifies the content/action
• state change data - event with data, occurance signifies time dependence, data the content/action depending on current state
• combination of the above ?

Devices can deliver combination of the above data classes. A consciencous discrimination can simplify application development. For example the ARToolkit library delivers location data as continous time data but also state change data by telling you whether a certain marker is visible or not. Some aspects of an application require only one or another of these types of events.

Applications might require certain types of events but devices do not deliver them. Therefore transformations might be necessary. Which transformations are there and how can they be defined and implemented ?