Signpost 2.0 Design

See the Signpost Project Page by Michael Knapp.

A short presentation on the ideas detailed here.

Content

• User Interface components
• Design Ideas
• User Interface Specifications
• Collaboration new !
• Software Design new !

User Interface components

Augmentation

Augmentation is world stabilized graphics that convey some information or highlight features. They should appear overlayed to the real world geometry. This includes

Walls

• transparent
• wire frame
• hidden line
• translucent
• only the current room
• all rooms
• only along the path, etc

Portals

• same as above
• different color from walls

Markers

• for debugging
• as fixed position for navigation hints

Other static geometry

• navigation hints such as red line, arrows on walls or floors, etc.
• maps displayed on walls as user interface to select destination room etc.

Navigation help

• Head stabilized arrows pointing in the right direction.
• analog ones
• only a set of discreed directions.
• Pointing straight to the next portals ?
• augmentation of direction of waypoints (map in the hat feature). not so meaning ful indoors, more interesting outdoors.

Lines along the floor or in the air, throughout the building along the path, bread crumps on the floor, arrows on walls and floor

World in miniature model

• Floating in front (as in Höllerer UIST2002), coming into view when looking down.
• different rendering then augmentation.
• Model clipped to a certain region ( a whole map might be too big), and scalable, clipped part scalable as well.
• This model would be world oriented but positionally head stabilized.
• visualization of user's position (and other users in a shared environment)
• switchable to be presented on the pip as well.

Selection of destination

• from a list presented on the pip
• from the wim model
• from a map rendered to be world stabilized over a marker.

Level of Error for presentation selection

• We should add a dynamic representation of tracking error (by increasing error as long as no new marker has been seen etc.). Then we can switch between different presentation modes such as augmentation, directional arrows and simple WIM models.
• could be visualized by a sphere with changing radius in a WIM model.

Design Ideas

• WIM is rendered in a window with clipping on and (in)visible borders. scale and offset can be manipulated ( by dragging operations ?).
• improve ARToolkit performance by turning of unused markers ?? (might be a challange to implement in the opentracker framework, we will see :).

The main observation is that we have a single source of geometry that is could be reused but needs to be rendered in very different styles. The room geometry needs to be rendered with different styles for walls, portals, markers. Not all rooms should be rendered at all times. Some rendering techniques, such has hidden line and translucent + wire frame require several rendering passes over parts or all of the geometry. Basically we would like to have a matrix of rendering styles and geometry parts to be flexible setable :

To implement this we will have one scene graph that stores the geometry in useful chunks such as rooms, further divided into semantically different geometry parts. Then there will be hooks at the appropriate places where different drawstyles and materials (and other gadgets such as GLTweak nodes to have only z-buffer rendering etc.) can be added. These will be switched depending on the context of the current traversal. Each different representation will use this scenegraph in an appropriate graph (necessary transformations to have the correct display) and set it's context. It can then easily switch between different styles of representation by changing only it's own drawstyles.

The different parts of the user interface such as Augmentation, WIM, way finding and path computation (this includes creating the line for example), should be as independend of each other as possible. Their common denominator is the use of the same geometry to express and compute their functionality. Therefore it appears as a good design choice to factor out this common scene graph and let each component interact only with this common scene graph but not with the other modules (or only as far as necessary). This leads to the following components :

• A Master Scene graph manager. It does no rendering, but holds the central scene graph of the building model.
• An Augmentation module. It renders the building model as an augmentation of the real world geometry.
• A WIM module. It renders the building model as a box floating in front of the user.
• A Singpost / path finding module.  It presents the destination selection user interface and computes the path to the destination. It also supplies additional geometry based on that computation. Any additional information it uses (adjacency graph) is local to this module.
• ....

We opt to implement these modules as single dedicated Studierstube applications. This way we can investigate how to design good interfaces between Stb apps. It should promote the reuse of these advanced interface components in other applications and simplify development as well. The single modules should also be more accessible to other developers than a large black box style application.

Idea : we reuse the one geometry several times for different rendering styles plus selections. We can augment the central geometry and only get a selection of components. that allows us to directly only display a subset such as the current path.

User Interface Specifications

WIM

Floating WIM

• visible part scalable by changing window
• moveable/scalable by dragging some invisible geometry inside the window
• centered around users position or fixed location (by moving the model)
• orientationally world stabilized

PIP

• Toggle Button to switch floating WIM on/off
• Slider to change scale of model within the window
• Push button to recenter around users position
• WIM display - same rendering as floating WIM
• change of rendering style - RadioButtons
  • wireframe
  • translucent walls
  • wireframe + walls
  • walls only for chosen path ?
• ToggleButton for auxiliary geometry on/off

Augmentation

Augmentation

• rendering styles
  • wireframe
  • translucent walls
• selection of rendering
  • current room
  • current portals
  • whole model
  • rooms along chosen path
  • portals along chosen path
  • nothing

Pip

• RadioButtons for nothing - current room - chosen path - everything
• RadioButton for room geometry nothing - wireframe - translucent walls
• RadioButton for portal geometry nothing - wireframe - translucent walls
• ToggleButton for auxiliary geometry on/off

Local Navigation

HUD display

• arrows showing direction - head stabilized
• current room name
• destination room name
• ????

Augmented display

• line - arrows on the floor for the whole way / or only part of that
• render portals to go through

Pip

• selection of rooms
  • via list of rooms
  • entering name gesturekit
  • ?selecting in map? maybe not so useful, maps are big, location of dest. room generally not known !
• destination room / current room (if errors occur)
• togglebutton for augmented display
• togglebutton for hud arrows
• togglebutton for textual hud information
• togglebutton for portal rendering

Map Overview application

Augmentation

• overview map displayed on a wall, for example close to a marker.
• go close to marker, do command, map appears ..., maybe automatically.
• fixed direction up north ??

Pip

• different modes
  • automatic map display if close to marker
  • active trigger

Collaboration application

Only PIP based to send commands to other users

• selection of user to send commands to. the other users are derived from the slave instances of the collaboration app.
• selection of target room via list of rooms or other means
• rendevouz mode, where a path from the current location to the other users is computed, then in the middle a rendevouz point is set.
• change the current rendevouz point (will update other users target as well !).

Collaboration

We want to add the following collaboration features:

Shared Views

A passive form of collaboration will only support shared views. A local user can see the same information as a remote user. Moreover its local display can be controlled by the local user, whereas the data itself is only manipulated by the remote user either implicitly (by moving around) or explicitly (selecting a new path etc.).

• User location : The location of any other user should be visible to the local user in different ways.
  • In WIM with some symbolic representation and labelling
  • Only the room, if accuracy is not good enough...
  • Direction to user - augmented navigation aid ?
  • Augmented overlay - symbolic representation overlayed as augmentation like an XRay view
• Selected Path : The path selected by the user should be visible to the local user
  • on the WIM
  • maybe as arrows etc augmented as well
  • essentially in the same ways as for the local path

Tasks

An active form of collaboration supports the users in completing certain tasks. The following tasks could be interesting :

• Find another User : This essentially is a local path selection where the destination is always the current room of the other user. Basically we have now a dynamic destination room that is updated from the remote users location.
  1. Select remote user to follow
  2. local path will be computed to the remote users location
• Rendevouz : Two users want to meet in the same place. Basically the path finding will compute a destination that has the shortest path for both users. Then the locally displayed paths will update accordingly. It should also allow the users to change the meeting point to a more appropriate location if necessary. This could also be extended to more than two users.
1. select remote user to rendevouz with (some collaboration by other user required ?? maybe not for simple demo)
2. rendevouz point computed
3. local and remote path will by updated to guide to rendevouz point
4. destination can always be changed by both users and will update on both displays.
• Guide User : A user could guide another user by setting the information for her navigation system. This could involve the following actions.
  • Select destination
  • Select source
  • Directly control navigation aids. What aids and how could they be controlled ? Setting a destination could already be used to control the navigation aids, by simply setting always the next room as destination.
  • Set a number of waypoints, could again be simulated by always setting the next destination. Could be implemented fixed by a script or an application at least!
1. Select a remote user
2. change her parameters such as destination, source
3. maybe repeat 2. if some condition holds
• Follow User : select a user to be your leader. Then the navigation system will always show navigation aids to find/follow her. Maybe could be just a dynamic version of Find another User. Or also involve close range navigation aids, such as arrows, augmenting position etc.

Software Design

The different parts of the overall Signpost application will be implemented as individual Studierstube applications that communicate via OpenInventor type interfaces, generally fields and routes. They also display or manipulate a shared state encapsulated in a common scene graph which is reused by each Studierstube application as it needs. The following components exist :

Components

All components share the following design attributes :

• All state is accesible via OpenInventor fields. All user interface state or application state should be exported via fields of an appropriate type. The general application logic is to observe these fields and update the applications scene graph/data according to changes. User interface components should not be observed by dedicated callbacks but rather change these states.
• States that express outcome of computation are used as outputs. These are fields as well, but they need not be observed by the application. They are only set after some computation finishes with the result.
• Any configuration of the components such as names of other applications to attach to, files to read, etc. should by configurable via fields again. They need not be observed for changes, if this is only a start up configuration.
• These 3 different uses of fields should be documented clearly !

BAUKit

The BAUKit application simply stores the central scene graph. This is an elaborate structure of geometry that offers a lot of hooks to render it differently or parts of it. This is detailed in the scene graph diagrams on Michaels web page. Some details of how to influence the rendering of the scene graph :

WIMKit

The WIMKit takes care of the visualization in a World-In-Miniature model. It uses several BAUClient nodes to render the model in a window. The window is either attached to the PIP or floating in front of the user, orientationally world stabilized. We also want to add the look down feature described by Höllerer later. On the PIP it provides options to change the visualization. The model itself in the window can be scaled and dragged to show only parts of the whole model. It has the following fields :

SoSFBool floatingWIM

a flag that denotes whether the wim floats in front of user or is attached to pip

SoSFFloat scale

scale of model in wim

SoSFVec3f center

current center of the model in the wim

SoSFEnum renderStyle

defines how to draw the geometry in the WIM. Possible values are WIREFRAME, TRANSLUCENT

SoSFEnum content

defines what to render in the WIM. This can be either only the current room, all rooms along the desired path or the whole model.Possible values are ROOM, PATH, ALL

SoSFBool auxGeometry

defines whether to render any auxiliary geometry in the common scene graph.

SoSFBool visible

defines whether the WIM is visible or not at all

SoMFString path

sets the rooms that belong to the current path. This is an input field for other components to use.

SoSFString currentRoom

sets the current room of the user. This is an input field for other components to use.

Other future options might include other rooms to highlight as well.

AugmentationKit

The AugmentationKit renders augmentation information, that is overlays that are registered with the real world, such as wireframe models of walls etc. On the PIP it provides options to change the visualization. It has the following fields :

SoSFEnum content

defines which part of the model to render. The options are only the current room, the rooms along the path, or all. Possible values are ROOM, PATH, ALL

SoSFEnum roomStyle

defines how to render the room geometry. This includes not at all, as wireframes, hiddenlines or translucent walls.Only walls, floors, etc. are rendered but no portals ! Possible values are INVISIBLE, WIREFRAME, HIDDENLINE, TRANSLUCENT

SoSFEnum portalStyle

defines how to render the portal geometry of selected portals !. This includes not at all, as wireframes, hiddenlines or translucent. Possible values are values INVISIBLE, WIREFRAME, HIDDENLINE, TRANSLUCENT

SoSFBool visible

switch augmentation of completely

SoSFBool auxGeometry

show / hide auxiliary geometry

SoMFString path

input field for the chosen path

SoSFString currentRoom

input field for current room

NavigationKit

The navigation kit is an application without looks but a lot of brain :). It only renders navigational aids, but does the wayfinding algorithms and similar stuff. Actually the navigational aids could be moved to another application as well. It has the following fields :

Input fields

SoSFString destinationRoom

the destination room, this is an input field used to set the room, the user wants to go to.

SoSFVec3f postion

user position, from an SoTrakEngine

SoSFRotation orientation

user orientation, from same SoTrakEngine as above

Output fields

SoSFString currentRoom

the current room, I think this is an output field. If merged with startRoom it will also be an input field

SoMFString path

output field with path of rooms to follow.

SoSFString portalIndex

index of portal to next room in path

SoMFVec3f linePoints

2 values to draw the stroked arrow geometry between user and portal

SoSFRotation arrowDirection

rotation output for HUD arrow

SoSFInt32 arrowState

switch output for HUD arrow

State fields related to user interface

These fields are not necessary on the kit itself, they are directly provided by button widgets on the PIP.

SoSFBool showPortals

render only the special portals or not. This sets for all visualizations if they render the portals to go through or not.

SoSFBool showLine

show red line through the model, this could also be done with fixed arrows at turning points (where the red line has to bend a lot) !!

SoSFBool showArrow

show the directional arrow, a 2D icon

SoSFBool showHUD

show heads up display with textual information on:

• current room
• destination room
• rooms left to cross

Note: indoor / outdoor navigation has different paradigmas. Outdoor waypoints are main points, indoors areas or spaces are the objects one deals with. There is an obvious duality between the two, but still can we integrate this or not ??

CollaborationKit

This kit implements collaborative features by manipulating another users components. The general design has one instance per user and the master instance runs on the users kit. The slave instances manipulate the other components described before. This is a sort of proxy model, where the remote slave clients act as proxies for the master copy on the users computer. As for each user such a collaboration kit exists, we do not need to switch master properties and it is easily extendible to more users (allthough it might not scale to well to large numbers :). The following figure gives an overview how the interaction between collaboration kits and others will work.

The CollaborationKit also provides a visualization of the user for the other users by inserting an avatar icon into the auxiliary geometry of the other users BAUKit scene graph. It controls this geometry via tracking information and the updates received from the master copy.

It has the following fields :

These fields control the appearance of the user on remote kits and passes information to the remote kits about its state (current room, path etc.)

SoSFInt32 user

the user id of the user it represents

SoSFBool visible

whether the users position should be visible (to others) as avatar

SoSFString currentRoom

the users current room

SoMFString path

the users current path to its destination room.

These variables are used to control other users. The slave copy residing on the machine of the target user will set the information set in the other fields accordingly. The protocol is simple. Any change to targetRoom or targetPath is set for the current targetUser. Therefore you first set the targetUser and then can send several changes by updating the targetRoom.

SoSFInt32 targetUser

user id of the user who's state is to be changed.

SoSFString targetRoom

set the destionation room of the target user

SoMFString targetPath

set the path of the target user (does this make sense, if we can set the destination room already ?)