| Home | Research | Interests | Notes | IMS Home |
This page contains detailed workplans for the VRLU projects.
The goal is to use a PDA as a 3D interaction device. It should be tracked with 6DOF and provide button information using its build in buttons. This interaction device will then act as pen replacement but also be used as a ray casting device to do remote object manipulation.
Tools:
Deliverables:
Details:
See the unified RayPicker for the ray interaction design.
Connection between the Button interception program and OpenTracker will be done as follows. There is iPaq side Java program communicating with a desktop side java program via the serial port. The desktop side program then sends multicast packets conforming to the OpenTracker network protocol which can be picked up by a local OpenTracker instance. This is not only simpler to implement but a less risky development if we start to use wireless lan to communicate with the iPAQ.
See the following OpenTracker source code files, for protocol details :
A simple test OpenTracker file to see if the format and connection is working is :
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE OpenTracker SYSTEM "opentracker.dtd">
<OpenTracker> <configuration> <ConsoleConfig headerline="Multicast Receiver"/> </configuration> <ConsoleSink comment="0"> <NetworkSource number="0" multicast-address="224.100.200.101" port="123345"/> </ConsoleSink> </OpenTracker>
Deliverables:
Details:
The ray picker should use the 2D mode of the unified ray picker node described below.
A combination of these two classes can be defined by switching between these two modes, plus adding a third mode using a 3D position and the camera instead of the 2D screen coordinates. It's interface looks like this :
SoRayPicker {
# general parameters
SoSFBool active FALSE
SoSFInt32 station -1
SoSFNode pickGraph NULL
# pick parameters
SoSFFloat radius 1.0
SoSFFloat nearDistance -1.0
SoSFFloat farDistance 1.0
#modes
SoSFEnum mode RAY # | STICK
SoSFEnum coordinates NORMALIZED # | ABSOLUTE
SoSFEnum input POSE # | POSITION | SCREEN
geometry Group {
DrawStyle {}
Material {}
Cylinder ()
}
# SoSFInt32 rayStation -1
SoSFPath pickedPath NULL
SoSFVec3f pickedPoint 0 0 0
SoSFRotation pickedOrientation 0 0 1 0
SoSFVec3f pickedNormal 0 0 0
SoSFVec4f pickedTextureCoord 0 0 0 0
SoSFInt32 button 0
}
The input enum field defines in what input mode the ray picker works. POSE uses 6 DOF input from the station to generate a ray. This works similar to a laserpointer you hold in your hand. POSITION uses only the 3D position of the station to generate a ray between the eye point and the tracked point. SCREEN uses the x and y coordinate of the station to describe a 2D point on the screen. This is then used together with the user's camera to generate the ray.
The POSITION and SCREEN mode work only if the ray picker is associated with a user by adding it to its user kit. The input from the station is assumed to be mouse coordinates as provided by the StbMouseSource. The field coordinates selects between normalized and absolute coordinates as in the StbMouseSource. This node is only valid if used within an SoUserKit, because the it requires a camera to do its picking. For example, it could be stored in the auxiliary geometry of the user. It will also update the ray of the camera of the user moves. In stick mode the stick should actually be attached to the camera !
The basic behaviour is to use the tracking input from station station and generate a raypick using the given position and orientation. This raypick is visualized by a line that extends from the current position to the picked point or to infinity, if there is no hit. The visualization of the line can be specified in the geometry part. The RayPicker will transform the geometry by scaling it along the X axis and positioning it so that it stretches between the input point and the picked point. It also outputs the found node via the pickedPath field and the found position via pickedPoint. The orientation of the ray given by either the interaction device or the up direction of the camera is output in pickOrientation. The button state of the input station is reflected by pickButton. There will be no output rayStation station. This is done by a special OpenTracker node the takes output from Studierstube nodes as input. In addition to that, the fields pickedNormal and pickedTextureCoord just output information retrieved from the RayPickAction (see there).
If a button is pressed it operates in one of two modes selected by the mode field: RAY or STICK. In RAY mode it behaves in the same manner, if a button is pressed or not, that is it always casts rays for updates to the input station. In STICK mode it behaves like a fixed stick once any button is pressed, that is the current length is fixed and a new position and orientation is computed from the fixed length of the stick and posted via the rayStation.
The ray pick action also needs a scenegraph to operate on. This can either be set explicitly by using the field pickGraph or implicitly by using the head of the path passed via the event traversal. We decided to use the geometry root in a fixed manner. To access the geometry root node use the method StbWorkspace.getGeometryRoot().
The fields radius, nearDistance and farDistance control the respective parameters of a SoRayPickAction. radius defines the size the ray in pixels. nearDistance and farDistance provide a clipping along the ray.
This node takes the output from a set of node fields and passes the data as events into the OpenTracker data flow graph. For each part of the state data class, one can specify a dedicated field of a node by using the def name and field name "nodeName.field". The field is tested to have the correct type for the part. The following associations are defined :
| OpenInventor Field Type | State part |
|---|---|
| SFVec3f | position |
| SFRotation | orientation |
| SFInt32 | button |
| SFFloat | confidence |
The implementation will try to find the correct fields with lazy evaluation so that the fields need not exist at the time the node is instantiated. The node will repeatedly test for the fields to exist and if they have the right type, register a SoFieldSensor on them. It will also check for deletion of fields and return to lazy evaluation again in this case. This node is implemented in the patch 1 for the VRLU software.
The goal is to integrate a Chromium server backend into Studierstube to be able to render the output of any OpenGL based application within Studierstube. The integration should allow as to transform the streamed OpenGL commands so that they are rendered interactively from the correct viewpoint and maybe also involve other input such as lightning. The second part includes writing an OpenInventor texture node based on VNC. It takes the desktop image of a remote computer generated by VNC and uses it as a live texture in OpenInventor. This allows the inclusion of any 2D desktop application into Stb. Together with a RayPicker we could also implement mouse based interaction with the application.
Tools:
Deliverables:
Details:
The Chromium processing endpoints are called Servers. They establish by themselves a connection to the Mothership to get information on connected input clients. Then they setup the network connections to receive the stream of OpenGL commands. This stream is passed on to Stream Processing Units (SPUs) on the server side which then process and render the received stream. We need to figure out how to include and drive this processing from the Studierstube/OpenInventor side and how to synchronize the rendering of the stream with the OpenInventor render traversal.
A proposed design was to run the Ch. server in a dedicated process and modify a SPU to synchronize the rendering with an OpenInventor render traversal. We need to investigate the basic data flow, main loop, possible buffering and synchronization within SPUs.