The chess automaton with a life-size mechanical “Chess Turk”, sitting at a table and playing chess against challengers, was presented in Vienna at the court of empress Maria Theresia by baron Wolfgang von Kempelen in 1769.
At the beginning of every show, all of the doors of the chess-table were opened and gave insight into the mechanical interior of the table. Nobody was able to see the human chess-player inside who hided perfectly behind the mechanics and the drawer of the chess-table. With complex and exact movements of its mechanical arm, the Chess Turk carried out the moves in a very realistic way.
Figure 1: The historical automaton
Kempelen went on tour with his Chess Turk and achieved international success. Only excellent chess-players hided inside the chess-table, therefore most of the games were won by the Chess Turk. The fact that the automaton lost a game sometimes made it even more popular. Many authors and newspapers tried to find explanations of the automaton’s function principle.
Twenty years went by until the mystery was considered to be solved almost completely. Even Napoleon had played against the automaton – and lost. After Kempelen’s death (1804), the Chess Turk was in possession of the inventor Maelzel primarily. He showed the automaton in Europe until 1825. At that time, too many people knew the solution of the mystery from publications. So Maelzel went to America where he had success with the Chess Turk again. But in contrast to Europe, here show business already was in high gear. Other enterprising people made copies of the automaton soon, and Maelzel had to leave the Broadway for other American towns. The last successful shows held by Maelzel took place in Havanna in 1837. Unfortunately, in the end the automaton burnt down in a museum in Philadelphia in 1854.
During a game of chess, the human player inside the table got light from a candle. The chessmen were magnetic inside. When the other player lifted up a chessman, a metallic pad fell down along a twine from the underside of the corresponding field. And when the person put down the piece somewhere else, the corresponding metallic pad moved up along its twine. In this way, the chess-player inside the table always knew which moves were done. He did them in parallel for himself on a small chess-board he had in front of him.
Through very precise mechanics, his own arm and finger movements were transmitted to the Chess Turk at a larger scale whenever it was his turn to make a move. This kind of power transmission was unprecedented for the state of technology at that time.
On the whole, the demonstrator “Turkish Chess-Player” for the Technical Museum Vienna shall represent a modern reproduction of the historical automaton. Visitors shall be able to play chess against the Chess Turk. The Virtual Showcase application shall give insight into the function principle of the historical automaton as well as into the technological background of the demonstrator itself. Finally, the installation shall reflect society’s attitude to technology, as it did in former times.
Besides, if the Virtual Showcase setup is kept as compact as possible, the Chess Turk can even go on tour as in former times.
The visitor of our Virtual Showcase demonstrator will find a kind of big table with a truncated pyramid mounted onto the table top. He also sees a display integrated in the front of the table that shows some mechanical equipment, and then some weird masks lying around. People hided their eyes behind such masks at masked balls in former times… He takes a mask and looks at the front of the pyramid. Suddenly he sees a lively Turkish man in 3D, sitting at the table and inviting him to play a game of chess. There is a hole at the bottom of the pyramid, and as soon as the visitor puts his hand into it, a virtual hand follows his movements. Easily he manages to lift up one of the virtual chessmen and to make his move.
Figure 2: The modern Turkish Chess Player
Another visitor joins. He also takes a mask and watches the run of the game. Then he takes a closer look at the other sides of the installation. Amazing, one side view of the pyramid shows the mechanics of the Chess Turk’s arm while the Turk is playing, and that as plastically as if the visitor could touch it. There is another display integrated in the side of the table. Astonishingly, it shows a real human acting inside the table. What will the other side show? Again the visitor can watch the Chess Turk play against the virtual hand of the other visitor, but here the function principle of 3D computer graphics becomes clear. The Turk’s 3D mesh and some bones behind are shown.
Meanwhile the chess playing visitor is stunned: He has lost control of his virtual hand that now seems to play chess against the Turk all alone, and the game runs faster and faster…
The Chess Turk and every other 3D content that happens above the chess-table will be presented to the visitors from three sides of the installation through three different passive stereo-projections onto a projection pyramid and without head-tracking.
At the moment, it is not intended to show the back side of the chess automaton.
In the suggested scenario, a truncated pyramid seems to be the most intuitive solution in providing the projection planes. Projections will take place onto the front, left, and right plane of this pyramid. The back side will not be used and can be a vertical plane. The front plane shows a hole at the bottom into which the user can put his hand in order to play chess against the Turk (interaction hole).
The three projection planes will be non-transparent. The edges of the pyramid could be framed, e.g. with wood, because no seamless transition from the front view to one of the side views is realized.
One great advantage of a pyramid in comparison to vertical projection planes is that in our case, a pyramid approximates the dimensions of the projected 3D scene best. Thus, the visitor can bend over the chess-table without knocking against a wall of glass with his head, as he would be able to do in case of a real game of chess.
The front plane will display the Chess Turk, the chessboard, the chessmen, and a virtual hand representing the user’s hand. The 3D scene will be rendered for an average viewpoint of a user who is sitting in front of the table. A sitting posture of the visitor shows an advantage in comparison to a standing posture because differences in the heights of sitting visitors are decreased.
Figure 3: Projection onto a truncated pyramid
The side planes will display basically the same content as the front plane, but a slightly different content layer and rendered for an average viewpoint of a user who is standing to the left or to the right of the table.
For those visitors, who are just watching the whole scenario, an exact viewpoint is less important than for a visitor who is interacting with the 3D scene.
The number of projectors used, the direction of projection (back or front) as well as the projection angle and the possible need of mirroring projections still seem to be open points that will have to be worked out with BAR.
Back projection from inside the chess-table seems to knock out the proposed user interaction that should take place best through an interaction hole as described below. The user would of course cross the projection rays with his lower arm during interaction.
No essential reason for the use of transparent projection planes has been found for this demonstrator of the Virtual Showcase project:
One could use real chessmen for the playing visitor and the Chess Turk and move those of the Chess Turk with magnetism, or one could use virtual chessmen for the Chess Turk and real chessmen for the visitor. In both cases, knocking out a chessman of the visitor’s real pieces constitutes a problem.
Furthermore, magnetical movements of the Chess Turk’s real pieces would not correspond to the historical chess automaton where the Chess Turk really lifted up the chessmen and put them down at another place. The next problem is given by the fact that real chessmen will easily get lost in a museum.
Exact merging of real and virtual chessmen is hard to realize because of the small dimensions of chessmen and fields, and very good tracking and calibration technologies are required.
Due to the mentioned facts, augmentation of real elements with virtual projections does not seem to make sense for the proposed setup.
The most augmentative way of interaction with the 3D chess scene is given through a hole in the front projection plane, which we want to call “interaction hole”. The visitor can play chess against the Turk through an appropriate interaction of his hand and lower arm within this hole. The horizontal movements of his hand (left-right, back-forth) will be followed accordingly by a virtual hand in the 3D scene projection.
For the proposed interaction purpose, the hand can be tracked optically through a single camera installed inside the pyramid and that at its top. Since there will be quite dimmed light within the pyramid, either a light source has to be installed, or one will have to use an infrared lens and infrared LEDs to assist image processing. It is a great advantage for the image analysis that the background of the real hand, which in our case is the table surface or a real chessboard, always stays static.
Figure 4: Visitor interacting with the 3D scene
One can imagine the correspondance of the virtual hand to the real hand as to be similar to the correspondance of a 2D screen cursor to a mouse device.
A critical point seems to be the recognition of the user’s intention to grab/release a virtual chessman. It is clear to the system that a lifting operation has to be followed by a release operation and vice versa. Thus, for lifting up a chessman, the same recognition solution can be used as for putting down the piece. One possible solution is given through appropriate image analysis, using the fact that the silhouette of a hand with thumb and index finger close together (grabbing) is different to that of a relaxed hand.
In case of a camera mounted on top of the pyramid, the user’s hand has to be oriented in a way that relative movements of thumb and other fingers are recognized from above. If the virtual hand shows the appropriate orientation (which will always stay the same) to the user, his willingness of imitation will guarantee proper interaction.
The major advantage of this interaction technology is that it works without the need of contact and that grabbing/release operations can be transferred to the system in an intuitive way.
A technical solution for the tracking of the user’s hand interaction that is provided as an off-the-shelf solution is given through a touch pad which is integrated in the table surface. The position where the user is touching the pad is assigned to the horizontal position of the virtual hand. Grabbing/release operations are communicated to the system through pressing the touch pad.
This solution is not as intuitive as optical tracking, but here grabbing/release can be identified more clearly.
The user’s grabbing of a chessman is carried out by the virtual hand, which lifts the piece up and follows the user’s hand with that piece to a place where he wants to put it down. For this purpose, there is no need of tracking the vertical position of the user’s hand. The chessman that shall be lifted and the field where it shall be released are defined through the horizontal position of the virtual hand, in correspondance to the horizontal position of the user’s hand. Some highlighting of the chessmen (grabbing intention) or fields (releasing intention), correspondingly to the position of the virtual hand, could assist the user in the choice of the desired pieces/fields.
Since the vertical position of the user’s hand is not taken under consideration for the virtual hand, there is no problem of any occlusion of the virtual hand and the chessmen. Once the choice of a certain chessman or field is specified through the horizontal position of the user’s hand and his grabbing/releasing interaction, picking up and putting down the piece by the virtual hand has to be carried out properly by the animation system.
Appropriate interaction as described above can only be achieved if the visitor takes a sitting posture in front of the interaction hole. In this case the user can bend his arm sufficiently in order to reach the whole virtual chessboard in a comfortable way.
Since the virtual hand corresponds to the visitor’s real hand like a mouse cursor, the dimensions of the real chessboard area within the interaction hole and those of the virtual chessboard need not be exactly the same. But the more realistic the correspondency, the more intuitive is the feeling of 3D immersion to the visitor.
A real chessboard without chessmen can be put on the table in order to be seen if somebody takes a closer look into the interaction hole of the projection pyramid. This could serve as a kind of magic or surprise to the visitor. However, augmentation with the projected chessboard is not necessary and is therefore not planned.
In the sides of the chess-table, appropriate displays will be installed. They will be used to show the technical interior of the table and the human chess-player operating inside through videos and still images.
Since the scene is rendered from three different viewpoints - one for each side of the installation - and no seamless transition from the front view to one of the side views is necessary for the visitors, the idea came to our mind to show different layers of 3D content for the three viewpoints.
In the front view, the virtual Chess Turk will be presented to the chess playing visitor, and to others who watch the installation, in a way that imitates its original appearance.
A display that is integrated in the front of the chess-table will show the mechanical interior life of the table as it was presented to the audience in former times. In those days, the front doors and the drawer were opened at the beginning of the game of chess, and the spectators could only see the mechanics, but not the human chess-player who was acting inside the table.
One side view of the installation can show how the chess automaton worked in former times. For example, the mechanics of the Chess Turk itself, particularly of its arm, can be shown. Additionally, a display integrated in the side of the chess-table can show the human chess-player acting inside the table through a video showing a “real” person.
The other side view can show how the installation works today. For example, the Chess Turk can be shown as a wireframe model with bones inside that are used for animation. Additionally, in the case of optical tracking of the user’s hand, the camera mounted on top of the projection pyramid can be shown within the 3D scene. Thus, the interaction technology is explained to the visitors.
For both of the side views, a kind of slider can be used to move between the described content layer and the “normal” view that corresponds to the appearance of the Chess Turk as it is seen from the front.
The virtual hand representing the chess-playing visitor could get out of the user’s control after some amount of time and continue the game against the Chess Turk all alone, maybe faster and faster. This shall symbolize the danger that man may lose control of technology one day.
In order to bring the irritating run to a good end, the Chess Turk will be the winner though. Thus, technology has tried to take over control but has no chance in the end.
Beside the Chess Turk, the virtual lower arm representing the visitor can also be designed in a highly appealing way. It could also show some old textile or jewellery convenient for that time. The virtual hand will be one of the most important features to turn the scenario into an augmented reality installation.
The polarisation glasses that are necessary for watching passive stereo-projections could be designed as masks that people used on masked balls at that time.