High School Graduate Refines Gyromouse Interface For Virtual Reality; Pre-teens Play Crucial Role
Matt Duncan was a graduating senior at Yorktown High School in Arlington Virginia when he visited our university for a three-week for-credit internship. The goal was for him to learn about 3D graphic design and interaction design while helping us develop our Virtual Reality (VR) technology. The result was beneficial for everyone. He learned how to build virtual environments and to compare and refine VR navigation and selection techniques though collaboration with users. We got a virtual obstacle course for VR navigation training and a very intuitive interaction for VR navigation with the Gyromouse.
By Matt Duncan, Matthew Kelley and Jeffrey Jacobson
In this paper, Kelley and Jacobson provide some background to put Duncan’s work in context. The rest is the final paper Duncan wrote to complete his internship. His time here was jointly sponsored by Ken Sochats’ Visual Information Systems Center (VISC) and Dr. Michael Lewis’ Usability Studies Lab (ULab), which are both part of the School of Information Sciences (SIS) at the University of Pittsburgh. He used the Virtual Theater (Jacobson 2005c), a multi-screen immersive display at SIS, based on CaveUT technology. The Carnegie Museum of Natural History also made an important contribution through Kerry Handron, who brought middle school students (always under her supervision) who worked with different variations on our controller interaction design.
In the first and second weeks, Mathew Kelley, a PHD student and Project Supervisor at the VISC taught Duncan how to build simple virtual environments for the Virtual Theater. His primary learning exercise was to build a virtual obstacle for navigation object selection training, shown here.
In the second and third weeks, Jeffrey Jacobson, a PHD student at the ULab showed Duncan how to compare the effectiveness of two navigation controllers, the Gyromouse and the Wingman Extreme (described below). The process required repeatedly adjusting the input parameters for each controller to optimize interaction, continually integrating lessons learned from testing with small groups of middle school children. In his final paper, below, Duncan describes the testing and adjustment process and the conclusions it led us to.
Along with the obstacle course, the main technical result of this study is the choice of the Gyromouse over the Wingman Extreme, as well as an optimal configuration for the Gyromouse. We will use it for most of our VR applications in the Virtual Theater and in the Earth Theater, a much larger immersive display at the Carnegie Museum.
During his controller study, Mr. Duncan worked with “CaveUT”, the VR software used in the Virtual Theater at SIS and the Earth Theater at the Carnegie Museum. CaveUT is a set of modifications to the game, Unreal Tournament 2004 (UT2004), which allows it to display a contiguous view across multiple screens. The screens can be arranged around the user, creating an immersive, CAVE-like display. CaveUT is free and open source, available to the public at “http://publicvr.org/ut/CaveUT.html” (Jacobson 2005i). One such display is the Virtual Theater (VT) at the School of Information Sciences at the University of Pittsburgh (Jacobson 2005c). Researchers and developers can use the extensive authoring and programming capabilities that come with UT2004 to develop their own applications (Lewis 2002). The primary authoring tool for UT2004 is UnrealEd, a 3D model editor and programming suite. Any application developed for UT2004 can display in any CaveUT-based composite display, like the obstacle course shown above.
The latest version of CaveUT (not yet released) supports a single cursor, which the user can move across all the screens of the composite display using the main controller device. The user can select an object by moving the cursor “over” it, treating the composite screen like a very large image map on a flat web page. However, the targeting is three dimensional, allowing the user to select the same object from many directions. For example, the image below shows a hawk statue in the Virtual Egyptian Temple (Jacobson 2005e) with the targeting cursor on it. Object selection was a key part of Duncan’s investigations.
Like most large immersive displays, the Virtual Theater and the Earth Theater needed a low-cost controller that would allow the user to stand and walk. One reasonably effective solution is the Gyromouse (Patel 2001, Herpers 2005, Olwal 2002, Hafner 2000, Winograd 2000), a cordless mouse that maps rotation in two orthogonal directions to the standard X and Y mouse movement. Similarly, the Logitech Wingman Extreme provides X and Y plane input from the user by simply tilting it. Along with many buttons, which the Gyromouse lacks, the Wingman’s tilt-input is intuitive enough to be worth serious consideration. Using either device for navigation and object selection in CaveUT only requires attaching it to the server machine and engaging the right options. For the reasons above, Mr. Jacobson chose the Gyromouse and the Wingman Extreme as the two best candidate controllers for further study.
Matt Duncan’s Final Paper
Between May 29th and June 16th this year I did a three-week unpaid internship at the University of Pittsburgh VISC Lab (Visual Information Systems Center), which is headed by Professor Ken Sochats (my sponsor). For the first half of the internship, I sometimes worked under Mr. Sochats’ direction, but more often I worked with Matt Kelley, a researcher at the Lab. Many of the other members of the lab were also very friendly and easy to work with.
The second half of my internship was spent working with Jeffrey Jacobson, a graduate student at the University, who is also the godfather of CaveUT. CaveUT is an open-source freeware used for displaying applications based on the Unreal Tournament engine on multi-screen immersive displays. The primary focus of my internship was a study of navigation control schemes used in the Virtual Theatre (Cave).
In order to begin my study I first had to get acquainted with UnrealEd: the editor that comes with most software based on the Unreal Engine. This Editor allows you to change almost any aspect of the game and allows you to make your own levels (or maps), characters, structures, etc. You can also import files such as textures from other programs such as Photoshop, which was another application I learned during my internship.
Before using the editor I had had experience with another 3d modeling program called Blender. UnrealEd is a bit different from other editors because the work space you are given isn’t just empty space, but is actually completely “mass” and when using subtraction or addition tools (brushes) you make the space that you need, and add in matter that you need within that space.
Using online tutorials I mastered the editor, which was surprisingly easy to learn. Then I started to make an obstacle-course map which would be used to test navigation inside the cave. This obstacle course also has another purpose; it would prepare people for navigation in Jeffrey Jacobson’s Virtual Egyptian Temple. The first course was small and unsatisfying because it was an all-indoor map. The next evolved through four versions, with the final version finished on the Friday (6/9) before testing. Finished Map with path:
The map was built to test the three navigation/control tasks inside the cave: turning/looking around, moving forwards and backwards, and using the cursor to point at and select things. The user begins in a place with three pillars around him: one on the left, one in front and one on the right. They are then asked to look at, and then point at (with the cursor) all three pillars. The final movement control that is taught is moving forwards and backwards.
After the exercises with the pillars and movement the user then goes into the open area in front of them and practices moving around. The next exercise is walking around fences by stopping, turning, and moving forward. After that segment the user has to walk along three s-shaped paths, each becoming increasingly narrower and having sharper turns.
The key for those is to practice turning while moving, and to prepare for the next exercise which is doing a figure eight around two massive rocks, which is a much harder maneuver.
The user then moves on from the rocks and goes into a building whose first feature is a long hallway with three other hallways that intersect with it on the right. At the end of those hallways there are holes on either side of the wall with letters inside and the user must point at them with the cursor and tell me what letters they see. The final obstacle in the course is another smaller figure eight, which means sharper turns making it harder to navigate.
We had 6 or 7 participants run the obstacle course. All the participants were students in grades 6 to 10. For the last four years I have been a skateboard instructor working with kids in the same age groups, and with adults as well. Needless to say I was very comfortable assuming a teaching position during the trials. It has been my experience in teaching skateboarding, that when kids are having fun, they are more willing to learn because the situation is no longer mundane but interesting. I saw this attitude in the kids that were participants in the navigation study. All of the kids were more then happy to learn how to navigate through the cave and none got frustrated with it. Even a boy that could barely navigate the cave still wouldn’t give up. It seemed evident that the more fun a kid was having, the greater his or her success in learning.
The controller that was used by the first three participants was the gyro-mouse:
By holding a trigger on the underside of the mouse one could activate the ability to turn, which would then be done by moving the mouse in the desired direction one wanted to turn/look. The user could move forwards and backwards by pressing the left and right mouse buttons. Finally, to exit navigation mode and to go into cursor mode the user would press the mouse wheel and then move the cursor around the same way they looked around in navigation mode.
The first participant, an eighth-grader, found the navigation to be too slow, especially turning. He found the figure-eight obstacles especially hard to navigate through. After this first test we decided to increase the game speed, and make all forms of movement faster. Before we admitted our next participant, we had an adult come and navigate the course with the new settings. He found the mouse turning easier, but told us that the forwards and backwards speed was too fast, and disorienting. After this we reduced the game speed somewhat, but not to the level of the first trial. Then we doubled the mouse speed.
The next two participants went one right after the other and nothing was changed in between the trials. The first kid was a sixth-grader who was our best navigator. He said he thought the mouse speed was too fast, but when we reduced the mouse speed it seemed that, in retrospect, he favored the faster mouse speed.
The third kid was also a sixth-grader, but he had a much harder time navigating. He tended to fixate on the wrong spot - he would be looking in the corner and trying to navigate when he should have been looking forward. Also, he wasn’t able to master turning as quickly as the others. He eventually achieved adequate navigation skills, but even when we slowed the game speed he showed no real overall improvement.
The second controller we used was this game controller:
The user can move forwards and backwards by tilting the controller forward or backwards. Tilting the controller in the desired direction will turn you to the left and right as well. A new form of movement available to this controller that we didn’t get with the gyro-mouse is the ability to strafe left and right, which is performed by pushing the left or right triggers underneath the controller. Looking up and down with this controller is done by pressing and holding the D-Pad up or down. Finally, pressing the big blue button on the bottom row all the way to the right changes the user to cursor mode. After entering cursor mode, tilting the controller in the direction one wants the cursor to go moves the cursor and one can also still strafe, and look up and down in this mode.
The first participant to use the game pad was a tenth-grade girl. She went through the course with almost no problems. For her, doing a figure eight was hard with the game pad, however she found that the acceleration of the movement being proportional to the amount she tilted the pad was a nice feature (one the gyro mouse does not possess). Afterwards I let her try out the gyro-mouse, which wasn’t on the speed setting I had set after the previous three trials. Because of this, the mouse was more jumpy and much too sensitive, which is why she said preferred the game pad.
Our second volunteer for the game pad was our first participant with the gyro- mouse. Compared to the mouse he said he found the cursor control harder, also he said that trying to not accidentally tilt the controller the wrong way was difficult as well. On the bridge over the pit he had a harder time. Unlike when he used the gyro-mouse and did it all in one motion, when using the game pad he had to stop and turn to go through the obstacle course. Finally the figure eights were harder for him as well, because the turning radius for the controller is much larger then for the gyro-mouse.
The third participant to use the game pad probably had the hardest time of all. She decided to do the course sitting down, which in hindsight was probably a bad idea since it constricts one’s movement. Immediately after we gave her the controller to hold she couldn’t keep it steady, the view started spiraling, and she unintentionally “ran” backwards away from the starting area. Once we got the player situated at the starting point again we continued the test. She took the course at a much slower rate, which was probably the product of being unfamiliar with the controls and “cautious driving”. She had trouble with the pit, and had to attempt it four times. On the fourth time I literally had to hold her hand through it. The reason for this is because she just couldn’t stop the “player” from moving. When asked about her test she said that she found the controls and navigation very hard, but that she still found it kind of fun. This girl used the gyro-mouse at the very end of the trials, and said that it was much easier to use and maneuver with and that she preferred it to the game pad.
Our last participant used the gyro-mouse. She started very well and only fell into the pit once. She had a little trouble on the first figure eight but immediately tried again and succeeded perfectly. When she was done we asked her what she thought about it all. She replied that it was fun and that the navigation and controls were easy to learn and easy to use. Finally she said that she did feel a little motion-sick during the trial but not intensely so.
In conclusion, I would say that the gyro-mouse was the real winner in this set of trials. Despite the few extra navigation features the game pad has over it (strafe and the ability to look up and down while in cursor mode), it just isn’t as easy to use and the controls are much harder to learn. This was my experience while using the controllers and seemed to be the norm among the trial participants.
My internship at Pitt had other experiences woven into it besides the 3D modeling and virtual reality projects. While working at the VISC Lab I got a taste of what working in a research lab is actually like. I participated in many demonstrations in the lab, and also learned how to make contacts and start projects with others. At the end of these three weeks I feel like I’m walking away from this with an understanding of academia that stems not from an undergraduate point of view, but instead from a graduate student perspective. Also, the benefit of knowing that I can go out into the real world and have the self-discipline to do this sort of work, and not slack off or become bored with a career in the subject I’m studying is monumental. The motivation I have received from this internship is certain to be helpful when I actually enter college. I would like to thank Jeffrey Jacobson and the VISC Lab for all the help and guidance they have given me over these three weeks.
Hafner, U., Matthias, B., Magg, R. (2000). Wireless Interaction in Cost-Effective Display Environments. Immersive Projection Technology Workshop (IPT2000).
Herpers, R., Hetmann, F., Hau, A., Heiden, W. (2005). Immersion Square - A Mobile Platform for Immersive Visualizations. Proc. Virtual Environment on a PC Cluster Workshop, Protvino, Russia, 2002.
Jacobson, J., Kelley, M., Ellis, S., Seethaler, L. (2005c) Immersive Displays for Education Using CaveUT. World Conference on Educational Multimedia Hypermedia & Telecommunications, Montreal, Canada, June 27-July 2. http://planetjeff.net/IndexDownloads/Jacobson2005c.pdf
Jacobson, J., Holden, L. (2005e). The Virtual Egyptian Temple. World Conference on Educational Media, Hypermedia & Telecommunications (ED-MEDIA), Montreal, Canada. http://planetjeff.net/IndexDownloads/Jacobson2005e.pdf
Jacobson, J., Lewis, L. (2005i). Game Engine Virtual Reality With CaveUT. IEEE Computer, 38, 79-82. http://planetjeff.net/IndexDownloads/Jacobson2005i.html
Lewis, M., Jacobson, J. (2002). Game Engines In Scientific Research. Communications of the ACM, 45, 27-31. http://planetjeff.net/IndexDownloads/Lewis2002.pdf
Olwal, A. (2002). Unit – A Modular Framework for Interaction Technique Design, Development and Implementation. Master’s Thesis: Dept. of Numerical analysis and Computer Science at the Royal Institute of Technology (KTH), Stockholm, Sweden, executed in the Computer Graphics and User Interfaces Laboratory at the Dept. of Computer Science at Columbia University (CU), New York, USA. http://www.nada.kth.se/~alx/text/thesis.pdf
Patel, H., et al. (2001). VIEW of the future, Information Society Technologies (IST), Technical Report: IST-2000-26089 www.view.iao.fraunhofer.de/pdf/D3_2.pdf
Winograd, T. (2000) Interaction Spaces for 21st Century Computing, in John Carroll, Ed., HCI in the New Millennium, Addison Wesley http://hci.stanford.edu/~winograd/papers/21st/