Collection of Posts by David Kabala

Input

Input for VR systems has primarily been concerned with tracking the position and orientation of users. This information is required for immersive stereoscopic display when calculating the correct viewpoint for each eye. 6-degrees of freedom (6-DOF), 3D position and orientation, data are necessary for this calculation. There are three main types of devices that have been produced for 6-DOF tracking; magnetic, sonic, and image based. The following table is a comparison of the cost of these systems. Recently there has been research into using low cost cameras and visual fidutials in a fully enclosed cave for tracking (Reference VF paper). This technique has been shown to be accurate, with moderate latency. Using visual fidutials is currently limited by the requirement to be used in fully enclosed cave environments, and additional processing on the image data from the camera.

Other than tracking devices, VR environments use common computer interaction devices, the keyboard and mouse being the most prevalent. Touch screen devices, gamepads, 3-degree of freedom trackers (Wiimote, PS3 controller), microphone (audio processing). There are additional interfaces including haptics, smell, taste, and even neural interfaces.

---

HCI of RTSs and console vs pc interfaces(Mouse)

Real time strategy(RTSs) games have historically been confined to PCs. The main reason appears to be that the use of a mouse and keyboard for control are superior to the controls provided by console video game platforms. Notable attempts to bring RTSs to consoles are the Nitendo 64(N64) version of Starcraft and the more recent Halo Wars. Halo Wars has been more successful, as it was designed from the ground up to use a control pad whereas the N64 Starcraft was simply a port of the PC game.

So what is it about a mouse and keyboard that is superior to gamepads for RTSs? I argue that there are two principle ways that separate advantages of the mouse and keyboard. I will break up the discussion of these in to two blogs. First the Mouse:
1. RTSs require very accurate and precise selection of items on screen.
A mouse is better suited for this than a joystick(s) on a gamepad. RTSs require the quick selection/deselection of friendly and enemy units, they also require accurate control of selecting small units within larger groups and accurate placement of commands for selected units. A joystick is limited to, usually, 128(8 bits) of precision on each axis. However modern mice have a much larger range of placement, and greater control over the speed of movement. In the future other interfaces may prove better at this currently then mice touch devices may be very good at this as it allows for direct connection of selection and the display itself.

---

HCI of RTSs and console vs pc interfaces(Keyboard)

Now for the Keyboard.
2. RTSs require a wide range of commands to be quickly executable. US traditional keyboards have at least 101 keys, in contrast to gamepads that have ~16 buttons max. When using a mouse an keyboard at the same time, as is done for RTSs, only one hand is covering the keyboard while the other is covering the mouse. This means that the whole keyboard cannot be covered at once, only ~30 keys are covered at one time. This can also be added to the 2-3 buttons on the mouse. However the hand covering the keyboard can be moved to cover different portions of the keyboard depending on the situation, so all of the keys can be readily available. Given the number of possible unit commands, magic commands, building commands, selection commands, and camera placement commands the ~16 max buttons of most gamepads are inadequate.

---

The software available for developing VR applications can be divided into two categories: code development libraries and end-user applications. The development libraries: Cavelibs, vrjuggler, OpenSceneGraph, OpenSG, and vrTools are targeted to application programmers. The End-user applications: Quest3D, Unigine, Alice, Agent Sheets, and Vizard are targeted for application designers.

end-user applications
Alice
3D programming environment
Focuses on teaching programming
Agent sheets
Create games and computational science applications with music, speech, and visualizations
License restrictions
No VR support
Unigine
Primarily a development library
Unigine viewer allows modification of scene
Licensing restrictions
Quest3D
development tool for creating real-time 3D applications
Licensing restrictions
Expensive
Windows-only

---

Software Licenses

When considering the accessibility of software and libraries it's important to understand how it is affected by the licensing. Bruce Perens describes four main categories of software licenses: proprietary, “gift”, “sharing with rules”, and “in-between” licenses[Parens]. Proprietary software is licensed such that it may not be modified or used in another package, doing so would be copyright infringement. The Open-source “gift” licenses like the Apache license[Apache] allow modification and use of the software in any derivative work including proprietary software. Open-source “sharing with rules” licenses allow modification and use of software as long as the derivative work is also shared. The General Public License version 3(GPL3)[GPL3] is an example of an open-source “sharing with rules” license. Open-source “in-between” licenses like the Lesser General Public License version 3(LGPL3)[LGPL] allow modification and use of the software in derivative work, including proprietary software, with the condition that the original software code be made available with the derivative work.

There may be software that provides the functionality needed in a new application, but because of licensing, that software may not be legally usable. This makes the general use of proprietary software and systems inaccessible to many. But under open-source licenses, there is legally solid ground for users to use, modify, and share derivative work

---

Expertise

The complexity of utilizing the hardware and software necessary for a VR system still requires specialized knowledge. The development of an application’s software has the most effect on accessibility as compared to hardware. This is because there are many different VR hardware arrangements ranging from very complex to support, 6-sided cave, to relatively simple to support, a single computer with attached HMD. However, developing a VR application with VR software is complex for the range of computer and HMD to 6-sided cave hardware arrangements. To utilize the available software requires software engineering, graphics, and VR domain knowledge. Because of this, most VR projects have an additional cost to pay for personnel that have this specialized knowledge.