Dev #04: VR Whiteboards

After a longer break, we are back with our devblog. A lot has happened in the past weeks and months: We tested a first prototype together with students, developed a virtual showroom for and implemented new features, including one of our first reusable Figments: An interactive whiteboard.

Dev #04: Current activities

In the winter semester 2021/2022, a first practical trial of took place at Hamm-Lippstadt University of Applied Sciences – there will be a separate article on this soon. During this trial, we were able to both observe how is used in practice and identify (numerous) errors.

Much of the work in recent months has therefore been concerned with making more stable and robust.

However, to distract us from working on netcode and fixing bugs, we still have some new features planned or in development.

Writing and drawing in VR

With a whiteboard, for example, we have developed a first interactive figment that can be used for drawing in virtual reality. Analogous to real objects, these can be placed freely in virtual space. Users can simply grab the markers or chalk in VR and then write on the whiteboard.

The whiteboard image is synchronized, i.e. all participants can see what others are writing at any time (we describe below that this is not always trivial).

Figments: Whiteboard Variants

The whiteboards – as prepared figments – can be imported into an existing scene or learning module at will and positioned there. We currently have several conventional whiteboards equipped with markers and erasers. In addition, there is a glassboard that can be written on with neon pens, and – very oldschool – a chalkboard. 

Different Whiteboards als GLTF Files, created by License: Public Domain - CC 0

Technical implementation

Under the hood, the whiteboards consist – in addition to the geometry, which is available as a GLTF file – of two essential components, a script for the whiteboard interface (including netcode) and a script for the interaction between markers and whiteboard. These components can be used to implement custom whiteboards or additional functionality in the future.

Script: Whiteboard

The functionality of the whiteboard is realized by a transparent render texture, on which is drawn when a marker comes in contact with it. This happens when a Raycast emanating from the marker collides with the whiteboard. Additional functions of the whiteboard control e.g. the positioning of the marker on the whiteboard while writing or the recovery of markers that have been lost (or thrown away 🙂 ). The whiteboard also generates the materials used for writing based on the color of the currently used marker.

Script: Whiteboard Marker

A raycast is performed from the tip of a marker to determine if it is pointed at or touching a whiteboard. Additional features include customizable pen colors, VR integration (e.g., gripping the pen with the hand or controller), and haptic feedback on contact with the whiteboard.

Functions, bugs and outlook

The features described above are the basis of the Whiteboard feature, but they are not final yet. As can be seen below, there are still one or two edge cases that we are considering in further development, as well as bugs that still need to be ironed out.

Depending on the requirements and the feedback we receive from practical user studies, we may add new features, including saving and exporting complete whiteboard images, for example.

Table of Contents