Towards a stable release
Our commitment is to deliver a framework that enables you to create interactive and immersive learning experiences. Regrettably, certain unforeseen bugs have surfaced during the final phases of our launch preparation that could impact your experience. To uphold the standards we’ve set for Figments.nrw, we have chosen to postpone the release briefly, for approximately two weeks.
We are dedicated to rectifying these issues diligently even after the funding period of our research project AR/VR.nrw ends. We hope that Figments.nrw, when it becomes available, will meet and surpass your expectations. We apologize for any inconvenience this might cause, but we’re confident that this extra time will allow us to address some of these bugs, give the user interfaces a final polish and create additional helpful content on how to use and implement Figments.nrw on this website.
Why do we delay?
There are two main reasons for the delay:
- Our goal is to enable collaborative learning in immersive worlds. For this, we implemented methods for network communications, scalable server orchestration and voice chat. While we used and tested these extensively during development, we transitioned parts of these methods to more open and freely usable counterparts in the last year. Taking Voicechat as an example, we used Agora Engine during development, as this commercial solution offered a cost-free service through it’s free plan, which was very helpful to get our first prototypes running as quickly as possible. As this solution would not scale well with the growth of Figments.nrw (as we would likely exceed the volume the free plan covers), we transitioned to WebRTC in preparation for the launch. This was also a needed step to make the later release of our source code free of any commercial dependencies.
Additionally, while the primary intention at the start of the project was to build a pc-based virtual reality application, we learned in the last years that there is a high demand for applications that run on stand alone headsets. This alleviates costs involved when using virtual reality in groups, as one does not need a high end pc, and makes preparing and running a virtual reality seminar easier as well, as the often cumbersome setup-step of getting all pcs, headsets and their connections running is no longer needed. To realize this we changed the base software architecture to support android-based stand alone headsets and implemented this exemplary for Pico devices. Changing such a large part of our core components this late was a risk we were willing to take, seeing as the benefits of this would allow for scaling Figments.nrw in the future and providing you with a version of Figments.nrw you can simply install on your headset.
While testing Figments.nrw in networked sessions over dedicated servers or dynamic kubernetes clusters some bugs emerged in these fields that we did not anticipate and that do not manifest when using Figments.nrw locally or as a single player. To fix these bugs and provide you with a bug-free collaborative immersive experience, we need some additional time.
- We built a solid foundation for immersive learning applications on the basis of being open and free from the ground up in the last three years (which in of itself were marked by their own challenges outside of the scope of our project). While we built our features, in the project group or through one of the many contributions (see spotlights), the first goal was always to get a stable and reliable tool, with aspects like user interfaces being of a lower priority. Through user studies and through first practical applications (e. g. a Fork of Figments.nrw is being used in the project FoProVR and maintained by Nils Mack at the University of Wuppertal) we found that the experience of creating virtual and immersive worlds lacked a coherent design language and user experience. Over the last months we went over all our features with a fine comb, built a (hopefully) coherent user interface design pattern and implemented this for almost all of our tools. To refactor the remaining features (e.g. the editing of virtual spaces and linear learning contents) we need a little bit more time, but hope that this provides you with a seamless experience in the end.
We are very sorry for any inconvenience this delay causes.
Feature Spotlights
We would like to shine a spotlight on some of the key features of Figments.nrw, features that were either directly developed by our core team or were only possible through student contributions and the research assistants that supported us on our journey. To give all of them a place in this spotlight, please find following a non-exhaustive list of all the contributions we received over the last three years.
The list is sorted alphabetically and does not reflect the size of the contribution or the role in the research project AR/VR.nrw – we are deeply grateful for each and every one. Thank you all!
- Daniel Bachmann: Partial Project Coordination, Visual Computing, Software Architecture, Rendering, File IO, Learning Spaces and States, Wrist UI
- Sevinc Eroglu: Authoring Tools, Natural Interactions, NVPL-Implementation, Virtual Camera
- Christian Dominic Fehling: UI/UX (Desktop UI, XR UI), Didactic Design, Transfer, Showroom, 3D Asset Generation
- David Gilbert: Netcode, Natural Interactions, Pico/Android Port, File IO, Git CI/CD, Infrastructure, System Administration, Server Orchestration, Scene Travel
- Christian Helwig: Personal Space, ReadyPlayer.me Integration
- Anton Kindsvater: Crystalline Structure Importer, NVPL-Implementation
- Nils Adrian Mack: Initial Android/Pico Port, Fork-Maintainer FoProVR of BUW
- Dr. Jens Maiero: Project Coordination, Audio Annotations, File IO, Quick UI
- Dr. Sinja Müser: Evaluation, Transfer, Didactic Design
- Vanessa Speeth: Wrist UI, Player States, Showroom
- Felix Voigtländer: Node-based Visual Scripting (NVPL)
- Nico Walheim: Grab Mode Navigation
- Sven Warmuth: Development Support, WebRTC, User/Role Management
- Sebastian Wiederspohn: XR Interactions (Ray, Direct, Poke), Interaction Networking, Object Highlighting, Permission Management
Importing Assets Seamlessly
One of the core features of Figments.nrw is its ability to seamlessly import assets from both your local disk and webdav servers like Sciebo.nrw. Experimentally, we tested if third-party platforms could be integrated as well. For this we implemented the Sketchfab API. As this currently depends on our private API access, we postponed the official support for this, but plan to provide documentation on how you can build this into your own version of Figments.nrw at a later date.
Whether you’re working with 3D models, images, slides, audio files, or video clips, our software empowers you to effortlessly integrate these elements into your virtual reality environment. This capability not only enhances the visual appeal of your immersive worlds but also facilitates a more engaging and enriching learning experience.
At the start, Figments.nrw supports the following types of assets:
- 3D-Models (.glb, .gltf)
- Avatars (ReadyPlayer.me)
- 3D-Sketches (native Figments-Format)
- Audio-Files (.wav)
- Texts (.txt)
- Images and Presentations (.jpg, .png)
- Crystalline Structures (.cif)
Conceptually FileIO is one of the core components of figments that integrates deeply with other systems. We have built this system to be as extendible as possible: As such the support for crystalline structures in the .cif-file-format was added at a later stage in the context of a student project. We hope that this extensibility allows us to import more files later, from other common formats for the existing file type to whole new types of content.
Natural Interaction and Navigation
Source of the 3D-Model shown in the video is Smoggybeard | Solar System Model (Orrery)
The virtual worlds we create inside of Figments.nrw are typically larger than the physical space we have available, so it is not always possible to benefit from room-scale navigation metaphors. While we prefer to naturally move around in our worlds, we implemented ways to navigate larger spaces as well:
- Natural movement: Simply move and your avatar will follow. This in combination with teleportation and snap turn is our default navigation scheme.
- Teleportation: Use the analog stick or touchpad of your controller to aim and teleport to distant places in your virtual world.
- Snap Turn: Simply tap the analog stick left or right to turn yourself around.
- Flying: You can switch from teleportation to flying in the locomotion settings. Afterwards, simply use the analog stick and your hand to control in which direction you want to fly.
- Grab Move: While teleportation and flying allow a fairly comfortable navigation through small and medium sized environments, for even larger spaces we preferred a faster navigation method. Grab Move literally allows you to grab the world and either drag yourself through it or scale the world according to your needs. Need to visit the other end of the world? Simply scale the world to the size of a marble, turn it around and scale it back up. This mode can be used in addition to teleportation or flying.
- Comfort Features: Especially navigating via Fly or Grab Move can induce motion sickness. To alleviate this you can enable some helpful comfort features in the locomotion settings, e.g. constant or dynamic vignettes that reduce your field of view while navigating or disabling the rotational component of Grab Move.
Creating Interactions with Visual Scripting
Figments.nrw empowers you to become the architect of interactions within your virtual environments using visual scripting. By adding nodes to 3D objects, you can intricately design their behaviors and interactions. From making objects grabbable to defining logic-driven behaviors and object-to-object interactions, you have the tools to craft dynamic and responsive virtual worlds. This feature opens up a realm of possibilities for creating immersive scenarios that captivate learners and foster active engagement.
This experimental feature currently enables you to create one or multiple small programs inside your virtual scene. Each program can contain a multiple of nodes with different functionalities. We are in the process of writing a dedicated blog post about this, so feel free to check back again at a later date.
Thanks to Felix Voigtländer, former student of the RWTH, for his excellent contribution!
Crafting Spaces and Learning Modules
Unlock your creativity and build interactive virtual worlds using the comprehensive suite of tools offered by Figments.nrw. Beyond importing assets and scripting interactions, you can design entire spaces and learning modules that seamlessly integrate into your educational journey. Define states for these spaces, ensuring a coherent progression throughout the experience. Our software empowers you to structure your content effectively, enabling the realization of immersive virtual reality learning and training environments that are both informative and captivating.
Internally we named this feature set our “Power Point”-metaphor, as we tried to make the creation of dynamic virtual environments as easy as possible. Truth be told, it is still more complex than building a power point, but it takes a lot of mental load off your mind and frees you from creating whole new applications with large overheads all the time.
Simply build your learning space by importing and arranging 3d-assets (this is somewhat akin to a slide master) and then create new states for this environment. Each state can contain modified assets of the base state, new assets or deletions, and your learners can traverse this learning content freely. This allows for the replication of mainly linear processes and activities: We hope to be able to explore this in future endeavors further, for example for the creation of interactive non-linear media.
Collaborate with others
Figments.nrw provides support to run gameservers on many different backends. This allows for multiple users to experience shared immersive worlds. With the right setup these worlds can become persistent as well, which allows the creation of a virtual campus. Let us know if you plan to build something like this!
The most common supported server-solutions are:
- Local Server / PC: This is the simplest case and only requires you to either run the Figments.nrw client in localhost mode (accessible via the Settings) or install the linux server build on a local machine. Both cases share similar limitations, as you and all your clients need to be in the same shared network and the Firewall and Ports need to be configured accordingly. From experience this is not a feasible solution in university networks or eduroam directly, but can be setup fairly easy in your own network.
- Dedicated Server: This solution is similar to the use of client or server binaries, but provides you with a preconfigured docker image. This containerization allows for a more flexible control over installation and updates of your server, as it can be integrated in existing git workflows.
- Kubernetes Cluster: For scalable and persistent immersive worlds Figments.nrw can be run on Kubernetes clusters, e.g. an environment of many containerized nodes, each capable of running the aforementioned docker image. We are using the open source Agones framework to facilitate server orchestration, so your server needs to support this. Setting this up is a little bit more involved than the other solutions, so we will provide a dedicated support page with more instructions at a later date.
Desktop Mode
In our initial concepts we saw Figments.nrw as a virtual reality application first and foremost. This has not changed, but we identified the need to enable users that cannot participate in immersive experiences to join their peers in these virtual worlds. In addition we found that some parts of the authoring process of a virtual environment can be more easily done in a conventional desktop base interaction mode. For this reason we extended our support for the less-immersive use of Figments.nrw without head-mounted displays. In this mode you navigate your avatar in a third-person perspective, more or less like in a conventional video game. While you do not have the tools at hand to realize immersive navigation or interaction metaphors (like grabbing and manipulating objects with your hands), you can import and load assets and design a virtual space none the less.
The future
The funding period our AR/VR.nrw ended on 31.08.2023. We plan to provide additional support up until the (delayed) release, mainly in the additional work needed to fix the aforementioned bugs. This work has to be done outside of the scope of AR/VR.nrw, which means some if not all of us now have other academic or professional commitments. We sincerely hoped for an earlier release, so that we would have been able to address your needs and your feedback more directly while still working inside the project scope and are sad, that we did not achieve this.
But we are convinced, that Figments.nrw as an open source framework for immersive worlds, is now more relevant than ever and hope to find ways to transform it into a sustainable solution in the future.
Until then, you can still send us your questions and inquiries via info@figments.nrw, where we voluntarily provide small scale support.
