A-Frame Windows Mixed Reality standalone app

Building a standalone (aka kiosk) virtual reality (VR) application on the Windows Mixed Reality (WMR) platform has some challenges, even more, if it is a WebXR app, based on the A-Frame framework.

Two challenges which need to be solved:

• Starting up the VR application on every startup of the Windows 10 system. That is, opening the app inside the WMR platform.
• The WMR platform goes into a standby mode after some time if the headset is not used. If the headset is used again it boots into the standard WMR app and not into the WebXR app.

Challange 1: Startup

The A-Frame VR application is generally running inside a web browser. If the typical start-vr button is pressed on the web-browser the app will load inside the WMR environment. The start-vr requires a “real” user interaction a simple dom click event is not enough (this just leads to an exception).

Therefore Puppeteer was chosen to create the click on the start-vr button. Additionally, the Puppeteer script was started by the Windows startup (aka autostart) feature.

const puppeteer = require('puppeteer-core');

(async () => {
  const browser = await puppeteer.launch({
    executablePath: 'C:/Program Files/.../chrome.exe',
    headless: false,
    userDataDir: 'C:/.../browser-data'
  });

  const page = await browser.newPage();
  await page.goto('http://link.to/a-frame-app.html');

  // if the app requires some loading time, then it should be waited for the respective load event

  const button = await page.$('#start-vr-button');
  button.click();
})();

The above javascript should be put inside an arbitrary script file (e.g., index.js) and put somewhere on the filesystem. Within an arbitrary bat file the script should be started (node index.js). The bat file (e.g., startup.bat) should be placed inside the directory (replace … with the respective user name):

C:\Users\...\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup

This procedure lets the VR app start as a standalone/kiosk app upon every startup of the Windows 10 system.

Challenge 2: Sleep mode

This is a challenge also reported by others. The approach which was taken is to prevent the WMR platform to go into sleep mode. This was accomplished by restarting the application after a certain time when the headset was unused.

To find out if the headset is unused the visibilityState of the XRSession was used. Each headset has a proximity sensor, which senses if somebody is wearing the headset or not. The visibilityState is either hidden or visible (There is also visible-blurred, which was never reported on the tested headsets).

Thus, the visibilityState of the current XRSession exposed by the A-Frame a-scene is queried every 5 seconds. If for “a long time” the state is hidden then the app is restarted.

The following snippet shows the general idea. The snippet is also part of the puppeteer script, which is staring the app in the first place.

let hiddenCount = 0;

const checkVr = async () => {
  const scene = await page.$('a-scene');
  const visibilityState = await scene.evaluate(node => node.xrSession.visibilityState);

  if (visibilityState == "hidden") {
    hiddenCount++;
    if (hiddenCount > 50) {
      await page.goto('http://link.to/a-frame-app.html');
      hiddenCount = 0;
    }
  } else {
    hiddenCount = 0;
  }
    
  setTimeout(checkVr, 5000);
};

checkVr();

Conclusion

Building standalone VR applications seem like a common requirement. WebXR applications (e.g., built with A-Frame) seem to be a good fit for getting appropriate results within a fast and easy development cycle (as common for web development in general). Building a standalone VR application for the WMR platform posed some challenges, which were solved as described above.