Immersive Twins
Philippe Coval <purl.org/rzr> #LGM2020
$ whois Phil Coval
- Software Engineer from Rennes, France
- OpenSource Contributor, Mozilla Rep (IoT)
- Involved in industry's OSS:
- Tizen (Intel), Yocto, IoTivity (Samsung)
- Contact: https://purl.org/rzr @RzrFreeFr
- Presentations, Videos, Social…
- Currently available for cooperation
From "flat" reality to…
- Validating concepts
- 1950: "Experience Theater", Sensorama
- 1980: Cyberspace : SciFi William Gibson
- 1993: Doom FPS (native)
- 3D online
- 1995: VRML, X3D (browser plugin)
- 2003: SecondLife (native)
- 2006: WebGL (GPU in browser)
- Devices & ubiquity
- 2014: Cardboard (on mobile)
- 2016: Pokemon Go (AR/ Overlay)
- 2017: OpenXR: devices layer
- 2018: WebXR (devices + apps)
Openness
- OpenXR: to "Solve AR/VR Fragmentation"
- Standard from Khronos Group
- API for applications and Device layer
- FLOSS: Monado, OpenHMD
- WebXR: Web Browser API
- to abstract devices for apps:
- HMD, sensors (6 DoF), controllers
- Cross devices/browsers:
- API: navigator.xr
Play WebXR apps
- From a VR/AR headset
- supporting Firefox reality
- or device's web-browser
- Mobile with Card-box (or similar)
- use orientation sensors
- From a regular browser:
Build WebXR apps
- Many frameworks:
- WebGL: JS bindings to OpenGL (GPU)
- Three.js: Scene on WebGL
- A-Frame: based on Three.js
- HTML for 3D Scene : "<a-box />"
- Babylon.js: based on WebGL
- glTF: GL Transmission Format
- Compress assets (JPEG of 3D)
- JSON structure
- Tools: Blender3D
Web as a platform
- Not only flat (HTML)
- 3D Native (WebGL)
- Immersive WebXR (Supersede WebVR)
- Dynamic (JS), programmable (Services)
- Transversal:
- between 3D worlds
- among people, cultures (social)
- or even connected devices
Back to real reality
- IoT: connected devices on the Internet
- API for Sensors and actuators
- Web Of Things: devices on the WWW
- Digital Twin:
- "Live" Replica of a physical entity (model)
- Web Of Twins: A DT experiment on the web
- Bind WoT to XR
- Control WebThings using A-Frame
Mozilla WebThings
- Mozilla's SmartHome platform
- Gives control back to users
- UI Dashboard: automation using rules
- Extensible with addons
- for new protocols or services
- Simplified W3C WoT Description (in JSON)
From Reality to XR:
- IO: Sensors (or actuators) are served
- WebThings API (RESTful or WebSockets)
- Webthings are connected to gateway (in LAN)
- Can be shared to the Internet
- WebXR application listen to webthing changes
- update "twin" node accordingly
- FLOSS: https://purl.org/aframe-webthing
Resources
- Feedback welcome:
- Extra demos on slides:
- Sources: