Positional tracking on smart glasses and webcams
Experimenting with @nianticspatial.bsky.social’s 8thWall engine, allowing us to determine position using camera and motion data
Tune your guitar with your smart glasses
With our wearables SDK, we’re able to display pitch, allowing musicians to easily tune their instruments
Works with compatible smartglasses, in this case the Brilliant Labs Frame running custom firmware
Display sheet music on your smart glasses, and tap your foot to turn the page
With our wearables SDK, not only can we render musicXML files on compatible smartglasses, but we’re able to use our smart insoles to trigger page turns by running an @edgeimpulse.com model that detects foot taps on device
Visualizing pressure and motion data from our smart insoles on smart glasses
With our wearables SDK we can receive realtime pressure and motion data from our smart insoles, and then display them on compatible smart glasses (in this case the Brilliant Labs’ Frame running custom firmware)
Visualizing jump height on smart glasses
With our wearables SDK, we’re able to calculate short-range positional displacement using imu data from our smart insoles & motion modules, visualizing the trajectory on compatible smart glasses (in this case the Brilliant Labs Frame running custom firmware)
Work out with realtime feedback on your smartglasses
With our Wearables SDK, we can visualize imu data from our motion modules (which can be mounted on the wrist or ankle depending on the workout), guiding the user to perform reps at a controlled tempo for maximum effectiveness
Running an Image Classifier on smart glasses, using @edgeimpulse.com to create the model
Streaming smart glasses camera data to a Quest 3
Using our Wearables SDK, we’re able to stream images from the Omi Glass (based on the @seeedstudio.com XIAO ESP32-S3 running custom Firmware) to a Quest 3
Local AI Assistant on Smart Glasses
Using our wearables SDK to take a picture, speak a prompt with Whisper’s speech to text, and use chrome’s Prompt API to generate a response to be displayed back on the glasses’ display
Spotify Player with Lyrics on Smart Glasses
Using our Wearables SDK to experiment with a music player on compatible smartglasses (in this case the Brilliant Labs’ Frame running custom firmware)
Also using a Mudra Band and TapXR for hand controls
Realtime maps in your smart glasses
Using our wearables SDK to display a simple map overlay, using @openstreetmap.bsky.social data to retrieve nearby data
Recognize people with smart glasses
Using our wearables SDK to look for certain people, and displaying relevant information when someone is recognized
Works with compatible smartglasses (in this case the Brilliant Labs’ Frame running custom firmware)
Experimenting with hand controls on smart glasses
With our Wearables SDK, we can both display a 3d wireframe scene on compatible smartglasses, as well as control a cursor by strapping one of our motion modules to our wrist
We’re able to detect pinches by running an @edgeimpulse.com model
View yourself in the third person with smart glasses
Using our Wearables SDK, we’re able to visualize MediaPipe tracking data (body, hands, and face) on compatible smartglasses (in this case the Brilliant Labs Frame running our custom firmware)
Great for yoga, working out, dancing, etc
Translation on Smart Glasses
Added foreign language support to our Wearables SDK, allowing us to display languages like Chinese, Japanese, and Korean
In this case we’re using @hf.co models for both realtime transcription and translation - all done locally (no cloud required)
Boxing on smart glasses
Added wireframe support to our Wearables SDK, allowing us to display realtime 3d graphics (like A-Frame and @threejs.org) on compatible smart glasses (in this case the Brilliant Labs Frame)
Realtime Subtitles on Smart Glasses, using our Wearables SDK
We used @hf.co’s Whisper model to transcribe microphone data locally, and then display the transcription on the smart glasses’ display (in this case the Brilliant Labs’ Frame) - no cloud required
Added sprite support to our wearables SDK
Great for stuff like emulating 3d graphics via billboarding
Added Font support to our Brilliant Sole Wearables’ JavaScript SDK
Added Bitmap support to our Brilliant Sole Wearables’ JavaScript SDK
Our wearables’ JavaScript SDK works great with react-like environments
This is a simple example using @nextjs.org, @supabase.com, and @tailwindcss.com to make a basic data collection website that can both record and view sensor data
Detect head gestures on your smart glasses
Made a simple @edgeimpulse.com model to detect nods and head shakes
Smart glasses webcam
Created a virtual webcam using OBS and our SwiftUI camera demo
Express yourself with smart glasses
Using MediaPipe’s face tracking to visualize a user’s eyes and eyebrows on a smart glasses display (in this case the Brilliant Labs’ Frame running our custom firmware)
Our smart glasses display simulator also works in WebXR, allowing us to preview ar glasses applications on a Quest 3
Added a smart glasses display simulator to our JavaScript SDK
Preview what your ar glasses applications will look like using your webcam, a video file, or even an interactive 3D scene
Added display support for smart glasses to our JavaScript SDK
We created a Canvas-like interface for drawing primitives onscreen for smartglasses with displays, like the Brilliant Labs Frame
Easily draw rectangles, circles, rounded rectangles, ellipses, polygons, and segments
Running speech recognition on smart glasses 👓
As we port our firmware to the Brilliant Labs’ Frame, we added microphone support to our SDK’s, allowing us to run @hf.co’s Whisper Web to transcribe speech in the browser
Hand, face, and object tracking on smart glasses
Running MediaPipe and @hf.co models on the Brilliant Labs Frame camera data
Streaming camera data from smart glasses to a smart watch via Bluetooth
As we port our firmware to the Brilliant Labs’ Frame, we added camera support to our SDK’s
