Sunny Ray sits down with Brent Auster, founder of Orbby, a startup building AI companion robots for the home. Auster spent 26 years making virtual things look and move real, working in 3D graphics and simulation at Electronic Arts, Autodesk, and NVIDIA, where he spent a decade demonstrating early AI to enterprise customers before deciding to build product himself. Before Orbby's current AI avatar (demoed live as a character named Rachel), the team built James, a holographic robotic bartender that mixed real drinks and became a hit demo at NVIDIA's GTC conference. Auster explains why he chose companionship over warehouse labor as Orbby's beachhead, how the company pivoted away from early spiking neural network AGI plans toward Yann LeCun's JEPA architecture, and how NVIDIA Isaac Sim lets the team train a physical robot's skeleton and actuators before hardware exists. He also discusses data privacy, keeping conversations local, and what a companion robot could mean for people who live alone. The physical Orbby robot is targeted for 2028, though the AI and avatar product already work today.
A former NVIDIA and Lucasfilm veteran on building a holographic robot bartender before pivoting to AI companion robots for the home.
What is Orbby and what problem are you solving with it?
Orbby is developing companion robots for the home. Right now it exists as an AI avatar you can chat with, ask for advice, or get step by step recipes from, like how to make lasagna or hamburgers. The avatar already talks about the specs it will have once it becomes a physical robot, including its intelligence, strength, and agility. The AI itself is done and working, we could put a paywall up tomorrow and have a paying product.
You spent 26 years in 3D graphics and simulation. What does that teach you about robotics that roboticists tend to miss?
I spent a lot of time working with humanoids in animation, building skeletons, bones, blending animations, and creating 3D content for them to move around in. That work is very similar to robotics, where you also create a skeleton, then bring it into Isaac Sim and add actuators, physics, and real movement. You can actually simulate a completed robot in Isaac Sim, which is how a lot of people build robots today.
Before the companion robot there was James, a robotic bartender that actually poured drinks. What did building James teach you?
We wanted a demo of AI instantiated in physical form that people could interact with. We coated a glass shower door with translucent film and projected a six foot holographic bartender onto it, hooked up to a robotic mixing machine that could make about two dozen drinks, even layered ones. We only had about half a million dollars from friends and family, so we needed something we could take to trade shows, and it was a hit at GTC.
Everyone funded in humanoids is chasing labor. You went for companionship. Why is the living room the beachhead and not the warehouse?
I want to build something attractive, something beautiful, something people can have emotions about. A home companion robot is a much more nuanced and detailed robot to build than a labor robot that nobody's ever going to see.
You've been at this since January 2018. What did you believe in year one that you've since thrown out?
Originally we planned to do AGI with spiking neural networks. About two or three years in, it became obvious the hardware, software, and algorithms weren't there yet and it would be a long haul. So we pivoted to working with JEPA, the AI Yann LeCun is building, first at Meta and now at his own company, and decided to build a real robot with personality, not just physical task capability.
You work at the mid level, turning a stated goal into servo instructions. Why is that the layer nobody talks about?
It's the hardest part. Take baking cookies as an example, you break it into subtasks, like taking a stick of butter and sugar and mixing it together. We use JEPA for the next level down, taking high level commands like put butter in bowl and turning them into instructions for the actuators and hands. There are a couple levels of hierarchy, and JEPA is really good at that hierarchical structure for issuing commands down to the final servos.
A companion robot that reads emotion, lives in a house, and listens. What's your position on what it records?
I think it has to keep all the data local and not record unless you explicitly ask it to, because I don't think people want every last thing they confide to the robot recorded for all eternity.
If Orbby works, what breaks in the world that people aren't ready for?
It could enable a whole new industry. Right now people have to do their chores themselves or hire someone. With this, they can have something that does it for them. For the many people living alone and lonely, having a presence, even if it's a robot, that talks, converses, is friendly, and can discuss a range of topics, and as that evolves into a stronger AI, it's going to be revolutionary.
Building something daring? Sunny talks to founders like this every day. Fifteen minutes to see if your story belongs on the stage.
Claim your pre-interview