As the robotics industry increasingly delegates control to generative AI models, a new company called Safeworld is emerging from stealth to address the inherent unpredictability of these probabilistic systems. Founded by Dr. Ding Zhao, director of the Safe AI lab at Carnegie Mellon University, along with veteran startup executive Kyle Wong and machine learning engineer Simo Rachidi, Safeworld aims to establish safety standards for robots before they are widely deployed in human-centric environments.
What Happened
Safeworld announced its launch today, backed by a seed round of more than $12 million. The funding was led by Shine Capital and a16z Speedrun, with additional investment from Box Group, the Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel. The company’s core offering involves evaluating robotic control systems using simulations populated with realistic human models. This approach allows developers to test scenarios such as blind corners in factories or human unpredictability—like tripping or falling—without the physical risk or logistical difficulty of testing on real people.
The technology leverages simulation platforms like Genesis or MuJoCo, where a digital version of a specific environment is created, and the robot’s actual software is driven through thousands of scenarios. Dr. Zhao notes that while traditional algorithms are predictable, generative AI introduces probabilistic risks that require new methods of underwriting safety. "The safety challenge that we’re talking about is a combination of, one, really advanced generative AI probabilistic evals... The second part that’s really hard is the trust part, and you need both to deploy a robot," Zhao stated.
Why It Matters
The shift toward generative AI in robotics brings significant safety challenges because these models operate in unstructured environments where safety standards can vary by facility. Jonathan Lai, a partner at a16z Speedrun, emphasized the urgency of establishing industry standards now, warning that waiting until robots are in households and causing safety incidents would be "way too late." Unlike autonomous vehicles, which face predictable road structures, robots must account for highly variable human behaviors and appearances, such as different clothing, body configurations, and movements.
The need for third-party validation is growing among robot developers. Vishal Dugar, CTO of Gritt Robotics, which uses AI to help workers install photovoltaic panels, noted that formal mathematical proofs of safety are difficult for these systems. "It necessarily has to be done empirically," Dugar said, highlighting the complexity of ensuring robotic arms do not collide with humans who might be kneeling, crouching, or running. Safeworld positions itself as a necessary partner for companies seeking to share safety case information and validate their systems against edge cases that are difficult to predict in a vacuum.
The Bottom Line
Safeworld enters the market at a critical juncture as generative AI becomes central to robotic control. While the company is still determining whether to operate as a platform or a service-based entity, its founders believe that safety validation will become a mandatory cost for deployment. "We’ll probably be the first profitable company in this field," Zhao predicted, arguing that any entity wanting to deploy robots at scale will need to pay for rigorous safety verification. The success of Safeworld will depend on its ability to build trust in probabilistic systems before they encounter the unpredictable reality of human interaction.