Fluxa
An AI agent that generates robotics simulation training scenes from natural language.
Robotics simulation platforms are notoriously time-consuming and difficult to learn. Most require users to navigate extensive documentation and lengthy tutorials before achieving even simple tasks. In tools like Isaac Sim, setting up synthetic data generation scenes can be especially tedious – users must either spend hours exploring numerous configuration tabs to assemble environments or write complex custom scripts to place specific objects. This steep learning curve slows down experimentation and limits usability for researchers and developers alike. Furthermore, training robot policies across a range of environments, embodiments, and tasks is crucial to enable generalization in the real world. However, collecting this data on real robots at the scale necessary to train robust policies is impractical. Simulation provides a useful framework to generate large amounts of high-fidelity training data, but existing simulators are limited in their ability to easily and rapidly configure diverse environments.
Solution
Fluxa is an AI agent that autonomously generates robotics simulation training scenes from natural language. I built an integrated system where NVIDIA Isaac Sim is streamed through WebRTC in a React front end, orchestrated by a customized open-source MCP server. The MCP server was extended with Dedalus API model-calling logic, enabling Claude Opus to generate custom robot, object, and environment configurations by invoking the Isaac Sim API. This automation reduced multi-file workflow setup and GUI scene assembly from days/weeks to minutes—improving synthetic data generation efficiency for robot learning research by over 95%. Fluxa provides a web-hosted conversational interface that makes simulation setup intuitive and visually effortless, eliminating tedious configuration or manual coding. I aim to show that using Fluxa to create and train robot policies in diverse, dynamically generated environments improves sim-to-real transfer performance compared to baseline policies trained in default simulation setups.
