Imagine taking a stroll and stumbling upon robots skillfully threading their way through an amazed crowd. Intriguing, yes, but these marvels of modern technology aren’t quite ready to be our all-in-one sous chefs or factory workhorses. A significant part of the puzzle they are yet to solve is efficient learning. Like us humans, they learn best through practical experience. However, immersing robots in diverse learning environments can be quite the task – it’s strenuous and time-consuming. But don’t worry, we have solutions on the horizon.
One potential solution gaining traction is the use of simulations as learning environments for robots. Professor Russ Tedrake from MIT, who also happens to be an investigator at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), outlines that advances have been made on the physics engines of robotic simulators. The catch? Crafting authentic simulations that reflect the complexity of the real world still remains an uphill task.
This is where AI agents step in. These semi-autonomous programs specialized in performing well-defined tasks could be exactly what’s needed to create realistic virtual spaces for robotic training. A faction of researchers from MIT CSAIL and Toyota Research Institute have conceived a system dubbed “SceneSmith.” Comprising a trio of AI agents, SceneSmith masterfully crafts objects, walls, and whole 3D scenes that represent settings like restaurants and hotels. Consequently, robots are able to hone their skills and strategies in these settings before deploying in the real world, saving engineers a significant amount of time.
SceneSmith leverages a vision-language model, known as GPT-5.2, which comes loaded with extensive internet text and image information. There are three central players in this setup – a “designer” that creates scene elements, a “critic” that ensures these elements reflect reality, and an “orchestrator” – the main hand – that signals job completion. Once the agents finish this fruitful collaboration, the scene is set for integration into physics simulation software.
Nicholas Pfaff, an MIT EECS Ph.D. student and CSAIL researcher, shared insights about the system’s ability to build 3D scenes, akin to a human designer. In fact, one leading vision-language model was used to create over 1,300 scenes, resulting in wonderfully creative and diverse environments without even the need for specific prompts.
Platforma SceneSmith umożliwia użytkownikom tworzenie wirtualnych środowisk na podstawie szczegółowych poleceń. Na przykład użytkownicy mogą poprosić o garaż wyposażony w samochód, stół warsztatowy, stos opon w jednym rogu oraz drabinę opartą o ścianę. Dzięki takiemu poziomowi szczegółowości środowiska te stanowią bogate pole do nauki dla robotów, które mogą doskonalić w nich umiejętności, takie jak przenoszenie puszki napoju z półki na stół.
Aby ocenić, na ile realistyczne są generowane środowiska, naukowcy wprowadzili do przestrzeni SceneSmith wstępnie wytrenowane algorytmy działania robotów. Roboty wykazały się sprawnością, wykonując zadania takie jak przeniesienie jabłka z miski na deskę do krojenia, co stanowi wyraźną wskazówkę, że wirtualne sceny w znacznym stopniu odzwierciedlają rzeczywiste warunki.
SceneSmith’s AI agents collaborate to gradually flesh out scenes – from creating a floor plan to populating it with furniture and objects. The “designer” vision-language model commences the layout, the “critic” takes a gander at it, and finally, the “orchestrator” wraps up the design. SceneSmith also sets itself apart with its ability to generate environments packed with objects and details compared to other methods, making it a crowd-pleaser among over 200 users for its vivid visuals and adherence to prompts.
There is one small hitch, however. Crafting those stunning realistic scenes takes a fair bit of time – sometimes hours for a single scene. But as computing power bolts ahead, SceneSmith’s efficiency should improve significantly. Perhaps it could even expand to include much more complex entities like deformable objects, should sufficient 3D libraries become available, as the CSAIL engineers hope.
AI automation is a burgeoning space with much to offer for the robotics realm, as exemplified by SceneSmith’s splendid contributions. For an in-depth look at this groundbreaking research, feel free to visit the original news article tutaj. Jeśli Twoja firma chce wykorzystać potencjał sztucznej inteligencji, implementi.ai może mieć dla Ciebie idealne rozwiązania.
Ta strona używa plików cookie.