Introducing LLM-Powered Robots: MachinaScript for Robots
Build modular ai-powered robots in your garage.
A little prologue from the author:
It all begun with Transformers — not the robots — a deep learning technique released by google for predicting words in a very good sequence. Suddenly we have achieved artificial intelligence, at least some form of it. But the true question remains:
Where the hell are my robots!
Well, fear not dear roboticist. MachinaScript is here to help you bring next-gen robotics to your garage. You can start making incredibly intelligent robots right away.
Anakin built C3PO when he was 9 years old. How about you?
- babycommando, made with love for the maker community.
Introducing MachinaScript For Robots,
A set of tools and a LLM-JSON-based language designed to empower humans in the creation of their own mechanical companions.
It facilitates the animation of generative movements, the integration of motion personality, and teaching a set of instructions (skills) with a high degree of autonomy.
With MachinaScript, you can control a wide range of electronics like Arduinos, Raspberry Pis, servo motors, cameras, sensors, and much more. Our goal is to make a wide modular generative format more accessible, as no robot design is the same for anyone.
How does it work, and some tips
A robot is naturally built on top of computers, motors and other cool electronical components. They are also built around a physical design.
There’s a lot of “how to arduino robots” out there already. They work pretty much like remote-controller toys: robot dogs, humanoids, R2D2 clones…
We are just switching the controls to a multimodal large language model. And trust me, they perform very well on this. Motion design can display a lot of personality.
Consider for example a robot that have a have a pair of cat ears. LLMs knows incredibly accurate ways to move them to express feelings when talking.
You just need a way to tell the robot they have cat ears…
Enter The MachinaScript Language
The MachinaScript language is built on a LLM-JSON-based synthax with incredibly modular capailities because it is generative. As long as your “Brain” code is adapted to understand it, you literally have no ending for your creativity.
Hook up all your components and limit a set of rules on how the robot may use them to fulfill the future inputs.
The current version of MachinaScript was built in this format:
- At the top layer there are a set of actions.
- Inside actions we have a set of movements to be executed to achieve a goal in an animated motion way. Movements contain the set of motors, degrees for each to move and additional configuration like movement speed.
- Also inside of an action, we have the “useSkill” — a set of skills to be used in an order to also achieve a goal. They are similar to function calling, with some code ready to be activated. Take a picture, text to speech, blink a led, etc.
Sudo make me a sandwich? Let’s go.
Note you can even change the complete synthax for the language structure for your needs! You see, MachinaScript is more of a philosophy for building generative robots than anything else. This is the true power of natural language processing.
Additionally, you may want to finetune a llm to make even better/faster generations for your own machines and specs. Training local models are pretty easy nowadays with tools like gradient.ai or even the official openAi one if you are using their models.
A Brain and a Body
Robot design patterns for MachinaScript to work are pretty straight forward: a computer represents brain, a microcontroller represents the body. Motors, sensors and other dispositives can be hooked to them in different setups and controlled from the MachinaScript generative set of instuctions and skills.
The actions, movements and skills are then serially encoded in a sequence to be executed — in deep specification if needed. For example:
Move slowly for taking a whole-night long-exposure picture of the milky way center. Start now at 45 degrees north and only stop at 5 am.
This set of orders comes from the LLM encoded in the MachinaScript JSON format. For arduino serial communication to be very fast, we must serialize this data and pass from the “Brain” to the “Body”.
Engineers and hackers, behold the github repo here for a full working example ready to be customized.
How to Make a Robot
Robots can be made from very cheap parts to extremely expensive ones. MachinaScript enables everyone to work with any case.
One of the wildests capabilities of the MachinaScript is the hability to metamorph into any robotic body shape. It learns in no-time how to control your design, being it two or four legged, aerial, terrestrial or submarine. As long as the capabilities are well prompted, mapped and parsed in the and the Body/Brain codes, you are good to go.
Grab some scrap metal, some very cheap servos and start playing around. Find an enclosure, make it have a personality. Or maybe ask it who it wants to be. Who knows.
And suddenly the dream of building intelligent robots is not a dream anymore.
Join the discord community, share your designs, skills, 3D models, hacks and crafts with us! We really wanna see your robots :]
MachinaScript for Robots is and always will be free and open source for everyone.
Contribute to the github repo here. You can find some early code examples to start making your robots right now.
MachinaScript for Robots is in very early beta. Use at your own risk.
