We noticed Spot run, soar, and even dance… however now we will see Spot speak. In a considerably unsettling video posted by Boston Dynamics, we see its robotic canine outfitted with a high hat, mustache, and googly eyes because it chats with employees members in a British accent, taking them on a tour of the corporate’s services.
“Shall we start our journey?” Spot asks. “The charging stations, the place Spot robots relaxation and recharge, is our first focal point. Observe me, gents.” As proven within the demo, Spot is able to answering questions and even opens its “mouth” to make it appear to be it’s really talking.
To make Spot “speak,” Boston Dynamics used OpenAI’s ChatGPT API, together with some open-source massive language fashions (LLM) to fastidiously prepare its responses. It then outfitted the bot with a speaker, added text-to-speech capabilities, and made its mouth — er… gripper — mimic speech “just like the mouth of a puppet.”
Matt Klingensmith, the principal software program engineer at Boston Dynamics, says the group gave Spot a “very transient script” for every of the rooms at its services. The bot then mixed that script with the imagery it will get from the cameras on its gripper and physique, permitting it to “get extra details about what it sees earlier than producing a response.” In response to the corporate, Spot makes use of Visible Query Answering fashions to primarily caption pictures and reply questions on them.
“Generator hums low in a room devoid of pleasure. Very similar to my soul.”
The “fancy butler” isn’t the one persona Spot assumes through the video. The four-legged bot additionally takes on the persona of a Nineteen Twenties archaeologist, a youngster, and a Shakespearean time traveler. It even assumes a sarcastic persona, which, when requested to give you a haiku, mentioned: “Generator hums low in a room devoid of pleasure. Very similar to my soul.”
Boston Dynamics says it uncovered just a few surprises when experimenting with Spot as a tour information. In a single occasion, the group requested Spot who its “mother and father” had been, and it went over to the place the older Spot fashions are displayed within the firm’s workplace. The corporate additionally notes that it nonetheless bumped into some situations the place the LLM made issues up, corresponding to suggesting that Stretch, its robotic designed to maneuver packing containers, was made for yoga.
“We’re excited to proceed exploring the intersection of synthetic intelligence and robotics,” Klingensmith writes in a put up on Boston Dynamics’ web site. “These fashions [LLMs] can assist present cultural context, basic commonsense data, and adaptability that may very well be helpful for a lot of robotics duties — for instance, with the ability to assign a activity to a robotic simply by speaking to it could assist scale back the training curve for utilizing these techniques.”
