At first, it’s not quite clear what you’re looking at. The yellow robots, sitting in silence, are lined up so closely that they almost resemble deck chairs stacked on top of one another. Then, in time with the music, their arms shoot out into a fluid series of patterns, giving the appearance of a single, many-armed robot peacocking for the camera. Finally, the seven Spots separate, synchronizing their steps and appearing to play off of one another’s dance moves. 

The steps are so perfectly timed, and the motions so smooth and harmonious, that you might think the robots are actually listening and responding to the music. But they’re not; they’re listening to their synchronized inner clocks. For all their sophisticated sensors, the robots don’t even know that music is playing. And their ability to move through space without stepping on one another’s toes is the result of painstaking positioning and programming, rather than Spot’s obstacle avoidance feature. 

The end result is seamless, sleek, and even a bit startling. But hiding behind the curtain lie hours of choreography and engineering effort – all coming together for a 77-second performance. 

“There were a lot of challenges around getting the vision of our choreographer, who’s used to dealing with human dancers, into our software,” notes Eric Whitman, a Boston Dynamics roboticist who had a lead role in the “Spot’s On It” video released on June 29. “Everything had to be worked out in advance and scripted precisely. Robots have the advantage over humans in that they’re very repeatable: Once you get it right, it stays right. But they have the disadvantage that you have to tell them every little detail. They don’t improvise at all.”

FORM, FUNCTION & FUN

But rewind a moment. This isn’t Spot’s first foray onto the dance floor. In 2018, Spot strutted its stuff to “Uptown Funk.” And late last year Spot joined Atlas and Handle (two other Boston Dynamics robots) in a dance performance to the 1962 classic “Do You Love Me.” 

Once these videos end, the obvious question lingers: Why dance? After all, Spot is already deployed commercially. The robot is completing inspections at energy plants and construction sites. What’s the purpose of programming the robot to shake its tail feather? 

“An athletic performance like dance stresses the mechanical design of the robot, and it also stresses the algorithms in the software,” says Marc Raibert, founder and chairman of Boston Dynamics. “If you look at the dance we did, it’s got dozens of different behaviors, so the team had to make tools that allow us to create those behaviors fast enough to do the project in a reasonable amount of time. That’s the practical answer. 

“But for me,” Raibert adds, “this was mostly a way of expressing creativity and having fun.” 

Whitman notes that it has taken years to develop routine behavior for Spot, such as reliable walking and climbing. Putting together a dance video like this one, he says, gives developers a creative target that leads to rapid innovation in how the robot can move. Dance also uncovers potential improvements in the physical design of the robot. Or, as Whitman puts it: “Dancing is a form of highly accelerated lifecycle testing for the hardware.” For instance, after dance testing revealed that slamming Spot’s knee with too much force could be damaging, the company set out to find better materials for the robot’s upper legs. 

“When we do these projects we always have two goals,” Whitman says. “One is to use the video as motivation to improve the product. The secondary goal is to have fun making the video.”

FROM IDEA TO IMPLEMENTATION

For the “Do You Love Me” video, Raibert reached back to his youth for inspiration. He was around 12 years old when the song first played on the radio, and during development he wanted the Atlas team to bring more soul to its “mashed potato.” BTS’s “Ioniq: I’m On It” was released in 2020, but Raibert again wanted to bring a vintage aesthetic to the new video. 

“When I was a kid you’d see dancers on Ed Sullivan and other variety shows doing kaleidoscope simulations,” he says. “Sometimes they’d be lying down on the stage with the camera overhead, and they would do these patterns that were symmetric in that circular spectrum. And if you looked at it, it looked just like what a kaleidoscope looks like. So that was the starting point. With the opening shot in the video, at first you’re not supposed to be able to tell that it’s more than one robot.” 

Monica Thomas, a professional choreographer and former dancer, took the lead on choreography for “Spot’s On It”, as she did for Boston Dynamics’ previous dance videos. Outlining a dance routine for robots, she notes, presents a whole separate set of challenges from traditional choreography. For one thing, she’s in constant communication with Boston Dynamics engineers to explore which steps the robots are physically able to do. And, of course, Spot has twice as many legs as a human dancer. “When I try to replay the choreography on my own body, my knees bend the wrong way, even if I put myself on all fours,” Thomas says. “I have less knowledge about what things even could look like, and so I have a lot more flexibility about what it does look like.” 

For the Spot’s On It video, Thomas was interested in the idea of the Spots acting as both individual dancers, and as part of a unified whole. In the early stages, she moved stones around on a table to represent the “marching band” style synchronization of the routine and show engineers what she wanted. 

Thomas handed off dance steps to Jakob Welner, a robotic movement consultant based in Denmark, who animated them in Autodesk Maya. “You can do anything you want in the software, artistically,” Welner says. “But as soon as you want it to work on a robot, you need to comply with physics, and you need to comply with the mechanics and dynamics of the robot.” 

Finally, the Spot team created the dance moves in Choreographer, a Spot software module created by Boston Dynamics specifically for Spot dance routines. Choreographer is in public release, and is being used by a handful of Boston Dynamics entertainment customers, such as sports teams and theme parks. The software features a graphical, drag-and-drop interface, which allows virtually anyone (tech background or no) to program a dance routine into Spot.  

“I would send new performance elements to the controls team. They would run it through a simulation, and then we would try it on the physical robot,” Welner says. “Then we would iterate on that to figure out what worked, what didn’t work, and what could be done to fix it. We very quickly established this rapid behavior pipeline, where you could animate a move and be reasonably confident it would run on the robot.” 

PUTTING IT OUT TO THE PUBLIC

One thing the “Spot’s On It” video doesn’t show is just how reliable and consistent the robots’ performance has become. The public tends to only see the perfected, final result of robotics demonstrations – the very best take. Raibert remembers a photo shoot at his lab in the early ’80s, when a photographer from Scientific American spent an entire day trying to get a shot of a robot jumping over a stack of blocks. The shot never came together, and the magazine had to publish a photo that Raibert and his team had previously taken. 

“That’s what it was always like back then,” Raibert says. “But now it is amazing how well Spot performs, precisely repeating moves over and over. We had the concept, we did a lot of work on getting the concept executed, but man, those robots just performed once we got there.” 

Raibert says he isn’t looking for any specific response when Boston Dynamics releases a video like “Spot’s On It.” “The people here love what we’re doing,” he says. “The videos are a little bit of showing off – showing our friends and our fans what we are up to.  Viewers need to be free to have whatever reaction they like.”