What happened since ...

It has been some time since posting the last time here on the blog. But a lot stuff happened. First we taught TIAGo to bring us beer for the NVIDIA Jetson Developer Challenge. The video we posted got quite some attention on Youtube (for our standard :)) and some news reports followed on popular pages like (CNN Chile, IEEE Spectrum, Sputnik, RTV, mail.ru …). Sadly we didn’t make it to the finals, but still we had a lot of fun during the implementation.

Best you see the resulting video where TIAGo autonomously opens the fridge and takes a beer to deliver it (click to play):

Bring me a beer from the fridge

Brining beer sounds easy for us to do. But when you start translating it in order to teach it to a robot it’s so much more. First the robot needs to learn the different types of beer and localize it. Same for the handle. The robot needs to be aware of where it actually is located right now and it also needs to be aware of it’s current state. That means it has to now it’s current joint angles of each joint at a high frequency in order to execute arm path to grasp the handle and then the beer. And and and …

Opening the fridge
Opening the fridge.
Getting the beer
Getting the beer.
Closing the fridge
Closing the fridge.

For localizing the beer we developed a semantic segmentation network, trained with quite some images of the different types of beer and another network trained for detecting the handle of the Fridge. Depending on the current state of the robot we execute a different model, therefore are able to set our focus of search depending on what we are looking for. Based on these estimates we programmed motions for gripping the beer and opening as well as closing the fridge.

Prediction result
Prediction result.

Then next, as a follow up to our attendance in the European Robotics League in Edinburgh we were invited to a studio talk and one day later Campus TV came by to report about our team and filmed the beer delivery.

Currently Raphael is in Tampere at the European Robotics Forum and presented the highlights from the last European Robotics League. This evening the award ceremony of the European Robotics League will be held.

European Robotics League Highlights from homer