Editorial Feature

Teaching Robots about the World using 'Robo Brain'

Image Credit: Carlos Amarillo/Shutterstock.com

Science fiction films like A.I. and I, Robot portray a world entwined with artificial intelligence (AI). However, what we don't see on the big screen is how this technology could work in the real world. A joint partnership involving researchers from Cornell University has started to create a giant repository of knowledge collected from the internet that robots can use to learn new skills.   

Robots will need to fully understand how the world works and how humans behave before they serve as our helpers in our homes, offices and factories. 

In recent years researchers around the world have been teaching them individual commands: How to find your keys, pour a drink, put away a dish and when not to interrupt two people having a conversation.

The Brain

The team involved in the project have dubbed the package Robo Brain, a huge repository of information collected from various sources online and then stored in a robot-friendly format that robots can access whenever they need it. 


Our laptops and cell phones have access to all the information we want. If a robot encounters a situation it hasn’t seen before it can query Robo Brain in the cloud.

Ashutosh Saxena, assistant professor of computer science at Cornell University

Saxena and his team started to download roughly one billion images, 120,000 YouTube videos and 100 million how-to documents and appliance manuals earlier this year.

Identifying Objects

Robo Brain is designed to process images by picking out the objects, and by connecting images and videos with text, it will learn to identify objects and how they are used. 

For most of use picking up a coffee mug comes as second nature, however for a robot it can become a challenging process. Using Robo Brain a robot can learn not only that it's a coffee mug, but that it can be used for holding liquids, that it can be held by the handle, and that it should be carried upright when full.

The system uses what computer scientists call "structured deep learning," which stores information in multiple layers of abstraction. For example, a chair is a part of a class of chairs which can be classed as furniture. Sitting is something you can use a chair for, but a human can also sit on the floor or a stool. 

The robot can store what it has learned in what is called the Markov Model, which can be depicted as a series of points connected by lines. The nodes could represent objects, actions or parts on an image, and each one is then assigned a probability. When searching for knowledge in any given situation, a robot's brain will make its own chain and look for one in the knowledge base that matches those probability limits.

The Robo Brain will look like a gigantic, branching graph with abilities for multidimensional queries.

Aditya Jami, a visiting researcher at Cornell who designed the large-scale database for the brain

Thanks to crowdsourcing Robo Brain will be able to learn more about the world. The Robo Brain website will display things the brain has learnt, and visitors can make additions and corrections. 

The Future of the Brain

It may be some time yet before we call on Will Smith to save us from an artificially intelligent super computer, but this technology does represent a significant development in the race to successfully develop AI.

However, the technology does demonstrate just how much a robot must learn before we can live and work alongside each other successfully.    


Stuart Milne

Written by

Stuart Milne

Stuart graduated from the University of Wales, Institute Cardiff with a first-class honours degree in Industrial Product Design. After working on a start-up company involved in LED Lighting solutions, Stuart decided to take an opportunity with AZoNetwork. Over the past five years at AZoNetwork, Stuart has been involved in developing an industry leading range of products, enhancing client experience and improving internal systems designed to deliver significant value for clients hard earned marketing dollars. In his spare time Stuart likes to continue his love for art and design by creating art work and continuing his love for sketching. In the future Stuart, would like to continue his love for travel and explore new and exciting places.


Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Milne, Stuart. (2014, September 01). Teaching Robots about the World using 'Robo Brain'. AZoRobotics. Retrieved on April 24, 2024 from https://www.azorobotics.com/Article.aspx?ArticleID=193.

  • MLA

    Milne, Stuart. "Teaching Robots about the World using 'Robo Brain'". AZoRobotics. 24 April 2024. <https://www.azorobotics.com/Article.aspx?ArticleID=193>.

  • Chicago

    Milne, Stuart. "Teaching Robots about the World using 'Robo Brain'". AZoRobotics. https://www.azorobotics.com/Article.aspx?ArticleID=193. (accessed April 24, 2024).

  • Harvard

    Milne, Stuart. 2014. Teaching Robots about the World using 'Robo Brain'. AZoRobotics, viewed 24 April 2024, https://www.azorobotics.com/Article.aspx?ArticleID=193.

Tell Us What You Think

Do you have a review, update or anything you would like to add to this article?

Leave your feedback
Your comment type

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.