Advanced AI Model Enables Coherent Scene Recognition for Autonomous Vehicles

Bicycles, people, cars or sky, road, grass: Which pixels of an image depict unique foreground objects or people in front of a self-driving car, and which pixels depict background classes?

Red for people, blue for cars: A new method uses artificial intelligence (AI) model that enables coherent recognition of visual scenes more quickly and effectively. Image Credit: Abhinav Valada.

This so-called panoptic segmentation task is a fundamental problem that finds applications in several fields like robotics, self-driving cars, biomedical image analysis, and augmented reality. Dr Abhinav Valada, an Assistant Professor for Robot Learning and a member of BrainLinks-BrainTools, at the Department of Computer Science of the University of Freiburg focuses on this research question.

Valada and his colleagues designed the most advanced “EfficientPS” artificial intelligence (AI) model that allows coherent recognition of visual scenes effectively and more rapidly.

According to Valada, this task is mostly addressed by employing a machine learning method called deep learning, wherein artificial neural networks developed based on the human brain learn from huge amounts of data. Public benchmarks like Cityscapes play a vital role in quantifying the advancement in such methods.

For many years, research teams, for example from Google or Uber, compete for the top place in these benchmarks.

Rohit Mohan, Member of Valada’s Team, University of Freiburg

The technique developed by the computer scientists from Freiburg, which has been designed to perceive urban city scenes, has been graded first in Cityscapes, the most powerful leaderboard for research on scene understanding in the field of autonomous driving.

Moreover, EfficientPS constantly sets new standards for other established benchmark datasets like IDD, Mapillary Vistas, and KITTI.

Valada demonstrated examples of the way the researchers trained several AI models on various datasets. The findings are overlaid on the corresponding input image, wherein the colors indicate the object class to which the pixel is assigned by the model. For instance, people are marked in red, cars in blue, buildings in gray, and trees in green.

Furthermore, the AI model forms a border around every object that it believes is an individual entity. The Freiburg scientists have been successful in training the model to convert the learned information of urban scenes from Stuttgart to New York City. The AI model did not know how a city in the United States would appear, yet it was able to precisely identify the New York City scenes.

Earlier techniques for tackling this issue have large model sizes and are computationally costly for use in real-world applications like robotics that are highly resource-restricted.

Our EfficientPS not only achieves state-of-the-art performance, it is also the most computationally efficient and fastest method. This further extends the applications in which EfficientPS can be used.

Dr Abhinav Valada, Assistant Professor for Robot Learning, Member of BrainLinks-BrainTools, University of Freiburg


Tell Us What You Think

Do you have a review, update or anything you would like to add to this news story?

Leave your feedback
Your comment type
Azthena logo powered by Azthena AI

Your AI Assistant finding answers from trusted AZoM content

Azthena logo with the word Azthena

Your AI Powered Scientific Assistant

Hi, I'm Azthena, you can trust me to find commercial scientific answers from

A few things you need to know before we start. Please read and accept to continue.

  • Use of “Azthena” is subject to the terms and conditions of use as set out by OpenAI.
  • Content provided on any AZoNetwork sites are subject to the site Terms & Conditions and Privacy Policy.
  • Large Language Models can make mistakes. Consider checking important information.

Great. Ask your question.

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.