The robo-dog, which is designed to negotiate chaotic situations with accuracy, has the potential to transform search-and-rescue missions, disaster response, and a wide range of emergency operations.
Thanks to its advanced memory and voice-command capabilities, it could be a game changer in emergency missions.
Sandun Vitharana, an engineering technology master's student, and Sanjaya Mallikarachchi, an interdisciplinary engineering doctoral student, led the development of the robotic dog that neither forgets where it has been nor what it has seen.
It recognizes voice instructions and uses artificial intelligence and camera input for path planning and detecting objects, and its memory is programmed to ensure it never forgets.
Fundamentally, the mechano-animal is a terrestrial robot equipped with a memory-driven navigation system, powered by a multimodal large language model (MLLM).
This system analyzes visual inputs and makes routing decisions by combining environmental picture capture, high-level reasoning, and path optimization with a hybrid control architecture that allows for both strategic planning and real-time modifications.
Robot navigation has evolved from simple landmark-based strategies to complex computational systems that utilize multiple sensory inputs. However, navigating unexpected and unstructured situations, such as disaster zones or isolated areas, has often proven problematic in autonomous exploration, where efficiency and agility are crucial.
While robot dogs and large language model-based navigation already exist in several different iterations, combining a bespoke MLLM with a visual memory-based system is a new concept, particularly in a general-purpose and flexible framework.
Some academic and commercial systems have integrated language or vision models into robotics. However, we haven’t seen an approach that leverages MLLM-based memory navigation in the structured way we describe, especially with custom pseudocode guiding decision logic.
Sandun Vitharana, Master Student, Engineering Technology, Texas A&M University
The robot, like humans, exhibits reactive and deliberative actions, as well as deliberate decision-making. It responds rapidly to prevent collisions and conducts high-level planning by analyzing its present perspective and determining the optimal course of action.
Moving forward, this kind of control structure will likely become a common standard for human-like robots.
Sanjaya Mallikarachchi, Doctoral Student, Interdisciplinary Engineering, Texas A&M University
The robot's memory-based approach enables it to recall and reuse previously traveled courses, increasing navigation efficiency by decreasing recurrent exploration. This capability is vital in search-and-rescue efforts, particularly in unmapped locations and GPS-denied settings.
The possible uses might go well beyond emergency response. Robots might help hospitals, warehouses, and other major institutions become more efficient. Its superior navigation system could also assist those with visual impairments in navigating minefields or conducting reconnaissance in dangerous regions.
Dr. Isuru Godage, an assistant professor at the Department of Engineering Technology and Industrial Distribution, advised the project.
The core of our vision is deploying MLLM at the edge, which gives our robotic dog the immediate, high-level situational awareness and emotional intelligence previously impossible. This allows the system to bridge the interaction gap between humans and machines seamlessly. Our goal is to ensure this technology is not just a tool, but a truly empathetic partner, making it the most sophisticated and first responder-ready system for any unmapped environment.
Dr. Isuru Godage, Assistant Professor, Department of Engineering Technology and Industrial Distribution, Texas A&M University
Meet the AI Robot Dog Built to Save Lives
Video Credit: Logan Jinks/Texas A&M University College of Engineering
Vitharana and Mallikarachchi presented and showed the robot's capabilities at the 22nd International Conference on Ubiquitous Robots. The findings were published in "A Walk to Remember: MLLM Memory-Driven Visual Navigation."
Journal Reference:
Vitharan, S, S., et al. (2025) A Walk to Remember: Mllm Memory-Driven Visual Navigation. IEEE. DOI: 10.1109/UR65550.2025.11078086. https://ieeexplore.ieee.org/document/11078086.