ROS-LLM: A Framework for Embodied AI

In this work, we present a framework for embodied artificial intelligence (AI). Our proposed system integrates a Large language model (LLM) based AI agent with the popular Robot Operating System (ROS). Key features of the framework include: integration of ROS with an AI agent connected to many open-source and commercial LLMs, automatic extraction of a behavior from the LLM output and execution of ROS actions/services, support for several behavior execution modes (e.g. code, behavior tree), imitation learning for adding new actions to the atomic action library, automated atomic action optimization, and LLM reflection via human and environmental feedback. Extensive experiments validate the framework, showcasing robustness, scalability, and versatility in diverse scenarios and embodiments, including long-horizon tasks, tabletop rearrangements, dynamic task optimization, and remote supervisory control. Moreover, all the results presented in this work were achieved by utilizing open-source models.