SayCan

SayCan addresses the challenge of getting robots to follow high-level natural language instructions, which often fail because language models don't understand real-world contexts. It works by combining large language models, like PaLM, with robotic affordances—essentially, it uses probability scores to pick skills that are both useful for the task and feasible in the environment. For example, if you tell it 'I spilled my drink, can you help?', it might plan steps like finding a sponge and bringing it to you, all while checking if those actions are possible. This approach benefits robotics researchers and developers by enabling robots to handle long, multi-step tasks without manual intervention, and it even supports multilingual instructions and reasoning through chain-of-thought prompts. However, the initial version doesn't incorporate real-time feedback, so it might struggle with unexpected changes or failures, though follow-up work is improving this.