Robots operating in real-world environments must handle open-ended tasks, novel objects, and instructions from non-expert users. However, current systems often fail to generalize beyond their training data because they lack commonsense knowledge about the world. In this talk, I present a framework for building general-purpose robots by representing and reasoning about commonsense. The framework consists of three components: (1) structured representations of objects, spatial relations, and tasks that capture compositional world knowledge; (2) computational models that jointly reason over language, observation, and action to infer and execute plans; and (3) scalable sources of knowledge, including human demonstrations, natural language, and foundation models. By integrating these components, our approach enables robots to interpret instructions, infer missing information, and adapt their behavior to new environments and tasks. I will demonstrate how this framework improves generalization in long-horizon manipulation and moves toward robots that can assist in everyday settings.
