In the past, robots have been required to be explicitly programmed to carry out tasks. Learning robots differ from this notion by allowing them to learn from experience, which allows them to get better the more they perform a task. This chapter will covers the basics of learning in robotics and give an overview of how machines can learn skills via reinforcement learning (RL). Topics that will be discussed include states, actions, rewards, and policy to better help understand how agents can learn from experience to create rewarding behaviours. Practical implementations will also be discussed, such as training robots in simulation before placing them into the real world. Important topics such as training in stochastic environments will also be discussed to help the reader better understand some of the difficulty that comes when training in these environments. To help exemplify some of these topics, Google Everyday Robot will be used as a case study to discuss some of the successes and failures of learning robots in the real world. By reviewing some of the issues with training in the real world vs. simulation, learners will gain a better understanding of some of the limitations. By the end of this chapter, learners will have a basic understanding of how robots can learn from their environment. Learners will be able to use this information to help train robots to best behave in uncertain environments.
Programming lies at the heart of intelligent robotics, and modern robotic systems are extremely dependent on software to make sense of sensory input from the outside world, make decisions, and drive motors. Typically, these activities are performed using modular code architecture connected by a middleware layer such as ROS 2 so that disparate parts of the system can communicate with one another. Concepts from programming languages are taught through the lens of Python, illustrating how robots can sense their environment, compute control commands, and perform actions. Topics covered in this chapter will show students how to create reactive robot behaviours that adapt to their surroundings. Fundamental programming concepts such as storing information in variables, making decisions with conditionals and loops, and using functions to write reusable code blocks are reviewed. Topics related to representing and sharing data, especially between concurrent processes like the ones found on robots, are also discussed. Students will gain hands-on experience with how robots perceive their environment and make decisions to follow a line using code to connect sensing, logic, and actuation. Key concepts such as writing modular, readable, and testable code are stressed. Debugging techniques are also demonstrated.
Robotic systems require well-organised and maintainable software to operate as intended. This chapter presents fundamental concepts from software engineering applied to robotics. Students will learn how to approach the design, implementation, and maintenance of complex robotic systems. Topics covered include the software development life cycle, object-oriented design principles, version control tools, and Agile development methodologies to promote organised and iterative development. Additionally, software design patterns that enable modularity and scalability will be discussed. Modularity allows systems to be decomposed into interchangeable components that can be developed and tested independently. By creating reusable components, students will learn how to scale their software to meet increasing complexity. These concepts will be demonstrated through examples in ROS, a popular robotics framework that facilitates communication between distributed components in robotic systems. To solidify their understanding, students will work in groups to iteratively design a modular ROS-based controller. This project is meant to emphasise the importance of teamwork, communication, and flexibility when working in multidisciplinary teams. Upon completion of this chapter, students will have the tools necessary to design, maintain, and manage complex robotics software systems.
