Reinforcement Learning for an Aerial Manipulator

Project Overview

This final-year group project focuses on developing an RL-based dynamic compensation controller for a hexacopter equipped with a 3-DoF robotic arm. The goal is to achieve stable flight and reliable task execution while the manipulator moves, which introduces forces and torques that disturb the UAV. The work is developed in simulation first and later validated on hardware.

Supervisor
Supervisor: Prof. Rohan Munasinghe

Motivation

Aerial manipulators can support inspection, interaction, and pick-and-place tasks in environments that are difficult or unsafe for humans. However, combining UAV flight control with arm motion creates strong coupling effects and stability challenges. This project aims to address those challenges using learning-based control alongside a safe flight baseline.

My Contribution (Main Focus Areas)

This is a group project, and my work mainly focuses on the simulation and learning pipeline:

• Hexacopter modeling in Gazebo: I am creating and refining the UAV model for realistic simulation behavior, including mass, inertia, geometry, and actuator setup.
• Hexacopter + arm integration: I am integrating the manipulator with the hexacopter model to represent a complete aerial manipulator system.
• PX4 SITL integration: I am setting up and validating the simulation workflow using PX4 SITL with the simulation environment.
• RL training + environment scripting: I am developing the RL training workflow and Gymnasium-based environment scripting to support policy learning and evaluation.
• Literature review: I am reviewing and summarizing key research papers to guide controller choices, training strategy, and simulation-to-real considerations.

Technical Approach (High Level)

The system uses a physics-based simulator for the aerial manipulator and a training pipeline to learn a compensation strategy under arm-induced disturbances. The approach emphasizes repeatable experiments, clear evaluation metrics, and gradual progression from simulation to real-world validation.