Gym Pybullet Drones AI Agent

[!TIP] For research work with symbolic dynamics and constraints, also try safe-control-gym

For GPU-accelerated, differentiable, JAX-based simulation, also try crazyflow

For real-world deployment of PX4/ArduPilot + ROS2 + JetPack, use aerial-autonomy-stack

gym-pybullet-drones

This is a minimalist refactoring of the original gym-pybullet-drones repository, designed for compatibility with gymnasium, stable-baselines3 2.0, and betaflight SITL.

NEWS: gym-pybullet-drones was featured in GitHub's Maintainer Spotlight 2026

NOTE: if you want to access the original IROS 2021 codebase, please git checkout [paper|master]

<img src="gym_pybullet_drones/assets/helix.gif" alt="formation flight" width="325"> <img src="gym_pybullet_drones/assets/helix.png" alt="control info" width="425">

Installation

Tested on Intel x64/Ubuntu 24.04 and Apple Silicon/macOS 26.

git clone https://github.com/learnsyslab/gym-pybullet-drones.git
cd gym-pybullet-drones/

conda create -n drones python=3.12
conda activate drones

# Beyond Python 3.10, `pybullet` has no pre-built wheel
# On Ubuntu, install `gcc` to let `pip3 install` build `pybullet`
sudo apt install build-essential
# On macOS, build and install `pybullet` with
CFLAGS="-Dfdopen=fdopen" pip install pybullet --no-cache-dir

pip3 install -e .

# check installed packages with `conda list`, deactivate with `conda deactivate`, remove with `conda remove -n drones --all`

Use

Control examples

cd gym_pybullet_drones/examples/
python3 pid.py
python3 pid_velocity.py
python3 mrac.py

Downwash effect example

cd gym_pybullet_drones/examples/
python3 downwash.py

Reinforcement learning examples (SB3's PPO)

cd gym_pybullet_drones/examples/

# single agent, task: single drone hover at z == 1.0
python learn.py
LATEST_MODEL=$(ls -t results | head -n 1) && python play.py --model_path "results/${LATEST_MODEL}/best_model.zip"

# multi-agent, task: 2-drone hover at z == 1.2 and 0.7
python learn.py --multiagent true
LATEST_MODEL=$(ls -t results | head -n 1) && python play.py --multiagent true --model_path "results/${LATEST_MODEL}/best_model.zip"

<img src="gym_pybullet_drones/assets/rl.gif" alt="rl example" width="375"> <img src="gym_pybullet_drones/assets/marl.gif" alt="marl example" width="375">

Multi-agent RL with PettingZoo

PettingZooWrapper exposes any multi-drone aviary through PettingZoo's Parallel API, with one agent per drone (drone_0, drone_1, ...). That makes it usable with MARL libraries built on PettingZoo.

from gym_pybullet_drones.envs.MultiHoverAviary import MultiHoverAviary
from gym_pybullet_drones.utils.PettingZooWrapper import PettingZooWrapper

env = PettingZooWrapper(MultiHoverAviary(num_drones=3))
observations, infos = env.reset(seed=42)
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)
env.close()

Each agent receives its own row of the aviary's observation. The aviary's team reward and episode flags are shared by all agents, and env.state() returns every drone's kinematics for centralised critics.

cd gym_pybullet_drones/examples/

# MWE based on `learn.py --multiagent true`
python pettingzoo_multiagent.py

Betaflight SITL example (Ubuntu only)

# one-time setup: from the repo's top folder, build one SITL executable per drone (e.g. 2), if needed, `apt install curl`
cd gym-pybullet-drones/
./gym_pybullet_drones/assets/clone_bfs.sh 2

# run the example
cd gym_pybullet_drones/examples/
python3 beta.py --num_drones 2
# --num_drones must be <= the number passed to clone_bfs.sh

Run all tests

# from the repo's top folder
cd gym-pybullet-drones/
pytest tests/

Citation

If you wish, please cite our IROS 2021 paper (and original codebase) as

@INPROCEEDINGS{panerati2021learning,
      title={Learning to Fly---a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control}, 
      author={Jacopo Panerati and Hehui Zheng and SiQi Zhou and James Xu and Amanda Prorok and Angela P. Schoellig},
      booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
      year={2021},
      volume={},
      number={},
      pages={7512-7519},
      doi={10.1109/IROS51168.2021.9635857}
}

References

• Erwin Coumans and Yunfei Bai (2023) PyBullet Quickstart Guide
• Carlos Luis and Jerome Le Ny (2016) Design of a Trajectory Tracking Controller for a Nanoquadcopter
• Nathan Michael, Daniel Mellinger, Quentin Lindsey, Vijay Kumar (2010) The GRASP Multiple Micro-UAV Testbed
• Benoit Landry (2014) Planning and Control for Quadrotor Flight through Cluttered Environments
• Julian Forster (2015) System Identification of the Crazyflie 2.0 Nano Quadrocopter
• Antonin Raffin, Ashley Hill, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, and Noah Dormann (2019) Stable Baselines3
• Guanya Shi, Xichen Shi, Michael O’Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, and Soon-Jo Chung (2019) Neural Lander: Stable Drone Landing Control Using Learned Dynamics
• C. Karen Liu and Dan Negrut (2020) The Role of Physics-Based Simulators in Robotics
• Yunlong Song, Selim Naji, Elia Kaufmann, Antonio Loquercio, and Davide Scaramuzza (2020) Flightmare: A Flexible Quadrotor Simulator

---

UTIAS / Learning Systems and Robotics Lab / Vector Institute / University of Cambridge's Prorok Lab

<!-- ## TODOs - [ ] Add motor delay, advanced ESC modeling by implementing a buffer in `BaseAviary._dynamics()` - [ ] Replace `rpy` with quaternions (and `ang_vel` with body rates) by editing `BaseAviary._updateAndStoreKinematicInformation()`, `BaseAviary._getDroneStateVector()`, and the `.computeObs()` methods of relevant subclasses -->