Robotic Arm Build and Policy Training

Custom robotic arm with reinforcement learning capabilities

Robotic Arm Project

Project Overview

This project involves the development of a custom 6-axis robotic arm with integrated reinforcement learning capabilities. The aim was to create an affordable, yet precise robotic arm that could learn manipulation tasks through trial and error, rather than explicit programming.

The project combines mechanical engineering, electronics, and machine learning to create a platform for exploring advanced robotics concepts without industrial-grade equipment costs.

Technical Specifications

  • • Degrees of Freedom: 6-axis configuration
  • • Motors: Dynamixel XM430-W350 for primary joints, XM430-W210 for wrist
  • • End Effector: Custom 3D printed two-finger gripper
  • • Reach: 60cm maximum extension
  • • Payload: 500g maximum
  • • Control System: ROS2 with custom Python interface
  • • Sensors: Force feedback in gripper, RGB-D camera for perception
  • • Learning Framework: PyTorch with custom reinforcement learning implementations

Development Process

Mechanical Design

The arm was designed in Fusion 360 with a focus on weight optimization and rigidity. Parts were primarily 3D printed in PETG for strength, with some aluminum components for high-stress areas. The design went through three major iterations, each improving on stability and reducing backlash in the joints.

Electronics & Control

The control system uses a Raspberry Pi 4 as the main controller, with a custom PCB for power distribution and signal management. The arm is controlled through a ROS2 framework with custom nodes for trajectory planning, inverse kinematics, and sensor integration. A custom Python API allows for high-level control and integration with the reinforcement learning pipeline.

Policy Training

The reinforcement learning pipeline uses Proximal Policy Optimization (PPO) with a custom reward function. The system was trained on three primary tasks: pick-and-place, object sorting, and simple assembly. Training was conducted in simulation first using PyBullet physics engine, then transferred to the physical robot with domain randomization techniques to bridge the reality gap.

Results & Performance

After approximately 100 hours of training, the arm achieved a success rate of 85% on pick-and-place tasks with novel objects, 78% on sorting tasks, and 65% on simple assembly tasks. The most significant challenge was adapting to varying lighting conditions and object textures, which required additional data augmentation and domain randomization techniques.

Project Gallery

Robotic Arm CAD Design Electronics Assembly Training in Progress Final Assembly

Key Learnings

This project provided valuable insights into the challenges of developing affordable robotic systems with advanced capabilities. Key learnings include:

  • Mechanical precision is crucial for effective reinforcement learning - even small inconsistencies in joint movement can significantly impact policy performance.
  • Simulation-to-reality transfer remains challenging, but domain randomization dramatically improves real-world performance.
  • Dynamixel servos provide an excellent balance of precision and affordability for custom robotics projects.
  • ROS2's improved real-time capabilities make it significantly better than ROS1 for reinforcement learning applications.
  • Task decomposition and curriculum learning accelerate training by an order of magnitude compared to end-to-end learning.

Future work will focus on improving visual perception capabilities and implementing more complex manipulation tasks that require multi-step planning.

Code & Resources

The project code and resources are available on GitHub:

github.com/adamcroft/robotic-arm-rl

The repository includes:

  • CAD files for 3D printing
  • ROS2 control package
  • PyTorch reinforcement learning implementation
  • Simulation environment for training
  • Documentation and build instructions