KARL

Official Implementation of KARL: Kalman-Filter Assisted Reinforcement Learner for Dynamic Object Tracking and Grasping

By Kowndinya Boyalakuntla, Abdeslam Boularias, Jingjin Yu (Rutgers University)

Published at: International Conference on Intelligent Robots and Systems (IROS) 2025

Abstract. We present Kalman-Filter Assisted Reinforcement Learner (KARL) for dynamic object tracking and grasping over eye-on-hand (EoH) systems, significantly expanding such systems’ capabilities in challenging, realistic environments. In comparison to the previous state-of-the-art, KARL (1) incorporates a novel six-stage RL curriculum that doubles the system’s motion range, thereby greatly enhancing the system’s grasping performance, (2) integrates a robust Kalman filter layer between the perception and reinforcement learning (RL) control modules, enabling the system to maintain an uncertain but continuous 6D pose estimate even when the target object temporarily exits the camera’s field-of-view or undergoes rapid, unpredictable motion, and (3) introduces mechanisms to allow retries to gracefully recover from unavoidable policy execution failures. Extensive evaluations conducted in both simulation and real-world experiments qualitatively and quantitatively corroborate KARL’s advantage over earlier systems, achieving higher grasp success rates and faster robot execution speed.

PDF • 5 MIN VIDEO

Getting Started

We provide a Docker image to simplify installation and ensure reproducibility.

1. Install Docker (skip if already installed)

Follow the official instructions:

Docker Engine: https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository
NVIDIA Container Toolkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#with-apt-ubuntu-debian
Configure Docker to use NVIDIA runtime: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#configuring-docker

2. Pull the Docker Image

sudo docker pull kowndidocker/karl:v0

3. Run the Container

sudo bash docker/run_container.sh # This will create a container with the name 'karl_container' and start it.

Container Management

Restart (Fix display issues)

If DISPLAY=:0 is lost:

sudo docker stop karl_container
sudo docker rm karl_container
sudo bash docker/run_container.sh

Start an Existing Container

sudo docker start karl_container
sudo docker attach karl_container

Open a New Terminal in the Container

sudo docker exec -it karl_container bash

Stop the Container

sudo docker stop karl_container

Remove the Container

sudo docker rm karl_container

Code Location Inside Container

<KARL_FOLDER_PATH> # Same as the path on your host machine, but inside the container.

UR5e + Intel L515 Hardware Setup

Camera Calibration

If the camera mount has not been modified, calibration is typically unnecessary as the error is negligible.

Otherwise, follow the MoveIt hand-eye calibration tutorial: https://ros-planning.github.io/moveit_tutorials/doc/hand_eye_calibration/hand_eye_calibration_tutorial.html

Relevant package:

vsg_real_robot/vsg_ws/src/moveit_calibration

Required Configuration

Ensure the joint zdummy_gripper_zcamera_joint is properly defined with calibrated values:

Real Robot URDF:

vsg_real_robot/vsg_ws/src/universal_robot/ur_description/urdf/inc/my_ur_macro.xacro

Simulation URDF:

visual_feedback_rl/assets/urdf/ur5e/ur5e_simplified_gripper.urdf

Camera Mount Mesh

Located at:

visual_feedback_rl/assets/urdf/ur5e/meshes

Robot Pendant Configuration

Select:

gripper_earl

Ensure the following URCaps are installed:

Robotiq
External Control

KARL's Real-Robot Execution (UR5e)

You need open 9 terminals to run the full system inside the created karl_container. In each of these terminals, you need to navigate to the KARL code directory before running the below respective commands.

terminal 1 (ros - ur5 and camera)

conda activate ros
source vsg_ws/setup.sh
roslaunch ur_robot_driver my_ur5e_bringup_vel.launch robot_ip:=172.17.139.103 json_file_path:=<KARL_FOLDER_PATH>/vsg_ws/l515.json

terminal 2 (python - publish robot info)

conda activate ros
source vsg_ws/setup.sh
cd vsg_tracking
python pub_act_jacob_ros_ur5e.py

terminal 3 (python - R2D2 Feature extraction)

cd vsg_tracking/r2d2
conda activate r2d2
python feature_server.py --model models/faster2d2_WASF_N16.pt --top-k 300