LTAMP

Learning for Task and Motion Planning (LTAMP)

Overview

Robotic multi-step manipulation planning for a PR2 robot using both learned and engineered models of primitive actions.

• Learned: pour, push, scoop, stir
• Engineered: move, pick, place, press

References

Zi Wang*, Caelan Reed Garrett*, Leslie Pack Kaelbling, Tomás Lozano-Pérez. Learning compositional models of robot skills for task and motion planning, The International Journal of Robotics Research (IJRR), 2020.

Zi Wang, Caelan R. Garrett, Leslie P. Kaelbling, Tomás Lozano-Pérez. Active model learning and diverse action sampling for task and motion planning, International Conference on Intelligent Robots and Systems (IROS), 2018.

Installation

$ git clone --recursive git@github.com:caelan/LTAMP.git
$ cd LTAMP
LTAMP$ ./setup.bash

Inverse Kinematics (IK)

setup.py - compiles an IKFast analytical IK program for both of the PR2's arms

LTAMP$ cd control_tools/ik/
LTAMP$ control_tools/ik/$ python setup.py build

See README for details about using the existing and generating new IK solvers.

---

Examples

Planning, experimentation, and learning in a simulated PyBullet environment.

Planning

run_simulation.py: tests the planning module in simulation

LTAMP$ python -m plan_tools.run_simulation -h
usage: run_simulation.py [-h] [-c] [-e] [-p {test_block,test_coffee,test_holding,test_pour,test_push,test_shelves,test_stack_pour,test_stacking,test_stir}] [-s SEED] [-v] [-d]

optional arguments:
  -h, --help            show this help message and exit
  -c, --cfree           When enabled, disables collision checking (for debugging).
  -e, --execute         When enabled, executes the plan using physics simulation.
  -p {test_block,test_coffee,test_holding,test_pour,test_push,test_shelves,test_stack_pour,test_stacking,test_stir}, --problem {test_block,test_coffee,test_holding,test_pour,test_push,test_shelves,test_stack_pour,test_stacking,test_stir}
                        The name of the problem to solve.
  -s SEED, --seed SEED  The random seed to use.
  -v, --visualize_planning
                        When enabled, visualizes planning rather than the world (for debugging).
  -d, --disable_drawing
                        When enabled, disables drawing names and forward reachability.

Data Collection

collect_simulation.py: collects manipulation-primitive data in simulation

LTAMP$ python -m learn_tools.collect_simulation -h
usage: collect_simulation.py [-h] [-f FN] [-n NUM] -p {pour,push,scoop,stir} [-t TIME] [-v]

optional arguments:
  -h, --help            show this help message and exit
  -f FN, --fn FN        The parameter function to use.
  -n NUM, --num NUM     The number of samples to collect.
  -p {pour,push,scoop,stir}, --problem {pour,push,scoop,stir}
                        The name of the skill to learn.
  -t TIME, --time TIME  The max planning runtime for each trial.
  -v, --visualize       When enabled, visualizes execution.

---

Modules

Planning

The planning module generates plans using the learned primitives.

Relevant planning submodules:

• pddlstream - Task and Motion Planning (TAMP)
• ss-pybullet - PyBullet Robot Planning

Learning

The learning module conducts manipulation-primitive data collection experiments and learns models from the collected data.

Control

The control module provides an interface for executing both simulated and real motion.