The International Planning Competition (IPC) happens every ~2 years since 1998. Its main goal is to provide a common dataset made of several planning instances written in the same language, PDDL, that are close to real world applications. Planners compete in several tracks, searching for step-optimal (smaller plans) or satisficing (less planning time) solutions and exploiting common or uncommon description features, such as numerics and optimization. There was no common language for Hierarchical Task Network (HTN), although the second (2000) and third (2002) IPC had tracks on 'Hand Tailored Systems'. Most systems relied on some input format of their own, making comparisons hard to happen. HTN IPC was proposed during ICAPS 2019 to improve this situation, pushing HDDL as the standard HTN language.
The IPC 2020 proposed two tracks: total order and partial order. In the total order track the subtasks must be decomposed following one order, either respecting the order in which tasks were described or sorting them based on their constraints. The partial order track contain domains in which there is more than one way to decompose tasks, which also makes possible to interleave subtasks, at the cost of complex bookkeeping. Few domains were acyclic (subtasks that may decompose to themselves) and therefore no extra tracks happened. The IPC domains and submitted domains are available online.
HyperTensioN participated in the total order track. Note that the HyperTensioN used in the IPC 2020 is slightly different from the original, it is available in its competition repository. During the competition only HDDL_Parser, Typredicate, Pullup, Dejavu and Hyper_Compiler modules were loaded by Hype. The debug outputs in the planning method were commented out. A few bugs made the competition release of HyperTensioN not able to parse Entertainment and Monroe (partially and fully observable) domains correctly, these are now fixed. The new timings were obtained in an Intel E5500 @ 2.8GHz with 3.25GB of RAM, which match IPC timings. Currently, the first 5 of 12 Entertainment instances are solved in less than a second, the sixth in 42s, the seventh in 740s, and the eighth in 235s, while others will take more than 1800s. All 20 Monroe-Fully-Observable instances are solvable, most in few seconds and the last two in 32s. The Monroe-Partially-Observable instances are still not solvable within a 1800s time limit. Other domains are unaffected by this change. The following table does not represent the current state of HyperTensioN, due to new extensions and compilers, only the difference between the IPC results and the bug fixes that happened soon after the competition.
| Domain/Planner | Total | HyperTensioN (2.1) | HyperTensioN (IPC) | Lilotane | PDDL4J-TO | PDDL4J-PO | HPDL | pyHiPOP |
|---|---|---|---|---|---|---|---|---|
| AssemblyHierarchical | 30 | 3 | 3 | 5 | 2 | 1 | 0 | 0.5 |
| Barman-BDI | 20 | 20 | 20 | 16 | 11 | 5.5 | 10 | 0 |
| Blocksworld-GTOHP | 30 | 16 | 16 | 22.1 | 16 | 8.5 | 6.6 | 0.5 |
| Blocksworld-HPDDL | 30 | 30 | 30 | 1 | 0 | 0 | 0 | 0 |
| Childsnack | 30 | 30 | 30 | 29 | 20.9 | 10.5 | 11 | 0 |
| Depots | 30 | 24 | 24 | 23.4 | 23 | 11.4 | 11 | 0 |
| Elevator-Learned | 147 | 147 | 147 | 147 | 2 | 1 | 5.5 | 1 |
| Entertainment | 12 | ~5.9 | 0 | 4.6 | 4.6 | 1.5 | 0 | 0.5 |
| Factories-simple | 20 | 3 | 3 | 4 | 0 | 0 | 0 | 0.5 |
| Freecell-Learned | 60 | 0 | 0 | 7.7 | 0 | 0 | 0 | 0 |
| Hiking | 30 | 25 | 25 | 21.3 | 17 | 7.3 | 0 | 0 |
| Logistics-Learned | 80 | 22 | 22 | 43.2 | 0 | 0 | 0 | 0 |
| Minecraft-Player | 20 | 5 | 5 | 1 | 1 | 0.5 | 1.5 | 0 |
| Minecraft-Regular | 59 | 57.1 | 57.1 | 29.2 | 23 | 11.5 | 17.5 | 0 |
| Monroe-FO | 20 | ~17.7 | 0 | 20 | 20 | 9.9 | 3.2 | 0 |
| Monroe-PO | 20 | 0 | 0 | 20 | 1 | 0.5 | 0 | 0 |
| Multiarm-Blocksworld | 74 | 8 | 8 | 4 | 0 | 0 | 0.5 | 0 |
| Robot | 20 | 20 | 20 | 11 | 6 | 3 | 0 | 0.5 |
| Rover-GTOHP | 30 | 30 | 30 | 21.3 | 27.5 | 12.8 | 15 | 3 |
| Satellite-GTOHP | 20 | 20 | 20 | 15 | 20 | 5 | 0 | 3.5 |
| Snake | 20 | 20 | 20 | 17.1 | 20 | 10 | 3.5 | 1 |
| Towers | 20 | 17 | 17 | 10 | 16 | 7.5 | 5.5 | 1 |
| Transport | 40 | 40 | 40 | 35 | 33.2 | 16.5 | 0.5 | 8.6 |
| Woodworking | 30 | 7 | 7 | 30 | 6 | 3 | 1.5 | 2 |
| Total | 892 | ~567.7 | 544.1 | 537.9 | 270.2 | 126.9 | 92.8 | 22.5 |
The planner was executed as ruby --disable=all Hype.rb $DOMAINFILE $PROBLEMFILE typredicate pullup dejavu run to save a few milliseconds from Ruby start up time.
Due to a limit in the amount of stack available to the Ruby interpreter in the Ubuntu 20.04 + Ruby 2.7 it was decided to use an older version, Ubuntu 18.04 + Ruby 2.5, to be able to solve more planning instances.
Some large planning instances require more stack, which is possible with export RUBY_THREAD_VM_STACK_SIZE=$(($MEMORY * 512 * 1024)).
HyperTensioN did not exploit the seed variable provided during the competition, although it is possible that randomizing parts of the planning instance may improve timing in certain domains.
A more detailed explanation is provided in the IPC paper HyperTensioN: A three-stage compiler for planning and the extended paper HyperTensioN and Total-order Forward Decomposition optimizations.
The effects of each extension in the domain structures can be visualized online in the HyperTensioN HTN IPC 2020 Explorer.
The plan format output required by the IPC to analyze plan correctness was different from the one used by HyperTensioN. Some examples and tests expect the original non-IPC output, the reason why it is disabled by default. The IPC output format was hardcoded during the competition, but now can be enabled by defining the IPC flag to the Ruby interpreter or C++ compiler, note that this adds a small overhead.
ruby -s Hype.rb -IPC $DOMAINFILE $PROBLEMFILE {extensions} runruby Hype.rb $DOMAINFILE $PROBLEMFILE {extensions} cpp && CXX -D IPC -O3 $DOMAINFILE.cpp -o $DOMAINFILE.bin && ./$DOMAINFILE.bin
Plans in the IPC format can be visualized using the HTN Plan Viewer. The competition results are available, and the presentation is on YouTube:
