Free Foil

Generating Efficient and Scope-Safe Abstract Syntax with Free (Scoped) Monads and Template Haskell (with a partial port to Scala).

About

This project provides an implementation of the efficient scope-safe representation for syntax with binders (think λ-abstraction, let-binding, for-loop, etc.). The underlying representation is based on the IFL 2022 paper by Maclaurin, Radul, and Paszke «The Foil: Capture-Avoiding Substitution With No Sharp Edges»¹. This project extends the foil with patterns, as well as two techniques for free generation of the foil. The details are presented in the paper «Free Foil: Generating Efficient and Scope-Safe Abstract Syntax»². Also see the slides from the ICCQ 2024 presentation (also video).

In brief, following the paper:

1. We introduce CoSinkable typeclass to generalize NameBinders to more complex patterns.
2. We provide Template Haskell machinery to generate the proper scope-safe AST definitions as well as conversion functions, and helpers for patterns. This approach works particularly well with the code generated by tools like BNFC or BNFC-meta.
3. We define a variant of free scoped monads³ with the foil instead of nested data types of Bird and Paterson. This approach allows implementing certain functions once for a large class of languages with binders. Here we implement only substitution, but more involved algorithms, such as generic higher-order preunification³ should be possible.

Additionally, we implement:

1. Template Haskell generation for the free foil.
2. Generalized support for patterns using withPattern method (currently in CoSinkable).
3. Generic α-equivalence for the free foil and α-equivalence support for patterns using UnifiablePattern typeclass.
4. Generic conversion functions for the free foil.

In addition to this repository, we have benchmarks for the foil and the free foil against each other and other implementation of λ-calculus. See KarinaTyulebaeva/lambda-n-ways, a fork of Stephanie Weirich's benchmark suite (sweirich/lambda-n-ways).

Project structure

Haskell

The Haskell code is organized into two packages as follows:

free-foil package

• Control.Monad.Foil provides basic definitions for the foil¹, including safe scopes, sinking, name maps, and scope constraints. In addition to the standard definitions, we contribute

  • CoSinkable class (dual to Sinkable, for patterns that generalize NameBinder)
  • a general total NameMap which is useful for some implementations (e.g. conversion functions)
  • unification of binders, which is useful for efficient α-equivalence implementations

• Control.Monad.Foil.TH provides Template Haskell functions that generate scope-safe representation types together with some conversion functions and helpers for patterns. All of this is generated from a raw recursive representation of syntax, that has to be split into 4 types: type of variable identifiers, type of terms, type of scoped terms (to clearly understand which parts get under binders), and type of patterns.

• Control.Monad.Foil.Relative defines a variation of a relative monad typeclass specifically indexed by scope type variables of kind S. This is merely for the demonstration that term type constructors are monads relative to Name.

• Control.Monad.Free.Foil defines a variation of free scoped (relative) monads relying on the foil for the scope-safe efficient handling of the binders. This module also defines

  • generic substitution for this generic representation of syntax with binders
  • generic α-normalization and α-equivalence checks
  • generic conversion helpers

• Control.Monad.Free.Foil.TH provides Template Haskell functions that generate helpers specifically for free foil, including the generation of the signature bifunctor, convenient pattern synonyms, conversion helpers, and instances needed to enable general α-equivalence.

lambda-pi package

• Language.LambdaPi.Impl.Foil defines a simple interpreter for λΠ-calculus with pairs, using the manually constructed scope-safe representation and conversion to and from the raw representation generated by BNFC.

• Language.LambdaPi.Impl.FoilTH defines a simple interpreter for λΠ-calculus with pairs, using Template Haskell to generate a large portion of the code.

• Language.LambdaPi.Impl.FreeFoil defines a simple interpreter for λΠ-calculus with pairs, using the free foil approach.

• Language.LambdaPi.Impl.FreeFoilTH defines a simple interpreter for λΠ-calculus with pairs, using the free foil approach together with Template Haskell generation. This implementation has the most automation and generality.

Scala

Scala implementation is not documented yet.

What is not (yet) in this project

In Haskell:

1. We do not (yet) generate α-equivalence via Template Haskell or GHC.Generics. However, we do provide generic α-equivalence functions for AST sig n, and demonstrate how to implement the check manually for λΠ-terms using the foil.
2. We do not (yet) demonstrate an implementation of the typechecker for λΠ in these representations. It should be largely straightforward, since the main non-trivial part is probably the equality for the Π-types, which relies on the α-equivalence from the previous item.
3. We provide general support for patterns, but at the moment it feels somewhat ad hoc (pending proper generalization of patterns) and also does not allow terms or scoped terms inside patterns. Supporting terms in patterns would be desirable to enable type annotations in patterns or things like ViewPatterns in Haskell.
4. We do not (yet) provide strict versions of the foil and free foil here. The benchmarks, however, do implement these variations.
5. We derive CoSinkable and Sinkable instances via Template Haskell at the moment. However, a safer and more flexible derivation should be possible via GHC.Generics.
6. Many functions, including rbind take in an extra argument of type Scope n, which is meant to be the runtime counterpart of the (phantom) type parameter n :: S. It might be possible to use singletons or a similar technique to avoid explicit passing of this argument, making the interface somewhat cleaner. However, since this would change the interface a lot, we do not plan on implementing this approach here.
7. Dealing with scopes generically might enable generic delayed substitution, which plays a big part in normalization by evaluation (NbE) algorithms (which outperform naïve substitution-based evaluation). It should be possible to provide a generic framework for closures of scoped terms and delayed substitutions, but we have not yet investigated that fully.
8. Free foil should allow the same generic implementation of higher-order unification as free scope monads². In fact, the parametrized metavariables should be added in the same data types à la carte fashion. However, we do not (yet) provide any generic binder-sensitive algorithms apart from substitution and α-equivalence.
9. We do not (yet) provide a Church-encoded version of the free foil, which should improve its performance.

Footnotes

1. Dougal Maclaurin, Alexey Radul, and Adam Paszke. 2023. The Foil: Capture-Avoiding Substitution With No Sharp Edges. In Proceedings of the 34th Symposium on Implementation and Application of Functional Languages (IFL '22). Association for Computing Machinery, New York, NY, USA, Article 8, 1–10. https://doi.org/10.1145/3587216.3587224 ↩ ↩²

2. Nikolai Kudasov, Renata Shakirova, Egor Shalagin, Karina Tyulebaeva. 2024. Free Foil: Generating Efficient and Scope-Safe Abstract Syntax. 4th International Conference on Code Quality (ICCQ), Innopolis, Russian Federation, 2024, pp. 1-16 http://doi.org/10.1109/ICCQ60895.2024.10576867 (arXiv version at https://arxiv.org/abs/2405.16384) ↩ ↩²

3. Nikolai Kudasov. Free Monads, Intrinsic Scoping, and Higher-Order Preunification. To appear in TFP 2024. https://arxiv.org/abs/2204.05653 ↩ ↩²