DanielBrice.net

Math-Haskell Rosetta Stone - Part 1

Tue, 24 Sep 2024 00:00:00 +0000

This post begins a short series meant to serve as an informal guide to reading Haskell code and translating back and forth with mathematics. It’s meant to help members of r/CategoryTheory understand posts that use Haskell code to convey ideas. My hope is that this series should also find use among Haskell programmers, as exposure to some of the basic methods and terminology used in modern math.

Part 1 (this post)
Part 2 (pending)
- Type variables
- Type classes
- Constructor classes
Part 3 (pending)
- Kinds
- Type-level programming

Basic types and functions

Math is concerned with defining sets and functions. Analogously, Haskell code is concerned with defining types and functions.

data RealVec2 = RV2 RealNumber RealNumber

rv2Norm :: RealVec2 -> RealNumber
rv2Norm (RV2 u v) = sqrt (u**2 + v**2)

rv2Scale :: RealNumber -> RealVec2 -> RealVec2
rv2Scale r (RV2 x y) = RV2 (r*x) (r*y)

rv2Add :: RealVec2 -> RealVec2 -> RealVec2
rv2Add (RV2 a1 a2) (RV2 b1 b2) = RV2 (a1+b1) (a2+b2)

The above code roughly translate to the following definitions:

A set $\mathrm{RealVec2}$,

\[\mathrm{RealVec2} = \{ (\mathrm{RV2}, x, y) \mid x, y \in \mathbb{R} \}\]

A single-variable function $\mathrm{rv2Norm}$,

\[\mathrm{rv2Norm} : \mathrm{RealVec2} \to \mathbb{R} \\ \mathrm{rv2Norm}\big((\mathrm{RV2}, u, v)\big) = \sqrt{u^2 + v^2}\]

A two-variable function $\mathrm{rv2Scale}$,

\[\mathrm{rv2Scale} : \mathbb{R} \times \mathrm{RealVec2} \to \mathrm{RealVec2} \\ \mathrm{rv2Scale}\big( r, (\mathrm{RV2}, x, y) \big) = ( \mathrm{RV2}, rx, ry )\]

A two-variable function $\mathrm{rv2Add}$, and

\[\mathrm{rv2Add} : \mathrm{RealVec2} \times \mathrm{RealVec2} \to \mathrm{RealVec2} \\ \mathrm{rv2Add}\big( (\mathrm{RV2}, a_1, a_2), (\mathrm{RV2}, b_1, b_2) \big) = ( \mathrm{RV2}, a_1 + b_1, a_2 + b_2 )\]

A two-variable function $\mathrm{RV2}$.

\[\mathrm{RV2} : \mathbb{R} \times \mathbb{R} \to \mathrm{RealVec2} \\ \mathrm{RV2}(r_1, r_2) = ( \mathrm{RV2}, r_1, r_2 )\]

Straightforward enough (except maybe for the $\mathrm{RV2}$ bit). Let’s dig into the syntax of the Haskell code to understand each part.

data RealVec2 = RV2 RealNumber RealNumber
-- ^ `data`, keyword that begins a type definition
--   ^^^^^^^^ `RealVec2`, the name we give this function
--              ^^^ `RV2`, a "data constructor", or "tag", we define
--                  ^^^^^^^^^^ ^^^^^^^^^^ `RealNumber`, a reference to an already-defined type
--                                        we define `RV2` as having two "fields," or "arguments"
--                                        each field contains a data object of type `RealNumber`

Pattern matching and variables

The tag RV2 does a number of things.

It ensures that the type RealVec2 has no members in common with any other type (this is one reason why it’s called a tag).
It acts as a function that allows the programer to create data objects of type RealVec2. This is why it’s called a data constructor, and we use RV2 in this capacity in the formulae of rv2Scale and rv2Add.
Third, notice that RV2 is an injective function, allowing us to use RV2 to refer to arbitrary members of RealVec2. This is the main sense in which RV2 is a tag: every member of RealVec2 has a unique form RV2 x y for some particular x and y in RealNumber. We take advantage of this fact in the definitions of rv2Norm, rv2Scale, and rv2Add.

rv2Norm :: RealVec2 -> RealNumber
-- ^^^^ define a function named `rv2Norm`
--      ^^ `rv2Norm` has type `RealVec2 -> RealNumber`
--         ^^^^^^^^ `rv2Norm` has domain type `RealVec2`
--                     ^^^^^^^^^^ `rv2Norm` has codomain type `RealNumber`

rv2Norm (RV2 u v) = sqrt (u**2 + v**2)
--                ^ equation defining `rv2Norm`
--      ^^^^^^^^^ An arbitrary member of `RealVec2` must match this pattern
--                  ^^^^^^^^^^^^^^^^^^ formula that computes the value of `rv2Norm (RV2 u v)`

Expressing a data object in terms of a data constructor is called pattern matching. Pattern matching is ubiquitous in Haskell. To define rv2Norm, we match its argument against the pattern that the RV2 data constructor follows: namely, the symbol RV2 followed by two variables representing the data objects contained in the fields of RV2. We do likewise in the definitions of rv2Scale and rv2Add.

rv2Scale :: RealNumber -> RealVec2 -> RealVec2
rv2Scale r (RV2 x y) = RV2 (r*x) (r*y)
--       ^ `r`, a variable representing the first argument of `rv2Scale`
--         ^^^^^^^^^ `(RV2 x y)`, pattern matching the second argument of `rv2Scale`
--              ^ `x`, a variable representing the contents of the first field of `RV2`
--                ^ `y`, a variable representing the contents of the second field of `RV2`
--                     ^^^ `RV2`, used to construct a data object
--                         ^^^^^ `(r*x)`, the contents we intend for the first field
--                               ^^^^^ `(r*y)` the contents we intend for the second field

rv2Add :: RealVec2 -> RealVec2 -> RealVec2
rv2Add (RV2 a1 a2) (RV2 b1 b2) = RV2 (a1+b1) (a2+b2)
--     ^^^^^^^^^^^ pattern match the first argument of `rv2Add`
--                 ^^^^^^^^^^^ pattern match the second argument of `rv2Add`
--          ^^ `a1`, name we choose for the first field of the first argument
--             ^^ `a2`, name we choose for the second field of the first argument
--                      ^^ `b1` we name we choose for the first field of the second argument
--                         ^^ `b2`, name we choose for the second field of the second argument

Again, the names we choose for variables are arbitrary.

Before we move on, I should take a moment to be clear about what I mean by the word variable in this context. You might already know that programmers often use the word variable to refer to a region of program memory that can be referenced, read from, and written to by the program at various times during execution. That is not what I mean by variable in this sense. In this post, I use variable to mean an abstract symbol used as a placeholder for a functions eventual argument, that is, the usual meaning of the word, as it’s used in math.

A note on multi-variable functions

Next, it’s time to have a little talk about the number of variables a Haskell function has.

rv2Scale :: RealNumber -> RealVec2 -> RealVec2
-- ^^^^^ defining a function named `rv2Scale`
--          ^^^^^^^^^^ domain type is `RealNumber`
--                        ^^^^^^^^^^^^^^^^^^^^ codomain type is `RealVec2 -> RealVec2`
rv2Scale r (RV2 x y) = RV2 (r*x) (r*y)

When reading type signatures, the right-most function arrow (the ->) takes precedence. The above code would be equivalent to the following (valid) code.

rv2Scale :: RealNumber -> (RealVec2 -> RealVec2)
rv2Scale r = f
  where f (RV2 x y) = RV2 (r*x) (r*y)

Earlier, I conceived of the math counterpart to rv2Scale as mapping $\mathbb{R} \times \mathrm{RealVec2}$ to $\mathrm{RealVec2}$. Were I to maintain fuller fidelity to the meaning of the Haskell code, though, I would have used the following instead.

\[\mathrm{rv2Scale} : \mathbb{R} \to \mathop{\mathrm{Hom}}\big(\mathrm{RealVec2},\mathrm{RealVec2}\big)\]

A Haskell function may take one, and only one, argument. When we speak of a multi-variable function in Haskell, we really mean a function of one variable that returns a function. This might seem odd at first, but please consider that Haskell actually agrees with our notion of function arguments in math. When we speak colloquially of a two-variable (math) function, what we truly speak of is a function of a single variable—the domain merely happens to be a cartesian product.

\[\mathrm{rv2Scale} : \underbrace{\mathbb{R} \times \mathrm{RealVec2}}_{\text{this is still just one set!}} \to \mathrm{RealVec2} \\ \mathrm{rv2Scale} \underbrace{\big( r, (\mathrm{RV2}, x, y) \big)}_{\text{this is still just one argument!}} = ( \mathrm{RV2}, rx, ry )\]

As it is in math, so it is in Haskell: all functions are single-variable functions.

But still, instead of returning a function, I could have defined rv2Scale as taking a pair as argument. Haskell has built-in syntax for ordered pairs, even. The type (RealNumber, RealVec2) is the Haskell analog of the set $\mathbb{R} \times \mathrm{RealVec2}$. We could have used that to define rv2Scale.

rv2Scale :: (RealNumber, RealVec2) -> RealVec2
rv2Scale (r, RV2 x y) = RV2 (r*x) (r*y)

The reason haskellers favor using the formulation RealNumber -> (RealVec2 -> RealVec2) (which is call the curried form) over the equivalent formulation (RealNumber, RealVec2) -> RealVec2 (which is call the uncurried form) is simply due to Haskell’s syntax. You must have noticed by now that Haskell’s syntax rules allow you to omit the parentheses normally used to indicate function application. Function application parentheses may be omitted whenever the argument consists of a single symbol.

rv2Reflect :: RealVec2 -> RealVec2
rv2Reflect (RV2 x y) = RV2 y x
--                         ^ first argument is a single-symbol, no parentheses needed
--                           ^ second argument is a single-symbol, no parentheses needed
--         ^^^^^^^^^ argument is a compound expression, requires parentheses

Because Haskell allows you to omit function-application parentheses, the curried formulation ends up being far more convenient to use and (arguably) easier to read than the uncurried formulation. There isn’t a deeper reason than that.

In summary, the code we’ve seen so far defines:

a type, RealVec2;
a function RV2, with domain type RealNumber and codomain type RealNumber -> RealVec2;
a function rv2Norm, with domain type RealVec2 and codomain type RealNumber;
a function rv2Scale, with domain type RealNumber and codomain type RealVec2 -> RealVec2;
a function rv2Add with domain type RealVec2 and codomain type RealVec2 -> RealVec2; and
a function rv2Reflect with domain type RealVec2 and codomain type RealVec2.

Since their codomains are function types, rv2Scale, rv2Add, and RV2 all yield functions. Even though it’s a little bit of a lie, it’s often convenient to think of such function-yielding-functions as having multiple variables.

Think of rv2Scale as having two inputs, with types RealNumber and RealVec2, and an output of type RealVec2.
Think of rv2Add as having two inputs, both with type RealVec2, and an output of type RealVec2.
Think of RV2 as having two inputs, both with type RealNumber, and an output of type RealVec2. RV2 is injective when thought of this way, a fact we use when we pattern match.

Readers may choose to skip this next aside, unless they feel like really nerding out.

Often when used in programming, the word function means something more like procedure or subroutine. In most programming languages, functions tend to “do” things; they’re state transitions, reading from and operating on state variables defined in the running program. Such functions might have no explicit arguments, as they can read state variables defined in the program. Similarly, they might not have a explicit return values, as such functions can communicate their data by modifying the values of state variables defined in the program. In this manner, program state variables act as implicit arguments and return values, accessible to all function in that program. Programmers seem to like this state of affairs, at it makes it trivially easy to propagate data throughout the program.

From a mathematical point of view, modelling such functions as sets of ordered pairs can prove challenging. Suppose $S$ is the set of all possible states of a program $P$, and suppose f is a function in $P$ with set of permissible inputs $A$ and set of potential outputs $B$. We model f mathematically via a counterpart function $f : A×S → S×B$. By including an $S$ component in the domain and codomain, $f$ makes explicit the dependencies and consequences f has on program state. This approach usually works well enough for modelling single-threaded programs. It is utterly inadequate for modelling multi-threaded programs, as the presence of simultaneous threads of computation can cause the $S$ argument to change unpredictably midway through computing f.

Functions in Haskell can’t do any of that. The language syntax simply makes it impossible to describe state variables, let alone reading or writing to them. So, a function in Haskell is a genuine function, in the sense that it yields a computed value that is completely determined by its arguments, conceptually equivalent to a set of ordered pairs. As a result, data propagation in Haskell is always limited entirely to functions’ arguments and its computed results.

Simple sum types and branching

Haskell ships with a standard library called base. base comes with several useful data types predefined. Among them are Bool and Ordering.

data Bool = False | True
data Ordering = LT | EQ | GT

In math, they would look something like this.

\[\mathrm{Bool} = \{ \mathrm{False}, \mathrm{True} \} \\ \mathrm{Ordering} = \{ \mathrm{LT}, \mathrm{EQ}, \mathrm{GT} \}\]

Each of False, True, LT, EQ, and GT is a data constructor with no fields. Conceptually they are constants, uniquely identified and referenced by their respective names.

Branching logic

The base library defines various operations on these data types, such as the familiar boolean operations.

not :: Bool -> Bool
not True = False
not False = True

(||) :: Bool -> Bool -> Bool
True || True = True
True || False = True
False || True = True
False || False = False

(&&) :: Bool -> Bool -> Bool
p && q =
  case (p, q) of
    (True, True) -> True
    (True, False) -> False
    (False, True) -> False
    (False, False) -> False

Here we see our first examples of branching logic in a Haskell function. Fundamentally, all branching in Haskell is done by pattern matching.

The branching in not and (||) is accomplished by defining several different equations. Each such equation matches a different pattern of the arguments, in other words, a different combination of data constructors.
(&&) branches using a case expression. Each case of the case expression matches a different pattern.

I could have defined (&&) with multiple equations, like (||), but I wanted to have an example that used a case expression.

We also see our first example of operators. By giving them symbolic names, Haskell knows that we intend the functions (&&) and (||) to be binary operators. This is reflected in the equations defining these functions.

Final note on these before moving on, each of (&&) and (||) really just needs two cases. In fact, defining these functions this way may cause the program to do more computations than necessary. Accordingly, Haskell syntax allows us to use an underscore (the character _) to indicate a match-anything pattern.

(||) :: Bool -> Bool -> Bool
False || False = False
_ || _ = True

(&&) :: Bool -> Bool -> Bool
True && True = True
_ && _ = False

If-then-else and guards

base provides a type Double that implements IEEE 754 double-precision floating point arithmetic. Let’s define a function that compares Doubles and yields an Ordering.

compareDoubles :: Double -> Double -> Ordering
compareDoubles x y =
  case x < y of
    True -> LT
    False -> case x > y of
      True -> GT
      False -> EQ

x < y is a member of Bool, which we then pattern match. Pattern matching a member of Bool is such common idiom that Haskell has a syntactic shortcut for it.

compareDoubles :: Double -> Double -> Ordering
compareDoubles x y =
  if x < y
    then LT
    else if x > y
      then GT
      else EQ

We can style that in various different ways, to our preference.

compareDoublesAlt1 :: Double -> Double -> Ordering
compareDoublesAlt1 x y =
  if x < y then
    LT
  else if x > y then
    GT
  else
    EQ

compareDoublesAlt2 :: Double -> Double -> Ordering
compareDoublesAlt2 x y =
  if x < y then LT else
  if x > y then GT else
  EQ

compareDoublesAlt3 :: Double -> Double -> Ordering
compareDoublesAlt3 x y = if x < y then LT else if x > y then GT else EQ

All of those are still a bit clunky, so Haskell syntax has a shorthand for checking a series of conditions. The syntax is reminiscent of defining a piecewise function.

compareDoubles :: Double -> Double -> Ordering
compareDoubles x y
  | x < y = LT
  | x > y = GT
  | otherwise = EQ

The translation to math is straightforward.

\[\mathrm{compareDoubles} : \mathrm{Double} \times \mathrm{Double} \to \mathrm{Ordering} \\ \mathrm{compareDoubles}(x, y) = \begin{cases} \mathrm{LT} & \text{if $x < y$} \\ \mathrm{GT} & \text{if $x > y$} \\ \mathrm{EQ} & \text{otherwise} \end{cases}\]

Algebraic data types

Recall the types we’ve seen so far.

data RealVec2 = RV2 RealNumber RealNumber

data Bool = False | True

data Ordering = LT | EQ | GT

RealVec2 builds a type by packaging data objects from several other types into a single conceptual unit. Virtually all programming languages have this feature, known under various different names. Some call such types structs, some languages call them objects, other languages call them records, and so on.

Bool and Ordering are defined by enumerating their members. Many programming languages support defining enumeration types, or enums, similarly.

In Haskell lingo, RealVec2 is an example of a product type. Bool and Ordering as special cases of sum types, though we will see sum types that are much more general. We will now see that the terminology, “product” and “sum,” is deserved.

Product types and universal projection maps

If you’re very clever, you may already have recognize RealVec2 as a categorical product of RealNumber with itself. We need to define the universal projection maps and verify the universal product property to make sure we’re right.

Claim 1

RealVec2 is a product of RealNumber and RealNumber, with projection maps p1 :: RealVec2 -> RealNumber and p2 :: RealVec2 -> RealNumber defined by p1 (RV2 x _) = x and p2 (RV2 _ y) = y.

Proof

Let T be a type, and suppose we have f1 :: T -> RealNumber and f2 :: T -> ReadNumber. Define f :: T -> RealVec2 by f t = RV2 (f1 t) (f2 t). We need to show that f1 and f2 factor through f. To this end, take any t in T. We have

(p1 . f) t === p1 (f t) === p1 (RV2 (f1 t) (f2 t)) === f1 t

and

(p2 . f) t === p2 (f t) === p2 (RV2 (f1 t) (f2 t)) === f2 t

so f1 === p1 . f and f2 === p2 . f, as required.

Next, we need to show that f is the only function through which f1 and f2 factor in this way. Suppose that g :: T -> RealVec2 is any function satisfying f1 === p1 . g and f2 === p2 . g. Take any t in T, and let g t === RV2 u v for some u and v in RealNumber. We have

f1 t === (p1 . g) t === p1 (g t) === p1 (RV2 u v) === u

and

f2 t === (p2 . g) t === p2 (g t) === p2 (RV2 u v) === v

so f1 t === u and f2 t === v. Then

f t === RV2 (f1 t) (f2 t) === RV2 u v === g t

so g === f, which completes the proof.

So, Haskell admits pairwise products of types. In fact, Haskell can express the product of any number of types (up to machine limitations). If A1, A2, …, AN are types, we define their product as so.

data P = PCons A1 A2 ... AN

p1 :: P -> A1
p1 (PCons x _ ... _) = x

p2 :: P -> A1
p2 (PCons _ x ... _) = x

...

pN :: P -> AN
pN (PCons _ _ ... x) = x

The names P, PCons, p1, p2, …, pN are arbitrary. The ... is pseudo-code for however many types you have/underscores you need.

Defining the functions p1, p2, …, pN can be rather tedious. Haskell gives us a shorthand way to define both the product type and the projection maps at once, using named fields.

data P = PCons {p1 :: A1, p2 :: A2, ..., pN :: AN}

This has all the effects of the first code block. It creates the function p1, p2, …, pN and the function PCons :: A1 -> A2 -> ... -> AN -> P. It also has the additional effect that we may choose to supply arguments to PCons either positionally or by name, to our preference. Here’s what I mean.

x :: P
x = PCons x1 x2 ... xN -- `PCons` with arguments supplied positionally.

y :: P
y = PCons {p1 = y1, p2 = y2, ..., pN = yN} -- `PCons` with arguments supplied by name.

When supplying arguments by name, we don’t need to remember the order in which the various fields were originally declared. Any permutation of the fields is valid. The below example is perfectly valid.

data Person = Person {name :: String, age :: Integer}

daniel :: Person
daniel = Person {age = 21, name = "Daniel"}

I slipped in a new concept there that Haskellers like to call “name punning.” I used “Person” as the name of the type and as the name of the data constructor. This is legal and valid haskell, and it’s pretty common, especially when a data type has only one data constructor.

Sum types and universal inclusion maps

Bool and Ordering are each defined by explicitly listing their members. Most languages support this, and it’s typically called enumerated types, or enums. In Haskell, a types that have more than one data constructor are called sum types.

One of Haskell’s innovative features is that it allows you to freely combine both product type and sum types.

data Payment
  = Cash {amount :: Dollars}
  | Card {amount :: Dollars, cardNo :: CardNumber, zip :: ZipCode}
  | Complimentary {amount :: Dollars} -- still need to keep track of how much we're giving away!

Here we have a data type with three data constructors. The novel feature of Haskell is that it allows a programmer to give different fields to each constructor in an enumeration. (I know, it’s crazy that this is a novel feature in programming languages!)

Notice that the constructors Cash and Complimentary have the same fields. Because of their tags, they will create different copies of Dollar in Payment. Thus, we have defined a tagged union.

\[\begin{align*} \mathrm{Payment} &= \{ (\mathrm{Cash}, x) | x \in \mathrm{Dollars} \} \\ &\cup \{ (\mathrm{Card}, x, y, z) | x \in \mathrm{Dollars}, y \in \mathrm{CardNumber}, z \in \mathrm{ZipCode} \} \\ &\cup \{ (\mathrm{Complimentary}, x) | x \in \mathrm{Dollars} \} \end{align*}\]

Tagged unions are coproducts of sets. Similarly, sum types in Haskell are coproducts of types.

Claim 2

Payment is the coproduct of Dollars, (Dollars, CardNumber, ZipCode) and Dollars, where the inclusion maps are defined as i1 x = Cash x, i2 (x, y, z) = Card x y z, and i3 x = Complimentary x.

Proof

Let T be a type, and suppose we have f1 :: Dollars -> T, f2 :: (Dollars, CardNumber, ZipCode) -> T, and f3 :: Dollars -> T. Define f :: Payment -> T as follows.

f (Cash x) = f1 x
f (Card x y z) = f2 (x, y, z)
f (Complimentary x) = f3 x

We need to show that f1, f2 and f3 factor through f. For f1, take any x in Dollars. Then

(f . i1) x === f (i1 x) === f (Cash x) === f1 x

so f1 === f . i1.

For f2, take any (x, y, z) in (Dollars, CardNumber, ZipCode).

(f . i2) (x, y, z) === f (i2 (x, y, z)) === f (Card x y z) === f2 (x, y, z)

so f2 === f . i2.

For f3, take any x in Dollars.

(f . i3) x === f (i3 x) === f (Complimentary x) === f3 x

so f3 === f . i3. Thus, f1, f2, and f3 factor through f, as required.

Next, we need to show that f is the only function through which f1, f2, and f3 factor in this way. Suppose that g :: Payment -> T is any function satisfying f1 === g . i1, f2 === g . i2, and f3 === g . i3. Take any p in Payment. We have three cases to consider.

If p === Cash x for some x in Dollars, then

f (Cash x) === f1 x === (g . i1) x === g (i1 x) === g (Cash x)

so g p === f p in case p === Cash x.

If p === Credit x y z for some x in Dollars, y in CardNumber, and z in ZipCode, then

f (Credit x y z) === f2 (x, y, z) === (g . i2) (x, y, z) === g (i2 (x, y, z)) === g (Credit x y z)

so g p === f p in case p === Credit x y z.

If p === Complimentary x for some x in Dollars, then

f (Complimentary x) === f3 x === (g . i3) x === g (i3 x) === g (Complimentary x)

so g p === f p in case p === Complimentary x.

In all cases, g p === f p, so g === f, completing the proof.

A programming language supports definition of product type and coproduct types, it is said to support algebraic data types.

Closing

In this post, we say how arbitrary products and coproducts of Haskell types can be expressed as Haskell types. Roughly, we have a category similar to $\mathrm{SET}$, where Haskell types are the objects and Haskell functions are the maps. That’s more-or-less the case, although there are a few caveats that we might have time for at the end. In the mean time, it’s useful to limit our discussion to the fragment of Haskell were it is true. This observation, though, is far from the total extant to which Category theory bares its teeth in Haskell.

As a parting note, notice that in the proofs of claims 1 and 2, we defined the induced map in a very ad hoc, manual way. In the next post, we’ll see how to write a Haskell program that generates these induced maps, naturally. Moreover, slipping the word naturally in there was intentional. We will rely on a concept called type variables. With type variables, we will be able to turn the machinery of Haskell in on itself. Type variables will allow us to embed—and leverage, to great utility—functors and natural transformation within Haskell programs.

Understanding Partial Application of Function Composition

Fri, 13 Sep 2024 00:00:00 +0000

A recent post on r/haskell solicited help understanding the expression

isAllergicTo :: Allergen -> Int -> Bool
isAllergicTo = (. allergies) . elem

where Allergen is a type and allergies :: Int -> [Allergen].

It’s straightforward to rewrite this function pointfully (ie. with formal parameters in place), but doing so doesn’t help one develope an intuition for thinking about function composition, partially applied, on a higher semantic level. This post is my attempt at helping people develop that high-level intuition.

The Problem

The original problem had allergies :: Int -> [Allergen], but I am going to change Int there to PatientId. I find that this helps to ground the problem.

data PatientId
data Allergen
instance Eq Allergen

allergies :: PatientId -> [Allergen]
allergies = undefined "elided"

isAllergicTo :: Allergen -> PatientId -> Bool
isAllergicTo = (. allergies) . elem

Our job is to “git gud” at Haskell to the point that we can glance at the definition of isAllergicTo and intuitively grasp how it works. I’m not quite there yet, either, so don’t feel bad.

Another thing that I find helps me understand the problem more deeply is to eliminate incidental details. We can reformulate the essential concepts of the problem, without distracting details, as follows. We postulate that some types A and B exist along with a function f :: A -> B between them. We’ll also need B to have a notion of equality between members. To avoid dragging lists into the problem unnecessarily, I’ll replace elem with (==). I’ll also define and alias eq = (==) for code readability.

data A
data B
instance Eq B

eq :: B -> B -> Bool
eq = (==)

f :: A -> B
f = undefined "elided"

isEq :: B -> A -> Bool
isEq = (. f) . eq

Our goal is to be able to read and understand things like isEq = (. f) . eq when we come across them in code. Notice that isEq in our stripped-down version of the problem is directly analogous to isAllergicTo from the original version of the problem.

In order to appeal to the broadest number of different areas of your brain, I’ll carry on referring to both the isAllergicTo form of the problem and the isEq form of the problem, in parallel. Focus on the one that resonates with you the most, or compare and contrast them as we go.

Prologue: Rewriting in Pointful Style

Let’s first make sure that we have a very good understanding of isAllergicTo before we move on. To do so, we’ll rewrite it with explicit parameters.

\[\begin{align*} \mathtt{isAllergicTo} &= \left( \mathop\circ \mathtt{allergies} \right) \mathop\circ \mathtt{elem} & \\ \mathtt{isAllergicTo} \, a &= \left( \left( \mathop\circ \mathtt{allergies} \right) \mathop\circ \mathtt{elem} \right) \, a &\text{(Plug in $a$ on both sides)} \\ &= \left( \mathop\circ \mathtt{allergies} \right) \left( \mathtt{elem} \, a \right) &\text{(Evaluate $\mathop\circ$)} \\ &= \mathtt{elem} \, a \mathop\circ \mathtt{allergies} &\text{(Operator section syntax)} \\ \mathtt{isAllergicTo} \, a \, p &= \left( \mathtt{elem} \, a \mathop\circ \mathtt{allergies} \right) \, p &\text{(Plug in $p$ on both sides)} \\ &= \left( \mathtt{elem} \, a \right) \, \left( \mathtt{allergies} \, p \right) &\text{(Evaluate $\mathop\circ$)} \\ \end{align*}\]

We end up with isAllergicTo a p = elem a (allergies p). A moment’s intuition tells us that this function answers the question, is a one of p’s allergies. Seeing isAllergicTo in pointful form, its inner workings are immediately clear to me. Its inner workings were not at all clear to me having seen isAllergicTo in its pointfree form, but we can fix that.

Before we go on, I want to draw your attention back to a step that I find particularly interesting.

\[\begin{align*} &= \left( \mathop\circ \mathtt{allergies} \right) \left( \mathtt{elem} \, a \right) \\ &= \mathtt{elem} \, a \mathop\circ \mathtt{allergies} \\ \end{align*}\]

Two things pique my curiosity here.

First, notice how $ \mathtt{elem} \, a $ sits on the right-hand side of $ \left( \mathop\circ \mathtt{allergies} \right) $, so it’s earlier in the computational pipeline. But this step flips it, so that $ \mathtt{elem} \, a $ ends up on the left-hand side of $ \left( \mathop\circ \mathtt{allergies} \right) $, so later in the computational pipeline. It’s as though something is swapped around.

Second, I like the interplay between function composition and function application in this step. Before the step, the function $ \left( \mathop\circ \mathtt{allergies} \right) $ is applied to its argument $ \mathtt{elem} \, a $. After the step, we’re left with the composition of the functions $ \mathtt{allergies} $ and $ \mathtt{elem} \, a$. It’s as though function application on one level becomes function composition at another level.

Put a couple pins in those observations.

For completeness, here’s isEq’s rewrite. It’s a little bit easier on the eyes, thus a little bit easier to follow, in my opinion.

\[\begin{align*} \mathtt{isEq} &= \left( \mathop\circ \mathtt{f} \right) \mathop\circ \mathtt{eq} & \\ \mathtt{isEq} \, b &= \left( \left( \mathop\circ \mathtt{f} \right) \mathop\circ \mathtt{eq} \right) \, b &\text{(Plug in $b$ on both sides)} \\ &= \left( \mathop\circ \mathtt{f} \right) \left( \mathtt{eq} \, b \right) &\text{(Evaluate $\mathop\circ$)} \\ &= \mathtt{eq} \, b \mathop\circ \mathtt{f} &\text{(Operator section syntax)} \\ \mathtt{isEq} \, b \, a &= \left( \mathtt{eq} \, b \mathop\circ \mathtt{f} \right) \, a &\text{(Plug in $a$ on both sides)} \\ &= \left( \mathtt{eq} \, b \right) \, \left( \mathtt{f} \, a \right) &\text{(Evaluate $\mathop\circ$)} \\ \end{align*}\]

We’d like to train ourselves to be able to read things like (. f) . eq or (. allergies) . elem naturally. The process of such training is a bit involved, but we only have to go through the process once. By the time we’re through, we will have a clear, concise, high-level intuition for what things like (. f) and (. allergies) mean.

Building a Higher Vantage Point

I like thinking in pictures. We presuppose a function allergies :: PatientId -> [Allergen] (or f :: A -> B, respectively). That looks like this.

Figure 1: the function allergies depicted as an arrow from PatientId to [Allergen]

In Figure 1 and the subsequent figures, we depict types as nodes in a directed graph. Then a function can be visualized as an arrow from its source type to its target type.

Now, consider some arbitrary function $ g $ with source type [Allergen] (respectively, B) and target type $ w $. Adding a node for $ w $ and an arrow for $ g $ gives us a graph with a path in it. This graph will also have an implied arrow from PatientId (res. A) to $ w $: specifically, I’m talking about the composition of $ g $ and allergies (res. f). This composed function can be thought of as the result of following the path from PatientId, along allergies through [Allergen], along $ g $ to $ w $ (res. from A, along f through B, along $ g $ to $ w $).

Figure 2: a two-arrow path representing the composition of allergies with a function g from [Allergen] to w

Figure 2 has a bonus feature that I haven’t brought up yet. You may have noticed it. It’s $ \mathop{\widehat{\mathtt{allergies}}} g $ (res. $ \mathop{\widehat{\mathtt{f}}} g $). This is a new operation on functions that I’d like to now define. For any function $ x $, define $ \widehat{x} $ to be the higher-order function that takes a function $ y $ to the composition $ y \mathop\circ x $ (if the composition is defined). That’s quite a mouthful. Let’s try to see it, in symbols; seeing the same thing two ways always helps me. Compare the symbolic description below to the verbal description above and practice translating between the two.

\[\begin{align*} \mathop{\widehat{x}} y &= y \mathop\circ x &\text{if $\mathrm{source}(y) = \mathrm{target}(x)$} \end{align*}\]

I’ll call this operation “hat,” and I’ll refer to things like $ \widehat{x} $ as “ex-hat.” As a third way to think of the hat operation, this is what it looks like in code.

hat :: (a -> b) -> ((b -> w) -> (a -> w))
hat f = (. f)

Hopefully it’s becoming clear why I would go to the trouble of defining the hat operation. The hat operation exists in order to give a name to the code pattern we’re trying to understand: function composition partially applied.

Look again at the signature of hat. I could have written (a -> b) -> (b -> w) -> a -> w, since that’s equivalent and omits implied parentheses. I chose to include all of the implied parentheses when I wrote hat’s signature. We aim to focus on what we get when we partially-apply function composition. Equivalently, we now focus on the function returned by hat.

        f ::  a       ->  b
    hat f :: (b -> w) -> (a -> w)

Figure 3 represents that code visually. It depicts the function $ \mathtt{f} $ going from its source type $ \mathtt{A} $ to its target type $ \mathtt{B} $ and the function $ \widehat{\mathtt{f}} $ going from its source type $ \mathtt{B} \to w $ to its source type $ \mathtt{A} \to w $. In the concrete setting, that’s the function $ \mathtt{allergies} $ going from its source type $ \mathtt{PatientId} $ to its target type $ [\mathtt{Allergen}] $ and the function $ \widehat{\mathtt{allergies}} $ going from its source type $ [\mathtt{Allergen}] \to w $ to its target type $ \mathtt{PatientId} \to w $.

Figure 3: a series of three arrows representing the action of the hat operation on allergies

Here is some intuition we can gain from Figure 3. First, in addition to transforming functions, $ \widehat{(\;)} $ appears to have some kind of action on types, sending a type $ x $ to $ x \to w $. Second, $ \widehat{(\;)} $ seems to reverses the direction of arrows it transforms. We depict this in Figure 4.

Figure 4: the hat operation transforming functions with source type PatientId and target type [Allergen] into higher-order functions with source type [Allergen] -> w and target type PatientId -> w

So $ \widehat{(\;)} $ takes a function and returns a higher-order function with reversed arguments, in some sense. That’s a start, but it’s still very vague. We can temper this understanding by zooming in and considering what $ \widehat{\mathtt{allergies}} $ specifically does (res. what $ \widehat{\mathtt{f}} $ does).

As a function, $ \widehat{\mathtt{allergies}} $ has source type $ [\mathtt{Allergen}] \to w $ and target type $ \mathtt{PatientId} \to w $. (Respectively, $ \widehat{\mathtt{f}} $ has source type $ \mathtt{B} \to w $ and target type $ \mathtt{A} \to w $.)

\[\begin{align*} \widehat{\mathtt{f}} &: \left( \mathtt{B} \to w \right) \to \left( \mathtt{A} \to w \right) \\ \widehat{\mathtt{allergies}} &: \left( [\mathtt{Allergen}] \to w \right) \to \left( \mathtt{PatientId} \to w \right) \end{align*}\]

Figure 5 depicts the action of $ \widehat{\mathtt{allergies}} $ (res. $ \widehat{\mathtt{f}} $) on an arbitrary function $ g $. You can think of this as “stretching out” the source and target sets in the signature of $ \widehat{\mathtt{allergies}} $ (res. $ \widehat{\mathtt{f}} $) given above.

Figure 5: the action of the hat-transformed allergies function on an arbitrary function g.

And this is the lesson. $ \widehat{\mathtt{f}} $ is a function that you use in order to modify the source type of other functions. Specifically, hat f changes a function’s source type from B to A. How does it do that? If you think about it for a moment, there’s really only one way it can do that.

\[\begin{align*} g &: \mathtt{B} \to w \\ \mathop{\widehat{\mathtt{f}}} g &: \mathtt{A} \to w \end{align*}\]

The only possible way hat f can work is if there is some pre-processing on the input A value. In fact, it must necessarily feed that A value to f in order to get a B value that it can then feed to $ g $. We remind ourselves, what was hat f? It was (. f) all along. But now we know that it has to be function composition, specifically preprocessing. The signatures themselves lock this requirement into place. There’s no other way to satisfy the signatures.

In summary, here’s how to think about (. f). (. f) is a higher-order function that you use to convert a B-eating functions into an A-eating functions. (. allergies) is a higher-order function that you use to convert a [Allergen]-eating function into a PatientId-eating function.

Epilogue: Case in Point

We’d like to have a function that tells us whether or not a patient is allergic to a particular allergen. Specifically, we’d like a path from (Allergen, PatientId) to Bool. Alternatively (and equivalently), we need a path from Allergen to PatientId -> Bool.

To each patient is associated the list of allergens harmful to them. Our task presupposes that this association is given.

allergies :: PatientId -> [Allergen]

For any particular allergen, it’s a straightforward task to write a function that checks for the presence of that allergen in any given list of allergens. In fact, this is such a common, formulaic task that the standard library provides a function for generating such allergen-list-checking functions.

Figure 6: elem is a function that generates functions that eat lists of allergens

elem takes us from Allergen to [Allergen] -> Bool. The last thing we need is a way to get from [Allergen] -> Bool to PatientId -> Bool. In other words, we need a way of converting [Allergen]-eating functions into PatientId-eating functions.

\[\underline{\phantom{\widehat{\mathtt{allergies}}}} : ([\mathtt{Allergen}] \to \mathtt{Bool}) \to (\mathtt{PatientId} \to \mathtt{Bool})\]

Since allergen goes from PatientId to [Allergen], (. allergen) converts [Allergen]-eating functions into PatientId-eating functions. Thus, (. allergen) takes us from [Allergen] -> Bool to PatientId -> Bool.

The function we’d like to have is the result of following the path from Allergen, along elem through [Allergen] -> Bool, then along (. allergies) to PatientId -> Bool. Following a path of arrows represents the composition of the functions represented by those arrows.

Figure 7: following the path along elem and the hat-transformed allergies function gives us the function we needed

Hence

isAllergicTo = (. allergies) . elem

Looking for a Needle that Might Not Be in an Infinite Haystack

Wed, 26 Oct 2022 00:00:00 +0000

Hand-wavy explanations of my lightning talk of the same title at Mercury’s October 2022 PDX Haskell meetup. Based on “Infinite sets that admit fast exhaustive search” by Martín Escardó.

{-# OPTIONS_GHC -Wall -fno-warn-orphans -fno-warn-name-shadowing #-}
{-# LANGUAGE DerivingVia, FlexibleInstances #-}

module Escardo where

import Prelude hiding (Real)

import Data.List (find)
import Data.Maybe (fromMaybe)
import Numeric.Natural (Natural)

Searchable types

We start by defining a class, $ \mathtt{Searchable} \, a $ for types $ a $ that admit exhaustive search. Given $ \phi : a \to \mathtt{Bool} $ and $ a_0 : a $, we say $ a_0 $ satisfies $ \phi $ if $ \phi \, a_0 = \mathtt{True} $. We say $ \phi $ is satisfiable if at least one non-bottom member of $ a $ satisfies $ \phi $.

type Search a = (a -> Bool) -> a

class Searchable a where
  -- Law: if `phi` is satisfiable then `search phi` satisfies `phi`.
  search :: Search a

A type $ a $ is searchable if there is a total function $ \mathtt{search} : (a \to \mathtt{Bool}) \to a $ where $ \mathtt{search} \, \phi $ satisfies $ \phi $ for every satisfiable total predicate $ \phi : a \to \mathtt{Bool} $.

\[\mathtt{Searchable} \, a \iff \left( \forall_{\phi : a \to \mathtt{Bool}} \, \left( \exists_{x : a} \, \phi \, x = \mathtt{True} \right) \Rightarrow \phi \left( \mathtt{search} \, \phi \right) = \mathtt{True} \right)\]

In other words, $ \mathtt{search} \, \phi $ will find a member of $ a $ that satisfies $ \phi $, if such a member exists. $ \mathtt{search} \, \phi $ still produces a member of $ a $ even if $ \phi $ is not satisfiable, though, so it’s the caller’s responsibility to check and see whether or not the search result satisfies $ \phi $. Let’s automate that check!

query :: Searchable a => (a -> Bool) -> Maybe a
query phi =
  case search phi of
    a0 | phi a0    -> Just a0
       | otherwise -> Nothing

In typical Haskell fashion, we use our API to eliminate ambiguity. If $ \mathtt{query} \, \phi $ is $ \mathtt{Just} \, a_0 $, then $ \phi $ is satisfiable and $ a_0 $ satisfies $ \phi $. If $ \mathtt{query} \, \phi $ is $ \mathtt{Nothing} $, then no member of $ a $ will satisfy $ \phi $, so $ \phi $ is not satisfiable. Let’s automate that reasoning, too!

exists :: Searchable a => (a -> Bool) -> Bool
exists = not . null . query

forAll :: Searchable a => (a -> Bool) -> Bool
forAll phi = (not . exists) (not . phi)

Given this framework, Our basic problem is to discover which types are searchable and to implement $ \mathtt{search} $ for such types.

Finite types are searchable

Duh…

A finite type $ a $ is searchable, because we can implement $ \mathtt{search} $ as follows. When given a predicate $ \phi : a \to \mathtt{Bool} $ we can iterating over the members of $ a $, applying $ \phi $ to each. If we find a member of $ a $ that satisfies $ \phi $, we stop and return it. If we exhaust $ a $, we return the last member we checked.

newtype Finite a = Finite a
  deriving (Bounded, Enum) via a

instance (Bounded a, Enum a) => Searchable (Finite a) where
  search phi = fromMaybe maxBound (find phi [minBound .. maxBound])

This is a brute-force approach, and many searchable types will have a more-elegant, more-efficient implementation of $ \mathtt{search} $. Here’s an implementation for $ \mathtt{Bool} $

instance Searchable Bool where
  search phi = phi True

To verify that our instance is lawful, we consider three cases.

Suppose $ \phi \, \mathtt{True} $ is $ \mathtt{True} $. Then $ \phi \left( \mathtt{search} \, \phi \right) = \phi \left( \phi \, \mathtt{True} \right) = \phi \, \mathtt{True} = \mathtt{True} $, as required.
Suppose $ \phi \, \mathtt{True} = \mathtt{False} $ and $ \phi \, \mathtt{False} = \mathrm{True} $. Then $ \phi \left( \mathtt{search} \, \phi \right) = \phi \left( \phi \, \mathtt{True} \right) = \phi \, \mathtt{False} = \mathtt{True} $, as required.
Suppose $ \phi \, \mathtt{True} = \phi \, \mathtt{False} = \mathrm{False} $. Then $ \mathtt{search} \, \phi $ is not required to satisfy $ \phi $, so there’s nothing to check: we’re done.

These cases exhaust all possibilities, thus the claim is verified.

This is effectively the solution to the Knight/Knave Riddle. Two guards stand in front of two doors; one door leads to certain death. One guard, the knave, always lies, and the other, the knight, always tells the truth. You have no way of telling which is which. You may ask a single question to one of the guards. You need to craft your question so that it’s guaranteed to reveal which door is ~~safe~~ less lethal, no matter which guard you ask.

Building searchable types inductively

If each of $ a $ and $ b $ is searchable, then their product is searchable.

instance (Searchable a, Searchable b) => Searchable (a, b) where
    search phi = (a0, b0)
      where
        a0 = search phi1
        phi1 a = exists (\b -> phi (a, b))

        b0 = search phi2
        phi2 b = phi (a0, b)

We claim that the above instance is lawful. To that end, suppose $ \phi $ is satisfiable. We need to show that $ (a_0, b_0) $ satisfies $ \phi $. Since $ \phi $ is satisfiable, $ \phi_1 $ is satisfiable, because $ \mathtt{fst} : a \times b \to a $ maps solutions of $ \phi $ to solutions of $ \phi_1 $. Since $ \phi_1 $ is satisfiable, $ a_0 = \mathtt{search} \, \phi_1 $ satisfies $ \phi_1 $. Since $ a_0 $ satisfies $ \phi_1 $, $ \mathtt{exists} \left( b \mapsto \phi \, (a_0,b) \right) $ is $ \mathtt{True} $. But notice $ \mathtt{exists} \left( b \mapsto \phi \, (a0,b) \right) $ is identically $ \mathtt{exists} \, \phi_2 $, thus $ \mathtt{exists} \, \phi_2 $ is $ \mathtt{True} $. Since $ \mathtt{exists} \, \phi_2 $ is $ \mathtt{True} $, $ \phi_2 $ is satisfiable. Since $ \phi_2 $ is satisfiable, $ b_0 = \mathtt{search} \, \phi_2 $ satisfies $ \phi_2 $. That is to say $ \phi \, (a_0, b_0) $ is $ \mathtt{True} $, completing the proof.

So we now know that the product of any two searchable types is searchable. This generalizes in a straight-forward manner, allowing us to conclude that the product of finitely-many searchable types is searchable. What about the product of infinitely-many searchable types? In particular, let’s consider infinite-length tuples of a searchable type $ a $. Such infinite tuples are, of course, more appropriately thought of as infinite sequences of members of $ a $. If $ a $ is searchable, is the type comprised of infinite sequences of members of $ a $ searchable? Surprisingly, it is!

type Sequence a = Natural -> a

instance Searchable a => Searchable (Sequence a) where
  search = tychonoff (const search)

$ \mathtt{Sequence} \, a $ has infinitely-many non-bottom members as long as $ a $ has at least two non-bottom members, so we now have examples of infinite types that are still searchable. $ \mathtt{tychonoff} $ is doing all the heavy lifting here. $ \mathtt{tychonoff} $ is a function that takes a sequence of searchers and uses that to build a searcher of sequences. We thus define $ \mathtt{search} $ on $ \mathtt{Sequence} \, a $ by applying $ \mathtt{tychonoff} $ to the constant sequence that identically returns $ \mathtt{search} $ on $ a $.

tychonoff :: Sequence (Search a) -> Search (Sequence a)
tychonoff searchers phi = result
  where
    result = decode . encode $ \i ->
      searchers i $ \a ->
        psi i a $
          tychonoff
            (\i' -> searchers (i' + i + 1))
            (psi i a)

    psi i a as =
      phi $ \i' -> case compare i' i of
        LT -> result i'
        EQ -> a
        GT -> as (i' - i - 1)

I’ve read Escardó’s paper four times now, and I still have no idea how $ \mathtt{tychonoff} $ works. Maybe on my fifth reading I’ll see the light. I will attempt to convey the gist of why this is even plausible in the first place, though. I defined searchable using the word total to describe predicates. The crucial fact that this gives us is that a total predicate $ \phi : a \to \mathtt{Bool} $ must be non-bottom for all non-bottom members of $ a $. Now, consider the case of a predicate defined on infinite sequences. In order for a predicate $ \phi : \mathtt{Sequence} \, a \to \mathtt{Bool} $ to be non-bottom on all non-bottom members of $ \mathtt{Sequence} \, a $, there must necessarily be a largest natural number $ n $ beyond which $ \phi \, x $ does not use any terms of $ x $. This is because the expression $ \langle\phi \, x\rangle $ reduces to an expression $ \langle\phi_1 \, [x \, 1] \, x\rangle $ which reduces to an expression $ \langle\phi_2 \, [x \, 1] \, [x \, 2] \, x\rangle $ and so on.

\[\langle\phi \, x\rangle \Rightarrow \langle\phi_1 \, [x \, 1] \, x\rangle \Rightarrow \langle\phi_2 \, [x \, 1] \, [x \, 2] \, x\rangle \Rightarrow \dots \Rightarrow \langle\phi_i \, [x \, 1] \, ... \, [x \, i] \, x\rangle \Rightarrow \dots\]

(Where $ \langle\cdot\rangle $ is $ \mathtt{apply} $ and $ [\cdot] $ is $ \mathtt{eval} $.)

This reduction process generates an infinite sequence of expressions. Because we require $ \phi \, x $ be non-bottom, the tail of the sequence eventually stabilizes at either the expression $ \langle\mathtt{True}\rangle $ or the expression $ \langle\mathtt{False}\rangle $. So given any non-bottom sequence $ x $, we know that $ \phi \, x $ will terminate.

This explains how $ \mathtt{tychonoff} \, (\mathtt{const} \, \mathtt{search}) \, \phi $ terminates in the case where $ \phi $ is satisfiable. If we assume for the moment that we are clairvoyant and we know that $ \phi $ is satisfiable before the fact, then we further know that iterating over all non-bottom sequences will eventually yield a sequence that satisfies $ \phi $.

What’s still not clear to me is how $ \mathtt{tychonoff} \, (\mathtt{const} \, \mathtt{search}) \, \phi $ terminates in the case where $ \phi $ is not satisfiable. Here’s my best guess, though. Suppose a predicate $ \phi $ is not satisfiable. For each non-bottom sequence $ x $, let $ N_\phi(x) $ be the smallest natural number $ n $ where

\[\langle\phi_n \, [x \, 1] \, ... \, [x \, n] \, x\rangle \in \{\langle\mathtt{True}\rangle, \langle\mathtt{False}\rangle\} \text{.}\]

(I.e., $ N_\phi(x) $ is the point at which the sequence of reductions of the expression $ \langle \phi \, x \rangle $ stabilizes.) We know that $ N_\phi(x) < \infty $ for each non-bottom $ x $. What we’d like is for $ N_\phi $ to have a maximum, and we’d like to be able to compute that maximum. If $ N_\phi $ had a computable maximum, we’d then know when to make $ \mathtt{tychonoff} \, (\mathtt{const} \, \mathtt{search}) \, \phi $ stop looking. Now, this is just one approach, one very hopeful approach. It’s totally conceivable that there’s some predicate $ \phi $ out there where $ N_\phi $ doesn’t have a maximum. So maybe the stopping condition for $ \mathtt{tychonoff} \, (\mathtt{const} \, \mathtt{search}) \, \phi $ is found some other way.

Tangentially, we want our search to be fast, so we memoize our sequence. A common memoization strategy in Haskell is passing a function through a concrete data structure; here we use a binary tree. This step is optional: the search algorithm takes longer but still works without it.

data Tree a = Branch a (Tree a) (Tree a)

encode :: Sequence a -> Tree a
encode f =
  Branch (f 0) (encode (\n -> f (2 * n + 1)))
               (encode (\n -> f (2 * n + 2)))

decode :: Tree a -> Sequence a
decode (Branch x l r) n =
  case n of
    0             -> x
    n | odd n     -> decode l ((n - 1) `div` 2)
      | otherwise -> decode r ((n - 2) `div` 2)

Here’s where we stand. Pick your favorite searchable type $ a $. Thanks to $ \mathtt{tychonoff} $, we know that the type comprised of infinite sequences of members of $ a $—clearly an infinite type if $ a $ has at least two non-bottom members—is also searchable. Exhaustively. In finite time. Mind blowing.

Equality of functions

Functions from $ a $ to $ b $ may be tested for equality—extensional equality, not reference equality—deterministically and in finite time if a is searchable and b instances Eq.

instance (Searchable a, Eq b) => Eq (a -> b) where
  f == g = forAll (\a -> f a == g a)

Real numbers between zero and one (inclusive) may be represented by infinite sequences of bits. This is the binary encoding of the number’s decimal expansion, as decoded by approx. The Show instance for Real makes use of only eight bits of information, but this will prove to be fairly expensive. In our demo, showing Real numbers will end up taking longer than testing Real-variable functions for equality.

type Real = Sequence Bool

approx :: Natural -> Real -> Double
approx bits x =
  sum [recip $ 2 ^ (i + 1) | i <- take (fromIntegral bits) [0 ..], x i]

instance Show Real where
  show x = show (approx 8 x) <> "..."

ints :: Enum a => Sequence a -> Integer -> Integer
ints x = toInteger . fromEnum . x . fromIntegral

Demonstration

Here’s the code I used to verify the searchability of Real, a file called demo.hs.

import Prelude hiding (Real, (^))
import qualified Prelude

import Escardo
import Numeric.Natural (Natural)
import System.CPUTime (getCPUTime)

(^) :: Num a => a -> Natural -> a
(^) = (Prelude.^)

We define several Integer-valued functions of a Real variable to test on. We avoid returning a Real or a Double because Eq (a -> b) requires Eq b, and while Double has an Eq instance, equality for Double values is somewhat flaky. I didn’t want risk having that flakiness change the outcome of any of the tests.

f :: Real -> Integer
f x =
  x'
    $   10 * x' (3 ^ 80)
    +  100 * x' (4 ^ 80)
    + 1000 * x' (5 ^ 80)
  where
    x' = ints x

g :: Real -> Integer
g x =
  x'
    $   10 * x' (3 ^ 80)
    +  100 * x' (4 ^ 80)
    + 1000 * x' (6 ^ 80)
  where
    x' = ints x

h :: Real -> Integer
h x =
  if x' (4 ^ 80) == 0
    then x'        j
    else x' (100 + j)
  where
    i = if x' (5 ^ 80) == 0 then 0 else 1000
    j = if x' (3 ^ 80) == 1 then 10 + i else i
    x' = ints x

Our demo will test these functions for equality, finding inputs on which they disagree if applicable. We’ll annotate our test cases with various stringy labels and also the time they take to run. In order to accurately measure this time, we force evaluation by branching on length . show of the output before grabbing the end time. Otherwise, laziness could cause misleading timings.

type Lbl = (,) String
type Mil = (,) Integer

diff :: (Searchable a, Eq b) => (a -> b) -> (a -> b) -> Maybe (Lbl b, Lbl b)
diff f g =
  fmap
    (\x -> (("f x", f x), ("g x", g x)))
    (query (\x -> f x /= g x))

diffShow :: (Searchable a, Eq b) => (a -> b) -> (a -> b) -> Maybe (Lbl b, Lbl b, Lbl a)
diffShow f g =
  fmap
    (\x -> (("f x", f x), ("g x", g x), ("x", x)))
    (query (\x -> f x /= g x))

timed :: IO a -> IO (Mil a)
timed x = do
  t0 <- getCPUTime
  y <- x
  t1 <- getCPUTime
  pure (div (t1 - t0) (1000 * 1000 * 1000), y)

told :: Show a => String -> a -> IO ()
told ann x = do
  print (ann, x)

did :: Show a => a -> IO a
did x = do
  case length (show x) of
    0 -> pure x
    _ -> pure x

main :: IO ()
main = do
  told "f == g" =<< timed (did $ f == g)
  told "g == h" =<< timed (did $ g == h)
  told "h == f" =<< timed (did $ h == f)

  told "f x /= g x" =<< timed (did $ diff f g)
  told "g x /= h x" =<< timed (did $ diff g h)

  told "f x /= g x" =<< timed (did $ diffShow f g)
  told "g x /= h x" =<< timed (did $ diffShow g h)

Interpreted

Ran the main action of demo.hs in GHCi, which precludes any kind of optimizations and is also slower than native binaries generally. It took about 2.5 seconds and 2.6 seconds to determine that f and g disagree and that g and h disagree, respectively. Interestingly, it only took about 1.1 seconds to determine that f and h agree at every input. It took about 2.6 seconds each to find inputs where f and g disagree and where g and h disagree, unsurprisingly. To find and also print the input on which they disagree took a whopping 10.4 and 10.7 seconds, respectively. Printing the input takes longer than finding it.

Times in the output are in milliseconds.

daniel@newport:~/Desktop
[11:32:12] $ ghci demo.hs
GHCi, version 8.10.7: https://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /Users/daniel/.ghci
[1 of 2] Compiling Escardo          ( Escardo.hs, interpreted )
[2 of 2] Compiling Main             ( demo.hs, interpreted )
Ok, two modules loaded.
ghci> main
("f == g",(2534,False))
("g == h",(2684,False))
("h == f",(1135,True))
("f x /= g x",(2694,Just (("f x",0),("g x",1))))
("g x /= h x",(2667,Just (("f x",1),("g x",0))))
("f x /= g x",(10438,Just (("f x",0),("g x",1),("x",0.99609375...))))
("g x /= h x",(10769,Just (("f x",1),("g x",0),("x",0.99609375...))))

Compiled

Native code generated without optimizations fared much better. Finding inputs now takes less than half a second in all cases. Finding and printing takes nearly two seconds.

daniel@newport:~/Desktop
[11:33:21] $ ghc -O0 demo.hs
[1 of 2] Compiling Escardo          ( Escardo.hs, Escardo.o )
[2 of 2] Compiling Main             ( demo.hs, demo.o )
Linking demo ...
daniel@newport:~/Desktop
[11:34:56] $ ./demo
("f == g",(435,False))
("g == h",(403,False))
("h == f",(202,True))
("f x /= g x",(437,Just (("f x",0),("g x",1))))
("g x /= h x",(427,Just (("f x",1),("g x",0))))
("f x /= g x",(1909,Just (("f x",0),("g x",1),("x",0.99609375...))))
("g x /= h x",(1766,Just (("f x",1),("g x",0),("x",0.99609375...))))

Compiled with optimizations

Optimizations further bring these numbers down, to less than a quarter of a second to find inputs and less than a half of a second extra if you want to print them.

daniel@newport:~/Desktop
[11:35:52] $ ghc -O2 demo.hs
[1 of 2] Compiling Escardo          ( Escardo.hs, Escardo.o )
[2 of 2] Compiling Main             ( demo.hs, demo.o )
Linking demo ...
daniel@newport:~/Desktop
[11:35:59] $ ./demo
("f == g",(193,False))
("g == h",(216,False))
("h == f",(115,True))
("f x /= g x",(205,Just (("f x",0),("g x",1))))
("g x /= h x",(209,Just (("f x",1),("g x",0))))
("f x /= g x",(645,Just (("f x",0),("g x",1),("x",0.99609375...))))
("g x /= h x",(702,Just (("f x",1),("g x",0),("x",0.99609375...))))

A decent and reliable Haskell dev setup for VS Code in 2022

Mon, 18 Jul 2022 00:00:00 +0000

The sickest experience possible for Haskell development in July 2022 is undoubtedly VS Code with the Haskell plugin powered by Haskell Language Server (HLS). When HLS works for your project, that is. This guide will demonstrate a simpler, lower-featured alternative that hopefully is reliable enough to work with your project in those cases where HLS does not.

Changelog

2022-07-26

Add --warnings to our Ghcid invocations. This causes tests to be run (updating ctags and hoogle) even in the presense of warnings.
On M1, the Stack distributed from haskellstack.org fails on some tasks. Use only the Stack distributed with GHCUp.
On M1, Hasktags might fail if you have LLVM 13 or 14. To work around this, install brew install llvm@12 and put its bindir near the front of your path.
Restart Ghcid (via ./start-ide.sh) whenever you modify stack.yaml, package.yaml, or your .cabal file.

Synopsis

HLS is superb and provides a rich Haskell experience that genuinely boosts productivity. However, there are various reasons why HLS might not work with a particular Haskell project. The project might be built using a version of GHC that HLS doesn’t support. The project might be very TemplateHaskell-heavy or otherwise just too damn big for HLS to function with acceptable latency. That said, progress continues on HLS (and on GHC), making it faster and more stable, with the goal that we’ll be able to use it for all of our Haskell projects.

In the mean time, we still would like some of the luxuries our text editors are capable of providing when given a suitable source of information. Following this guide, you’ll get:

Basic code suggestions (i.e. autocomplete) based on textual analysis of your source file.
Goto definition for top-level symbols and modules within your project.
Project-specific Hoogle search and Haddock documentation for your project and its dependencies (and transitive dependencies).
Problems reporting.
- Compiler errors and warnings will appear both in VS Code’s Problems pane, allowing you to easily jump to the source locatoin, and
- Inline squiggly underlines in your source code.

Finally, we’ll discus tying these features together into a cohesive, convenient workflow.

Why VS Code?

This guide is for VS Code, because that’s the editor I know and use. I’m sure you can get similar features for Emacs and the various Vis using the approaches described here, though. If you do, drop a comment and link to your blog post :-D

Why Stack?

I rely on Stack to reduce the number of dev tools I need to manage manually. Alternatively, the same setup should work if you install the necessary tools yourself (at the correct versions for your project). Or if you know of Cabal commands that more-or-less correspond to the Stack commands I use, then these workflows should work with those commands just fine.

You can use Stack for these editor integrations even if you don’t use Stack to build your project. In your project root directory (rather, the directory that has the Cabal file defining your package), use stack init to create a stack.yaml file. Check stackage.org and find your version of GHC on the list Latest releases per GHC version; click into that. Somewhere on the page, you’ll see something that looks more or less like

resolver: lts-18.28

resolver: nightly-2022-07-18

Replace the resolver property in your stack.yaml file with the one you found on Stackage. You should be good to go, unless you’re on Apple Silicon. Stack will work on Apple Silicon, but it takes a bit of finagling (please read on).

Stack on Apple silicon

Because of incompatibilities between GHC versions, Stack treats GHC as a project dependency: Stack will download and use the appropriate version of GHC for your project. However, the GHC binaries that Stack currently distributes are incompatible with Apple M1 (and presumably M2) chips. Moreover, I’ve gotten reports that the Stack binaries distributed by Homebrew and haskellstack.org exhibit unexpected errors. These are known issues, and they will hopefully be fixed soon.

Meanwhile, you’ll need to install both Stack and GHC from GHCUp. You’ll also need to configure Stack to use the GHC installed by GHCUp.

Remove any extand Stack distribution that didn’t come from GHCUp.

For example, if you installed Stack from Homebrew or from haskellstack.org, remove it.
Install GHCUp, and use it to install Stack and Cabal (latest versions).
Make sure GHCUp’s bin directory is included in your shell’s PATH variable.

For example, on my Mac OS 12.4 system, GHCUp’s bin directory is /Users/daniel/.ghcup/bin, and this directory appears near the front of my PATH variable.
Use GHCUp to install and set the version of GHC you want for your project.
Add a stack.yaml file to your project root if there isn’t one already.
In stack.yaml, add the following top-level properties:

   system-ghc: true
   install-ghc: false
   skip-ghc-check: true

Stack will now use the GHC binary provided by GHCUp.

One more thing. I’ve heard reports of Haskell projects unexpectedly failing to build on M1 due to LLVM incompatibilities. We need to install LLVM version 12 and we need to put its bindir on our path so that it’s the first LLVM found.

brew install llvm@12
echo 'export PATH="/usr/local/opt/llvm/bin:$PATH"' >> ~/.profile && . ~/.profile

Basic code suggestions

VS Code ships with a rudimentary form of code suggestions (a.k.a. autocomplete) built-in. It’s not very smart, being driven by a simple textual analysis of your source file, but it’s better than nothing.

Here are VS Code settings that work for me. You might want to tweak some of them to fit your preferred workflow.

{
  "editor.acceptSuggestionOnCommitCharacter": false,
  "editor.acceptSuggestionOnEnter": "smart",
  "editor.quickSuggestions":
  {
    "comments": "inline",
    "other": "inline",
    "strings":  "inline"
  },
  "editor.quickSuggestionsDelay": 2000,
  "editor.snippetSuggestions": "none",
  "editor.suggest.filterGraceful": false,
  "editor.suggest.preview": true,
  "editor.suggest.showStatusBar": true,
  "editor.suggest.showWords": true,
  "editor.suggestOnTriggerCharacters": false,
  "editor.suggestSelection": "first",
  "editor.wordBasedSuggestionsMode": "matchingDocuments"
}

With these settings, you’ll see the top suggestion as ghost text. You can apply the suggestion by hitting the Tab key. You can see the full list of suggestions by hitting Control+Space.

I have these settings in my user-level VS Code settings because I want them to apply to every workspace. (Workspace is just the word VS Code uses for what I’ve been calling your project.) Alternatively, you might want these settings to apply only in specific workspaces. If so, create a .vscode/settings.json in your project and put these properties there.

Goto definition

This feature depends on the ctagsx VS Code plugin.

We will use a Haskell tool called Hasktags to generate a file named tags. The tags file contains symbol information from your project source code. ctagsx relies on this file to provide Goto definition functionality in VS Code.

It’s not as great as what you get with HLS. HLS is able to index all of the defined entities in your source code, including local variables. Hasktags is only capable of indexing module-level definitions, but that’s better than nothing. It indexes types, classes, and variables—whether or not they’re exported—and it indexes modules. Right click on the entity you want to chase, and pick Goto Definition.

Caveat: to jump to a module you’ll need to select the whole module name before you right click. For example, to be taken to the source code for a module named Foo.Bar you need to select that entire module name—from F to r—before right clicking.

All you need to do is install the ctagsx plugin and generate a tags file for your source code.

Generate tags file:

stack exec hasktags -- --ctags .

If you want an up-to-date index, you’ll need to regenerate your tags file every time you modify your code. Obviously, doing this manually would be very tedious. We’ll automate it later.

Hoogle and Haddock

This feature depends on the vscode-goto-documentation plugin.

By now we should have Goto Definition working for the types, functions, classes, and modules in our project. Unfortunately, Goto Definition won’t work for entities defined in our dependencies. The next best thing, though, is having Hoogle search and Haddock documentation for them (and Haddock includes source code, so we kinda do get Goto Definition in a roundabout way).

Build initial Hoogle database (also builds docs):

stack hoogle --rebuild

Start Hoogle server (server will listen at localhost:8080):

stack hoogle --server &

You’ll now have Hoogle and Haddock for your project and its dependencies when you open your browser to localhost:8080.

As with our tags file, we’ll have to update Hoogle’s database every time our source code changes. Fortunately, we can do this while the Hoogle server is running. This needs to be done every time we change a source file (so we’ll automate it later).

Hot-rebuild Hoogle database:

stack hoogle -- generate --local <package_name_of_your_project>

When we’re done working for the day, we’ll want to tear down our server to free up Port 8080.

Kill Hoogle server:

pkill hoogle

Now, opening our browser, navigating to localhost:8080, and keying in the name of a function or type is laborious. Fortunately, we can get all this with just a right click once we configure our plugin appropriately. Create a .vscode/settings.json in your project if there isn’t one already, and add the following property:

{
  "goto-documentation.customDocs": {
    "hs": "http://localhost:8080/?hoogle=${query}"
  }
}

You can now right-click on a function, class, module, or type in your source code and pick Goto Document to get taken to the Hoogle results. This works both for symbols we get from our dependencies and symbols we’ve defined ourselves. The best part is we don’t even need a live internet connection.

Problems reporting

This feature depends on the haskell-ghcid VS Code plugin.

Here’s the real meat of our faux-IDE setup.

Ghcid is a simple, reliable file watcher that spawns a Ghci session and presses :r for you (Ghci’s reload command) whenever a Haskell source file changes. It’s reliable precisely because it has no notion of projects, build tools, or text editors. It relies simply on Ghci. It’ll even let you choose how to start Ghci, so that all of your dependencies will be registered and all of your modules will be in scope.

(I don’t know why I felt the need to explain all that. It’s hard to imagine anyone is writing Haskell and doesn’t know about Ghcid.)

In the old days, we were forced to use Ghcid in a crude way. We’d have it running in a terminal, side-by-side with our editor. Today, we have editor plugins for Emacs, Neovim, and VS Code. Go install the plugin, you’ll be happy you did!

The plugin is very simple. It reads a file, named ghcid.txt. It assumes the file contains info on compiler errors and warnings, and it parses that info and feeds it into VS Code’s problems reporting API. Your errors and warnings show up in VS Codes Problems pane (which provides Jump-to-source-location functionality) and inlined into your source code as squiggly underlines.

Your responsibility here is to bootstrap this whole process by starting Ghcid in a terminal and then forgetting about it. You’ll need to use Ghcid’s --outputfile option to tell Ghcid that it should be writing info to ghcid.txt.

Start Ghcid with file output:

stack exec ghcid -- --command 'stack repl --ghc-options "-fno-code -ignore-dot-ghci -ferror-spans"' --outputfile ghcid.txt

Replace stack repl ... with whatever command you use to load your project into Ghci. For example, if you use Cabal to build your project, it might be something like this

stack exec ghcid -- --command 'cabal repl --ghc-options "-fno-code -ignore-dot-ghci -ferror-spans"' --outputfile ghcid.txt

The most important thing here is --outputfile ghcid.txt. This Ghcid option it to write errors and warnings to ghcid.txt, where our VS Code plugin is expecting to find them.

The second most important thing here is -fno-code -ignore-dot-ghci -ferror-spans. These options tell Ghci that it should only be type checking (for faster reloads) and that it should be reporting full source spans for errors (so your editor can make squiggles). -ignore-dot-ghci is more superstitious than anything, but it’s in there because sometimes your .ghci file can mess up your project REPL. (If you rely on a .ghci file to load your project REPL, then you probably already know enough about Ghci to know that you should overrule my suggestion and omit that option.)

Now, just minimize your terminal and pull your editor up. Chasing compiler errors has never been easier!

Putting it all together

Our problems reporting (and squiggles!) are powered by Ghcid running in a minimized terminal somewhere. Well, Ghcid has a little-known feature that allows you to run an arbitrary Ghci command every time it reloads (so, every time a source file in your project changes). We’ll use this feature to automate the re-indexing of our tags file and our Hoogle database. This Ghcid feature is controlled by the --test option.

But we don’t want to run a Ghci command, we want to run a couple shell commands. Well, Ghci has a little-known feature that allows you to run arbitrary shell commands. At the Ghci prompt, type :! and then a space and then any shell command. Try it with something innocuous, like ls -l

ghci> :! ls -l

That’s rather delightful!

So, when we invoke Ghcid, we’ll use the --test option to get Ghci to run the shell commands that re-index our tags file and Hoogle database.

Here’s everything conveniently in a shell script:

# start-ide.sh
stack hoogle --rebuild # build Hoogle database for our dependencies
stack hoogle --server 2> /dev/null 1> /dev/null & # Hoogle on localhost:8080
(stack exec ghcid -- \ # on file saves...
  --command 'stack repl --ghc-options "-fno-code -ignore-dot-ghci -ferror-spans"' \ # type-check
  --test ':! stack exec hasktags -- --ctags . && stack hoogle -- generate --local ${your_package_name}' \ # and re-index
  --warnings \ # re-index even if there were warnings, dammit!
  --outputfile ghcid.txt) || true # write errors and warnings to ghcid.txt
(pkill ghcid && echo "Killed Ghcid.") || echo "No Ghcid process to kill." # superstitiously cleanup ghcid process (ctrl+c should have done it, though)
(pkill hoogle && echo "Killed Hoogle.") || echo "No Hoogle process to kill." # really cleanup Hoogle process

(You can find the name of your package in your project’s Cabal file.)

So, when I go sit down to work, I open a terminal to my project directory and I run ./start-ide.sh. Then I minimize that terminal and forget about it. I open my project in VS Code, and I have

basic code suggestions;
goto definition for my types, classes, functions, and modules;
goto documentation for all types, classes, functions, and modules;
errors and warnings in VS Code’s Problems pane, with goto source location; and
inline squigglies in my source code for errors and warnings.

And I have it all reliably, whether or not HLS works for my project.

Summary

Here are the steps written as tersely as possible, free of commentary, while still being totally complete. Some users will be able to skip some of these steps.

Initial Setup:

If on an Apple M1 or M2 system (optional otherwise), install GHCUp and:
1. Use it to install a version of GHC appropriate for your project.
2. Configure your shell so that GHCUp’s bindir is on your PATH.
Install Stack.
Install ctagsx VS Code plugin.
Install vscode-goto-documentation plugin.
Install haskell-ghcid VS Code plugin.
Create .vscode/settings.json in your project directory (contents shown below):
If your project doesn’t have a stack.yaml file, create one with stack init, and
1. Check stackage.org to find the latest resolver that matches your version of GHC.
2. Change the resolver property in stack.yaml to the resolver you just found.
If on an Apple M1 or M2 system, modify your stack.yaml (contents show below):
Create an executable script named start-ide.sh in your project dir with contents:

// ./.vscode/settings.json
{
  "editor.acceptSuggestionOnCommitCharacter": false,
  "editor.acceptSuggestionOnEnter": "smart",
  "editor.quickSuggestions":
  {
    "comments": "inline",
    "other": "inline",
    "strings":  "inline"
  },
  "editor.quickSuggestionsDelay": 2000,
  "editor.snippetSuggestions": "none",
  "editor.suggest.filterGraceful": false,
  "editor.suggest.preview": true,
  "editor.suggest.showStatusBar": true,
  "editor.suggest.showWords": true,
  "editor.suggestOnTriggerCharacters": false,
  "editor.suggestSelection": "first",
  "editor.wordBasedSuggestionsMode": "matchingDocuments",
  "goto-documentation.customDocs": {
    "hs": "http://localhost:8080/?hoogle=${query}"
  }
}

# add to ./stack.yaml
system-ghc: true
install-ghc: false
skip-ghc-check: true

# ./start-ide.sh
stack hoogle --rebuild
stack hoogle --server 2> /dev/null 1> /dev/null &
(stack exec ghcid -- \
  --command 'stack repl --ghc-options "-fno-code -ignore-dot-ghci -ferror-spans"' \
  --test ':! stack exec hasktags -- --ctags . && stack hoogle -- generate --local ${your_package_name}' \
  --warnings \
  --outputfile ghcid.txt) || true
(pkill ghcid && echo "Killed Ghcid.") || echo "No Ghcid process to kill."
(pkill hoogle && echo "Killed Hoogle.") || echo "No Hoogle process to kill."

Every time you work on your project:

Run ./start-ide.sh in a terminal and minimize it.
Each time you modify your .cabal file, stack.yaml, or package.yaml, you need to kill Ghcid and restart ./start-ide.sh.

Sets, Classes, and Wabbits

Mon, 14 Mar 2022 00:00:00 +0000

Set theory is a bit of a trickster. Some of its ideas go back to Plato (forms) and Aristotle (categorical syllogisms), but sets don’t quite capture the objects of these disciplines. Rather, the objects of those disciplines are captured more closely by the concept of classes (as in classifications). Sets vs. classes? What’s the difference? Why split hares? This blog post will go into how Set Theory fails to capture our intuitive notions around classification.

We intuitively classify things, drawing comparisons and making distinctions. Questions about a platonic form are best understood as questions about the class—not necessarily set—of all things that meets some defining criteria. Similarly, Categorical Syllogism is the logic of inferring class—not necessarily set—membership based on logical predicates. We’d like a mathematical theory of sets that corresponds to our intuitive notions of classification. That is, we’d like to be able to take an arbitrary logical predicate and then form the set of all things that satisfy said predicate. Unfortunately, such a vision is untenable, as its unrestricted application leads to paradox.

How about a nice, close shave? Teach your whiskers to behave.
Lots of lather, lots of soap. Please hold still, don’t be a dope.
Now we’re ready for the scraping There’s no use to try escaping.
Yell and scream and rant and rave. It’s no use, you need a shave!

Bugs Bunny, from The Rabbit of Seville

In Seville, Spain there lives and works a sole barber, Conejito. Now, Seville is a rather big city, with the better part of a million sevillians. It’d be absurd to think that Conejito is every sevillian’s barber. Instead, Conejito is barber to those sevillians, and only those sevillians, who are not their own barber. So then, who is Conejito’s barber? Either Conejito is his barber or he is his own barber. If Conejito is his barber, then that just means that Conejito is Conejito’s barber. That’s fine, broadly speaking a person can be their own barber, but is there any reason why Conejito in particular can’t be his own barber? There is a reason. An established facts about Conejito is that he is barber to a sevillian if and only if that sevillian is not their own barber. If someone is their own barber, then the only if part entails that Conejito can’t be that person’s barber. Applied to Conejito, that means Conejito can’t be his own barber. So that’s fine, we’ve established that Conejito can’t be his own barber. Except now, the if part entails that Conejito is the barber of any sevillian that isn’t their own barber. In particular, this would make Conejito his own barber. Adopting either alternative leads us immediately to the other alternative, yet they are exclusive alternatives, and so we have no viable resolution. This is to say that the very idea of there being a person such as Conejito is self-contradictory.

This might sound scary and bad at first, but it’s really not a big problem at all. It’s certainly not the paradox I alluded to earlier. We could have seen Conejito’s self-contradictory nature formally. Let $B(u,v)$ be the proposition that $u$ is $v$’s barber. Then, \[ (\forall x) \big( B(y,x) \iff \neg B(x,x) \big) \] is the proposition that $y$ is barber to everyone who is not their own barber. Assume for contradiction that at least one such barber exists, \[ (\exists y) \Big( (\forall x) \big( B(y,x) \iff \neg B(x,x) \big) \Big) \] Now, let $c$ stand for one such barber in particular, so we have \[ (\forall x) \big( B(c,x) \iff \neg B(x,x) \big) \text{.} \] Since this statement applies generally to all $x$, it applies in particular when $x$ is $c$, yielding \[ B(c, c) \iff \neg B(c, c) \text{.} \] At this point, the contradiction should be apparent, but let’s finish the proof just out of caution. Splitting the bi-implication, we have \[ \big( B(c, c) \implies \neg B(c, c) \big) \wedge \big( \neg B(c, c) \implies B(c,c) \big) \text{.} \] Applying Material Implication to both conjuncts gives \[ \big( \neg B(c, c) \vee \neg B(c, c) \big) \wedge \big( B(c, c) \vee B(c,c) \big) \] which reduces to \[ \neg B(c, c) \wedge B(c, c)\text{,} \] a contradiction, whereby we conclude the negation of our assumption, \[ \neg (\exists y) \Big( (\forall x) \big( B(y,x) \iff \neg B(x,x) \big) \Big) \text{.} \]

All this amounts to is the fact that the class of all sevillians who are barber to those, and only those, sevillains who aren’t their own barber is empty. This is not a paradox, it’s simply an observation. \[ \Big\{ y \,\Big\vert\, (\forall x) \big( B(y,x) \iff \neg B(x,x) \big) \Big\} = \emptyset \]

Now we’re ready to begin the main course. Let’s assume we have a coherent theory of sets, where we can take any logical predicate and form a set consisting of all entities in the universe that satisfy said predicate (ie. imagine we have unrestricted set comprehension).

Consider in particular the predicate that $x$ is not a member of the set $y$, $x \notin y$. For various reasons that I won’t get into today, we’d like to consider something like a subuniverse of all entities that are well-behaved in the following sense. Specifically, we’d like to only consider sets that do not contain themselves. Thus, it’s reasonable to consider the set of all sets that do not contain themselves, taking that to be our favored subuniverse. \[ U = \{ x \,\vert\, x \notin x \} \]

The question we now face: is $U$ a member of $U$? By $U$’s definition, we find that $U$ is a member of $U$ if and only if $U \notin U$. \[ U \in U \iff U \notin U \text{.} \]

We’ve seen this exact logical form before, when we saw why Conejito can’t exist. In the same way, it tells us that no such set $U$ can exist. This is not to say that $U$ is empty. If $U$ were merely empty we’d be disappointed, but it wouldn’t necessarily be a paradox. But the situation we have here is far more subtle. We see that the set $U$ itself can’t exist, empty or otherwise, in exactly the same way that Conejito can’t exist: nothing that satisfies the logical form \[ \varphi(x,x) \iff \neg \varphi(x,x) \] can exist no matter what the predicate $\varphi$ may be (as we demonstrated above, in fact).

This flies in the face of our desire to be able to form sets corresponding to arbitrary logical predicates, showing us that no coherent system of set theory will correspond exactly to our intuitive notion of classes. The solution—which I might write up in a subsequent blog post one of these days—is to be very careful and precise about what sets you allow yourself to form. We end up relying on a few basic constructions and then build up our universe of sets inductively (ie. recursively) from there.

Haskell GoF - Abstract Factory

Wed, 19 Jan 2022 00:00:00 +0000

The beginning of a series wherein we show how to implement selected design patterns from Design Patterns: Elements of Reusable Object-Oriented Software. This week, we take a look at the venerable Abstract Factory pattern.

A common question new Haskell programmers have is, “How do I implement <this-or-that> OOP design pattern in Haskell.” These curious and resolute dames and gents are (quite reasonably) hoping to leverage their background in object-oriented programming towards their new goal of learning functional programming and Haskell. Unfortunately, it’s not always straightforward to translate patterns that rely on the familiar notions of class hierarchies, inheritance, instantiation, virtual methods, collaborators, encapsulation, and mutation to the alien landscape of datatypes, typeclasses, parametricity, immutability, pattern matching, and higher-order functions that is Haskell. Worse, many a young and impressionable Haskellers develop misleading sets of analogies between OOP concepts and Haskell features, ultimately leading to confusion and frustration down the line. For example, Haskell typeclasses bear only the most superficial resemblance to Java interfaces.

This series has two goals: (1) to provide new Haskell programmers with a menagerie of translations between familiar OOP code idioms and Haskell, and (2) to dispel dubious analogies between OOP languages features and Haskell language features by replacing them with well-founded and practical analogies. As the book of design patterns is often affectionately (and sometimes derisively) referred to as Gang of Four, we’ll unimaginatively call this series Haskell GoF.

Expect new posts once every blue moon.

Identifying the Pattern

We should begin our explorations with a shared understanding of the topic at hand. In particular, what makes something a design pattern? The concept of a design pattern became popular in programming circles due to the broadly influential book on (brick and mortar) architectural design, The Timeless Way of Building by Christopher Alexander, architect and emeritus professor at the University of California, Berkeley.

Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice.

Christopher Alexander

To paraphrase Dr. Alexander, a design pattern is an outline of a solution to a common problem that does not admit a generic solution. In programming, we often come across a problem that does admit its own generic solution. Since the solution to such a problem is generic, we need only write the solution down once, package that solution into a library, and import the library wherever the need arises. Such solutions are not design patterns. A design pattern for a recurring programming problem is a solution template that resists codification.

Next, we need to understand why certain solutions (or outlines of solutions) can’t be codified. One reason is that sometimes the problem itself is vague. Often, we can spot similarities in various coding problems—similarities that lead to similar but somewhat different solutions—without being able to identify a precise defining characteristic of such a class of problems. Thus, while a vague problem usually can’t have a precise solution, it can have a vague solution. This vague solution becomes a design pattern.

Another reason a solution might not be realizable as a library is that we might be missing the right language features. There’s lots of talk about the relative abstraction power of various programming languages. Really, this so-called power of abstraction is simply the ability to parametrize repeated syntactical forms by the sub-expressions by which those forms vary, enabling elimination of the syntactic repetition. (What else do you think a function is?) Through their features and syntactic rules, different languages enable and prohibit abstraction of various syntactical forms.¹ The repeated syntactic forms that can’t be abstracted become design patterns instead of libraries.

Before proceeding, there’s one more point of terminology on which we should agree, since it shows up so frequently in discussions of patterns: the usage of the word abstract. Programmers tend to attach a peculiar meaning to the word abstract. They often mean “complicated implementation hidden behind a simple API,” but this is not a description of something abstract. On the contrary, this is a description of something very real and concrete; we have a full implementation inside there, after all. A human body is quite complex inside, but the (non-lethal) ways in and out are clearly defined and simple, so should we call human bodies abstract? On top of that, programmers already have another word for this concept of implementation hiding: encapsulation. Let’s not use the term abstract to describe a complex, concrete system with a small, tidy surface when we already have a much more evocative word to describe it. Abstract and concrete are at odds, anyway.

Consider: what’s abstract about abstract art? A central feature of abstract art is that it avoids depiction of identifiable, concrete objects. Instead, we have pure geometry of shapes, lines, brushstrokes, and color. Instead of an art piece having a definite meaning, different onlookers will be able to imagine different meanings. The piece is open to the interpretation of the onlooker, and this property cuts to the heart of what it means for something to be abstract. Indeed, abstract means “open to interpretation.”

Abstraction helps us decouple our code in ways that encapsulation can’t. On the left, we see the dependency structure we get when we encapsulate a third-party library. On the right, we abstract it. When we abstract it, fewer of our code units depend on the third-party library, decoupling our application code from our external dependencies.

Now that we agree (at least for the purposes of this discussion) on what our words mean—on what design patterns are and why we might need them and on the nature of abstraction more generally—we’re ready to tackle our first pattern, the Abstract Factory.

Abstract Factory

Provide an interface for creating families of related or dependent objects without specifying their concrete classes.

GoF

Suppose we’d like to write an algorithm or build up a data structure using a bespoke set of primitives while allowing the interpretation of that algorithm or data structure into several different concrete forms. The Abstract Factory Pattern gives us a way to accomplish this.

That’s a mouthful, so let’s consider a practical example: graphical user interfaces (GUIs). Each software platform (Linux, Mac, Web, Windows, etc.) provides its own toolkit for rendering things to the screen. These toolkits don’t necessarily have any commonalities between them, so if we want to write a cross-platform program, we’re left to repeatedly implement our GUI for each platform we target. Alternatively, we can come up with our own set of primitives for describing our program’s GUI. We can write more complicated components in terms of our primitives, and in turn combine those components to create our complete GUI, abstractly (that is, without reference to any of the concrete, platform-dependent toolkits). Then, instead of implementing every high-level component in each toolkit, we need merely define how each primitive (and each combining rule) is implemented in each of the various toolkits. We thus interpret the exact same abstract, platform-independent GUI code into concrete GUIs appropriate to each target platform. Qapla’!

Let’s look at an implementation in classic Java.²

interface Widget {}

class HtmlWidget implements Widget {
  void addStyle(String key, String value) {...}
  void addAttribute(String key, String value) {...}
}

class Text extends HtmlWidget {
  Text(String content) {...}
}

class Tag extends HtmlWidget {
  Tag(String tag, HtmlWidget[] children) {...}
}

class LatexWidget implements Widget {}

class BareText extends LatexWidget {
  BareText(String text) {...}
}

class Command extends LatexWidget {
  Command(String name, String[] parameters) {...}
}

class Environment extends LatexWidget {
  Environment(String name, String[] parameters, LatexWidget content) {...}
}

interface AbstractWidgetFactory {
  Widget column(Widget[] children);
  Widget row(Widget[] children);
  Widget image(String path);
  Widget text(String content);
  Widget heading(String content);
}

(The symbol ... indicates details that we’ve omitted for brevity.)

Each of HtmlWidget and LatexWidget wraps³ a third-party toolkit for one of our target platforms. Notice that they have very different APIs. Nevertheless, the differences in their APIs won’t stop us from finding a mutual abstraction: we will still be able to make an implementation of AbstractWidgetFactory for both of them, despite their idiosyncrasies.

Before moving on, it’s important to understand that AbstractWidgetFactory is not an (abstract widget) factory. Rather, it is an abstract (widget factory). In other words, it is not a concrete factory that produces hypothetical widgets; it hypothesizes a factory that produces real widgets. The factory itself is abstract, not the widgets. AbstractWidgetFactory is abstract in the sense that its methods are not implemented and must be given an interpretation by any implementing classes.⁴

Using the primitives provided by AbstractWidgetFactory, we can build up larger reusable components, and ultimately combine those components to build out our entire GUI. This can be accomplished without any knowledge of (e.g. without depending on) the various concrete subtypes of Widget. As such, we have a component that works with any subtype of Widget—even ones that haven’t been conceived of yet—so long as we’re given a factory that will produce widgets of that type. In this way, we completely decouple the implementation of our GUI from the hardware toolkits our target platforms provide. Here’s an example of one reusable component, a table.

interface TableCol {
  Widget title();
  Widget getField(TableRow row);
}

interface TableRow {
  Widget field(TableCol col);
}

Widget table(AbstractWidgetFactory factory, TableCol[] cols, TableRow[] rows) {
  ArrayList rows0 = new ArrayList();

  ArrayList labels = new ArrayList();
  for (TableCol col : cols) {
    labels.add(col.title());
  }
  rows0.add(factory.row((Widget[]) labels.toArray()));

  for (TableRow row : rows) {
    ArrayList fields = new ArrayList();
    for (TableCol col : cols) {
      fields.add(row.field(col));
    }
    rows0.add(factory.row((Widget[]) fields.toArray()));
  }

  return factory.column((Widget[]) rows0.toArray());
}

Since table is written in terms of the AbstractWidgetFactory primitives, we will be able to use it generically with any of the platform toolkits by passing in different implementations of AbstractWidgetFactory. “Write once run anywhere” never felt so visceral!

Finally, let’s see the implementation of AbstractWidgetFactory for HtmlWidget.⁵

class HtmlWidgetFactory implements AbstractWidgetFactory {

  public HtmlWidget column(Widget[] children) {
    for (Widget child : children) {
      ((HtmlWidget) child).addStyle("display", "block");
    }
    return new Tag("div", (HtmlWidget[]) children);
  }

  public HtmlWidget row(Widget[] children) {
    for (Widget child : children) {
      ((HtmlWidget) child).addStyle("display", "inline-block");
    }
    return new Tag("div", (HtmlWidget[]) children);
  }

  public HtmlWidget image(String path) {
    HtmlWidget img = new Tag("image", new HtmlWidget[]{});
    img.addAttribute("source", path);
    return img;
  }

  public HtmlWidget text(String content) {
    return new Tag("p", new HtmlWidget[]{new Text(content)});
  }

  public HtmlWidget heading(String content) {
    return new Tag("h2", new HtmlWidget[]{new Text(content)});
  }
}

One glaring flaw in this design is that we have to perform unsafe type casts from Widgets to HtmlWidgets in order to implement column and row. Nothing can stop us from accidentally passing LatexWidgets into an HtmlWidgetFactory, causing a runtime error that will crash our program. Indeed, the GoF authors are aware of and admit to this flaw. The flaw can be completely overcome with the use of type parameters, but lacking type parameters, the GoF authors suggest that this danger can be mitigated by ensuring that only one factory is ever in scope at a time and that all Widgets in your program be created from that one factory.⁶

And that’s all there is to the Abstract Factory. We’ve accomplished our goal of writing our UI once against a set of abstract primitives (i.e. the methods of AbstractWidgetFactory), affording us the ability to interpret that UI against each of our concrete toolkits by simply interpreting the primitives (i.e. by implementing a concrete subclass of AbstractWidgetFactory). This is the crucial characteristic of the Abstract Factory pattern. Our operative code does not have a dependency on either HtmlWidget or LatexWidget, nor does it have a dependency on their respective factories. Neither does AbstractFactory have a dependency on either of HtmlWidget or LatexWidget. This is important.

As a point of contrast, we can imagine a code structure where we encapsulate HtmlWidget or LatexWidget (or both) behind an API that is more convenient for our immediate usage. This is a great practice, but it’s not an abstract factory.⁷ Sometimes you want such a code structure, but for the problem we’re considering today it’s inadequate; it doesn’t accomplish our goal of having platform-independent code. Our code would depend on the wrapper class, which in turn depends on the concrete, platform-dependent subclass(es) of Widget that the wrapper encapsulates. This restricts us to only those toolkits that the author of the wrapper class thought to include. It also forces us to depend on all the included toolkits, when we’d likely only want to depend on one per any given executable.

On the other hand, the genuine abstraction characteristic of the Abstract Factory Pattern (versus the encapsulation provided by the Adapter Pattern) allows us to decouple our operative code from dependency on any concrete toolkit. Without abstraction, we cannot achieve this separation.

Haskell Abstract Factory

All in one go, I’ll show you my Haskell version of the above code. We’ll go into more detail below, so just give it a quick skim for now.

{-# LANGUAGE DerivingVia, NamedFieldPuns #-}

module AbstractFactory where

import Data.Foldable (fold)
import Data.Map (Map, insert)

----
-- GUI primitives.

class Widget a where
  column :: [a] -> a
  row :: [a] -> a
  image :: FilePath -> a
  text :: String -> a
  heading :: String -> a

----
-- Toolkit-independent GUI components.

table :: Widget a => [(a, r -> a)] -> [r] -> a
table cols rows =
  column (headingRow : dataRows)
  where
    headingRow = row (fmap fst cols)
    dataRows = fmap makeDataRow rows
    makeDataRow r = row (fmap (($ r) . snd) cols)

aboutMe :: Widget a => a
aboutMe =
  column
    [ heading "About me"
    , text
      "Welcome to my site.\
      \ My name is Daniel.\
      \ Thanks for visiting."
    , row
      [ image "skip.png"
      , text
        "Please enjoy a picture of my dog,\
        \ Scipio, or ‟Skip” for short.\
        \ Skip is a German Shepherd-Australian\
        \ Cattle Dog mix, and even though he's five\
        \ years old, he's got boundless energy to keep\
        \ me company on our long hikes through the hills.\
        \ Skip and I do almost everything together."
      ]
    , row
      [ text
        "One thing Skip can't do with me, though,\
        \ is Swing dance. Here's a gif of me doing\
        \ a Lindy Circle, which looks way more impressive\
        \ that it really is. (It's pretty much just the\
        \ basic step...)\n\
        \\n\
        \I'm learning, slowly 😅"
      , image "lindy-circle.gif"
      ]
    , heading "Places I've lived"
    , let
        tblCols =
          [ (text "City", fmtCity)
          , (text "Duration (years)", fmtYears)
          ]
        tblRows =
          [ ("Rialto", 19.5)
          , ("Camarillo", 4.5)
          , ("Auburn", 7)
          , ("Bakersfield", 1)
          , ("Santa Barbara", 1)
          , ("Ventura", 4)
          , ("San Francisco", 0.2)
          ]
        fmtCity (c, _) = text c
        fmtYears (_, y) = text (show y)
      in
        table tblCols tblRows
    ]

----
-- Toolkit 1: HTML

-- Yes, partial fields. Don't 'at' me!
data Html
  = Tag
    { tag :: String
    , styles :: Map String String
    , attrs :: Map String String
    , children :: [Html]
    }
  | Text String

html :: String -> [Html] -> Html
html tag0 =
  Tag tag0 mempty mempty

style :: String -> String -> Html -> Html
style key val html0 =
  case html0 of
    Text {} -> style key val $ html "span" [html0]
    Tag {styles} -> html0 {styles = insert key val styles}

attr :: String -> String -> Html -> Html
attr key val html0 =
  case html0 of
    Text {} -> attr key val $ html "span" [html0]
    Tag {attrs} -> html0 {attrs = insert key val attrs}

instance Widget Html where
  column chl =
    html "div" (fmap (style "display" "block") chl)

  row chl =
    html "div" (fmap (style "display" "inline-block") chl)

  image path =
    attr "source" path (html "image" [])

  text str =
    html "p" [Text str]

  heading str =
    html "h2" [Text str]

----
-- Toolkit 2: Latex

newtype Latex = Latex [String]
  deriving (Semigroup, Monoid) via [String]

bare :: String -> Latex
bare str = Latex [str]

command :: String -> [String] -> Latex
command name params =
  Latex ["\\" <> name <> mkParams params]

env :: String -> [String] -> Latex -> Latex
env name params body =
  fold
    [ Latex $ ["\\begin{" <> name <> "}" <> mkParams params]
    , body
    , Latex $ ["\\end{" <> name <> "}"]
    ]

mkParams :: [String] -> String
mkParams params =
  fold ["{" <> param <> "}" | param <- params]

instance Widget Latex where
  column chl =
    env "center" [] $
      env "tabular" ["c"] $
        foldMap (<> Latex [" \\\\"]) chl

  row chl =
    env "center" [] $
      env "tabular" [fold ["c " | _ <- chl]] $
        foldMap (<> Latex [" & "]) chl

  image path =
    command "includegraphics" [path]

  text = bare

  heading str =
    command "section" [str]

The first thing one might notice is the curious absence of anything called a factory. With that in mind, let’s take a closer look at the Widget typeclass.

class Widget a where
  column :: [a] -> a
  row :: [a] -> a
  image :: FilePath -> a
  text :: String -> a
  heading :: String -> a

data Html = ...
instance Widget Html where ...

data Latex = ...
instance Widget Latex where ...

Our Widget typeclass represents the notion that certain types can be said to be Widget types.⁸ A type is a Widget type if we have ways to create and combine values of that type the in the ways we expect to be able to create and combine widgets (namely, that appropriate functions column, row, image, text, and heading exist). That is to say, in this translation of the above Java code, it’s the typeclass Widget that plays the role of the abstract factory, and it’s the typeclass instances instance Widget Html and instance Widget Latex that play the roles of the concrete factory implementations. The type parameter a, oddly enough, plays the role of the Java version’s Widget interface.

The most important question is this: does the Haskell version solve the core problem that the Abstract Factory Pattern is meant to solve? Recall the core problem, as stated by GoF.

Provide an interface for creating families of related or dependent objects without specifying their concrete classes.

We see that, indeed, the Widget typeclass provides us with a means of creating GUI widgets without specifying their concrete ~~classes~~ types. The evidence lies in table and aboutMe.

table :: Widget a => [(a, r -> a)] -> [r] -> a
aboutMe :: Widget a => a

table and aboutMe each creates a widget of an unspecified type a. That is, table and aboutMe can be used to create widgets of any type, even widget types we haven’t thought of yet: they’re polymorphic over all widget types. By writing polymorphic functions with a Widget typeclass constraint, we’re able to completely decouple our GUI code from any concrete GUI toolkit. Furthermore, as long as we can write a Widget typeclass instance to bootstrap the process, we’re able to translate our GUI code into any existing or future toolkit.

Alternate Haskell Versions

Lest you develop the mistaken notion that typeclasses correspond exactly to instances of the Abstract Factory Pattern, we’ll take a look at two other Haskell versions of the same program.

Record Encoding

You might be thinking that my analogy between the Widget typeclass and the Abstract Factory Pattern is somewhat thin. After all, where’s the actual factory? Consider an alternative, more literal Haskell translation of the Java version. Suppose that we had no Widget typeclass, and instead had a WidgetFactory record.

data WidgetFactory a =
  WidgetFactory
    { column :: [a] -> a
    , row :: [a] -> a
    , image :: FilePath -> a
    , text :: String -> a
    , heading :: String -> a
    }

In this encoding, we would explicitly pass a value of type WidgetFactory a to table and aboutMe.

table :: WidgetFactory a -> [(a, r -> a)] -> [r] -> a
table factory cols rows = ...

aboutMe :: WidgetFactory a -> a
aboutMe factory = ...

Instead of invoking a top-level rows to a list of widgets childWidgets, you’d extract the rows field from your factory and apply to the list of widgets, vis a vis (rows factory) childWidgets. Instead of class instances, you’d write functions that return WidgetFactorys.

data Html = ...
htmlFactory :: WidgetFactory Html
htmlFactory =
  WidgetFactory
    { columns = \chl ->
        html "div" (fmap (style "display" "block") chl)
    , row = \chl ->
        html "div" (fmap (style "display" "inline-block") chl)
    , image = \path ->
        attr "source" path (html "image" [])
    , text = \str ->
        html "p" [Text str]
    , heading = \str ->
        html "h2" [Text str]
    }

data Latex = ...
latexWidgetFactory :: WidgetFactory Latex
latexWidgetFactory =
  WidgetFactory
    { columns = ...
    , ...
    }

Notice that in both the typeclass version and the record version, the type parameter a guards us from passing the wrong kind of widget into a factory. In the Haskell versions, the types Html and Latex don’t need a common subtype the way they do in the Java version. In fact, the type parameter completely obviates the need for a type hierarchy, as the only reason we had to have a type hierarchy in the Java version was so that we could provide signatures for the AbstractWidgetFactory methods.

Another thing to notice is that the record version allows us to have multiple factories for the same type in scope. We could have two different values of type WidgetFactory Html, and we could choose between them for whatever various reasons. Sometimes this is what you want, and it’s slightly annoying to pull off in the typeclass version, making the record encoding preferable in such cases. In typical cases, though, having multiple factories for the same type is decidedly not what we want. We usually want to ensure that there’s only one factory of a given type in scope. The typeclass version ensures that this is the case, while the record version leaves this as a backdoor for potential bugs.⁹

Algebraic Datatype Encoding

While the typeclass version and the record version are morally the same encoding (the record encoding follows the typeclass encoding in an entierly formulaic way¹⁰), we can yet model the Abstract Factory Pattern in an altogether different way by inverting the priority of the principle players. By that, I mean we change our notion of which concepts are primitive and which concepts are composite.

data Widget
  = Column [Widget]
  | Row [Widget]
  | Image FilePath
  | TextW String -- the constructor name `Text` is already taken by `Html`
  | Heading String

table :: [(Widget, r -> Widget)] -> [r] -> Widget
table cols rows =
  Column (headingRow : dataRows)
  where
    headingRow = Row (fmap fst cols)
    dataRows = fmap makeDataRow rows
    makeDataRow r = Row (fmap ((S r) . snd) cols)

asHtml :: Widget -> Html
asHtml widget = case widget of
  Column chl ->
    html "div" (fmap (style "display" "block" . asHtml) chl)
  Row chl ->
    html "div" (fmap (style "display" "inline-block" . asHtml) chl)
  Image path ->
    attr "source" path (html "image" [])
  TextW str ->
    html "p" [Text str]
  Heading str ->
    html "h2" [Text str]

asLatex :: Widget -> Latex
asLatex widget = case widget of
  Column chl -> ...
  ...

In this encoding, the type Widget consists of abstract syntax trees for our notions of what a widget should be and how we should be able to manipulate and combine widgets. We write our GUI code in terms of the abstract syntax. We then write the interpreters asHtml and asLatex that interpret the abstract syntax in terms of the respective concrete toolkits.

Concluding Remarks

Notice that, at the end of the day, we end up writing what’s basically all the same code in all three versions. The cases for asHtml end up having nearly identical code as the methods of the typeclass instance Widget Html and the fields of the record htmlWidgetFactory :: WidgetFactory Html. None of these versions results in significantly less code than either of the others.

All three encodings have their strengths and weaknesses. For example, a nice thing about the datatype encoding is that Widgets are now serializable. A nice thing about the record encoding is that you can have multiple factories in scope and select one or the other at runtime, if that’s a thing you want to do. You could even construct new factories at runtime. Some nice things about the typeclass encoding are that it’s very DRY—your Widget functions are top-level, so you neither need to keep pulling them out of some record nor need to keep passing said record down the call stack—and it ensures that there’s only one factory in scope for any given type, if that’s a thing you want to ensure.

I wanted to show you three different encodings because I don’t want you to get the idea that typeclasses are necessarily some kind of manifestation of the Abstract Factory Pattern. Typeclasses are their own thing, and it just so happens that they provide a way to emulate the Abstract Factory Pattern. But so do records and so do algebraic datatypes, so there’s no special connection between abstract factories and typeclasses. I also don’t want you to think that typeclasses correspond to OOP interfaces (really, records are closer to OOP interfaces, but still not quite the same). In general, we shouldn’t try to directly map OOP concepts onto Haskell concepts, because to do so is a form of the XY Problem. Instead, we should always take a deep breath and go back to the root problem we’re trying to solve. We may find a simple answer.

Recall that the basic problem the Abstract Factory Pattern solves is fundamentally about decoupling. The three above Haskell versions demonstrate two simple ideas. First, we have the record and typeclass encodings, which illustrate the utility of type parameters and callbacks for decoupling code. The record (respectively, the typeclass instance) is really just a way of passing in a bunch of callbacks to a function, when you think about it. Second, the datatype encoding illustrates the utility of domain-specific languages (DSLs) and interpreters for decoupling code. Write your program in terms of a simple, declarative grammar with a few primitives and basic combination rules. Then, interpret your program by simply interpreting the primitives and basic rules, a form of induction/recursion.

Summary

Design patterns are templates of solutions that—for whatever reason—evade being codified into a reusable function, component, or library.
Abstractions are open to interpretation. Abstract stands in contrast to concrete.
Merely hiding concrete implementation details doesn’t make something abstract; rather, it encapsulates (which is fine, encapsulation is appropriate at times).
Abstraction decouples code. Encapsulation does not.
An Abstract Widget Factory is an abstract (widget factory), not an (abstract widget) factory. Explicitly, it’s the factory that’s abstract, not the widgets. An abstract factory is a placeholder for a concrete factory that will be given to your code at some future time.
An OOP-style abstract factory can be modeled in Haskell using classes, records, or an algebraic datatype. None is clearly better than the others.
You can use type parameters and callbacks to decouple code.
You can use domain-specific languages and interpreters to decouple code.
In general, try to avoid thinking about how to directly translate OOP solutions to Haskell. Instead, go back to the original problem.
Keep it simple and don’t overthink things. At the end of the day, it’s all just passing arguments to functions.

Notice that the incredible facility for abstraction in Lisp has less to do with having copious language features (as Lisp has hardly any) and more to do with the elegant simplicity of its syntax. ↩
Java, first released in 1995, lacked type parameters before its fifth edition in 2004. This lack of type parameters had a profound impact on the nature of the code people wrote in Java. You wouldn’t think so, but the lack of type parameters inevitably leads to the style of mutation-heavy, void-method-driven APIs we see in so many Java libraries today. When we give examples in this series, we’ll stick to classic Java, that is, Java without type parameters. This fits in better with the style of programming in GoF (published in 1994), which also lacks type parameters. ↩
We have to wrap the third-party toolkits because Widget is our class, so they can’t extend it. Later on, we’ll see how to do this wrapping using the Adapter Pattern. ↩
Indeed, this was the case for all interfaces before Java’s eighth edition. Interfaces are a mechanism for abstraction. As such, the Abstract in AbstractWidgetFactory is redundant. We already know it’s abstract, because it’s an interface. One can only presume that the GoF authors used this redundant naming convention as a pedantic device to drive home their points. Unfortunately, the obtusely-redundant naming convention became part of the Java culture, leading to such absurdities as “Since ConcreteWidgetFactory implements AbstractWidgetFactory, a ConcreteWidgetFactory is an AbstractWidgetFactory.” No wonder programmers are so confused about the meaning of the word abstract! ↩
I’m not going to implement AbstractWidgetFactory for LatexWidget in Java, because it’s a huge pain in the ass. I will implement it in Haskell, though. ↩
The GoF authors suggest that the programmer accomplish this careful control of scope using the Singleton pattern, which we will examine in a subsequent post. ↩
It is an example of the Adapter Patter, though, which we’ll examine in a subsequent post. ↩
This is where we get the term type class. A type class is a class (or set) of types that all satisfy some required conditions. Type class instances define precisely how a particular member of the type class satisfies the required conditions. ↩
It’s usually not that big of a deal, to be honest. ↩
See the legendary blog post Scrap Your Type Classes by the incomperable Gabriella Gonzalez for details. ↩

Of function instances and abstract syntax

Tue, 20 Apr 2021 00:00:00 +0000

Some Haskell classes class Myclass a admit an instance for functions instance Myclass a => Myclass (x -> a) based on the instance for a. All of these instances have a few things in common: (1) they implement the class methods in a straightforward way as mymethod f = \x -> mymethod (f x), and (2) they are polarizing among Haskell practitioners. The sequel is a case study of why I find such instances compelling and useful.

{-# LANGUAGE NoImplicitPrelude,
             RankNTypes,
             RebindableSyntax,
             ViewPatterns #-}

module Polynomial where

import Data.Void (Void)
import Prelude (Either, Integer, String, foldl, (==), (<>))
import qualified Prelude

The dictionary of functions that comprise a Num instance is roughly (if you omit abs and signum) the signature of what math folk call a commutative ring.

class Ring a where
  (+) :: a -> a -> a
  (*) :: a -> a -> a
  negate :: a -> a
  fromInteger :: Integer -> a

A commutative ring $R$ is a set (or type) on which associative, commutative operations $+$ and $\cdot$ are defined, a unary operation $-$ is defined that takes a ring element to its $+$ inverse, $\cdot$ distributes over $+$, there is an element called $0$ that is an identity for $+$, there is an element called $1$ that is an identity for $\cdot$, and $0$ and $1$ are distinct. This gives us our +, *, and negate functions in Haskell’s Num.

For any commutative ring $R$, it is possible to define one (and only one) function $i$ from the integers to $R$ with the following special properties:

$i$ distributes over the usual $+$ and $\cdot$ of the integers, yielding the $+$ and $\cdot$ of $R$ when it does, i.e.
\[\begin{align*} i(x + y) &= i(x) + i(y) \\ i(x \cdot y) &= i(x) \cdot i(y) \end{align*}\]
where $+$ and $\cdot$ on the left hand side refer to the usual operations defined for integers, but $+$ and $\cdot$ on the right hand side refer to the ring operations defined for $R$.
$i(0)$ is the $0$ of $R$, and $i(1)$ is the $1$ of $R$.

Facts 1 and 2 tell us that every commutative ring (abbreviated to just ring for the remainder of this discussion) has an image of the integers inside of it. This function $i$ is analogous to our fromInteger function in Haskell’s Num. At the same time, implementing fromInteger for a Num instances implicitly defines the $0$ and $1$ of our ring vis a vis fromInteger 0 and fromInteger 1.

instance Ring Integer where
  (+) = (Prelude.+)
  (*) = (Prelude.*)
  negate = Prelude.negate
  fromInteger = Prelude.fromInteger

Obviously $\mathbb{Z}$, the set of all integers, is a ring. In a sense, $\mathbb{Z}$ is the most prototypical ring. But there are many other rings, some exotic. For example, Take any set $X$, and form the set $\mathop{\mathrm{Pow}}(X)$ consisting of all subsets of $X$. Recall $\cap$ is the usual intersection of sets, $\cup$ is the usual union of sets, and $\setminus$ is the usual difference of sets (i.e., $E_1 \setminus E_2 = \left\{ x \mid x \in E_1 \text{ and } x \notin E_2 \right\}$). Define $\mathop{\Delta}$ by $E_1 \mathop{\Delta} E_2 = (E_1 \cup E_2) \setminus (E_1 \cap E_2)$. Then, perhaps surprisingly, $\mathop{\mathrm{Pow}}(X)$ is a ring using $\mathop{\Delta}$ as $+$, $\cap$ as $\ast$, and the identity function as $-$ (i.e. every set is its own $\Delta$ inverse).

newtype Set a = Set {includes :: a -> Bool}

union :: Set a -> Set a -> Set a
s1 `union` s2 = Set $ \x -> s1 `includes` x || s2 `includes` x

intersection :: Set a -> Set a -> Set a
s1 `intersection` s2 = Set $ \x -> s1 `includes` x && s2 `includes` x

exclusion :: Set a -> Set a -> Set a
s1 `exclusion` s2 = Set $ \x -> s1 `includes` x && not (s2 `includes` x)

none :: Set a
none = Set $ const False

every :: Set a
every = Set $ const True

instance Ring (Set a) where
  s1 + s2 = union s1 s2 `exclusion` intersection s1 s2
  (*) = intersection
  negate = id

  -- Here is the only suitable function, since
  --
  --     fromInteger (2*n) = fromInteger (n + n)
  --                       = fromInteger n + fromInteger n
  --                       = none
  --
  -- and
  --
  --     fromInteger (2*n + 1) = fromInteger (n + n + 1)
  --                           = fromInteger n + fromInteger n + fromInteger 1
  --                           = none + every
  --                           = every
  fromInteger x = if even x then none else every

If $R_1$ and $R_2$ are rings, then $R_1 \times R_2$ (the set of ordered pairs) is a ring using $(x_1, x_2) + (y_1, y_2) = (x_1 + y_1, x_2 + y_2)$ and $(x_1, x_2) \cdot (y_1, y_2) = (x_1 \cdot y_1, x_2 \cdot y_2)$. Towards our main goal, if $R$ is a ring and if $X$ is any arbitrary set, then the set of all functions from $X$ to $R$ is a ring using $f + g = x \mapsto f(x) + g(x)$ and $f \cdot g = x \mapsto f(x) \cdot g(x)$.

instance (Ring a, Ring b) => Ring (a, b) where
  (x1, x2) + (y1, y2) = (x1 + y1, x2 + y2)
  (x1, x2) * (y1, y2) = (x1 * y1, x2 * y2)
  negate (x1, x2) = (negate x1, negate x2)
  fromInteger n = (fromInteger n, fromInteger n)

instance Ring a => Ring (x -> a) where
  f + g = \x -> f x + g x
  f * g = \x -> f x * g x
  negate f = \x -> negate (f x)
  fromInteger n = \_ -> fromInteger n

Finally, the polynomials: if $X$ is any set, define the polynomials in $X$, which we’ll write as $P(X)$, as the set of all formal expressions of the form

\[n_1 x_{1,1} x_{1,2} ... x_{1,k_1} + n_2 x_{2,1} x_ {2,2} ... x_ {2,k_ 2} + ... + n_m x_{m,1} x_{m,2} ... x_{m,k_m}\]

where the $n_i$ are integers, the $k_i$ and $m$ are non-negative integers, and the $x_{i,j}$ are elements of $X$.

Put more constructively:

each individual integer is a polynomial in $X$,
a string of element of $X$ is a polynomial in $X$,
if $p_1$ is a polynomial in $X$ of Form 1 and $p_2$ is a polynomial in $X$ of Form 2, then the juxtaposition $p_1 p_2$ is a polynomial in $X$, and
if $p_1$ and $p_2$ are a polynomials in $X$, then the formal expression $p_1 + p_2$ is a polynomial in $X$.

We could directly encode this as a Haskell datatype as follows.

data PExpr x
  = Form1 Integer
  | Form2 [x]
  | Form3 Integer [x]
  | Form4 (PExpr x) (PExpr x)

Doing so, we notice that Form3 subsumes Form1 and Form2, so we will work with a simplified definition.

data PExpr x
  = Form3 Integer [x]
  | Form4 (PExpr x) (PExpr x)

Traditionally elements of $X$ are called the variables, generators, or indeterminates of $P(X)$, so you will often see $P(X)$ referred to as the polynomials generated by $X$ or as the polynomials with indeterminates in $X$. Elements of $X$ may be lifted into $P(X)$ (i.e., thought of as elements of $P(X)$) in light of Form 2.

liftVar :: x -> PExpr x
liftVar x = Form3 1 [x]

$P(X)$ forms a ring, independent of (e.g. ignoring) any existing ring structure on $X$ itself.

instance Ring (PExpr x) where
  p1 + p2 = Form4 p1 p2

  Form4 p1 p2 * p3 = Form4 (p1 * p3) (p2 * p3)
  p1 * Form4 p2 p3 = Form4 (p1 * p2) (p1 * p3)
  Form3 n1 xs1 * Form3 n2 xs2 = Form3 (n1 * n2) (xs1 <> xs2)

  negate (Form3 n ss) = Form3 (negate n) ss
  negate (Form4 p1 p2) = Form4 (negate p1) (negate p2)

  fromInteger n = Form3 n []

Reasonable sets to use for $X$ include any singleton set, from which arise polynomials in one variable, any pair set, from which arise polynomials in two variables, etc. You may derive the set of polynomials in infinitely-many variables by using $\mathbb{Z}$ for $X$, but you must take care when you do: polynomials of Form 1 inherit the ring structure of the integers, but polynomials of Form 2 are merely abstract symbols with no ring structure other than that of $P(\mathbb{Z})$ defined above. Thus, $P(\mathbb{Z})$ contains two distinct copies of $\mathbb{Z}$ with different algebraic properties, and it’s up to you to always be cognizant whether an integer was lifted into $P(\mathbb{Z})$ using Form 1 or using Form 2. Because of this possibility for confusion, it is often more convenient to use the set of character strings as $X$ to give rise to the ring of polynomials in infinitely-many variables, and you get the added benefit of being able to name variables. Using the empty set for $X$ gives a ring that is indistinguishable (from the point of view of ring theory) from $\mathbb{Z}$.

type P_1 = PExpr () -- polynomials in one variable
type P_2 = PExpr (Either () ()) -- polynomials in two variables
type P_inf = PExpr Integer -- polynomials in infinitely-many unfortunately-named variables
type P_named = PExpr String -- polynomials in infinitely-many nicely-named variables
type Z = PExpr Void -- polynomials, but with no variables (a.k.a, the integers)

In computery terms, the polynomials in $X$ are the “ring expressions”, with variables denoted by elements of $X$. They are the abstract syntax trees of (commutative) ring theory. That is, the polynomials are the expressions that “make sense” and can be interpreted in any ring $A$, once you throw in a dictionary that assigns an element of $A$ to each element of $X$. This framing suggests a deep fact that we’ll discuss next.

Given a set $X$ and a ring $A$, we may define the evaluation map of $X$ to $A$ that takes a dictionary from $X$ to $A$ and a polynomial in $X$ and determines a value of $A$ by first substituting values of $A$ for elements of $X$ as they appear in the polynomial and then reducing the polynomial according to the ring operations of $A$ (treating juxtaposition as $\cdot$ and treating the formal $+$ exactly as the notation suggests).

evalPExpr :: Ring a => (x -> a) -> PExpr x -> a
evalPExpr dict (Form3 n xs) = foldl (\a x -> a * dict x) (fromInteger n) xs
evalPExpr dict (Form4 p1 p2) = evalPExpr dict p1 + evalPExpr dict p2

Now, remember that at its core, a function from $X$ to $A$ is simply a selection of $X$-many elements of $A$. For example, defining a function from a one-element set to the set of integers is equivalent to merely picking an integer. Defining a function from a two-element set to the set of character strings is equivalent to selecting two particular character strings (in a particular order). In this sense, we can think of a function $X$ to $A$ as a substitution assignment of values in $A$ to the indeterminates of $P(X)$. Thus, an element of $P(X)$ is interpreted as a function on $A$ with $X$-many variables (recovering fully the classical notion of a polynomial as being one of those familiar functions from high-school Math). Thus, the polynomials are the functions that can be defined wholly in terms of the rings operations, and thus may be applied in any ring. This fact, encoded above as evalPExpr, is the essential property of polynomials from which all of their other properties derive.

It took us quite a lot of work to get here, between the definition of the datatype PExpr and its Ring instance, not to mention the definition of evalPExpr. It would be nice if there were an easier way to reach the light at the end of the tunnel. Notice what happens when we rearrange the arguments and constraint on evalPExpr, though.

evalPExpr :: PExpr x -> (forall a. Ring a => (x -> a) -> a)

My hope is that this reminds you of a comment I made above. To paraphrase and summarize the above discussion:

The polynomials with indeterminates in $X$ are exactly the functions of $X$-many variables that make sense in any ring.

Now look at the signature. The (x -> a) -> a is a function on a with x-many variables, where the x -> a argument is thought of as a dictionary selecting x-many elements of a to substitute into the function. The forall a. Ring a => indicates that this function makes sense in any Ring. The type forall a. Ring a => (x -> a) -> a) is in fact the ring of polynomials generated by x in disguise.

-- | The polynomials in @x@.
newtype Polynomial x = P (forall a. Ring a => (x -> a) -> a)

Lifting $X$ values into $P(X)$ is merely function application.

var :: x -> Polynomial x
var x = P (\dict -> dict x)

The evaluation map, the map that promotes an arbitrary $X$ to $A$ function into a $P(X)$ to $A$ function, is also just application.

evalMap :: Ring a => (x -> a) -> Polynomial x -> a
evalMap dict (P f) = f dict

And the ring structure on Polynomial x (i.e., the Ring instance) is the familiar Ring instance for functions.

instance Ring (Polynomial x) where
  P f + P g = P (f + g)
  P f * P g = P (f * g)
  negate (P f) = P (negate f)
  fromInteger n = P (fromInteger n)

My claim is that the ring Polynomial x is indistinguishable (from the point of view or ring theory) from the ring PExpr x. To convince you of this, I need to define a function from Polynomial x to PExpr x that (1) is invertible and that (2) distributes over the ring operations. These two facts together would mean that any ring-theoretic statement of fact you could make about PExpr x would have a corresponding statement of fact about Polynomial x, and vice versa.

Here’s that function.

toExpr :: Polynomial x -> PExpr x
toExpr = evalMap liftVar

Doing the easy job first, we’ll establish that toExpr distributes over the ring operations.

toExpr (P f + P g)
  = toExpr (P (f + g)) -- def. of `+` for `Polynomial x`
  = (f + g) liftVar -- def. of `toExpr` and `evalMap`
  = f liftVar + g liftVar -- def. of `+` for functions
  = toExpr (P f) + toExpr (P g) -- def. of `evalMap` and `toExpr`

toExpr (P f * P g)
  = toExpr (P (f * g)) -- def. of `*` for `Polynomial x`
  = (f * g) liftVar -- def. of `toExpr` and `evalMap`
  = f liftVar * g liftVar -- def. of `*` for functions
  = toExpr (P f) * toExpr (P g) -- def. of `evalMap` and `toExpr`

toExpr (negate (P f))
  = toExpr (P (negate f)) -- def. of negate for `Polynomial x`
  = (negate f) liftVar -- def. of `toExpr` and `evalMap`
  = (\dict -> negate (f dict)) liftVar -- def. of `negate` for functions
  = negate (f liftVar) -- beta reduction
  = negate (toExpr (P f)) -- def. of `evalMap` and `toExpr`

toExpr (fromInteger n)
  = toExpr (P (fromInteger n)) -- def. of `fromInteger` for `Polynomial x`
  = (fromInteger n) liftVar -- def. of `toExpr` and `evalMap`
  = (\dict -> fromInteger n) liftVar -- def. of `fromInteger` for functions
  = fromInteger n -- beta reduction

Now, we need to see that toExpr is invertible. To do that, we’ll define a function going the other way, and then show that they cancel each other.

fromExpr :: PExpr x -> Polynomial x
fromExpr = evalPExpr var

Using the definitions of the Ring operations defined in the instance for PExpr x, we’ll see that toExpr . fromExpr = id.

toExpr (fromExpr (Form3 n [x_1, ..., x_k]))
  = toExpr (evalPExpr var (Form3 n [x_1, ..., x_k]))
  = toExpr (foldl (\a x -> a * var x) (fromInteger n) [x_1, ..., x_k])
  = toExpr (fromInteger n * var x_1 * ... * var x_k
  = toExpr (
      P (\dict -> fromInteger n)
    * P (\dict -> dict x_1)
    * ...
    * P (\dict -> dict x_k))
  = toExpr (P (\dict -> fromInteger n * dict x_1 * ... * dict x_k))
  = (\dict -> fromInteger n * dict x_1 * ... * dict x_k) liftVar
  = fromInteger n * liftVar x_1 * ... * liftVar x_k
  = Form3 n [] * Form3 1 [x_1] * ... * Form3 1 [x_k]
  = Form3 n [x_1, ..., x_k]

So toExpr . fromExpr fixes Form3 expressions. Likewise for Form4 expressions. For this, we use induction. Suppose that expr1 and expr2 are in PExpr x and that (toExpr . fromExpr) expr1 = expr1 and (toExpr . fromExpr) expr2 = expr2.

toExpr (fromExpr (Form4 expr1 expr2))
  = toExpr (evalPExpr var (Form4 expr1 expr2))
  = toExpr (evalPExpr var expr1 + evalPExpr var expr2)
  = toExpr (fromExpr expr1 + fromExpr expr2)
  = toExpr (fromExpr expr1) + toExpr (fromExpr expr2)
  = expr1 + expr2
  = Form4 expr1 expr2

We’ve left to show that fromExpr . toExpr = id. This is highly technical and only true if we assume termination. We would proceed using induction on the set of all functions of the form Ring a => (x -> a) -> a. We know that any (terminating) function of that form can only apply the dictionary to inhabitants of x and use the Ring methods, so we’d have base cases \dict -> fromInteger n and \dict -> dict x, and we have induction cases \dict -> f dict + g dict, \dict -> f dict * g dict, and \dict -> negate (f dict). We’d show the base cases all satisfy (fromExpr . toExpr) p = p, and we’d show that each induction case satisfies (fromExpr . toExpr) p = p when we assume the hypotheses (fromExpr . toExpr) f = f and (fromExpr . toExpr) g = g. We’ll spare the details, but assuming this work is done, we will have accomplished our goal of showing that Polynomial x and PExpr x are indistinguishable from the point of view of ring theory.

x :: Polynomial String
x = var "x"

y :: Polynomial String
y = var "y"

constant :: Integer -> Polynomial x
constant = fromInteger

ellipticCurve :: Integer -> Integer -> Polynomial String
ellipticCurve (constant -> a) (constant -> b) = y^2 - x^3 - a*x - b

Now some philosophy. The Ring instance for the function type X -> A defines the ring operations for functions in terms of the operations on the type A. In a sense, X -> A inherits its ring structure (its Ring instance) from the ring structure of A. An element of Polynomial x, that is, a function Ring a => (x -> a) -> a, is polymorphic in a. Any ring may be slotted in for a, and the polynomial can be evaluated at that type. Yet Ring a => (x -> a) -> a is a function type; it inherits its ring structure from the target ring a, which–again–can be any ring. In a sense, Polynomial x inherits its ring structure from every ring.

Recall that we earlier said that the ring of integers is the most- prototypical ring. Every other ring contains a projection of the ring of integers, witnessed by its fromInteger implementation. The polynomial rings, inheriting their ring structure from every ring, are the counterpoint to the integers’ maximal prototypicality. By inheriting the ring structure of every ring, the polynomial rings are the most general rings.

There is a way to make this notion of maximal generality mathematically precise. Direct your attention to the signature of evalMap.

evalMap :: Ring a => (x -> a) -> Polynomial x -> a

I’m very deliberate about the choice of name here. It should remind you of a common Haskell function you’re hopefully familiar with.

foldMap :: Monoid a => (x -> a) -> [x] -> a

Suspending legitimate concerns about termination and infinite lists, Haskell’s [] functor is commonly thought of as the free monoid construction. What does that mean? It means that [] is a factory that produces free monoids. Explicitly, it means that the type [()] is the free monoid on one generator (a.k.a. “in one variable”), the type [Either () ()] is the free monoid on two generators (a.k.a. “in two variables”). The type [x] is the free monoid generated by x (i.e., monoid expressions with indeterminates in x). Elements of the free monoid generated by x are abstract monoid-syntax trees, using the elements of x as variables. foldMap is the function that allows us to interpret the abstract syntax, promoting a dictionary of assignments x -> a to a function[x] -> a. This explanation of what it means to be a free monoid should sound familiar to you, because it’s analogous to the discussion we had earlier about polynomials, whereby our intuitive notion that the polynomials Polynomial x are the x-variable functions that make sense in any ring is made precise. The polynomial ring Polynomial x is the free (commutative) ring generated by x, in the same sense that the list type [x] is the free monoid generated by x.

Summary:

Types that are instances of Haskell’s Num class are analogous to (commutative) rings.
The ring of integers is the most-prototypical ring. Every ring contains an image of the integers.
If $A$ is a ring and if $X$ is any set, then the set of functions $X \to A$ is a ring in a non-arbitrary way, in that the ring operations on functions are defined in terms of the ring operations on $A$. $X \to A$ inherits its ring structure from $A$.
A polynomial with indeterminates in $X$ is an $X$-variable function that makes sense in any ring. In Haskell terms, that is precisely a polymorphic function with a Ring (or Num) constraint. Equivalently, we can think of polynomials as ring expressions, abstract syntax built from the ring operations.
The polynomials in $X$ is a ring in a non-arbitrary way, because the polynomials are functions into a ring (really, functions into any ring), and functions inherit the ring structure of their target ring.
Counterpoint to the role the integers play as the most-prototypical ring, The polynomial rings are the most-general rings. The polynomials are functions into any target ring, so the ring of polynomials inherit the ring structure of every ring. Made precise, this is the statement that the polynomial rings are the free (commutative) rings.

(-) :: Ring a => a -> a -> a
x - y = x + negate y

(^) :: Ring a => a -> Integer -> a
x ^ 0 = 1
x ^ n = x * (x ^ (n - 1))

infixr 8 ^
infixl 7 *
infixl 6 +
infixl 6 -

Proving My Point

Sat, 21 Nov 2020 00:00:00 +0000

A coworker of mine has been kind enough to spend his scant spare time guiding a small group of future Haskellers through Hutton’s Programming in Haskell, assigning weekly problems and judging the solutions submitted along the way. What follows here today¹ is my smart-ass solution to my colleague’s Problem 1.

Executing Programs on Paper

Week 1 of the reading group was to read Chapter 1 in Hutton and complete a few simple problems based on the reading. Chapter 1 is mostly an inspirational overview, but it does show how one can write simple, recursive functions on lists and how one can evaluate such functions on paper. For example, Hutton defines sum as follows.

sum [] = 0
sum (x : xs) = x + sum xs

He then shows how one may evaluate sum [1, 2, 3] on paper.

\[\begin{align*} \texttt{sum}\,[1, 2, 3] &= \texttt{sum}\,(1 : 2 : 3 : []) \\ &= 1 + \texttt{sum}\,(2 : 3 : []) \\ &= 1 + (2 + \texttt{sum}\,(3 : [])) \\ &= 1 + (2 + (3 + \texttt{sum}\,[])) \\ &= 1 + (2 + (3 + 0)) \\ &= 1 + (2 + 3) \\ &= 1 + 5 \\ &= 6 \end{align*}\]

Week 1 Problem 1

Using the above example as a guide, said group leader assigned what would be the first problem for week 1.

Problem 1: Prove that for all x, sum [x] is x.

This is a great problem, and I’m sure the intended solution was to evaluate the program on paper for a general value $x$. By that, I mean we assume that $x$ is some fixed value and we proceed to evaluate $\texttt{sum}\,[x]$ on paper as though we knew what value $x$ were.

\[\begin{align*} \texttt{sum}\,[x] &= \texttt{sum}\,(x : []) \\ &= x + \texttt{sum}\,[] \\ &= x + 0 \\ &= x \end{align*}\]

Since our answer did not depend on any specific characteristics of $x$, the reasoning is valid no matter what value $x$ were, and so we know that this same conclusion can be drawn about any such value. That is, we have shown that for all x, sum [x] is x. (Notice that the above evaluation could not have been carried out to conclusion had the formula for sum directed us to pattern match $x$. Also, we must assume a lawful Num instance, as we tacitly use $x + 0 = x$.)

Proofs that Compile

The above is certainly a fine proof, and—I stress again—surely the intended solution to Problem 1.

But I was bored. And I wanted to be extra, super-duper sure of the veracity of the claim.

To this very end, I wrote a little program in Haskell. And as we know about Haskell, if it compiles, it works.

module Chapter1 where

import Prelude hiding (sum)

data Nil
data Cons x xs

We’re going to prove Problem 1, but for type-level lists, using classes as predicates on types. Our type-level list is implemented using Cons “cells” terminating with a Nil. For example, the type level list consisting of Int, Double, and String (in that order) is quite simply Cons Int (Cons Double (Cons String Nil)). This type has no inhabitants, but that’s fine—we won’t be needing any.

{-# LANGUAGE FunctionalDependencies #-}

...

import Data.Proxy (Proxy(..))

...

class Sum list x | list -> x where
  sum :: Proxy list -> Proxy x
  sum _ = Proxy

We’ve added a class to represent the predicate The type x is the sum of the type list. That is, x is the sum of list if and only if the compiler can find an instance Sum list x. The syntax | list -> x means that attempting to create two instances Sum list x1 and Sum list x2 is a compiler error. We want this, because the sum of list (if a sum exists) should be uniquely and unambiguously determined by list. The FunctionalDependencies language extension enables this syntax and the associated check. The extension also aids type inference, since—in light of the restriction imposed by | list -> x—the type x can now be inferred from the type list.

The class method sum is notable for both its signature and its implementation. Its signature Proxy list -> Proxy x tells us that sum gives us a value-level way to find the type x that is the sum of the type list.

To see how, we need to talk about Proxy. Proxy is defined as data Proxy a = Proxy. So, Proxy Int has the one inhabitant Proxy :: Proxy Int. And Proxy Bool has the one inhabitant Proxy :: Proxy Bool. And Proxy [IO String] has the one inhabitant Proxy :: Proxy [IO String]. In fact, no matter what type is used for the parameter, we have just one inhabitant, Proxy, that carries no data. What gives?

Well, the interesting thing about Proxy isn’t the value, it’s the type parameter. In code, the data constructor Proxy is often given an explicit type annotation as a way to pass types as arguments into Haskell functions. This is exactly how we’re going to use it with our sum function.

If we have some type list in mind and we want to know its sum, all we have to do is ask the compiler to inferred type of sum (Proxy :: Proxy list). The compiler will infer a type Proxy x if it can find an instance Sum list x, from which we see that type x is the sum of type list, or else it will reject the program if it fails to find an instance. This is how we get the compiler to perform type-level computations for us (such as summing lists) at compile time.

The implementation sum _ = Proxy works because the compiler doesn’t need to inspect the argument (it has no data anyway) as it already knows the type of the argument, and from that it can infer the type of the result Proxy value.

Moving on, we still have to create instances of class Sum. To see this viscerally, let’s try to get the compiler to infer the type of sum (Proxy :: Proxy (Cons 1 (Cons 2 (Cons 3 Nil)))).

{-# LANGUAGE PolyKinds #-} -- gives us, among other things, type-level numbers

...


sum123 :: _ -- the underscore here asks GHC to infer this type
sum123 = sum (Proxy :: Proxy (Cons 1 (Cons 2 (Cons 3 Nil))))

We get a compiler error:

No instance for (Sum (Cons 1 (Cons 2 (Cons 3 Nil))) x0)
  arising from a use of ‘sum’

Although we humans know the type of sum123 should be Proxy 6, the compiler error is actually quite reasonable: we haven’t written any instances of class Sum yet!

The instances are where we teach the compiler how to compute sums. In other words, instances are the actual programming in this flavor of type-level programming.

{-# LANGUAGE FlexibleInstances #-} -- probably should be enabled by default
{-# LANGUAGE TypeFamilies, TypeOperators #-} -- lets us add type-level numbers
{-# LANGUAGE UndecidableInstances #-} -- lets the compiler solve hard class constraints

...

instance Sum Nil 0
instance (Sum rest n2, n ~ (n1 + n2)) => Sum (Cons n1 rest) n

-- sum123 :: _
-- sum123 = sum (Proxy :: Proxy (Cons 1 (Cons 2 (Cons 3 Nil))))

Commenting out sum123 for suspense, we’ve ported to the type system the standard recursive definition of sum that we encountered at the start of this post. We have two instances, a base case and a recursive case. Our base case instance Sum Nil 0 tells the compiler that the sum of an empty list is 0. Our recursive case instance (Sum rest n2, n ~ (n1 + n2)) => Sum (Cons n1 rest) n tells the compiler that the sum of Cons n1 rest is n, provided that the sum of rest is n2 and that n equals n1 + n2.

...

sum123 :: _
sum123 = sum (Proxy :: Proxy (Cons 1 (Cons 2 (Cons 3 Nil))))

Found type wildcard ‘\_’ standing for ‘Proxy 6’

Uncommenting sum123, we see the compiler now infers the correct type, namely Proxy 6. That is, the sum of the type Cons 1 (Cons 2 (Cons 3 Nil)) is the type 6.

Let’s fill the underscore with Proxy 6 in our code and press on to the main event.

{-# LANGUAGE ExplicitForAll #-}

...

sum123 :: Proxy 6
sum123 = sum (Proxy :: Proxy (Cons 1 (Cons 2 (Cons 3 Nil))))

problem1 :: forall (n :: Nat). Proxy (Cons n Nil) -> Proxy n
problem1 = sum

And, would you look at that! problem1 says that sum has the type forall (n :: Nat). Proxy (Cons n Nil) -> Proxy n, which is what we wanted to show. It compiles, so it works ;-)

Pulling Back on the Reins

If you found this post confusing and maybe even a little scary, don’t worry. Programming in Haskell is really nothing like this. For that reason, I won’t go out of my way to share the above “solution” with the reading group. I did it simply because I, personally, found it to be an enjoyable puzzle on which to pass an hour.

If, on the other hand, you did enjoy this post, you might want to take a look at this lovely story called Typing the Technical Interview, by Aphyr, from which I learned this style of programming. And it’s a riot, to boot. You might also enjoy the excellent talk 4 Programming Paradigms in 45 Minutes by Aja Thagomizer Hammerly. Finally, we provide the above program (and some bonus features) conveniently as a single file.

Thanks for reading!

I know I owe you a retro on the linear regression post, including a clever observations made by a user on Twitter.com and a constant-space solution. (My solution uses memory proportional to the size of the input—yuck!) I haven’t forgotten. ↩

Simple Linear Regression in One Pass

Fri, 12 Jun 2020 00:00:00 +0000

I recently had to implement linear regression for a hobby project. As I’d never learned how to calculate it, this was a great opportunity to learn. As I dug in, I saw that I’d need to be making two passes over my data. Let’s see if we can fix that.

The Problem

Simple linear regression is where we take a set of points in the plane and compute the slope and $y$-intercept of the line that most-closely fits the points.

Precisely, if our data is $ \left\{ (x_i, y_i) \vert i \in I \right\} $, then we are looking for the pair $m$, $b$ which minimize the cost

\[ \sum_{i \in I} \left( m x_i + b - y_i \right)^2 \text{,} \]

where $m$ is the slope of our model, $b$ is the $y$-intercept of our model, and $y = mx + b$ is the value our model predicts for arbitrary input $x$.

There are many approaches to this problem, ranging from multi-variable Calculus to geometry to linear algebra. We will use a statistical approach.

(Stating without proof) We have the following:

\[ m = r \frac{s_y}{s_x} \]

and

\[ b = \bar{y} - m \bar{x} \]

where

$\bar{x}$, $\bar{y}$ are the means of $x_i$, $y_i$,
$s_x$, $s_y$ are the standard deviations of $x_i$, $y_i$, and
$r$ is Pearson’s correlation coefficient, defined as

\[ r = \frac{ \sum_i \left( (x_i - \bar{x}) (y_i - \bar{y}) \right) }{ \sqrt{ \sum_i (x_i - \bar{x})^2 \sum_i (y_i - \bar{y})^2 } }\text{.} \]

(This useful video walks you through an example by hand.)

Recall the formula formula for standard deviation $s_x$ (and analogous for $s_y$),

\[ s_x = \sqrt{\frac{\sum_i (x_i - \bar{x})^2}{n - 1}} \text{,} \]

where $n$ is the size of the sample $I$.

First, let’s simplify the formula for $m$.

\[ m = r \frac{s_y}{s_x} = r \frac{ \sqrt{\frac{\sum_i (y_i - \bar{y})^2}{n - 1}} }{ \sqrt{\frac{\sum_i (x_i - \bar{x})^2}{n - 1}} } = r \sqrt{\frac{\sum_i (y_i - \bar{y})^2}{\sum_i (x_i - \bar{x})^2}} \]

\[ = \frac{ \sum_i \left( (x_i - \bar{x}) (y_i - \bar{y}) \right) }{ \sqrt{ \sum_i (x_i - \bar{x})^2 \sum_i (y_i - \bar{y})^2 } } \sqrt{\frac{\sum_i (y_i - \bar{y})^2}{\sum_i (x_i - \bar{x})^2}} \]

\[ = \sum_i \left( (x_i - \bar{x}) (y_i - \bar{y}) \right) \sqrt{\frac{ \sum_i (y_i - \bar{y})^2 }{ \sum_i (x_i - \bar{x})^2 \sum_i (y_i - \bar{y})^2 \sum_i (x_i - \bar{x})^2 }} \]

\[ = \sum_i \left( (x_i - \bar{x}) (y_i - \bar{y}) \right) \sqrt{\frac{1}{ \left( \sum_i (x_i - \bar{x})^2 \right)^2 }} \]

\[ = \frac{ \sum_i \left( (x_i - \bar{x}) (y_i - \bar{y}) \right) }{ \sum_i (x_i - \bar{x})^2 } \]

This final form is much nicer. To work with it, we no longer need to know the standard deviations $s_x$, $s_y$, though we still need to know the means $\bar{x}$, $\bar{y}$.

Two-pass Implementation

The formulas for $m$ and $b$ above give us a straight-forward way to implement simple linear regression:

Find avg_x and avg_y,
Use avg_x and avg_y in the above formula for slope $m$, and
Use avg_x, avg_y, and slope in the above formula for intercept $b$.

simpleLinearRegression :: Fractional a => [(a, a)] -> (a, a)
simpleLinearRegression points =
    (slope, intercept)
    where
    avg_x = sum [x | (x, _) <- points] / fromIntegral (length points)
    avg_y = sum [y | (_, y) <- points] / fromIntegral (length points)

    xys = sum [(x - avg_x) * (y - avg_y) | (x, y) <- points]
    xxs = sum [(x - avg_x) * (x - avg_x) | (x, _) <- points]

    slope = xys / xxs
    intercept = avg_y - slope * avg_x

This works, but it makes six passes over the list points. If this list is very long, then each pass will take a non-trivial amount of time. We can refactor this using monoids and foldMap to reduce the number of list passes we need to make.

class Semigroup a => Monoid a where
    (<>) :: a -> a -> a -- from `Semigroup`
    mempty :: a

A type a is a monoid if it has an associative binary operation <> with an identity element mempty.

foldMap :: Monoid a => (x -> a) -> [x] -> a -- specialized to lists

foldMap takes a list [x] and a mapping function x -> a and, if a is a monoid, applies the function to the list elements and combines the results using <>. Concretely, foldMap f [x1, x2, x3] == f x1 <> f x2 <> f x3. foldMap of an empty list gives mempty.

We’ll use the monoids Sum a and (a, b) in our refactor.

-- Import from `Data.Monoid`
newtype Sum a = Sum a

instance Num a => Monoid (Sum a) where
    Sum x <> Sum y = Sum (x + y)
    mempty = Sum 0

instance (Monoid a, Monoid b) => Monoid (a, b) where
    (a1, b1) <> (a2, b2) = (a1 <> a2, b1 <> b2)
    mempty = (mempty, mempty)

-- ... similar instances for 3-tuples, 4-tuples, etc ...

Armed with these components, let’s take a closer look at calculating avg_x and avg_y. We’re currently passing over the list four times to calculate them.

avg_x = sum [x | (x, _) <- points] / fromIntegral (length points)
avg_y = sum [y | (_, y) <- points] / fromIntegral (length points)

We can get this down to three passes if we share the length.

n = fromIntegral (length points)
avg_x = sum [x | (x, _) <- points] / n
avg_y = sum [y | (_, y) <- points] / n

Next, we can use the Sum a and (a, b) monoids to get both sums in one pass.

n = fromIntegral (length points)
(Sum xs, Sum ys) = foldMap (\(x, y) -> (Sum x, Sum y)) points
avg_x = xs / n
avg_y = ys / n

We’re down to two passes, but we can get the length at the same time we get the sums with this one neat trick: count the number of list elements by packing a Sum 1 into the result of our mapping function.

(Sum n, Sum xs, Sum ys) = foldMap (\(x, y) -> (Sum 1, Sum x, Sum y)) points
avg_x = xs / n
avg_y = ys / n

This is a bit tricky, so let’s look at an example to see how it works:

foldMap (\(x, y) -> (Sum 1, Sum x, Sum y)) [(x1, y1), (x2, y2), (x3, y3)]
    == (Sum 1, Sum x1, Sum y1)
        <> (Sum 1, Sum x2, Sum y2)
            <> (Sum 1, Sum x3, Sum y3)
    == (Sum 1 <> Sum 1 <> Sum 1,
        Sum x1 <> Sum x2 <> Sum x3,
            Sum y1 <> Sum y2 <> Sum y3)
    == (Sum 3, Sum (x1 + x2 + x3), Sum (y1 + y2 + y3))

So we’ve gotten both averages in just one pass! Let’s refactor the rest of the program.

simpleLinearRegression :: Fractional a => [(a, a)] -> (a, a)
simpleLinearRegression points =
    (slope, intercept)
    where
    avg (x, y) = (Sum 1, Sum x, Sum y)

    (Sum n, Sum xs, Sum ys) = foldMap avg points
    avg_x = xs / n
    avg_y = ys / n

    reg (x, y) =
        (Sum $ x' * y', Sum $ x' * x')
        where
        x' = x - avg_x
        y' = y - avg_y

    (Sum xys, Sum xxs) = foldMap reg points
    slope = xys / xxs
    intercept = avg_y - slope * avg_x

We make two passes over our list, one using the mapping function avg and one using the mapping function reg. This is already a huge improvement over our first draft.

Monoid-valued Functions

At first glance, it seems like two passes is the best we can hope for, since reg needs to use of the results avg_x and avg_y of the first pass. Fortunately, as programmers we have a one-size-fits-all tool for abstracting away information that we don’t yet have: functions. We can make reg a function that gets avg_x and avg_y as inputs.

reg (x, y) (avg_x, avg_y) =
    (Sum $ x' * y', Sum $ x' * x')
    where
    x' = x - avg_x
    y' = y - avg_y

I assert that this solves all of our problems, but the astute reader might now notice a subtlety. reg has the special requirement that it needs to map our list elements into a monoid. In its original form we have reg :: (a, a) -> (Sum a, Sum a), which does indeed map a list element (a, a) into the monoid (Sum a, Sum a). Our new version of reg, though, has a different signature. We now have reg :: (a, a) -> ( (a, a) -> (Sum a, Sum a) ), which maps a list element (a, a) into the function type (a, a) -> (Sum a, Sum a). It turns out that the type (a, a) -> (Sum a, Sum a) is a perfectly reasonable monoid already.

instance Monoid a => Monoid (x -> a) where
    (f <> g) x = f x <> g x
    mempty = const mempty

The instance above says that any function type where the target type a is a monoid is itself a monoid in a formulaic way and with no conditions on the source type x. In other words, monoid-valued functions always form a monoid. We just need to make sure that our definition of <> is associative and that const empty is an identity for <>.

(f <> (g <> h)) x
    == f x <> (g <> h) x -- definition of `<>` on type `x -> a`
    == f x <> (g x <> h x) -- definition of `<>` on type `x -> a`
    == (f x <> g x) <> h x -- by associativity of `<>` on type `a`
    == ((f <> g) x) <> h x -- definition of `<>` on type `x -> a`
    == ((f <> g) <> h) x -- definition of `<>` on type `x -> a`

(f <> const mempty) x
    == f x <> const mempty x -- definition of `<>` on type `x -> a`
    == f x <> mempty -- definition of `const`
    == f x -- by identity of `mempty` on type `a`

(const mempty <> f) x
    == const mempty x <> f x -- definition of `<>` on type `x -> a`
    == mempty <> f x -- definition of `const`
    == f x -- by identity of `mempty` on type `a`

The three stanzas above demonstrate that f <> (g <> h) is (f <> g) <> h, f <> const mempty is f, and const mempty <> f is f. Thus, x -> a is a valid monoid.

One-pass Implementation

Armed with this new method of building up monoids, we refactor our program to make just one pass over points.

simpleLinearRegression :: Fractional a => [(a, a)] -> (a, a)
simpleLinearRegression points =
    (slope, intercept)
    where
    avg (x, y) = (Sum 1, Sum x, Sum y)

    reg (x, y) (avg_x, avg_y) =
        (Sum $ x' * y', Sum $ x' * x')
        where
        x' = x - avg_x
        y' = y - avg_y

    ((Sum n, Sum xs, Sum ys), getReg) = foldMap (\p -> (avg p, reg p)) points

    avg_x = xs / n
    avg_y = ys / n
    (Sum xys, Sum xxs) = getReg (avg_x, avg_y)
    slope = xys / xxs
    intercept = avg_y - slope * avg_x

Analysis

Using the ideas above, I wrote three implementations of simple linear regression and compared their performance on a sample data set. I compiled each implementation to a standalone Macos binary using GHC 8.8.3 and the -O2 optimization flag.

The data set (provided by Kaggle) features ocean temperatures vs. salinity and consists of 864,862 data points, which I then quadrupled to 3,459,448 data points using Unix cat. I ran each implementation four times, using Unix time to record how long each run took.

Notably, I’m ignoring memory in this analysis, though memory complexity would be an important question to answer in a more comprehensive study. Also, for reasons unknown to me, the second implementations gives a slightly different result than the first and third. I should probably look into why that is.

Naive Implementation

We directly port the math over to Haskell, ignoring performance concerns such as multiple list passes and common subexpressions. This one tends to finish in about 15 seconds.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
-- naive-linear-regression.hs
import Data.Maybe (mapMaybe)

main :: IO ()
main =
    print . simpleLinearRegression . parse =<< getContents

readM :: Read a => String -> Maybe a
readM str =
    case reads str of
        [(x, "")] -> Just x
        _ -> Nothing

parse :: String -> [(Double, Double)]
parse =
    mapMaybe parseLine . lines . filter (/= '\r')
    where
    parseLine raw =
        let (x, y) = break (== ',') raw
        in (,) <$> readM x <*> readM (tail y)

simpleLinearRegression :: Fractional a => [(a, a)] -> (a, a)
simpleLinearRegression points =
    (slope, intercept)
    where
    avg_x = sum [x | (x, _) <- points] / fromIntegral (length points)
    avg_y = sum [y | (_, y) <- points] / fromIntegral (length points)

    xys = sum [(x - avg_x) * (y - avg_y) | (x, y) <- points]
    xxs = sum [(x - avg_x) * (x - avg_x) | (x, _) <- points]

    slope = xys / xxs
    intercept = avg_y - slope * avg_x

$ time ./naive-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.520773058533541e-2,34.4409093436518)
real  0m14.858s
user  0m14.063s
sys 0m0.535s

$ time ./naive-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.520773058533541e-2,34.4409093436518)
real  0m15.055s
user  0m14.489s
sys 0m0.540s

$ time ./naive-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.520773058533541e-2,34.4409093436518)
real  0m15.084s
user  0m14.521s
sys 0m0.542s

$ time ./naive-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.520773058533541e-2,34.4409093436518)
real  0m14.882s
user  0m14.323s
sys 0m0.539s

One-pass Implementation

We use built-in Monoid instances to condense our calculations into one list pass. Surprisingly, this one takes longer than the naive implementation, tending to finish in about 22 seconds.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
-- one-pass-linear-regression.hs
import Data.Maybe (mapMaybe)
import Data.Monoid (Sum(Sum))

main :: IO ()
main =
    print . simpleLinearRegression . parse =<< getContents

readM :: Read a => String -> Maybe a
readM str =
    case reads str of
        [(x, "")] -> Just x
        _ -> Nothing

parse :: String -> [(Double, Double)]
parse =
    mapMaybe parseLine . lines . filter (/= '\r')
    where
    parseLine raw =
        let (x, y) = break (== ',') raw
        in (,) <$> readM x <*> readM (tail y)

simpleLinearRegression :: Fractional a => [(a, a)] -> (a, a)
simpleLinearRegression points =
    (slope, intercept)
    where
    avg (x, y) = (Sum 1, Sum x, Sum y)

    reg (x, y) (avg_x, avg_y) =
        (Sum $ x' * y', Sum $ x' * x')
        where
        x' = x - avg_x
        y' = y - avg_y

    ((Sum n, Sum xs, Sum ys), getReg) =
        foldMap (\p -> (avg p, reg p)) points

    avg_x = xs / n
    avg_y = ys / n
    (Sum xys, Sum xxs) = getReg (avg_x, avg_y)
    slope = xys / xxs
    intercept = avg_y - slope * avg_x

$ time ./one-pass-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.5207730585339065e-2,34.44090934365883)
real  0m22.002s
user  0m20.592s
sys 0m1.190s

$ time ./one-pass-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.5207730585339065e-2,34.44090934365883)
real  0m21.829s
user  0m20.619s
sys 0m1.181s

$ time ./one-pass-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.5207730585339065e-2,34.44090934365883)
real  0m22.520s
user  0m21.262s
sys 0m1.212s

$ time ./one-pass-linear-regression < 2190_3685_bundle_archive/quad.csv
(-5.5207730585339065e-2,34.44090934365883)
real  0m21.883s
user  0m20.666s
sys 0m1.179s

One-pass Unboxed Implementation

Not content to have the naive implementation win, I still had a few tricks up my sleeve to squeeze more calculations out of each CPU cycle. For one, since we know we’re consuming our whole input list, we can switch from foldMap to the strict foldMap'. For another, the one-pass implementation achieves a satisfying level of code reuse through polymorphism and composition of tuples, but those extra pointers add up. We can gain some performance by monomorphising and unboxing all of our data, at the cost of implementing Semigroup and Monoid instances by hand. This one tends to finish in under 12 seconds.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
-- one-pass-strict-unboxed-linear-regression.hs
{-# LANGUAGE BangPatterns #-}

import Data.Foldable (foldMap')
import Data.Maybe (mapMaybe)

main :: IO ()
main =
    print . simpleLinearRegression . parse =<< getContents

readM :: Read a => String -> Maybe a
readM str =
    case reads str of
        [(x, "")] -> Just x
        _ -> Nothing

parse :: String -> [Pair]
parse =
    mapMaybe parseLine . lines . filter (/= '\r')
    where
    parseLine raw =
        let (x, y) = break (== ',') raw
        in Pair <$> readM x <*> readM (tail y)

data Pair =
    Pair {-# UNPACK #-} !Double {-# UNPACK #-} !Double
    deriving Show

instance Semigroup Pair where
    Pair x1 y1 <> Pair x2 y2 = Pair (x1 + x2) (y1 + y2)

instance Monoid Pair where
    mempty = Pair 0 0

data Stat =
    Stat
        {-# UNPACK #-} !Double
        {-# UNPACK #-} !Double
        {-# UNPACK #-} !Double
        !(Pair -> Pair)

instance Semigroup Stat where
    Stat n1 x1 y1 f1 <> Stat n2 x2 y2 f2 =
        Stat (n1 + n2) (x1 + x2) (y1 + y2) (f1 <> f2)

instance Monoid Stat where
    mempty = Stat 0 0 0 (const mempty)

simpleLinearRegression :: [Pair] -> Pair
simpleLinearRegression points =
    Pair slope intercept
    where
    avgReg !(Pair x y) =
        Stat 1 x y $ \(Pair !avg_x !avg_y) ->
            let
                x' = x - avg_x
                y' = y - avg_y
            in
                Pair (x' * y') (x' * x')

    Stat n xs ys getReg = foldMap' avgReg points

    avg_x = xs / n
    avg_y = ys / n
    Pair xys xxs = getReg (Pair avg_x avg_y)
    slope = xys / xxs
    intercept = avg_y - slope * avg_x

$ time ./one-pass-strict-unboxed-linear-regression < 2190_3685_bundle_archive/quad.csv
Pair (-5.520773058533541e-2) 34.4409093436518
real  0m11.762s
user  0m11.435s
sys 0m0.148s

$ time ./one-pass-strict-unboxed-linear-regression < 2190_3685_bundle_archive/quad.csv
Pair (-5.520773058533541e-2) 34.4409093436518
real  0m11.678s
user  0m11.530s
sys 0m0.147s

$ time ./one-pass-strict-unboxed-linear-regression < 2190_3685_bundle_archive/quad.csv
Pair (-5.520773058533541e-2) 34.4409093436518
real  0m11.680s
user  0m11.536s
sys 0m0.144s

$ time ./one-pass-strict-unboxed-linear-regression < 2190_3685_bundle_archive/quad.csv
Pair (-5.520773058533541e-2) 34.4409093436518
real  0m11.563s
user  0m11.412s
sys 0m0.147s

Conclusions

I was caught off-guard by the test results. I thought it’d go without saying that the one-pass version would trounce the naive version. Instead, the opposite was true. The naive version did better than the polymorphic one-pass version, and I found myself having to really reach to beat the naive implementation in the one-pass unboxed version. It’d be interesting to compare the Core GHC produces for these three implementations to see how many list passes the naive version really ends up doing.

I think this blog post is less a story about how awesome monoids are (They are.) and more a story about how awesome GHC is (It is.). To me, it’s amazing that GHC can take the straightforward (and, frankly, kinda sloppy) naive implementation and compile it down to efficient code. To me, this reinforces a general theme in Haskell: do the obvious, simple thing first.

Appendix

In trying to find out why the six-pass naive version did so well compared to the first one-pass version, I went ahead and implemented the six-pass naive version in python.

import sys

def main():
    print(simpleLinearRegression(read(sys.stdin)))

def read(input):
    points = []
    for line in input:
        try:
            [x, y] = line.split(",")
            points.append( (float(x), float(y)) )
        except:
            pass
    return points

def simpleLinearRegression(points):
    avg_x = sum([x for (x, _) in points]) / len(points)
    avg_y = sum([y for (_, y) in points]) / len(points)

    xys = sum([(x - avg_x) * (y - avg_y) for (x, y) in points])
    xxs = sum([(x - avg_x) * (x - avg_x) for (x, _) in points])

    slope = xys / xxs
    intercept = avg_y - slope * avg_x

    print(len(points))

    return (slope, intercept)

if __name__ == "__main__":
    main()

The six-pass python version runs in about 3 seconds.

$ time python naive.py < 2190_3685_bundle_archive/quad.csv
(-0.05520773058533541, 34.4409093436518)
real    0m3.384s
user    0m3.156s
sys 0m0.224s

$ time python naive.py < 2190_3685_bundle_archive/quad.csv
(-0.05520773058533541, 34.4409093436518)
real    0m3.375s
user    0m3.159s
sys 0m0.211s

$ time python naive.py < 2190_3685_bundle_archive/quad.csv
(-0.05520773058533541, 34.4409093436518)
real    0m3.395s
user    0m3.166s
sys 0m0.222s

$ time python naive.py < 2190_3685_bundle_archive/quad.csv
(-0.05520773058533541, 34.4409093436518)
real    0m3.352s
user    0m3.134s
sys 0m0.209s

So my new question is, why the hell is my Haskell so slow? What am I doing wrong here. Will update when I find out.

Appendix B

Profiling immediately pointed to readM as the culprit, taking 78% of the time. After a quick search for “Haskell is read slow?”, I found this SO question. The answer suggested refactoring to use bytestring instead, and that cut a big chunk of time out.

Refactored to use bytestring, the naive implementation takes just over 4 seconds, the polymorphic one-pass version takes just over 7 seconds, and the strict unboxed one-pass version takes just under 4 seconds, still slower than the naive Python implementation, which takes just under 3.5 seconds.

I think the data set I’m using (about 3.5 million list elements, about 42 megabytes) is just too small to make multiple passes a problem; however, on a significantly larger dataset, such as one that could not fit in memory, a one-pass implementation would OOM building up the closure needed to compute xys and xxs, so I’m not sure what to do in that situation. If you have an idea leave a comment, or reply on Twitter.

Appendix 3

Sat down with my math friends (virtually) and we came up with a way to do it in constant memory. Expect that, and some interesting observations made by my Twitter friends, in a follow up post.

The IO Rosetta Stone

Wed, 01 Jan 2020 00:00:00 +0000

Recently to a friend, I quipped that it’d be a good exercise towards demystifying Haskell’s IO type to write a comparable IO type in your favorite language. In this blog post, I do that for Java, Javascript, Python, and Scala.

TL;DR: Here’s the code.

Understanding Functional APIs

My favorite way of understanding a type is by looking at its constructors, combinators, and eliminators. But what do I mean by constructors, combinators, and eliminators? Let me show you by looking at a more familiar type first before we tackle IO.

data Set a

We call Set an opaque type because its authors chose to hide its implementation. Instead of letting us muck around with the internals, they give us some functions that we can use to create sets.

empty     ::                 Set a
singleton ::            a -> Set a
fromList  :: Ord a => [a] -> Set a

Notice that each of those functions returns a set, and none of them require a set in any of its inputs, so we call these functions constructors.

There are also a few functions for manipulating sets.

insert       :: Ord a =>     a -> Set a -> Set a
delete       :: Ord a =>     a -> Set a -> Set a
union        :: Ord a => Set a -> Set a -> Set a
intersection :: Ord a => Set a -> Set a -> Set a

Each function in this group takes one or more sets in its input and returns a set. These functions are thought of as combining their inputs in some way to create new sets, so we call them combinators.

Finally, we need a way to get usable information out of a set. The authors provide us with a few functions that take sets as input but don’t mention Set in their output, so we call them eliminators.

null       ::                   Set a -> Bool
member     :: Ord a =>     a -> Set a -> Bool
size       ::                   Set a ->  Int
isSubsetOf :: Ord a => Set a -> Set a -> Bool

Any functional API for a type can be understood by listing out its constructors, combinators, and eliminators, and the dreaded IO type is no different.

API of the IO Type

So, what are the constructors, combinators, and eliminators of the IO type?

data IO a

Like Set, IO is opaque. To find IO’s constructors, we notice that IO is an instance of the Monad type class, which provides the return function:

return :: a -> IO a

What are the other constructors? Well, Prelude is full of them:

putStrLn :: String ->     IO ()
getLine  ::           IO String
...

At first glance, these don’t feel like constructors, but trust me for now.

We get combinators from the Monad and Functor type classes:

fmap  :: (a -> b) ->        IO a -> IO b
(>>=) ::     IO a -> (a -> IO b) -> IO b
(>>)  ::     IO a ->        IO b -> IO b
...

These functions take one or more IOs in their input and produce an IO in their output, so they’re perfectly reasonable combinators.

One thing you might notice, though, is a curious lack of eliminators. In fact, Haskell has a single designated way of eliminating an IO value: naming it main.

main :: IO ()

Naming your IO value main and running your program is how you eliminate the IO type and get access to the useful information inside.

The crucial thing about Haskell’s IO type is that it’s referentially transparent, which means that a value of type IO a doesn’t perform any action, it only describes an action. putStrLn "Hello!" doesn’t print to the screen: it describes printing to the screen. getLine doesn’t get a line of input, it describes the action of getting a line of input. You could call this the Command Pattern if that helps (but don’t dwell on it if it doesn’t).

This is why it’s reasonable to call putStrLn and getLine constructors: they allow you to construct IO values that describe some actions, and then you can further modify those action using the combinators fmap and bind ((>>=)). As you write your program, you are describing the actions that should take place, and the only action that ultimately does take place is the one you named main.

A Simple Program

I wrote this simple program in order to have something to port over to the other languages. It uses the IO constructors, combinators, and eliminator we’ve listed above ((>>=) and (>>) are used implicitly in do notation), so we’ll have to port those to the other languages as well.

-- App.hs
appLogic :: String -> String -> String
appLogic x y = "Result: " <> show (read x + read y)

printMaybe :: Maybe String -> IO ()
printMaybe = maybe (return ()) putStrLn

prompt :: Maybe String -> (String -> Maybe String) -> IO String
prompt greet confirm = do
  printMaybe greet
  l <- getLine
  printMaybe (confirm l)
  return l

app :: IO ()
app = do
  x <- prompt
    (Just "Please input two numbers.")
    (Just . ("Got first input: " <>))
  y <- prompt
    Nothing
    (Just . ("Got second input: " <>))
  putStrLn (appLogic x y)

main :: IO ()
main = app

In each language, I port the IO type and its API in one module and I port the app in another module. This is to mimic practical usage, where you’ll want the IO type to be in its own library.

Java

First, the IO library.

// IO.java
import java.util.function.Function;
import java.util.function.Supplier;

public final class IO<A> {
  private final Supplier<A> run;

  private IO(Supplier<A> run) {
    this.run = run;
  }
// ...

Like in Haskell, Java’s IO is an opaque type. We accomplish this by making the class constructor and sole instance field private. We provide a few static factory methods as part of the public API.

// ...

  /** Constructors */

  public static IO<String> getLine = new IO<String>( () ->
    System.console().readLine() );

  public static IO<Void> putStrLn(String str) {
    return new IO<Void>( () -> {
      System.out.println(str);
      return null;
    });
  }

  public static <A> IO<A> _return(A x) {
    return new IO<A>( () -> x );
  }
// ...

Combinators are instance methods.

// ...

  /** Combinators */

  public <B> IO<B> map(Function<A, B> f) {
    return new IO<B>( () ->
      f.apply(this.unsafeRunIO()) );
  }

  public <B> IO<B> bind(Function<A, IO<B>> f) {
    return new IO<B>( () ->
      f.apply(this.unsafeRunIO()).unsafeRunIO() );
  }

  public <B> IO<B> and(IO<B> other) {
    return new IO<B>( () -> {
      this.unsafeRunIO();
      return other.unsafeRunIO();
    });
  }
// ...

A single eliminator, with an ominous name, rounds out the API.

// ...

  /** Eliminators */

  public A unsafeRunIO() {
    return run.get();
  }
}

Now, the app. Unlike the library, the app is not very pretty, but it does demonstrate a few key points.

// App.java
import java.util.function.Function;
import java.util.Optional;

public final class App {

// ...

We’re able to maintain a clean separation between core logic and the tedium of talking to the outside.

// ...

  private static String appLogic(String x, String y) {
    return "Result: " + Integer.toString(
      Integer.parseInt(x) + Integer.parseInt(y));
  }
// ...

The IO type is extensible, in the sense that we may define our own custom IO operations. We’re not limited to those found in the IO library.

// ...

  private static IO<Void> printMaybe(Optional<String> x) {
    return x.map(IO::putStrLn).orElse(IO._return(null));
  }

  private static IO<String> prompt( Optional<String> greet,
                                    Function<String, Optional<String>> confirm ) {
    return printMaybe(greet)
      .and(IO.getLine).bind( l ->
        printMaybe(confirm.apply(l))
          .and(IO._return(l)));
  }
// ...

We maintain referential transparency by waiting until main to use unsafeRunIO. The result is that app is a first-class value. We are free to reuse it anywhere, or inline it, or otherwise refactor as we see fit.

// ...

  public static IO<Void> app =
    prompt( Optional.of("Please input two numbers."),
            l -> Optional.of("Got first input: " + l) ).bind( x ->
    prompt( Optional.empty(),
            l -> Optional.of("Got second input: " + l) ).bind( y ->
    IO.putStrLn(appLogic(x, y)) ));

  public static void main(String[] args) {
    app.unsafeRunIO();
  }
}

Javascript

The IO library in Javascript is delightfully small.

// IO.js
const IO = (function() {
  'use strict';

  const IO = (run) => {
    return {
      unsafeRunIO: run,

      map: (f) => IO(() => f(run())),

      bind: (f) => IO(() => f(run()).unsafeRunIO()),

      and: (x) => IO(() => {
        run();
        return x.unsafeRunIO();
      })
    };
  };

  return {
    getLine: IO(() => window.prompt("")),

    putStrLn: (str) => IO(() => {
      window.alert(str);
      return null;
    }),

    return: (x) => IO(() => x)
  };
}());

The outer IO refers to the name of the module. The inner IO refers to the name of the class. We’re able to hide the class by only exporting our three constructors.

We need an HTML file to run our Javascript app in the browser. It’s in this file that we call unsafeRunIO.

<!-- App.html -->
<html>
  <head>
    <script type="text/javascript" src="./IO.js"></script>
    <script type="text/javascript" src="./App.js"></script>
  </head>
  <body>
    <script type="text/javascript">
      App.main.unsafeRunIO()
    </script>
  </body>
</html>

Finally, our Javascript port of our app.

// App.js
const App = (function() {
  'use strict';

  const appLogic = (x, y) => 'Result: ' + (parseInt(x) + parseInt(y));

  const printMaybe = (strM) => strM == null ? IO.return(null) : IO.putStrLn(strM)

  const prompt = (greet, confirm) =>
    printMaybe(greet)
      .and(IO.getLine).bind( l =>
        printMaybe(confirm(l))
          .and(IO.return(l)));

  const app =
    prompt( 'Please input two numbers.',
            l => 'Got first input: ' + l ).bind( x =>
    prompt( null,
            l => 'Got second input: ' + l ).bind( y =>
    IO.putStrLn(appLogic(x, y)) ));

  return {
    main: app
  };
}());

Python

Python is a delightful language that seems to revel in side effects (mostly in the form of assignment statements). Let’s see if we can encapsulate IO here.

# IO.py
class IO:

  def __init__(self, run):
    self.unsafeRunIO = run

  def map(self, f):
    return IO(lambda: f(self.unsafeRunIO()))

  def bind(self, f):
    return IO(lambda: f(self.unsafeRunIO()).unsafeRunIO())

  def _and(self, other):
    def go():
      self.unsafeRunIO()
      return other.unsafeRunIO()
    return IO(go)

def _return(x):
  return IO(lambda: x)

def putStrLn(str):
  def go():
    print(str)
    return None
  return IO(go)

getLine = IO(input)

Hopefully, the IO library looks second-nature by now. Remember, the object of the game is to create a data structure that describes the effects to be done, but doesn’t do them until unsafeRunIO is called.

The only tricky part in Python is that lambdas can’t span multiple lines. It’s easy enough to get around this by defining and using a named function, though.

# App.py
import IO

def appLogic(x, y):
  return "Result: " + str(int(x) + int(y))

def printMaybe(strM):
  return IO.putStrLn(strM) if strM is not None else IO._return(None)

def prompt(greet, confirm):
  salute = printMaybe(greet)._and(IO.getLine)
  certify = lambda l: printMaybe(confirm(l))._and(IO._return(l))
  return salute.bind(certify)

app = prompt( "Please input two numbers.",
              lambda l: "Got first input: " + str(l) ).bind(lambda x:
      prompt( None,
              lambda l: "Got second input: " + str(l) ).bind(lambda y:
      IO.putStrLn(appLogic(x, y)) ))

if __name__ == "__main__":
  app.unsafeRunIO()

Scala

The Scala IO library is even shorter than the Javascript one. We make the IO type opaque by marking it as sealed and by making the companion apply method private.

// IO.scala
sealed trait IO[A] {

  def unsafeRunIO: A

  final def map[B](f: A => B): IO[B] = IO(f(unsafeRunIO))

  final def flatMap[B](f: A => IO[B]): IO[B] = IO(f(unsafeRunIO).unsafeRunIO)

  final def and[B](x: IO[B]): IO[B] = IO {
    unsafeRunIO
    x.unsafeRunIO
  }
}

object IO {

  private def apply[A](x: => A): IO[A] = new IO[A] {
    def unsafeRunIO: A = x
  }

  val getLine: IO[String] = IO(io.StdIn.readLine)

  def putStrLn(str: String): IO[Unit] = IO(println(str))

  def _return[A](x: A): IO[A] = IO(x)
}

Since Scala supports call-by-name arguments, we have a very convenient syntax IO(...) for “promoting” side-effectful code blocks into first-class actions.

In Scala we benefit from having for comprehensions, so our port follows the original Haskell much more closely (though the port is not quite as pretty as the original).

// App.scala
object App {

  def appLogic(x: String, y: String): String =
    "Result: " + (x.toInt + y.toInt).toString

  def printMaybe(x: Option[String]): IO[Unit] =
    x.fold(IO._return(()))(IO.putStrLn)

  def prompt( greet: Option[String],
              confirm: String => Option[String]
            ): IO[String] = for {
    _ <- printMaybe(greet)
    l <- IO.getLine
    _ <- printMaybe(confirm(l))
  } yield l

  val app: IO[Unit] = for {
    x <- prompt( Some("Please input two numbers."),
                 l => Some("Got first input: " + l) )
    y <- prompt( None,
                 l => Some("Got second input: " + l) )
    _ <- IO.putStrLn(appLogic(x, y))
  } yield ()

  def main(args: Array[String]) = app.unsafeRunIO
}

Thanks for reading. I hope you enjoyed this post, and I sincerely hope that I helped eliminate some of the mysticism for you. It’s not magic, after all.