I'm publishing this series ahead of Code BEAM Europe this October, where I'll reveal Bondy, a new declarative language for the BEAM: statically typed, functional, relational and rule-based.
To complect is to take two or more separate concepts and entangle them so tightly that you can no longer reason about or change one without risking the other.
Rich Hickey, the creator of Clojure, revived this old word to name the root cause of software complexity. His point is that complexity doesn't come from size or age. It comes from tying together things that should have stayed apart. Every "we can't touch that without breaking something else" is one of those knots. His prescription is to keep separate things separate: plain maps and sets instead of objects, values instead of mutable state, components side by side instead of twisted together.
I think Rich is right, and I try to follow this. But there is a gap in it, and the gap is where most of the day-to-day pain in a real codebase lives.
The gap
If your data is "just" maps and sets, then everything you know about that data has nowhere to live except in your code. A bare map is typed by convention and by hope. Suppose you want to say "this map must have a name and an age, and the age is a non-negative integer below 130". The moment you do, you have left the data model and entered procedure.
So we write validators. On a good day we reach for a declarative DSL: Ecto schemas and NimbleOptions in Elixir, babel and maps_utils in Erlang (at Leasight), JSON Schema, clojure.spec, malli. These are good tools. But each is a second language bolted onto the first, and each can say some of what you mean and not the rest. Here is where the declarative part stops:
-module(example).
-export([spec/0]).
-export([new/1]).
%% What the DSL can say: shape and per-field type.
spec() ->
#{
price => #{
required => true,
datatype => pos_integer
},
quantity => #{
required => true,
datatype => pos_integer
},
discount => #{
required => false,
datatype => non_neg_integer
},
approved_by => #{
required => false,
datatype => binary
}
}.
%% What the DSL cannot say. Three constraints as declarative as the
%% four above, written as control flow because the spec has no slot
%% for them.
new(Map0) ->
Map = maps_utils:validate(Map0, spec()),
#{price := Price, quantity := Qty} = Map,
Total = Price * Qty,
%% Default to zero
Discount = maps:get(discount, Map, 0),
%% Cross-field: discount may not exceed 10% of the line.
Discount =< Total div 10 orelse error({invalid, discount}),
%% Conditional on a computed value: large orders need
%% an approver.
case Total > 10000 andalso not is_map_key(approved_by, Map) of
true ->
error({missing, approved_by});
false ->
ok
end,
Map.Read spec/0 and you learn the shape and the types. Read new/1 and you learn nothing until you execute it in your head. Then you learn that Total is derived, that discount is bounded by it, and that approved_by is required sometimes. The condition lives three lines away from the field. "Discount is at most a tenth of the total" is as much a rule about the data as pos_integer is. But the DSL only understands one field at a time, so anything that involves two fields becomes code.
We complected the constraints into the constructor. Not by choice. In a vocabulary of maps, sets and functions a value can only ever be one thing, a datum. "Discount is at most a tenth of the total" is not a datum and it is not a function. It is a statement about what must hold, and the language has nowhere to put one.
One noun, more roles
My conclusion is that the language should let the noun it already has play more roles.
In most languages a value is one thing: a datum in a variable, waiting to be passed to a function. But in every real program the same value lives other lives.
That value is a fact that holds, a relation. It is an invariant that is imposed, a constraint. It is something that happened, an event. It is a truth that persists until something ends it, a fluent. These are not four data structures. They are four roles one typed value can play, each with its own semantics.
The computation side is just as thin. There is one noun, the function, and one verb, call. Two definitions are missing beside it: the rule, which runs when the world makes it true, and the signal, which never runs at all because it always equals its definition.
For almost every one of these concepts, there is an entire programming paradigm built by taking that single idea seriously and making it a primitive.[1] Each paradigm discovered one element, turned it into its entire world, and made the others awkward to use. I believe the working programmer needs them all—in the same program, at the same time. At least, I always have. When a language only supplies one or two, we are forced to awkwardly bolt the rest onto whatever tools are at hand.
This is why I don't describe Bondy as multi-paradigm. It has one way to describe data, the shape, and one type system. You describe a kind of value once, as a named shape. Its roles are one-line declarations on top of it, and this compiles today:
%% The order line from new/1. Amounts are in pence.
type OrderLine = {
price: Int & > 0,
quantity: Int & > 0,
total: .price * .quantity when .total =< 10000 or .approved_by !== null,
discount: Int & >= 0 = 0 when .discount * 10 =< .total,
approved_by?: String
}
%% Its two roles
relation OrderLine
constraint OrderLine
%% A type and its role, declared together
event LineApproved = {line: OrderLine, by: String}Every rule from new/1 is in the type. total is computed from its siblings. discount defaults to zero and may not exceed a tenth of the total. A total above 10000 needs an approved_by. The when on a field can read the other fields of the same value, so a rule that relates two fields sits next to the fields it relates.
That is my answer to the obvious objection, that four roles and two nouns sounds like six little languages bolted onto a seventh. They are not bolted on, because they share the same values. An OrderLine in a query, an OrderLine in a rule and an OrderLine arriving at the boundary are the same typed value, checked by the same type checker.
It also fits Hickey's own line between simple (not tangled up) and easy (familiar). A language where a value can only play one role makes you tangle the others into your code. That is complex in his sense, even with fewer keywords. A few separate roles, each doing one thing, is simpler, even if there is more to learn.
Alan Perlis’ famous epigram states that "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures". There is still one data structure here. What multiplies is what it is allowed to mean.
Three words I'll lean on. A noun is a kind of thing you can declare and name. A verb is an operation only that thing supports. A role is what a value is doing at runtime.
Shapes
A shape answers one question: which values are allowed? Int is a shape. So is Int & >= 18, the integers from 18 up, and so is {name: String, age: Int & >= 0}, a record with two fields. A shape with named fields is a structural shape, and when you give one a name, as in type Person = {...}, you get a structural type.
Shapes are values in their own right. You can bind one to a variable, pass it around, and combine it with others at runtime:
let adult = Int & >= 18
age & adult %% the age if it is 18 or more; a failure saying why, if notThe & is unification: it combines two descriptions into one that satisfies both, or reports that they conflict. Int & >= 18 is itself a unification, of Int with >= 18. Unify a value with a shape and you have validation. Unify two shapes and you have a stricter shape. Unify data that is only half there and you get a value that knows which fields are still missing. The third post shows each of these running.
What makes "object" a slur in Hickey's vocabulary is not the record but everything OO added to it: identity, mutable state behind that identity, behaviour dispatched through it. Keep the record, throw the rest overboard, and you have what a bare map cannot give you: a description of which values are allowed, with the rules inside it. discount: Int & >= 0 = 0 when .discount * 10 =< .total is part of the type, not a check performed downstream. That settles the gap inside one value. It does not settle what must hold between values, or what follows when it does. That takes roles.
Relations: a value, held
A relation answers: how are things connected? It is Codd's relational model, the one under every SQL database and Datalog. A value on its own describes one thing. A value in the relation role is a fact, and a set of facts describes how things are connected, which you can query and join.
What it replaces in your code is the object graph. order.customer and customer.orders are the same connection stored twice, kept in sync by hope, and each only works in one direction. A relation stores the connection once, and you query it from either side.
Its verbs are Codd's: select, project, join, insert, delete. There is no update. Update is delete-then-insert, and pretending otherwise is how the object graph smuggled mutation back in.
Constraints: a value, imposed
A constraint answers: what must hold? A relation is a set of facts you consult. A constraint is a fact you impose: you tell it to a store, and the store is then obliged to keep it consistent with everything else it has been told.
Inside one value, the shape already holds what fell off the edge of the validation DSL and landed in new/1. A constraint takes the same idea across values. The invariant still lives with the type, not in a procedure the next reader has to execute in their head. And a constraint is not only something to check after the fact. Other facts can be derived from it: if the store knows age >= 0 and learns age = x - 1, it follows that x >= 1.
In Bondy, rules over the store are how you say what follows. Validation is the simplest case, where you already have all the values and only ask yes or no.
Its verbs are tell and ask: tell adds a constraint to the store, ask checks whether what the store knows already implies one. A schema has one verb, validate, and validate is tell-everything-then-ask-once. A constraint you can only validate has most of its meaning cut off.
The natural objection, "isn't a constraint just a rule?", draws the line. A constraint is what must be true. A rule is when this, then that. One is a role a value plays. The other is a noun, and it gets its own section.
Events: a value, emitted
An event answers: what happened? It has an instant, not a duration. It is never updated and never retracted, because it already happened.
What it replaces is the callback and the log line. In most languages "something happened" is either a function call or a string in a log. The call welds the occurrence to whoever is listening, and the log line keeps the occurrence but throws away its type. An event keeps both: a typed value, addressed to nobody in particular. In Bondy, events are what rules emit and what processes act on. What comes in is told to the store as a constraint, and what goes out is an event.
An event has no inertia at all. The next role is nothing but inertia.
Fluents: a value, latched
A fluent answers: what is true now, and since when? The idea comes from the event calculus: a fluent becomes true when something starts it and stays true until something ends it. "The light is on" becomes true when someone flips the switch and stays true until someone flips it back.
The example that shows why this is its own role is an alarm. Define "alarm is active" as temperature > 100 and the alarm clears itself the instant the reading dips under the threshold. Real alarms latch. The alarm is initiated by the crossing, terminated by an acknowledgement, and in between it holds regardless of what the temperature does. That is what a mutable boolean throws away. The boolean has no record of when it changed or what changed it, and you spend the rest of the project reconstructing that from logs.
The query is holds-at: was this true at time t? Nothing sets a fluent. The only way to change one is to name what changed it.
Fluents are designed and on the roadmap. The post on time covers the design.
Signals
A signal answers: what value does this always equal? The cleanest intuition is a spreadsheet cell. B1 = A1 * 2 is a signal. You never assign to B1: it always equals twice A1, and keeping that true is not your problem. Its verbs are define and sample. There is no set, because B1 = A1 * 2 would be meaningless if anyone could write to B1.
A signal is not a role, because it is not a fact playing a part. A signal is a definition, like a function. A function is a computation you call. A signal is a computation you never call, because its answer is already there.
The alarm shows why fluents and signals are two things. The temperature is a signal: it always has a current value, computed from the sensor, and it keeps no history. "Temperature exceeds 100" is a signal too: it is true while the reading is above 100 and false as soon as it is not. The moment the reading goes above 100 is an event: it happens once, at one point in time. The latched alarm is a fluent: that event makes it true, and it stays true until someone acknowledges it, even after the temperature falls. Defining the alarm as temperature > 100 treats all four as one signal, and that is why it clears itself.
Signals are designed alongside fluents, on the same incremental engine.
Rules
A rule answers: what follows, and what reacts? It has two kinds, one for relations and one for constraints.
Derivation rules work over relations: they work out new facts from existing ones, until nothing new appears. This is Datalog, and it ships:
relation Edge = {from: String, to: String}
relation Reachable(src: String, dst: String) :-
Edge(from: src, to: dst)
relation Reachable(src: String, dst: String) :-
Edge(from: src, to: mid), Reachable(src: mid, dst: dst)Rewrite rules work over constraints: they match several facts in a store at once, check a condition, and replace what they matched. This is CHR, Constraint Handling Rules, and it ships too. A rule matches facts with on, takes the ones it names out of the store with remove, and adds constraints and events with tell. The facts it does not remove stay in the store.
constraint Bucket = {id: String, tokens: Int, max_tokens: Int}
constraint Request = {id: String}
event Allowed = {id: String, remaining: Int}
rule consume_token =
on Request(id: rid) as req, Bucket(id: rid, tokens: t, max_tokens: m) as bucket
remove req, bucket
when t > 0
tell Bucket(id: rid, tokens: t - 1, max_tokens: m),
Allowed(id: rid, remaining: t - 1)
endMatching several facts at once is what a function can't do. You call a function with its input. A rule matches a combination of facts in the store (rid in both patterns is a join), and the engine decides when it fires.
A function runs when called. A rule runs when the world makes it true.
Because a rule is a declaration and not control flow, the compiler can inspect it. Today it checks that every guard has no side effects and always gives the same answer. It also checks that a guard cannot go from true to false as more facts arrive, so the order of arrival cannot change the result. Checking that a rule set always terminates is on the roadmap.
Run the alarm once more, end to end. The temperature is a signal. When it crosses the threshold, the crossing is told to the store. A rewrite rule matches it, latches the alarm fluent, and emits an event so the rest of the system hears about it. When an acknowledgement arrives, another rule terminates the fluent. Four roles and two nouns, none standing in for another.
Where this lands
Put them side by side and the table is clean enough that I take it as evidence these are real, not invented:
| Noun or role | The question it answers | Its verbs | What it separates |
|---|---|---|---|
| Shape (the noun) | Which values are allowed? | unify (&) | The description of a value from identity, state and behaviour |
| as relation | How are things connected? | select, project, join, insert, delete | A connection from the one direction you happened to store it in |
| as constraint | What must hold? | tell, ask | Validity from the procedure that checks it |
| as event | What happened? | emit, on | An occurrence from whoever is listening |
| as fluent | What is true now, and since when? | initiate, terminate, holds-at | A persistent truth from its cell and its cause |
| Signal | What value does this always equal? | define, sample | A derived value from the glue that recomputes it |
| Rule | What follows, and what reacts? | match, remove, derive, fire | Derivation and reaction from hand-wired control flow |
Unification belongs to the shape, and because every role is played by values of a shape, it works in all of them. It is what holds the table together: it is how a rule matches a constraint in the store and how a query matches a fact in a relation.
Everything else is specific. Nothing is joined except a relation. Nothing is told except a constraint. Nothing fires except a rule. And none of the verbs is assign.
Rich is right that complecting is the disease and that maps and sets beat the object soup they replaced. My one amendment is that maps and sets answer one of these seven questions, what is this value. Offering them as the answer to all seven is itself a complecting. You get one role and one verb, and every other life a value leads gets smeared across procedures, DSLs and mutable cells. A language that ignores relation, constraint, event, fluent, signal and rule doesn't spare you those concepts. It makes you pay for them by hand, forever, with assign.
I've spent the last few years putting these under one type system, on the BEAM. Shapes, relations, constraints, events and both kinds of rule are in the compiler today. Fluents and signals are designed and come next. The rest of this series takes them one at a time, with code you can compile. It starts with shapes, and with how the compiler and the runtime check the order line above.
The relational model and Datalog took the relation. Constraint programming took the constraint, and concurrent constraint programming (Saraswat) gave it tell and ask. The situation and event calculi (McCarthy and Hayes, then Kowalski and Sergot) took the event and the fluent. Logic programming took the rule. Synchronous and reactive languages (Lustre, Esterel, FRP) took the signal. Object orientation took the shape of the value and welded identity, state and behaviour onto it. ↩︎