Trie

Functional key-value hash maps.

Functional maps (and sets) whose representation is "canonical", and independent of operation history (unlike other popular search trees).

The representation we use here comes from Section 6 of ["Incremental computation via function caching", Pugh & Teitelbaum](https://dl.acm.org/citation.cfm?id=75305).

User’s overview

This module provides an applicative (functional) hash map. Notably, each put produces a new trie and value being replaced, if any.

Those looking for a more familiar (imperative, object-oriented) hash map should consider TrieMap or HashMap instead.

The basic Trie operations consist of: - put - put a key-value into the trie, producing a new version. - get - get a key’s value from the trie, or null if none. - iter - visit every key-value in the trie.

The put and get operations work over Key records, which group the hash of the key with its non-hash key value.

import Trie "mo:base/Trie";
import Text "mo:base/Text";

type Trie<K, V> = Trie.Trie<K, V>;
type Key<K> = Trie.Key<K>;

func key(t: Text) : Key<Text> { { key = t; hash = Text.hash t } };

let t0 : Trie<Text, Nat> = Trie.empty();
let t1 : Trie<Text, Nat> = Trie.put(t0, key "hello", Text.equal, 42).0;
let t2 : Trie<Text, Nat> = Trie.put(t1, key "world", Text.equal, 24).0;
let n : ?Nat = Trie.put(t1, key "hello", Text.equal, 0).1;
assert (n == ?42);

Trie

type Trie<K, V> = {#empty; #leaf : Leaf<K, V>; #branch : Branch<K, V>}

Binary hash tries: either empty, a leaf node, or a branch node

Leaf

type Leaf<K, V> = { size : Nat; keyvals : AssocList<Key<K>, V> }

Leaf nodes of trie consist of key-value pairs as a list.

Branch

type Branch<K, V> = { size : Nat; left : Trie<K, V>; right : Trie<K, V> }

Branch nodes of the trie discriminate on a bit position of the keys' hashes. we never store this bitpos; rather, we enforce a style where this position is always known from context.

AssocList

type AssocList<K, V> = AssocList.AssocList<K, V>

Key

type Key<K> = { hash : Hash.Hash; key : K }

equalKey

func equalKey<K>(keq : (K, K) -> Bool) : ((Key<K>, Key<K>) -> Bool)

Equality function for two `Key<K>`s, in terms of equality of `K’s.

isValid

func isValid<K, V>(t : Trie<K, V>, enforceNormal : Bool) : Bool

Checks the invariants of the trie structure, including the placement of keys at trie paths

Trie2D

type Trie2D<K1, K2, V> = Trie<K1, Trie<K2, V>>

A 2D trie maps dimension-1 keys to another layer of tries, each keyed on the dimension-2 keys.

Trie3D

type Trie3D<K1, K2, K3, V> = Trie<K1, Trie2D<K2, K3, V>>

A 3D trie maps dimension-1 keys to another layer of 2D tries, each keyed on the dimension-2 and dimension-3 keys.

empty

func empty<K, V>() : Trie<K, V>

An empty trie.

size

func size<K, V>(t : Trie<K, V>) : Nat
 Get the number of key-value pairs in the trie, in constant time.
Get size in O(1) time.

branch

func branch<K, V>(l : Trie<K, V>, r : Trie<K, V>) : Trie<K, V>

Construct a branch node, computing the size stored there.

leaf

func leaf<K, V>(kvs : AssocList<Key<K>, V>, bitpos : Nat) : Trie<K, V>

Construct a leaf node, computing the size stored there.

This helper function automatically enforces the MAX_LEAF_SIZE by constructing branches as necessary; to do so, it also needs the bitpos of the leaf.

fromList

func fromList<K, V>(kvc : ?Nat, kvs : AssocList<Key<K>, V>, bitpos : Nat) : Trie<K, V>

Transform a list into a trie, splitting input list into small (leaf) lists, if necessary.

clone

func clone<K, V>(t : Trie<K, V>) : Trie<K, V>

Clone the trie efficiently, via sharing.

Purely-functional representation permits O(1) copy, via persistent sharing.

replace

func replace<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v : ?V) : (Trie<K, V>, ?V)

Replace the given key’s value option with the given one, returning the previous one

put

func put<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v : V) : (Trie<K, V>, ?V)

Put the given key’s value in the trie; return the new trie, and the previous value associated with the key, if any

get

func get<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool) : ?V

Get the value of the given key in the trie, or return null if nonexistent

find

func find<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool) : ?V

Find the given key’s value in the trie, or return null if nonexistent

merge

func merge<K, V>(tl : Trie<K, V>, tr : Trie<K, V>, k_eq : (K, K) -> Bool) : Trie<K, V>

Merge tries, preferring the right trie where there are collisions in common keys.

note: the disj operation generalizes this merge operation in various ways, and does not (in general) lose information; this operation is a simpler, special case.

mergeDisjoint

func mergeDisjoint<K, V>(tl : Trie<K, V>, tr : Trie<K, V>, k_eq : (K, K) -> Bool) : Trie<K, V>

Merge tries like merge, except signals a dynamic error if there are collisions in common keys between the left and right inputs.

diff

func diff<K, V, W>(tl : Trie<K, V>, tr : Trie<K, W>, k_eq : (K, K) -> Bool) : Trie<K, V>

Difference of tries. The output consists are pairs of the left trie whose keys are not present in the right trie; the values of the right trie are irrelevant.

disj

func disj<K, V, W, X>(tl : Trie<K, V>, tr : Trie<K, W>, k_eq : (K, K) -> Bool, vbin : (?V, ?W) -> X) : Trie<K, X>

Map disjunction.

This operation generalizes the notion of "set union" to finite maps.

Produces a "disjunctive image" of the two tries, where the values of matching keys are combined with the given binary operator.

For unmatched key-value pairs, the operator is still applied to create the value in the image. To accomodate these various situations, the operator accepts optional values, but is never applied to (null, null).

join

func join<K, V, W, X>(tl : Trie<K, V>, tr : Trie<K, W>, k_eq : (K, K) -> Bool, vbin : (V, W) -> X) : Trie<K, X>

Map join.

This operation generalizes the notion of "set intersection" to finite maps. The values of matching keys are combined with the given binary operator, and unmatched key-value pairs are not present in the output.

foldUp

func foldUp<K, V, X>(t : Trie<K, V>, bin : (X, X) -> X, leaf : (K, V) -> X, empty : X) : X

This operation gives a recursor for the internal structure of tries. Many common operations are instantiations of this function, either as clients, or as hand-specialized versions (e.g., see , map, mapFilter, some and all below).

prod

func prod<K1, V1, K2, V2, K3, V3>(tl : Trie<K1, V1>, tr : Trie<K2, V2>, op : (K1, V1, K2, V2) -> ?(Key<K3>, V3), k3_eq : (K3, K3) -> Bool) : Trie<K3, V3>

Map product.

Conditional catesian product, where the given operation op conditionally creates output elements in the resulting trie.

The keyed structure of the input tries are not relevant for this operation: all pairs are considered, regardless of keys matching or not. Moreover, the resulting trie may use keys that are unrelated to these input keys.

iter

func iter<K, V>(t : Trie<K, V>) : I.Iter<(K, V)>

Returns an Iter over the key-value entries of the trie.

Each iterator gets a persistent view of the mapping, independent of concurrent updates to the iterated map.

Build

let Build

Represent the construction of tries as data.

This module provides optimized variants of normal tries, for more efficient join queries.

The central insight is that for (unmaterialized) join query results, we do not need to actually build any resulting trie of the resulting data, but rather, just need a collection of what would be in that trie. Since query results can be large (quadratic in the DB size), avoiding the construction of this trie provides a considerable savings.

To get this savings, we use an ADT for the operations that would build this trie, if evaluated. This structure specializes a rope: a balanced tree representing a sequence. It is only as balanced as the tries from which we generate these build ASTs. They have no intrinsic balance properties of their own.

fold

func fold<K, V, X>(t : Trie<K, V>, f : (K, V, X) -> X, x : X) : X

Fold over the key-value pairs of the trie, using an accumulator. The key-value pairs have no reliable or meaningful ordering.

some

func some<K, V>(t : Trie<K, V>, f : (K, V) -> Bool) : Bool

Test whether a given key-value pair is present, or not.

all

func all<K, V>(t : Trie<K, V>, f : (K, V) -> Bool) : Bool

Test whether all key-value pairs have a given property.

nth

func nth<K, V>(t : Trie<K, V>, i : Nat) : ?(Key<K>, V)

Project the nth key-value pair from the trie.

Note: This position is not meaningful; it’s only here so that we can inject tries into arrays using functions like Array.tabulate.

toArray

func toArray<K, V, W>(t : Trie<K, V>, f : (K, V) -> W) : [W]

Gather the collection of key-value pairs into an array of a (possibly-distinct) type.

isEmpty

func isEmpty<K, V>(t : Trie<K, V>) : Bool

Test for "deep emptiness": subtrees that have branching structure, but no leaves. These can result from naive filtering operations; filter uses this function to avoid creating such subtrees.

filter

func filter<K, V>(t : Trie<K, V>, f : (K, V) -> Bool) : Trie<K, V>

Filter the key-value pairs by a given predicate.

mapFilter

func mapFilter<K, V, W>(t : Trie<K, V>, f : (K, V) -> ?W) : Trie<K, W>

Map and filter the key-value pairs by a given predicate.

equalStructure

func equalStructure<K, V>(tl : Trie<K, V>, tr : Trie<K, V>, keq : (K, K) -> Bool, veq : (V, V) -> Bool) : Bool

Test for equality, but naively, based on structure. Does not attempt to remove "junk" in the tree; For instance, a "smarter" approach would equate #bin {left = #empty; right = #empty} with #empty. We do not observe that equality here.

replaceThen

func replaceThen<K, V, X>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v2 : V, success : (Trie<K, V>, V) -> X, fail : () -> X) : X

Replace the given key’s value in the trie, and only if successful, do the success continuation, otherwise, return the failure value

putFresh

func putFresh<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, v : V) : Trie<K, V>

Put the given key’s value in the trie; return the new trie; assert that no prior value is associated with the key

put2D

func put2D<K1, K2, V>(t : Trie2D<K1, K2, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool, v : V) : Trie2D<K1, K2, V>

Put the given key’s value in the 2D trie; return the new 2D trie.

put3D

func put3D<K1, K2, K3, V>(t : Trie3D<K1, K2, K3, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool, k3 : Key<K3>, k3_eq : (K3, K3) -> Bool, v : V) : Trie3D<K1, K2, K3, V>

Put the given key’s value in the trie; return the new trie;

remove

func remove<K, V>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool) : (Trie<K, V>, ?V)

Remove the given key’s value in the trie; return the new trie

removeThen

func removeThen<K, V, X>(t : Trie<K, V>, k : Key<K>, k_eq : (K, K) -> Bool, success : (Trie<K, V>, V) -> X, fail : () -> X) : X

Remove the given key’s value in the trie, and only if successful, do the success continuation, otherwise, return the failure value

remove2D

func remove2D<K1, K2, V>(t : Trie2D<K1, K2, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool) : (Trie2D<K1, K2, V>, ?V)

remove the given key-key pair’s value in the 2D trie; return the new trie, and the prior value, if any.

remove3D

func remove3D<K1, K2, K3, V>(t : Trie3D<K1, K2, K3, V>, k1 : Key<K1>, k1_eq : (K1, K1) -> Bool, k2 : Key<K2>, k2_eq : (K2, K2) -> Bool, k3 : Key<K3>, k3_eq : (K3, K3) -> Bool) : (Trie3D<K1, K2, K3, V>, ?V)

Remove the given key-key pair’s value in the 3D trie; return the new trie, and the prior value, if any.

mergeDisjoint2D

func mergeDisjoint2D<K1, K2, V>(t : Trie2D<K1, K2, V>, k1_eq : (K1, K1) -> Bool, k2_eq : (K2, K2) -> Bool) : Trie<K2, V>

Like [mergeDisjoint](#mergedisjoint), except instead of merging a pair, it merges the collection of dimension-2 sub-trees of a 2D trie.