Storage formats

A relate storestore A relate database: a directory, a single .rl file, or :memory:, opened with --database. Read more in the glossary has one of three layouts, picked by the path given to --database: in memory, a directory, or a single .rl file. A durable store also has one codeccodec The on-disk encoding of a store's snapshot: Parquet (the default), JSON or markdown, fixed when the store is created. Read more in the glossary, the encoding its checkpoints write the snapshotsnapshot The committed graph state a checkpoint writes to disk, loaded on open before the WAL replays; not the snapshot durability mode or --read-only freshness. Read more in the glossary in: parquet (the default), json or markdown. Every layout and codec runs the same engine: the graph is held in memory while the store is open, and every commit goes to the store's write-ahead logWAL Append-only log each commit goes to before it applies; under the default full durability mode it is synced first, so a crash loses no acknowledged write. Read more in the glossary (WAL) before it applies. The layout and codec decide only what a checkpointcheckpoint The step that writes a fresh snapshot of committed state and recycles old WAL segments: every 300 seconds by default, on clean exit, and on .checkpoint. Read more in the glossary writes to disk.

Synopsis

relate --database <path> [--read-only[=live]] [--snapshot-format parquet|json|markdown|parquet-wide] [--wal-dir <dir>] [<statement>]
relate clone [--wal-dir <dir>] [--to-snapshot-format parquet|json|markdown] <source> <dest>
relate info --database <path> [--json]

A missing path is created on the first writable open. A --read-only open of a missing path is an error and creates nothing. .cl files written by releases up to 0.16 open as single-file stores. These pages call the single-file layout a .rl file; relate info prints it as layout: .rl bundle, and the directory layout as layout: native directory.

Layouts

A writable open of a durable store that another process holds open for writing fails with storage locked: <path> (exit code 5). The writer lock is held while the store is open.

Codecs

The codec is chosen when a store is created, with --snapshot-format or the RELATE_SNAPSHOT_FORMAT environment variable, and recorded in the store. An existing store keeps its recorded codec, so later opens need no flag. One store holds one codec.

graph.json holds every node and relationship with its labels or type and properties, then the index and constraint catalog. A store holding one KNOWS relationship between two people starts like this:

{"value_encoding":2,"nodes":[{"id":0,"labels":["Person"],"properties":{"name":"Ada"}},{"id":1,"labels":["Person"],"properties":{"name":"Grace"}}],"relationships":[{"id":0,"type":"KNOWS","start_node":0,"end_node":1,"properties":{"since":2024}}],"indexes":[],"constraints":[], …

A markdown note keeps a node's properties and outgoing relationships in YAML frontmatter, with relationships as [[Label/name]] links. --markdown-body-properties names the properties written as the note's body (default: body). A note edited by hand while no process has the store open is read at the next open.

---
type: Person
name: Ada
role: engineer
KNOWS:
  - "[[Person/Grace]]"
WORKS_ON:
  - since: 2024
    target: "[[Project/relate]]"
---

relate clone --to-snapshot-format <codec> copies a store into a new store in another codec; the source is not written. In relate 0.1.2 a markdown store clones into a .rl file only with --to-snapshot-format parquet or json (relate clone).

Wide-column parquet

--snapshot-format parquet-wide keeps the parquet codec and changes how its data files hold properties: a property key that the rows of a file share with one type (boolean, integer, float or string) gets a typed column of its own, named p_<key>, and the remaining properties stay in the tagged-JSON properties column. relate info reports the codec as parquet and the format version as 5 (6 while the store holds an ontology). These commands create a wide directory store:

relate --database ./wide --snapshot-format parquet-wide "CREATE (:Person {name: 'Ada', role: 'engineer'}), (:Person {name: 'Grace', role: 'engineer'})"

relate info --database ./wide | head -n 3

layout: native directory
codec: parquet
format version: 5

The flag also applies to an existing parquet store, unlike the other values: the store's next checkpoint rewrites every data file in the wide form. From then on the manifest keeps the store wide, and later opens need no flag. It works on a directory store and a .rl file, and on relate serve. On a json or markdown store it fails the open with storage error: snapshot codec mismatch: directory is <codec>, requested parquet. RELATE_SNAPSHOT_FORMAT and relate clone --to-snapshot-format don't take parquet-wide, and a clone of a wide store is written at format version 3; open the copy once with --snapshot-format parquet-wide to make it wide again.

Directory store files

A markdown store keeps manifest.json and the WAL in its .cypherlite/ folder (.cypherlite/manifest.json, .cypherlite/wal/NNNNNNNN.log, .cypherlite/wal/.cypherlite.lock, .cypherlite/wal/.cypherlite.writer) and the root lock files .cypherlite.lock and .cypherlite.writer beside its notes; the parquet and json codecs keep them at the root and in wal/.

The snapshot files hold the state at the last checkpoint; the WAL holds every commit since. Opening a store loads the snapshot, then replays the WAL. A copy of the snapshot files without the WAL folder holds the state at the last checkpoint only, and a --read-only open of it fails.

On-disk names that start with .cypherlite and the magic numbers in the version markers table keep the product's former name, so stores written by earlier releases open unchanged.

The schema copy is plain text. After two schema statements on a directory store, schema.cypher holds one statement per index and constraint:

relate --database ./people --mode envelope "CREATE CONSTRAINT person_name IF NOT EXISTS FOR (p:Person) REQUIRE p.name IS UNIQUE"
relate --database ./people --mode envelope "CREATE INDEX person_role IF NOT EXISTS FOR (p:Person) ON (p.role)"

cat people/.cypherlite/schema/schema.cypher

CREATE RANGE INDEX `person_role` FOR (n:`Person`) ON (n.`role`);
CREATE CONSTRAINT `person_name` FOR (n:`Person`) REQUIRE (n.`name`) IS UNIQUE;

The copy is checked on every open, so an edited file stops the store from opening, read-only included:

echo "// edited" >> people/.cypherlite/schema/schema.cypher
relate --database ./people --read-only --mode envelope "RETURN 1 AS ok"
echo "exit=$?"

Failed to open Relate: storage error: schema mirror .cypherlite/schema/schema.cypher differs from the schema this store's checkpoint persisted; it was edited by hand or copied from another store; delete the .cypherlite/schema/ directory and reopen: the next checkpoint regenerates it from the store's schema
{"error":{"code":"fatal_error","exit_code":1,"kind":"storage","message":"storage error: schema mirror .cypherlite/schema/schema.cypher differs from the schema this store's checkpoint persisted; it was edited by hand or copied from another store; delete the .cypherlite/schema/ directory and reopen: the next checkpoint regenerates it from the store's schema","retryable":false}}
exit=1

Version markers

Each durable part of a store carries a version marker. A store is checked against every marker before anything is written.

A store at an older readable version is upgraded in place when it is opened writable, and older releases can't open it afterward. A store written by a newer release fails to open, with the unsupported version in the error. Upgrade a store covers both. A --read-only open never migrates.

Size on disk

These commands write the same 100,000 nodes into a parquet directory store, a json directory store and a .rl file, then clone the .rl file, and print each store's size as relate info reports it:

relate --database ./items --mode envelope "UNWIND range(1, 100000) AS i CREATE (:Item {id: i, name: 'item-' + toString(i), score: i * 0.5})"
relate --database ./items-json --snapshot-format json --mode envelope "UNWIND range(1, 100000) AS i CREATE (:Item {id: i, name: 'item-' + toString(i), score: i * 0.5})"
relate --database ./items.rl --mode envelope "UNWIND range(1, 100000) AS i CREATE (:Item {id: i, name: 'item-' + toString(i), score: i * 0.5})"
relate clone ./items.rl ./items-clone.rl

{"columnTypes":[],"columns":[],"notifications":[],"planCache":"miss","rows":[],"timings":{"elapsed_ms":382.339583,"execute_ms":382.13495800000004,"lex_parse_ms":0.005,"lower_ms":0.006709,"parse_ms":0.194375,"plan_ms":0.0043749999999999995,"result_generation_ms":0.000167,"rewrite_ms":0.000375},"truncated":false}
{"columnTypes":[],"columns":[],"notifications":[],"planCache":"miss","rows":[],"timings":{"elapsed_ms":369.608875,"execute_ms":369.399625,"lex_parse_ms":0.006833,"lower_ms":0.0077919999999999994,"parse_ms":0.1995,"plan_ms":0.004292,"result_generation_ms":0.00029099999999999997,"rewrite_ms":0.000375},"truncated":false}
{"columnTypes":[],"columns":[],"notifications":[],"planCache":"miss","rows":[],"timings":{"elapsed_ms":387.91008300000004,"execute_ms":387.68233299999997,"lex_parse_ms":0.007416,"lower_ms":0.008875000000000001,"parse_ms":0.21529099999999998,"plan_ms":0.006167000000000001,"result_generation_ms":0.000166,"rewrite_ms":0.000542},"truncated":false}

for store in items items-json items.rl items-clone.rl; do printf '%s: ' "$store"; relate info --database "./$store" | grep 'total bytes'; done

items: total bytes: 2696219
items-json: total bytes: 9245067
items.rl: total bytes: 19070976
items-clone.rl: total bytes: 2723840

Size a store and its memory covers the memory a store needs while it is open.

Surfaces

RELATE_SNAPSHOT_FORMAT selects the codec of a new store, a directory or a .rl file, on all three.

Errors

On stderr, an error at open is prefixed Failed to open Relate: . The errors reference lists the storage errors with their remedies.

See also

• Choose a store layout: which layout and codec fits a job, with the commands that create each.
• Back up and restore a store: relate clone and file copies.
• Storage and durability: the write path, the WAL, and checkpoints.
• Limits and sizing: the limits on graph size, commits, and values.
• Keep the schema as files: reading, reviewing and applying .cypherlite/schema/.
• Open a store as an Obsidian vault: the markdown codec in use.