Choose a store layout

This guide shows you how to choose where a relate storestore A relate database: a directory, a single .rl file, or :memory:, opened with --database. Read more in the glossary lives and how it is written to disk. The path you pass to --database picks the layout: :memory: keeps the graph in the process, a path ending in .rl is one file, and any other path is a directory. Use :memory: for throwaway work, a .rl file for a store you copy or hand to someone, and a directory storedirectory store A store kept as a folder of snapshot files, a WAL folder and lock files; the layout for any --database path that is not :memory: and does not end in .rl. Read more in the glossary when a second process reads beside the writer or you want a text snapshot you can diff.

Prerequisites

• relate on your PATH (Install relate).
• For the client section: the Python client (relate-db) or the Node.js client (@relate/relate).

Compare the layouts

Every layout runs the same engine: the whole graph lives in memory while the store is open, and every commit goes to the store's write-ahead log (WALWAL Append-only log each commit goes to before it applies; under the default full durability mode it is synced first, so a crash loses no acknowledged write. Read more in the glossary) first. The layouts differ in what sits on disk.

Startup grows with the size of the graph in every durable layout, because relate loads all of it into memory before the first statement runs. The storage formats reference has the file layout of each and measured sizes.

Start in memory

With no --database, a statement runs against an in-memory store that is gone when the process exits:

relate --mode envelope "RETURN 1 AS ok"

{"columnTypes":["INTEGER"],"columns":["ok"],"notifications":[],"planCache":"miss","rows":[{"ok":1}],"timings":{"elapsed_ms":0.356541,"execute_ms":0.20325000000000001,"lex_parse_ms":0.020542,"lower_ms":0.02775,"parse_ms":0.09895799999999999,"plan_ms":0.042292,"result_generation_ms":0.000209,"rewrite_ms":0.007292},"truncated":false}

Use it for tests, scratch statements and anything you rebuild from a script.

Create a single-file store

The rest of this guide uses one small graph. Save its seed statement as team.cypher:

cat > team.cypher <<'EOF'
CREATE (ada:Person {name: 'Ada', role: 'engineer'}),
       (grace:Person {name: 'Grace', role: 'engineer'}),
       (noor:Person {name: 'Noor', role: 'maintainer'}),
       (relate:Project {name: 'relate', language: 'Rust'}),
       (tck:Project {name: 'opencypher-tck', language: 'Gherkin'}),
       (ada)-[:WORKS_ON {since: 2024}]->(relate),
       (grace)-[:WORKS_ON {since: 2023}]->(relate),
       (grace)-[:WORKS_ON {since: 2022}]->(tck),
       (noor)-[:MAINTAINS]->(relate),
       (ada)-[:KNOWS]->(grace);
EOF

Run it into a path that ends in .rl. relate creates the file on the first open:

relate query --database ./team.rl --mode envelope < team.cypher

{"columnTypes":[],"columns":[],"notifications":[],"planCache":"miss","rows":[],"timings":{"elapsed_ms":2.00025,"execute_ms":1.69775,"lex_parse_ms":0.01425,"lower_ms":0.02775,"parse_ms":0.283083,"plan_ms":0.013583,"result_generation_ms":0.000167,"rewrite_ms":0.000583},"truncated":false}

relate info reads a store's metadata without opening it:

relate info --database ./team.rl

layout: .rl bundle
superblock version: 1
codec: parquet
format version: 3
checkpoint LSN: 1
commit timestamp: 1
total bytes: 36864
wal segments past checkpoint: 1
nodes (at checkpoint LSN 1): 5
relationships (at checkpoint LSN 1): 5
indexes: 0
constraints: 0

Choose a .rl file when the store travels: one file to copy, attach, or check in. To read it into memory without holding the file, pass it to --load; the in-memory copy is read-only:

relate --load ./team.rl --mode envelope "MATCH (p:Person) RETURN count(p) AS people"

{"columnTypes":["INTEGER"],"columns":["people"],"notifications":[],"planCache":"miss","rows":[{"people":3}],"timings":{"elapsed_ms":0.363917,"execute_ms":0.18412499999999998,"lex_parse_ms":0.03225,"lower_ms":0.043709,"parse_ms":0.11108299999999999,"plan_ms":0.060209,"result_generation_ms":0.00025,"rewrite_ms":0.00825},"truncated":false}

Create a directory store

Any path that does not end in .rl becomes a directory store:

relate query --database ./team --mode envelope < team.cypher

{"columnTypes":[],"columns":[],"notifications":[],"planCache":"miss","rows":[],"timings":{"elapsed_ms":1.223083,"execute_ms":0.926084,"lex_parse_ms":0.015042,"lower_ms":0.021459,"parse_ms":0.28016599999999997,"plan_ms":0.010917000000000001,"result_generation_ms":0.000209,"rewrite_ms":0.0005},"truncated":false}

The folder holds the snapshot (manifest.json and the Parquet files under nodes/ and relationships/) and the WAL:

ls team

manifest.json
nodes
relationships
wal

Choose a directory store when another process needs to read while one writes: a --read-only open of a directory works beside its writer, and a .rl file refuses it. Share one store between two processes covers that setup.

Pick a codec for a directory store

A codec is the encoding a checkpoint writes the snapshot in. The default is Parquet: compact, binary columnar files. JSON writes the whole graph as one readable graph.json, which a code review can diff. Pass --snapshot-format json when you create the store:

relate query --database ./team-json --snapshot-format json --mode envelope < team.cypher

{"columnTypes":[],"columns":[],"notifications":[],"planCache":"miss","rows":[],"timings":{"elapsed_ms":1.260125,"execute_ms":0.930708,"lex_parse_ms":0.013375,"lower_ms":0.061,"parse_ms":0.313042,"plan_ms":0.010166000000000001,"result_generation_ms":0.000042,"rewrite_ms":0.001084},"truncated":false}

ls team-json

graph.json
manifest.json
wal

The codec is fixed when the store is created and recorded in manifest.json, so later opens need no flag:

relate --database ./team-json --read-only --mode envelope "MATCH (p:Person) RETURN p.name AS name ORDER BY name"

{"columnTypes":["STRING"],"columns":["name"],"notifications":[],"planCache":"miss","rows":[{"name":"Ada"},{"name":"Grace"},{"name":"Noor"}],"timings":{"elapsed_ms":0.536292,"execute_ms":0.376833,"lex_parse_ms":0.029833000000000002,"lower_ms":0.038042,"parse_ms":0.10087499999999999,"plan_ms":0.050334000000000004,"result_generation_ms":0.00020800000000000001,"rewrite_ms":0.008},"truncated":false}

RELATE_SNAPSHOT_FORMAT=json makes the same choice for a new store, including from the clients. A third codec, markdown, writes one note per node for a folder people edit by hand; the formats reference describes it.

Open a store from Python or Node.js

The clients take the same three spellings: ":memory:", a directory path, or a path ending in .rl. Each opens writable by default and creates a missing store. In Python, relate.connect(path) opens one, as in the Python tutorial, and read_only=True opens it read-only. This program writes the graph into each layout, then opens the .rl file read-onlyread-only An open mode (--read-only) that never writes, checkpoints or migrates the store; writes fail, and a directory store can be read beside its writer. Read more in the glossary:

import relate

seed = """
CREATE (ada:Person {name: 'Ada', role: 'engineer'}),
       (grace:Person {name: 'Grace', role: 'engineer'}),
       (noor:Person {name: 'Noor', role: 'maintainer'}),
       (relate:Project {name: 'relate', language: 'Rust'}),
       (tck:Project {name: 'opencypher-tck', language: 'Gherkin'}),
       (ada)-[:WORKS_ON {since: 2024}]->(relate),
       (grace)-[:WORKS_ON {since: 2023}]->(relate),
       (grace)-[:WORKS_ON {since: 2022}]->(tck),
       (noor)-[:MAINTAINS]->(relate),
       (ada)-[:KNOWS]->(grace)
"""

for path in [":memory:", "./team", "./team.rl"]:
    conn = relate.connect(path)
    conn.execute(seed)
    people = conn.execute("MATCH (p:Person) RETURN count(p) AS people").fetchone()["people"]
    print(f"{path}: {people} people")
    conn.close()

reader = relate.connect("./team.rl", read_only=True)
rows = reader.execute("MATCH (p:Person) RETURN p.name AS name ORDER BY name").fetchall()
print("read-only:", [row["name"] for row in rows])
reader.close()

:memory:: 3 people
./team: 3 people
./team.rl: 3 people
read-only: ['Ada', 'Grace', 'Noor']

In Node.js, Database.open takes the same paths, and { readOnly: true } opens a store read-only. The Node.js handles in this program stay open until it exits, so the read-only open goes to the directory store, which allows a reader beside its writer:

const { Database } = require("@relate/relate");

const seed = `
CREATE (ada:Person {name: 'Ada', role: 'engineer'}),
       (grace:Person {name: 'Grace', role: 'engineer'}),
       (noor:Person {name: 'Noor', role: 'maintainer'}),
       (relate:Project {name: 'relate', language: 'Rust'}),
       (tck:Project {name: 'opencypher-tck', language: 'Gherkin'}),
       (ada)-[:WORKS_ON {since: 2024}]->(relate),
       (grace)-[:WORKS_ON {since: 2023}]->(relate),
       (grace)-[:WORKS_ON {since: 2022}]->(tck),
       (noor)-[:MAINTAINS]->(relate),
       (ada)-[:KNOWS]->(grace)`;

for (const path of [":memory:", "./team", "./team.rl"]) {
  const db = Database.open(path);
  const conn = db.connect();
  conn.query(seed);
  const people = conn.query("MATCH (p:Person) RETURN count(p) AS people").rows[0].people;
  console.log(`${path}: ${people} people`);
}

const reader = Database.open("./team", { readOnly: true }).connect();
const names = reader.query("MATCH (p:Person) RETURN p.name AS name ORDER BY name").rows;
console.log("read-only:", names.map((row) => row.name));

:memory:: 3 people
./team: 3 people
./team.rl: 3 people
read-only: [ 'Ada', 'Grace', 'Noor' ]

Switch layouts later

relate clone copies a store into a new store of either layout, and --to-snapshot-format changes the codec on the way. The source is opened read-only and never written; the destination must not exist yet. Turn the .rl file into a directory store:

relate clone ./team.rl ./team-copy

Then make a JSON copy of that:

relate clone --to-snapshot-format json ./team-copy ./team-copy-json

Switch when the job changes:

Verify

relate info names the layout and codec of a store. The JSON copy reports both on its first two lines:

relate info --database ./team-copy-json | head -n 2

layout: native directory
codec: json

If it goes wrong

• storage error: snapshot codec mismatch: directory is json, requested parquet (exit code 1). --snapshot-format named a codec other than the one the directory was created with. Asking for Parquet on the JSON store fails:

  relate --database ./team-json --snapshot-format parquet --read-only --mode envelope "RETURN 1 AS ok"

  Failed to open Relate: storage error: snapshot codec mismatch: directory is json, requested parquet
  {"error":{"code":"fatal_error","exit_code":1,"kind":"storage","message":"storage error: snapshot codec mismatch: directory is json, requested parquet","retryable":false}}

  Drop the flag, or make a copy in another codec with relate clone --to-snapshot-format <codec> <source> <dest>.

• storage locked: <path> (exit code 5). Another process holds the store. A .rl file can't be opened at all while a writer holds it, even --read-only. If a reader has to run beside the writer, move to a directory store with relate clone. The locked error has the details.

• Failed to clone database: storage error: clone destination already exists (exit code 1). relate clone never overwrites. Pick a new destination path.

Related

• Storage formats: the files each layout and codec writes, versions, and measured sizes.
• Back up and restore a store: relate clone as the backup command.
• Storage and durability: the write path every layout shares.
• Share one store between two processes: a writer and readers on one directory store.