Size a store and its memory

This guide shows you how to estimate the memory a store needs, measure the size of a real one, and cap its growth. relate holds the whole graph in memory: opening a storestore A relate database: a directory, a single .rl file, or :memory:, opened with --database. Read more in the glossary loads its last snapshotsnapshot The committed graph state a checkpoint writes to disk, loaded on open before the WAL replays; not the snapshot durability mode or --read-only freshness. Read more in the glossary and replays the WALWAL Append-only log each commit goes to before it applies; under the default full durability mode it is synced first, so a crash loses no acknowledged write. Read more in the glossary into RAM, and there is no buffer pool or spill to disk. The graph you query has to fit in the process's memory, with room for the statements that run on it.

Prerequisites

• relate on your PATH (Install relate).
• Either the node, relationship, and property counts you expect, or a store to measure.

Estimate from counts

relate counts the size of a graph in logical bytes. The logical graph sizelogical graph size relate's own byte count of the live nodes, relationships, properties and index entries; what --logical-graph-size-limit caps. Read more in the glossary is its own tally of the live nodes, relationships, properties, and index entries. That tally is what --logical-graph-size-limit compares against. It leaves out the WAL buffers, statement working memory and allocator overhead, which the next section adds back.

These figures were measured with the relate version this page is verified against: 10,000 elements of one shape written into a fresh .rl store, the store's logical size read with the method in Measure a real store, and the difference divided by 10,000.

Property costs grow in steps, not one property at a time, so a node with 5 properties costs nearly as much as one with 8.

Work an example

Take a memory graph for an agent: 1,000,000 :Memory nodes with an integer id, a 20-character name and an 8-character kind, 5,000,000 RELATES_TO relationships (5 out of each node) with an integer since, and a range index on id.

A one-hundredth model of the same graph (10,000 nodes, 50,000 relationships, the same index) measured 25,901,217 logical bytes, which scales to 2,590,121,700: the estimate is within 0.2% of it.

Allow for process memory

The process needs more memory than the logical size, and the most while a store opens. Two measurements of this release:

For the worked example (2,586,000,000 logical bytes), that is about 3.2 to 4.0 GB once loaded, and about 6.5 GB while the store opens, at the 2.5 ratio of the largest measured store (rounded to 0.1 GB). The open is the peak, so size the machine from it. Statements need working memory on top: one value a statement builds, such as a list, may reach 256 MiB under the default --expression-memory-limit.

The size on disk is not a guide to memory: a 1,000,000-node .rl store of 187,416,576 bytes, measured with this release, holds 1,390,890,109 logical bytes.

Measure a real store

relate info reads a store's metadata without opening it, including the node and relationship counts at its last checkpointcheckpoint The step that writes a fresh snapshot of committed state and recycles old WAL segments: every 300 seconds by default, on clean exit, and on .checkpoint. Read more in the glossary. To follow along, create the shared example graph first:

cat > team.cypher <<'EOF'
CREATE (ada:Person {name: 'Ada', role: 'engineer'}),
       (grace:Person {name: 'Grace', role: 'engineer'}),
       (noor:Person {name: 'Noor', role: 'maintainer'}),
       (relate:Project {name: 'relate', language: 'Rust'}),
       (tck:Project {name: 'opencypher-tck', language: 'Gherkin'}),
       (ada)-[:WORKS_ON {since: 2024}]->(relate),
       (grace)-[:WORKS_ON {since: 2023}]->(relate),
       (grace)-[:WORKS_ON {since: 2022}]->(tck),
       (noor)-[:MAINTAINS]->(relate),
       (ada)-[:KNOWS]->(grace);
EOF
relate query --database ./team.rl < team.cypher

Read its counts:

relate info --database ./team.rl

layout: .rl bundle
superblock version: 1
codec: parquet
format version: 3
checkpoint LSN: 1
commit timestamp: 1
total bytes: 36864
wal segments past checkpoint: 1
nodes (at checkpoint LSN 1): 5
relationships (at checkpoint LSN 1): 5
indexes: 0
constraints: 0

The counts go straight into the estimate. To read the logical size itself, try a write against a size limit of 1 byte: the refusal reports the live size as used. --load with a new --database path does it on a copy, probe.rl, so the original store is never opened for writing:

relate --load ./team.rl --database ./probe.rl --logical-graph-size-limit 1 --mode envelope "CREATE (:Probe)"

The first two lines are stderr, the last is the error object on stdout:

--load ./team.rl: loaded into ./probe.rl
capacity exceeded: used=6555 limit=1
{"error":{"code":"fatal_error","exit_code":1,"kind":"capacity","message":"capacity exceeded: used=6555 limit=1","retryable":false}}

The example graph is 6,403 logical bytes. Remove the copy when you're done:

rm ./probe.rl

Cap the graph's size

--logical-graph-size-limit SIZE caps the logical size. It takes bytes, or a number with mb or gb (powers of 1000) or mib or gib (powers of 1024). The default is unlimited, and 0 means unlimited too. RELATE_LOGICAL_GRAPH_SIZE_LIMIT sets it for every run in an environment, and relate serve takes the same flag.

Set the cap below the memory you can give the process, divided by the open-peak ratio from Allow for process memory. A write that would grow the graph past the cap then fails with an error you can read, instead of the operating system ending the process.

This statement tries to add a node to team.rl under a cap of 6,500 bytes, which the 6,403-byte graph leaves no room for:

relate --database ./team.rl --logical-graph-size-limit 6500 --mode envelope "CREATE (:Person {name: 'Ines', role: 'engineer'})"
echo "exit=$?"

capacity exceeded: used=6555 limit=6500
{"error":{"code":"fatal_error","exit_code":1,"kind":"capacity","message":"capacity exceeded: used=6555 limit=6500","retryable":false}}
exit=1

What the cap does when a write reaches it:

Verify

Read the store under the same cap, read-only. The read runs, and the refused write left the 3 people as they were:

relate --database ./team.rl --read-only --logical-graph-size-limit 6500 --mode envelope "MATCH (p:Person) RETURN count(p) AS people"

{"columnTypes":["INTEGER"],"columns":["people"],"notifications":[],"planCache":"miss","rows":[{"people":3}],"timings":{"elapsed_ms":0.34058299999999997,"execute_ms":0.164833,"lex_parse_ms":0.029375,"lower_ms":0.042624999999999996,"parse_ms":0.112292,"plan_ms":0.05450000000000001,"result_generation_ms":0.000292,"rewrite_ms":0.008417},"truncated":false}

If it goes wrong

• capacity exceeded: used=… limit=…, exit code 1. A write reached --logical-graph-size-limit. Raise the cap if the machine has the memory, delete data you no longer need, or split the graph across stores. The errors reference has the entry.
• This statement tried to commit a single WAL record of at least <used>. One commit's WAL record would be over the 64 MiB per-record cap, which is separate from the size limit. Commit the data in smaller transactions, such as batches of a bulk load (WAL payload cap).
• Error: invalid --logical-graph-size-limit: semantic error: invalid memory limit unit 'kb'; use bytes, mb, mib, gb, or gib, exit code 2. The size has a unit relate doesn't accept. Write it in bytes or with one of the four units.
• The process ends with no relate error while the graph grows. The operating system stopped it for memory. Under the default full durability mode, every write it acknowledged is in the WAL and comes back on the next open (Recover from a crash). Size the machine from the open peak, and set a cap below it.

Related

• Limits and sizing: every limit relate enforces, with its default.
• --logical-graph-size-limit in the flags reference.
• Choose a durability mode: what a crash can lose while the graph is in memory.