Local-First Impact Graph Change Tracking
This guide explains how a local-first impact graph tracks changes across its nodes and edges, enabling offline-first collaboration and conflict-free synchronization. You’ll learn the core mechanisms—version vectors, change logs, and merge strategies—and how to implement them step by step.
What You’ll Need
- A basic understanding of graph data structures (nodes, edges, properties).
- Familiarity with a programming language (e.g., JavaScript, Python) for the example snippets.
- A local storage system (e.g., IndexedDB, SQLite, or a simple JSON file).
- Optional: A sync library like Automerge or Yjs, but the guide covers manual implementation.
Step-by-Step
- Define the graph model. Represent each node and edge with a unique ID (e.g., UUID). Store properties as key-value pairs. For example, a node might be
{ id: 'n1', type: 'person', props: { name: 'Alice' } }. - Add a version vector to each entity. A version vector is a map of replica ID to a counter. Each time a replica modifies an entity, it increments its own counter and updates the vector. For instance, after replica A makes a change, the vector becomes
{ A: 1 }; if replica B also changes it, the vector becomes{ A: 1, B: 1 }. - Maintain a change log. For every mutation (add/update/delete), append an entry to a local log. Each entry should contain: the entity ID, the operation (e.g., ‘set’, ‘delete’), the new value (or a diff), the timestamp, and the current version vector of that entity.
- Implement a merge function. When receiving changes from another replica, merge them into your local graph. For each incoming change, compare its version vector with the local one. If the incoming vector is strictly greater (i.e., all counters are >= and at least one is >), apply the change. If they are concurrent (neither dominates), use a conflict resolution strategy (see Tips).
- Handle deletions with tombstones. Instead of physically removing a node, mark it as deleted with a tombstone. This ensures that deletions propagate correctly across replicas. A tombstone is a special property like
deleted: true. - Sync periodically. When a replica goes online, exchange change logs. Send all log entries that the other replica hasn’t seen (based on their version vectors). Apply incoming changes using the merge function.
- Garbage collect tombstones. After all replicas have acknowledged a deletion, you can remove the tombstone. This requires a mechanism to know when all replicas have seen the deletion, e.g., by tracking the maximum version vector across all replicas.
Common Mistakes
- Ignoring concurrent updates. If two replicas change the same property simultaneously, you must have a strategy (e.g., last-write-wins, merge, or conflict flag).
- Using wall-clock timestamps for ordering. Clocks can be out of sync; use version vectors instead.
- Deleting data without tombstones. This can cause a deleted node to reappear on another replica.
- Not persisting the change log. If you lose the log, you lose the ability to sync correctly.
- Over-complicating the merge. Start with a simple last-write-wins for scalar values and a merge for sets.
Tips
- Use a library if possible. Automerge and Yjs handle version vectors and merge logic for you, saving time and reducing bugs.
- Keep the change log compact. Instead of storing full snapshots, store diffs or patches to reduce storage and sync bandwidth.
- Test with multiple replicas. Simulate offline scenarios and concurrent edits to ensure your merge logic is correct.
- Document your conflict resolution strategy. This helps users understand what happens when edits conflict.
- Use a consistent hashing for IDs. This ensures that the same entity has the same ID across replicas.
Troubleshooting
- Changes are not syncing: Check that both replicas have a common baseline. Ensure the change log is being sent and received correctly. Verify that version vectors are being updated on every change.
- Conflicts are not resolved as expected: Review your merge function. If using last-write-wins, ensure the timestamp is the version vector, not wall-clock time. If using a merge, ensure you handle nested structures correctly.
- Deleted nodes reappear: This usually means tombstones are not being propagated. Make sure deletions are logged and sent to all replicas before garbage collection.
- Storage grows too large: Implement garbage collection for tombstones and compress the change log. Consider using a delta-based approach instead of storing full snapshots.
- Sync is slow: Optimize the change log by sending only the missing changes. Use a binary format for efficiency.
Frequently Asked Questions
What is a version vector?
A version vector is a map of replica IDs to counters. It tracks how many changes each replica has contributed to an entity. It helps determine causality between updates.
How do I handle concurrent edits to the same property?
You can use last-write-wins (based on version vector order), merge the values (if they are sets or maps), or flag the conflict for manual resolution. Choose based on your domain.
Do I need to use a specific database?
No. You can use any local storage, from IndexedDB to SQLite. The key is to store the change log and version vectors alongside the data.
Can I use this for real-time collaboration?
Yes, but you’ll need a network layer to propagate changes. Local-first means the app works offline and syncs when online.
How do I know when it’s safe to delete a tombstone?
You need to know that all replicas have received the deletion. This can be done by tracking the maximum version vector across all replicas and comparing it to the tombstone’s vector.
Is this the same as CRDTs?
Yes, this approach is a form of Conflict-Free Replicated Data Type (CRDT), specifically a state-based CRDT. It ensures eventual consistency without a central server.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.