Basic Versioning Example
Let’s create a table with sample data to demonstrate LanceDB’s versioning capabilities:Set Up the Table
First, let’s create a table with some sample data:Check Initial Version
After creating the table, let’s check the initial version information:Modify Data
When you modify data through operations like update or delete, LanceDB automatically creates new versions.Update Existing Data
Let’s update some existing records to see versioning in action:Add New Data
Now let’s add more records to the table:Check Version Changes
Let’s see how the versions have changed after our modifications:Rollback to Previous Versions
LanceDB supports fast rollbacks to any previous version without data duplication.View All Versions
First, let’s see all the versions we’ve created:Restore a Version Snapshot
Now let’s restore a captured version snapshot:Tag-Based Versioning
Numeric table versions likev3 or v17 are precise but hard to remember. Tags
let you attach human-readable labels (e.g., "prod", "baseline",
"q3-evaluation") to specific versions and check those out by name. They are
conceptually similar to git tags, and unlike numeric versions, tagged versions
are preserved when old versions are pruned (see the cleanup note at the bottom
of this page).
The tags API supports the standard CRUD operations — create, list, update, delete —
plus checking out by tag name.
Deleting a tag only removes the label, not the version it points to. After
deletion, the underlying table version becomes eligible for cleanup again.
Branches
Branches are isolated, writable lines of history forked from another branch (or a specific version). Writes to a branch do not affectmain, which makes
branches ideal for:
- Running experiments (new indexes, schema changes, reprocessing) without
disturbing production reads on
main. - Backfills or migrations that you want to review before promoting.
- Sharing a frozen point-in-time fork with a collaborator while continuing to
write to
main.
Branches are currently supported on local and namespace-backed tables in
LanceDB OSS. Branch support for LanceDB Enterprise (remote) tables is not yet
available.
Create, Write, and Reopen a Branch
Create a branch frommain (the default source) and write to it. The base table
continues to point at main and is unaffected by writes on the branch. You can
also reopen an existing branch either from a table handle or directly from the
database connection when opening the table.
list returns a mapping of branch names to branch metadata, including which
branch each was forked from. main is the reserved default branch: wherever a
branch is optional, leaving it out means main. To fork from somewhere else,
pass a branch name, a specific version, or both when creating the branch.
Deleting a branch removes that branch and its branch-local history; data on
main is not affected. There is no automatic merge operation. To promote
changes, copy the desired data back explicitly.
Branches vs. tags vs. checkout
- Use a tag to bookmark an existing version with a human-readable name. Tags are read-only and protect that version from cleanup.
- Use
checkout(version)to read from a historical version ofmainwithout forking. The handle is read-only until yourestore. - Use a branch when you need to write on top of a point in history
without touching
main.
Delete Data From the Table
Let’s demonstrate how deletions also create new versions:Go Back to Latest Version
First, let’s return to the latest version:Delete Data
Now let’s delete some data to see how it affects versioning:Version History and Operations
On a fresh table, the snippets in this guide produce this version sequence:v1: create table (create_table/createTable/create_table)v2: update rows (update)v3: add rows (add)v4: restore snapshot (restore) fromversion_after_mod/versionAfterModv5: delete rows (delete)
list_versions/listVersions, version, checkout, checkout_latest/checkoutLatest) do not create new versions.
The version metadata fields can differ by backend. Direct table-backed version listing exposes a
timestamp, while namespace-backed listing may expose fields such as manifest_path,
manifest_size, e_tag, and timestamp_millis. In deployments that use managed versioning,
prefer the table version APIs exposed by LanceDB Enterprise or the namespace service instead of
mixing in lower-level Lance file operations.
System OperationsSystem operations like
optimize(), index updates, and table compaction also increment table version numbers.
In LanceDB OSS and Enterprise, optimize() can prune older versions based on its retention setting (cleanup_older_than, 7 days by default),
which is when old-version files are removed and disk space is reclaimed.Tagged versions are exempt from cleanup. A version with a tag pointing at it is
retained regardless of age, and its files are not removed by optimize(). To make
a tagged version eligible for pruning, delete the tag first.