Large Files and Binary Assets

Most source code works well as-is. git’s delta compression is efficient for text, the ingest path is streaming, and forking a popular open-source repo costs essentially nothing in storage. This page is about the cases where size or content type calls for a different posture.

Source code and normal binaries — handled well today

For the overwhelming majority of repositories — source code, config files, scripts, documentation, build artifacts under a few hundred megabytes, images for a web app, compiled binaries for a CLI tool — neosource uses standard git pack format stored on distributed object storage. This is the same format git uses locally, which means:

• git clone, git push, and git fetch all work without any client-side changes
• delta compression between versions is efficient
• shallow and blobless clones work normally
• forking a repo shares the upstream’s stored objects and adds no new storage cost

The ingest path streams the wire pack directly to durable object storage without buffering the whole pack in memory, so even a multi-gigabyte initial push of a large project lands cleanly.

Files larger than approximately 512 MiB — planned, not yet available

Git stops applying delta compression to objects larger than its core.bigFileThreshold (approximately 512 MiB by default). Past this threshold, git stores blobs verbatim, so the pack-delta win disappears. For truly large individual files — video masters, large ML model weights, high-resolution CAD exports, AAA-scale uncompressed assets — the storage profile shifts: each version is stored essentially in full, with no cross-version savings.

What’s planned: a size-gated externalization path (“Tier W”) that handles objects above the threshold differently: each blob is stored as a whole-object, content-addressed by BLAKE3, compressed with zstd, outside the main git pack. This provides whole-object dedup (two repos that hold the exact same large blob share one stored copy) and keeps the large objects out of the pack index. The design is documented in plans/binary-tier-tier-w.md.

Current status: not yet built. If you’re pushing repositories with many files above 512 MiB each, those files land in packs and are charged at full decompressed size against your quota. Git LFS is an option for deferring those objects to a separate store, but neosource’s native large-file path is not yet available.

Actively-edited large structured assets — gamedev and media workflows

For game development, visual effects, and similar workflows involving large binary assets that are edited repeatedly by multiple people — .blend files, .fbx meshes, .psd layered images, Godot .scn scenes, large audio files — the challenges are different from raw file size:

1. Binary files cannot be merged. Merge and conflict tooling is powerful for source code, but a .png or a .blend cannot be 3-way-merged. Two simultaneous edits to the same binary file have exactly one safe resolution: someone’s work wins and the other’s is discarded.
2. Large frequently-changed binaries accumulate quickly even with delta compression, because binary formats often change wholesale between saves.

The complete answer for this workflow involves two things that are not yet fully available:

• File locking: exclusive locks on binary assets so collaborators see who has a file checked out before they start editing it. The design is documented in ADR 0037. It is planned as a complement to the Git LFS locking API so standard game-engine git integrations continue to work.
• A native large-asset backend (Lore): a substrate built around whole-object dedup and content-defined chunking for the kinds of files that change in predictable ways (e.g., appending frames to a video, extending a binary container). This is a separate design decision from the git-pack backbone, gated on the gamedev vertical being a product priority.

If your workflow is already using Git LFS, neosource works with LFS clients today — LFS pointer files live in the git pack normally, and the LFS server endpoint is available. The native large-asset experience described above is planned for when the gamedev vertical is prioritised.

Large blobs that re-encode wholesale — retention and file locking

Some binary assets change completely with each edit rather than accumulating incrementally. A rendered video frame, a baked lighting map, or a compiled shader cache has essentially no relationship to its predecessor at the byte level. For these:

• Delta compression provides no win, and CDC chunking provides no win either. Measured on real gamedev corpora (godot-demo-projects, bun), git’s own pack format matches or beats content-defined chunking on real binary asset sets.
• Retention policy matters more than encoding. If each version of a large baked asset is kept forever, storage grows with every commit. The practical answer is retention: decide how many historical versions to keep, and prune older ones. This integrates with file locking — the lock lifecycle includes releasing old versions.

File locking design is in ADR 0037; the retention mechanics are connected to the operation-log retention work described in the internal ADR 0033.

Storage and quota

All stored content counts against your workspace’s storage quota, measured as the logical decompressed size of the objects your repos reach, minus the bytes you inherit through fork lineage (a fork of a public repo pays for zero bytes from the upstream). Incompressible binary content that the compactor cannot shrink is charged at full stored size — there is no discount for content that resists compression.

File size caps are enforced at ingest: individual pushes above 16 GiB are rejected, and individual objects above 2 GiB are rejected before they land. These limits exist to keep the read path well-behaved; contact support if your workflow genuinely requires larger individual objects.