Storage & distribution
How model bytes move in and out of ScaiAtlas, and how clusters share them.
Storage backends#
ScaiAtlas stores metadata in a database and bytes in an object store. Two backends:
- S3-compatible (default) — production deployments. Multipart uploads and range reads happen against the bucket, mediated by the API.
- Filesystem — single-node deployments. Files are served through signed,
time-limited URLs (
/api/v1/files/{key}) validated by HMAC signature and expiry, so no auth token is needed for the actual byte transfer.
Either way, clients never hold storage credentials. The API proxies or signs every transfer.
Uploads are chunked and verified#
Uploads are multipart. The lifecycle:
- Create the version —
POST …/versionsputs it inuploadingstate. - Init a file —
POST …/upload/initwith the file's path, size, and SHA-256; the server returns anupload_id, a recommendedpart_size, andpart_count. - Push parts —
PUT …/upload/{upload_id}/parts/{n}for each chunk; each returns anetag. - Complete —
POST …/upload/{upload_id}/completewith the part list; the file is assembled and checksum-verified.
The SDK's upload session does all of this for you — see Uploading a model. Default chunk size is 10 MB; you can override it.
Downloads resume and verify#
Downloads support HTTP range requests, so an interrupted transfer restarts from where it stopped rather than from zero:
GET …/downloadreturns file metadata and URLs for a version (optionally a specificvariantorfile).GET …/download/files/{path}streams a file, honoring aRange: bytes=start-endheader and replying206 Partial Contentwith aContent-Range. It also setsX-Checksum-SHA256andAccept-Ranges: bytes.
The SDK verifies each file's SHA-256 after download and skips files already present with a matching checksum. See Downloading a model.
Variants: many formats, one version#
A version's original upload is its primary variant. You can request additional
variants — e.g. a GGUF quantization of a safetensors original — with
POST …/variants, which returns 202 Accepted and creates the variant in pending
state while conversion runs (pending → converting → available). Downloads can
target a specific variant with the variant parameter.
Peer-to-peer distribution#
For clusters that pull the same model repeatedly, ScaiAtlas can distribute via P2P instead of every node hitting origin storage.
Manifests#
A distribution manifest breaks a version into fixed-size chunks (default 64 MB), each with its own checksum:
1 2 | |
Chunked distribution is not implemented today. No manifest is generated, so
GETanswers404andPOSTanswers501. Clusters that share model bytes between nodes currently fetch a whole version from one peer and verify it against the per-file SHA-256 checksums the version already carries. The chunked form below is the design, not the running behaviour.
The manifest lists every file, its chunks, and per-chunk offsets/sizes/checksums — enough for a peer to fetch chunks from wherever they're available and reassemble with integrity.
The tracker#
When P2P is enabled, peers announce what they hold and discover each other through
the tracker under /api/v1/p2p:
POST /p2p/announce— a peer advertises its address and which version chunks it has (seeding vs. still leeching).GET /p2p/peers/by-version?version_id=…— find peers holding a version, split into seeders and leechers.GET /p2p/peersandGET /p2p/stats— the full roster and tracker-wide counts. Administrators only: together they list every node's address and port. A peer looking for a swarm usesby-version, which answers for one version at a time.
If P2P is disabled on a deployment, these endpoints return 503.
A node id has an owner#
The first announce of a node_id claims it for the caller that made it. From
then on only that caller can move the peer — change its address, port or chunk
list — and anyone else gets 403. Since clients resolve peer addresses through
by-version, an unowned node id would let anyone redirect a swarm.
Ownership is tied to the credential, not kept forever. Revoking the key that claimed a peer releases it, so a machine rebuilt under a new worker key simply reclaims its node id on the next announce. An administrator can also take a node over, and a node can delete its own peer when it retires.
P2P is an optimization layer. Correctness never depends on it — origin storage is always authoritative, and every chunk is checksum-verified on arrival.