---
summary: Where bytes live, resumable transfer, and P2P.
title: Storage & distribution
path: concepts/storage-and-distribution
status: published
---

# Storage & distribution

How model bytes move in and out of ScaiAtlas, and how clusters share them.

## Storage backends

ScaiAtlas stores metadata in a database and **bytes** in an object store. Two backends:

- **S3-compatible** (default) — production deployments. Multipart uploads and range
  reads happen against the bucket, mediated by the API.
- **Filesystem** — single-node deployments. Files are served through signed,
  time-limited URLs (`/api/v1/files/{key}`) validated by HMAC signature and expiry,
  so no auth token is needed for the actual byte transfer.

Either way, **clients never hold storage credentials.** The API proxies or signs
every transfer.

## Uploads are chunked and verified

Uploads are multipart. The lifecycle:

1. **Create the version** — `POST …/versions` puts it in `uploading` state.
2. **Init a file** — `POST …/upload/init` with the file's path, size, and SHA-256; the server returns an `upload_id`, a recommended `part_size`, and `part_count`.
3. **Push parts** — `PUT …/upload/{upload_id}/parts/{n}` for each chunk; each returns an `etag`.
4. **Complete** — `POST …/upload/{upload_id}/complete` with the part list; the file is assembled and checksum-verified.

The SDK's upload session does all of this for you — see
[Uploading a model](/docs/scaiatlas/guides/uploading-a-model). Default chunk size is
10 MB; you can override it.

## Downloads resume and verify

Downloads support HTTP **range** requests, so an interrupted transfer restarts from
where it stopped rather than from zero:

- `GET …/download` returns file metadata and URLs for a version (optionally a specific `variant` or `file`).
- `GET …/download/files/{path}` streams a file, honoring a `Range: bytes=start-end` header and replying `206 Partial Content` with a `Content-Range`. It also sets `X-Checksum-SHA256` and `Accept-Ranges: bytes`.

The SDK verifies each file's SHA-256 after download and skips files already present
with a matching checksum. See [Downloading a model](/docs/scaiatlas/guides/downloading-a-model).

## Variants: many formats, one version

A version's original upload is its **primary** variant. You can request additional
variants — e.g. a GGUF quantization of a safetensors original — with
`POST …/variants`, which returns `202 Accepted` and creates the variant in `pending`
state while conversion runs (`pending → converting → available`). Downloads can
target a specific variant with the `variant` parameter.

## Peer-to-peer distribution

For clusters that pull the same model repeatedly, ScaiAtlas can distribute via P2P
instead of every node hitting origin storage.

### Manifests

A **distribution manifest** breaks a version into fixed-size chunks (default 64 MB),
each with its own checksum:

```bash
POST …/versions/{version}/manifest    # generate (201)
GET  …/versions/{version}/manifest    # fetch
```

The manifest lists every file, its chunks, and per-chunk offsets/sizes/checksums —
enough for a peer to fetch chunks from wherever they're available and reassemble with
integrity.

### The tracker

When P2P is enabled, peers announce what they hold and discover each other through
the tracker under `/api/v1/p2p`:

- `POST /p2p/announce` — a peer advertises its address and which version chunks it has (seeding vs. still leeching).
- `GET /p2p/peers/by-version?version_id=…` — find peers holding a version, split into seeders and leechers.
- `GET /p2p/stats` — tracker-wide counts.

If P2P is disabled on a deployment, these endpoints return `503`.

> P2P is an optimization layer. Correctness never depends on it — origin storage is
> always authoritative, and every chunk is checksum-verified on arrival.