Storage & secrets

Persistent volumes, named secrets, KV stores, and cloud bucket mounts.

Volumes

Persistent block-storage volumes. Data survives across deploys, retries, and restarts. Use for model weights, datasets, and checkpoints. Create volumes in the dashboard or with the CLI.

python
1models = gw.Volume.from_name("models", region="US")2 3@app.task(gpu=gw.Gpu("A100"), volumes={"/models": models})4async def load_model():5    import torch6    model = torch.load("/models/checkpoint.pt")7    return model

Secrets

Secrets are stored in the tenant AES-GCM vault and injected as environment variables at task startup. Create with `gworker resource create secret MY_SECRET`. Never commit secrets to source.

python
1@app.task(secrets=[2    gw.Secret.from_name("hf-token"),3    gw.Secret.from_name("openai-key"),4])5async def run():6    import os7    token = os.environ["HF_TOKEN"]  # injected automatically

KV store

Distributed key-value store accessible from any task or stage. Useful for caching results, counters, and cross-task coordination.

python
1cache = gw.KV.from_name("embeddings-cache")2 3@app.task(kvs=[cache])4async def embed(text: str) -> list[float]:5    # read cached result6    cached = await cache.get(text)7    if cached:8        return cached9    vec = compute_embedding(text)10    await cache.set(text, vec, ttl=3600)11    return vec

Cloud bucket mounts

Mount an S3 or GCS bucket as a filesystem path inside the container. Uses FUSE — higher latency than a Volume, but good for large datasets you don't want to copy.

python
1dataset = gw.CloudBucketMount.from_s3(2    bucket="my-training-data",3    mount_path="/data",4    secret="aws-creds",  # secret containing AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY5    read_only=True,6)7 8@app.task(gpu=gw.Gpu("A100"), cloud_mounts=[dataset])9async def train():10    import os11    files = os.listdir("/data")  # S3 objects appear as files12    ...