Storage Engines

filestore ships with five storage engines. All share the same interface — you switch backends by swapping the engine instance passed to FileStore.

Engines are configured once at construction and reused across requests. Cloud engines create their SDK client lazily and cache it, so there is no per-request client setup cost.

Engine	Extra	Presigned URLs	Delete
LocalEngine	—	✗	✓
MemoryEngine	—	✗	✗
S3Engine	s3	✓	✓
GCSEngine	gcp	✓	✓
AzureBlobEngine	azure	✓	✓

Local Engine

Writes files to the local filesystem. This is the default engine when none is given.

from filestore import FileStore, LocalEngine

storage = FileStore(
    "document",
    engine=LocalEngine(
        base_dir="uploads/documents",
        base_url="/media/documents",
    ),
)

Features

• Atomic writes — files are written to a temporary file first, then renamed into place. No partial uploads on crash.
• Collision handling — when overwrite=False (the default), a numeric suffix is appended: doc.pdf → doc-1.pdf → doc-2.pdf.
• Streaming size checks — max_file_size is enforced while writing; oversized uploads are aborted mid-stream and cleaned up.
• Directory creation — destination directories are created automatically.
• Safe deletion — await engine.delete(key) resolves against base_dir and rejects keys that escape it.

Constructor

Argument	Default	Description
base_dir	Current directory	Directory used when config has no destination; root for delete()
base_url	None	Default public URL prefix (config base_url takes precedence)

Result

The returned FileData includes:

• path — absolute Path to the written file
• key — path relative to the destination directory
• url — public URL (if base_url was set)

---

Memory Engine

Reads the entire file into memory. No disk I/O. Useful for pipelines that process the payload without persisting it.

from filestore import FileStore, MemoryEngine

storage = FileStore("image", count=3, engine=MemoryEngine())

Result

• file — the raw bytes payload (excluded from JSON serialization)
• path and url are None

---

S3 Engine

Upload to Amazon S3 or any S3-compatible service (MinIO, LocalStack, Cloudflare R2, DigitalOcean Spaces, etc.).

Requires extra

Install with pip install "filestore[s3]"

from filestore import FileStore, S3Engine

storage = FileStore(
    "asset",
    engine=S3Engine(bucket="my-bucket", region="us-east-1"),
    config={"destination": "uploads/assets"},  # S3 key prefix
)

Credentials

Credentials are resolved through the standard boto3 chain — environment variables, ~/.aws/credentials, or IAM roles. The engine always signs with SigV4, which is required for presigned URLs against modern buckets.

S3-Compatible Services

engine = S3Engine(
    bucket="my-bucket",
    endpoint_url="http://localhost:9000",  # MinIO
)

Constructor

Argument	Env fallback	Description
bucket	AWS_BUCKET_NAME	Required. S3 bucket name
region	AWS_DEFAULT_REGION	AWS region
endpoint_url	—	Custom endpoint for S3-compatible services
client	—	Pre-built boto3 client (dependency injection / testing)

Result

• url — public S3 URL
• key — full S3 object key
• metadata["bucket"] — bucket name

The file's content type is set as the object's ContentType; use extra_args for additional put_object() kwargs (e.g. {"ACL": "public-read"}).

---

Google Cloud Storage Engine

Upload to GCS buckets.

Requires extra

Install with pip install "filestore[gcp]"

from filestore import FileStore, GCSEngine

storage = FileStore(
    "asset",
    engine=GCSEngine(bucket="my-gcs-bucket", project="my-project-id"),
    config={"destination": "uploads/assets"},
)

Credentials

Uses Application Default Credentials by default. Pass credentials= for an explicit credentials object.

Signed URLs

V4 signed URLs require credentials with a private key — typically a service account. Workload-identity tokens alone cannot sign.

Constructor

Argument	Env fallback	Description
bucket	GCP_BUCKET_NAME	Required. GCS bucket name
project	GCP_PROJECT / GOOGLE_CLOUD_PROJECT	Google Cloud project ID
credentials	ADC	Explicit credentials object
endpoint_url	—	Custom endpoint for emulators
client	—	Pre-built storage.Client

Behaviour

• When overwrite=False (default), uploads use if_generation_match=0 so existing objects are never clobbered.

Result

• url — public GCS URL
• key — full object key
• metadata["bucket"] — bucket name

---

Azure Blob Engine

Upload to Azure Blob Storage containers.

Requires extra

Install with pip install "filestore[azure]"

from filestore import AzureBlobEngine, FileStore

storage = FileStore(
    "asset",
    engine=AzureBlobEngine(
        container="my-container",
        connection_string="UseDevelopmentStorage=true",
    ),
    config={"destination": "uploads/assets"},
)

Authentication

Connection StringAccount URL + Managed Identity

engine = AzureBlobEngine(
    container="my-container",
    connection_string="DefaultEndpoints...",
)

engine = AzureBlobEngine(
    container="my-container",
    account_url="https://myaccount.blob.core.windows.net",
    # Uses DefaultAzureCredential automatically
)

SAS URLs

Presigned (SAS) URLs require an account key, so use connection-string or shared-key authentication if you plan to presign.

Constructor

Argument	Env fallback	Description
container	AZURE_STORAGE_CONTAINER	Required. Blob container name
connection_string	AZURE_STORAGE_CONNECTION_STRING	Connection-string auth
account_url	AZURE_STORAGE_ACCOUNT_URL	Account URL auth
credential	—	Explicit credential object
client	—	Pre-built BlobServiceClient

Result

• url — blob URL
• key — full blob name
• metadata["container"] — container name
• metadata["etag"] — ETag (when available)

---

Per-Field Engines

Different fields in the same request can use different engines:

from filestore import FileField, FileStore, MemoryEngine, S3Engine

storage = FileStore(
    fields=[
        FileField(name="archive", engine=S3Engine(bucket="archives")),
        FileField(name="thumbnail"),  # falls back to the store engine
    ],
    engine=MemoryEngine(),
)

Custom Engines

Subclass StorageEngine and implement save() — see the Storage Engines API reference.