For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt, and this page is available as Markdown at /docs/features/object-storage.md.

Object Storage

tsdevstack provides unified object storage across all cloud providers. Add a bucket, import StorageModule, and your code works identically on local RustFS, AWS S3, GCP Cloud Storage, and Azure Blob Storage.

Quick start

Add a storage bucket to your project:

npx tsdevstack add-bucket-storage --name uploads

This updates config.json. Then sync to regenerate docker-compose, secrets, and start RustFS:

npx tsdevstack sync

Import the module in your NestJS service:

import { Module } from '@nestjs/common';
import { StorageModule } from '@tsdevstack/nest-common';

@Module({
  imports: [
    StorageModule.forRoot({ buckets: ['uploads'] }),
  ],
})
export class AppModule {}

Use it in a service:

import { Injectable } from '@nestjs/common';
import { InjectStorage } from '@tsdevstack/nest-common';
import type { StorageProvider } from '@tsdevstack/nest-common';

@Injectable()
export class FileService {
  constructor(
    @InjectStorage('uploads') private readonly storage: StorageProvider,
  ) {}

  async uploadFile(key: string, data: Buffer, contentType: string): Promise<void> {
    await this.storage.upload(key, data, contentType);
  }

  async getDownloadUrl(key: string): Promise<string> {
    return this.storage.getPresignedUrl(key, 3600); // 1 hour
  }
}

How it works

The StorageProvider interface is the same across all environments. The adapter is selected automatically at runtime based on the SECRETS_PROVIDER environment variable:

SECRETS_PROVIDERStorage AdapterUsed When
localS3 (RustFS)Local development
awsS3 (AWS)AWS cloud
gcpGCSGoogle Cloud
azureAzure BlobAzure cloud

No code changes are needed between environments. The same StorageProvider calls work everywhere.

Local development (RustFS)

When storage buckets are configured, docker-compose.yml includes RustFS, an S3-compatible object storage server (Apache-2.0).

ServiceURLPurpose
RustFS APIhttp://localhost:9000S3-compatible API endpoint
RustFS consolehttp://localhost:9001Web-based storage browser

Default credentials: rustfsadmin / rustfsadmin

What happens on docker compose up

  1. RustFS container starts on ports 9000 + 9001
  2. rustfs-init runs once and creates the configured buckets with s5cmd
  3. Bucket names follow the cloud naming pattern: {project}-{name}-dev

Data persistence

RustFS data is stored in ./data/rustfs/ (bind mount), the same pattern used by PostgreSQL (./data/{name}/) and Redis (./data/redis/). Data persists across container restarts.

StorageModule setup

Static configuration

Use forRoot when bucket names are known at compile time:

StorageModule.forRoot({ buckets: ['uploads', 'media-assets'] })

Async configuration

Use forRootAsync when you need dynamic configuration (e.g., reading from SecretsService during initialization):

StorageModule.forRootAsync({ buckets: ['uploads', 'media-assets'] })

Both register the module globally — import once in AppModule and @InjectStorage works everywhere.

Multi-bucket access

For services that need multiple buckets, inject each one separately:

@Injectable()
export class MediaService {
  constructor(
    @InjectStorage('uploads') private readonly uploads: StorageProvider,
    @InjectStorage('media-assets') private readonly media: StorageProvider,
  ) {}
}

Or use StorageService to access any bucket dynamically:

import { Injectable } from '@nestjs/common';
import { StorageService } from '@tsdevstack/nest-common';

@Injectable()
export class DynamicService {
  constructor(private readonly storageService: StorageService) {}

  async copyToMedia(key: string): Promise<void> {
    const uploads = this.storageService.getProvider('uploads');
    const media = this.storageService.getProvider('media-assets');

    const data = await uploads.download(key);
    await media.upload(key, data);
  }
}

File Uploads

There are two approaches to handling file uploads. Choose based on your requirements.

Traditional upload (through backend)

Files flow through the full stack: Client → Next.js → Kong → NestJS → Storage.

@Post('upload')
@UseInterceptors(FileInterceptor('file'))
async uploadFile(
  @UploadedFile() file: Express.Multer.File,
): Promise<{ url: string }> {
  await this.storage.upload(`files/${file.originalname}`, file.buffer, file.mimetype);
  const url = await this.storage.getPresignedUrl(`files/${file.originalname}`, 3600);
  return { url };
}

Use when: You need server-side validation, custom naming, synchronous post-processing, or atomic database + file operations.

Size limits:

LayerDefaultConfigurable
Kong (client_max_body_size)10MBkong.maxUploadSize in infrastructure.json
GCP Cloud Run32MBNo (hard ceiling)
AWS ALB / ECS FargateNo documented limit—
Azure Front Door100MB (Standard)Tier-dependent
NestJS / MulterNo defaultlimits.fileSize on FileInterceptor

Configure the Kong limit in infrastructure.json:

{
  "prod": {
    "kong": {
      "maxUploadSize": "25m"
    }
  }
}

Presigned URL upload (direct to storage)

Files go directly from the client to the storage provider, bypassing WAF, load balancer, Kong, and NestJS entirely.

// Backend: generate a presigned upload URL
const uploadUrl = await this.storage.getPresignedUrl('files/report.pdf', {
  action: 'put',
  expiresInSeconds: 3600,
  contentType: 'application/pdf',
});
return { uploadUrl };

// Client: upload directly to the URL
await fetch(uploadUrl, {
  method: 'PUT',
  body: file,
  headers: { 'Content-Type': 'application/pdf' },
});

Use when: Files are large (>10MB), you want to avoid server-side memory/CPU load, or you need to exceed provider WAF/LB size limits.

Key details:

  • Works across all providers (GCS v4 signatures, S3 presigned URLs, Azure SAS tokens)
  • Maximum URL expiration: 7 days (all providers), configurable via expiresInSeconds
  • No size limit imposed by WAF or Kong — the file goes directly to the storage bucket
  • The backend only handles URL generation (fast, no file data in memory)

StorageProvider API

All methods are available on every provider adapter.

MethodSignatureDescription
upload(key: string, data: Buffer | Readable, contentType?: string) => Promise<void>Upload a file. Supports Buffer or stream.
download(key: string) => Promise<Buffer>Download a file as Buffer.
downloadStream(key: string) => Promise<Readable>Download a file as a readable stream. Use for large files.
delete(key: string) => Promise<void>Delete a file. No error if the file doesn't exist.
list(prefix?: string) => Promise<StorageObject[]>List files, optionally filtered by prefix.
copy(sourceKey: string, destinationKey: string) => Promise<void>Copy a file within the same bucket.
getMetadata(key: string) => Promise<StorageMetadata>Get file metadata (size, content type, last modified).
getPresignedUrl(key: string, expiresInSeconds?: number) => Promise<string>Generate a temporary download URL. Default expiry varies by provider.
exists(key: string) => Promise<boolean>Check if a file exists.
getNativeClient() => S3Client | Storage | BlobServiceClientGet the underlying provider SDK client for advanced operations.

Types

interface StorageObject {
  key: string;
  size: number;
  lastModified: Date;
}

interface StorageMetadata {
  contentType: string;
  contentLength: number;
  lastModified: Date;
  etag?: string;
}

Bucket naming

PropertyRule
Logical nameKebab-case, 2-30 characters (e.g., uploads, media-assets)
Cloud name (GCP){project}-{name}-{env}-{suffix}, the suffix 6 characters from the GCP project ID (max 63 characters). Environments first deployed with v0.12.0 or earlier keep {project}-{name}-{env}
Cloud name (Azure){project}-{name}-{env} (max 63 characters)
Cloud name (AWS){project}-{name}-{env}-{accountId} (max 63 characters)
Azure storage account{project}{env}st{suffix}, the suffix 6 characters from the subscription ID (alphanumeric only, max 24 characters). Environments set up with v0.12.0 or earlier keep {project}{env}storage
Reserved namesrustfs, minio, storage, gateway, kong, redis, postgres, etc.

Cloud bucket names are derived automatically. You only specify the logical name. add-bucket-storage keeps room for the longest account part (AWS's 13 characters), so {project}-{name}-{env} can have at most 50.

Secrets

Storage secrets are framework-managed and shared-scope — they are auto-assigned to all services via secret-map.json. You never need to manually add them to .secrets.user.json.

Local secrets

SecretValuePurpose
STORAGE_ENDPOINThttp://localhost:9000RustFS API endpoint
STORAGE_ACCESS_KEYrustfsadminRustFS root access key
STORAGE_SECRET_KEYrustfsadminRustFS root secret key
STORAGE_BUCKET_{NAME}{project}-{name}-devBucket name per configured bucket

Cloud secrets

In cloud environments, only STORAGE_BUCKET_{NAME} secrets exist. Credentials are handled by IAM roles — no access keys in containers:

ProviderAuthenticationSecrets
AWSECS Task Role (IAM)STORAGE_BUCKET_* only
GCPCloud Run service accountSTORAGE_BUCKET_* only
AzureManaged Identity + AZURE_STORAGE_ACCOUNT_NAME env varSTORAGE_BUCKET_* only

For deeper coverage, see How Secrets Work.

Cloud deployment

Running infra:deploy handles everything:

  1. Terraform creates cloud buckets — S3 buckets (AWS), GCS buckets (GCP), or Azure Blob containers
  2. IAM permissions are granted — service roles get read/write access to buckets
  3. Post-Terraform sync pushes STORAGE_BUCKET_* secret values to the cloud secret manager
  4. Services are deployed with access to the new bucket names via SecretsService

Per-provider details

AWSGCPAzure
ResourceS3 bucketGCS bucketBlob container in shared storage account
AccessBlock all public access; requests without HTTPS refusedUniform bucket-level access; public access prevention enforcedPrivate container; HTTPS only, TLS 1.2 or later
Deleted or overwritten filesKept 7 days (versioning)Kept 7 days (soft delete)Kept 7 days (soft delete)
EncryptionAES-256 server-sideGoogle-managedMicrosoft-managed
IAMs3:GetObject/PutObject/DeleteObject/ListBucket on service task rolesroles/storage.objectUser + roles/iam.serviceAccountTokenCreator on each service's own accountStorage Blob Data Contributor on managed identity
CORS***
Access logsCloudTrail data events in CloudWatch LogsCloud Storage audit logs in Cloud LoggingBlob logs in Log Analytics

CORS is set to * on all providers. Buckets are private — presigned URLs are the security boundary, not CORS. Browser access requires a presigned URL obtained via your API (which is protected by Kong's CORS).

Access logs

Every read and write of a bucket's files is logged in the cloud, with who did it and from where. Read them with infra:logs, one source per bucket:

npx tsdevstack infra:logs bucket-uploads --env dev

See Cloud Logs. Each access costs a little; see your provider's cost page. On GCP, the deploy account needs the Logs Configuration Writer role, which infra:bootstrap adds. A project set up with an older CLI runs npx tsdevstack infra:bootstrap --env <env> once.

For provider-specific architecture details, see GCP Architecture, AWS Architecture, or Azure Architecture.

Removing a bucket

npx tsdevstack remove-bucket-storage --name uploads

What it does locally:

  • Removes bucket from config.json
  • Regenerates docker-compose.yml (removes RustFS entirely if this was the last bucket)
  • Regenerates secrets (removes STORAGE_* secrets if this was the last bucket)
  • Does NOT delete local RustFS data — it remains in ./data/rustfs/

What it does NOT do:

  • Does not remove cloud resources (S3 buckets, GCS buckets, Azure Blob containers)
  • Does not remove STORAGE_BUCKET_* secrets from cloud secret managers
  • Does not delete any stored files

To clean up cloud resources:

  1. Run infra:deploy --env {env} to apply Terraform changes
Warning

This permanently deletes all data in the cloud bucket. Back up your data first if needed.

  1. Remove cloud secrets: cloud-secrets:remove STORAGE_BUCKET_UPLOADS --env {env}
  2. If this was the last bucket, also remove StorageModule imports from your NestJS code