Service Configuration
Customize how your services run in the cloud using infrastructure.json.
Overview
The .tsdevstack/infrastructure.json file lets you override default settings per environment:
Creating infrastructure.json
If you don't have an infrastructure.json file yet, create it in your .tsdevstack directory:
Then add the minimum required content:
IDE Autocomplete
The $schema property enables intelligent autocomplete and validation in VS Code and other editors that support JSON Schema.
What you get:
- Autocomplete for all configuration options (CPU, memory, database tiers, etc.)
- Inline validation for invalid values
- Hover documentation for each property
How it works:
- Add
"$schema": "./infrastructure.schema.json"as the first property in yourinfrastructure.json - The framework includes
infrastructure.schema.jsonin the.tsdevstackdirectory - Your IDE automatically provides suggestions and validation
If autocomplete isn't working, ensure the infrastructure.schema.json file exists in your .tsdevstack directory. Run npx tsdevstack infra:init --env <environment> to generate it.
Service Options
The JSON schema is provider-aware — IDE autocomplete will only show values valid for your cloud provider.
On AWS, a service, Next.js app or worker without cpu and memory runs at 0.25 vCPU / 512 MiB, the smallest Fargate size. If you set only one of the two, the other comes from 0.5 vCPU / 1 GiB. Fargate runs only certain pairs (0.25 vCPU with 512Mi, 1Gi or 2Gi; 0.5 with 1Gi to 4Gi; 1 with 2Gi to 8Gi; 2 with 4Gi to 16Gi; 4 with 8Gi to 30Gi), and infra:generate stops at a pair it can't run, the merged one included.
On Azure, Next.js apps run on App Service, not Container Apps: cpu and memory don't apply to them, and their instance counts follow the plan's SKU. See Azure App Service plans.
Cloud Run rejects some combinations, so infra:generate, infra:plan, the deploy commands and infra:status check them first:
cpubelow 1 needs"concurrency": 1(each instance then serves one request at a time). Workers and Kong needcpu1 or more.- A NestJS service with
minInstances1 or more needscpu1 or more: its CPU stays on between requests (see minInstances), which Cloud Run allows from 1 vCPU. - Memory per CPU: up to
512Mibelow 0.5 vCPU,1Gibelow 1,4Giat 1,8Giat 2. 4 vCPU needs at least2Gi, 8 vCPU at least4Gi. :::
Scaling Configuration
minInstances
Controls the minimum number of instances always running.
1 or more: the service always runs. It answers requests, and it keeps working between them: message handlers (@OnMessage), timers, anything that waits on Redis. On GCP, a NestJS service with minInstances 1 or more keeps its CPU between requests (Cloud Run's instance-based billing). Cloud Run otherwise gives a service CPU only while it handles a request, so its message handlers would wait for the next request. That costs more: about $50 a month instead of about $10 for one instance at 1 vCPU and 512 MiB (list prices). Next.js apps get CPU only during requests.
0: the service scales to zero when idle. A good fit for dev and staging, which sit idle most of the time. Problematic for production:
- Messages for the service wait in Redis until a request starts it. Nothing is lost, but the work happens late: a user signs up, and the service that gives them their plan does it on its next request.
- The first request after an idle period waits for a cold start.
- Between requests the service has no CPU. Work a request started and didn't finish pauses until the next request, and its traces arrive late.
Every service that subscribes to messages should keep minInstances at 1 or more. Workers always run.
Scale-to-zero is a platform feature of Cloud Run and Container Apps, and cold start times vary:
On Azure, scale-to-zero covers the services on Container Apps. Kong and Next.js apps run on App Service, which cannot scale to zero: minInstances: 0 runs one instance there.
:::warning AWS has no scale-to-zero
On AWS every ECS service (NestJS services, Next.js frontends, workers and Kong) runs at least one task. minInstances: 0 fails validation in infra:generate, infra:plan, infra:deploy and infra:status, with a hint to set 1 or more. Backend services on AWS are reachable only inside the VPC, so there is no public entry point that could wake a stopped service. See AWS Cost Estimation for what always-on costs.
Recommendations:
- Dev and staging on GCP or Azure:
0saves money. Messages and background work wait for the next request, and cold starts are acceptable - Production:
1+(the default) for every service - Detached workers:
1+- Workers must always be running to poll Redis queues. On GCP and Azure the framework raises0to 1 for workers; on AWS it fails validation like any other service
maxInstances
Limits how many instances can run during high traffic.
- Higher values handle more concurrent users
- Each instance costs money while running
- The container runtime auto-scales based on CPU utilization and request queue. On AWS, NestJS services, Next.js frontends, workers and Kong scale on CPU (target tracking at 70%) between
minInstancesandmaxInstances - On Azure, Kong and Next.js apps only autoscale on a Standard or Premium App Service plan. See Azure App Service plans
Example: If each instance handles 80 concurrent requests and you expect 800 peak concurrent users, set maxInstances: 10 minimum.
Resource Configuration
CPU
Memory
GCP has the widest range (256Mi–8Gi). AWS supports larger instances (up to 16Gi). Azure is the most constrained (0.5Gi–4Gi, no sub-Gi values).
Kong Gateway
Configure the API gateway separately:
Kong defaults:
Keep Kong at minInstances: 1 or more: every API request goes through it. On AWS, 0 fails validation.
On AWS, Kong without cpu and memory runs at 0.25 vCPU / 1 GiB. Its task also holds the trace collector, which takes 256 MiB of it. If you set only one of the two, the other comes from 1 vCPU / 2 GiB.
On Azure, leave out cpu and memory: Kong runs on an App Service plan, and the plan's SKU sets them (setting them fails validation). Kong's maxInstances only takes effect on a Standard or Premium plan. See Azure App Service plans.
Azure App Service plans
On Azure, Kong and the Next.js apps run on App Service: Kong on its own plan, and every Next.js app on a second, shared plan. The plan's SKU sets CPU and memory per instance, and decides what minInstances and maxInstances do.
Here Kong autoscales between 1 and 4 instances, and the frontend Next.js app runs a fixed 2.
- At least one instance. A plan runs
minInstancesinstances. App Service cannot scale to zero, sominInstances: 0runs one. - Autoscale needs Standard or Premium. When
maxInstancesis aboveminInstances, the plan gets an autoscale rule: one more instance when average CPU stays above 70% for 5 minutes, one fewer when it stays below 30% for 10 minutes. A redeploy keeps the current count. - Basic plans run a fixed count. A
maxInstancesaboveminInstanceson a Basic plan fails validation. With the defaults (KongmaxInstances: 5onB1),infra:generatenotes the fixed count and names the SKU that autoscales. - Next.js apps share one plan. It uses the largest
minInstancesandmaxInstancesamong them. - Health checks with more than one instance. App Service pings Kong's
/statusand each Next.js app's/healthevery minute. An instance that fails for 2 minutes leaves rotation; one that stays unhealthy for an hour is replaced. At most half of an app's instances leave rotation at a time. - No sticky sessions. Requests spread across instances. Kong and the Next.js apps keep no session state.
- Limits are checked before deploy. Counts above the tier's maximum, and
cpuormemoryon Kong or a Next.js app, fail validation ininfra:generate,infra:planandinfra:deploy. For more CPU or memory per instance, pick a bigger SKU (B2,S2,P1v3and up).
App Service bills per instance, so every instance a plan runs costs as much as the first. Make sure your subscription's App Service quota covers the largest count a plan can reach: see Azure account setup and Azure cost estimation.
Detached Workers
Configure BullMQ worker scaling and resources:
Workers support the same options as services (minInstances, maxInstances, cpu, memory). On GCP, workers need cpu 1 or more: they keep their CPU allocated between requests, which Cloud Run allows from 1 vCPU (the 0.25 above is for AWS). The framework enforces minInstances >= 1 because workers must always be running to poll Redis queues: on GCP and Azure 0 is overridden to 1, on AWS it fails validation.
A worker adds instances when its average CPU goes above a target, up to maxInstances, and removes them when it drops: 70% on AWS and Azure, and Cloud Run's own 60% on GCP (workers there keep their CPU allocated, so Cloud Run counts their CPU even between requests).
Workers scale on CPU, not on the number of jobs waiting. So match the job to the knob:
- A job that waits on the network (sending an email, calling an API) barely uses the CPU, so adding workers never triggers. Let one worker run several at once with the processor's
concurrency. If the API has a rate limit, add alimiter: it holds the whole queue, across every worker, and jobs over it wait. - A job that uses the CPU (resizing images, building reports) keeps the default of one at a time per worker, and more work adds workers.
The auth template's email processor does the first:
If a queue still falls behind, raise concurrency (within the API's limit) or minInstances.
Changing a worker's maxInstances affects the database connection pool calculation. After changing scaling config, redeploy all services and workers to rebalance pool sizes.
Database
Configure managed PostgreSQL:
High availability (ha) keeps a standby that takes over when the primary fails. It roughly doubles the database cost.
- GCP: regional Cloud SQL instance. Needs
backup: true. - AWS: Multi-AZ, with the standby in another Availability Zone.
- Azure: zone-redundant standby. Only on General Purpose (
GP_) and Memory Optimized (MO_) tiers, and only in regions with availability zones.infra:deploychecks the region before it starts.
Backups (backup):
- GCP:
falseturns automated backups and point-in-time recovery off. - AWS:
falseturns automated backups and point-in-time recovery off. The final snapshot oninfra:destroyis still taken. - Azure: backups can't be turned off.
truekeeps them for 35 days,falsefor 7 (the minimum).
Deletion protection (deletionProtection):
- GCP and AWS: the provider's own deletion protection on the instance.
infra:destroyturns it off before it deletes anything. - Azure: a delete lock on the PostgreSQL server (
tsdevstack-database-delete-lock), put on byinfra:deploy. The lock also covers the databases on the server.infra:remove-servicelifts it while it deletes a service's database and puts it back after.infra:destroylifts it first, and stops before deleting anything if it can't. Withfalse, the next deploy takes the lock off before it applies. - Azure: with the lock on, a deploy that would delete a database (a service whose
hasDatabaseyou turned off, or a service you took out ofconfig.jsonwithoutinfra:remove-service) fails at that step. To drop the database on purpose, setdeletionProtectiontofalsefor that deploy, then back totrue.
On AWS, changes to ha, backup and tier on a running database are applied in the next maintenance window (Mondays 04:00 to 05:00 UTC). Turning backups on or off causes a short outage.
Database tiers by provider (the full list each provider accepts; deploys size the connection pool from each tier's connection limit, and infra:generate and the deploy commands reject other values):
GCP (Cloud SQL):
db-f1-micro- Development (shared CPU)db-g1-small- Small production (shared CPU)db-n1-standard-1- Standard production (1 vCPU)db-n1-standard-2- Larger production (2 vCPU)db-n1-standard-4- High-traffic production (4 vCPU)db-n1-standard-8,db-n1-standard-16- 8 and 16 vCPU
AWS (RDS PostgreSQL):
db.t3.micro- Development (~$15/mo)db.t3.small,db.t3.medium- Small production (burstable)db.t4g.micro,db.t4g.small,db.t4g.medium- Burstable, ARM (Graviton)db.r6g.large- Memory optimized, ARM (Graviton)
Azure (PostgreSQL Flexible Server):
B_Standard_B1ms- Development (burstable, ~$14/mo)B_Standard_B2s,B_Standard_B2ms- Small production (burstable)GP_Standard_D2s_v3,GP_Standard_D2ds_v5- Standard production (general purpose)GP_Standard_D4s_v3,GP_Standard_D4ds_v5- High-traffic production (general purpose)MO_Standard_E4s_v3,MO_Standard_E4ds_v5- Memory optimized
The _v5 sizes are Azure's current generation, with the same vCores, memory and connection limits as their _v3 twins. A region can offer one generation to your subscription and not the other (see PostgreSQL Region Access).
Data of a removed service
infra:remove-service saves the service's database before it drops it. The file goes to a private bucket of the environment, made by infra:deploy, and is deleted after 7 days. infra:destroy deletes the bucket with the environment. Keeping a file costs less than a cent per GB.
The bucket is a Cloud Storage bucket on GCP, an S3 bucket on AWS, and a container in a storage account of its own on Azure. Only the database job reads and writes the files.
To bring the data back, deploy the service again (cloud-secrets:push, then infra:deploy), then run:
Without that file (removed with --skip-export, or more than 7 days ago), the data is only in the database server's automated backups (see backup above). Each backup covers the whole server, not one database. Restore a backup into a new server, at a time just before the removal. Then copy the service's database from it into the live server, and delete the new one:
- GCP: clone the Cloud SQL instance with point-in-time recovery.
- AWS: a point-in-time restore of the RDS instance. It creates a new instance.
- Azure: a point-in-time restore of the Flexible Server. It creates a new server.
On GCP, don't restore a backup onto the live instance: that rolls back every service's database to the backup's time.
Custom server name (Azure)
On Azure the PostgreSQL Flexible Server name is a global DNS name (<name>.postgres.database.azure.com), so it has to be unique across all of Azure, not only in your subscription. tsdevstack names it {project}-{env}-postgres-{suffix}, the suffix 6 characters from the subscription ID (environments set up with v0.12.0 or earlier: {project}-{env}-postgres). If someone else already holds that name, the deploy fails with ServerNameAlreadyExists. Typical causes: another project uses the same project name, or a server with that name still exists in a subscription you deleted.
Pick a different name for that environment with serverName:
Rules: 3 to 63 characters, lowercase letters, digits and hyphens, no hyphen at the start or the end. Set it per environment; environments without it keep the default name. Your services' connection strings follow on their own, since the deploy builds them from the server address Terraform reports.
The field is Azure only. On GCP and AWS the schema flags it in your editor and infra:status reports it as an error.
Set serverName before the first deploy of an environment. Changing it on an environment that is already running makes Terraform replace the server: the old server and every database on it are destroyed and a new, empty one is created. With deletionProtection on (the default), the delete lock makes that deploy fail instead. If you really need to rename a live server, back up first (pg_dump) and check npx tsdevstack infra:plan before you deploy.
Redis
Configure managed Redis:
GCP (Memorystore):
BASIC- Single instance, no failoverSTANDARD_HA- High availability with automatic failover
AWS (ElastiCache):
cache.t3.micro- Development (~$12/mo)cache.t3.small- Small productioncache.r6g.large- Standard production
Azure (Managed Redis):
Balanced_B0- Development (~$26/mo for two nodes, clustered)Balanced_B1- Small productionBalanced_B3- Standard production
Azure Managed Redis uses EnterpriseCluster policy. BullMQ requires a {bull} prefix to avoid CROSSSLOT errors — see Azure Architecture for details.
Frontend Domains
Configure domains for frontend services:
Load Balancer
Configure domain redirects and API domain:
When you own multiple domains (e.g., .com, .io, .app), add alternates to redirectDomains. All traffic to those domains redirects to your canonical domain set in the DOMAIN secret.
Access Control
Keep non-production environments out of search engines:
Environment password protection was removed. The accessControl.protected and accessControl.cookieTtlHours fields are no longer accepted, and a config that still sets them fails validation. Remove them from infrastructure.json.
Scheduled Jobs
Configure cron-based scheduled jobs in the scheduledJobs array. See the Scheduled Jobs guide for full configuration, service-side implementation, authentication, and provider architecture.
With the auth template, every environment needs the sync-api-key-usage job; infra:generate warns and prints the entry when it is missing. See API Keys.
WAF Rules
The framework includes default WAF rules for common attacks (SQL injection, XSS, path traversal, command injection, SSRF, scanner fingerprints, and more). You can customize the global rate limit and add custom rules per provider.
Global rate limit
Override the default rate limit (1000 requests per 60 seconds per IP) across all providers:
The framework translates this to each provider's native format:
- GCP: Direct pass-through to Cloud Armor throttle rule
- AWS: Scaled to 5-minute window (AWS minimum). 1000/60s becomes 5000 per 5 minutes
- Azure: Scaled to per-minute buckets. 500/30s becomes 1000 per 1 minute
Custom rules (GCP)
GCP uses CEL expressions for custom rules:
Custom rules (AWS)
AWS uses statement-based rules with byte match, rate-based, and geo match types:
Custom rules (Azure)
Azure uses match conditions with operators and transforms:
Match condition options:
Schema validation
The infrastructure.schema.json is provider-aware — it only shows the custom rule format relevant to your cloud provider. IDE autocomplete will guide you to the correct syntax.
Upload Size Limits
Configure the maximum traditional upload size through Kong:
This sets Kong's client_max_body_size. Requests exceeding this limit receive a 413 Content Too Large response.
Hard ceilings per provider (cannot be exceeded even with higher configuration):
For files larger than these limits, use presigned URL uploads which bypass the WAF/LB/Kong chain entirely. See Object Storage for details.
Tracing
Every request is traced by default. tracing.sampleRate sets the share of new traces each environment keeps:
Trace volume drives the trace bill at high traffic. See Traces and Dashboards.
Cost Optimization
Development environments
A GCP example: the database and Redis tiers are GCP's. Kong stays at 1, because every API request goes through it. On AWS nothing scales to zero: minInstances: 0 fails validation there.
Production environments
Applying Changes
After modifying infrastructure.json:
Changes are applied on the next deployment. Some changes (like database tier) may cause brief downtime.