Infrastructure capacity and costs
Explains infrastructure bottlenecks and scaling costs using actual connection, memory and CPU constraints rather than server counts alone.
capacity-infrastructure-costs
See how it helpsWhen is it useful?
Use when adding servers increases costs without resolving latency or outages.
What does it help you do?
- Finds database connection limits, memory pressure and CPU throttling.
- Relates instance counts and pool sizes to downstream capacity.
- Proposes scaling limits and cost controls around the measured bottleneck.
Try asking this
Explain why doubling application containers exhausts database connections and propose a bounded capacity configuration. Calculate the total connection demand from every replica and worker. Set a tested pool limit and show the remaining database headroom.
Detailed documentation
Read the skill’s original instructions, examples and checks in their authored language.
Show less
Capacity & Infrastructure Costs
Right-size infrastructure from measured workload characteristics rather than guesswork, and guard every auto-scaling or pay-per-use resource with hard upper ceilings so a traffic spike, retry loop, or runaway job cannot cause an outage or a billing shock.
Bottleneck Diagnosis (The USE Method)
Before scaling up hardware or adding replicas, check Utilization, Saturation, and Errors for every resource along the request path:
- Database & Connection Pools (Most Common Real Bottleneck):
- Adding 20 application containers that each open a 20-connection pool (
400connections) will overwhelm a database configured formax_connections = 100. - Size per-instance pool limits so $\text{max_instances} \times \text{pool_size} + \text{admin_reserve} < \text{DB max_connections}$, or place a connection pooler (PgBouncer / RDS Proxy / Supavisor) in front of the database.
- Adding 20 application containers that each open a 20-connection pool (
- Memory Working Set vs. Limits (OOMKills):
- Monitor container memory working set (
container_memory_working_set_bytes) and language runtime heap limits (e.g., Node--max-old-space-sizeset to ~75–80% of container memory limit so GC runs before the Linux kernel OOM-killer terminates the process with exit code137).
- Monitor container memory working set (
- CPU Throttling vs. Average CPU:
- Check container CFS throttling (
container_cpu_cfs_throttled_periods_total) and event-loop lag alongside average CPU utilization; bursty workloads can experience severe tail-latency spikes even at 40% average CPU if limits are overly tight.
- Check container CFS throttling (
Safe Autoscaling & Cost Guardrails
| Cost / Capacity Risk | Mitigation to Enforce |
|---|---|
| Unbounded Horizontal Autoscaling | Always set a hard maxReplicas / max_instances cap paired with edge rate-limiting and queue backpressure so a DDoS or retry storm cannot spin up hundreds of instances. |
| Network Egress & NAT Gateway Multipliers | Serve static assets, media, and large downloads through a CDN with proper Cache-Control headers (immutable for fingerprinted assets); use VPC endpoints / private networking for high-volume internal storage/DB traffic. |
| Log, Trace & Metric Cardinality Explosion | Never put unbounded user IDs, URLs with query strings, or UUIDs into metric labels (Prometheus/Datadog tag cardinality explosion). Sample high-volume debug/info traces and set retention TTLs on log buckets. |
| Orphaned Storage & Preview Environments | Automate teardown of ephemeral PR preview environments, unattached block volumes, old container registry tags, and expired build artifacts via lifecycle rules. |
When proposing an infrastructure or scaling change, state the expected capacity headroom, failure mode at ceiling, and estimated monthly cost delta explicitly.
Included files 1
SKILL.md