Skip to main content

​
Replicas

Each service can run multiple replicas across your cluster. Configure how many replicas run on each server from the service settings. Replica count ranges from 1 to 32 per service. Automatic placement supports two replica-management modes:
  • Fixed keeps the configured replica count running.
  • Autoscaled keeps the active count within a configured minimum and maximum from 1 through 32. CPU and memory each use a fixed 60% target; thresholds are not configurable.
Autoscaling is available only for automatic-placement, stateless, non-serverless services without volumes. Both CPU and memory limits must be configured because utilization is measured against those limits. Saving stages the policy; deploying snapshots it into the immutable revision and activates it. The controller evaluates trailing three-minute CPU and memory averages every minute. Scale-down additionally requires five minutes of complete low-usage observations. Missing or stale metrics hold the current count. A scaling attempt starts a ten-minute cooldown, including failed attempts. Autoscaling V1 scales up directly to the CPU- or memory-based recommendation. After stabilization, it scales down one replica at a time. Each change uses the normal rolling full-fleet replacement. Existing containers remain active until the complete replacement fleet is healthy and routing has converged, but this can temporarily require both the old and replacement capacity and can reset long-lived connections. Changing the configured range clamps an active count outside the new bounds directly to the nearest bound in one rollout.

​
Serverless scaling

Public HTTP services can be configured to sleep when idle on proxy nodes. A sleeping deployment keeps active traffic intent, records stopped runtime intent, and stops the local container. The next public HTTP request wakes local proxy-hosted deployments and is held until an upstream is ready or the wake timeout is reached. For services with proxy-hosted serverless replicas, public traffic must be sent only to proxy nodes that host a local proxy replica for that service. Non-owner proxies do not emit a public HTTP route for the service, so DNS or the external load balancer must avoid those proxies for that service’s domains. Serverless settings are configured per service: The serverless settings appear only when the service has a public HTTP port with a domain. Removing the final qualifying endpoint disables serverless in the pending service configuration. A cold wake starts the sleeping local proxy replicas for that host. Held requests resume when one upstream is ready. Serverless services require at least one configured replica.

​
Placement

Stateless services support automatic and manual placement. Automatic placement stores either a fixed count or an autoscaling range and distributes the concrete replica target across online, configured nodes during rollout. Manual placement selects exact target servers and replica counts. Serverless services support automatic placement across online, configured proxy nodes. Public ingress must use health checks to avoid proxy nodes that do not currently own a replica for the service.

​
Server Pinning

Stateful services (those with volumes) are automatically pinned to a single server. This ensures the container always mounts the same local data path and avoids accidentally starting on a server that does not have the volume. Pinning does not provide high availability. Volume data is not replicated across servers, and if the pinned server is lost, the service must be recovered from completed backups. You can also manually lock any service to a specific server by setting the locked server. This is useful for workloads that need to run on a particular machine.

​
Limitations

  • Stateful services are limited to 1 replica.
  • Stateful services are always pinned to their locked server.
  • Stateful services do not automatically fail over to another server.
  • Maximum 32 replicas per service.
  • Serverless scaling requires a public HTTP service domain.
  • Sleep and wake are proxy-local; serverless replicas must be placed on proxy nodes.
  • Serverless traffic must be routed only to proxy nodes that own a local proxy replica for that service.
  • Proxy agents report sleep and wake transitions through normal status reports.