Containers concurrency
Concurrency overview
Concurrency determines the number of incoming requests a single instance of a container can process before scaling up. Serverless Containers automatically scale according to the number of instances needed to handle the incoming workload.
A higher number of instances processing requests at the same time implies a greater usage of memory and vCPU, and consequently a higher cost.
Concurrency setting
When deploying a container, Scaleway Serverless Containers allows you to configure the concurrency, which is the maximum number of requests a single container instance can handle at the same time. The concurrency setting ranges from 1 to 1,000 simultaneous instances.
By default, Serverless Containers allow 80 concurrent requests per container instance. We recommend using the default value, as Serverless Containers are designed to efficiently function with it, but you can lower it to better fit specific requirements.
Impact of concurrency on container scaling
If the number of incoming requests exceeds the set concurrency value for a container, Serverless Containers automatically creates new container instances to handle the additional load.
For example, if a container is set to handle 10 concurrent requests, and 50 requests or events arrive at the same time, Serverless Containers will create at least 5 container instances to handle the incoming load.
Benefits of concurrency
Efficiency: Setting a higher concurrency value allows a single container instance to handle more requests, which can reduce the total number of container instances, therefore reducing costs.
Responsiveness: Setting a lower concurrency (down to 1 concurrent container instance) allows to reduce latency for individual requests, since the container focuses on fewer tasks at a time.
Implications for your service
Compute-bound workloads: Workloads relying on heavy CPU or memory usage per request benefit from lower concurrency (e.g., 1 or 2) to avoid resource contention within the container.
I/O-bound workloads: Workloads relying on smaller input/output operations, such as waiting for external APIs or databases benefit from higher concurrency, since the container can handle multiple requests while waiting on I/O.
Examples
Low concurrency
If your Serverless Container hosts a resource-intensive application that can only handle a small number of requests with the provisioned memory and vCPU, you can reduce the number of maximum concurrent requests per instance to scale quicker and avoid bottlenecks and queued requests.
You can set the Maximum concurrent requests per instance of your container to 1 if it is designed to handle a single request at a time. However, a low maximum concurrency value may affect the performance of your deployment, as several instances of your container will start if it receives a high number of concurrent requests.
High concurrency
If your Serverless Container hosts an application that receives a high number of requests that are easily processed with the provisioned memory and vCPU, you can set a greater number of maximum concurrent requests per instance.
By setting a greater concurrency value than the average number of concurrent requests your container can handle, it will only scale up in the event of a spike in received requests.