Use session pools in Azure Container Apps

Session pools provide subsecond session allocation times for new pools, and are responsible for the management and lifecycle of each session.

Configuration

You can control the behavior of session pools using the following parameters available to az containerapp sessionpool create.

Create a pool

The process for creating a pool is slightly different depending on whether you're creating a code interpreter pool or a custom container pool.

Code interpreter pool

To create a code interpreter session pool using the Azure CLI, ensure you have the latest versions of the Azure CLI and the Azure Container Apps extension with the following commands:

# Upgrade the Azure CLI
az upgrade

# Install or upgrade the Azure Container Apps extension
az extension add --name containerapp --upgrade --allow-preview true -y

Use the az containerapps sessionpool create command to create the pool. The following example creates a Python code interpreter session pool named my-session-pool. Make sure to replace <RESOURCE_GROUP> with your resource group name before you run the command.

az containerapp sessionpool create \
    --name my-session-pool \
    --resource-group <RESOURCE_GROUP> \
    --___location westus2 \
    --container-type PythonLTS \
    --max-sessions 100 \
    --cooldown-period 300 \
    --network-status EgressDisabled

You can define the following settings when you create a session pool:

Get the management API endpoint

To use code interpreter sessions with LLM framework integrations or by calling the management API endpoints directly, you need the pool's management API endpoint.

The endpoint is in the format https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>.

To retrieve the management API endpoint for a session pool, use the az containerapps sessionpool show command. Make sure to replace <RESOURCE_GROUP> with your resource group name before you run the command.

az containerapp sessionpool show \
    --name my-session-pool \
    --resource-group <RESOURCE_GROUP> \
    --query 'properties.poolManagementEndpoint' -o tsv

Custom container pool

To create a custom container session pool, you need to provide a container image and pool configuration settings.

You invoke or communicate with each session using HTTP requests. The custom container must expose an HTTP server on a port that you specify to respond to these requests.

az upgrade
az extension add --name containerapp --upgrade --allow-preview true -y

az containerapp sessionpool create \
    --name my-session-pool \
    --resource-group <RESOURCE_GROUP> \
    --environment <ENVIRONMENT> \
    --registry-server myregistry.azurecr.io \
    --registry-username <USER_NAME> \
    --registry-password <PASSWORD> \
    --container-type CustomContainer \
    --image myregistry.azurecr.io/my-container-image:1.0 \
    --cpu 0.25 --memory 0.5Gi \
    --target-port 80 \
    --cooldown-period 300 \
    --network-status EgressDisabled \
    --max-sessions 10 \
    --ready-sessions 5 \
    --env-vars "key1=value1" "key2=value2" \
    --___location <LOCATION>

az containerapp sessionpool show \
    --name <SESSION_POOL_NAME> \
    --resource-group <RESOURCE_GROUP> \
    --query "properties.poolManagementEndpoint" \
    --output tsv

{
  "type": "Microsoft.App/sessionPools",
  "apiVersion": "2024-08-02-preview",
  "name": "my-session-pool",
  "___location": "westus2",
  "properties": {
    "environmentId": "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ContainerApps/environments/<ENVIRONMENT_NAME>",
    "poolManagementType": "Dynamic",
    "containerType": "CustomContainer",
    "scaleConfiguration": {
      "maxConcurrentSessions": 10,
      "readySessionInstances": 5
    },
    "dynamicPoolConfiguration": {
      "executionType": "Timed",
      "cooldownPeriodInSeconds": 600
    },
    "secrets": [
      {
        "name": "registrypassword",
        "value": "<REGISTRY_PASSWORD>"
      }
    ],
    "customContainerTemplate": {
      "registryCredentials": {
          "server": "myregistry.azurecr.io",
          "username": "myregistry",
          "passwordSecretRef": "registrypassword"
      },
      "containers": [
        {
          "image": "myregistry.azurecr.io/my-container-image:1.0",
          "name": "mycontainer",
          "resources": {
            "cpu": 0.25,
            "memory": "0.5Gi"
          },
          "command": [
            "/bin/sh"
          ],
          "args": [
            "-c",
            "while true; do echo hello; sleep 10;done"
          ],
          "env": [
            {
              "name": "key1",
              "value": "value1"
            },
            {
              "name": "key2",
              "value": "value2"
            }
          ]
        }
      ],
      "ingress": {
        "targetPort": 80
      }
    },
    "sessionNetworkConfiguration": {
      "status": "EgressEnabled"
    }
  }
}

Management endpoint

The following endpoints are available for managing sessions in a pool:

You build the full URL for each endpoint by concatenating the pool's management API endpoint with the endpoint path. The query string must include an identifier parameter containing the session identifier, and an api-version parameter with the value 2024-02-02-preview.

For example: https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/code/execute?api-version=2024-02-02-preview&identifier=<IDENTIFIER>

To retrieve the session pool's management endpoint, use the az containerapp sessionpool show command:

az containerapp sessionpool show \
    --name <SESSION_POOL_NAME> \
    --resource-group <RESOURCE_GROUP> \
    --query "properties.poolManagementEndpoint" \
    --output tsv

All requests to the pool management endpoint must include an Authorization header with a bearer token. To learn how to authenticate with the pool management API, see Authentication.

Each API request must also include the query string parameter identifier with the session ID. This unique session ID enables your application to interact with specific sessions. To learn more about session identifiers, see Session identifiers.

Image caching

When a session pool is created or updated, Azure Container Apps caches the container image in the pool. This caching helps speed-up the process of creating new sessions.

Any changes to the image aren't automatically reflected in the sessions. To update the image, update the session pool with a new image tag. Use a unique tag for each image update to ensure that the new image is pulled.

Related content

• Session types: Learn about the different types of dynamic sessions:

  • Code interpreter sessions
  • Custom container sessions

• Tutorials: Work directly with the REST API or via an LLM agent:

  • Use an LLM agent:
    • AutoGen
    • LangChain
    • LlamaIndex
    • Semantic Kernel
  • Use the REST API
    • JavaScript Code interpreter