Connection Pooling

Reuse TCP connections and leverage HTTP/2 multiplexing for dramatically faster parallel requests.

Overview

Connection pooling groups requests by host and executes them through a single curl process with HTTP/2 multiplexing. This eliminates redundant TCP handshakes and TLS negotiations, dramatically improving performance for parallel requests to the same host.

1 TCP Handshake

Per host instead of per request

1 TLS Negotiation

Single handshake for HTTPS

HTTP/2 Streams

Multiplexed on single connection

1 DNS Lookup

Cached for all requests

How It Works

When connection pooling is enabled, curl-runner groups requests by host and executes each group in a single batched curl process using curl -Z --parallel --http2.

Before vs After

# WITHOUT Connection Pooling (default)
# Each request = new process + new TCP + new TLS

Request 1 → curl ... → TCP handshake → TLS → HTTP
Request 2 → curl ... → TCP handshake → TLS → HTTP
Request 3 → curl ... → TCP handshake → TLS → HTTP
# Total: 3 TCP handshakes, 3 TLS negotiations

# WITH Connection Pooling
# All same-host requests share ONE connection

Requests 1,2,3 → curl -Z --http2 → TCP → TLS → [stream1, stream2, stream3]
# Total: 1 TCP handshake, 1 TLS negotiation

# WITHOUT Connection Pooling (default)
# Each request = new process + new TCP + new TLS

Request 1 → curl ... → TCP handshake → TLS → HTTP
Request 2 → curl ... → TCP handshake → TLS → HTTP
Request 3 → curl ... → TCP handshake → TLS → HTTP
# Total: 3 TCP handshakes, 3 TLS negotiations

# WITH Connection Pooling
# All same-host requests share ONE connection

Requests 1,2,3 → curl -Z --http2 → TCP → TLS → [stream1, stream2, stream3]
# Total: 1 TCP handshake, 1 TLS negotiation

Performance Impact

For 10 requests to the same host over HTTPS, connection pooling can save ~200ms of TCP/TLS overhead per request, resulting in up to 2 seconds of total time saved.

Basic Usage

Enable connection pooling with connectionPool.enabled in your global configuration.

Basic Connection Pooling

global:
  execution: parallel
  connectionPool:
    enabled: true

requests:
  - name: Get Users
    url: https://api.example.com/users
  - name: Get Posts
    url: https://api.example.com/posts
  - name: Get Comments
    url: https://api.example.com/comments

global:
  execution: parallel
  connectionPool:
    enabled: true

requests:
  - name: Get Users
    url: https://api.example.com/users
  - name: Get Posts
    url: https://api.example.com/posts
  - name: Get Comments
    url: https://api.example.com/comments

CLI Options

Enable and configure connection pooling directly from the command line.

Option	Description
--connection-pool	Enable TCP connection pooling with HTTP/2 multiplexing
--max-streams <n>	Max concurrent streams per host (default: 10)
--keepalive-time <sec>	TCP keepalive time in seconds (default: 60)
--connect-timeout <sec>	Connection timeout in seconds (default: 30)

CLI Examples

# Enable connection pooling via CLI
curl-runner api.yaml -p --connection-pool

# With custom settings
curl-runner api.yaml -p --connection-pool --max-streams 20

# Via environment variables
CURL_RUNNER_CONNECTION_POOL=true curl-runner api.yaml -p

# Full example with env vars
CURL_RUNNER_CONNECTION_POOL=true \
CURL_RUNNER_MAX_STREAMS_PER_HOST=15 \
CURL_RUNNER_KEEPALIVE_TIME=120 \
curl-runner api.yaml -p

# Enable connection pooling via CLI
curl-runner api.yaml -p --connection-pool

# With custom settings
curl-runner api.yaml -p --connection-pool --max-streams 20

# Via environment variables
CURL_RUNNER_CONNECTION_POOL=true curl-runner api.yaml -p

# Full example with env vars
CURL_RUNNER_CONNECTION_POOL=true \
CURL_RUNNER_MAX_STREAMS_PER_HOST=15 \
CURL_RUNNER_KEEPALIVE_TIME=120 \
curl-runner api.yaml -p

Environment Variables

Configure connection pooling using environment variables for CI/CD pipelines.

Variable	Description
CURL_RUNNER_CONNECTION_POOL	Enable connection pooling (true/false)
CURL_RUNNER_MAX_STREAMS_PER_HOST	Max concurrent streams per host
CURL_RUNNER_KEEPALIVE_TIME	TCP keepalive time in seconds
CURL_RUNNER_CONNECT_TIMEOUT	Connection timeout in seconds

Configuration Options

Fine-tune connection pooling behavior with these configuration options.

Option	Type	Default	Description
enabled	boolean	false	Enable connection pooling and HTTP/2 multiplexing
maxStreamsPerHost	number	10	Maximum concurrent HTTP/2 streams per host connection
keepaliveTime	number	60	TCP keepalive probe interval in seconds
connectTimeout	number	30	Connection establishment timeout in seconds

Full Configuration

global:
  execution: parallel
  connectionPool:
    enabled: true
    maxStreamsPerHost: 10    # Max concurrent streams per connection
    keepaliveTime: 60        # TCP keepalive in seconds
    connectTimeout: 30       # Connection timeout in seconds

requests:
  - name: Get Users
    url: https://api.example.com/users
    method: GET
  - name: Get Posts
    url: https://api.example.com/posts
    method: GET
  - name: Create Order
    url: https://api.example.com/orders
    method: POST
    body:
      productId: 123
  - name: Get Inventory
    url: https://api.example.com/inventory
    method: GET

global:
  execution: parallel
  connectionPool:
    enabled: true
    maxStreamsPerHost: 10    # Max concurrent streams per connection
    keepaliveTime: 60        # TCP keepalive in seconds
    connectTimeout: 30       # Connection timeout in seconds

requests:
  - name: Get Users
    url: https://api.example.com/users
    method: GET
  - name: Get Posts
    url: https://api.example.com/posts
    method: GET
  - name: Create Order
    url: https://api.example.com/orders
    method: POST
    body:
      productId: 123
  - name: Get Inventory
    url: https://api.example.com/inventory
    method: GET

Multi-Host Behavior

Requests are automatically grouped by host (scheme + hostname + port). Each host group gets its own pooled connection, and groups execute in parallel.

Multi-Host Grouping

# Requests are automatically grouped by host
# Each host group uses its own pooled connection

global:
  execution: parallel
  connectionPool:
    enabled: true
    maxStreamsPerHost: 5

requests:
  # Group 1: api.example.com (1 connection, 3 streams)
  - url: https://api.example.com/users
  - url: https://api.example.com/posts
  - url: https://api.example.com/comments

  # Group 2: auth.example.com (1 connection, 2 streams)
  - url: https://auth.example.com/token
  - url: https://auth.example.com/validate

  # Group 3: cdn.example.com (1 connection, 1 stream)
  - url: https://cdn.example.com/assets

# Requests are automatically grouped by host
# Each host group uses its own pooled connection

global:
  execution: parallel
  connectionPool:
    enabled: true
    maxStreamsPerHost: 5

requests:
  # Group 1: api.example.com (1 connection, 3 streams)
  - url: https://api.example.com/users
  - url: https://api.example.com/posts
  - url: https://api.example.com/comments

  # Group 2: auth.example.com (1 connection, 2 streams)
  - url: https://auth.example.com/token
  - url: https://auth.example.com/validate

  # Group 3: cdn.example.com (1 connection, 1 stream)
  - url: https://cdn.example.com/assets

When to Use

Recommended For

Multiple parallel requests to the same host

Performance-critical test suites

Load testing with many concurrent requests

APIs with high TLS negotiation overhead

Not Needed For

Sequential execution (requests run one at a time)

Single request executions

Requests to many different hosts

When request isolation is required

Limitations

Parallel mode only: Connection pooling only works with execution: parallel. Sequential mode needs ordered results for store variables.

Retry handling: Individual request retries within a batch are not supported. Failed requests in a batch report the batch error.

Curl version: Requires curl 7.66+ for parallel mode support (-Z flag).

HTTP/2 SupportEnable HTTP/2 protocol Parallel ExecutionRun requests simultaneously Performance ProfilingMeasure connection timing metrics