WebSocket Reconnection

Overview

The III SDK automatically reconnects when the WebSocket connection drops, using exponential backoff with jitter to prevent thundering herd problems. All functions, triggers, and services are automatically re-registered after reconnection.

Default Behavior

By default, the SDK reconnects indefinitely with these settings:

import { DEFAULT_BRIDGE_RECONNECTION_CONFIG } from 'iii-sdk'

const defaults = {
  initialDelayMs: 1000,      // Start with 1 second delay
  maxDelayMs: 30000,         // Cap at 30 seconds
  backoffMultiplier: 2,      // Double delay each attempt
  jitterFactor: 0.3,         // ±30% randomization
  maxRetries: -1             // Retry forever
}

from iii import DEFAULT_RECONNECTION_CONFIG

defaults = {
    'initial_delay_ms': 1000,      # Start with 1 second delay
    'max_delay_ms': 30000,         # Cap at 30 seconds
    'backoff_multiplier': 2.0,     # Double delay each attempt
    'jitter_factor': 0.3,          # ±30% randomization
    'max_retries': -1              # Retry forever
}

Reconnection Algorithm

The SDK calculates retry delays using exponential backoff with jitter:

const exponentialDelay = initialDelayMs * (backoffMultiplier ** attemptNumber)
const cappedDelay = Math.min(exponentialDelay, maxDelayMs)
const jitter = cappedDelay * jitterFactor * (2 * Math.random() - 1)
const finalDelay = cappedDelay + jitter

Example retry sequence (with defaults):

Attempt 1: ~1s (1000ms ± 300ms)
Attempt 2: ~2s (2000ms ± 600ms)
Attempt 3: ~4s (4000ms ± 1200ms)
Attempt 4: ~8s (8000ms ± 2400ms)
Attempt 5: ~16s (16000ms ± 4800ms)
Attempt 6+: ~30s (30000ms ± 9000ms) - capped

Custom Configuration

Basic Configuration

import { init } from 'iii-sdk'

const iii = init('ws://localhost:49134', {
  reconnectionConfig: {
    initialDelayMs: 500,       // Faster initial retry
    maxDelayMs: 10000,         // Lower cap (10s max)
    backoffMultiplier: 1.5,    // Gentler backoff
    jitterFactor: 0.2,         // Less jitter
    maxRetries: 10             // Give up after 10 attempts
  }
})

from iii import III, InitOptions, ReconnectionConfig

iii = III('ws://localhost:49134', InitOptions(
    reconnection_config=ReconnectionConfig(
        initial_delay_ms=500,       # Faster initial retry
        max_delay_ms=10000,         # Lower cap (10s max)
        backoff_multiplier=1.5,     # Gentler backoff
        jitter_factor=0.2,          # Less jitter
        max_retries=10              # Give up after 10 attempts
    )
))

OpenTelemetry Connection Reconnection

The OpenTelemetry telemetry system uses a separate WebSocket connection for traces, metrics, and logs. You can configure its reconnection independently:

import { init } from 'iii-sdk'

const iii = init('ws://localhost:49134', {
  // Main connection (functions, triggers)
  reconnectionConfig: {
    maxRetries: -1  // Never give up
  },
  
  // Telemetry connection (traces, metrics, logs)
  otel: {
    enabled: true,
    reconnectionConfig: {
      maxRetries: 5,           // Give up after 5 attempts
      initialDelayMs: 2000,    // Slower retries (less critical)
      maxDelayMs: 60000        // Higher cap (1 minute)
    }
  }
})

from iii import III, InitOptions, ReconnectionConfig
from iii.telemetry_types import OtelConfig

iii = III('ws://localhost:49134', InitOptions(
    # Main connection (functions, triggers)
    reconnection_config=ReconnectionConfig(max_retries=-1),
    
    # Telemetry connection (traces, metrics, logs)
    otel={
        'enabled': True,
        'reconnection_config': {
            'max_retries': 5,           # Give up after 5 attempts
            'initial_delay_ms': 2000,   # Slower retries (less critical)
            'max_delay_ms': 60000       # Higher cap (1 minute)
        }
    }
))

Connection State Monitoring

Track connection state

Monitor the connection lifecycle to implement custom logic:

import type { IIIConnectionState } from 'iii-sdk'

const unsubscribe = iii.onConnectionStateChange((state: IIIConnectionState) => {
  console.log(`Connection state: ${state}`)
  
  switch (state) {
    case 'disconnected':
      // Initial state or after close
      break
    case 'connecting':
      // First connection attempt
      break
    case 'connected':
      // Successfully connected
      console.log('✓ All functions re-registered')
      break
    case 'reconnecting':
      // Attempting to reconnect after disconnect
      console.warn('Connection lost, reconnecting...')
      break
    case 'failed':
      // Max retries exceeded
      console.error('Connection failed permanently')
      // Trigger alert, switch to backup, etc.
      break
  }
})

// Later: stop monitoring
unsubscribe()

from iii import IIIConnectionState

def handle_state(state: IIIConnectionState) -> None:
    print(f'Connection state: {state}')
    
    if state == 'disconnected':
        # Initial state or after close
        pass
    elif state == 'connecting':
        # First connection attempt
        pass
    elif state == 'connected':
        # Successfully connected
        print('✓ All functions re-registered')
    elif state == 'reconnecting':
        # Attempting to reconnect after disconnect
        print('Connection lost, reconnecting...')
    elif state == 'failed':
        # Max retries exceeded
        print('Connection failed permanently')
        # Trigger alert, switch to backup, etc.

unsubscribe = iii.on_connection_state_change(handle_state)

# Later: stop monitoring
unsubscribe()

Query current state

Check the current connection state synchronously:

const state = iii.getConnectionState()

if (state === 'connected') {
  await iii.call('my-function', data)
} else {
  console.log('Not connected, queueing for later')
}

state = iii.get_connection_state()

if state == 'connected':
    await iii.call('my-function', data)
else:
    print('Not connected, queueing for later')

Automatic Re-registration

When the connection is re-established, the SDK automatically re-registers:

All registered functions (local and HTTP)
All registered trigger types
All registered triggers
All registered services
Pending invocation messages (queued while disconnected)

// No action needed - happens automatically on reconnect
iii.registerFunction({ id: 'users::get' }, getUserHandler)
iii.registerTrigger({ type: 'http', function_id: 'users::get', config: {} })

// Connection drops and reconnects
// → Both function and trigger are re-registered automatically

# No action needed - happens automatically on reconnect
iii.register_function('users::get', get_user_handler)
iii.register_trigger('http', 'users::get', {})

# Connection drops and reconnects
# → Both function and trigger are re-registered automatically

Invocation Behavior During Reconnection

Queuing Messages

Messages sent while disconnected are queued (up to 1000 messages) and sent when the connection is restored:

// Connection is down
iii.callVoid('log::info', { message: 'Hello' })  // Queued
iii.callVoid('log::info', { message: 'World' })  // Queued

// Connection restored → both messages sent immediately

# Connection is down
iii.call_void('log::info', {'message': 'Hello'})  # Queued
iii.call_void('log::info', {'message': 'World'})  # Queued

# Connection restored → both messages sent immediately

Timeout Behavior

Invocations with await that were sent before disconnect will timeout normally:

try {
  // Connection drops after this is sent
  const result = await iii.call('my-function', data, 5000)
} catch (error) {
  // Error: Invocation timeout after 5000ms: my-function
  console.error('Timed out waiting for response')
}

try:
    # Connection drops after this is sent
    result = await iii.call('my-function', data, timeout=5.0)
except TimeoutError as e:
    # TimeoutError: Invocation of 'my-function' timed out after 5.0s
    print('Timed out waiting for response')

Configuration Examples

Development (Fast Retries)

const iii = init('ws://localhost:49134', {
  reconnectionConfig: {
    initialDelayMs: 100,   // Very fast initial retry
    maxDelayMs: 2000,      // Low cap for quick feedback
    maxRetries: 3          // Give up quickly
  }
})

iii = III('ws://localhost:49134', InitOptions(
    reconnection_config=ReconnectionConfig(
        initial_delay_ms=100,   # Very fast initial retry
        max_delay_ms=2000,      # Low cap for quick feedback
        max_retries=3           # Give up quickly
    )
))

Production (Resilient)

const iii = init('ws://production-engine:49134', {
  reconnectionConfig: {
    initialDelayMs: 1000,
    maxDelayMs: 60000,     // 1 minute max
    backoffMultiplier: 2,
    jitterFactor: 0.3,
    maxRetries: -1         // Never give up
  }
})

iii = III('ws://production-engine:49134', InitOptions(
    reconnection_config=ReconnectionConfig(
        initial_delay_ms=1000,
        max_delay_ms=60000,     # 1 minute max
        backoff_multiplier=2.0,
        jitter_factor=0.3,
        max_retries=-1          # Never give up
    )
))

Ephemeral Workers (No Retries)

// For short-lived processes (CI, cron jobs)
const iii = init('ws://localhost:49134', {
  reconnectionConfig: {
    maxRetries: 0  // Don't retry, fail immediately
  }
})

# For short-lived processes (CI, cron jobs)
iii = III('ws://localhost:49134', InitOptions(
    reconnection_config=ReconnectionConfig(
        max_retries=0  # Don't retry, fail immediately
    )
))

Debugging Reconnection

// Enable debug logging (Node.js)
process.env.DEBUG = 'iii:*'

// Logs:
// [iii] Reconnecting in 1234ms (attempt 1)...
// [iii] Reconnecting in 2456ms (attempt 2)...

import logging

# Enable debug logging (Python)
logging.basicConfig(level=logging.DEBUG)
logging.getLogger('iii.iii').setLevel(logging.DEBUG)

# Logs:
# [iii.iii] Reconnecting in 1234ms (attempt 1)
# [iii.iii] Reconnecting in 2456ms (attempt 2)

Best Practices

Reconnection Strategy Checklist:

✅ Use infinite retries (maxRetries: -1) in production
✅ Monitor connection state for critical paths
✅ Use jitter (jitterFactor > 0) to prevent thundering herd
✅ Set reasonable maxDelayMs (30-60s) to balance responsiveness and load
✅ Configure telemetry reconnection separately (less critical than functions)
✅ Test failure scenarios (network partitions, engine restarts)
✅ Implement alerting when state reaches 'failed'

Get Started

Core Concepts

Guides

WebSocket Reconnection

Overview

Default Behavior

Reconnection Algorithm

Custom Configuration

Basic Configuration

OpenTelemetry Connection Reconnection

Connection State Monitoring

Automatic Re-registration

Invocation Behavior During Reconnection

Queuing Messages

Timeout Behavior

Configuration Examples

Development (Fast Retries)

Production (Resilient)

Ephemeral Workers (No Retries)

Debugging Reconnection

Best Practices

​Overview

​Default Behavior

​Reconnection Algorithm

​Custom Configuration

​Basic Configuration

​OpenTelemetry Connection Reconnection

​Connection State Monitoring

​Automatic Re-registration

​Invocation Behavior During Reconnection

​Queuing Messages

​Timeout Behavior

​Configuration Examples

​Development (Fast Retries)

​Production (Resilient)

​Ephemeral Workers (No Retries)

​Debugging Reconnection

​Best Practices

Overview

Default Behavior

Reconnection Algorithm

Custom Configuration

Basic Configuration

OpenTelemetry Connection Reconnection

Connection State Monitoring

Automatic Re-registration

Invocation Behavior During Reconnection

Queuing Messages

Timeout Behavior

Configuration Examples

Development (Fast Retries)

Production (Resilient)

Ephemeral Workers (No Retries)

Debugging Reconnection

Best Practices