Telemetry

OpenTelemetry APIs for distributed tracing, metrics, and logging

The III SDK includes comprehensive OpenTelemetry support for observability, automatically propagating traces across function calls and exporting telemetry to the III Engine.

Initialization

initOtel()

Manually initialize OpenTelemetry (called automatically by init()).

import { initOtel } from 'iii-sdk/telemetry'

initOtel(config?)

config · OtelConfig

OpenTelemetry configuration

Show OtelConfig properties

enabled · boolean · default: "true"

Enable/disable OpenTelemetry. Also reads OTEL_ENABLED env var

serviceName · string · default: "iii-node"

Service name for telemetry. Also reads OTEL_SERVICE_NAME env var

serviceVersion · string · default: "unknown"

Service version. Also reads SERVICE_VERSION env var

serviceNamespace · string

Service namespace. Also reads SERVICE_NAMESPACE env var

serviceInstanceId · string

Service instance ID. Also reads SERVICE_INSTANCE_ID env var. Auto-generated UUID by default

engineWsUrl · string · default: "ws://localhost:49134"

III Engine WebSocket URL. Also reads III_BRIDGE_URL env var

metricsEnabled · boolean · default: "true"

Enable metrics export. Also reads OTEL_METRICS_ENABLED env var

metricsExportIntervalMs · number · default: "60000"

Metrics export interval in milliseconds (60 seconds)

fetchInstrumentationEnabled · boolean · default: "true"

Auto-instrument global fetch() calls for HTTP client tracing

instrumentations · Instrumentation[]

Custom OpenTelemetry instrumentations (e.g., Prisma, MongoDB)

reconnectionConfig · Partial<ReconnectionConfig>

WebSocket reconnection configuration for telemetry connection

Example: Custom Configuration

import { initOtel } from 'iii-sdk/telemetry'
import { PrismaInstrumentation } from '@prisma/instrumentation'

initOtel({
  serviceName: 'api-service',
  serviceVersion: '1.2.3',
  serviceNamespace: 'production',
  metricsExportIntervalMs: 30000, // 30 seconds
  instrumentations: [
    new PrismaInstrumentation()
  ]
})

shutdownOtel()

Shutdown OpenTelemetry and flush pending data.

import { shutdownOtel } from 'iii-sdk/telemetry'

await shutdownOtel()

Distributed Tracing

getTracer()

Get the OpenTelemetry tracer instance.

import { getTracer } from 'iii-sdk/telemetry'

const tracer = getTracer()

tracer · Tracer | null

OpenTelemetry tracer, or null if OTel is disabled

withSpan()

Create and run a function within a new span.

import { withSpan, SpanKind } from 'iii-sdk/telemetry'

const result = await withSpan(name, options, fn)

name · string · required

Span name

options · object · required

Show Options

kind · SpanKind

Span kind: INTERNAL, SERVER, CLIENT, PRODUCER, CONSUMER

traceparent · string

Parent trace context (W3C traceparent header)

fn · (span: Span) => Promise<T> · required

Async function to execute within the span

result · T

The function’s return value

Example: Custom Spans

import { withSpan, SpanKind, getContext } from 'iii-sdk'
import { init } from 'iii-sdk'

const iii = init('ws://localhost:49199')

iii.registerFunction(
  { id: 'orders::process' },
  async (data: { order_id: string; items: string[] }) => {
    const { logger } = getContext()
    
    // Create a custom span for database query
    const order = await withSpan(
      'db.query.orders',
      { kind: SpanKind.CLIENT },
      async (span) => {
        span.setAttribute('db.system', 'postgresql')
        span.setAttribute('db.operation', 'SELECT')
        
        const order = await db.orders.findUnique({
          where: { id: data.order_id }
        })
        
        span.addEvent('Order fetched')
        return order
      }
    )
    
    // Process items in their own span
    for (const item of data.items) {
      await withSpan(
        'process.item',
        { kind: SpanKind.INTERNAL },
        async (span) => {
          span.setAttribute('item.id', item)
          // Process item
          logger.info('Processing item', { item })
        }
      )
    }
    
    return { order_id: data.order_id, status: 'processed' }
  }
)

Trace Context Propagation

The SDK automatically propagates W3C trace context across function calls:

iii.registerFunction(
  { id: 'service_a::handler' },
  async (data) => {
    // This span is automatically created
    const { trace } = getContext()
    
    trace?.setAttribute('service', 'A')
    
    // Call another function - trace context is propagated
    const result = await iii.call('service_b::handler', data)
    
    return result
  }
)

iii.registerFunction(
  { id: 'service_b::handler' },
  async (data) => {
    // This function receives the trace context from service_a
    const { trace } = getContext()
    
    trace?.setAttribute('service', 'B')
    
    // Both spans are linked in the distributed trace
    return { processed: true }
  }
)

Context Extraction and Injection

import {
  currentTraceId,
  currentSpanId,
  injectTraceparent,
  extractTraceparent,
  injectBaggage,
  extractBaggage
} from 'iii-sdk/telemetry'

// Get current trace info
const traceId = currentTraceId()
const spanId = currentSpanId()

// Inject trace context for external HTTP calls
const traceparent = injectTraceparent()
const baggage = injectBaggage()

await fetch('https://api.example.com/data', {
  headers: {
    'traceparent': traceparent!,
    'baggage': baggage!
  }
})

// Extract trace context from incoming requests
const parentContext = extractTraceparent(req.headers.traceparent)

Baggage

Baggage propagates key-value pairs across service boundaries.

setBaggageEntry()

Set a baggage entry in the current context.

import { setBaggageEntry, context } from 'iii-sdk/telemetry'

const newContext = setBaggageEntry('user_id', '123')
context.with(newContext, () => {
  // user_id baggage is now available
})

getBaggageEntry()

Get a baggage entry from the current context.

import { getBaggageEntry } from 'iii-sdk/telemetry'

const userId = getBaggageEntry('user_id')

getAllBaggage()

Get all baggage entries.

import { getAllBaggage } from 'iii-sdk/telemetry'

const baggage = getAllBaggage()
console.log(baggage) // { user_id: '123', tenant_id: 'abc' }

removeBaggageEntry()

Remove a baggage entry.

import { removeBaggageEntry } from 'iii-sdk/telemetry'

const newContext = removeBaggageEntry('user_id')

Metrics

getMeter()

Get the OpenTelemetry meter instance for creating custom metrics.

import { getMeter } from 'iii-sdk/telemetry'

const meter = getMeter()

meter · Meter | null

OpenTelemetry meter, or null if OTel is disabled

Example: Custom Metrics

import { getMeter } from 'iii-sdk/telemetry'
import { init, getContext } from 'iii-sdk'

const iii = init('ws://localhost:49199')
const meter = getMeter()

if (meter) {
  // Create a counter
  const requestCounter = meter.createCounter('requests.total', {
    description: 'Total number of requests'
  })
  
  // Create a histogram
  const latencyHistogram = meter.createHistogram('request.duration', {
    description: 'Request duration in milliseconds',
    unit: 'ms'
  })
  
  // Create a gauge (via observable gauge)
  const activeConnectionsGauge = meter.createObservableGauge('connections.active', {
    description: 'Number of active connections'
  })
  
  let activeConnections = 0
  
  activeConnectionsGauge.addCallback((result) => {
    result.observe(activeConnections)
  })
  
  // Use metrics in functions
  iii.registerFunction(
    { id: 'api::handler' },
    async (data) => {
      const startTime = Date.now()
      activeConnections++
      
      try {
        requestCounter.add(1, { endpoint: 'api::handler' })
        
        // Process request
        const result = await processRequest(data)
        
        const duration = Date.now() - startTime
        latencyHistogram.record(duration, { endpoint: 'api::handler' })
        
        return result
      } finally {
        activeConnections--
      }
    }
  )
}

Logging

getLogger()

Get the OpenTelemetry logger instance.

import { getLogger, SeverityNumber } from 'iii-sdk/telemetry'

const logger = getLogger()

logger · Logger | null

OpenTelemetry logger, or null if OTel is disabled

Example: Direct Logging

import { getLogger, SeverityNumber } from 'iii-sdk/telemetry'

const logger = getLogger()

if (logger) {
  logger.emit({
    severityNumber: SeverityNumber.INFO,
    body: 'Application started',
    attributes: {
      'service.name': 'my-service',
      'environment': 'production'
    }
  })
}

Most applications should use the Context Logger instead. See Context API.

Instrumentation

Add custom instrumentations for automatic tracing of libraries:

import { init } from 'iii-sdk'
import { PrismaInstrumentation } from '@prisma/instrumentation'
import { HttpInstrumentation } from '@opentelemetry/instrumentation-http'

const iii = init('ws://localhost:49199', {
  otel: {
    instrumentations: [
      new PrismaInstrumentation(),
      new HttpInstrumentation()
    ]
  }
})

Environment Variables

Configure OpenTelemetry via environment variables:

# Enable/disable OpenTelemetry
OTEL_ENABLED=true

# Service identification
OTEL_SERVICE_NAME=my-service
SERVICE_VERSION=1.0.0
SERVICE_NAMESPACE=production
SERVICE_INSTANCE_ID=instance-1

# Engine connection
III_BRIDGE_URL=ws://localhost:49199

# Metrics
OTEL_METRICS_ENABLED=true

Best Practices

Add meaningful span attributes

await withSpan('db.query', { kind: SpanKind.CLIENT }, async (span) => {
  span.setAttribute('db.system', 'postgresql')
  span.setAttribute('db.operation', 'SELECT')
  span.setAttribute('db.table', 'users')
  span.setAttribute('db.query_count', 1)
  
  return await db.query(sql)
})

Use span events for important milestones

await withSpan('process_order', {}, async (span) => {
  span.addEvent('Validating order')
  await validateOrder(order)
  
  span.addEvent('Charging payment')
  await chargePayment(order)
  
  span.addEvent('Order completed')
  return order
})

Use baggage for cross-cutting concerns

// Set tenant ID at the entry point
const ctx = setBaggageEntry('tenant_id', req.headers['x-tenant-id'])

context.with(ctx, async () => {
  // All downstream services can access tenant_id
  await iii.call('service::handler', data)
})

Create custom metrics for business events

const meter = getMeter()
const ordersCounter = meter?.createCounter('orders.created')
const revenueCounter = meter?.createCounter('revenue.total', {
  unit: 'USD'
})

ordersCounter?.add(1, { product_type: 'subscription' })
revenueCounter?.add(order.amount, { currency: 'USD' })