> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/iii-hq/sdk/llms.txt
> Use this file to discover all available pages before exploring further.

# Telemetry

> OpenTelemetry APIs for distributed tracing, metrics, and logging

The III SDK includes comprehensive OpenTelemetry support for observability, automatically propagating traces across function calls and exporting telemetry to the III Engine.

## Initialization

### initOtel()

Manually initialize OpenTelemetry (called automatically by `init()`).

```typescript theme={null}
import { initOtel } from 'iii-sdk/telemetry'

initOtel(config?)
```

<ParamField path="config" type="OtelConfig">
  OpenTelemetry configuration

  <Expandable title="OtelConfig properties">
    <ParamField path="enabled" type="boolean" default="true">
      Enable/disable OpenTelemetry. Also reads `OTEL_ENABLED` env var
    </ParamField>

    <ParamField path="serviceName" type="string" default="iii-node">
      Service name for telemetry. Also reads `OTEL_SERVICE_NAME` env var
    </ParamField>

    <ParamField path="serviceVersion" type="string" default="unknown">
      Service version. Also reads `SERVICE_VERSION` env var
    </ParamField>

    <ParamField path="serviceNamespace" type="string">
      Service namespace. Also reads `SERVICE_NAMESPACE` env var
    </ParamField>

    <ParamField path="serviceInstanceId" type="string">
      Service instance ID. Also reads `SERVICE_INSTANCE_ID` env var. Auto-generated UUID by default
    </ParamField>

    <ParamField path="engineWsUrl" type="string" default="ws://localhost:49134">
      III Engine WebSocket URL. Also reads `III_BRIDGE_URL` env var
    </ParamField>

    <ParamField path="metricsEnabled" type="boolean" default="true">
      Enable metrics export. Also reads `OTEL_METRICS_ENABLED` env var
    </ParamField>

    <ParamField path="metricsExportIntervalMs" type="number" default="60000">
      Metrics export interval in milliseconds (60 seconds)
    </ParamField>

    <ParamField path="fetchInstrumentationEnabled" type="boolean" default="true">
      Auto-instrument global `fetch()` calls for HTTP client tracing
    </ParamField>

    <ParamField path="instrumentations" type="Instrumentation[]">
      Custom OpenTelemetry instrumentations (e.g., Prisma, MongoDB)
    </ParamField>

    <ParamField path="reconnectionConfig" type="Partial<ReconnectionConfig>">
      WebSocket reconnection configuration for telemetry connection
    </ParamField>
  </Expandable>
</ParamField>

### Example: Custom Configuration

```typescript theme={null}
import { initOtel } from 'iii-sdk/telemetry'
import { PrismaInstrumentation } from '@prisma/instrumentation'

initOtel({
  serviceName: 'api-service',
  serviceVersion: '1.2.3',
  serviceNamespace: 'production',
  metricsExportIntervalMs: 30000, // 30 seconds
  instrumentations: [
    new PrismaInstrumentation()
  ]
})
```

### shutdownOtel()

Shutdown OpenTelemetry and flush pending data.

```typescript theme={null}
import { shutdownOtel } from 'iii-sdk/telemetry'

await shutdownOtel()
```

## Distributed Tracing

### getTracer()

Get the OpenTelemetry tracer instance.

```typescript theme={null}
import { getTracer } from 'iii-sdk/telemetry'

const tracer = getTracer()
```

<ResponseField name="tracer" type="Tracer | null">
  OpenTelemetry tracer, or `null` if OTel is disabled
</ResponseField>

### withSpan()

Create and run a function within a new span.

```typescript theme={null}
import { withSpan, SpanKind } from 'iii-sdk/telemetry'

const result = await withSpan(name, options, fn)
```

<ParamField path="name" type="string" required>
  Span name
</ParamField>

<ParamField path="options" type="object" required>
  <Expandable title="Options">
    <ParamField path="kind" type="SpanKind">
      Span kind: `INTERNAL`, `SERVER`, `CLIENT`, `PRODUCER`, `CONSUMER`
    </ParamField>

    <ParamField path="traceparent" type="string">
      Parent trace context (W3C traceparent header)
    </ParamField>
  </Expandable>
</ParamField>

<ParamField path="fn" type="(span: Span) => Promise<T>" required>
  Async function to execute within the span
</ParamField>

<ResponseField name="result" type="T">
  The function's return value
</ResponseField>

### Example: Custom Spans

```typescript theme={null}
import { withSpan, SpanKind, getContext } from 'iii-sdk'
import { init } from 'iii-sdk'

const iii = init('ws://localhost:49199')

iii.registerFunction(
  { id: 'orders::process' },
  async (data: { order_id: string; items: string[] }) => {
    const { logger } = getContext()
    
    // Create a custom span for database query
    const order = await withSpan(
      'db.query.orders',
      { kind: SpanKind.CLIENT },
      async (span) => {
        span.setAttribute('db.system', 'postgresql')
        span.setAttribute('db.operation', 'SELECT')
        
        const order = await db.orders.findUnique({
          where: { id: data.order_id }
        })
        
        span.addEvent('Order fetched')
        return order
      }
    )
    
    // Process items in their own span
    for (const item of data.items) {
      await withSpan(
        'process.item',
        { kind: SpanKind.INTERNAL },
        async (span) => {
          span.setAttribute('item.id', item)
          // Process item
          logger.info('Processing item', { item })
        }
      )
    }
    
    return { order_id: data.order_id, status: 'processed' }
  }
)
```

### Trace Context Propagation

The SDK automatically propagates W3C trace context across function calls:

```typescript theme={null}
iii.registerFunction(
  { id: 'service_a::handler' },
  async (data) => {
    // This span is automatically created
    const { trace } = getContext()
    
    trace?.setAttribute('service', 'A')
    
    // Call another function - trace context is propagated
    const result = await iii.call('service_b::handler', data)
    
    return result
  }
)

iii.registerFunction(
  { id: 'service_b::handler' },
  async (data) => {
    // This function receives the trace context from service_a
    const { trace } = getContext()
    
    trace?.setAttribute('service', 'B')
    
    // Both spans are linked in the distributed trace
    return { processed: true }
  }
)
```

### Context Extraction and Injection

```typescript theme={null}
import {
  currentTraceId,
  currentSpanId,
  injectTraceparent,
  extractTraceparent,
  injectBaggage,
  extractBaggage
} from 'iii-sdk/telemetry'

// Get current trace info
const traceId = currentTraceId()
const spanId = currentSpanId()

// Inject trace context for external HTTP calls
const traceparent = injectTraceparent()
const baggage = injectBaggage()

await fetch('https://api.example.com/data', {
  headers: {
    'traceparent': traceparent!,
    'baggage': baggage!
  }
})

// Extract trace context from incoming requests
const parentContext = extractTraceparent(req.headers.traceparent)
```

## Baggage

Baggage propagates key-value pairs across service boundaries.

### setBaggageEntry()

Set a baggage entry in the current context.

```typescript theme={null}
import { setBaggageEntry, context } from 'iii-sdk/telemetry'

const newContext = setBaggageEntry('user_id', '123')
context.with(newContext, () => {
  // user_id baggage is now available
})
```

### getBaggageEntry()

Get a baggage entry from the current context.

```typescript theme={null}
import { getBaggageEntry } from 'iii-sdk/telemetry'

const userId = getBaggageEntry('user_id')
```

### getAllBaggage()

Get all baggage entries.

```typescript theme={null}
import { getAllBaggage } from 'iii-sdk/telemetry'

const baggage = getAllBaggage()
console.log(baggage) // { user_id: '123', tenant_id: 'abc' }
```

### removeBaggageEntry()

Remove a baggage entry.

```typescript theme={null}
import { removeBaggageEntry } from 'iii-sdk/telemetry'

const newContext = removeBaggageEntry('user_id')
```

## Metrics

### getMeter()

Get the OpenTelemetry meter instance for creating custom metrics.

```typescript theme={null}
import { getMeter } from 'iii-sdk/telemetry'

const meter = getMeter()
```

<ResponseField name="meter" type="Meter | null">
  OpenTelemetry meter, or `null` if OTel is disabled
</ResponseField>

### Example: Custom Metrics

```typescript theme={null}
import { getMeter } from 'iii-sdk/telemetry'
import { init, getContext } from 'iii-sdk'

const iii = init('ws://localhost:49199')
const meter = getMeter()

if (meter) {
  // Create a counter
  const requestCounter = meter.createCounter('requests.total', {
    description: 'Total number of requests'
  })
  
  // Create a histogram
  const latencyHistogram = meter.createHistogram('request.duration', {
    description: 'Request duration in milliseconds',
    unit: 'ms'
  })
  
  // Create a gauge (via observable gauge)
  const activeConnectionsGauge = meter.createObservableGauge('connections.active', {
    description: 'Number of active connections'
  })
  
  let activeConnections = 0
  
  activeConnectionsGauge.addCallback((result) => {
    result.observe(activeConnections)
  })
  
  // Use metrics in functions
  iii.registerFunction(
    { id: 'api::handler' },
    async (data) => {
      const startTime = Date.now()
      activeConnections++
      
      try {
        requestCounter.add(1, { endpoint: 'api::handler' })
        
        // Process request
        const result = await processRequest(data)
        
        const duration = Date.now() - startTime
        latencyHistogram.record(duration, { endpoint: 'api::handler' })
        
        return result
      } finally {
        activeConnections--
      }
    }
  )
}
```

## Logging

### getLogger()

Get the OpenTelemetry logger instance.

```typescript theme={null}
import { getLogger, SeverityNumber } from 'iii-sdk/telemetry'

const logger = getLogger()
```

<ResponseField name="logger" type="Logger | null">
  OpenTelemetry logger, or `null` if OTel is disabled
</ResponseField>

### Example: Direct Logging

```typescript theme={null}
import { getLogger, SeverityNumber } from 'iii-sdk/telemetry'

const logger = getLogger()

if (logger) {
  logger.emit({
    severityNumber: SeverityNumber.INFO,
    body: 'Application started',
    attributes: {
      'service.name': 'my-service',
      'environment': 'production'
    }
  })
}
```

<Info>
  Most applications should use the Context Logger instead. See [Context API](/nodejs/api/context).
</Info>

## Instrumentation

Add custom instrumentations for automatic tracing of libraries:

```typescript theme={null}
import { init } from 'iii-sdk'
import { PrismaInstrumentation } from '@prisma/instrumentation'
import { HttpInstrumentation } from '@opentelemetry/instrumentation-http'

const iii = init('ws://localhost:49199', {
  otel: {
    instrumentations: [
      new PrismaInstrumentation(),
      new HttpInstrumentation()
    ]
  }
})
```

## Environment Variables

Configure OpenTelemetry via environment variables:

```bash theme={null}
# Enable/disable OpenTelemetry
OTEL_ENABLED=true

# Service identification
OTEL_SERVICE_NAME=my-service
SERVICE_VERSION=1.0.0
SERVICE_NAMESPACE=production
SERVICE_INSTANCE_ID=instance-1

# Engine connection
III_BRIDGE_URL=ws://localhost:49199

# Metrics
OTEL_METRICS_ENABLED=true
```

## Best Practices

<AccordionGroup>
  <Accordion title="Add meaningful span attributes">
    ```typescript theme={null}
    await withSpan('db.query', { kind: SpanKind.CLIENT }, async (span) => {
      span.setAttribute('db.system', 'postgresql')
      span.setAttribute('db.operation', 'SELECT')
      span.setAttribute('db.table', 'users')
      span.setAttribute('db.query_count', 1)
      
      return await db.query(sql)
    })
    ```
  </Accordion>

  <Accordion title="Use span events for important milestones">
    ```typescript theme={null}
    await withSpan('process_order', {}, async (span) => {
      span.addEvent('Validating order')
      await validateOrder(order)
      
      span.addEvent('Charging payment')
      await chargePayment(order)
      
      span.addEvent('Order completed')
      return order
    })
    ```
  </Accordion>

  <Accordion title="Use baggage for cross-cutting concerns">
    ```typescript theme={null}
    // Set tenant ID at the entry point
    const ctx = setBaggageEntry('tenant_id', req.headers['x-tenant-id'])

    context.with(ctx, async () => {
      // All downstream services can access tenant_id
      await iii.call('service::handler', data)
    })
    ```
  </Accordion>

  <Accordion title="Create custom metrics for business events">
    ```typescript theme={null}
    const meter = getMeter()
    const ordersCounter = meter?.createCounter('orders.created')
    const revenueCounter = meter?.createCounter('revenue.total', {
      unit: 'USD'
    })

    ordersCounter?.add(1, { product_type: 'subscription' })
    revenueCounter?.add(order.amount, { currency: 'USD' })
    ```
  </Accordion>
</AccordionGroup>