OpenTelemetry Collection

OpenTelemetry collection exports distributed traces, logs, and metrics from Daytona SDK operations and sandbox runtimes to your observability stack. Data is sent over the OpenTelemetry Protocol (OTLP) to any OTLP-compatible collector or backend.

Daytona supports two independent telemetry paths:

• Sandbox telemetry: collects traces, logs, and metrics from inside sandboxes, including CPU, memory, and filesystem metrics, application logs, and HTTP spans
• SDK tracing: instruments Daytona API operations and SDK calls in your application process

You can enable one or both. SDK tracing covers the control path from your application into Daytona. Sandbox telemetry covers what runs inside the sandbox. Together they provide end-to-end visibility across both sides.

Configure sandbox collection

Configure a sandbox collection endpoint.

1. Go to Daytona Dashboard ↗

2. Navigate to OpenTelemetry section (visible to organization owners)

3. Configure the following fields:

   • OTLP Endpoint: OpenTelemetry collector endpoint

     Example: https://otel-collector.example.com

   • Headers: authentication headers as key/value pairs

     Example: api-key = YOUR_API_KEY

curl 'https://app.daytona.io/api/organizations/ORGANIZATION_ID/otel-config' \
  --request PUT \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --data '{
  "endpoint": "https://otel-collector.example.com",
  "headers": {
    "api-key": "YOUR_COLLECTOR_API_KEY"
  }
}'

All sandboxes automatically export their telemetry data to the specified OTLP endpoint.

Collected data

View collected telemetry for a sandbox.

1. Go to Daytona Dashboard ↗
2. Navigate to Sandboxes section
3. Open the Sandbox Details sheet for any sandbox
4. Use the Logs, Traces, and Metrics tabs to inspect the collected telemetry data

# Historical resource metrics (CPU, memory, disk)
samples = sandbox.get_metrics()
for s in samples:
    print(f"{s.timestamp}: CPU {s.cpu_used_pct}%")

Metrics:

• daytona.sandbox.cpu.utilization: CPU usage percentage (0-100%)
• daytona.sandbox.cpu.limit: CPU cores limit
• daytona.sandbox.memory.utilization: Memory usage percentage (0-100%)
• daytona.sandbox.memory.usage: Memory used in bytes
• daytona.sandbox.memory.limit: Memory limit in bytes
• daytona.sandbox.filesystem.utilization: Disk usage percentage (0-100%)
• daytona.sandbox.filesystem.usage: Disk space used in bytes
• daytona.sandbox.filesystem.available: Disk space available in bytes
• daytona.sandbox.filesystem.total: Total disk space in bytes

Traces:

• HTTP requests and responses
• Custom spans from your application code

Logs:

• Application logs (stdout/stderr)
• System logs
• Runtime errors and warnings

Resource labels

All sandbox telemetry is automatically annotated with the following OTel resource attributes:

• daytona_organization_id: the organization the sandbox belongs to
• daytona_region_id: the region the sandbox is running in
• daytona_snapshot: the snapshot used to create the sandbox

Custom resource labels

Attach custom resource labels on a sandbox. Labels are a comma-separated list of key=value pairs. The labels are added as OTel resource attributes to all traces, logs, and metrics emitted by the sandbox. Use them to filter and group telemetry by custom dimensions in your observability platform.

1. Set the DAYTONA_SANDBOX_OTEL_EXTRA_LABELS environment variable on a sandbox:

DAYTONA_SANDBOX_OTEL_EXTRA_LABELS="team=backend,env=staging,app=my-service"

Organization metrics

In addition to per-sandbox telemetry, Daytona exports organization-level resource metrics to your configured OTLP endpoint. These metrics are pushed every 60 seconds and provide a high-level view of resource consumption and quotas across your organization.

Organization metrics are exported automatically when you have a sandbox collection endpoint configured. No additional setup is required. The same OTLP endpoint receives both sandbox telemetry and organization metrics.

Exported metrics

Metric attributes

Each metric includes the following attributes for filtering and grouping:

• organization.id (resource attribute): the organization the metrics belong to
• region.id (data point attribute): the region the resource usage and quota applies to

SDK tracing

SDK tracing instruments Daytona SDK operations in your application process and exports them as OpenTelemetry traces. When enabled, the SDK creates spans for calls your application makes into Daytona, then sends those traces over OTLP to your observability backend.

Send traces to any OTLP-compatible backend:

• New Relic
• Jaeger
• Grafana Cloud
• Datadog

1. Pass the otelEnabled flag when initializing the Daytona client, or set the DAYTONA_OTEL_ENABLED environment variable to true:

export DAYTONA_OTEL_ENABLED=true

from daytona import Daytona, DaytonaConfig

# Using async context manager (recommended)
async with Daytona(DaytonaConfig(otel_enabled=True)) as daytona:
    sandbox = await daytona.create()
    # All operations will be traced
# OpenTelemetry traces are flushed on close

daytona = Daytona(DaytonaConfig(otel_enabled=True))
try:
    sandbox = await daytona.create()
    # All operations will be traced
finally:
    await daytona.close()  # Flushes traces

Configure OTLP exporter

The SDK uses standard OpenTelemetry environment variables for configuration.

# OTLP endpoint (without the /v1/traces path)
OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.nr-data.net:4317

# Authentication headers (format: key1=value1,key2=value2)
OTEL_EXPORTER_OTLP_HEADERS="api-key=your-api-key-here"

New Relic

OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.nr-data.net:4317
OTEL_EXPORTER_OTLP_HEADERS="api-key=YOUR_NEW_RELIC_LICENSE_KEY"

See the New Relic dashboard example for detailed setup steps.

Jaeger (local)

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

Grafana Cloud

OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-gateway-prod-<region>.grafana.net/otlp
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <BASE64_ENCODED_CREDENTIALS>"

1. Go to Grafana Cloud Portal
2. Open Connections
3. Click Add new connection
4. Search for OpenTelemetry (OTLP)
5. Follow the wizard to create an access token. The endpoint and headers are provided in the instrumentation instructions. See the Grafana dashboard example for detailed setup steps.

Datadog

Datadog exposes a native OTLP intake endpoint, so you can send telemetry directly to Datadog without a Datadog Agent.

OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.datadoghq.com
OTEL_EXPORTER_OTLP_HEADERS="dd-api-key=YOUR_DATADOG_API_KEY"
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

1. Enter the OTLP endpoint for your Datadog site and add a single header dd-api-key with your Datadog API key:

Use the base endpoint without a /v1/... path. The path is appended automatically.

2. Generate an API key under Datadog > Organization Settings > API Keys. See the Datadog dashboard example for an importable dashboard and verification steps.

Metrics appear under Metrics > Summary (search for daytona.sandbox), where you can also confirm the exact tag keys (for example, service, region.id).

Example

Complete example of OpenTelemetry tracing with the Daytona SDK:

import asyncio
import os
from daytona import Daytona, DaytonaConfig

# Set OTEL configuration
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otlp.nr-data.net:4317"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "api-key=YOUR_API_KEY"

async def main():
    # Initialize Daytona with OTEL enabled
    async with Daytona(DaytonaConfig(otel_enabled=True)) as daytona:

        # Create a sandbox - this operation will be traced
        sandbox = await daytona.create()
        print(f"Created sandbox: {sandbox.id}")

        # Execute code - this operation will be traced
        result = await sandbox.process.code_run("""
import numpy as np
print(f"NumPy version: {np.__version__}")
""")
        print(f"Execution result: {result.result}")

        # Upload a file - this operation will be traced
        await sandbox.fs.upload_file("local.txt", "/home/daytona/remote.txt")

        # Delete sandbox - this operation will be traced
        await daytona.delete(sandbox)

    # Traces are automatically flushed when exiting the context manager

if __name__ == "__main__":
    asyncio.run(main())

Traced operations

The Daytona SDK automatically instruments the following operations:

SDK operations

• create(): sandbox creation and initialization
• get(): retrieving sandbox instances
• list(): listing sandboxes
• start(): starting sandboxes
• stop(): stopping sandboxes
• delete(): deleting sandboxes
• All sandbox, snapshot and volume operations

HTTP requests

• All API calls to the Daytona backend
• Request duration and response status codes
• Error information for failed requests

Trace attributes

Each trace includes the following metadata:

• Service name and version
• HTTP method, URL, and status code
• Request and response duration
• Error details (if applicable)

Troubleshooting

Verify the exporter before digging into application code:

1. Check that environment variables are set correctly
2. Verify your OTLP endpoint is reachable
3. Confirm API keys and headers are valid
4. Check your observability platform for incoming traces
5. Look for connection errors in application logs