OTel Agent

A host agent is a small OpenTelemetry Collector that runs as a background service on a server, scrapes host metrics (CPU, memory, disk, filesystem, network) on an interval, and ships them to your TracePath project over OTLP/HTTP. No code changes, no application restart: it watches the machine, not the process.

This page covers running one against TracePath today.

Install

OTELCOL_VERSION=<version-from-the-installation-docs>
curl -fsSL -o otelcol-contrib.deb \
  "https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v${OTELCOL_VERSION}/otelcol-contrib_${OTELCOL_VERSION}_linux_amd64.deb"
sudo dpkg -i otelcol-contrib.deb

receivers:
  hostmetrics:
    collection_interval: 60s
    root_path: /
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      load:
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      disk:
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true
      network:
 
processors:
  batch:
    send_batch_size: 1000
    timeout: 10s
  memory_limiter:
    check_interval: 5s
    limit_mib: 400
  resource/identity:
    attributes:
      - key: service.name
        value: ${env:TRACEPATH_SERVICE_NAME}
        action: upsert
 
exporters:
  otlphttp/tracepath:
    endpoint: ${env:TRACEPATH_ENDPOINT}
    headers:
      Authorization: Bearer ${env:TRACEPATH_TOKEN}
    compression: gzip
    retry_on_failure:
      enabled: true
    sending_queue:
      enabled: true
      queue_size: 1000
 
service:
  pipelines:
    metrics:
      receivers: [hostmetrics]
      processors: [memory_limiter, resource/identity, batch]
      exporters: [otlphttp/tracepath]

sudo tee /etc/otelcol-contrib/otelcol-contrib.conf >/dev/null <<'EOF'
TRACEPATH_TOKEN=<your-project-token>
TRACEPATH_ENDPOINT=https://ingest.tracepath.dev/api/otel
TRACEPATH_SERVICE_NAME=api-prod-eu-1
EOF
sudo chmod 600 /etc/otelcol-contrib/otelcol-contrib.conf
 
sudo systemctl restart otelcol-contrib
sudo systemctl status otelcol-contrib

TRACEPATH_ENDPOINT is the OTLP/HTTP base URL. The Collector appends /v1/metrics, /v1/logs and /v1/traces itself, so give it https://ingest.tracepath.dev/api/otel and nothing more.

What gets captured

Every scrape interval the configuration above collects:

• CPU: utilization (% per core per state), load averages (1m / 5m / 15m).
• Memory: usage in bytes, utilization %.
• Disk: throughput bytes, IOPS, I/O time per device.
• Filesystem: usage bytes and utilization % per mount.
• Network: throughput bytes, packets, errors, and open connections per interface.

Per-process metrics are available by adding a process scraper, which needs the Collector to run as root on Linux:

receivers:
  hostmetrics:
    scrapers:
      process:
        include:
          names: [myapp, postgres]
          match_type: strict

That adds per-process CPU time, RSS, and virtual memory, keyed by process.executable.name. It is off by default because process series multiply quickly.

How hosts are identified

Metrics are tagged with server_name, which comes from the service.name resource attribute — that is what TRACEPATH_SERVICE_NAME sets in the config above. Give every host a distinct value or the hosts cannot be told apart: two machines sharing a service.name collapse into one instance row no matter how different everything else about them is.

You can also group and filter by the attributes each scraper puts on its own data points, for example cpu and state on CPU metrics, device and direction on disk and network metrics, and device and mountpoint on filesystem metrics.

TracePath keeps a fixed allowlist of resource attributes beyond service.name, covering host and platform metadata (host.name, host.id, host.arch, os.type, os.description, cloud.provider, cloud.region, cloud.availability_zone) and per-process identity (process.pid, process.executable.name, process.command_line, process.owner, which is what keeps the per-process series apart). Everything else is dropped, because every distinct value multiplies the number of series stored. See Resource Attributes for the full list.

Adding the resourcedetection processor fills that metadata in automatically, including on EC2, GCE and Azure VMs:

processors:
  resourcedetection:
    detectors: [env, system, ec2, gcp, azure]
    timeout: 5s

Put it in the metrics pipeline ahead of resource/identity. The retained metadata is what the organization overview shows beside each server: OS, architecture, cloud region, and the host name when it differs from the service name. It is descriptive only — server_name remains the identity.

Shipping log files too

To tail application logs from the same agent, add a filelog receiver and a logs pipeline:

receivers:
  filelog:
    include: [/var/log/app/*.log, /var/log/nginx/access.log]
    start_at: end
 
service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [memory_limiter, resource/identity, batch]
      exporters: [otlphttp/tracepath]

start_at: end means the agent does not replay history on first start. The Collector keeps its read position across restarts. Lines arrive on the Logs page, tagged with the service name.

What it opens

Add a health check and the agent exposes one loopback-only port:

extensions:
  health_check:
    endpoint: 127.0.0.1:13133
 
service:
  extensions: [health_check]

curl -fsS http://127.0.0.1:13133/ is then a quick way to confirm it is still running. It is not reachable from off the host. The Collector's own internal metrics are served on localhost:8888 by default.

Ingest, quotas, and limits

Host metrics count against your plan's monthly ingest allowance like any other telemetry. See Billing for the per-plan figures and what happens at the limits.

Two responses are worth handling in an agent that runs unattended:

A 60-second scrape interval on a single host does not come close to the 10 MB cap; the batch processor settings above keep exports far below it even on a busy machine.

Uninstall

sudo systemctl stop otelcol-contrib
sudo systemctl disable otelcol-contrib
sudo dpkg -r otelcol-contrib
sudo rm -rf /etc/otelcol-contrib

Your TracePath project data is untouched.

Next steps

• Organization Overview: what the Servers page does with host metrics.
• Dashboards: building widgets on these metric names.
• Kubernetes: the cluster-wide equivalent.
• Alerts: paging on a host running out of memory or disk.