- Documentation
- Observability
- Traces
Traces
Guara Cloud captures distributed traces from every service running on the platform — without requiring a single line of instrumentation in your application code.
How it works
Tracing on Guara Cloud is built on three components:
- Beyla runs as an eBPF-based daemon on every node and discovers HTTP, gRPC, Redis, NATS, Postgres and other protocols at the kernel level. No SDK to install.
- OpenTelemetry collectors receive Beyla’s spans, batch them, and forward them to Tempo.
- Grafana Tempo stores traces and exposes them to the dashboard and the API for querying by trace ID, time range, status, and minimum duration.
This gives you end-to-end traces of internal service-to-service calls and external requests with zero application changes.
Cross-service visualization
When a single request hops through several services, the trace detail page shows it as a unified picture with three coordinated views:
- Mini service map at the top — every service that participated in the trace, drawn as a small graph. Click a node to filter the waterfall to spans from or to that service. Real services are clickable and jump to the service detail page; Internet and External sentinels (see below) are non-navigable on purpose.
- Participants strip — compact chips listing every service in the trace, plus any Internet / External sentinel nodes. Useful when the map is too large to scan at a glance.
- Waterfall grouped by service lane — instead of a flat list, spans land in lanes named after the service they belong to. Server, internal, producer, and consumer spans live in their own service’s lane; client spans are attributed to the destination service they call. This means every call to
payment-apicollects in a single lane, regardless of which caller initiated it.
The trace list also shows a services involved chip on each row — a count of unique real services touched by the trace, so you can spot complex requests at a glance.
Internet and external sentinels
A trace often touches more than just your services — calls to third-party APIs, requests coming from the public internet, hits to a managed database outside the project. Two synthetic nodes make those visible without polluting the participant list with one entry per host:
- Internet — represents traffic that originated from outside your project (typically end-user requests entering through the edge). One sentinel per trace; root spans without a parent service hang off here.
- External (
<host>) — represents an outbound destination outside your project. One sentinel per unique destination host. Calls toapi.stripe.com,mongodb.atlas.cloud, etc. are grouped under their respective External sentinels.
Sentinels appear as nodes on the mini map and as lanes in the waterfall, with a distinct visual treatment so you can tell at a glance where the boundary of your project is.
Viewing traces in the dashboard
Open a service in the dashboard and switch to the Traces tab. You can:
- Filter by time range, status (
ok/error) and minimum duration. - Click a trace to see the full waterfall of spans across services, with timing per span and any error attributes attached.
- Click any span on the waterfall to see its full attributes and to jump into the related service.
- Pivot from a trace into the related logs and metrics for the same service window.
Viewing traces from the CLI
The same data is available from the terminal — useful when investigating from a shell session or scripting checks in CI:
guara services traces list— list recent traces with filters that mirror the dashboard.guara services traces get <traceId>— render the full waterfall directly in the terminal, with errored spans highlighted in red.
Cron workers in traces
Cron worker fires are instrumented with their own spans and propagate the W3C traceparent header to their destination. A scheduled HTTP fire to one of your services therefore appears as a single trace covering the cron span, the HTTP request span on the receiving service, and any work that service does as a result. See Cron Workers for details.
From the CLI
Two commands surface the same data in the terminal — pair them when you’re investigating from a shell session or wiring up CI checks.
Live tail
guara services traces list --watch is an append-only live feed — each tick adds only the traces it hasn’t seen before, so you can leave the command running and treat your terminal as a tail. Combine with --status error to wait for failures.
guara services traces list --watch
guara services traces list --watch --status error --min-duration-ms 500
guara services traces list --watch --interval 3 --service my-api
Sample feed output:
14:32:01 ● 124ms GET /healthz → my-api 5 spans
14:32:02 ● 312ms POST /orders → my-api 18 spans
14:32:04 ✕ 30001ms POST /webhooks/stripe → payments-api 9 spans
14:32:05 ● 89ms GET /users/me → my-api 4 spans
14:32:08 ● 451ms POST /orders → my-api 22 spans
Green ● for ok, red ✕ for traces with at least one errored span. Each row is appended exactly once even if the same trace remains in the polling window.
Waterfall
guara services traces get <traceId> renders the full waterfall directly in the terminal — every span on its own row, positioned by start time, sized by duration, status colored. Pipe the output into less for navigation, or use --errors to dim non-error spans when triaging.
guara services traces get 4f2ab91d
guara services traces get 4f2ab91d --errors --depth 3
guara services traces get 4f2ab91d --json | jq '.spans[] | select(.status == "error")'
Sample waterfall (truncated, no color):
trace 4f2ab91d · POST /webhooks/stripe · 30001ms · ✕ error
payments-api · POST /webhooks/stripe ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 30001ms ✕
payments-api · verifySignature ▇ 18ms ok
payments-api · db.query ▇▇▇ 142ms ok
payments-api · stripe.events.dispatch ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 29800ms ✕
External (api.stripe.com) · POST /events ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 29800ms ✕
Whenever you have a trace ID in hand — copied from a Grafana panel, a log line, or another tool — guara services traces get <trace-id> is the single fastest way to go from “something failed” to “this exact span timed out”.