-
Notifications
You must be signed in to change notification settings - Fork 533
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
It seems that OpenTelemetry traces don't inherit the Span parent ID from the http request (traceparent header).
Steps to reproduce (if applicable)
Tl,dr: on a running quickwit instance, call the /api/v1/_elastic/{indexId}/_search http endpoint with a traceparent header, and observe that the associated root_search span doesn't have a parent ID.
More specifically, using this minimal docker compose:
docker-compose.yaml
services:
otel-collector:
image: otel/opentelemetry-collector-contrib:0.119.0
volumes:
- ./otel-collector-config.yaml:/etc/otel/config.yaml
command: ["--config", "/etc/otel/config.yaml"]
ports:
- 4317:4317
- 4318:4318
quickwit:
image: quickwit/quickwit:edge
volumes:
- qwdata:/quickwit/qwdata
ports:
- 7280:7280
environment:
- QW_ENABLE_OPENTELEMETRY_OTLP_EXPORTER=true
- OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
- OTEL_TRACES_SAMPLER=always_on
depends_on:
- otel-collector
command: ["run"]
test-client:
image: curlimages/curl:latest
depends_on:
- quickwit
# Wait for Quickwit to be ready, send one request with a well-known traceparent,
# then sleep to allow the OTLP batch exporter to flush before the container exits.
entrypoint:
- sh
- -c
- |
until curl -sf http://quickwit:7280/health/livez; do sleep 1; done
curl -sf http://quickwit:7280/api/v1/_elastic/otel-traces-v0_9/_search \
-H 'Content-Type: application/json' \
-H 'traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01' \
-d '{"query":{"match_all":{}},"size":1}'
sleep 15
volumes:
qwdata:
otel-collector-config.yaml
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
exporters:
debug:
verbosity: detailed
service:
pipelines:
traces:
receivers: [otlp]
exporters: [debug]
logs:
receivers: [otlp]
exporters: [debug]
metrics:
receivers: [otlp]
exporters: [debug]
I can see the following span:
Span #9
Trace ID : 9a66a9922558a74ccd7bc36d01cd25c3
Parent ID :
ID : 34136b5bd0bb77ac
Name : root_search
Kind : Internal
Start time : 2026-03-26 10:22:21.380716543 +0000 UTC
End time : 2026-03-26 10:22:21.409089709 +0000 UTC
Status code : Unset
Status message :
Attributes:
-> code.file.path: Str(quickwit-search/src/root.rs)
-> code.module.name: Str(quickwit_search::root)
-> code.line.number: Int(1200)
-> thread.id: Int(5)
-> thread.name: Str(main_runtime_thread)
-> target: Str(quickwit_search::root)
-> busy_ns: Int(27969792)
-> idle_ns: Int(404000)
Events:
SpanEvent #0
-> Name: root_search
-> Timestamp: 2026-03-26 10:22:21.390622376 +0000 UTC
-> DroppedAttributesCount: 0
-> Attributes::
-> level: Str(INFO)
-> target: Str(quickwit_search::root)
-> query_ast: Str({"type":"match_all"})
-> agg: Str()
-> start_ts: Str(0..0)
-> count_required: Str(UNDERESTIMATE)
-> num_docs: Str(0)
-> num_splits: Str(0)
-> code.file.path: Str(quickwit-search/src/root.rs)
-> code.module.name: Str(quickwit_search::root)
-> code.line.number: Int(1222)
Expected behavior
Span #9 Parent ID should be should not be null.
Configuration:
Quickwit version: quickwit version: 0.8.0 (aarch64-unknown-linux-gnu 2026-03-11T13:17:37Z 92a526b)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working