-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
A note for the community
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem
Summary
We use Vector as a metrics aggregator between Datadog Agents and Datadog intake. After upgrading the Datadog Agent from 7.50.3 to 7.71.0, certain metric tags (e.g. process_shard:integrations-*) disappear downstream when metrics are forwarded through Vector.
Based on our investigation so far, it is unclear whether this behavior is the result of an upstream change in the Datadog Agent or something occurring downstream. We are opening this issue to maybe get a set of eyes on any Vector-side influence in this case.
Environment
- Vector version: 0.44.0
- Source type: socket (TCP)
- Port: 8282
- Protocol: non-HTTP, binary (Datadog Agent internal format)
- Deployment: Kubernetes
- Vector role: metrics aggregator / forwarder to Datadog
note: No Vector configuration changes were made between working and broken states.
Architecture (simplified)
Application Pods
→ DogStatsD (8125)
→ Datadog Agent
→ TCP (8282)
→ Vector
→ Datadog
note: Multiple Datadog Agent DaemonSets forward metrics to the same Vector instance.
Observed Behavior
Working state
Datadog Agent version: 7.50.3-jmx
Metrics forwarded through Vector include tags such as:
- process_shard:integrations-*
- service_family
- customer_tier
Broken state
Datadog Agent version: 7.71.0-jmx (also reproducible on 7.68.0+)
Metrics still arrive, but certain tags are missing
Issue only occurs when Vector forwarding is enabled
Disabling Vector restores all tags
Vector logs and vector tap confirm that the missing tags are not present in events as received by Vector.
- No Vector config changes between versions
- Same Vector instance handles both working and broken agents
- Tags are present inside the Datadog Agent prior to forwarding
- Tags are missing by the time metrics reach Vector
- Rolling back Agent immediately restores tags
Given this setup:
Can Vector’s TCP source or any default metrics handling mutate, filter, or drop tags from Datadog Agent–originated payloads without explicit transforms?
Configuration
Version
0.44.0
Debug Output
Example Data
No response
Additional Context
No response
References
No response