Developer-Friendly Guide to Sending Local Logs and Traces to Datadog
Written on
Chapter 1: Introduction to Local Tracing
This guide is a continuation of the earlier part, which detailed how to send local logs to Datadog. Here, we will focus on transmitting traces from your local machine to Datadog. It is essential to have the prerequisites from Part 1 to follow the steps outlined here.
How can you send traces locally?
The key component is the Datadog Agent, set up to accept traces via TCP on port 8126. An application, referred to as datadog-demo in this context, utilizes Opentracing (a vendor-neutral API for distributed tracing) to send these traces.
Section 1.1: Running the Datadog Agent
To initiate the process, you need to run a Datadog agent within a Docker container. Use the following command:
docker run --rm --name datadog-agent
--cgroupns host
-v /var/run/docker.sock:/var/run/docker.sock
-v /proc/:/host/proc/:ro
-v /sys/fs/cgroup:/host/sys/fs/cgroup:ro
-p 8126:8126/tcp
-e DD_HOSTNAME=my-local
-e DD_API_KEY=${DD_API_KEY}
-e DD_SITE=datadoghq.eu
-e DD_APM_ENABLED=true
-e DD_APM_NON_LOCAL_TRAFFIC=true
public.ecr.aws/datadog/agent
Breakdown of the Command
- The parameters from --cgroupns to -v /sys... are necessary for successful tracing, although the internal workings of the Datadog trace agent remain unclear.
- The -p 8126:8126/tcp option binds the Datadog trace agent's port 8126 to the host's port 8126.
- The -e DD_HOSTNAME, DD_API_KEY, and DD_SITE variables should be defined as per Part 1.
- The flag -e DD_APM_ENABLED=true activates the Datadog trace agent.
- The -e DD_APM_NON_LOCAL_TRAFFIC=true flag allows the agent to handle tracing from the datadog-demo application, which runs in a different container.
Section 1.2: Configuring Automatic Instrumentation
This step is thoroughly explained in the Automatic Instrumentation guide provided by Datadog. For Java applications, you typically use the Datadog Java tracer JAR file (commonly named dd-java-agent.jar). You can download this from dtdg.co or the dd-trace-java releases.
To integrate this, pass the absolute path of the JAR file in the --javaagent parameter. For those using IntelliJ, you can modify your Run/Debug configurations to include this in the VM options.
Passing the Javaagent Parameter in IntelliJ
This configuration ensures that seamless integration occurs with various supported Java frameworks.
Section 1.3: Implementing Custom Instrumentation (Optional)
For those interested in further customization, you can implement your own instrumentation in the application using the Datadog Library. Detailed instructions for this can be found in the Custom Instrumentation guide. I have also created a sample project that you can explore and experiment with.
In the next installment of this series, we will discuss how to send metrics locally.
Chapter 2: Additional Resources
The video titled "Logging, Metrics, and Tracing with Node.js – Thomas Hunter II" provides valuable insights into the integration of logging and tracing within Node.js applications. Make sure to check it out for a deeper understanding of these concepts.