This repository contains labs for the OpenTelemetry Hands-On Session. The tech stack used are Go and Kafka.

Prerequisites

What You'll Learn

In this step we will prepare the environment for the hands on exercises

Step 1: Enable OpenTelemetry instrumentation

Step 2: Install OneAgent

Step 3: Clone the repos

You've arrived

In this step we extend a sample program with an additional OpenTelemetry Span. The sample consists of an HTTP Server which is able to calculate any Fibonacci number. The result of that calculation is getting sent to a Kafka Broker.

Compile and launch the program

  $ source ~/.profile
  $ go version
  go version go1.15.7 linux/amd64

  $ cd ex1
  $ go build

  $ ./fib

  $ curl http://localhost:28080/fib?n=1;echo

  $ curl http://localhost:28080/fib?n=1;echo

1-1 Declaring a Global Tracer

  if err := initGlobalTracer(nil); err != nil {
     panic(err)
  }

1-2 Producing SPAN on http request

  	tracer := otel.Tracer("http")
	ctx := context.Background()

	var span trace.Span
	ctx, span = tracer.Start(ctx, "http-request")
	defer span.End()

Recompile and execute transactions again

  $ go build
  $ ./fib
  $ curl http://localhost:28080/fib?n=2;echo

You've arrived

The OpenTelemetry Span we just created is more or less redundant. The HTTP Sensor of Dynatrace already created the necessary PurePath Node. Whatever we have introduced via OpenTelemetry doesn't really enrich our experience. In this task our goal is to grant users of Dynatrace deeper insight into what is going on when the Fibonacci number is getting calculated.

Prerequisites

2-1 Passing the trace context into Fibonacci function

  func New(ctx context.Context) Fibonacci {
	return &fibonacci{Context: ctx}
	
  type fibonacci struct {
	Context context.Context
  }

2-2 Producing SPAN inside Calc method

	var span trace.Span
	tracer := otel.Tracer("")
	f.Context, span = tracer.Start(f.Context, fmt.Sprintf("fib(%d)", n))
	defer span.End()

2-3 Creating a SPAN attribute

  span.SetAttributes(label.Key("fib.result").Int(result))

2-4 Execution of Fibonacci function in main.go now requires trace context

  result, numIterations := fibonacci.New(ctx).Calc(n)

Recompile and execute transactions again

  $ go build
  $ ./fib
  $ curl http://localhost:28080/fib?n=3;echo

2-5 Tell Dynatrace which OpenTelemetry Attributes are of interest for you

2-6 Capture a Request Attribute based on the OpenTelemetry Key

  $ curl http://localhost:28080/fib?n=5;echo
  $ curl http://localhost:28080/fib?n=6;echo

You've arrived

With Exercise 2 we actually overshot. First of all, we had to modify the existing source code of the Fibonacci Calculator. This is rarely possible for third party libraries. But even then you'd rather prefer to use the original libraries instead of patching them. Second, our solution traces ALL the invocations of the Calc function. Trace sizes easily explode given a sufficiently high input value.

Prerequisites

3-1 Wrapping trace+context over Fibonacci function

  $ cd ..
  $ cd ex3

  // Wrap produces a Fibonacci Calculator with tracing capabilities
  func Wrap(ctx context.Context, f Fibonacci) Fibonacci {
  	return &tracingFib{Context: ctx, Fibonacci: f}
  }
  
  type tracingFib struct {
  	Fibonacci Fibonacci
  	Context   context.Context
  }
  
  func (tf *tracingFib) Calc(n int) (int, int) {
  	var span trace.Span
  	tracer := otel.Tracer("")
  	tf.Context, span = tracer.Start(tf.Context, fmt.Sprintf("fib(%d)", n))
  	defer span.End()
  	result, iterations := tf.Fibonacci.Calc(n)
  	span.SetAttributes(label.Key("fib.result").Int(result))
  	return result, iterations
  }

3-2 Changes are required on main.go - Producing SPAN on http request

3-2 Execute Fibonacci function that has the trace WRAPPER function

  result, numIterations := fibonacci.Wrap(ctx, fibonacci.New()).Calc(n)

Recompile and execute transactions again

  $ go build
  $ ./fib
  $ curl http://localhost:28080/fib?n=7;echo
  $ curl http://localhost:28080/fib?n=8;echo

You've arrived

Compile Time instrumentation might be fun to some extent, but it's usually not your job to do that. The idea behind OpenTelemetry is that third party software either already comes fully instrumented (with all the necessary source code for OpenTelemetry included) or alternatively is getting provided as an optional feature via a helper library.

Prerequisites

4-1 Kafka Broker URL

4-2 Introduce the Sarama lib wrapper

  producer = otelsarama.WrapSyncProducer(config, producer)

Recompile and execute transactions again

  $ go build
  $ ./fib
  $ curl http://localhost:28080/fib?n=9;echo

You've arrived

Apart from getting deeper insight into what's going on within an application on a transactional level, OpenTelemetry also covers the ability to gather metrics. In this lesson we learn how to enable the OpenTelemetry Metric Exporter for Dynatrace.

Prerequisites

MetricIngestion

5-1 Instantiate Dynatrace Exporter

    func main() {
        if err := initGlobalTracer(nil); err != nil {
            panic(err)
        }
        initMetricsProvider()
        http.HandleFunc("/fib", FibServer)
        http.HandleFunc("/favicon.ico", faviconHandler)
        http.ListenAndServe(":28080", nil)
    }

5-2 Using OneAgent endpoint for metrics ingestion

Recompile and execute transactions again

  $ go build
  $ ./fib
  $ curl http://localhost:28080/fib?n=1;echo
  $ curl http://localhost:28080/fib?n=2;echo
  $ curl http://localhost:28080/fib?n=3;echo
  $ curl http://localhost:28080/fib?n=4;echo
  $ curl http://localhost:28080/fib?n=5;echo
  $ curl http://localhost:28080/fib?n=6;echo
  $ curl http://localhost:28080/fib?n=7;echo
  $ curl http://localhost:28080/fib?n=8;echo
  $ curl http://localhost:28080/fib?n=9;echo
  $ curl http://localhost:28080/fib?n=10;echo
  $ curl http://localhost:28080/fib?n=11;echo
  $ curl http://localhost:28080/fib?n=12;echo

Visualize the metrics using the Data Explorer

5-3 Using DynatraceAPI endpoint for metrics ingestion

Recompile and execute transactions again

  $ go build
  $ ./fib
  $ curl http://localhost:28080/fib?n=12;echo
  $ curl http://localhost:28080/fib?n=13;echo
  $ curl http://localhost:28080/fib?n=14;echo
  $ curl http://localhost:28080/fib?n=15;echo
  $ curl http://localhost:28080/fib?n=16;echo

You've arrived

We hope you enjoyed this lab and found it useful. We would love your feedback!

How was your overall experience with this lab?

Excellent Good Average Fair Poor

What did you benefit most from this lab?

Using OneAgent Operator to deploy in Kubernetes Setting up Kubernetes integation Enabling early access feature flags Learning Kubernetes View in Dynatrace

How likely are you to recommend this lab to a friend or colleague?

Very Likely Moderately Likely Neither Likely nor unlikely Moderately Unlikely Very Unlikely