opentelemetry-net-instrumentation

Aaronontheweb/dotnet-skillsMonitoringAudit passedSnapshot 6cfa18bd0930

Installs: 0

Summary

Provides guidance for implementing OpenTelemetry instrumentation in .NET codebases, covering tracing (Activities/Spans), metrics, logs, naming conventions, error handling, performance, SDK setup, resources, context propagation, and API design best practices.

SKILL.md

OpenTelemetry .NET Instrumentation Skill

When to Use

Adding OpenTelemetry instrumentation to .NET code (traces, metrics, logs)
Creating or modifying ActivitySources, Meters, or ILogger usage
Setting up the OpenTelemetry SDK, resources, exporters, or sampling
Reviewing telemetry implementations for spec compliance
Optimizing instrumentation performance
Designing telemetry APIs that become part of the public surface
Implementing context propagation across service boundaries

Architecture: .NET Is Different

CRITICAL: The .NET OpenTelemetry implementation is fundamentally different from other platforms. .NET provides tracing, metrics, and logging APIs in the framework itself. That means OTel does not provide a separate instrumentation API — it uses the built-in .NET APIs and acts as the collection/export layer.

The Three Built-in .NET APIs (Primary — Zero Dependencies)

Signal	.NET Framework API	Namespace
Tracing	`ActivitySource` / `Activity`	`System.Diagnostics`
Metrics	`Meter` / `Counter<T>` / `Histogram<T>` / etc.	`System.Diagnostics.Metrics`
Logging	`ILogger<T>`	`Microsoft.Extensions.Logging`

These are the primary and only APIs library authors should use for instrumentation. They ship with the .NET runtime — .

Package	Purpose	When to add
`OpenTelemetry.Extensions.Hosting`	DI integration for ASP.NET Core / generic host	Application only
`OpenTelemetry.Exporter.Console`	Console exporter (dev/testing)	Application only
`OpenTelemetry.Exporter.OpenTelemetryProtocol`	OTLP exporter (production)	Application only
`OpenTelemetry.Exporter.Prometheus*`	Prometheus metrics endpoint	Application only
`OpenTelemetry.Instrumentation.AspNetCore`	Auto-instrument ASP.NET Core requests	Application only
`OpenTelemetry.Instrumentation.Http`	Auto-instrument HttpClient calls	Application only
`OpenTelemetry.Instrumentation.SqlClient`	Auto-instrument SQL calls	Application only

// ✅ CORRECT: Use ActivitySource, not DiagnosticSource
public class MyFeature
{
    // Primary ActivitySource - name typically matches the component or NuGet package name
    private static readonly ActivitySource ActivitySource = new("MyApp.MyComponent", "1.0.0");

    // Specialized ActivitySource for opt-in scenarios
    private static readonly ActivitySource DetailedActivitySource = new("MyApp.MyComponent.Detailed", "1.0.0");
}

// ✅ Check HasListeners, null-check, then guard expensive work behind IsAllDataRequested
if (ActivitySource.HasListeners())
{
    using var activity = ActivitySource.StartActivity("ProcessItem", ActivityKind.Internal);
    if (activity != null && activity.IsAllDataRequested)
    {
        activity.DisplayName = "Processing order #12345";
        activity.SetTag("app.item_id", itemId);
        activity.SetTag("app.item_type", itemType);
    }
}

// ❌ WRONG: Don't start activities in fire-and-forget tasks where the
// using scope ends before the async work completes (AsyncLocal context is lost)
async Task HelperAsync()
{
    using var activity = ActivitySource.StartActivity("Helper");
    _ = Task.Run(() => DoWorkAsync()); // ❌ activity disposed before task completes
}

// ✅ Unique operation name, friendly display name (null-check before accessing)
using var activity = ActivitySource.StartActivity(
    name: "ProcessItem",              // Unique, identifies class of spans
    kind: ActivityKind.Internal
);
if (activity != null)
    activity.DisplayName = "Processing order #12345"; // User-friendly, can be specific

// ❌ WRONG: Don't include runtime data in operation name
using var badActivity = ActivitySource.StartActivity($"Process_{itemId}"); // ❌

`ActivityKind`	OTel SpanKind	When to use
`Internal`	`INTERNAL`	Default — in-process operations not crossing a remote boundary
`Server`	`SERVER`	Processing an incoming request/response call (HTTP server, gRPC server, RPC server)
`Client`	`CLIENT`	Making an outgoing request/response call (HTTP client, database client, RPC call)
`Producer`	`PRODUCER`	Enqueuing/publishing deferred work (message queue publish, event emit, job enqueue)
`Consumer`	`CONSUMER`	Dequeuing/processing deferred work (message queue receive, event handle, job dequeue)

// ✅ Application code: use your own namespace
activity?.SetTag("myapp.order_id", orderId);
activity?.SetTag("myapp.payment.status", "confirmed");

// ✅ Manual infrastructure instrumentation: use semantic conventions
// activity?.SetTag("db.system.name", "postgresql"); // custom database client
// activity?.SetTag("http.request.method", "GET"); // custom HTTP transport

// Values can be strings, numbers, booleans, or homogeneous arrays
activity?.SetTag("app.item_count", 42);
activity?.SetTag("app.related_ids", new int[] { 1, 2, 3 });

// ❌ WRONG: PascalCase, hyphen delimiter, plural, or unrelated namespace
activity?.SetTag("MyApp.OrderId", orderId);     // ❌ Wrong case
activity?.SetTag("myapp.order-id", orderId);    // ❌ Wrong delimiter

try
{
    await ProcessItemAsync(); // ✅ success: leave status Unset, do not call SetStatus(Ok) — see rules below
}
catch (Exception ex)
{
    if (activity != null)
    {
        activity.SetStatus(ActivityStatusCode.Error, ex.Message); // modern API
        activity.SetTag("error.type", ex.GetType().FullName);
    }
    throw;
}

// Mirrors the spec's OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN values: unset/anything else → spans only (today's default), "logs/dup" → both, "logs" → logs only.
private static readonly string? ExceptionSignalOptIn = Environment.GetEnvironmentVariable("OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN");
private static readonly bool EmitSpanEvents = ExceptionSignalOptIn != "logs";
private static readonly bool EmitLogs = ExceptionSignalOptIn is "logs" or "logs/dup";

try
{
    await ProcessItemAsync();
}
catch (Exception ex)
{
    activity?.SetStatus(ActivityStatusCode.Error, ex.Message);
    activity?.SetTag("error.type", ex.GetType().FullName);

    if (EmitSpanEvents) // ✅ default — what current .NET tooling and dashboards actually consume today
    {
        activity?.AddEvent(new ActivityEvent("exception", tags: new ActivityTagsCollection
        {
            ["exception.type"] = ex.GetType().FullName,
            ["exception.message"] = ex.Message,
            ["exception.stacktrace"] = ex.ToString()
        }));
    }

    if (EmitLogs) // opt-in — the spec's forward direction; pass the exception instance while `activity` is still Activity.Current so the SDK derives trace_id/span_id and exception.type/message/stacktrace from it
    {
        logger.LogError(ex, "Item processing failed");
    }

    throw;
}

var current = Activity.Current; // ❌ may be a user-created ambient span
using var ownedActivity = ActivitySource.StartActivity("MyOperation"); // ✅ captured reference
ownedActivity?.SetTag("myapp.key", value);

var links = new List<ActivityLink>
{
    new(activityContext1),
    new(activityContext2),
};

var activity = ActivitySource.StartActivity(
    ActivityKind.Internal, name: "batch-process", links: links);

public sealed class OrderProcessingMetrics : IDisposable
{
    private readonly Meter meter = new("MyApp.OrderProcessing", "1.0.0");
    private readonly Histogram<double> processingDuration =
        meter.CreateHistogram<double>("myapp.order.processing.duration", unit: "s");
    private readonly Counter<long> itemsProcessed =
        meter.CreateCounter<long>("myapp.order.processing.count", unit: "{order}");

    public void Dispose() => meter.Dispose();
}

Instrument	.NET API	Behavior	Typical Use
Counter	`CreateCounter<T>`	Monotonically increasing	Request counts, error counts
UpDownCounter	`CreateUpDownCounter<T>`	Increases or decreases	Queue size, active connections
Histogram	`CreateHistogram<T>`	Distribution of values	Durations, response sizes
Gauge	`CreateGauge<T>` (.NET 9+)	Synchronous instant value	Current measurement recording
ObservableCounter	`CreateObservableCounter<T>`	Async callback, monotonic	Periodically polled totals
ObservableGauge	`CreateObservableGauge<T>`	Async callback, non-monotonic	CPU/memory usage
ObservableUpDownCounter	`CreateObservableUpDownCounter<T>`	Async callback, bi-directional	Active tasks by priority

// ✅ Action/outcome-based naming, separate methods per outcome
public void OrderProcessingSucceeded(string orderType, TimeSpan duration) { /* Record */ }
public void OrderProcessingFailed(string orderType, Exception ex, TimeSpan duration) { /* Record */ }
public void ConnectionOpened() => connectionsOpen.Add(1);
public void ConnectionClosed() => connectionsOpen.Add(-1);

// ❌ WRONG: Name after metric, confusing signature
public void RecordOrderProcessingDuration(...) { } // ❌ don't name after metric
public void RecordError(bool succeeded, Exception? ex) { } // ❌ confusing signature

// ✅ Low-cardinality, predefined dimensions
processingDuration.Record(duration.TotalSeconds,
    new KeyValuePair<string, object?>("myapp.order_type", orderType),  // bounded set
    new KeyValuePair<string, object?>("outcome", "success"));         // bounded set

// ❌ High-cardinality: unbounded values cause cardinality explosion
failureCount.Add(1, new KeyValuePair<string, object?>("order_id", orderId)); // ❌ unbounded

// ✅ CORRECT: Guard with cheap checks
if (ActivitySource.HasListeners())
{
    using var activity = ActivitySource.StartActivity("Operation");
    // ... expensive work
}

// ✅ CORRECT: Use TagList (struct) for metrics
var tags = new TagList
{
    { "myapp.order_type", orderType },
    { "outcome", "success" }
};
counter.Add(1, tags);

// ✅ Timestamp math (no allocation)
var startTime = Stopwatch.GetTimestamp();
try { await ProcessAsync(); }
finally { var duration = Stopwatch.GetElapsedTime(startTime); metrics.OrderProcessingSucceeded(orderType, duration); }

// ❌ Allocates: Stopwatch.StartNew() or IDisposable timing wrappers

// ❌ Allocates: string interpolation without IsAllDataRequested guard
activity?.SetTag("item", $"Processing {itemId}"); // ❌

// ✅ Guard expensive work behind IsAllDataRequested
if (activity?.IsAllDataRequested == true)
    activity.SetTag("item", $"Processing {itemId}");

[Test]
public async Task Should_create_processing_span_with_correct_parent()
{
    // Arrange
    using var parent = new Activity("Parent").Start();

    // Act
    await handler.Handle(item);

    // Assert
    var processingSpan = recordedActivities.Single(a => a.OperationName == "ProcessItem");
    Assert.AreEqual(parent.Id, processingSpan.ParentId);
    Assert.AreEqual("myapp.item_type", processingSpan.Tags.First().Key);
}

[Test]
public void Should_not_introduce_breaking_changes_to_span_names()
{
    // Ensures string values in span names are under test
    Assert.AreEqual("ProcessItem", MyFeature.SpanName);
}

private static readonly ActivitySource ActivitySource = new("MyApp.MyComponent", "0.9.0");
private readonly Meter meter = new("MyApp.MyComponent", "0.8.0");

opentelemetry-net-instrumentation

Summary

SKILL.md

OpenTelemetry .NET Instrumentation Skill

When to Use

Architecture: .NET Is Different

The Three Built-in .NET APIs (Primary — Zero Dependencies)

The OTel Collection/Export Layer (Secondary — Application Root Only)

Package Decision Guide

Core Principles

Resiliency First

API Surface Awareness

Standards Compliance

Traces / Spans (Activities)

ActivitySource Setup

Creating Activities

Activity Naming

SpanKind Selection

Span Attributes (Tags)

Activity Status and Errors

Recording Exceptions — Keep Emitting Span Events, Add the Logs Opt-In

Accessing Activities

Span Links

Context Propagation

Metrics

Meter and Metrics Class Setup

Instrument Type Overview

Metric Recording Method Naming

Metric Dimensions

Performance Requirements

Zero-Allocation Fast Path

Timing

Avoid Hidden Allocations

Testing Requirements

Span Tests

Versioning

Logs

Reference Files

References

Related skills