Quota telemetry

Goal

Trace one Echo request and one metered MCP tool call. Export their spans and quota metrics to stdout, then stop the providers and limiter. This tutorial uses the same compiled program for the HTTP request and tool call. No collector or external model service runs.

Prerequisites

Install Echo Pro with Go 1.26 or later. The program binds a temporary loopback port and opens an in-process MCP connection. It needs no stable listening port, Redis, database or credential. If you add application attributes, do not ship this stdout to shared logs.

Step 1: run the demonstration

Find the installed module directory:

cd "$(go list -m -f '{{.Dir}}' echo.labstack.com/pro)"

Run the program:

go run ./examples/quota-otel

The program makes its own HTTP request and tool call and then exits. You do not need curl. Stdout contains OpenTelemetry JSON from the trace and metric exporters. Timestamps, trace IDs and durations vary. The HTTP span is named HTTP GET; the tool path meters search. Inspect the emitted quota decision events and token/request instruments.

Step 2: follow the HTTP context

The tracing middleware starts the span before quota and replaces the request context. Quota hooks therefore observe that span. quota.JoinHooks combines tracing, metrics and lifecycle observations. The limiter has plan team, a ten-unit daily request limit and a 1000-unit tokens limit on the token meter.

The HTTP callback returns fixed tenant demo. Replace it with authbind callbacks after authentication and tenant resolution in a deployed application.

Step 3: follow tool accounting

The program registers search on an MCP server and connects an in-process client. quotamcp applies an explicit tool policy and the same limiter. MCP telemetry uses a bounded allowlist for plan, tool and meter names. It never takes the tenant from tool arguments, which the model writes.

By default, a settled MCP call that records no Charge keeps its full billable reservation. This includes calls whose arguments fail SDK validation before the handler runs. The policy in this example therefore bills those failed calls.

To refund billable Cost for validation failures, set both Settle: true and ChargeFirst: true. The handler must call quotamcp.Charge(ctx, nil) before billable work and report actual usage afterward. Attempt meters remain charged. A successful result without any Charge keeps the reservation and reports a configuration error.

// RegisterChargeFirstSearch registers a tool and returns its metering policy.
func RegisterChargeFirstSearch(srv *mcp.Server, search func(context.Context, string) (string, int64, error)) quotamcp.Policy {
  mcp.AddTool(srv, &mcp.Tool{Name: "search"}, func(ctx context.Context, _ *mcp.CallToolRequest, args struct {
    Query string `json:"query"`
  }) (*mcp.CallToolResult, any, error) {
    quotamcp.Charge(ctx, nil) // Mark started before calling the provider.
    text, tokens, err := search(ctx, args.Query)
    if err != nil {
      return nil, nil, err
    } // Keep the reservation if usage is unknown.
    quotamcp.Charge(ctx, quota.Cost{"tokens": tokens})
    return &mcp.CallToolResult{Content: []mcp.Content{&mcp.TextContent{Text: text}}}, nil, nil
  })
  // Assign this policy to MiddlewareConfig.Policies["search"].
  return quotamcp.Policy{
    Cost: quota.Cost{"tokens": 100}, Settle: true, ChargeFirst: true,
  }
}

The full source owns both providers, creates the failure counter and joins the HTTP server and MCP sessions:

// Command quota-otel runs a bounded Echo request and a metered MCP tool call,
// emits spans and metrics to stdout, and shuts down all caller-owned resources.
package main

import (
  "context"
  "errors"
  "fmt"
  "io"
  "net"
  "net/http"
  "os"
  "time"

  "github.com/labstack/echo/v5"
  "github.com/modelcontextprotocol/go-sdk/mcp"
  "go.opentelemetry.io/otel/exporters/stdout/stdoutmetric"
  "go.opentelemetry.io/otel/exporters/stdout/stdouttrace"
  sdkmetric "go.opentelemetry.io/otel/sdk/metric"
  sdktrace "go.opentelemetry.io/otel/sdk/trace"
  "go.opentelemetry.io/otel/trace"

  "echo.labstack.com/pro/quota"
  "echo.labstack.com/pro/quota/quotamcp"
  "echo.labstack.com/pro/quota/quotamcp/mcpotel"
  "echo.labstack.com/pro/quota/quotaotel"
)

func runWithContext(ctx context.Context, output io.Writer) (result error) {
  spanExporter, err := stdouttrace.New(stdouttrace.WithWriter(output))
  if err != nil {
    return err
  }
  tp := sdktrace.NewTracerProvider(sdktrace.WithSyncer(spanExporter))
  defer func() {
    shutdown, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
    defer cancel()
    result = errors.Join(result, tp.Shutdown(shutdown))
  }()
  metricExporter, err := stdoutmetric.New(stdoutmetric.WithWriter(output))
  if err != nil {
    return err
  }
  mp := sdkmetric.NewMeterProvider(sdkmetric.WithReader(
    sdkmetric.NewPeriodicReader(metricExporter),
  ))
  defer func() {
    shutdown, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
    defer cancel()
    result = errors.Join(result, mp.Shutdown(shutdown))
  }()
  names := quotaotel.NameAllowlist{Plans: []string{
    "team",
  }, Limits: []string{"daily", "tokens"}, Meters: []string{
    "requests",
    "tokens",
  }}
  failures, err := quotaotel.NewFailureCounter(mp)
  if err != nil {
    return err
  }
  tracing, err := quotaotel.Tracing(quotaotel.TraceConfig{
    Names:    names,
    Failures: failures,
  })
  if err != nil {
    return err
  }
  metrics, err := quotaotel.New(mp)
  if err != nil {
    return err
  }
  lifecycle, err := quotaotel.NewLifecycle(mp, quotaotel.LifecycleConfig{Names: names})
  if err != nil {
    return err
  }
  limiter, err := quota.New(quota.Config{Store: quota.NewMemoryStore(
    quota.MemoryConfig{},
  ), Plans: []quota.Plan{{ID: "team", Limits: []quota.Limit{
    quota.PerDay("daily", 10),
    quota.PerDay("tokens", 1000).Meter("tokens"),
  }}}, Hooks: quota.JoinHooks(
    tracing,
    metrics.Hooks(),
    lifecycle.Hooks(),
  )})
  if err != nil {
    return err
  }
  defer func() {
    shutdown, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
    defer cancel()
    result = errors.Join(result, limiter.Close(shutdown))
  }()
  tracer := tp.Tracer("quota-otel-example")
  e := echo.New()
  // Tracing runs first, so quota receives the span in the request context.
  e.Use(func(next echo.HandlerFunc) echo.HandlerFunc {
    return func(c *echo.Context) error {
      requestCtx, span := tracer.Start(
        c.Request().Context(),
        "HTTP "+c.Request().Method,
      )
      defer span.End()
      c.SetRequest(c.Request().WithContext(requestCtx))
      return next(c)
    }
  })
  mw, err := (quota.MiddlewareConfig{
    Limiter: limiter,
    Identity: func(
      *echo.Context,
    ) (quota.Identity, error) {
      return quota.Identity{Tenant: "demo"}, nil
    },
  }).ToMiddleware()
  if err != nil {
    return err
  }
  e.GET("/", func(c *echo.Context) error {
    return c.String(
      http.StatusOK,
      "quota-otel demo\n",
    )
  }, mw)
  listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", "127.0.0.1:0")
  if err != nil {
    return err
  }
  server := &http.Server{
    Handler:           e,
    ReadHeaderTimeout: 5 * time.Second,
    ReadTimeout:       10 * time.Second,
    WriteTimeout:      10 * time.Second,
    MaxHeaderBytes:    16 << 10,
  }
  done := make(chan error, 1)
  go func() { done <- server.Serve(listener) }()
  defer func() {
    shutdown, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
    defer cancel()
    result = errors.Join(result, server.Shutdown(shutdown))
    if err := <-done; !errors.Is(err, http.ErrServerClosed) {
      result = errors.Join(result, err)
    }
  }()
  request, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://"+listener.Addr().String()+"/", http.NoBody)
  if err != nil {
    return err
  }
  client := &http.Client{Timeout: 5 * time.Second}
  response, err := client.Do(request)
  if err != nil {
    return err
  }
  if _, err := io.Copy(io.Discard, response.Body); err != nil {
    return errors.Join(err, response.Body.Close())
  }
  if err := response.Body.Close(); err != nil {
    return err
  }
  if response.StatusCode != http.StatusOK {
    return fmt.Errorf("HTTP demonstration returned %d", response.StatusCode)
  }
  return toolCall(ctx, limiter, mp, tracer, failures)
}

func toolCall(
  ctx context.Context,
  limiter *quota.Limiter,
  mp *sdkmetric.MeterProvider,
  tracer trace.Tracer,
  failures quotaotel.FailureSink,
) (
  result error,
) {
  names := mcpotel.NameAllowlist{
    Plans:  []string{"team"},
    Tools:  []string{"search"},
    Meters: []string{"tokens"},
  }
  tracing, err := mcpotel.MCP(mcpotel.MCPTraceConfig{Names: names, Failures: failures})
  if err != nil {
    return err
  }
  metrics, err := mcpotel.NewMCP(mp, mcpotel.MCPMetricsConfig{
    Names:    names,
    Failures: failures,
  })
  if err != nil {
    return err
  }
  a := metrics.Adapters()
  server := mcp.NewServer(&mcp.Implementation{
    Name:    "quota-otel-demo",
    Version: "v1",
  }, nil)
  mcp.AddTool(server, &mcp.Tool{
    Name:        "search",
    Description: "Local demonstration",
  }, func(ctx context.Context, _ *mcp.CallToolRequest, _ struct{}) (
    *mcp.CallToolResult,
    any,
    error,
  ) {
    quotamcp.Charge(ctx, quota.Cost{"tokens": 12})
    return &mcp.CallToolResult{Content: []mcp.Content{&mcp.TextContent{
      Text: "demo result",
    }}}, nil, nil
  })
  mw, err := (quotamcp.MiddlewareConfig{Limiter: limiter, Identity: func(
    context.Context,
    quotamcp.Caller,
  ) (
    quota.Identity,
    error,
  ) {
    return quota.Identity{Tenant: "demo"}, nil
  }, Policies: map[string]quotamcp.Policy{"search": {
    Cost:   quota.Cost{"tokens": 100},
    Settle: true,
  }}, OnCall: func(ctx context.Context, i quotamcp.CallInfo) {
    tracing.OnCall(ctx, i)
    a.OnCall(
      ctx,
      i,
    )
  }, OnMeteringError: func(ctx context.Context, f quotamcp.MeteringFailure) {
    tracing.OnMeteringError(ctx, f)
    a.OnMeteringError(ctx, f)
  }}).ToMiddleware()
  if err != nil {
    return err
  }
  server.AddReceivingMiddleware(mw)
  // In-memory transports do not propagate a client's span automatically. The
  // server session explicitly borrows this live application span for the demo.
  callCtx, span := tracer.Start(ctx, "MCP search")
  defer span.End()
  st, ct := mcp.NewInMemoryTransports()
  ss, err := server.Connect(callCtx, st, nil)
  if err != nil {
    return err
  }
  defer func() { result = errors.Join(result, ss.Close()) }()
  cs, err := mcp.NewClient(&mcp.Implementation{
    Name:    "demo-client",
    Version: "v1",
  }, nil).Connect(callCtx, ct, nil)
  if err != nil {
    return err
  }
  defer func() { result = errors.Join(result, cs.Close()) }()
  response, err := cs.CallTool(callCtx, &mcp.CallToolParams{Name: "search"})
  if err != nil {
    return err
  }
  if response.IsError {
    return errors.New("MCP demonstration was denied")
  }
  return nil
}

func main() {
  if err := run(); err != nil {
    fmt.Fprintln(os.Stderr, err)
    os.Exit(1)
  }
}

func run() error {
  ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
  defer cancel()
  if err := runWithContext(ctx, os.Stdout); err != nil {
    return err
  }

  return nil
}

Step 4: verify the lifecycle

Run the example tests:

go test ./examples/getting-started ./examples/quota-otel ./examples/admin-demo

The program stops HTTP serving, closes limiter-owned settlement work and shuts down meter and tracer providers through bounded cleanup contexts. It does not change global telemetry providers. If an exporter fails, the program exits with that error.

For a smaller tracing-only fixture, this executable example prints true quota.decision:

package quotaotel_test

import (
  "context"
  "errors"
  "fmt"
  "testing"

  sdkmetric "go.opentelemetry.io/otel/sdk/metric"
  "go.opentelemetry.io/otel/sdk/metric/metricdata"
  sdktrace "go.opentelemetry.io/otel/sdk/trace"
  "go.opentelemetry.io/otel/sdk/trace/tracetest"

  "echo.labstack.com/pro/quota"
  "echo.labstack.com/pro/quota/quotaotel"
)

func ExampleTracing() {
  exporter := tracetest.NewInMemoryExporter()
  provider := sdktrace.NewTracerProvider(sdktrace.WithSyncer(exporter))
  ctx, span := provider.Tracer("example").Start(context.Background(), "request")
  hooks, err := quotaotel.Tracing(quotaotel.TraceConfig{
    Names: quotaotel.NameAllowlist{
      Plans:  []string{"standard"},
      Limits: []string{"daily"},
    },
  })
  if err != nil {
    panic(err)
  }
  limiter, err := quota.New(quota.Config{Store: quota.NewMemoryStore(
    quota.MemoryConfig{},
  ), Plans: []quota.Plan{{ID: "standard", Limits: []quota.Limit{
    quota.PerDay("daily", 10),
  }}}, Hooks: hooks})
  if err != nil {
    panic(err)
  }
  d, err := limiter.Allow(ctx, quota.Request{
    Identity: quota.Identity{Tenant: "example"},
  })
  if err != nil {
    panic(err)
  }
  span.End()
  fmt.Println(d.Allowed, exporter.GetSpans()[0].Events[0].Name)
  if err := limiter.Close(ctx); err != nil {
    panic(err)
  }
  if err := provider.Shutdown(ctx); err != nil {
    panic(err)
  }
  // Output: true quota.decision
}

// ConfigureOutboxMetrics records delivery metrics after the exporter accepts an event.
func ConfigureOutboxMetrics(cfg quota.Config, metrics *quotaotel.Metrics, tracing, lifecycle quota.Hooks, deliver func(context.Context, quota.Event) error) quota.Config {
  cfg.Hooks = quota.JoinHooks(tracing, metrics.OutboxHooks(), lifecycle)
  cfg.Outbox = &quota.Outbox{Deliver: func(ctx context.Context, ev quota.Event) error {
    if err := deliver(ctx, ev); err != nil {
      return err
    }
    metrics.Count(ctx, ev)
    return nil
  }}
  return cfg
}

func TestConfigureOutboxMetrics(t *testing.T) {
  deliveryErr := errors.New("delivery failed")
  for _, tc := range []struct {
    name string
    err  error
    want string
  }{
    {"success", nil, "quota.consumption {unit} {meter=\"tokens\"} 5\n"},
    {"failure", deliveryErr, "\n"},
  } {
    t.Run(tc.name, func(t *testing.T) {
      reader := sdkmetric.NewManualReader()
      provider := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader))
      t.Cleanup(func() {
        if err := provider.Shutdown(context.WithoutCancel(t.Context())); err != nil {
          t.Error(err)
        }
      })
      metrics, err := quotaotel.New(provider)
      if err != nil {
        t.Fatal(err)
      }
      event := quota.Event{Consumption: &quota.ConsumptionEvent{Meter: "tokens", Amount: 5}}
      deliverCalls := 0
      cfg := ConfigureOutboxMetrics(quota.Config{}, metrics, quota.Hooks{}, quota.Hooks{},
        func(ctx context.Context, ev quota.Event) error {
          deliverCalls++
          if ev.Consumption != event.Consumption {
            t.Fatal("delivery received a different event")
          }
          var observed metricdata.ResourceMetrics
          if err := reader.Collect(ctx, &observed); err != nil {
            t.Fatal(err)
          }
          if len(observed.ScopeMetrics) != 0 {
            t.Fatalf("counted before delivery succeeded: %+v", observed)
          }
          return tc.err
        })
      if err := cfg.Outbox.Deliver(t.Context(), event); !errors.Is(err, tc.err) {
        t.Fatalf("Deliver error = %v, want %v", err, tc.err)
      }
      if got := render(t, reader); got != tc.want || deliverCalls != 1 {
        t.Fatalf("metrics=%q deliveries=%d; want %q and one delivery", got, deliverCalls, tc.want)
      }
    })
  }
}

Change for deployment

Replace stdout exporters with your own provider/export path. Keep the providers caller-owned and drain them at shutdown after work stops. Define allowlists from controlled plan, limit, tool and meter names. Do not label metrics with raw tenant IDs, credentials or arbitrary tool arguments.

When combining this program with the Stripe outbox example, replace metrics.Hooks() with metrics.OutboxHooks() in quota.JoinHooks. Keep tracing and lifecycle hooks. An outbox cannot be combined with synchronous consumption or threshold hooks. Count events after the exporter accepts them:

// ConfigureOutboxMetrics records delivery metrics after the exporter accepts an event.
func ConfigureOutboxMetrics(cfg quota.Config, metrics *quotaotel.Metrics, tracing, lifecycle quota.Hooks, deliver func(context.Context, quota.Event) error) quota.Config {
  cfg.Hooks = quota.JoinHooks(tracing, metrics.OutboxHooks(), lifecycle)
  cfg.Outbox = &quota.Outbox{Deliver: func(ctx context.Context, ev quota.Event) error {
    if err := deliver(ctx, ev); err != nil {
      return err
    }
    metrics.Count(ctx, ev)
    return nil
  }}
  return cfg
}

Count counts redeliveries again; these counters are delivery observations, not a deduplicated billing ledger. Stripe delivery still deduplicates by event ID.

Use Redis or Valkey for budgets shared across processes. Record allow/deny decisions and final usage as separate metrics. Tracing a denied attempt does not mean it consumed work. See quota accounting tasks for reserve/settle examples and the configuration reference for telemetry bounds.