Avoiding Manual Metric Boilerplate in gRPC

Instrumenting every gRPC handler by hand to record request counts, error codes, and latency histograms is repetitive and easy to forget on new endpoints. Instead, gRPC interceptors let us capture RED metrics (Rate, Errors, Duration) across all unary and streaming RPCs in one place.

Two Gotchas to Watch Out For

  1. Use go-grpc-middleware/providers/prometheus: The older grpc-ecosystem/go-grpc-prometheus repository is deprecated in favor of github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus.
  2. Expose a separate HTTP server for /metrics: gRPC serves binary HTTP/2 traffic, whereas Prometheus scrapes plain-text metrics over HTTP. You need a lightweight HTTP listener on a separate port (e.g. :9092) dedicated to /metrics.

Server-Side Setup

package main

import (
	"log"
	"net"
	"net/http"

	prom "github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promhttp"
	"google.golang.org/grpc"
)

func main() {
	// 1. Create server metrics with latency histogram buckets and register them
	srvMetrics := prom.NewServerMetrics(
		prom.WithServerHandlingTimeHistogram(
			prom.WithHistogramBuckets([]float64{0.001, 0.01, 0.1, 0.3, 0.6, 1, 3, 6, 9, 20}),
		),
	)
	reg := prometheus.NewRegistry()
	reg.MustRegister(srvMetrics)

	// 2. Wire unary and stream interceptors into the gRPC server
	grpcServer := grpc.NewServer(
		grpc.ChainUnaryInterceptor(srvMetrics.UnaryServerInterceptor()),
		grpc.ChainStreamInterceptor(srvMetrics.StreamServerInterceptor()),
	)

	// Register your gRPC service implementations here:
	// pb.RegisterYourServiceServer(grpcServer, &yourService{})

	// 3. Pre-initialize metric series for all registered RPC methods
	srvMetrics.InitializeMetrics(grpcServer)

	// 4. Expose /metrics on a separate HTTP port for Prometheus scraping
	go func() {
		mux := http.NewServeMux()
		mux.Handle("/metrics", promhttp.HandlerFor(reg, promhttp.HandlerOpts{}))
		log.Printf("Starting Prometheus metrics server on :9092")
		if err := http.ListenAndServe(":9092", mux); err != nil {
			log.Fatalf("failed to start metrics server: %v", err)
		}
	}()

	// 5. Start the gRPC server
	lis, err := net.Listen("tcp", ":50051")
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	log.Printf("Starting gRPC server on :50051")
	if err := grpcServer.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

Calling srvMetrics.InitializeMetrics(grpcServer) after registering your services ensures zero-valued counters exist for every RPC method at startup, avoiding missing series in Prometheus queries before the first request arrives.

Client-Side Metrics

For outgoing gRPC calls, use prom.NewClientMetrics(), register it with your Prometheus registry, and pass clientMetrics.UnaryClientInterceptor() and clientMetrics.StreamClientInterceptor() to grpc.NewClient (or grpc.Dial).