Lesson 11 · Security, telemetry & operations

Observability

Metrics, traces, and logs for every hop — for free — and the metric that already scales your services.

Your win: explain the three telemetry outputs a mesh gives you (metrics, traces, logs), and trace this repo's real pipeline: istio_requests_total from the Envoy sidecar → Prometheus → your KEDA autoscaler and alerts.

The mesh sees every request — so it can measure it

Because every call passes through an Envoy sidecar (Lesson 2), the mesh can emit telemetry for all traffic with no app instrumentation. Three outputs:¹

Output	What	Consumed by
Metrics	request count, duration, sizes — per source/destination	Prometheus → Grafana / Kiali
Traces	spans across services for one request	Jaeger / an OTel collector
Access logs	a line per request at the proxy	log aggregation

Istio's standard metrics cover most of the golden signals — latency, traffic, errors — per service, out of the box. The star is istio_requests_total: a counter incremented for every request an Envoy handles.²

This repo's telemetry pipeline

Anchor — metrics: sidecar → Prometheus → KEDA There's no Telemetry CRD here — metrics come from the Envoy sidecar's built-in Prometheus stats (Istio's enablePrometheusMerge), scraped via the prometheus.io/scrape pod annotation. Prometheus's kubernetes-pods job (platforms/monitoring/prometheus/values.yaml:424-462) picks them up. Then the same metric powers two things you've already met:

KEDA autoscaling (Course 2 L9): libs/util/templates/_keda.tpl:150-152 queries sum(rate(istio_requests_total{destination_service_namespace=…,destination_service_name=…}[2m])) against prometheus-server.monitoring — so request rate, measured by the sidecar, drives replica count.
Alerting — Prometheus rules query istio_requests_total directly for gRPC/5xx error rates and "is the ingress even receiving traffic" (prometheus/values.yaml:793-836).

So the metric you learned as "the thing that scales bob" originates here, in the Istio sidecar — this course closes that loop.

Anchor — traces & logs The mesh config (platforms/istio/istiod-values.yaml) turns on tracing — Zipkin-format spans to opentelemetry-collector.monitoring:9411 at sampling: 100.0 (:19-21) — and access logging in JSON (:14-15). So every request is traced and logged at the proxy, feeding the OTel collector (the full Prometheus/Grafana/Jaeger/Kiali stack is Course 6).

Kiali — the mesh's map Because the sidecars know every call, Kiali can draw the live service graph (who calls whom, at what error rate) and validate your Istio config — the single most useful "what is my mesh doing?" view. It's fed entirely by this telemetry. (Detailed in Course 6.)

Istio docs — Observability + Standard Metrics

The metrics/traces/logs model and the istio_requests_total family of standard metrics.

→ istio.io — Observability
→ istio.io — Standard Metrics

Check yourself (from memory)

Q1. The mesh can emit telemetry for all traffic because…

The sidecar sees every request, so it measures metrics/traces/ logs with no app instrumentation.

Q2. In this repo, istio_requests_total is produced by…

No Telemetry CRD — Envoy's merged Prometheus stats, scraped via the prometheus.io/scrape annotation.

Q3. That same metric already drives…

KEDA scales on rate(istio_requests_total…) (Course 2 L9), and Prometheus alerts query it for error rates.

Mesh observability — the three outputs and this repo's pipeline.

recall, then click to reveal

Because every call goes through an Envoy SIDECAR, the mesh emits telemetry for ALL traffic with NO app instrumentation. THREE OUTPUTS: METRICS (request count/ duration/sizes → Prometheus/Grafana/Kiali — covers golden signals; star = istio_requests_total counter), TRACES (spans → Jaeger/OTel), ACCESS LOGS (per request at the proxy). REPO: no Telemetry CRD — Envoy merged Prom stats scraped via prometheus.io/scrape (prometheus/values.yaml:424). Same metric powers KEDA (_keda.tpl:150, rate(istio_requests_total…) → replica count) AND alerts (:793-836). Tracing = Zipkin → opentelemetry-collector.monitoring:9411 @100% sampling; access logs JSON (istiod-values.yaml:14-21). Kiali draws the live service graph. (Full stack = Course 6.)

Want to see a Kiali graph, or why 100% trace sampling is unusual (and costly) in production? Ask me — the monitoring stack is Course 6.

1. Istio — Observability.

2. Istio — Standard Metrics.