Lesson 11 · Traces, logs & the whole picture

Logs

The pillar that answers why — structured, shipped to Cloud Logging, no extra infra.

Your win: explain structured vs unstructured logging, how this repo's logs reach GCP Cloud Logging with no in-cluster log stack, and why the log format is tuned for Cloud Logging severities.

Structured logging — logs you can query

A metric tells you the error rate rose; a trace tells you it died in the payment call. The log tells you why — the actual error, the inputs, the stack. But a free-text log line ("failed to charge user 42: timeout") is hard to search at scale. Structured logging emits logs as JSON with fields ({"severity":"error","user":"42","err":"timeout","service":"payment"}) so you can filter and aggregate — "all errors for user 42 in the last hour" becomes a query, not a grep.1

Anchor — zap → stdout JSON → Cloud Logging The logger golib is zap (internal/golibs/logger/logger.go). Outside local dev it builds a JSON encoder writing to stdout, with the field names deliberately mapped for GCP Cloud Logging: LevelKey: "severity", TimeKey: "time", MessageKey: "msg", and log levels mapped to Cloud Logging's severity strings (debug/info/warning/error/critical). Locally it uses a colored console encoder instead — same logger, human-readable in dev, machine-readable in prod.
The surprise: there's no log pipeline in the cluster You might expect a Fluentd/Fluent Bit → Loki/Elasticsearch stack. There isn't one. Services just write JSON to stdout, and GKE's node logging agent automatically forwards container stdout to GCP Cloud Logging. Because the JSON uses severity/time/msg, Cloud Logging parses it into structured, filterable entries for free. So "how do logs get shipped?" has a refreshingly boring answer: they don't ship anything — they print, and the platform collects it. No Loki, no Elasticsearch for app logs.
Istio logs the same way The Envoy access logs (Course 4) also go to /dev/stdout as JSON (istiod-prod-values.yaml:20-50) — same path to Cloud Logging. So both your app logs and the mesh's request logs land in the same place, queryable together.

Correlation — logs are pillar #3, not an island

The payoff of structured logs is joining them to the other pillars. If a log carries the trace ID (Lesson 9), you can jump from a slow trace straight to the log lines for that exact request — metrics said what, traces said where, and the log says why, all for one request. Correlation across the three pillars is the whole point (Lesson 12).

Read this next

GCP Cloud Logging + structured logging

How stdout becomes structured log entries, and the severity/JSON conventions.

→ cloud.google.com — Structured logging
→ cloud.google.com — Cloud Logging overview

Check yourself (from memory)

Q1. Structured logging means logs are…

JSON fields let you filter/aggregate ("errors for user 42") instead of grepping free text.

Q2. In this repo, app logs reach Cloud Logging by…

zap writes JSON to stdout; GKE's node agent auto-forwards it to Cloud Logging. No in-cluster log stack, no Loki.

Q3. The log field severity is used so that…

zap maps levels to Cloud Logging's severity field (+ time/msg), so GCP shows them as structured, filterable entries.
Structured logging + how this repo ships logs (the boring answer).
recall, then click to reveal
Logs answer WHY. STRUCTURED LOGGING = JSON with FIELDS (not free text) → filter/aggregate ("errors for user 42") instead of grep. REPO: logger = ZAP (logger.go); non-local builds a JSON encoder to STDOUT with field names mapped for GCP Cloud Logging (severity/time/msg, level→Cloud Logging severities); local = colored console. SHIPPING (the surprise): NO in-cluster pipeline — services print JSON to stdout, GKE's node agent auto-forwards to GCP CLOUD LOGGING (parses the JSON fields). NO Loki/Elasticsearch. Istio access logs go the same way (/dev/stdout JSON). CORRELATION: a log carrying the TRACE ID lets you jump trace→logs for one request (metrics=what, traces=where, logs=why).
Want to see the zap encoder config, or how to add the trace ID to a log for correlation? Ask me.

1. GCP — Structured logging; in-repo internal/golibs/logger/logger.go.