← Inside orch contents

Chapter 38 worked answers

These answers use baseline cbfc94ade5b57e449d6b73bf1485b1c5b1647022 and the chapter's linked execution records. Procedures remain scoped to disposable resources.

1. Follow the producer and consumer

instrumentedRuntime.measure increments operation attempts and returned errors. Agent registry exposes docker_operation_errors_total with operation pull_image. prometheus.yml scrapes the agent job. DockerOperationErrorsHigh sums its five-minute rate above0.1for10minutes, and the Docker Operation Errors dashboard groups the rate by operation. The teaching fixture actually executed the wrapped fake failure and registry exposition; scrape/rule/panel were inspected, not live-evaluated.

2. Requests and distinct transitions differ

api.go701–702 increments after each accepted request whose requested status is failed. MemoryService can accept a duplicate failed report without emitting another transition event. Therefore the metric may count two accepted reports for one task failure. The source-inspected semantics support this distinction; do not invent a distinct-task cardinality guarantee from the metric's name.

3. Rename proxies honestly

Task State Counts should read cumulative reported task state changes. Node Status Counts should read agent scrape reachability. The former sums task_state_changes_total; the latter counts up values. Authoritative current counts need current task/node state from a verified control-plane inventory or explicitly maintained current-state gauges. Counter subtraction cannot generally reconstruct state after arbitrary transitions or restarts. Rollout requests minus failures likewise is not terminal-success inventory.

4. Test temporal and absent-data behavior

Use an owned Prometheus instance with exact job labels and rule version. Inject a short burst, observe samples/rate, and verify the ten-minute hold does not fire for an insufficient sustained condition. Then inject sustained qualifying error rate and observe pending/firing/recovery timestamps. Separately stop the agent scrape and inspect up plus heartbeat-series absence. A single fixture increment supplies no rate history or evaluator time. Retain workload recovery evidence independently from alert disappearance; no such live rule run was claimed in this chapter.

See the corresponding chapter claim ledger for exact immutable source anchors and limitations.

Return to Chapter 38.