Analysis & Reporting

Data is not insight. Phase 5 converts gigabytes of measurements into specific, defensible answers to the questions the test was commissioned to settle.

Analysis sequence

Validate the run first. Before interpreting results we confirm the test itself was sound: target load actually applied, error rates within the test's own tolerances, no load-generator saturation, no environment interference. Invalid runs are discarded, not explained around.

Read the percentiles, not the average. Latency distributions are long-tailed; averages hide the tail your unhappiest users live in. We report p50/p90/p95/p99 over time, never a single summary number. (Why: Key Performance Metrics.)

Correlate client symptoms with server causes. Every client-side observation — a latency knee at 410 tps, an error burst at minute 42 — is matched against time-aligned server metrics to find the mechanism: a saturating pool, a GC pause cluster, a lock convoy, an autoscaling event. A symptom without a mechanism is reported as an open question, not dressed up as a conclusion.

Findings, not folklore

Each finding in our reports states: the observation (with data), the mechanism (with evidence), the business impact (what happens in production if unaddressed), and a specific recommendation with expected effect. Recommendations are prioritised by impact against the agreed objectives — not by how interesting the engineering is.

Finding F-03 (high priority)
  Observation : throughput plateaus at 410 tps; p99 rises
                from 0.9 s to 11 s between 400–450 tps
  Mechanism   : DB connection pool (max=50) saturates;
                threads queue at pool acquisition
                (evidence: pool wait time, thread dumps)
  Impact      : peak-hour forecast of 520 tps will breach
                SLOs with cascading timeouts
  Action      : raise pool to 120 (DB headroom confirmed
                to 200 conns); retest — expected ceiling
                > 600 tps

Reporting for two audiences

Executives get a one-page verdict: objectives met or not, top risks, decision recommendation. Engineers get the full technical report plus the raw data and analysis notebooks to verify or extend every conclusion. Both versions are walked through live, with your team, before we close the engagement.