Most quarterly technology reviews I've sat in are dashboard theater: a slide with uptime percentages and ticket counts, everyone nods, meeting ends, nothing changes. A real quarterly technology review doesn't need a dashboard. It needs one hour, five questions, and the willingness to hear an honest answer from the people actually using the systems.
The point of reviewing Q1 isn't to prove your systems are fine. It's to find the one thing that quietly cost you the most hours or money over the last three months, and fix that before Q2 repeats it. That requires talking to staff, not just reading a metrics panel.
Here's the one-hour agenda I run, and the five metrics that actually tell you whether your technology is helping or hurting.
Why dashboards alone miss the real story
A dashboard tells you a system was up 99.2% of the time. It doesn't tell you that staff have been manually re-entering data into a spreadsheet every morning because the sync between two systems breaks silently. Uptime and ticket volume are lagging, sanitized signals. The real cost sits in workarounds nobody logged as an incident because they just quietly adapted.
This is why the review has to include a conversation, not just a report pull. Ask your team directly: "What are you doing manually right now that you wish the system did for you?" That single question surfaces more real technical debt than any dashboard.
The one-hour agenda
- Minutes 0-10: What broke. Pull the incident list. Not just "system down" events, but anything that required an emergency fix, a late-night call, or a customer-facing apology.
- Minutes 10-20: What got slower. Compare cycle times for core processes (order to fulfillment, application to approval, ticket to resolution) against last quarter. Slower without a known cause is a leading indicator of accumulating friction.
- Minutes 20-35: What staff work around manually. This is the conversation part. Ask two or three people from different teams what they do by hand that the system should handle. Write every answer down, even the small ones.
- Minutes 35-50: What one change would save the most hours in Q2. Not a wishlist. One change, ranked by hours saved times how many people it affects.
- Minutes 50-60: Decide and assign. Pick the one change, assign an owner, set a date. If nothing gets assigned, the review was theater.
The five metrics that matter
| Metric | What it reveals | How to get it |
|---|---|---|
| Cycle time on core process | Whether the system is speeding up or slowing down real work | Compare start-to-finish time for your top 1-2 workflows quarter over quarter |
| Downtime and incident count | Reliability trend, not just a single quarter's average | Pull from your incident log, weight by business hours affected |
| Manual workaround count | Hidden technical debt no dashboard shows | Ask staff directly, log every answer |
| Support ticket recurrence rate | Whether root causes get fixed or just patched | Track how many tickets are repeats of a previous issue |
| Cost per transaction or per unit processed | Whether technology spend is actually reducing cost, not just adding tools | Total system + labor cost divided by volume, tracked quarter over quarter |
These five give you a fuller picture than uptime and ticket count alone, because they connect technology performance directly to business outcomes: speed, reliability, hidden labor, recurring failure, and cost.
A short case in point
A pharmacy chain I reviewed had a dashboard showing healthy uptime across their inventory system. The one-hour conversation surfaced that three branch managers were still manually cross-checking stock counts against a paper log every week, because the system's forecasting had been wrong often enough that nobody trusted it. That's a workaround a dashboard would never show you, and it was costing more staff hours than any recorded incident that quarter. For how that kind of gap gets closed with better forecasting discipline, see A Pharmacy Chain Let Data Decide Its Reorders.
Turning the review into next quarter's plan
Resist the urge to leave this meeting with ten action items. Pick the one change that saves the most hours across the most people, assign a single owner, and set a real date. If you consistently find yourselves with more findings than you can act on, that's itself useful data, it tells you where to set your next quarter's technology goals. I've written before about why three goals beat ten in Set 3 Technology Goals for the New Year (Not 10), and the same discipline applies here: fewer, funded, finished beats a long list that nobody owns.
If your team doesn't have the internal bandwidth to run this kind of review objectively, an outside technical partner can run it in a way that avoids the "everything's fine" bias internal teams sometimes default to. That's a conversation worth having before it becomes a Q2 fire drill, and it's the kind of engagement I take on through partner.
The takeaway
Anti-dashboard, pro-conversation. An hour spent asking what broke, what slowed down, and what your staff quietly work around will tell you more than a quarter of metrics snapshots. Pick one change, assign one owner, set one date, and you've turned a review into an actual plan instead of a status update nobody acts on.