Two software vendors can quote you the exact same price for the exact same feature, and one of them will ship it in two weeks with zero production incidents while the other takes six weeks and breaks something else on release day. The automated testing business case is the reason for that gap, and it's almost never something business owners think to ask about before signing a contract.
I've sat on both sides of this. As a technical lead, I've inherited systems with no test coverage and watched every release become a stressful, all-hands event. I've also built systems with strong automated tests where a junior developer can ship a change on a Friday afternoon with confidence, because the tests catch what a human reviewer would miss.
You don't need to write code to understand this. You need to know what question to ask your vendor, and what a healthy answer sounds like.
What Automated Testing Actually Is, in Plain Terms
Automated tests are code that checks other code. Instead of a QA person manually clicking through the app after every change to confirm nothing broke, a suite of automated tests runs in minutes and checks hundreds or thousands of scenarios: does the discount calculation still work, does the login still reject bad passwords, does the invoice total still match after a tax rate change.
The tests aren't a nice-to-have layer added at the end. In a well-run engineering team, they're written alongside the feature, sometimes before it, and they run automatically every time anyone proposes a change.
The Foundation Analogy
Ask a contractor about their foundation work and a competent one will tell you specifics: depth, rebar spacing, curing time, load calculations. An incompetent or corner-cutting one will wave you off with "it's fine, trust me."
Ask a software vendor about test coverage and the same pattern holds. A team that takes quality seriously will tell you specifics: what percentage of the codebase is covered, what kinds of tests run (unit, integration, end-to-end), and what happens automatically before code reaches production. A team cutting corners will say something like "we test everything manually before release" and change the subject.
The foundation is invisible until the building has a problem. Test coverage is invisible until your software has a problem, at which point it's expensive and public.
The Cost of No Tests, Concretely
I took over a project for a retail chain in Tangerang whose previous vendor had built a promotions engine with no automated tests. Every price rule change required manually testing dozens of combinations by hand, and it still broke roughly one release in four, usually discovered by a cashier at checkout, not by QA.
The direct costs:
- Slower releases. Every change required a manual regression pass, turning a two-hour feature into a two-day release cycle.
- Production incidents. Wrong prices at checkout, sometimes undercharging customers for days before anyone noticed.
- Fear-driven development. Developers became reluctant to touch old code because nobody could confirm what would break, so the codebase calcified.
- Onboarding drag. New developers had no safety net to learn the system against; every change was a leap of faith.
We rebuilt the core pricing logic with a test suite covering the business rules. Release cycles went from two days back to hours. Production pricing incidents dropped to roughly zero over the following year. The upfront cost of writing tests was real, but it was smaller than one bad pricing incident at scale.
What a Healthy Answer From a Vendor Sounds Like
When you ask a vendor or your internal team about testing, listen for specifics like these:
| Healthy signal | Warning sign |
|---|---|
| "We run automated tests on every change before merging." | "We test manually before each release." |
| "Coverage is around 70-80% on core business logic." | "We don't really track that number." |
| "Tests run in our CI pipeline automatically, in minutes." | "QA does a full pass, takes about a week." |
| Can name what's NOT tested and why (a real tradeoff) | Claims everything is fully tested (unlikely to be true) |
| Bugs found get a test added so they can't silently recur | Same category of bug keeps recurring across releases |
Nobody has 100% coverage, and that's fine. What matters is that the team can tell you honestly where the gaps are and why, the same way a contractor should be able to tell you which parts of a build carry more risk.
Why This Matters for Delivery Speed, Not Just Quality
The counterintuitive part: strong automated testing makes delivery faster, not slower. Teams without tests slow down over time as the codebase grows, because every change requires more manual verification to feel safe. Teams with tests can move at a near-constant pace for years, because the test suite does the verification work a human would otherwise redo from scratch every release.
If your current vendor is getting slower on delivery the longer you work together, that's often the tell. It's worth raising directly, and it's a legitimate item to negotiate into a contract renewal, alongside other terms covered in negotiating software contracts. It's also a fair question to raise during any quarterly tech budget review, since slowing delivery velocity is a cost even when no line item captures it directly.
The Practical Takeaway
Ask your vendor or internal team one question at your next project review: "What percentage of the core business logic is covered by automated tests, and what happens automatically before code reaches production?" A specific, confident answer is a good sign. A vague one, or a pivot to "we test manually," is your signal to ask harder questions before your next contract renewal, not after the next production incident.