Does AI Work? How to Measure Whether It Pays Off
Erwin Berkouwer
You measure whether AI works with three numbers: the time a process takes, the number of errors and how satisfied your customers are. You record them before the AI goes live and compare them afterwards. All you need is a tab in Google Sheets and a few weeks of discipline.
Say you've put €5,000 into AI. Three months later your managing director asks: "So, is it paying off?" If all you can say is that it seems to be working, you're on weak ground. Many SMEs feel things are going better, but can't prove it. That isn't an argument when you have to justify an investment.
Why is "it feels better" not an answer?
Ask whether AI is paying off and you often get answers like:
- "Yes, it works pretty well..."
- "The team is happy with it..."
- "We're saving time, I think..."
That isn't a measurement, it's a feeling. A feeling doesn't hold up when costs rise, when a new colleague asks why you do this, or when next year you want budget for the next step. Measuring isn't bureaucracy: it is how you win a second investment on solid grounds.
The trap is thinking measuring is something big, like a BI tool, a consultant or a dashboard. It starts with writing down how things are going now, before the AI goes live. That baseline is later your only honest point of comparison.
Which three things do you measure?
Forget the rest. For most AI projects in SMEs, these three KPIs give you the most insight.
1. Time: how many hours do you take out of the process?
The most tangible gain. Measure how many hours a week a process took before AI and how many it takes now.
- Before AI: how many hours a week does the task take?
- With AI: how many hours does it take now, including checking the output?
- Difference × hourly rate = your time saved in euros.
Use a realistic hourly rate: what that employee's time actually costs you. Don't take minimum wage, and don't take the rate you bill to clients either. As a rule of thumb, in SMEs this is often roughly between €45 and €75 an hour, depending on administrative or specialist work.
Measuring tip: don't just ask the team how long something takes, because people often guess wrong. Have them track it for two weeks in a free time-tracking tool such as Toggl or Clockify. Two weeks before AI, two weeks after.
2. Errors: how much rework do you prevent?
Fewer errors means fewer corrections, less hassle and fewer hidden costs. Count the error rate before and after you bring in AI and add the cost of rework.
An error that used to surface only at the customer and is now caught up front saves not just repair hours but goodwill too. Goodwill is hard to put in euros. So count only what you can back up, the correction hours, and mention the rest separately as a bonus, not included in the sum.
3. Customers: do they notice the difference?
Faster or more consistent service leads to happier customers, less churn and sometimes more revenue. This is the softest of the three, so keep the measuring simple: after an interaction, send one NPS question ("How likely are you to recommend us, 0 to 10?") and compare the average before and after AI.
Only count a rise in satisfaction in euros if you have a plausible link, for example fewer cancellations in the following quarter. Otherwise it is an indicator, not a return.
The three at a glance
| KPI | How do you measure it? | Handy tool | Biggest pitfall |
|---|---|---|---|
| Time saved | Track hours, 2 weeks before and after | Toggl, Clockify, Google Sheets | Estimating hours from memory |
| Fewer errors | Count errors per 100 tasks or orders | Existing system + tally sheet | A different definition of "error" |
| Customer satisfaction | One NPS question after the interaction | Email, simple survey | Too few responses to say anything |
A worked example: a construction company automating quotes
This is a constructed SME scenario, not a UnifyAI client, meant to show the proportions.
A construction company with about fifteen employees wants to produce quotes faster. The AI drafts them based on earlier quotes and standard items. The assumptions, in ranges:
- Investment: a setup of €2,500 to €5,000 and then €100 to €250 a month. That is a plausible range for a clearly defined automation. What else comes on top, you can read in what an AI agent project costs.
- Time: a quote used to take 3 to 4 hours, now 30 to 45 minutes. At 8 to 12 quotes a week, that is roughly 20 to 30 hours saved a week. At €50 to €60 an hour, that adds up to tens of thousands of euros a year.
- Errors: fewer wrong items and forgotten lines means less recalculation and less loss on projects.
- Customers: someone who gets a clean quote within a day instead of after a week may well choose you more often.
The exact ROI depends on your assumptions. In worked examples like this the payback time is short, but more important than the exact number is that you can now back it up, instead of saying it feels faster.
When is the best time to measure?
Timing decides whether your measurement is worth anything.
Not too early. Give a new system 4 to 6 weeks to settle in. The first weeks are messy: bugs, adjustments, people getting used to it. Measure then and you mostly measure the chaos.
Not too late. Measure within three months. Wait longer and you've lost your baseline, so you have nothing to compare with.
A workable timeline:
- Week 0: baseline. Record how the process stands now: hours, errors, satisfaction. Without this point, the rest is worthless.
- Week 6: first check. Is it working as intended? If not, adjust before you draw conclusions.
- Week 12: ROI measurement. Compare with week 0 and convert it to euros.
- After that, monthly. One line a month in the same sheet. That shows whether the gain holds or fades.
What do we see in practice?
We've automated a range of processes for Dutch companies, including payment reminders for an accountancy firm, routing support tickets, administrative email handling and an email responder that prepares draft replies. What those projects have in common isn't a special model, but a sharply defined process and a baseline up front.
The gain is rarely in AI taking everything over, but in removing repetitive, error-prone work nobody enjoyed. That can be measured with the three KPIs above. That is why we advise starting with one process instead of a fifty-page plan. You can measure one process, you can't measure a whole programme.
The flip side: a good share of AI projects in SMEs don't make it to the finish line, usually because the scope is too complex or too few people use it. You can read how to prevent that in why so many AI implementations fail. Measuring helps here: it shows early whether a process is taking off, so you can adjust before you put more into it.
How do you start with a baseline today?
You don't have to wait until the AI is live. If you don't measure now, you'll have nothing to compare later.
A concrete action: pick one of the three KPIs. Set up a tab in Google Sheets with three columns (what you measure, before, now) and have your team keep it up for the next four weeks. After that, you talk about AI from data, not from expectations.
Want to know which process in your business is best suited to measuring and automating? Run an AI scan for a first starting point. If you get stuck setting up a measurement plan, look at AI consultancy or book an introduction call.
Frequently asked questions
Short, clear answers so you can decide faster.
How long does it take before AI shows measurable results?
With process automation you often see results after a few weeks: fewer hours, fewer errors. For more complex uses you usually need two to three months of data for reliable conclusions. Always measure once before the implementation.
Which KPIs do I use to measure AI success?
The three most useful are time saved per task in hours, the error rate before and after, and a customer satisfaction score. Pick three at most and measure them consistently, otherwise you get noise and arguments about what really works.
What if the AI investment doesn't pay off enough?
First analyse why: was the process too complex, is the data quality off, or isn't the team using it? The problem is more often in process and adoption than in the technology. Optimise, pick a different process or stop on purpose.
How do I present the AI ROI to my management or CFO?
Use concrete numbers: hours saved × hourly rate, plus error reduction in euros, plus any extra revenue. Set that against the setup and monthly costs and state your assumptions explicitly.
Do I need an expensive dashboard to measure this?
No. A tab in Google Sheets or Excel with three columns (what you measure, before, now) is enough to start. For hours, use a time-tracking tool such as Toggl or Clockify, and for customer satisfaction, one NPS question by email.
