How do you run your first AI pilot, step by step?

Why most pilots stall

A pilot is a short, small test of one AI tool on one real task, run so you can decide whether to keep it. Most stall for three reasons. Nobody owns it, nobody recorded what the task looked like before, and it never ends, so it drifts on as an interesting experiment that never becomes a decision.

Set it up before anything starts

Pick one task that repeats and is a little dull, such as drafting replies to the same kinds of enquiry or matching supplier invoices. Name one owner and free real hours for them, about half a day a week for a single tool in one department. A pilot handed to an enthusiastic person on top of a full week gets attention for two weeks, then whenever there is time.

Then take the before number. For two weeks, record how long the task takes, how many are done and how often one comes back wrong. Write down what success looks like and what would make you stop, as numbers. Without them, the result is a discussion about impressions.

Run it on real work

Use a small group of real users on real work, including the bad Tuesday: the scanned fax, the customer who writes in fragments, the job with three revisions. A pilot tested on tidy examples chosen by someone who wants it to work tells you very little. A person checks every output, and nothing reaches a customer unread.

Hold a short review each week. Look at the mistakes, decide what to fix, and change one thing at a time: the instructions, the documents it reads, or the way the task is handed over. Write the fixes down, because the pattern in the mistakes is what the pilot is for.

End it on a date and decide in writing

Fix the end date at the start, a few weeks out, and hold to it. On that day, put the starting number beside the finishing number. Look at two more: how often a person had to fix the output, and how many of the people who had the tool used it each week. A tool with few corrections and little use solved a problem nobody had.

Then decide in writing: keep, widen or stop. Name the decision-maker before the pilot begins, because a pilot without one ends as a report that circulates and dies. Stopping is a fair result. You learned it for the price of a few weeks, and you can try the next task.

What to do this week

Choose the task and the owner today. Write a one-page plan: the task, the owner, the people using it, today's numbers, what success and stopping mean, the end date and who decides. Start recording the before number this week. Ask too what it would take to run it for real, such as connecting it, permissions and support, so the yes at the end is a yes to the real cost.