[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"article-body-an-evaluation-is-not-a-pilot":3},"\nThe enterprise pilot has a design flaw so standard it is invisible: it\nis scoped to succeed. The success criteria are vague enough to be\nunfailable, \"validate the use case\", \"explore the potential\". The\nmeasurement is chosen after the results exist, which guarantees a\nfavourable measurement exists to be chosen. And the vendor's best\npeople hover over the deployment in a way they never will again.\n\nThen the pilot succeeds, the purchase follows, the rollout meets\nreality, and the value fails to materialise at scale. The famous\nstatistic about enterprise AI pilots delivering no measurable impact\n([and how it gets misread](/insights/where-the-95-percent-goes)) is\nmanufactured in large part by this exact design: not by technology\nfailing, but by nothing being set up that could measure whether it\nworked.\n\n*A pilot proves the software runs. It was never designed to prove the\nsoftware matters.*\n\n## The one discipline that changes everything\n\nThe alternative is an evaluation, and nearly all of the difference is\na single discipline applied at the start: **the metric and the\nbaseline are agreed in week one, in writing, before anyone has seen a\nresult.**\n\nWhich number should move: forecast error on these items, alert\nfalse-positive rate in this queue, days earlier that this risk is\ndetected, hours out of this close process? What is that number today,\nmeasured honestly? What movement, by week eight, would justify a\npurchase, and what movement would not?\n\nEverything else in the eight weeks is execution:\n\n- **Weeks two to seven: run on your data, with your users, inside\n  real decisions.** Not a sandbox, not curated extracts, not the\n  vendor driving. The system takes the inputs your operation\n  actually produces, and its outputs land with the people who would\n  own them in production.\n- **Week eight: measure against the week-one baseline.** The result\n  is a number against a number agreed before the exercise could bias\n  it. Either the metric moved past the threshold, or it did not.\n\n> Both outcomes are wins. A clean \"yes\" is a purchase justified by\n> evidence. A clean \"no\" is a purchase avoided for the price of eight\n> weeks, which is the cheapest failed AI initiative your organisation\n> will ever run.\n\n## Why vendors resist, and what resistance tells you\n\nAn evaluation on these terms transfers risk from buyer to vendor,\nwhich is why the structure is rare. A vendor whose product creates\nmeasurable value inside eight weeks can accept the terms routinely;\nthe evaluation is simply their product doing what it does, observed.\nA vendor whose value story depends on multi-year transformation\nnarratives, integration programmes before any measurable effect, or\nmetrics chosen in retrospect, cannot accept them, and the ways they\ndecline are informative. \"Value takes longer to emerge\" sometimes\nmeans the value is real but slow. It reliably means the value is\nunmeasured.\n\nTwo honest caveats keep the standard credible. First, eight weeks\nsuits decisions with fast feedback: forecasts, alerts, triage,\nprocess cycle times. Outcomes that resolve over years need staged\nproxies, agreed with the same week-one discipline. Second, an\nevaluation needs enough historical data to establish the baseline\nhonestly; where the baseline itself is unmeasurable, that discovery\nis the first finding, and it is worth having before any purchase\nrather than after.\n\nThe deeper point is symmetry. The seller should be as exposed to the\nmeasurement as the buyer is to the purchase. Week-one metrics are\nhow a buyer finds out, cheaply, which vendors have been exposing\nthemselves to measurement all along.\n\n*Agree the number before anyone sees a result. Everything after that\nis just finding out.*\n\nThe eight-week value evaluation is how Prophesee starts every\nengagement. Your data, your users, a week-one baseline, and a week\neight measured against it, or the evidence that you should not buy.\nAgree your metric. [Start here](/contact).\n",1786799034544]