Who's Writing Your Success Criteria?

Most sales engineers I’ve worked with are nervous about bringing pre-written success criteria into a proof of concept. It feels presumptuous, like putting words in the customer’s mouth or steering the evaluation toward the answer you want.

I understand the instinct, you certainly don’t want to feel like you’re steam-rolling your customer. This is why it needs to be done thoughtfully and purposefully.

Important: Keep your foot out of your mouth

There should never be criteria introduced in an evaluation that your product hasn’t repeatedly proven it can fulfill in any environment.

This may seem obvious, but “hopeful” testing criteria abound. These delusions of evaluation grandeur come from many sources. Overzealous founders. Engineers with some new mind-blowing skunkworks project. But overwhelmingly, I see these types of criteria emerge from marketing documentation. Marketing allows for disclaimers and asterisks, evaluations don’t. If something was proven “in an independent study”, that’s fantastic for bringing new customers in, but unless you’re using the same lab with the same “controlled conditions” they did to acquire those findings, you may be setting yourself up for a failed proof-of-concept.

All of the success criteria available to a customer should be written and approved by your technical team before a proof of concept starts. In my experience it’s best to find the most skeptical engineer on your team to help with this. It’s better to have 10 criteria you can win every time than 50 “maybes”.

What you’re actually asking of someone

Testers have day jobs, they aren’t getting paid to only test your product. The hours they spend standing up your software and sitting on calls with you come out of the work they’re actually measured on.

In my experience prospects are genuinely appreciative when you arrive with prebuilt criteria. You’ve saved them the part of the job that requires expertise they have no reason to have.

Handing that person a blank page and asking them to define what a successful evaluation looks like is homework, assigned to someone already strapped for time. It’s like asking a student to write a test about a subject they only have cursory knowledge of, and then take the test.

Is writing test criteria rigging the eval in your favor?

Some will argue that pre-building the criteria is rigging the eval. Here’s my view.

Your customer is looking to you for expertise, otherwise they wouldn’t be testing your product in the first place. If I go to a sporting goods store and say “I need a bicycle that can go off-road,” and they tell me one of my buying criteria should be knobby tires, that’s just me benefiting from the store’s expertise. If they hadn’t told me that and I’d gone off and bought a road bike, I would have been pretty upset when my tire popped on the first rock I hit.

If your criteria accurately map to the customer’s needs, with sign-off, and the other products can’t meet them, that isn’t rigged either. That’s the evaluation working.

The word “accurately” is a key component here. Criteria that map to the customer’s actual needs showcase your team’s expertise and understanding of their goals with the evaluation. You should never deliver a list of criteria that’s just a list of features your product has. Not only does that come across as a waste of time, but it invites scope creep that could potentially crater your eval on an unforced error.

What’s the alternative?

There are two possibilities:

Prebuilt required. Mainly state-run entities and government accounts, where you will get a list of requirements, but in my experience these are in the extreme minority of deals.
Customer wants to write their own. Customers writing their own criteria can help keep the tester invested, but it isn’t required. A bit of leeway can be given if the customer wants to introduce criteria of their own, as long as they’re specific, measurable, defensible, and relevant. If your criteria are solid and well thought out, this generally shouldn’t happen.

For the latter, there’s a practical reason now that didn’t exist a few years ago. Any time-strapped, savvy security engineer is just going to ask Claude or ChatGPT to write their criteria anyway. If they use your criteria instead, they (and you) can rest assured they won’t be dealing with hallucinated requirements.

This is a point worth considering. The alternative to you writing the criteria isn’t going to be the customer carefully authoring their own from scratch. It’s a busy engineer generating a list in thirty seconds from an LLM that has never seen your product, has no idea what’s technically reasonable in your category, and might be inventing capabilities on its own. Then you spend the evaluation explaining why some of the criteria are impossible, which is a worse conversation than the one where you brought a curated list your team spent time and effort on creating.

Write them for everywhere

One caution about where your team lets customers run their evaluations.

Most enterprise networks are a mess. Aging hardware, mismatched subnetting, naming inconsistencies in multiple languages, every manner of misconfiguration imaginable. For a detection product to show value, there has to be something to detect, and you don’t control any of it. For a segmentation product to show value, there already has to be some semblance of order in the network your product is being deployed to.

Your product may shine in the chaos. But if you have even a shadow of a doubt that it might fold under the pressure, an evaluation in a customer’s production environment may not be the best route.