Culling Scorecard: Score AI on Your Own Gallery

The four numbers that matter

Score a tool on keepers found, false rejects, extras and minutes, in that order. Keepers found is the share of your own picks that the tool also selected. A false reject is a frame you would keep that the tool marked as a reject. Extras are frames the tool selected that you would not keep. Minutes is how long you spend after the tool finishes: reviewing, overriding and fixing.

How to run the test

1. Cull the gallery yourself first, without opening any tool, and write down how long it took. Mark your keepers and note which keepers are the only frame of a moment (no near-duplicate to fall back on).
2. Run the tool on the same untouched files. Where the tool lets you set a target count, you can also run it at your keeper count, but record that run separately: when the tool selects exactly as many frames as you kept, extras and false rejects are equal by construction, precision equals keepers found, and only the overlap is informative. Aftershoot's own pricing post lists a "Cull to Target" feature (vendor claim). Record the settings you used.
3. Count overlap. Count how many of your keepers the tool selected, and how many frames it selected in total.
4. Time the review. Start the clock when the tool finishes, stop when your final selection is ready. Include the time spent opening its reject pile; Adobe's Assisted Culling tutorial describes reviewing rejects and overriding them, and the habit applies to any tool.
5. Enter the numbers below and read the result.

Why the order matters: if you look at the tool's picks before making yours, you will anchor on them and your keeper list stops being independent. Our how to cull wedding photos guide covers building your own list, and what is photo culling explains why vendor accuracy percentages are not comparable with each other.

How to read the result on a single gallery

One gallery is a sample of one, and it tells you about that kind of day, not about the tool in general. Frames in a burst are correlated, so 3,000 frames are not 3,000 independent trials; the number that counts is closer to your count of distinct moments. Treat the result as a pass-fail screen for your own shoots.

• False rejects deserve the most weight. An extra costs you seconds to dismiss. A false reject can cost you a frame that exists nowhere else, unless you review the reject pile. Count the unrecoverable ones separately.
• Small counts are noisy. With 300 keepers and 270 found, a 95 percent Wilson interval runs from about 86 to 93 percent (illustrative arithmetic). Because bursts are correlated, the true range is wider than that.
• Minutes saved is personal. Your manual time is the baseline, so a slow baseline flatters the tool. Re-timing a gallery you have already culled is faster than the first pass, which makes a second-pass baseline unreliable.
• Do not compare tools on different galleries. Run each tool on the same files, with the same keepers list.

Per-tool context is in the Aftershoot review and Narrative Select review; for cost, use the cost per 1,000 images calculator.

·