Case study · Validation gate
An import is a database write
In short
A catalog import file that looks valid can still wipe live data: in a merge import, a blank cell deletes the live value and the import still reports success. alfa runs a validation layer between the build and the store: 143 gates in all, including 13 hooks that run before every write. No file leaves without a PASS in every category, and a file that would blank a live field fails before it ships.
Counts as of 28.10.2026. Dollar value: the 1-10-100 rule of data quality (Labovitz and Chang), $1 to stop an error at entry against $100 once it reaches the business.
A catalog is not a document
Handles group rows into products, variant rows hang off those handles, and image rows bind to both. An import file rewrites all three at once. The importer does exactly what the file implies, and the format has no way to say what its author meant.
That gap is where catalog updates go wrong, and they go wrong quietly. The import reports success and the row count matches. The damage sits in the records the file never meant to touch.
What the gate holds
The gate sits between build and delivery: locked rules, a certification suite by category, update-file checks, image-row checks and an image gate. Only the columns that change go into an update file, and every blank is checked. Each check below runs on the file itself, before it is allowed to leave.
Duplicate handles
The handle is the product identity. Rows that share a handle are one product: the first row carries the product fields, and every later row adds a variant, an image or an option value. A handle repeated by accident does not create a second product. It folds two products into one, and nothing in the file looks wrong while it happens.
The check: group the rows by handle before anything is sent. Reject the file if a handle repeats, or if a handle in the file is not already in the store.
validate_update_file.py# A handle is the product identity. Two rows sharing one are not two # products: the later row overwrites the earlier. counts = Counter(norm(r[HANDLE]) for r in rows[1:]) dups = [h for h, c in counts.items() if c > 1] check("no duplicate Handle (a later row overwrites an earlier one)", not dups, dups[:5]) # A handle that is not already in the store does not update the # product you meant, it creates a second one beside it. unknown = [h for h in counts if h not in live] check("every Handle exists in the store", not unknown, unknown[:5])
Blank cells in a column that is present
The unit of update is the column, not the cell. If a column is in the file, the importer reads every row as a statement about that column. A blank cell does not mean no opinion. It means the value is nothing, and the live value is overwritten with empty.
This is how a file built by filtering a full export down to the rows that should change erases fields it never meant to touch. A missing column means no opinion. A present column with an empty cell means delete.
The check: every column in the file has to be one the file deliberately sets. A blank cell in one of those columns fails the file unless the blank is declared as an intentional clear, and a declared clear still fails if it would wipe a value that looks valid.
validate_update_file.py# In a MERGE import an empty cell in a column that is present
# overwrites the live value with nothing. Find those before sending.
deletions = defaultdict(list)
for r in rows[1:]:
h = norm(r[HANDLE])
for i, field in update_cols:
if not norm(r[i]) and live.get(h, {}).get(field):
deletions[field].append((h, live[h][field]))
unexpected = {f: v for f, v in deletions.items() if f not in allow_delete}
check("zero deletions in a field not declared with --allow-delete",
not unexpected)
Command columns
The command column decides whether a section is merged into what already exists or replaces it. The same rows with the wrong command are the difference between adding a variant and replacing the variant set. An update file is usually built from an export that already carries a command, so the command is easy to inherit by accident.
The check: assert that every command column is present, populated and set to the operation this file is for. The baseline the file is judged against is the store as it is now: the last export plus every update imported since.
validate_update_file.py# The command decides whether a section is merged or replaced. An update # file inherits it from the export it was built out of, so assert it. if "Command" in header: ci = header.index("Command") bad = {norm(r[ci]) for r in rows[1:]} - {"MERGE"} check("Command = MERGE on every row", not bad, bad) # Judge against the store as it is now: the last export plus every # delta imported since. A stale export hides deletions. live = load_live(args.live, args.live_delta)
Image paths on variant updates
Images bind to variants by URL. A wrong path, an unreachable host or a stale query string raises no error at import time. The image does not attach, and the variant goes live without it. This one shows only on the storefront, which is the worst place to find it.
The check: resolve every image URL before the import runs. A bare file name fails the file, and so does any URL that does not come back as an image.
check_image_rows.py# A bare file name is not a URL. It fails silently at import time. bare = [r for r in rows if not str(r["Image Src"]).lower().startswith(("http://", "https://"))] check("every Image Src is an http(s) URL", not bare) # Then probe every distinct URL in parallel, before the import runs. with cf.ThreadPoolExecutor(max_workers=16) as ex: status = dict(ex.map(head, unique_urls)) check("every Image Src returns HTTP 200", all(v == 200 for v in status.values()))
How the gate grows
None of these checks were designed up front. Each was written on the day its failure mode was understood, and each has run on every file since.
A standard import pipeline handles the cases in its specification. This gate keeps every edge case it has met as a check, and stops it instead of reporting it. Every catalog that goes through it adds to it.
- A missing value stays blank. A guessed value is a defect.
- An update that would empty a full field on a live record fails the gate.
- No file leaves without a PASS in every category.
The files the gate checks come out of the build pipeline.