The errors that slip into a scraped dataset unnoticed rarely come from the fields everyone worries about. Prices and dates get normalization rules because they fail loudly; the fields below fail quietly, because a value such as a weight, a rating or a phone number looks valid on every row while meaning something different from source to source. Each of the seven fields gets the same treatment: the error that creeps in, a concrete check with a hypothetical example, and what that check catches and what it misses. None of these rules replaces the ones for normalizing scraped prices or dates and times in scraped data; they sit beside them.
One pattern repeats across all seven. Keep the raw string exactly as scraped, derive the normalized form in a separate column, and reserve an explicit unknown state for values the rule could not interpret, instead of forcing them into the nearest valid value. The raw column is what lets a rule be fixed and rerun months later; the unknown state is what makes the rule's limits visible in a count rather than hidden in a plausible-looking default. Every check below is a comparison between the raw and the normalized columns, usually grouped by source, because a source that systematically differs from the others is the most common way these errors show up.
1. Units and measurements
A weight column that reads 500, 0.5 and 1.1 across three sources is probably grams, kilograms and pounds, and an average computed over it is meaningless. The error enters because the number is parsed and the unit text next to it is dropped, or because one source states the unit in a column header rather than beside each value. The rule is to store three things: the raw string, a numeric value and an explicit unit code, then convert to one base unit in a separate column. Suppose a hypothetical catalog of two thousand kitchen appliances from four retailers; the check is to group the base-unit weight by source and compare the medians. If one source's median is a thousand times smaller than the others, its unit was misread. This check catches systematic unit confusion per source; it misses a single product listed in the wrong unit on an otherwise correct site, which only a range check per product category would find.
2. Ratings and scores
A rating of 4 means something different on a five-star scale, a ten-point scale and a scale where 4 is a percentage. Sources also differ on half-stars, on whether the figure is rounded for display, and on whether a product with no reviews shows zero or nothing at all. The rule is to store the raw rating, the scale maximum as its own column, the review count, and a normalized score on a fixed scale such as zero to one. The check, on the hypothetical appliance catalog, is a frequency table of normalized scores per source: a five-star source shows a handful of distinct values and a ten-point source shows more, and a source whose scores cluster at exactly 0.8 on every row is probably showing a default placeholder rather than real ratings. The check catches scale mismatches and placeholders; it misses the subtler bias of sources that display rounded figures, which can only be noted in the field documentation, not corrected.
3. Phone numbers
Phone numbers arrive with and without country codes, with spaces, dots, parentheses and dashes, with extensions in free text, and sometimes with two numbers in one field. Deduplication and matching against your own records fail when the same number appears in three spellings. The rule is to keep the raw string and derive a canonical form with a country code inferred from the source's country when the number lacks one, with the extension in a separate column, and to reject values whose digit count falls outside the plausible range for that country. Suppose a hypothetical directory of twelve hundred public business listings; the check is to count distinct canonical numbers against distinct raw strings, and to list raw strings that produced no canonical number. A large gap between the two counts means the normalization is collapsing spellings as intended, and the rejected list shows what the rule could not handle, such as vanity numbers written with letters. The check catches formatting variation and malformed values; it misses a number that is well formed but belongs to the wrong business, which is a sourcing error, not a formatting one.
4. Postal addresses and place names
Addresses from different sources abbreviate differently, order their parts differently, place unit numbers before or after the street, and name the same city in several ways. A grouping by city then splits one city into four. The rule is to parse the address into components with a rule set per country, to normalize the locality name against a reference list of place names, and to keep the raw string for anything the parser could not place. In the business directory example, the check is a count of listings per normalized locality compared with a count per raw locality string; the normalized count should be smaller and the difference should be made of recognizable variants. A second check is to look at the components labeled as unknown and confirm they are genuinely odd rather than a common pattern the parser missed. These checks catch spelling and abbreviation variants; they miss two different places that legitimately share a name, which requires the region or postal code to resolve.
5. Availability and yes-or-no flags
A field meant to be true or false, such as in stock, free shipping or verified, is rarely boolean at the source. It arrives as yes, no, in stock, only two left, ships in three weeks, call for availability, or as the presence or absence of a badge. Mapping all of it to true or false loses the states in between and treats an absent badge as a definite no. The rule is to map to a small set of named states, such as available, limited, backorder, unavailable and unknown, with the raw text preserved, and to reserve unknown for cases where the source showed nothing. The check, in the appliance catalog, is a list of every distinct raw string per source with its assigned state and its count; read it once and the mapping errors are visible, such as a limited-stock phrase mapped to unavailable. This catches mapping mistakes and new phrases a source introduced; it misses a source that shows the same badge for both in-stock and backorder items, which no mapping can separate and which belongs in the coverage notes.
6. Categories and tags
Every source has its own taxonomy. One has a three-level category path, another has flat tags, a third uses plurals where the first uses singulars, and a fourth changes the names between seasons. A combined dataset grouped by source category is unusable, and a naive lowercase-and-trim pass does not fix it. The rule is to keep the source category path verbatim, to maintain a mapping table from each source's categories to a target taxonomy you control, and to flag source categories that have no mapping yet rather than dropping the rows. Suppose the appliance catalog maps about ninety source categories to twenty target ones; the check is a weekly list of source categories that appeared in the data but not in the mapping table, with the number of rows affected. A non-empty list means the source changed its taxonomy or the scraper picked up a new section. This catches new and renamed categories; it misses a category that kept its name while changing what it contains, which shows up only as a shift in the products under it, the kind of drift described in when scraped values quietly change meaning.
7. Person and organization names
Names arrive in different case, with and without titles, in given-family or family-given order, with suffixes and legal forms attached to companies, with trailing whitespace, and sometimes with a role or a location appended in the same field. Matching and deduplication then fail, and any count of distinct authors, sellers or organizations is inflated. The rule is to store the raw name, a display form with consistent case and whitespace, and a matching key that strips punctuation, legal forms, titles and diacritics and lowercases the rest, with organization and person names handled by separate rules. In the business directory, the check is to sort by the matching key and look at adjacent keys that differ by one character or one token: those pairs are either genuine near-duplicates to merge or distinct entities the key is about to merge wrongly, and a human needs to look at a sample of each. This catches formatting variants and most duplicates; it misses the case where two different organizations have the same name in different places, so the key should never be used alone to merge records, only together with a location or an identifier. Where a provider such as ScrapingIsNotACrime delivers the data, ask for the raw field to be kept alongside any normalized form, because the raw value is the only thing that lets a rule be corrected after the fact.