Regular Expressions for Data Cleaning: Match Structure, Not Meaning

Regular expressions describe patterns in text. In data cleaning they are useful for three distinct jobs: validating a shape, extracting components, and transforming a known representation. Problems begin when a shape match is treated as proof of meaning.

Read a pattern in layers

For an IPv4-like shape:

^(\d{1,3}\.){3}\d{1,3}$

• ^ and $ anchor the whole value;
• \d{1,3} matches one to three digits;
• \. matches a literal dot;
• {3} repeats the grouped prefix three times.

It accepts 999.999.999.999, so it validates syntax but not the numeric domain. Parse the four components and enforce 0..255 separately.

In plain terms: regex can recognize the shape of an address on an envelope. It cannot prove the address exists.

Match, extract, and replace are different operations

import re

raw = "2026/9/7"
m = re.fullmatch(r"(\d{4})/(\d{1,2})/(\d{1,2})", raw)
if m:
    year, month, day = map(int, m.groups())

After extraction, use a date library to validate month lengths and leap years. Only then emit an ISO value such as 2026-09-07.

A replacement should target what was captured:

re.sub(r"^([^,]+),\s*([^,]+)$", r"\2 \1", "Lin, Johnny")

This transformation is safe only if the source convention is known. Names are not universally two comma-separated tokens.

Avoid the common traps

Trap	Symptom	Better approach
Missing anchors	Valid substring inside invalid text passes	Use full match for validation
Greedy wildcard	One group consumes too much	Use explicit delimiters or non-greedy match
Catastrophic backtracking	Runtime spikes on crafted input	Simplify nested repetition and bound length
One giant pattern	Impossible to review or explain	Parse in stages with named groups
Engine assumptions	Pattern works in one tool only	Record flavor and flags

Unicode also matters. \w, case folding, and character classes behave differently across engines and flags. Test accents, non-Latin scripts, and composed versus decomposed characters when the dataset contains them.

Build a test matrix before mass editing

Case	Input	Expected
Normal	john@example.com	accept
Boundary	one-character local part	accept if policy allows
Malformed	missing domain	reject
Near miss	trailing comment or space	normalize or reject explicitly
Adversarial	very long repeated text	finish quickly

Store the examples beside the cleaning rule. A regex without positive and negative fixtures is an undocumented guess.

Know when to stop using regex

Use a parser for nested formats, CSV quoting, JSON, URLs, dates, and programming-language syntax. Use reference data or business rules for identity and geography. Regex remains valuable at the boundary: extracting a candidate or rejecting an impossible shape before stronger validation.

Review card

• Anchor validation patterns to the whole value.
• Separate syntactic matching from domain and semantic validation.
• Extract components first; canonicalize only after validation.
• Record regex flavor, flags, and Unicode assumptions.
• Test valid, invalid, boundary, near-miss, and adversarial inputs.
• Prefer a real parser when the format has nested grammar.

^(\d{1,3}\.){3}\d{1,3}$

import re

raw = "2026/9/7"
m = re.fullmatch(r"(\d{4})/(\d{1,2})/(\d{1,2})", raw)
if m:
    year, month, day = map(int, m.groups())

re.sub(r"^([^,]+),\s*([^,]+)$", r"\2 \1", "Lin, Johnny")