Michael Limberger

Need me? Email mike@limberger.ca

The floor everything stands on

A fast refresher. If you were at Regex Therapy, this is the same alphabet at walking speed. If this is your first night with pattern matching, this gets you up to speed.

Atoms walk left to right

A regex pattern is made of atoms: the smallest unit the engine can match. A literal character is an atom. A character class is an atom. A group is an atom. The engine breaks your pattern into those pieces, then walks them left to right.

Here is what happens when the engine matches the pattern cat against the text the cat sat.

Text:    t h e   c a t   s a t
Position: 0 1 2 3 4 5 6 7 8 9

Position 0: Try 'c' against 't' · no match, slide forward
Position 1: Try 'c' against 'h' · no match, slide forward
Position 2: Try 'c' against 'e' · no match, slide forward
Position 3: Try 'c' against ' ' · no match, slide forward
Position 4: Try 'c' against 'c' · match! Try 'a' against position 5
Position 5: Try 'a' against 'a' · match! Try 't' against position 6
Position 6: Try 't' against 't' · match! All atoms matched!

Result: "cat" found at position 4

The engine is persistent. It tries every starting position until it finds a match or runs out of text. Every advanced feature tonight sits on this left-to-right, atom-by-atom walk.

Square brackets, one character

A character class is a set of allowed characters inside square brackets. The engine matches exactly one character from the set.

[aeiou]       One vowel
[abc]         The letter a, b, or c

Ranges use a dash inside the brackets:

[a-z]         Any lowercase letter
[A-Z]         Any uppercase letter
[0-9]         Any digit
[a-zA-Z]      Any letter, upper or lower
[a-zA-Z0-9]   Any letter or digit

Mix ranges and individual characters:

[a-z0-9_]     Lowercase letter, digit, or underscore

Negate with a caret at the start of the class. The caret is the ^ glyph:

[^0-9]        Anything that is NOT a digit
[^aeiou]      Anything that is NOT a vowel

Shorthand classes save typing. A backslash \ starts each one:

\d    Digit           same as [0-9]
\D    Not a digit     same as [^0-9]
\w    Word character  same as [a-zA-Z0-9_]
\W    Not word char   same as [^a-zA-Z0-9_]
\s    Whitespace      space, tab, newline
\S    Not whitespace

Show me

echo "Hello World 123" | grep -Po '[A-Z][a-z]+'

That matches a capital letter followed by one or more lowercase letters. Result: Hello and World.

The grammar glyphs

These characters have special meaning. They are the grammar. When the engine sees them, it does not match them literally. It interprets them.

.     Any single character (except newline by default)
^     Start of line or string
$     End of line or string
*     Zero or more of the previous atom
+     One or more of the previous atom
?     Zero or one of the previous atom (makes it optional)
|     Alternation (OR)
( )   Grouping and capturing
[ ]   Character class
{ }   Quantifier bounds like {3} or {2,5}
\     Escape character

When you need a literal meta character, escape it with a backslash:

\.    A literal dot
\$    A literal dollar sign
\^    A literal caret
\(    A literal opening parenthesis
\\    A literal backslash

A filename with a real dot:

photo\.jpg    matches "photo.jpg" (not "photoxjpg")
price: \$\d+  matches "price: $50"

The pipe means or

The pipe character | means or. The engine tries the left side first. If that does not match, it tries the right side.

cat|dog                 matches "cat" or "dog"
red|green|blue          matches any of the three
gray|grey               matches both spellings

Scope matters. The pipe applies to everything on each side unless parentheses contain it.

cat|dog food            matches "cat" OR "dog food" (the whole string)
(cat|dog) food          matches "cat food" OR "dog food"

A cleaner way to handle spelling variations:

gr(a|e)y                matches "gray" or "grey"
colo(u|)r               matches "color" or "colour"

You can build a whole day-of-week matcher:

(Mon|Tues|Wednes|Thurs|Fri|Satur|Sun)day

+-------------------------------------------------------+
|  If all of this is comfortable, you're ready for      |
|  tonight. If any of it feels fuzzy, the Regex         |
|  Therapy tutorial on this site covers      |
|  every piece in detail. Grab it for five bucks.       |
+-------------------------------------------------------+