python working with regular expressions


A regular expression is a sequence of symbols and characters expressing a string or pattern to be searched for within a longer piece of text.  By using RegEx (i.e regular expression) we process text. We perform operations like

  • finding a given pattern in a string/text
  • Replacing given pattern with another string.
  • Finding all strings that matches a given RegEx/pattern , etc.

Python Regular Expressions

  • To work with regular expressions in python we use "re" module in python. It is a in-built module that comes with python.
  • Every Regular Expression built to produce a pattern which is defined with  finite set of chararcters(i.e building blocks of regex). Every character has a special meaning to it.
  • Let's checkout the the regular expression's building blocks and their meanings below
  • Regex Special Characters:
    • ^  Matches the expression to its right at the start of a string. It matches every such instance before each \n in the string.
    • $  Matches the expression to its left at the end of a string. It matches every such instance before each \n in the string.
    • . Matches any character except line terminators like \n.

    • \ Escapes special characters or denotes character classes.

    • A|B  Matches expression A or B. If A is matched first, B is left untried.

    • +  Greedily matches the expression to its left 1 or more times.

    • *  Greedily matches the expression to its left 0 or more times.

    • ?  Greedily matches the expression to its left 0 or 1 times. But if ? is added to qualifiers (+*, and ? itself) it will perform matches in a non-greedy manner.

    • {m}  Matches the expression to its left m times, and not less.

    • {m,n} Matches the expression to its left m to n times, and not less.

    • {m,n}? Matches the expression to its left m times, and ignores n.

  • Regex Special Sequences:

    • \w  Matches alphanumeric characters, which means a-zA-Z, and 0-9. It also matches the underscore, _.

    • \d  Matches digits, which means 0-9.

    • \D  Matches any non-digits.

    • \s  Matches whitespace characters, which include the \t\n\r, and space characters.

    • \S  Matches non-whitespace characters.

    • \b  Matches the boundary (or empty string) at the start and end of a word, that is, between \w and \W.

    • \B  Matches where \b does not, that is, the boundary of \w characters.

    • \A  Matches the expression to its right at the absolute start of a string whether in single or multi-line mode.

    • \Z  Matches the expression to its left at the absolute end of a string whether in single or multi-line mode.

  • Regex Character Sets:

    • [ ] Contains a set of characters to match.

    • [amk]  Matches either am, or k. It does not match amk.

    • [a-z]  Matches any alphabet from a to z.

    • [a\-z] Matches a-, or z. It matches - because \ escapes it.

    • [a-] Matches a or -, because - is not being used to indicate a series of characters.

    • [-a] As above, matches a or -.

    • [a-z0-9] Matches characters from a to z and also from 0 to 9.

    • [(+*)] Special characters become literal inside a set, so this matches (+*, and ).

    • [^ab5] Adding ^ excludes any character in the set. Here, it matches characters that are not ab, or 5.

  • Regex Groups:

    • ( ) | Matches the expression inside the parentheses and groups it.

    • (? ) | Inside parentheses like this, ? acts as an extension notation. Its meaning depends on the character immediately to its right.

    • (?PAB) | Matches the expression AB, and it can be accessed with the group name.

    • (?aiLmsux) | Here, aiLmsu, and x are flags:

      • a — Matches ASCII only
      • i — Ignore case
      • L — Locale dependent
      • m — Multi-line
      • s — Matches all
      • u — Matches unicode
      • x — Verbose
    • A(?!B)  Negative lookahead assertion. This matches the expression A only if it is not followed by B.

    • (?<=B)A Positive lookbehind assertion. This matches the expression A only if B is immediately to its left. This can only matched fixed length expressions.

    • (?<!B)A  Negative lookbehind assertion. This matches the expression A only if B is not immediately to its left. This can only matched fixed length expressions.

    • (?P=name) Matches the expression matched by an earlier group named “name”.

    • (...)\1  The number 1 corresponds to the first group to be matched. If we want to match more instances of the same expresion, simply use its number instead of writing out the whole expression again. We can use from 1 up to 99 such groups and their corresponding numbers.

    • (?:A) Matches the expression as represented by A, but unlike (?PAB), it cannot be retrieved afterwards.

    • (?#...) A comment. Contents are for us to read, not for matching.

    • A(?=B) Lookahead assertion. This matches the expression A only if it is followed by B.

Python "re" module:

Let's find a sub string "the" in the the text "Experience is the name everyone gives to their mistakes."

import re

pattern = re.compile("the")
text = "Experience is the name everyone gives to their mistakes."
out = pattern.findall(text)
print(out)
# Output: ['the', 'the']
  • In the above code we have compiled our pattern "the" using python's re module function "compile".
  • After we have find all the strings that matches with the pattern using "findall" method (i.e bounded to the compiled pattern)