Introduction to Regular Expressions

  • 4,372 views
Uploaded on

^Regular Expressions is one of those tools that every developer should have in their toolbox. You can do your job without regular expressions, but knowing when and how to use them will make you a much …

^Regular Expressions is one of those tools that every developer should have in their toolbox. You can do your job without regular expressions, but knowing when and how to use them will make you a much more efficient and marketable developer. You'll learn how regular expressions can be used for validating user input, parsing text, and refactoring code. We'll also cover various tools that can be used to help you write and share expressions.$

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,372
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
213
Comments
0
Likes
6

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • I just wanted to get this famous quote out of the way from the beginning. Like all great quotes, it has been falsely attributed all over the place. See http://regex.info/blog/2006-09-15/247 for a comprehensive investigation on where the quote originates.
  • If you’re never seen a regular expression before, it might look like Q*bert’s language to you.
  • These are all examples of regular expressions. They were all created with the intent of matching email addresses. As you can see, they’re very different from one another.
  • Regular expressions are often called a “write only language” and aren’t easily understood. To a novice they don’t even look like they have any meaning.
  • Kleene is pronounced “clay-knee”
  • Kenneth Lane Thompson, or just ‘ken’ in hacker circles, is often credited for creating the Unix operating system along with Dennis Ritchie
  • Unix command line tool which borrowed Regular Expression pattern matching from the “ed” editor.
  • Theregex library was eventually used to develop the PCRE library – Perl Compatible Reguarl Expressions. Most major programming languages these days use regular expressions based on the PCRE, including Java, .NET and Ruby.
  • Without knowing regular expressions you could certainly accomplish this task, but it would take much longer, be more complex, and much more likely to contain bugs.
  • Here’s some c# code that I threw together in a few minutes to parse a text file. There are bugs in the code and requirements that it doesn’t meet. Also, if requirements were to be added, this code would be more difficult to refactor than a regex.
  • This solution is in Perl. I could have re-written it in c# for good comparison with my previous slide, but I decided that the original example solution looks much nicer. By the way, this example is taken from Mastering Regular Expressions by Jeffrey Friedl
  • Knowing regular expressions won’t make you a hero, but you may feel like one when you’ve saved lots of time.
  • The basics of regular expressions start with literal characters. Any character, except for a small list of reserved characters which will be covered next, is considered a literal character. This example has a regex “is” which is two literal characters.
  • The regex “a” matches any letter “a” in the target string, even if its inside a word.
  • Some regex engines have the option to turn off case sensitivity. Some engines may have this option on by default.
  • These are all reserved characters, also known as meta-characters.
  • Special characters can be escaped with a backslash, as demonstrated in the example.
  • The curly brace characters are reserved, but only when in the context of a repetition modifier, and don’t need to be escaped otherwise. You can escape them without any side effects.
  • There are other non-printable characters too, especially when you use unicode. You can also reference ASCII character codes, but I don’t see the need to show an example of this.
  • The period, or dot, character matches any single character. There is an exception to this – if the engine is in single line mode (which used to be the only mode, but now is usually off by default) then the period will not match a new line. Javascript and VBScript don’t have a multi-line option, but [sS] works.The period is the most common meta-character, but that is because it is often mis-used.
  • Also known as Character Sets match only one of the characters inside the square brackets.
  • The hyphen, or dash, character inside a character class indicates a range of characters. The example would match any hexadecimal value. Note that the hyphen can be escaped inside a character class, but doesn’t need to be escaped if its at the beginning or end because its not a range in that context.
  • A caret just inside a character negates the match. Note that the example won’t match the string “Iraq” because there is no character following the q. Also, “Qatar” isn’t matched because the Q is capitalized.
  • The caret and hyphen characters are only reserved when they’re used in a context that could suggest a negation (for carets) or a range (for hyphens).
  • These can be used inside and outside of character classes. All of these aren’t necessary, but are shortcuts that make it easier to write readable regular expressions – HA!Note that in the example the match includes the space before the numbers, but I couldn’t easily represent that with coloring.
  • These are the negated versions of the shorthand character classes from the previous slide.
  • Like the period, the asterisk is overused and dangerous.
  • Like the period, the asterisk is overused and dangerous.
  • The question mark can also modify a repetition symbol to make it non-greedy or lazy, but that’s a more advanced subject. Its all about context.
  • Anchors don’t match a character, they match a position which could be before, after or between characters. In this example, the string only matches because there is no whitespace before the word “vacation”
  • Note that anchors can result in a zero length match. For example, a regex of just “$” matches the position at the end of a line, but the match has no characters. This can be useful or cause issues in your code!
  • Note that another exception is a new line at the end of a file will not be matched by $ or . The z shorthand character class will handle this circumstance.
  • Note that the words “at”, “that” and “ate” don’t contain a match in the example.
  • Regular expressions are eager, which means they match as soon as they can. This means that the order of similar character classes in an alternation can affect the result. In the example, the “and” match is found first, even though the full word “android” exists. If the character classes were switched, the whole “and” word would still match, but the whole “android” word would match instead of just the beginning of the word.
  • Greediness causes the example to always match the full “android” instead of just “and” which is also a valid match.
  • The ? mark after a repetition operator (*+?) makes it lazy.
  • Using parenthesis, or round braces, to group a character set not only groups those characters together to apply repetition to them, but it also creates a back reference. A back reference stores the grouped part of the match for use later in the regex. Back references can also be used for replaces.In the example, “agici” doesn’t match because the group is “a” not “i”.
  • The groups can be named with single parenthesis instead of greater/less than characters.
  • This example matches a “q” not followed by a “u”. Unlike the example earlier in the presentation, this regex won’t match the character following the “q”.
  • This example matches a “q” followed by a “u”, but notice that the “u” is not included with that match.
  • This example matches a “q” followed by a “u”, but notice that the “u” is not included with that match.

Transcript

  • 1. Introduction toRegular Expressions
    Matt Casto
    http://google.com/profiles/mattcasto
  • 2. Introduction toRegular Expressions
    Matt Casto
    Quick Solutions
    http://google.com/profiles/mattcasto
  • 3. “Some people, when confronted with a problem, think “I know, I'll use regular expressions. Now they have two problems.”
    - Jamie Zawinski, August 12, 1997
  • 4.
  • 5. What are Regular Expressions?
    ^w+@[a-zA-Z_]+?.[a-zA-Z]{2,3}$
    [w-]+@([w-]+.)+[w-]+
    ^.+@[^.].*.[a-z]{2,}$
    ^([a-zA-Z0-9_-.]+)@(([[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.)|(([a-zA-Z0-9-]+.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(]?)$
  • 6.
  • 7. History
    Stephen Cole Kleene
    American mathematician credited for inventing Regular Expressions in the 1950’s using a mathematic notation called regular sets.
  • 8. History
    Ken Thompson
    American pioneer of computer science who, among many other things, used Kleene’s regular sets for searching in his QED and ed text editors.
  • 9. History
    grep
    Global Regular Expression Print
  • 10. History
    Henry Spencer
    Wrote the regex library which is what Perl and Tcl languages used for regular expressions.
  • 11. Why Should You Care?
    Example: finding duplicate words in a file.
    Requirements:
    • Output lines that contain duplicate words
    • 12. Find doubled words that expand lines
    • 13. Ignore capitalization differences
    • 14. Ignore HTML tags
  • 15. Why Should You Care?
    Example: finding duplicate words in a file.
    Solution:
    $/ = “. ”;
    while (<>) {
    next if !s/([a-z]+)((?:s<[^>]+>)+)(1)/e[7m$1e[m$2e[7m$3e[m/ig;
    s/^(?:[^e]* )+//mg;
    s/^/$ARGV: /mg;
    print;
    }
  • 16.
  • 17. Literal Characters
    Any character except a small list of reserved characters.
    regex
    is
    Jack is a boy
    match in target string
  • 18. Literal Characters
    Literals will match characters in the middle of words.
    regex
    a
    Jack is a boy
    matches in target string
  • 19. Literal Characters
    Literals are case sensitive – capitalization matters!
    regex
    j
    Jack is a boy
    NOT a match
  • 20. Special Characters
    [ ^ $ . | ? * + ( )
  • 21. Special Characters
    You can match special characters by escaping them with a backslash.
    1+1=2
    I wrote 1+1=2 on the chalkboard.
  • 22. Special Characters
    Some characters, such as { and } are only reserved depending on context.
    if (true) {
    else if (true) { beep; }
  • 23. Non-Printable Characters
    Some literal characters can be escaped to represent non-printable characters.
    – tab
    – carriage return
    – line feed
    a – bell
    e – escape
    f – form feed
    v – vertical tab
  • 24. Period
    The period character matches any single character.
    a.boy
    Jack is a boy
  • 25. Character Classes
    Used to match only one of the characters inside square braces.
    [Gg]r[ae]y
    Grayson drives a grey sedan.
  • 26. Character Classes
    Hyphen is a reserved character inside a character class and indicates a range.
    [0-9a-fA-F]
    The HTML codefor White is #FFFFFF
  • 27. Character Classes
    Caret inside a character class negates the match.
    q[^u]
    Qatar is home to quite a lot of Iraqi citizens, but is not a city in Iraq
  • 28. Character Classes
    Normal special characters are valid inside of character classes. Only ] ^ and – are reserved.
    [+*]
    6 * 7 and 18 + 24 both equal 42
  • 29. Shorthand Character Classes
    d – digit or [0-9]
    w – word or [A-Za-z0-9_]
    s – whitespace or [ ] (space, tab, CR, LF)
    [sd]
    1 + 2 = 3
  • 30. Shorthand Character Classes
    D – non-digit or [^d]
    W – non-word or [^w]
    S – non-whitespace or [^s]
    [D]
    1 + 2 = 3
  • 31. Repetition
    The asterisk repeats the preceding character class 0 or more times.
    <[A-Za-z][A-Za-z0-9]*>
    <HTML>Regex is <b>Awesome</b></HTML>
  • 32. Repetition
    The plus repeats the preceding character class 1 or more times.
    <[A-Za-z0-9]+>
    Watch out for invalid <HTML> tags like <1> and <>!
  • 33. Repetition
    The question mark repeats the preceding character class 0 or 1 times, in effect making it optional.
    </?[A-Za-z][A-Za-z0-9]*>
    <HTML>Regex is <b>Awesome</b></HTML>
  • 34. Anchors
    The caret anchor matches the position before the first character in a string.
    ^vac
    vacation evacuation
  • 35. Anchors
    The dollar sign anchor matches the position after the last character in a string.
    tion$
    vacation evacuation
  • 36. Anchors
    The caret and dollar sign anchors match the start and end of the line if the engine has multi-line turned on.
    tion$
    vacation evacuation
    has ruined my evaluation
  • 37. Anchors
    The A and  shorthand character classes are like
    ^ and $ but only match the start and end of the string.
    tion
    vacation evacuation
    has ruined my evaluation
  • 38. Word Boundaries
    The  shorthand character class matches…
    • position before the first character in a string (like ^)
    • 39. position after the last character in a string (like $)
    • 40. between two characters where one is a word character and the other is not
    4
    We’ve got 4 orders for 44 lbs of C4
  • 41. Word Boundaries
    The B shorthand character class is the negated word boundary – any position between to word characters or two non-word characters.
    BatB
    vacation evacuation at that
    time ate my evaluation
  • 42. Alternation
    The pipe symbol delimits two or more character classes that can both match.
    cat|dog
    A cat and dog are expected to follow
    the dogma that their presence with one
    another leads to catastrophe.
  • 43. Alternation
    Alternations include any character classes.
    cat|dog
    A cat and dog are expected to follow
    the dogma that their presence with one
    another leads to catastrophe.
  • 44. Alternation
    Use parenthesis to group alternating matches when you want to limit the reach of alternation.
    (cat|dog)
    A cat and dog are expected to follow
    the dogma that their presence with one
    another leads to catastrophe.
  • 45. Eagerness
    Eagerness causes the order of alternations to matter.
    and|android
    A robot and an android fight. The ninja wins.
  • 46. Greediness
    Greediness means that the engine will always try to match as much as possible.
    anS+
    A robot and an android fight. The ninja wins.
  • 47. Laziness
    Laziness, or reluctant, modifies a repetition operator to only match as much as it needs to.
    anS+?
    A robot and an android fight. The ninja wins.
  • 48. Limiting Repetition
    You can limit repetition with curly braces.
    d{2,4}
    1 111111111 11111
  • 49. Limiting Repetition
    The second number can be omitted to mean infinite.
    Essentially {0,} is the same as * and {1,} same as +.
    d{2,}
    1 11111111111111
  • 50. Limiting Repetition
    The a single number can be used to match an exact number of times.
    d{4}
    1 11 111 1111 11111
  • 51. Back References
    Parenthesis around a character set groups those characters and creates a back reference.
    ([ai]).1.1
    The magician said abracadabra!
  • 52. Named Groups
    Named groups let you reference matched groups by their name rather than just index.
    (?<vowel>[ai]).k<vowel>.1
    The magician said abracadabra!
  • 53. Negative Lookahead
    Negative lookaheads match something that is not there.
    q(?!u)
    Qatar is home to quite a lot of Iraqi citizens, but is not a city in Iraq
  • 54. Positive Lookahead
    Positive lookaheads match something that is there without having that group included in the match.
    q(?=u)
    Qatar is home to quite a lot of Iraqi citizens, but is not a city in Iraq
  • 55. Positive & Negative Lookbehind
    Lookbehinds are just like lookaheads, but working backwards.
    (?<=a)q
    Qatar is home to quite a lot of Iraqi citizens, but is not a city in Iraq
  • 56. Resources
    Lots of web pages
    http://del.icio.us/mattcasto/regex
    “Mastering Regular Expressions”
    by Jeffrey Friedl
    http://oreilly.com/catalog/9780596528126/