Regular Expressions
Performance
Optimizing event capture building
better Ossim Agent plugins
About A3Sec
● AlienVault's spin-off
● Professional Services, SIEM deployments
● Alienvault's Authorized Training Center (A...
About Me
● David Gil <dgil@a3sec.com>
● Developer, Sysadmin, Project Manager
● Really believes in Open Source model
● Prog...
Summary
1. What is a regexp?
2. When to use regexp?
3. Regex basics
4. Performance Tests
5. Writing regexp (Performance St...
Regular Expressions
What is a regex?
Regular expression:

(bb|[^b]{2})
Regular Expressions
What is a regex?
Regular expression:

(bb|[^b]{2})dd
Input strings:

bb445, 2ac3357bb, bb3aa2c7,
a2ab6...
Regular Expressions
What is a regex?
Regular expression:

(bb|[^b]{2})dd
Input strings:

bb445, 2ac3357bb, bb3aa2c7,
a2ab6...
Summary
1. What is a regexp?
2. When to use regexp?
3. Regex basics
4. Performance Tests
5. Writing regexp (Performance St...
Regular Expressions
To RE or not to RE
● Regular expressions are almost never the
right answer
○ Difficult to debug and ma...
Regular Expressions
To RE or not to RE
● Regular expressions are almost never the
right answer
○ Difficult to debug and ma...
Regular Expressions
To RE or not to RE
● Regular expressions are almost never the
right answer
○ Difficult to debug and ma...
Regular Expressions
To RE or not to RE
Example: URL parsing
● regex:
^(https?://)?([da-z.-]+).([a-z.]{2,6})([/w .-]*)*/?$
...
Regular Expressions
To RE or not to RE
But, there are a lot of reasons to use regex:
● powerful
● portable
● fast (with pe...
Summary
1. What is a regexp?
2. When to use regexp?
3. Regex basics
4. Performance Tests
5. Writing regexp (Performance St...
Regular Expressions
Basics - Characters
● d, D: digits. w, W: words. s, S: spaces
>>> re.findall('dddd-(dd)-dd', '2013-07-...
Regular Expressions
Basics - Repetitions
● *, +: 0-1 or more repetitions
>>> re.findall('FO+', 'FOOOOOOOOO')
>>> re.findal...
Regular Expressions
Basics - Groups
[...]: Set of characters
>>> re.findall('[a-z]+=[a-z]+', 'foo=bar')

...|...: Alternat...
Regular Expressions
Greedy & Lazy quantifiers: *?, +?
● Greedy vs non-greedy (lazy)
>>> re.findall('A+', 'AAAA')
['AAAA']
...
Regular Expressions
Greedy & Lazy quantifiers: *?, +?
● Greedy vs non-greedy (lazy)
>>> re.findall('A+', 'AAAA')
['AAAA']
...
Regular Expressions
Greedy & Lazy quantifiers: *?, +?
● Greedy vs non-greedy (lazy)
>>> re.findall('A+', 'AAAA')
['AAAA']
...
Summary
1. What is a regexp?
2. When to use regexp?
3. Regex basics
4. Performance Tests
5. Writing regexp (Performance St...
Regular Expressions
Performance Tests
Different implementations of a custom
is_a_word() function:
● #1 Regexp
● #2 Char it...
Regular Expressions
Performance Test #1
def is_a_word(word):
CHARS = string.uppercase + string.lowercase
regexp = r'^[%s]+...
Regular Expressions
Performance Test #1
def is_a_word(word):
CHARS = string.uppercase + string.lowercase
regexp = r'^[%s]+...
Regular Expressions
Performance Test #1
def is_a_word(word):
CHARS = string.uppercase + string.lowercase
regexp = r'^[%s]+...
Regular Expressions
Performance Test #2
def is_a_word(word):
for char in word:
if not char in (CHARS): return "NOP"
return...
Regular Expressions
Performance Test #2
def is_a_word(word):
for char in word:
if not char in (CHARS): return "NOP"
return...
Regular Expressions
Performance Test #2
def is_a_word(word):
for char in word:
if not char in (CHARS): return "NOP"
return...
Regular Expressions
Performance Test #3
def is_a_word(word):
return "YES" if word.isalpha() else "NOP"
Regular Expressions
Performance Test #3
def is_a_word(word):
return "YES" if word.isalpha() else "NOP"

timeit.timeit(s, '...
Regular Expressions
Performance Test #3
def is_a_word(word):
return "YES" if word.isalpha() else "NOP"

timeit.timeit(s, '...
Summary
1. What is a regexp?
2. When to use regexp?
3. Regex basics
4. Performance Tests
5. Writing regexp (Performance St...
Regular Expressions
Performance Strategies
Writing regex
● Be careful with repetitions (+, *, {n,m})
(abc|def){2,4} produc...
Regular Expressions
Performance Strategies
Writing regex
● Be careful with repetitions (+, *, {n,m})
(abc|def){2,4} produc...
Regular Expressions
Performance Strategies
Writing regex
● Be careful with repetitions (+, *, {n,m})
(abc|def){2,4} produc...
Regular Expressions
Performance Strategies
Writing regex
● Be careful with repetitions (+, *, {n,m})
(abc|def){2,4} produc...
Regular Expressions
Performance Strategies
Writing regex
● Be careful with repetitions (+, *, {n,m})
(abc|def){2,4} produc...
Regular Expressions
Performance Strategies
Writing regex
● Use the non-capturing group when no need
to capture and save te...
Regular Expressions
Performance Strategies
Writing regex
● Use the non-capturing group when no need
to capture and save te...
Regular Expressions
Performance Strategies
Writing regex
● Use the non-capturing group when no need
to capture and save te...
Regular Expressions
Performance Strategies
Writing regex
● Use the non-capturing group when no need
to capture and save te...
Summary
1. What is a regexp?
2. When to use regexp?
3. Regex basics
4. Performance Tests
5. Writing regexp (Performance St...
Regular Expressions
Performance Strategies
Writing Agent plugins
● A new process is forked for each loaded
plugin
○ Use th...
Regular Expressions
Performance Strategies
Writing Agent plugins
● A new process is forked for each loaded
plugin
○ Use th...
Regular Expressions
Performance Strategies
Writing Agent plugins
● Alphabetical order for rule matching
○ Order your rules...
Regular Expressions
Performance Strategies
Writing Agent plugins
● Alphabetical order for rule matching
○ Order your rules...
Regular Expressions
Performance Strategies
Tool1
Tool2
Tool3
Tool4
Tool5

20 logs/sec
20 logs/sec
20 logs/sec
20 logs/sec
...
Regular Expressions
Performance Strategies
Tool1
Tool2
Tool3
Tool4
Tool5

20 logs/sec
20 logs/sec
20 logs/sec
20 logs/sec
...
Summary
1. What is a regexp?
2. When to use regexp?
3. Regex basics
4. Performance Tests
5. Writing regexp (Performance St...
Regular Expressions
Tools for testing Regex
Python:
>>> import re
>>> re.findall('(S+) (S+)', 'foo bar')
[('foo', 'bar')]
...
Regular Expressions
Tools for testing Regex
Regex debuggers:
● Kiki
● Kodos
Online regex testers:
● http://gskinner.com/Re...
any (?:question|doubt|comment)+?
A3Sec
web: www.a3sec.com
email: training@a3sec.com
twitter: @a3sec
Spain Head Office
C/ Aravaca, 6, Piso 2
28040 Madrid
Tl...
A3 sec -_regular_expressions
Upcoming SlideShare
Loading in …5
×

A3 sec -_regular_expressions

342 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
342
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

A3 sec -_regular_expressions

  1. 1. Regular Expressions Performance Optimizing event capture building better Ossim Agent plugins
  2. 2. About A3Sec ● AlienVault's spin-off ● Professional Services, SIEM deployments ● Alienvault's Authorized Training Center (ATC) for Spain and LATAM ● Team of more than 25 Security Experts ● Own developments and tool integrations ● Advanced Health Check Monitoring ● Web: www.a3sec.com, Twitter: @a3sec
  3. 3. About Me ● David Gil <dgil@a3sec.com> ● Developer, Sysadmin, Project Manager ● Really believes in Open Source model ● Programming since he was 9 years old ● Ossim developer at its early stage ● Agent core engine (full regex) and first plugins ● Python lover :-) ● Debian package maintainer (a long, long time ago) ● Sci-Fi books reader and mountain bike rider
  4. 4. Summary 1. What is a regexp? 2. When to use regexp? 3. Regex basics 4. Performance Tests 5. Writing regexp (Performance Strategies) 6. Writing plugins (Performance Strategies) 7. Tools
  5. 5. Regular Expressions What is a regex? Regular expression: (bb|[^b]{2})
  6. 6. Regular Expressions What is a regex? Regular expression: (bb|[^b]{2})dd Input strings: bb445, 2ac3357bb, bb3aa2c7, a2ab64b, abb83fh6l3hi22ui
  7. 7. Regular Expressions What is a regex? Regular expression: (bb|[^b]{2})dd Input strings: bb445, 2ac3357bb, bb3aa2c7, a2ab64b, abb83fh6l3hi22ui
  8. 8. Summary 1. What is a regexp? 2. When to use regexp? 3. Regex basics 4. Performance Tests 5. Writing regexp (Performance Strategies) 6. Writing plugins (Performance Strategies) 7. Tools
  9. 9. Regular Expressions To RE or not to RE ● Regular expressions are almost never the right answer ○ Difficult to debug and maintain ○ Performance reasons, slower for simple matching ○ Learning curve
  10. 10. Regular Expressions To RE or not to RE ● Regular expressions are almost never the right answer ○ Difficult to debug and maintain ○ Performance reasons, slower for simple matching ○ Learning curve ● Python string functions are small C loops: super fast! ○ beginswith(), endswith(), split(), etc.
  11. 11. Regular Expressions To RE or not to RE ● Regular expressions are almost never the right answer ○ Difficult to debug and maintain ○ Performance reasons, slower for simple matching ○ Learning curve ● Python string functions are small C loops: super fast! ○ beginswith(), endswith(), split(), etc. ● Use standard parsing libraries! Formats: JSON, HTML, XML, CSV, etc.
  12. 12. Regular Expressions To RE or not to RE Example: URL parsing ● regex: ^(https?://)?([da-z.-]+).([a-z.]{2,6})([/w .-]*)*/?$ ● parse_url() php method: $url = "http://username:password@hostname/path?arg=value#anchor"; print_r(parse_url($url)); ( [scheme] => http [host] => hostname [user] => username [pass] => password [path] => /path [query] => arg=value [fragment] => anchor )
  13. 13. Regular Expressions To RE or not to RE But, there are a lot of reasons to use regex: ● powerful ● portable ● fast (with performance in mind) ● useful for complex patterns ● save development time ● short code ● fun :-) ● beautiful?
  14. 14. Summary 1. What is a regexp? 2. When to use regexp? 3. Regex basics 4. Performance Tests 5. Writing regexp (Performance Strategies) 6. Writing plugins (Performance Strategies) 7. Tools
  15. 15. Regular Expressions Basics - Characters ● d, D: digits. w, W: words. s, S: spaces >>> re.findall('dddd-(dd)-dd', '2013-07-21') >>> re.findall('(S+)s+(S+)', 'foo bar') ● ^, $: Begin/End of string >>> re.findall('(d+)', 'cba3456csw') >>> re.findall('^(d+)$', 'cba3456csw') ● . (dot): Any character: >>> re.findall('foo(.)bar', 'foo=bar') >>> re.findall('(...)=(...)', 'foo=bar')
  16. 16. Regular Expressions Basics - Repetitions ● *, +: 0-1 or more repetitions >>> re.findall('FO+', 'FOOOOOOOOO') >>> re.findall('BA*R', 'BR') ● ?: 0 or 1 repetitions >>> re.findall('colou?r', 'color') >>> re.findall('colou?r', 'colour') ● {n}, {n,m}: N repetitions: >>> re.findall('d{2}', '2013-07-21') >>> re.findall('d{1,3}.d{1,3}.d{1,3}.d{1,3}','192.168.1.25')
  17. 17. Regular Expressions Basics - Groups [...]: Set of characters >>> re.findall('[a-z]+=[a-z]+', 'foo=bar') ...|...: Alternation >>> re.findall('(foo|bar)=(foo|bar)', 'foo=bar') (...) and 1, 2, ...: Group >>> re.findall(r'(w+)=(1)', 'foo=bar') >>> re.findall(r'(w+)=(1)', 'foo=foo') (?P<name>...): Named group >>> re.findall('d{4}-d{2}-(?P<day>d{2}'), '2013-07-23')
  18. 18. Regular Expressions Greedy & Lazy quantifiers: *?, +? ● Greedy vs non-greedy (lazy) >>> re.findall('A+', 'AAAA') ['AAAA'] >>> re.findall('A+?', 'AAAA') ['A', 'A', 'A', 'A']
  19. 19. Regular Expressions Greedy & Lazy quantifiers: *?, +? ● Greedy vs non-greedy (lazy) >>> re.findall('A+', 'AAAA') ['AAAA'] >>> re.findall('A+?', 'AAAA') ['A', 'A', 'A', 'A'] ● An overall match takes precedence over and overall non-match >>> re.findall('<.*>.*</.*>', '<B>i am bold</B>') >>> re.findall('<(.*)>.*</(.*)>', '<B>i am bold</B>')
  20. 20. Regular Expressions Greedy & Lazy quantifiers: *?, +? ● Greedy vs non-greedy (lazy) >>> re.findall('A+', 'AAAA') ['AAAA'] >>> re.findall('A+?', 'AAAA') ['A', 'A', 'A', 'A'] ● An overall match takes precedence over and overall non-match >>> re.findall('<.*>.*</.*>', '<B>i am bold</B>') >>> re.findall('<(.*)>.*</(.*)>', '<B>i am bold</B>') ● Minimal matching, non-greedy >>> re.findall('<(.*)>.*', '<B>i am bold</B>') >>> re.findall('<(.*?)>.*', '<B>i am bold</B>')
  21. 21. Summary 1. What is a regexp? 2. When to use regexp? 3. Regex basics 4. Performance Tests 5. Writing regexp (Performance Strategies) 6. Writing plugins (Performance Strategies) 7. Tools
  22. 22. Regular Expressions Performance Tests Different implementations of a custom is_a_word() function: ● #1 Regexp ● #2 Char iteration ● #3 String functions
  23. 23. Regular Expressions Performance Test #1 def is_a_word(word): CHARS = string.uppercase + string.lowercase regexp = r'^[%s]+$' % CHARS if re.search(regexp, word) return "YES" else "NOP"
  24. 24. Regular Expressions Performance Test #1 def is_a_word(word): CHARS = string.uppercase + string.lowercase regexp = r'^[%s]+$' % CHARS if re.search(regexp, word) return "YES" else "NOP" timeit.timeit(s, 'is_a_word(%s)' %(w)) 1.49650502205 YES len=4 word 1.65614509583 YES len=25 wordlongerthanpreviousone.. 1.92520785332 YES len=60 wordlongerthanpreviosoneplusan.. 2.38850092888 YES len=120 wordlongerthanpreviosoneplusan.. 1.55924701691 NOP len=10 not a word 1.7087020874 NOP len=25 not a word, just a phrase.. 1.92521882057 NOP len=50 not a word, just a phrase bigg.. 2.39075493813 NOP len=102 not a word, just a phrase bigg..
  25. 25. Regular Expressions Performance Test #1 def is_a_word(word): CHARS = string.uppercase + string.lowercase regexp = r'^[%s]+$' % CHARS if re.search(regexp, word) return "YES" else "NOP" timeit.timeit(s, 'is_a_word(%s)' %(w)) 1.49650502205 YES len=4 word 1.65614509583 YES len=25 wordlongerthanpreviousone.. 1.92520785332 YES len=60 wordlongerthanpreviosoneplusan.. 2.38850092888 YES len=120 wordlongerthanpreviosoneplusan.. 1.55924701691 NOP len=10 not a word 1.7087020874 NOP len=25 not a word, just a phrase.. 1.92521882057 NOP len=50 not a word, just a phrase bigg.. 2.39075493813 NOP len=102 not a word, just a phrase bigg.. If the target string is longer, the regex matching is slower. No matter if success or fail.
  26. 26. Regular Expressions Performance Test #2 def is_a_word(word): for char in word: if not char in (CHARS): return "NOP" return "YES"
  27. 27. Regular Expressions Performance Test #2 def is_a_word(word): for char in word: if not char in (CHARS): return "NOP" return "YES" timeit.timeit(s, 'is_a_word(%s)' %(w)) 0.687522172928 YES len=4 word 1.0725839138 YES len=25 wordlongerthanpreviousone.. 2.34717106819 YES len=60 wordlongerthanpreviosoneplusan.. 4.31543898582 YES len=120 wordlongerthanpreviosoneplusan.. 0.54797577858 NOP len=10 not a word 0.547253847122 NOP len=25 not a word, just a phrase.. 0.546499967575 NOP len=50 not a word, just a phrase bigg.. 0.553755998611 NOP len=102 not a word, just a phrase bigg..
  28. 28. Regular Expressions Performance Test #2 def is_a_word(word): for char in word: if not char in (CHARS): return "NOP" return "YES" timeit.timeit(s, 'is_a_word(%s)' %(w)) 0.687522172928 YES len=4 word 1.0725839138 YES len=25 wordlongerthanpreviousone.. 2.34717106819 YES len=60 wordlongerthanpreviosoneplusan.. 4.31543898582 YES len=120 wordlongerthanpreviosoneplusan.. 0.54797577858 NOP len=10 not a word 0.547253847122 NOP len=25 not a word, just a phrase.. 0.546499967575 NOP len=50 not a word, just a phrase bigg.. 0.553755998611 NOP len=102 not a word, just a phrase bigg.. 2 python nested loops if success (slow) But fails at the same point&time (first space)
  29. 29. Regular Expressions Performance Test #3 def is_a_word(word): return "YES" if word.isalpha() else "NOP"
  30. 30. Regular Expressions Performance Test #3 def is_a_word(word): return "YES" if word.isalpha() else "NOP" timeit.timeit(s, 'is_a_word(%s)' %(w)) 0.146447896957 YES len=4 word 0.212563037872 YES len=25 wordlongerthanpreviousone.. 0.318686008453 YES len=60 wordlongerthanpreviosoneplusan.. 0.493942975998 YES len=120 wordlongerthanpreviosoneplusan.. 0.14647102356 NOP len=10 not a word 0.146160840988 NOP len=25 not a word, just a phrase.. 0.147103071213 NOP len=50 not a word, just a phrase bigg.. 0.146239995956 NOP len=102 not a word, just a phrase bigg..
  31. 31. Regular Expressions Performance Test #3 def is_a_word(word): return "YES" if word.isalpha() else "NOP" timeit.timeit(s, 'is_a_word(%s)' %(w)) 0.146447896957 YES len=4 word 0.212563037872 YES len=25 wordlongerthanpreviousone.. 0.318686008453 YES len=60 wordlongerthanpreviosoneplusan.. 0.493942975998 YES len=120 wordlongerthanpreviosoneplusan.. 0.14647102356 NOP len=10 not a word 0.146160840988 NOP len=25 not a word, just a phrase.. 0.147103071213 NOP len=50 not a word, just a phrase bigg.. 0.146239995956 NOP len=102 not a word, just a phrase bigg.. Python string functions (fast and small C loops)
  32. 32. Summary 1. What is a regexp? 2. When to use regexp? 3. Regex basics 4. Performance Tests 5. Writing regexp (Performance Strategies) 6. Writing plugins (Performance Strategies) 7. Tools
  33. 33. Regular Expressions Performance Strategies Writing regex ● Be careful with repetitions (+, *, {n,m}) (abc|def){2,4} produces (abc|def)(abc|def)((abc|def)(abc|def)?)?
  34. 34. Regular Expressions Performance Strategies Writing regex ● Be careful with repetitions (+, *, {n,m}) (abc|def){2,4} produces (abc|def)(abc|def)((abc|def)(abc|def)?)? (abc|def){2,1000} produces ...
  35. 35. Regular Expressions Performance Strategies Writing regex ● Be careful with repetitions (+, *, {n,m}) (abc|def){2,4} produces (abc|def)(abc|def)((abc|def)(abc|def)?)? (abc|def){2,1000} produces ... ● Be careful with wildcards re.findall(r'(ab).*(cd).*(ef)', 'ab cd ef')
  36. 36. Regular Expressions Performance Strategies Writing regex ● Be careful with repetitions (+, *, {n,m}) (abc|def){2,4} produces (abc|def)(abc|def)((abc|def)(abc|def)?)? (abc|def){2,1000} produces ... ● Be careful with wildcards re.findall(r'(ab).*(cd).*(ef)', 'ab cd ef') # slower re.findall(r'(ab)s(cd)s(ef)', 'ab cd ef') # faster
  37. 37. Regular Expressions Performance Strategies Writing regex ● Be careful with repetitions (+, *, {n,m}) (abc|def){2,4} produces (abc|def)(abc|def)((abc|def)(abc|def)?)? (abc|def){2,1000} produces ... ● Be careful with wildcards re.findall(r'(ab).*(cd).*(ef)', 'ab cd ef') # slower re.findall(r'(ab)s(cd)s(ef)', 'ab cd ef') # faster ● Longer target string -> slower regex matching
  38. 38. Regular Expressions Performance Strategies Writing regex ● Use the non-capturing group when no need to capture and save text to a variable (?:abc|def|ghi) instead of (abc|def|ghi)
  39. 39. Regular Expressions Performance Strategies Writing regex ● Use the non-capturing group when no need to capture and save text to a variable (?:abc|def|ghi) instead of (abc|def|ghi) ● Pattern most likely to match first (TRAFFIC_ALLOW|TRAFFIC_DROP|TRAFFIC_DENY)
  40. 40. Regular Expressions Performance Strategies Writing regex ● Use the non-capturing group when no need to capture and save text to a variable (?:abc|def|ghi) instead of (abc|def|ghi) ● Pattern most likely to match first (TRAFFIC_ALLOW|TRAFFIC_DROP|TRAFFIC_DENY) TRAFFIC_(ALLOW|DROP|DENY)
  41. 41. Regular Expressions Performance Strategies Writing regex ● Use the non-capturing group when no need to capture and save text to a variable (?:abc|def|ghi) instead of (abc|def|ghi) ● Pattern most likely to match first (TRAFFIC_ALLOW|TRAFFIC_DROP|TRAFFIC_DENY) TRAFFIC_(ALLOW|DROP|DENY) ● Use anchors (^ and $) to limit the score re.findall(r'(ab){2}', 'abcabcabc') re.findall(r'^(ab){2}','abcabcabc') #failures occur faster
  42. 42. Summary 1. What is a regexp? 2. When to use regexp? 3. Regex basics 4. Performance Tests 5. Writing regexp (Performance Strategies) 6. Writing plugins (Performance Strategies) 7. Tools
  43. 43. Regular Expressions Performance Strategies Writing Agent plugins ● A new process is forked for each loaded plugin ○ Use the plugins that you really need!
  44. 44. Regular Expressions Performance Strategies Writing Agent plugins ● A new process is forked for each loaded plugin ○ Use the plugins that you really need! ● A plugin is a set of rules (regexp operations) for matching log lines ○ If a plugin doesn't match a log entry, it fails in ALL its rules! ○ Reduce the number of rules, use a [translation] table
  45. 45. Regular Expressions Performance Strategies Writing Agent plugins ● Alphabetical order for rule matching ○ Order your rules by priority, pattern most likely to match first
  46. 46. Regular Expressions Performance Strategies Writing Agent plugins ● Alphabetical order for rule matching ○ Order your rules by priority, pattern most likely to match first ● Divide and conquer ○ A plugin is configured to read from a source file, use dedicated source files per technology ○ Also, use dedicated plugins for each technology
  47. 47. Regular Expressions Performance Strategies Tool1 Tool2 Tool3 Tool4 Tool5 20 logs/sec 20 logs/sec 20 logs/sec 20 logs/sec 20 logs/sec /var/log/syslog (100 logs/sec) 5 plugins with 1 rule reading /var/log/syslog 5x100 = 500 total regex/sec
  48. 48. Regular Expressions Performance Strategies Tool1 Tool2 Tool3 Tool4 Tool5 20 logs/sec 20 logs/sec 20 logs/sec 20 logs/sec 20 logs/sec /var/log/tool1 /var/log/tool2 /var/log/tool3 /var/log/tool4 /var/log/tool5 (100 logs/sec) 5 plugins with 1 rule reading /var/log/tool{1-5} 5x20 = 100 total regex/sec (x5) Faster
  49. 49. Summary 1. What is a regexp? 2. When to use regexp? 3. Regex basics 4. Performance Tests 5. Writing regexp (Performance Strategies) 6. Writing plugins (Performance Strategies) 7. Tools
  50. 50. Regular Expressions Tools for testing Regex Python: >>> import re >>> re.findall('(S+) (S+)', 'foo bar') [('foo', 'bar')] >>> result = re.search( ... '(?P<key>w+)s*=s*(?P<value>w+)', ... 'foo=bar' ... ) >>> result.groupdict() { 'key': 'foo', 'value': 'bar' }
  51. 51. Regular Expressions Tools for testing Regex Regex debuggers: ● Kiki ● Kodos Online regex testers: ● http://gskinner.com/RegExr/ (java) ● http://regexpal.com/ (javascript) ● http://rubular.com/ (ruby) ● http://www.pythonregex.com/ (python) Online regex visualization: ● http://www.regexper.com/ (javascript)
  52. 52. any (?:question|doubt|comment)+?
  53. 53. A3Sec web: www.a3sec.com email: training@a3sec.com twitter: @a3sec Spain Head Office C/ Aravaca, 6, Piso 2 28040 Madrid Tlf. +34 533 09 78 México Head Office Avda. Paseo de la Reforma, 389 Piso 10 México DF Tlf. +52 55 5980 3547

×