The document discusses regular expressions and finite state machines, showing examples of parsing routes with Journey and building non-deterministic and deterministic finite automata to represent the routes. It demonstrates how to simulate finite state machines and convert a non-deterministic finite automaton to a deterministic one to recognize route patterns more efficiently.

1. Who makes the best
Asado?

8. What country has the
best footballers?

10. Argentina!

11. river

12. river
Nacional B???

14. NOOOOOOOO!!!!

15. Aaron Patterson

16. @tenderlove

17. Ruby core team

18. Rails core team

19. ! WARNING !

20. ZOMG!

21. We are in Argentina!

23. José Yo

24. WWFMD?

25. me gusta

26. ¿Por qué Maria?

27. THANKS!!

29. Resource
Management

30. Path Management

31. Move bet ween theory
and practice
Pattern
Matching

32. Graphviz
http://graphviz.org

33. greeting
hello world

34. graph.dot
digraph nfa {
rankdir=LR;
world [shape = doublecircle];
hello [shape = circle];
hello -> world [label="greeting"];
}

35. graph.dot
digraph nfa {
rankdir=LR;
world [shape = doublecircle];
hello [shape = circle];
goodbye [shape = circle];
hello -> world [label="greeting"];
goodbye -> world
}

36. hello
greeting
world
goodbye

37. /articles(.:format)
/ articles . (?-mix:[^./?]+)
0 1 2 3 5
/
new
4 6 /articles/new(.:format)

38. Journey

39. Journey
Yes, it's named after the '70s rock sensation

40. Provides
1: URL generation
2: Path Recognition
3: Route parsing
What is a router?

41. Why a new router?

42. Maintenance

43. LOC
3000
2250
1500
750
0
Journey Journey-2 Rack-Mount

44. Known algorithms

45. References
• Compilers: Principles, Techniques, &
Tools (Aho, Lam, Sethi, Ullman)
• Intro to Formal Languages and
Automata (Linz)

46. Predictability

47. CPU Time

48. Memory Usage

49. Current Performance
rack-mount Journey
url generation path recognition route parsing

50. me gusta

51. Patterns

52. Why patterns matter

53. Regular Expressions
/om(g)!/

54. Rails Routes

55. resource :articles

56. /articles(.:format)
/articles/new(.:format)
/articles/:id/edit(.:format)
/articles/:id(.:format)

57. Parens don't capture

58. :whatever => /([^./?]+)/

59. /articles(.:format)
//articles(?:.([^./?]+))?$/

60. //articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

61. GET /articles/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

62. GET /articles/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

63. GET /articles/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/
200 OK!

64. //articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

65. GET /foos/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

66. GET /foos/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

67. GET /foos/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

68. GET /foos/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

69. GET /foos/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/

70. GET /foos/new
//articles(?:.([^./?]+))?$/
//articles/new(?:.([^./?]+))?$/
//articles/([^./?]+)/edit(?:.([^./?]+))?$/
//articles/([^./?]+)(?:.([^./?]+))?$/
404 Not Found

71. How long does this
take?

72. r = routes.length

73. x = regexp compare

74. O(r ⨉ x)

75. Can we do better?

76. Parse Trees
Automata
Finite State Machines

77. Parse Trees

78. Parsing regexp

79. (a|b)*abb

80. ○ Describe node types
○ b
○ b
a *
()
|
a b

81. Parsing rails routes

82. Journey::Parser
parser = Journey::Parser.new
ast = parser.parse '/articles(.:format)'
puts ast.to_dot

83. ○
○ ()
/ articles ○
. :format

84. To String!
parser = Journey::Parser.new
ast = parser.parse '/articles(.:format)'
puts ast.to_s
puts ast.to_regexp

85. OR AST
parser = Journey::Parser.new
trees = [
'/articles(.:format)',
'/articles/new(.:format)',
].map { |s| parser.parse s }
ast = Journey::Nodes::Or.new trees
puts ast.to_dot

86. |
○ ○
○ () ○ ()
/ articles ○ ○ new ○
. :format ○ / . :format
/ articles

87. SECRET FEATURE
parser = Journey::Parser.new
ast = parser.parse '/articles|books(.:format)'

88. SECRET FEATURE
parser = Journey::Parser.new
RE T!
EC
ast = parser.parse '/articles|books(.:format)'
RS
SU PE

89. Automata

90. (a|b)*abb

91. Double circle
is acceptance
state
b a
a b
b 0 1 a
a 2
3
b

92. baabb

93. b a
a b
b 0 1 a
a 2
3
b

94. b
b a
a b
b 0 1 a
a 2
3
b

95. ba
b a
a b
b 0 1 a
a 2
3
b

96. baa
b a
a b
b 0 1 a
a 2
3
b

97. baabb
b a
a b
b 0 1 a
a 2
3
b

98. aab
b a
a b
b 0 1 a
a 2
3
b

99. Deterministic Finite
Automaton

100. Only one path

101. Storage
class DFA
attr_reader :table
def initialize
@table = Hash.new { |h, from| h[from] = {} }
@table[0]['a'] = 1
@table[0]['b'] = 0
@table[1]['a'] = 1
@table[1]['b'] = 2
@table[2]['a'] = 1
@table[2]['b'] = 3
@table[3]['b'] = 0
@table[3]['a'] = 2
end
end

102. FSM Simulation

103. class Simulator
def initialize(dfa)
@dfa = dfa
end
def simulate(symbols)
state = 0
until symbols.empty?
state = @dfa.move(state, symbols.shift)
end
state
end
end

104. move function
class DFA
...
def move(from, symbol)
@table[from][symbol]
end
end

105. irb> sim = Simulator.new(DFA.new)
=> #<Simulator:0x007f95929a82e0 ...>
irb> sim.simulate %w{ b a a b b }
=> 3
irb> sim.simulate %w{ a a b }
=> 2

106. Time: O(n)
n = string.length

107. me gusta

108. Space: S + T
states.length + transitions.length

109. Nondeterministic
Finite Automaton

110. Has nil edges

111. Can't tell direction

112. Simulation of NFA

113. a|b
a
ε 2 3 ε
0 ε ε 6
b
4 5

114. nil-closure
a
ε 2 3 ε
0 ε ε 6
b
4 5

115. a
a
ε 2 3 ε
0 ε ε 6
b
4 5

116. Storage
class NFA
def initialize
@table = Hash.new { |h, from|
h[from] = Hash.new { |i,sym| i[sym] = [] }
}
@table[0][nil] << 2
@table[0][nil] << 4
@table[2]['a'] << 3
@table[4]['b'] << 5
@table[3][nil] << 6
@table[5][nil] << 6
end
def nil_closure(states)
states.map { |s| @table[s][nil] }.flatten
end
end

117. FSM Simulation
class Simulator
def initialize(nfa)
@nfa = nfa
end
def simulate(symbols)
states = @nfa.nil_closure([0])
until symbols.empty?
next_s = @nfa.move(states, symbols.shift)
states = @nfa.nil_closure(next_s)
end
states
end
end

118. Move function
class NFA
...
def move(states, symbol)
states.map { |s| @table[s][symbol] }.flatten
end
end

119. irb> sim = Simulator.new(NFA.new)
=> #<Simulator:0x007faa188a5f88 ...>
irb> sim.simulate %w{ a }
=> [6]
irb> sim.simulate %w{ b }
=> [6]
irb> sim.simulate %w{ b b }
=> []
irb> sim.simulate %w{ c }
=> []

120. Time: O(r ⨉ x)
r = operators.length, x = string.length

121. Who cares about
NFA?

122. NFA Construction

123. /articles(.:format)
○
○ ()
/ articles ○
. :format

124. cat nodes
/
0 1

125. /articles
/ articles
0 1 2

126. Optional
ε
0 ε ????? ε 3
1 2

127. ε
/ articles
0 1 2 ε ε 6
. :format
3 4 5

128. Journey::NFA
parser = Journey::Parser.new
ast = parser.parse '/articles(.:format)'
nfa = Journey::NFA::Builder.new ast
tt = nfa.transition_table
puts tt.to_dot

129. Converting
NFA to DFA

130. Eliminate nil
transitions

131. Collapse duplicate
edges

132. /articles(.:format)
ε
/ articles
0 1 2 ε ε 6
. :format
3 4 5

133. /articles(.:format)
/ articles . :format
0 1 2 3 4

134. CODES!
parser = Journey::Parser.new
ast = parser.parse '/articles(.:format)'
nfa = Journey::NFA::Builder.new ast
tt = nfa.transition_table.generalized_table
puts tt.to_dot

135. SHORTER CODES!
parser = Journey::Parser.new
ast = parser.parse '/articles(.:format)'
dfa = Journey::GTG::Builder.new ast
tt = dfa.transition_table
puts tt.to_dot

136. ANY NFA
converts to DFA

137. O(r ⨉ x) => O(x)
r = operations.length, x = string.length

138. me gusta

139. Converting Automata
to Regexp

140. /articles(.:format)
/ articles . :format
0 1 2 3 4

141. /articles(.:format)
/articles . :format
0 2 3 4

142. /articles(.:format)
/articles .:format
0 2 4

143. /articles(.:format)
/articles(.:format)?
0 4

144. Generalized
Transition Graph

145. resource :users
/users(.:format)
/users/new(.:format)
/users/:id/edit(.:format)
/users/:id(.:format)

146. resource :users
(?-mix:[^./?]+)
. 3 5
/ users
0 1 2 /
new . (?-mix:[^./?]+)
4 6 8 11
(?-mix:[^./?]+)
/ edit . (?-mix:[^./?]+)
7 9 12 14 15
.
(?-mix:[^./?]+)
10 13

147. The Plan?

148. Combine All Routes

149. Produce DFA

150. Simulate in O(n) Time

151. Routing To The
Future

152. JS Simulation

153. Table => JSON
parser = Journey::Parser.new
ast = parser.parse '/articles(.:format)'
dfa = Journey::GTG::Builder.new ast
tt = dfa.transition_table
puts tt.to_json

154. Table => SVG
parser = Journey::Parser.new
ast = parser.parse '/articles(.:format)'
dfa = Journey::GTG::Builder.new ast
tt = dfa.transition_table
puts tt.to_svg

155. JS Tokenizer
function tokenize(input, callback) {
while(input.length > 0) {
callback(input.match(/^[/.?]|[^/.?]+/)[0]);
input = input.replace(/^[/.?]|[^/.?]+/, '');
}
}

156. JS Simulator
tokenize(input, function(token) {
var new_states = [];
for(var key in states) {
var state = states[key];
if(string_states[state] && string_states[state][token]) {
var new_state = string_states[state][token];
highlight_edge(state, new_state);
highlight_state(new_state);
new_states.push(new_state);
}
}
if(new_states.length == 0) {
return;
}
states = new_states;
});

157. d3.js

159. Rails Console
irb> File.open('out.html', 'wb') { |f|
irb* f.write(
irb* Wot::Application.routes.router.visualizer
irb> )}
=> 69074

160. routes.rb marshalling

161. Test suggestions

162. Test coverage

163. Usage Heat Maps

164. AST
REGEXP NFA
DFA

165. ROFLSCALE!!!

166. DFA => YACC

167. DFA => RACC

168. DFA => Ragel

169. Open Questions

170. Is our GTG
deterministic?

171. /users/new|/users/:id
new 4
/ users / (?-mix:[^./?]+)
0 1 2 3
5

172. "new" =~ /new|[^./?]+/

173. Can we make it
deterministic?

174. L1 = {new}
L2 = {[^./?]+}

175. /users/new|/users/:id
L1 4
/ users / L2 - L1
0 1 2 3
5

176. Is it worth our effort?

177. AST
REGEXP NFA
DFA

178. Is it worth our effort?

179. Thank You!!

180. <3<3<3<3<3