1. Theory of Automata
&
Formal Languages
1

2. BOOKS
Theory of computer Science: K.L.P.Mishra &
N.Chandrasekharan
Intro to Automata theory, Formal languages and
computation: Ullman,Hopcroft
Motwani
Elements of theory of computation Lewis &
papadimitrou
2

3. Syllabus
Introduction
Deterministic and non deterministic Finite
Automata, Regular Expression,Two way
finite automata,Finite automata with
output,properties of regular sets,pumping
lemma, closure properties,Myhill nerode
theorem
3

4. Context free Grammar: Derivation trees,
Simplification forms
Pushdown automata: Def, Relationship
between PDA and context free
language,Properties, decision algorithms
Turing Machines: Turing machine
model,Modification of turing
machines,Church’s
thesis,Undecidability,Recursive and
recursively enumerable languages Post
correspondence problems recursive
functions
4

5. Chomsky Hierarchy: Regular grammars,
unrestricted grammar, context sensitive
language, relationship among languages
5

6. 6

7. 7

8. Finite Automaton
Input
• String
Output
Finite String
Automaton
8

9. Finite Accepter
Input
• String
Output
“Accept”
Finite
or
Automaton
“Reject”
9

10. Transition Graph
a,b
Abba -Finite Accepter
• q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
initial final
state state
transition
state “accept”
10

11. Initial Configuration
Input String
a b b a
•
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
11

12. Reading the Input
a b b a
•
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
12

13. a b b a
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
13

14. a b b a
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
14

15. a b b a
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
15

16. Input finished
a b b a
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
Output: “accept”16

17. Rejection
a b a
•
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
17

18. a b a
•
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
18

19. a b a
•
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
19

20. a b a
•
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
20

21. Input finished
a b a
a,b
Output:
q5 “reject”
a a,b
b a b
q0 a q1 b q2 b q3 a q4
21

22. Another Example
a a b
a a,b
q0 b a,b
q1 q2
22

23. a a b
a a,b
q0 b a,b
q1 q2
23

24. a a b
a a,b
q0 b a,b
q1 q2
24

25. a a b
a a,b
q0 b a,b
q1 q2
25

26. Input finished
a a b
a a,b
Output: “accept”
q0 b a,b
q1 q2
26

27. Rejection
b a b
a a,b
q0 b a,b
q1 q2
27

28. b a b
a a,b
q0 b a,b
q1 q2
28

29. b a b
a a,b
q0 b a,b
q1 q2
29

30. b a b
a a,b
q0 b a,b
q1 q2
30

31. Input finished
b a b
a a,b
q0 b a,b
q1 q2
Output: “reject”
31

32. • Deterministic Finite Accepter (DFA)
M Q, , , q0 , F
Q : Finite set of states
: input alphabet
: transition function
:Q X Q
q0 : initial state is a member of Q
F : set of final states
32

33. Input Alphabet
a, b
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
33

34. Set of States Q
Q q0 , q1, q2 , q3 , q4 , q5
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
34

35. Initial State q0
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
35

36. Set of Final States F
F q4
a,b
q5
a a,b
b a b
q0 a q1 b q2 b q3 a q4
36

37. Transition Function
:Q Q
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
37

38. q0 , a q1
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
38

39. q0 , b q5
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
39

40. q2 , b q3
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
40

41. a b Transition Function
q0 q1 q5
q1• q5 q2
q2 q2 q3
q3 q4 q5 a,b
q4 q5 q5
q5 q5 q5 q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
41

42. Extended Transition Function *
*: Q * Q
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
42

43. * q0 , ab q2
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
43

44. * q0 , abba q4
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
44

45. * q0 , abbbaa q5
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
45

46. Observation: There is a walk from q0 to q 5
with label abbbaa
* q0 , abbbaa q5
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
46

47. Recursive Definition
* q, q
• * q, wa ( * (q, w), a )
a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
47

48. * q0 , ab
* (q0 , a ), b
* q0 , ,a ,b
q0 , a , b
q1 , b
q2 a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
48

49. Languages Accepted by DFAs
• Take DFA M
• Definition:
– The language L M contains
– all input strings accepted by M
LM
– = { strings that drive M to a final state}
49

50. Example
LM abba M
• a,b
q5
b a a,b
a b
q0 a q1 b q2 b q3 a q4
accept
50

51. Another Example
LM , ab, abba M
• a,b
q5
b a a a,b
b
q0 a q1 b q2 b q3 a q4
accept accept accept
51

52. Formally
• For a DFA Language accepted by M :
LM w * : * q0 , w F
M Q, , , q0 , F
alphabet transition initial final
function state states
52

53. Observation
• Language accepted by M:
LM w * : * q0 , w F
• Language rejected by M :
LM w * : * q0 , w F
53

54. • More Examples
n
LM {a b : n 0}
a a,b
q0 b a,b
q1 q2
accept trap state
or dead state
54

55. LM = { all strings with prefix ab }
• a,b
q0 a q1 b q2
b a accept
q3 a,b
55

56. L M = { all strings without
substring 001 }
•
1 0 0,1
1
0 1
0 00 001
0
56

57. Regular Languages
• A language L is regular if there is
• a DFA M such that L L M
• All regular languages form a language
family
–
57

58. Example
L awa : w a, b *
a
• The language b
• is regular:
b
q0 a q2 q3
•
b a
q4
a,b 58

59. Non Deterministic
Automata
59

60. Non Deterministic Finite
Accepter
M Q, , , q0 , F
Q
:Q 2
60

61. Nondeterministic Finite Accepter (NFA)
Alphabet = {a}
q1 a q2
a
q0
a
q3
61

62. Nondeterministic Finite Accepter (NFA)
Alphabet = {a}
Two choices q1 a q2
a
q0
a
q3
62

63. Nondeterministic Finite Accepter (NFA)
Alphabet = {a}
Two choices q1 a q2 No transition
a
q0
a
q3 No transition
63

64. First Choice
a a
q1 a q2
a
q0
a
q3
64

65. First Choice
a a
q1 a q2
a
q0
a
q3
65

66. First Choice
a a
q1 a q2
a
q0
a
q3
66

67. First Choice
a a
All input is consumed
q1 a q2 “accept”
a
q0
a
q3
67

68. Second Choice
a a
q1 a q2
a
q0
a
q3
68

69. Second Choice
a a
q1 a q2
a
q0
a
q3
69

70. Second Choice
a a
q1 a q2
a
q0
a
No transition:
q3
the automaton hangs
70

71. Second Choice
a a
Input cannot be consumed
q1 a q2
a
q0
a
q3 “reject”
71

72. An NFA accepts a string:
when there is a computation of the NFA
that accepts the string
•All the input is consumed and the automaton
is in a final state
72

73. An NFA rejects a string:
when there is no computation of the NFA
that accepts the string
• All the input is consumed and the
automaton is in a non final state
• The input cannot be consumed
73

74. Example
aa is accepted by the NFA:
“accept”
q1 a q2 q1 a q2
a a
q0 q0
a a
q3 q3 “reject”
because this computation
accepts aa 74

75. Rejection example
a
q1 a q2
a
q0
a
q3
75

76. First Choice
a
q1 a q2
a
q0
a
q3
76

77. First Choice
a
“reject”
q1 a q2
a
q0
a
q3
77

78. Second Choice
a
q1 a q2
a
q0
a
q3
78

79. Second Choice
a
q1 a q2
a
q0
a
q3
79

80. Second Choice
a
q1 a q2
a
q0
a
q3 “reject”
80

81. Example
a is rejected by the NFA:
“reject”
q1 a q2 q1 a q2
a a
q0 q0
a a
q3 “reject” q3
All possible computations lead to rejection
81

82. Rejection example
a a a
q1 a q2
a
q0
a
q3
82

83. First Choice
a a a
q1 a q2
a
q0
a
q3
83

84. First Choice
a a a
q1 a q2
a
q0
a No transition:
the automaton hangs
q3
84

85. First Choice
a a a
Input cannot be consumed
q1 a q2 “reject”
a
q0
a
q3
85

86. Second Choice
a a a
q1 a q2
a
q0
a
q3
86

87. Second Choice
a a a
q1 a q2
a
q0
a
q3
87

88. Second Choice
a a a
q1 a q2
a
q0
a
No transition:
q3
the automaton hangs
88

89. Second Choice
a a a
Input cannot be consumed
q1 a q2
a
q0
a
q3 “reject”
89

90. aaa is rejected by the NFA:
“reject”
q1 a q2 q1 a q2
a a
q0 q0
a a
q3 q3 “reject”
All possible computations lead to rejection
90

91. Language accepted: L {aa}
q1 a q2
a
q0
a
q3
91

92. Lambda →Transitions
q0 a q1 q2 a q3
92

93. a a
q0 a q1 q2 a q3
93

94. a a
q0 a q1 q2 a q3
94

95. (read head doesn’t move)
a a
q0 a q1 q2 a q3
95

96. a a
q0 a q1 q2 a q3
96

97. all input is consumed
a a
“accept”
q0 a q1 q2 a q3
String aa is accepted 97

98. Rejection Example
a a a
q0 a q1 q2 a q3
98

99. a a a
q0 a q1 q2 a q3
99

100. (read head doesn’t move)
a a a
q0 a q1 q2 a q3
100

101. a a a
q0 a q1 q2 a q3
No transition:
the automaton hangs
101

102. Input cannot be consumed
a a a
“reject”
q0 a q1 q2 a q3
String aaa is rejected 102

103. Language accepted: L {aa}
q0 a q1 q2 a q3
103

104. Another NFA Example
q0 a q1 b q2 q3
104

105. a b
q0 a q1 b q2 q3
105

106. a b
q0 a q1 b q2 q3
106

107. a b
q0 a q1 b q2 q3
107

108. a b
“accept”
q0 a q1 b q2 q3
108

109. Another String
a b a b
q0 a q1 b q2 q3
109

110. a b a b
q0 a q1 b q2 q3
110

111. a b a b
q0 a q1 b q2 q3
111

112. a b a b
q0 a q1 b q2 q3
112

113. a b a b
q0 a q1 b q2 q3
113

114. a b a b
q0 a q1 b q2 q3
114

115. a b a b
q0 a q1 b q2 q3
115

116. a b a b
“accept”
q0 a q1 b q2 q3
116

117. Language accepted
L ab, abab, ababab, ...
ab
q0 a q1 b q2 q3
117

118. Another NFA Example
0
q0 q1 0, 1 q2
1
118

119. Language accepted
L(M ) = {λ, 10, 1010, 101010, ...}
= {10}*
0
q0 q1 0, 1 q2
1
119

120. Remarks:
•The symbol never appears on the
input tape
•Extreme automata:
M1 M2
q0 q0
L(M1 ) = {} L(M 2 ) = {λ}
120

121. •NFAs are interesting because we can
express languages easier than DFAs
a,b
NFA M1 DFA M2
q2
q0 a q1 b
a,b
q0 a q1
L( M1 ) = {a} L( M 2 ) = {a} 121

122. Formal Definition of NFAs
M Q, ,
, q0 , F
Q : Set of states, i.e. q0 , q1, q2
: Input alphabet, i.e. a, b
: Transition function
q0 : Initial state
F : Final states 122

123. Transition Function
q0 , 1 q1
0
q0 q1 0, 1 q
2
1
123

124. (q1,0) {q0 , q2}
0
q0 q1 0, 1 q
2
1
124

125. (q0 , ) {q0 , q2}
0
q0 q1 0, 1 q
2
1
125

126. (q2 ,1)
0
q0 q1 0, 1 q
2
1
126

127. Extended Transition Function *
* q0 , a q1
•
q4 q5
a a
q0 a q1 b q2 q3
127

128. * q0 , aa q4 , q5
q4 q5
a a
q0 a q1 b q2 q3
128

129. * q0 , ab q2 , q3 , q0
q4 q5
a a
q0 a q1 b q2 q3
129

130. Formally
qj * qi , w
It holds
if and only if
there is a walk from qi to q j
with label w
130

131. The Language of an NFA M
F q0 ,q5
q4 q5
• a a
q0 a q1 b q2 q3
* q0 , aa q4 , q5 aa L(M )
131

132. F q0 ,q5
q4 q5
a a
q0 a q1 b q2 q3
* q0 , ab q2 , q3 , q0 ab L M
132

133. F q0 ,q5
q4 q5
• a a
q0 a q1 b q2 q3
* q0 , abaa q4 , q5 aaba L(M )
133

134. F q0 ,q5
q4 q5
a a
q0 a q1 b q2 q3
* q0 , aba q1 aba L M
134

135. q4 q5
a a
q0 a q1 b q2 q3
LM aa ab * ab aa
135

136. Formally
• The language accepted by NFA M is:
LM w1, w2 , w3 ,...
* (q0 , wm ) {qi , q j ,...}
• where
qk F (final state)
136

137. w LM * (q0 , w)
• qi
w
qk qk F
q0 w
w qj
137

138. Equivalence of NFAs and DFAs
138

139. Equivalence of Machines
• For DFAs or NFAs:
• Machine M1 is equivalent to machine M
2
• if L M1 L M2
• 139

140. NFA M1
L M1 {10} * 0
q0 q1 0, 1 q2
•
1
DFA M2 0,1
L M2 {10}* 0
q0 q1 1 q2
1
0 140

141. • Since L M1 L M2 10 *
• machines M1 and M 2are equivalent
0
NFA M1 q0 q1 0, 1 q2
1
0,1
0
DFA M2 q0 q1 1 q2
1
0 141

142. Equivalence of NFAs and DFAs
Question: NFAs = DFAs ?
Same power?
Accept the same languages?
142

143. Equivalence of NFAs and DFAs
Question: NFAs = DFAs ? YES!
Same power?
Accept the same languages?
143

144. We will prove:
Languages Languages
accepted accepted
by NFAs by DFAs
NFAs and DFAs have the same
computation power
144

145. Step 1
Languages Languages
accepted accepted
by NFAs by DFAs
Proof: Every DFA is trivially an NFA
A language accepted by a DFA
is also accepted by an NFA 145

146. Step 2
Languages Languages
accepted accepted
by NFAs by DFAs
Proof: Any NFA can be converted to an
equivalent DFA
A language accepted by an NFA
is also accepted by a DFA 146

147. NFA to DFA
NFA M a
q0 a q1 q2
• b
DFA M
q0
147

148. NFA M a
q0 a q1 q2
• b
DFA M
q0 a
q1, q2
148

149. NFA M a
q0 a q1 q2
• b
DFA M
q0 a
q1, q2
b
149

150. NFA M a
q0 a q1 q2
• b
a
DFA M
q0 a
q1, q2
b
150

151. NFA M a
q0 a q1 q2
• b
b a
DFA M
q0 a
q1, q2
b
151

152. NFA M a
q0 a q1 q2
• b
b a
DFA M
q0 a
q1, q2
b
a, b
152

153. NFA M a LM L(M )
q0 a q1 q2
• b
a
DFA M b
q0 a
q1, q2
b
a, b
153

154. NFA to DFA: Remarks
• We are given an NFA M
• We want to convert it to an equivalent DFA M
LM L(M )
154

155. • If the NFA has states
q0 , q1, q2 ,...
• the DFA has states in the power set
•
, q0 , q1 , q1, q2 , q3 , q4 , q7 ,....
155

156. Procedure NFA to DFA
•
• 1. Initial state of NFA: q0
•
• Initial state of DFA: q0
156

157. NFA M a
q0 a q1 q2
• b
DFA M
q0
157

158. Procedure NFA to DFA
• 2. For every DFA’s state {qi , q j ,..., qm}
• Compute in the NFA
* qi , a ,
* q j,a , {qi , q j ,..., qm}
•
...
{qi , q j ,..., qm}, a {qi , q j ,..., qm}
•
158
Add transition to DFA

159. NFA M a
q0 a q1 q2
• b
* (q0 , a ) {q1, q2 }
DFA M
q0 a
q1, q2
q0 , a q1, q2 159

160. Procedure NFA to DFA
• Repeat Step 2 for all letters in alphabet,
• until
• no more transitions can be added.
160

161. NFA M a
q0 a q1 q2
• b
b a
DFA M
q0 a
q1, q2
b
a, b
161

162. Procedure NFA to DFA
• 3. For any DFA state {qi , q j ,..., qm}
• If some q j is a final state in the NFA
• Then, {qi , q j ,..., qm }
•
162
is a final state in the DFA

163. NFA M a
q0 a q1 q2 q1 F
• b
a
DFA M b
q0 a
q1, q2
b q1, q2 F
a, b
163

164. Theorem
Take NFA M
•
Apply procedure to obtain DFA M
Then M and M are equivalent :
LM LM
164

165. Finally
We have proven
Languages Languages
accepted accepted
by NFAs by DFAs
165

166. We have proven
Languages Languages
accepted accepted
by NFAs by DFAs
Regular Languages
166

167. We have proven
Languages Languages
accepted accepted
by NFAs by DFAs
Regular Languages Regular Languages
167

168. We have proven
Languages Languages
accepted accepted
by NFAs by DFAs
Regular Languages Regular Languages
Thus, NFAs accept the regular languages
168

169. Single Final State
for NFAs and DFAs
169

170. Observation
• Any Finite Automaton (NFA or DFA)
• can be converted to an equivalent NFA
• with a single final state
170

171. Example
a NFA
a b
b
a Equivalent NFA
a b
b
171

172. In General
NFA
Equivalent NFA
Single
final state
172

173. Extreme Case
NFA without final state
Add a final state
Without transitions
173

174. Some Properties of
Regular Languages
174

175. properties
For regular languages L1 and L
2
we will prove that:
Union: L1 L2
Concatenation: L1L2 Are regular
Languages
Star: L1 *
175

176. We Say:
Regular languages are closed under
Union: L1 L2
Concatenation: L1L2
Star: L1 *
176

177. Properties
For regular languages L1 and L2
we will prove that:
Union: L1 L2
Are regular
Concatenation: L1L2
Languages
Star: L1 *
177

178. Regular language L1 Regular language L2
L M1 L1 L M2 L2
NFA M1 NFA M2
Single final state Single final state
178

179. Example
a
n M1
L1 {a b} b
M2
L2 ba b a
179

180. L1 L2 Union
M1
• NFA for
M2
180

181. n
L1 L2 {a b} {ba}
NFA for
n
• L1 {a b}
a
b
L2 {ba}
b a
181

182. Concatenation
L1L2
• NFA for
M1 M2
182

183. Example
n n
L1L2 {a b}{ba} {a bba}
•
• NFA for
n
L1 {a b}
a L2 {ba}
b b a
183

184. Star Operation
• NFA for L *
1
L1 *
M1
184

185. Example
n
L1* {a b} *
• NFA for
n
L1 {a N>=}
b0
a
b
185

186. Procedure: NFA to DFA
1 Create a graph with vertex {q0}.Identify this vertex as
initial vertex of DFA.
2 Repeat the following steps until no more edges are
missing.Take any vertex {qi,qj,….qk} of G that has no
outgoing edge for some symbol a of the alphabet.
Compute *(qi, a), * (qj, a)…. *(qk, a)
Then form the union of all these yielding the set *{ql,
qm, …qn}.
Create a vertex for G labeled {ql, qm,…qn} if it does not
already exist.
Add to G an edge from {qi, qj,…qk} to {ql,qm…qn} and
label it with a.
3 Every state of G whose label contains any qf of F is
identified as a final vertex of DFA.
4 If NFA accepts then vertex {q0} in G is also made a
186
final vertex.

187. 1
0
q0 q1 q2
0,1 0,1
187

188. Start
Equivalent q0 1
0
DFA q
{q0, 1
q1} 1
0,1
1 q2
0
0,1 0
q0,q1,q2 1 q1,q2
0 0,1
188