This document appears to be notes from a presentation about the evolution of Travis CI from a monolithic application to a distributed, service-oriented architecture. It describes how Travis CI was originally a single Ruby on Rails application, but was broken out into multiple services for improved scalability, reliability and maintainability. Key points discussed include introducing background workers, switching to VirtualBox for isolated builds, adding a message queue and splitting functionality like the API into separate services/domains. Design philosophies around incremental changes, metrics, feature flags and database migrations are also covered.

1. привет
Tuesday, May 28, 13

2. Piotr Sarnacki
Tuesday, May 28, 13

3. @drogus
Tuesday, May 28, 13

4. TravisCI
Tuesday, May 28, 13

5. TravisCI
What’s inside?
Tuesday, May 28, 13

6. Poland
Tuesday, May 28, 13

7. Tuesday, May 28, 13

8. Tuesday, May 28, 13

9. Tuesday, May 28, 13

10. Tuesday, May 28, 13

11. Tuesday, May 28, 13

12. Tuesday, May 28, 13

13. ZOMG!
THIS IS MY DREAM JOB!
Tuesday, May 28, 13

14. Getting back to presentation...
Tuesday, May 28, 13

15. A few questions first
Tuesday, May 28, 13

16. Tuesday, May 28, 13

17. Distributed CI
Tuesday, May 28, 13

18. Distributed CI
for Open Source!
Tuesday, May 28, 13

19. Distributed CI
and for private projects
Tuesday, May 28, 13

20. Distributed CI
and for private projects
soon!
Tuesday, May 28, 13

21. We’re for builds
what RubyGems
is for libraries
Tuesday, May 28, 13

22. A few facts
Tuesday, May 28, 13

23. close to 23,000 builds daily
A few facts
Tuesday, May 28, 13

24. close to 23,000 builds daily
A few facts
over 5MLN builds total!
Tuesday, May 28, 13

25. close to 23,000 builds daily
A few facts
over 5MLN builds total!
ruby, python, java, node... etc.
Tuesday, May 28, 13

26. What about Jenkins?
Tuesday, May 28, 13

27. Реальный опыт работы с Jenkins
What about Jenkins?
Tuesday, May 28, 13

28. What’s the purpose
of this presentation?
Tuesday, May 28, 13

29. Open Source distributed
application
Tuesday, May 28, 13

30. Open Source distributed
application
we need to scale
Tuesday, May 28, 13

31. Open Source distributed
application
you can watch the entire process
we need to scale
Tuesday, May 28, 13

32. Open Source distributed
application
you can see us fail and learn from
our mistakes
you can watch the entire process
we need to scale
Tuesday, May 28, 13

33. Running an app is different
than running Open Source
library
Tuesday, May 28, 13

34. Tuesday, May 28, 13

35. A bit of a history
Tuesday, May 28, 13

36. It all started with a mustache
Tuesday, May 28, 13

37. Tuesday, May 28, 13

38. svenfuchs/travis-ci
Tuesday, May 28, 13

39. Tuesday, May 28, 13

40. It worked
Tuesday, May 28, 13

41. but...
Tuesday, May 28, 13

42. Monolithic application
Tuesday, May 28, 13

43. Tuesday, May 28, 13

44. Tuesday, May 28, 13

45. Two concerns
Tuesday, May 28, 13

46. Two concerns
serving web requests
Tuesday, May 28, 13

47. Two concerns
serving web requests
background worker
Tuesday, May 28, 13

48. Problems
Tuesday, May 28, 13

49. Problems
growing complexity
Tuesday, May 28, 13

50. Problems
growing complexity
growing test suite
Tuesday, May 28, 13

51. Solution: Travis Worker
Tuesday, May 28, 13

52. Tuesday, May 28, 13

53. Great improvement!
Tuesday, May 28, 13

54. but...
Tuesday, May 28, 13

55. Single box, no VMs
Tuesday, May 28, 13

56. No clean state
Tuesday, May 28, 13

57. Leftovers after tests
Tuesday, May 28, 13

58. Solution: VirtualBox
Tuesday, May 28, 13

59. VirtualBox
Tuesday, May 28, 13

60. VirtualBox
snapshotting
Tuesday, May 28, 13

61. VirtualBox
snapshotting
always clean state
Tuesday, May 28, 13

62. VirtualBox
snapshotting
always clean state
free
Tuesday, May 28, 13

63. Tuesday, May 28, 13

64. Still only one box
Tuesday, May 28, 13

65. Still a lot of coupling
Tuesday, May 28, 13

66. Tuesday, May 28, 13

67. Tuesday, May 28, 13

68. Master plan
Tuesday, May 28, 13

69. Master plan
use AMQP for messaging
Tuesday, May 28, 13

70. Master plan
use AMQP for messaging
create travis-hub
Tuesday, May 28, 13

71. Master plan
use AMQP for messaging
create travis-hub
use jruby for the worker
Tuesday, May 28, 13

72. Why AMQP?
Tuesday, May 28, 13

73. Unify messaging
Tuesday, May 28, 13

74. Log output
Tuesday, May 28, 13

75. Tuesday, May 28, 13

76. When rails app fails,
builds still run
Tuesday, May 28, 13

77. So many changes
Tuesday, May 28, 13

78. All at once?
Tuesday, May 28, 13

79. Not that smart move
Tuesday, May 28, 13

80. 3 months of
development
Tuesday, May 28, 13

81. A lot of firefighting
Tuesday, May 28, 13

82. Lesson learned
Tuesday, May 28, 13

83. Lesson learned
smaller incremental
changes are better
Tuesday, May 28, 13

84. Overally,
great improvement again
Tuesday, May 28, 13

85. but...
Tuesday, May 28, 13

86. Tuesday, May 28, 13

87. Tuesday, May 28, 13

88. Tuesday, May 28, 13

89. Meet travis-listener
Tuesday, May 28, 13

90. Tuesday, May 28, 13

91. Tuesday, May 28, 13

92. Tuesday, May 28, 13

93. Tuesday, May 28, 13

94. Tuesday, May 28, 13

95. Tuesday, May 28, 13

96. Tuesday, May 28, 13

97. Better reliability
Tuesday, May 28, 13

98. Rare changes
last deploy: 2012-12-05.11-27
Tuesday, May 28, 13

99. Small chance of loosing
an event
Tuesday, May 28, 13

100. It’s great!
Tuesday, May 28, 13

101. but...
Tuesday, May 28, 13

102. Travis Hub does a lot
Tuesday, May 28, 13

103. Tuesday, May 28, 13

104. Tuesday, May 28, 13

105. A great improvement again!
Tuesday, May 28, 13

106. but...
Tuesday, May 28, 13

107. API is coupled with client
Tuesday, May 28, 13

108. Tuesday, May 28, 13

109. Tuesday, May 28, 13

110. API does not change that often
Tuesday, May 28, 13

111. Easier deploys
Tuesday, May 28, 13

112. Separate domains?
Tuesday, May 28, 13

113. CORS to the rescue
Tuesday, May 28, 13

114. CORS
Tuesday, May 28, 13

115. CORS
Cross Origin Resource Sharing
Tuesday, May 28, 13

116. CORS
Cross Origin Resource Sharing
allows remote ajax requests
Tuesday, May 28, 13

117. CORS
Cross Origin Resource Sharing
allows remote ajax requests
client and API isolation
Tuesday, May 28, 13

118. Service Oriented Architectures
Tuesday, May 28, 13

119. SOA - benefits
Tuesday, May 28, 13

120. SOA - benefits
reliability
Tuesday, May 28, 13

121. SOA - benefits
reliability
scalability
Tuesday, May 28, 13

122. SOA - benefits
reliability
scalability
maintainability
Tuesday, May 28, 13

123. SOA - drawbacks
Tuesday, May 28, 13

124. SOA - drawbacks
more apps to change
Tuesday, May 28, 13

125. SOA - drawbacks
more apps to change
more overhead
Tuesday, May 28, 13

126. SOA - drawbacks
more apps to change
more overhead
distributed computing is hard
Tuesday, May 28, 13

127. From monolithic app to SOA!
Tuesday, May 28, 13

128. We’re awesome, aren’t we?
Tuesday, May 28, 13

129. we can always do better
Tuesday, May 28, 13

130. Can you spot the problem?
Tuesday, May 28, 13

131. Tuesday, May 28, 13

132. Still a lot of sharing
Tuesday, May 28, 13

133. Tuesday, May 28, 13

134. Tuesday, May 28, 13

135. Tuesday, May 28, 13

136. Future direction:
remove more coupling
Tuesday, May 28, 13

137. Well defined public APIs
Tuesday, May 28, 13

138. Our code sharing
is not optimal
Tuesday, May 28, 13

139. travis-core
Tuesday, May 28, 13

140. Tuesday, May 28, 13

141. Split travis-core
Tuesday, May 28, 13

142. A few tips
Tuesday, May 28, 13

143. Small changes
Tuesday, May 28, 13

144. Metrics
Tuesday, May 28, 13

145. Tuesday, May 28, 13

146. Don’t break public APIs
Tuesday, May 28, 13

147. If you need,
deprecate first
Tuesday, May 28, 13

148. Always deployable master
Tuesday, May 28, 13

149. Feature flips
Tuesday, May 28, 13

150. if Travis::Features.active? :foo
do_foo()
end
Tuesday, May 28, 13

151. Bulletproof DB changes
Tuesday, May 28, 13

152. if column_names.include? :branch
where(branch: 'master')
else
joins(:commit).
where(commit: { branch: 'master' })
end
Tuesday, May 28, 13

153. Google for:
Fallacies of distributed
computing
Tuesday, May 28, 13

154. Thank you
Tuesday, May 28, 13

155. Questions?
Tuesday, May 28, 13