3. • Java Storm
– More powerful features
– More stable
– More faster
What’s the JStorm
Alibaba
4. • JStorm Team was among one of the earliest that
uses Storm in China.
– Storm 0.5.1/0.5.4/0.6.0/0.6.2/0.7.0/0.7.1
– JStorm 0.7.1/0.9.0/0.9.1/0.9.2/0.9.3/…
• Our Duties
– Application Development
– JStorm System Development
– JStorm System Operation
Who we are?
Alibaba
5. • Storm community is not as active as
we’ve expected
– Tailored for enterprise environment
– Fixed critical bugs in Storm
– Provided professional technical support,
improved app development pace.
– Reduced operational cost.
Why start Jstorm?
Alibaba
6. • Too much requirement drive us move faster
– Release 11 version in 2014
– Refer to https://github.com/alibaba/jstorm/releases
Evolution speed
Alibaba
7. • Start design from 2012/02/07
• Release first version 0.7.1 2013/04/30
Jstorm history
Alibaba
8. • Most of powerful Chinese Company
Who are Using Jstorm?
Alibaba
9. • More than 3000 servers
• More than 3 trillion messages per day
• More than 300 topology
How big in Alibaba?
Alibaba
10. • Live Alibaba 11.11 room
– Trade amount/count
– PV/UV
– All kinds of KPI
• The peak volume of JStorm messaged being process
ed during 11.11,12.12 Shopping Feistivals is ten time
s as large compared to the peak volume on a normal
day.
User Scenario
Alibaba
11. • Realtime Recommended Ad
– Analysis user action, then recommend production
User Scenario
Alibaba
12. • Log Analysis
– Get all kinds of KPI
– Monitor
– Smart Customer Service
– Tlog/EagleEye
User Scenario
Alibaba
13. • Realtime Data sync pipeline
– DB
– Log
– Message
User Scenario
Alibaba
14. • 3 Examination every year
– 11/11
– 12/12
– Spring Festival, red package war
– Ten throughput peak period
Why Stable?
Alibaba
15. • Nimbus HA
• Support Resource Isolation with Cgroups
• Fix bugs under Hadoop-yarn
• Monitor every phase of tuple
• Tuning GC parameter
• Graceful worker shutdown
Improve stability
Alibaba
18. • More powerful scheduler
• More powerful metrics system
• Support Classloader
• More convenient Web UI/LogView
• Support sync mode for Netty RPC frameworker
• New transaction programming mode
• Self-adaption speed
More features
Alibaba
19. • More than 100 improvements
– https://github.com/alibaba/JStorm/blob/master/history.md
More details
Alibaba
20. • Make evolution faster
– Full time developer
– Full time tester
– Hundreds of application which can test new feature quickly
– Java core will bring more developer
What can we bring?
Alibaba