Mindtalk Technology - behind the scenes

1. Mindtalk Technology
Behind the Scenes
Robin Syihab (CTO Mindtalk)
MT: @robin
TW: @anvie

2. Apa itu “Scalability”?

3. “Scalability” bukan
• Performa yang tinggi.
• Mudah untuk menambahkan server.
• High availability.

4. Skalabilitas adalah
• Kemampuan dalam mengakomodir
peningkatan permintaan yang tinggi.
• Bukan hanya masalah server atau
teknologi, tetapi juga sumber daya
manusia dan proses di dalamnya.

5. Bagaimana Mindtalk
menghadapi masalah
skalabilitas?

6. Database
• MongoDB (main database).
• Redis (cache).
• Tokyo Cabinet (url shortener).

7. Database
Write
MongoDB
(master)
Read
MongoDB MongoDB
(shard)
(shard)
MongoDB
(replica)
MongoDB MongoDB
(shard)
(shard)
HAProxy
Service
Cache

8. Web Server
• Nginx
• HAProxy (failover)

9. Web Service
web service
HAProxy
consumer
rest API
service
Mobile web
service

10. Tips Optimasi
• Selalu aktifkan gzip.
• Jangan gunakan domain yang sama
untuk web dan serving static files.
• Tuning web level cache.
• nginx workers == jumlah core.
• Jika perlu gunakan varnish.

11. Elastic Search
• Mudah menambahkan shard, tidak
perlu servis tambahan seperti servis
kordinator, contoh: Zookeeper.
• Skema dokumen yang flexibel.
• Faceting.
• Pencarian ke banyak index.
• Automatic shard rebalancing.

12. Development
Process

13. Source Control Management : GIT

14. Continuous Integration: Buildbot

15. Code Inspection:
Gitlab

16. Monitoring System: PandoraFMS
http://www.lafermeduweb.net

17. Deployment System
Makefile
increment version
pre-build test
build distribution
staging
backup
updating

18. Logging
Levels:
DEBUG
file A (rotated)
INFO
WARNING
ERROR
file B (rotated)
file C (rotated)
email to maintainers
includes technical
information

19. Bahasa
pemrograman
• Python - Backend services.
• Scala - Stream processing.
• Node.js - Realtime notifier.
• PHP - Front end.

20. MID ENDS
FRONT ENDS
Go
Anscdn & AWS
Cache
BACK ENDS
Database
Replica
Closure
Library
Sock API
•3 instances
Search
Engine
•Includes
Database
Lucene Engine
round-robin by
hproxy
Ndayak 3 Engine
Notifier
•Web
Notification
long polling
web API
Cache
•Authentication
•API
Hadoop
Data mining storage
•Mobile Push
Notification
Message Queueing
•Purpose to:
• Generate analytics.
• Calculate search relevancy.
• Calculate suggestion like
PYMK or GYML.

21. Geek Wanted
• Java / Scala Programmer.
• NoSQL Database Engineer.
• System Administrator.
kirimkan CV ke:
job@mindtalk.com

22. Terima kasih