This document summarizes Mindtalk's approach to scalability. It discusses how Mindtalk uses databases like MongoDB and Redis with sharding and replication to handle high demand. It also covers Mindtalk's use of services like Nginx, HAProxy, Elastic Search, and message queues. The document provides tips on optimization, discusses Mindtalk's development processes involving tools like Git, Buildbot, and PandoraFMS, and lists some open job positions.

1. Mindtalk Technology
Behind the Scenes
Robin Syihab (CTO Mindtalk)
MT: @robin
TW: @anvie

2. Apa itu “Scalability”?

3. “Scalability” bukan
• Performa yang tinggi.
• Mudah untuk menambahkan server.
• High availability.

4. Skalabilitas adalah
• Kemampuan dalam mengakomodir
peningkatan permintaan yang tinggi.
• Bukan hanya masalah server atau
teknologi, tetapi juga sumber daya
manusia dan proses di dalamnya.

5. Bagaimana Mindtalk
menghadapi masalah
skalabilitas?

6. Database
• MongoDB (main database).
• Redis (cache).
• Tokyo Cabinet (url shortener).

7. Database
Write
MongoDB
(master)
Read
MongoDB MongoDB
(shard)
(shard)
MongoDB
(replica)
MongoDB MongoDB
(shard)
(shard)
HAProxy
Service
Cache

8. Web Server
• Nginx
• HAProxy (failover)

9. Web Service
web service
HAProxy
consumer
rest API
service
Mobile web
service

10. Tips Optimasi
• Selalu aktifkan gzip.
• Jangan gunakan domain yang sama
untuk web dan serving static files.
• Tuning web level cache.
• nginx workers == jumlah core.
• Jika perlu gunakan varnish.

11. Elastic Search
• Mudah menambahkan shard, tidak
perlu servis tambahan seperti servis
kordinator, contoh: Zookeeper.
• Skema dokumen yang flexibel.
• Faceting.
• Pencarian ke banyak index.
• Automatic shard rebalancing.

12. Development
Process

13. Source Control Management : GIT

14. Continuous Integration: Buildbot

15. Code Inspection:
Gitlab

16. Monitoring System: PandoraFMS
http://www.lafermeduweb.net

17. Deployment System
Makefile
increment version
pre-build test
build distribution
staging
backup
updating

18. Logging
Levels:
DEBUG
file A (rotated)
INFO
WARNING
ERROR
file B (rotated)
file C (rotated)
email to maintainers
includes technical
information

19. Bahasa
pemrograman
• Python - Backend services.
• Scala - Stream processing.
• Node.js - Realtime notifier.
• PHP - Front end.

20. MID ENDS
FRONT ENDS
Go
Anscdn & AWS
Cache
BACK ENDS
Database
Replica
Closure
Library
Sock API
•3 instances
Search
Engine
•Includes
Database
Lucene Engine
round-robin by
hproxy
Ndayak 3 Engine
Notifier
•Web
Notification
long polling
web API
Cache
•Authentication
•API
Hadoop
Data mining storage
•Mobile Push
Notification
Message Queueing
•Purpose to:
• Generate analytics.
• Calculate search relevancy.
• Calculate suggestion like
PYMK or GYML.

21. Geek Wanted
• Java / Scala Programmer.
• NoSQL Database Engineer.
• System Administrator.
kirimkan CV ke:
job@mindtalk.com

22. Terima kasih