The document discusses optimizing the system for a company called Ecomobi that has 30 million users. It outlines some of the main issues like high availability, response time, and bandwidth. It then discusses the solutions implemented, including using measurement tools like New Relic, selecting reliable service providers, programming languages like PHP and Golang, databases like MySQL and MongoDB, caching, asynchronous processing using message queues, scaling both the web and database layers, distributing services, and using CDN. Profiling was also used to analyze performance.

1. Tối ưu hóa hệ thống
30 triệu người dùng
IT Expert Club

2. CONTENT
1. Introduction
2. Solution
3. Caching
4. Asynchronous
5. Scale
6. Distributed
7. Profiling and Turning

3. 1. Introduction
CTO of Ecomobi
Contact info:
• EMAIL: TUYENDV@ECOMOBI.COM
• MOBILE: (+84)904.554.112
• SKYPE: ITBACHKHOA

4. 1. Introduction
Ecomobi is a CPA Network focus on SEA Market.

5. Main Flow

6. Daily Information
Affiliate Networks
• 1.5M+ DAU
• 2-3M CLICK
• 1TB TRANSFER
• USER: VN, ID, IN, BR...
Ads Networks
• 30M+ USERS
• 1M+ DAU
• 40M+ ADS REQUEST
• 70K+ RPM
• USERS: VN, ID, IN, BR...

7. Our Issues
• High Availability
• Minimize Response time
• Concurrency
• High Bandwidth
• International user

8. 2. Our Solutions
• Measurement tools
• Select Services Provider
• Programming language
• Database
• Others

9. 2. Measurement
• Analytics
• Quick or Slow
• Error or Accuracy
• Stable or Unstable
• Better or Worse

10. 2. Measurement Tools - New
Relic

11. 2. Measurement Tools - New Relic

12. Services Provider?
• Dedicated Server in Viet nam
• Private cloud
• Docker
• VPS Provider in Vietnamese
• Amazon Web Service
• Digital Ocean / Linode
• VPS in Local Provider

13. Programming language?
• PHP
• JAVA
• Erlang / Elixir
• Node JS / Golang

14. Database
• MySQL / MariaDB
• PostgreSQL
• MongoDB
• Cassandra / couchbase / Riak
• Redis / Memcache
• Elasticsearch

15. Message Queue
• RabbitMQ
• Redis
• Activemq / Zeromq
• Kafka
• Gearman (Job Server)

16. Other
• Caching
• Asynchronous
• Scale out
• Distributed

17. Our Stack

18. 3.Caching
• In-memory / variable caching
• Cache size => minimize (why)
• Increasing Cache Hit / Cache Miss
• Cache Read through issue

19. 3.Caching: memory
function getInfoWithCache($id) {
static $cache_contents = array();
if (isset($cache_contents[$id])) {
return $cache_contents[$id];
}
$data = getInfoWithoutCache($id)
$cache_contents[$id] = $data;
return $data;
}

20. Caching - Minimize cache size

21. 3.Caching - increasing cache hit
$candidates = Campaign for
• Active campaigns
• Allow Android 4.0.1
• Allow Vietnam
• Not gambling
• CPI or CPC
• IF CPC: PRICE > $0.03
• IF CPI: PRICE > $0.5
• Did not install this game / app
…
Candidates = Campaign for
• Active Campaigns
• Allow Vietnam
Candidates = Filter(Candidates)
• Allow Android 4.0.1
• Not gambling
• CPC or CPC
…

22. Cache read through issue!!!
function getData($id)
{
$data = getDataFromCache($id);
if (!empty($data) {
return $data;
}
$data = getDataWithoutCache($id);
setDataToCache($id, $data, $exprire_time);
return $data;
}

23. 4. Asynchronous
• Send a external HTTP request
• Send a email
• Working with file
• Working with DB
• …

24. 4. Asynchronous
• Decreasing Response time
• Deal with huge of request

25. 4. Asynchronous - MQ

26. 4. Asynchronous - MQ
while (true) {
$msg = getMessageQueue();
if (empty($msg)) {
sleep(1);
continue;
}
processMQ($msg);
if ($need_stop) {
break;
}
}

27. 5.1 Web Application Scale

28. 5.2 Database layer

29. 6. Distributed

30. 6. Distributed

31. 6. CDN

32. 7. Profiling