Modern problems in backend engineering, Marten Meikop
1.
About me
● Hobbies
○ Mindsports (chess, checkers)
○ Programming (vint.ee - 30 online
games)
● Wrote first line of code in 1999
● Professional career:
○ C coding for Bang & Olufsen
○ Software architect in Telia
○ Site lead in Proekspert
○ Startup founder
(atmosphere.ee)
2.
90k
Customers in
170 countries
600+
Employees
8
Offices
Pipedrive - The best sales CRM
2010
Founded
$90m
Total Venture Capital
Funding
3.
70+
Live deployments
every day
2
Data centers
1 billion
Requests
per week
Pipedrive Engineering in Numbers
250
People in
Engineering
250
Back-end
Services
1
Management
region
4.
The Beginning (2010)
PHP framework
CodeIgniter
Percona Server
for MySQL
Schema per customer
database architecture
LEMP Stack
5.
Natural Growth (2015)
Webapp
Websocket
Microservices
Microservice
DatabasePipedrive DB
Data Center in US
Company DB
6.
Megaparsec: Moving to
Multi-datacenter environment (2016-2018)
The Needs Challenges
Serving end users from physically
closer location
Legal aspects (GDPR)
Not to put all eggs in one basket
Running out of physical space - one DC
might grow too big
Gateway
Service Discovery
Backoffice / management
CI/CD multi-dc support
Monitoring
1 parsec = 3.0857x1016m
7.
Customer data in one DC
PipedriveDB in multi-master sync between datacenters
Universal domain name (pipedrive.com, not
eu.pipedrive.com)
Geolocation routing
Multi-DC architecture: Key concepts
Datacenter 1 Datacenter 2
Gateway Gateway
8.
Our own custom public gateway
Written in NodeJS
User authentication
(Crypto cookie, api-token, Oauth)
Rate limiting
Service discovery & load balancing
Enriching headers
Producing full access log
Routing between Datacenters
9.
Barista
eu-central-1
F5
BaristaIdentity
Microservice A
us-east-1
F5
BaristaIdentity
Microservice B