6. The good
Served us faithfully from 2012-2015
SQL queries are very easy to hand-test and share
Simple data can be handled with simple queries but...
...data doesn't stay simple forever
Each product is handled in a different way
Unscalable performance as data grew
Difficult to refactor complex SQL
The bad
Ever maintained a 1,000 line SQL query?
8. Goals
• Handle all products using a common flow (ie. business trip or cleaning)
• Infer events from data or consume in realtime from production
• Easily query-able subledger output
14. Platform Events
Product Type
Product Id
Datetime
Guest Info
Host Info
Funding Sources
Itemized Pricing
Taxes (currency / amount / remittance currency)
Product Type
Product Id
Datetime
Transaction Id
Payment Info
Currency
Amount
Effective currency rate
Reconciled rate
Payment Events
15. Build in Spark / Scala
Finance system is an offline component and don’t need to worry about the
latency;
Developer can focus on the business logic and don’t need to worry about the
scalability;
Performance (throughput): 7M events / min
--conf spark.default.parallelism=200
--num-executors 50
--executor-cores 8
16. Reports
Guest Receivable Future Host Payout
SELECT sum(IF(Operation = `Debit`,
amount, -amount))
FROM subledger_entries
WHERE account = ‘ReceivableAccount’
AND meta[‘Guest’] IS NOT NULL
AND event_date < ‘2015-01-01’;
SELECT sum(IF(Operation = `credit`,
amount, -amount))
FROM subledger_entries
WHERE account = ‘PayableAccount’
AND meta[‘Host’] IS NOT NULL
AND event_date < ‘2015-01-01’;
SELECT sum(IF(Operation = `credit`, amount,
-amount))
FROM subledger_entries
WHERE account = ‘DeferredRevenueAccount’
AND event_date < ‘2015-01-01’;
Deferred Revenue
17. Migration Process
1. Generate all the platform events and payment events based on existing
database account audit records;
2. Build reports based on Braavos to match up the existing reports;
3. Changing the upstream components to generate real events and compare
with existing results;
4. Switch to use the real upstream events;
18. Intercompany report
Airbnb transactions involve four entities: Airbnb Inc. / Airbnb Payment Inc. /
Airbnb UK / Airbnb Ireland and the number is increasing.
Guest / Host may belong to the different entities and their payment entities
might be different as well.
We need to report intercompany money movement across entities.
19. Entity A Entity B
Intercompany
Inbound Outbound
Operation: Debit
Amount: $150
Operation: Credit
Amount: $200
Entity C
Operation: Credit
Amount: $120
Entity D
Operation: Debit
Amount: $170
20. Future Work
Cash Reconciliation
Tie out internal data
with processor data
Automate Treasury Rebalancing
Robo-trade currency and hedge
against market fluctuation
Automate Everything
Build out financial back-office tools to
give better insight into our business
Improve Monitoring/Alerting
We should catch data issues in
minutes, not days
Stream-Based Processing
Consume events in realtime
from our production apps
