1. 1 hbasecon.com

2. HBASEATBLOOMBERG//
THE EVOLUTION
OF BLOOMBERG
DATA SYSTEMSMEDIUM DATA NEEDS FOR THE FINANCIAL INDUSTRY
MAY // 07 // 2015

3. HBASEATBLOOMBERG//
BLOOMBERG
3
Leading Data and Analytics provider to the financial industry

4. HBASEATBLOOMBERG//
DATA IS OUR BUSINESS
4

5. HBASEATBLOOMBERG//
September 28: Full Workshop at Bloomberg
September 30: Showcase at Strata Hadoop
Call for papers at:
bloomberglabs.com/data-science
DATA SCIENCE
FOR SOCIAL GOOD:
GOVERNMENT INNOVATION,
PUBLIC HEALTH, ENVIRONMENT,
EDUCATION

6. HBASEATBLOOMBERG//
6
• We have a “medium data” problem…
• Speed and availability are paramount
• Hundreds of thousands of users with
expensive requests
We’ve built many systems
to address
DATA MANAGEMENT TODAY

7. HBASEATBLOOMBERG//
DATA MANAGEMENT CHALLENGES
7
• Single security
analytics on Big Iron
• Replication of
Systems and Data
• Complexity kills
Top 500 Supercomputer list, 2013
>96% Linux. 100% of top 40.

8. HBASEATBLOOMBERG//
DATA MANAGEMENT TOMORROW
8
• Simplicity and
performance
• Benefit from external
developments
• Retain our
independence
• Details matter

9. HBASEATBLOOMBERG//
THE PREMISE
9
• Can apply big data techniques to our medium
data problem, by addressing gaps in existing
open systems
• HBase is a good bet
• Part of a broader whole
• The Biggest community wins

10. HBASEATBLOOMBERG//
CHALLENGES
Our requirements from HBase are:
• Read performance – fast with low variability
• High availability
• Operational simplicity
• Efficient use of good hardware
• Expressive power
Bloomberg has been investing in all these
aspects of HBase

11. HBASEATBLOOMBERG//
WE’VE MADE THAT BET
11

12. HBASEATBLOOMBERG//
WE’RE NOT THE ONLY ONES
12
Google Cloud Bigtable

13. HBASEATBLOOMBERG//
AIMING HIGHER
We can make things better
by working together
Let’s be the gold standard

14. HBASEATBLOOMBERG//
14

15. >>>>>>>>>>>>>>
CALL TO ACTION

16. HBASEATBLOOMBERG//
FURTHER BOLSTER RELIABILITY
16
Great strides such as HBASE-10070 but more to do
• Improved reconciliation of
state between Master,
META and ZK
• More determinism in
Admin/Master operations

17. HBASEATBLOOMBERG//
BENEFIT FROM MODERN HARDWARE
17
• 32 cores - 256GB RAM – SSD - untapped potential
• CPU load max 20% , inadequate throughput
• Multi-RS administratively painful
• Much better story with memory

18. HBASEATBLOOMBERG//
IMPROVE MULTI-TENANCY
18
• Mixed workloads challenging
• interactive vs batch
• read vs write
• different read access
patterns
• Many solutions in progress
• Administrative simplicity is key

19. HBASEATBLOOMBERG//
SPARK INTEGRATION
19
• Analytical frameworks need a distributed database
• Columnar file format != column database
• Integrate with HBase to move towards the
universal database

20. HBASEATBLOOMBERG//
ANALYTICS: EFFICIENCY
20
• Choice of row and columnar storage engines
• Expose primitives for efficiency:
• Column pruning
• Predicate pushdowns
• Data locality

21. HBASEATBLOOMBERG//
THE FUTURE IS BRIGHT
21
• The state of the “Hadoop Database” union is strong
– Increasing adoption
– Strong foundation
– Great community
• Prominent role in the data & analytics platform of
the future
• Let’s go create the future

22. >>>>>>>>>>>>>>
THANK YOU

23. 23 hbasecon.com