Downloaded 87 times
Uploaded byCloudera, Inc.
HBaseCon 2013: General Session
HBaseCon 2013 aimed to bring the Apache HBase community together, encouraging contributions and sharing solutions amongst participants. The conference highlighted the growth and health of the HBase ecosystem, featuring sessions from various companies while emphasizing the importance of community engagement. Key discussions included innovations in HBase's architecture and tools, and the ecosystem's evolving role in big data applications.
More Related Content
Similar to HBaseCon 2013: General Session
More from Cloudera, Inc.
HBaseCon 2013: General Session
- 2. Hosted by Welcome Michael Stack,Software Engineer, Cloudera & HBase PMC Chair
- 3. Goals of HBaseCon2013 Bring the Apache HBase community together Encourage contributions to the HBase ecosystem Share challenges and solutions for HBase 1 2 3
- 4. HBaseCon 2013 SessionTracks Operations Internals Ecosystem Case Studies Session Track 3Track 1 Track 2 Track 3 Track 4
- 5. HBaseCon 2013 ProgramCommittee Gary Helmling Lars Hofhansl Jonathan Hsieh Doug Meil Andrew Purtell Enis Söztutar Michael Stack – Chair Liyin Tang Architect Engineer Software Engineer Chief Software Architect Systems Architect Member of Technical Staff Software Engineer Software Engineer
- 6. Thank You toOur Sponsors Community Sponsor Conference Sponsors Media Sponsors
- 7. Visit Sponsors =Chance to Win =
- 8. Conference Notes • Pleasefill out the overall conference survey • Reception is 5:40pm – 8:00pm in the Yerba Buena Foyer • Connecting to the internet • Wireless network = Marriott Conference • Passcode = db075b
- 9. Hosted by The ApacheHBase Community: Best Ever and Getting Better Amr Awadallah, CTO and Co-founder, Cloudera @awadallah
- 10. The Apache HBaseCommunity Has Never Been Healthier JIRA ActivityCommits Activity
- 11. The Market forHBase Skills is Bigger than Ever
- 12. The HBase Ecosystemis Rich and Expanding HBaseCon 2013 speakers from these companies this year (logos below the dotted line are net-new from 2012!)
- 13. Top 5 ReasonsCloudera Loves HBase Its vibrant community is a benchmark for the entire Apache Hadoop ecosystem. It’s a first-class citizen inside the Hadoop stack. It allows us to offer support services for which a lot of customers will pay good money. It draws top-drawer engineer talent to Cloudera. It gives us an excuse to host this tremendous conference and throw a big party for the community! 1 2 3 4 5
- 14. Hosted by Thank Youfor Attending HBaseCon and Thank You for Contributing!
- 16. Hosted by State ofthe Apache HBase Union Michael Stack, Software Engineer, Cloudera & HBase PMC Chair and Lars Hofhansl, Architect, Salesforce.com
- 17. We are yourRelease Managers! • Mr. (0.94.x) Lars Hofhansl • Michael Stack (0.95.x/0.96.x)
- 18.
- 19.
- 20.
- 21. MVP …and the awardgoes to…
- 22.
- 23. Deploys • Multitenant multifariousfeature store • a.k.a dumping ground • Stumbleupon, Y!, Salesforce • Reconciliation store • eBay • Timeseries • OpenTSDB, Salesforce, FB ODS • Lots-o-entities store • Flurry, Kiji, Genome • Lots-o-entities BLOBs, FB Messages
- 24.
- 27.
- 28. # of Commits TotalFiles 2021 Total Lines of Code 832122 Total Commits 6615 Authors 39
- 29.
- 30.
- 31. JIRA: Opened vsClosed
- 32. New Committers (byFirst Commit) vs. Active Committers (One Commit/Month)
- 33. Commits/Month over Time(0.94/trunk)
- 35. • 419 jirasin 0.94.0 • 660 jiras in 0.94.1 – 0.94.8
- 36. • Frequent, smallreleases • Train model • 4–6 week cycles
- 37. • Wire compatible betweenreleases • Upgrade possible to any point release • Test stability
- 38. • Focus on: •Performance (FB, Salesforce) • Stability
- 39.
- 40.
- 41.
- 42.
- 44. • Hadoop HDFSFixes • Faster Recovery • Detection • Replay • Assign
- 47. • System tables •Filesystem • Up in zookeeper • Over the wire
- 48. RPC • Implements protobufservice • Specification! • Data on the side • Encoding • Compression PB DATA
- 49. Snapshots • By table •Snapshot, clone, restore, export • Inexpensive • Just metadata • Good for... • ackups • Replication • Offline processing
- 50. Compactions • Pluggable • Tiered •Striped • Trigger
- 51. Tests • Cluster testmodule • Standalone or cluster • Sizeable • x data • x runtime • “Borrows” test types from all over • Netflix “ChaosMonkey” • Apache Accumulo linked-list dataloss checker
- 52. Miscellaneous • Smarter loadbalancer • Revamped • Metrics • UI • Etc. • Hadoop 1.x and 2.x
- 53.
- 54.
- 56. kiji.org • Entity-centric, simplemodel • Types, complex, compound types • Each cell is schema versioned • Works across MR & REST, etc. • Production users • Open-source
- 59. SQL • “A SQLskin over HBase” • Coprocessors, custom filters, jdbc driver • https://github.com/forcedotcom/phoenix Phoenix
- 61.
- 62. Related: QoS Next • Latencyresilience/”Latency tolerance”* • Bring home the outliers * http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/people/jeff/Berkeley-Latency-Mar2012.pdf
- 63.
- 64. Next • More databasey!!! •Statistics • 2ndary indexing • Take 3 • Types • Serialization • Keep sort order
- 65.
- 67. Hosted by The ApacheHBase Ecosystem Aaron Kimball, Chief Architect, WibiData
- 68.
- 69. In the beginning… Therewas a search engine … and when building indexes got too hard Along came an elephant to help push:
- 70. The ASF isan ecosystem
- 71. And so isHadoop
- 72. HBase: The newecosystem Phoenix
- 73.
- 74. Big Data Applications Customer RelationsMobileWeb Applications Hadoop & HBase Storage Analytics Serving Real-time model scoring Investigative analytics
- 75.
- 76. Big Data Appsare hard to build • Serialization & versioning • Deployment • Communication between teams • Front end, back end, short request, batch, real time… Every Java developer should be able to build Big Data Apps – today it’s too hard.
- 77. Kiji Kiji is designedto help you build real-time Big Data Applications on Apache HBase + + 100% Apache 2 licensed
- 78.
- 79. Leading design decisions •Store your data in HBase • Encode it using Avro • An entity-centric table design • Manage a data dictionary around tables • Distribute writes across the cluster
- 80. Key features • Workwith big data in rich types with schema evolution • Guides users to successful application design • Scala-based modeling language • Integration with front-end systems • Deployment of real-time model scoring
- 81. Kiji • Go tokiji.org and download the BentoBox – Zero-config Hadoop + HBase + Kiji instance – “Batteries included” • 15-minute quickstart guide and a tutorial with full source code
- 82. Come attend ! Wanta deep dive on Kiji? KijiCon is tomorrow! A 1-day workshop of tutorials & hacking Register @ kijicon.eventbrite.com
- 83. Conclusions • Each monthshows new peak interest in HBase • The ecosystem is growing • Open source technologists are working hand in hand to make HBase more accessible • We’d love your help in the community!
- 84. aaron@wibidata.com Build big dataapplications.