A Tour of Internal Accumulo Testing
•Download as PPT, PDF•
1 like•1,707 views
Accumulo includes a remarkable breadth of testing frameworks, which helps to ensure its correctness, performance, robustness, and protection of your vital data. This presentation takes you on a tour from Accumulo's basic unit testing up through performance and scalability testing exercised on running clusters. Learn the extent to which Accumulo is put through its paces before it is released, and get ideas for how you can similarly enhance testing of your own code. Find this talk and others at http://www.slideshare.net/AccumuloSummit.
More Related Content
What's hot
What's hot (20)
Similar to A Tour of Internal Accumulo Testing
Similar to A Tour of Internal Accumulo Testing (20)
Recently uploaded
Recently uploaded (20)
A Tour of Internal Accumulo Testing
- 1. 11 A Tour of Internal Apache Accumulo Testing Bill Havanki Solutions Architect, Cloudera Government Solutions
- 2. 2 ©2014 Cloudera, Inc. All rights reserved. I’ll be your skipper for as far as we get 2 CC BY-ND 2.0 Loren Javier
- 3. 3 ©2014 Cloudera, Inc. All rights reserved. Itinerary • unit, functional, and integration • static analysis • continuous and randomwalk • memory stress and failures • upgrade • scalability and performance 3 CC BY-ND 2.0 Loren Javier
- 4. 4 Basic Tests Don’t worry if it’s crowded now ... there’ll be lots of room after these first few slides 4
- 5. 5 ©2014 Cloudera, Inc. All rights reserved. Unit testing • just over 1000 unit tests • JUnit with Easymock • can run under Jenkins • less focus here than long- running tests 5 CC BY-ND 2.0 Loren Javier
- 6. 6 ©2014 Cloudera, Inc. All rights reserved. Functional testing (through 1.5.x) • 72 or so • implemented in Java with Python wrappers • need HDFS / ZK running • either Accumulo installed or built • can run under MR 6 CC BY-ND 2.0 Loren Javier again
- 7. 7 ©2014 Cloudera, Inc. All rights reserved. Integration testing (1.6.0+) • just over 100 • pure Java • run under Maven • can run under Jenkins • uses MiniAccumuloCluster 7 CC BY-ND 2.0 guess who? Loren Javier
- 8. 8 ©2014 Cloudera, Inc. All rights reserved. Static analysis • Findbugs • PMD and CPD • Eclipse • Coverity 8 CC BY-ND 2.0 still Loren Javier
- 9. 9 Small Cluster Tests The python is one of the jungle’s most fascinating creatures. Look at how many animals get wrapped up in the subject 9
- 10. 10 ©2014 Cloudera, Inc. All rights reserved. Continuous ingest • two phases • ingest • verify • run for 24 or 72 hours 10 CC BY-ND 2.0 oh hello there, Loren Javier
- 11. 11 ©2014 Cloudera, Inc. All rights reserved. Continuous ingest other processes during ingest • walkers • scanners • agitators (≈ Chaos Monkey) 11 CC BY-ND 2.0 that’s about enough, Loren Javier
- 12. 12 ©2014 Cloudera, Inc. All rights reserved. Joke time Knock knock. I said, KNOCK KNOCK. Safari. Safari, so good. You haven’t fallen asleep yet. 12
- 13. 13 ©2014 Cloudera, Inc. All rights reserved. Randomwalk • traversal of graph of test actions • 8 fundamental graphs • several more combining • can be limited by hops or run time • usually run for 24 hours 13 CC BY-ND 2.0 look at me, I’m Loren Javier, I take all the best pictures
- 14. 14 ©2014 Cloudera, Inc. All rights reserved. Memory stress • two running parts • reader • writer • configure for large keys or values (e.g., > 1 MB) • run for about an hour 14 CC BY 2.0 Justin Ennis
- 15. 15 ©2014 Cloudera, Inc. All rights reserved. Failures • namenode failover • datanode failure • tablet server failure • ZooKeeper failure • loss of tablet server node • loss of master node 15 CC BY-ND 2.0 Loren Javier, back in the saddle
- 16. 16 ©2014 Cloudera, Inc. All rights reserved. Upgrades less formal, run manually Cloudera has a couple tests here • data compatibility test • ACL compatibility test 16 CC BY-ND 2.0 Loren “likes the ride a bit too much” Javier
- 17. 17 Large Cluster Tests Look at all the elephants! Go ahead and take pictures, they all have their trunks on 17
- 18. 18 ©2014 Cloudera, Inc. All rights reserved. Scalability • ingest millions of entries • run over tens of nodes (up to 80) • watch performance, esp. across versions 18 CC BY-ND 2.0 Loren Javier, shutterbug extraordinaire
- 19. 19 ©2014 Cloudera, Inc. All rights reserved. Joke time Why did the elephant quit her testing job? • She was tired of working for peanuts. • She had had it up to ear. • She herd about a better job. • There were just too many wrinkles. • Her desire to do it was stamped out. 19
- 20. 20 ©2014 Cloudera, Inc. All rights reserved. Performance - Benchmarks 8 benchmark implementations, e.g.: • lots of small records • create, delete many tables • insert one terabyte of data 20 CC BY-ND 2.0 Loren ... Javier
- 21. 21 ©2014 Cloudera, Inc. All rights reserved. Performance - Yahoo! Cloud Serving Benchmark • Accumulo binding • workloads with 2 phases • load data • run test • customizable record count, field length, etc. • multiple clients 21 CC BY-ND 2.0 Nancy Nally
- 22. 22 Returning to the Summit Make sure you have all your personal belongings with you ... smartphones, small children ... children left behind will be forced to write unit tests 22
- 23. 23 ©2014 Cloudera, Inc. All rights reserved. Still more to do • more unit and integration test coverage (≈30%) • attention to static analysis • chasing performance regressions • easier to run 23 CC BY-ND 2.0 nice work, Loren Javier
- 24. 24 ©2014 Cloudera, Inc. All rights reserved. Thanks • Sean Busbey & Mike Drob • Accumulo committers and community • fellow Clouderans • Loren Javier • my audience 24 CC BY 2.0 Justin Ennis
- 25. 25 ©2014 Cloudera, Inc. All rights reserved. Visit our booth! 25 CC BY-ND 2.0 Loren Javier, head photographer
- 26. 26 ©2014 Cloudera, Inc. All rights reserved. The most dangerous part of our journey 26 CC BY-ND 2.0 Loren Javier
- 27. 27 ©2014 Cloudera, Inc. All rights reserved. 27 Please exit the boat Watch your step and mind your head. If you hit your head, watch your language
Editor's Notes
- Feel free to gather together, it makes the heating system in the conference center work better. They say you can always judge the quality of a talk by the line to get in. ... My point, exactly.
- I was going to think of some barrel jokes, but I never got around to it. If I had, they would have tested your mettle.
- Anyone know what kind of snake that is in the picture? ... Starts with a P? ... No, it’s a plastic snake. Pythons make bad testers. They find the work too constricting. They also only get to work for scale.
- By renaming the functional tests, some people think we took the fun out of them. I don’t know, I think they turned inta great stuff. If anyone is wondering, those natives are from the Britney tribe. You can tell by the spears.
- Apes also make bad testers, they’re always monkeying around. Drives their managers bananas.
- Continuous tests ... I could go on and on about them. And I wouldn’t be discreet. You always need to double-check the test reports from one of these big cats, because they could always be a lyin’. I would have expected they’d take more pride in their work. That’s the mane thing.
- These monkeys make better testers than the apes. They really get into the swing of it.
- Failures are an unavoidable fact of life when working with Accumulo clusters ... or attending Accumulo talks. I’m really only giving you a quick crash course on failure testing here. I hope that’s plane to everyone.
- Crocodiles aren’t so bad as testers. They always have smiles on their faces, but they tend to get snappy.
- Being benchmarks, these tests sometimes don’t sit well with people. They cushion their criticism, though. Me, I think that’s just lazy, boy.
- Also documentation, especially Javadoc.
- This is Sam, our head recruiter for this area. Although he does complain his candidate pool has been shrinking.
- We are entering the most dangerous part of our journey ... crowds in the hall / the lunch rush / driving over DC-area roads. Thanks for riding along and have a great day. Or, as Christopher Walken would say, (as Walken). Or, as Charlie Brown’s teacher would say, “wa wa wa-wa wa”. If you enjoyed this talk, my name is Bill and I’m from Cloudera. If you didn’t, my name is Larry and I’m from Oracle.
- As you leave, please don’t step on anyone’s feet. You’ll feel like a heel, and we’ll have to call a toe truck. Then you’d have to foot the bill. Don’t go out the window, it’s pane-ful. I hope that’s clear. Of all the audiences I’ve had, you’re the most ... recent.