Wallaroo Labs Presentation
•
0 likes•358 views
Scale independent, disappearing infrastructure, code to scale, stateful algorithms, rapid deployment, data processing, Big Data meets Serverless, Python
Recommended
More Related Content
Similar to Wallaroo Labs Presentation
Similar to Wallaroo Labs Presentation (20)
Recently uploaded
Recently uploaded (20)
Wallaroo Labs Presentation
- 1. WallarooLABS Wallaroo – the streaming data framework that startups are adopting Vid Jain, CEO & Founder vid@wallaroolabs.com http://WallarooLabs.com
- 2. Purpose built team with experience in streaming data, big data, and fast machine algorithms. NYC-based startup WALLAROO Vid Jain, Founder & CEO Co-founded Adtech startup, drove Merrill Lynch trading overhaul, 3 patents, Ph.D. UC Berkeley. Sean Allen, VP Engineering Expert in high-performance distributed and AI systems, Author of Storm Applied Chuck Blake, SVP Operations 20+ years experience in delivery of products from concept to launch. Ex. VP Premium Services, TheStreet. Dan Petrozzo Former CTO Investment Mgmt, Goldman Sachs Michi Botlo Quantbot Managing Director Pat Finn Former SVP Cisco, Financial Services & Gov’t. Jean Safar Former Chief Architect, Calypso Francesca Krihely Developer Marketing, MongoDB Luke Kanies Founder, Puppet Owen Davis Founder, NYCSeed Alexis Le-Quoc CTO Datadog John Gannon Cloud Platform at DigitalOcean Advisors
- 3. ➜ Speed & quantity to increase 10X ➜ More sources & algorithmic complexity → 50X cost ➜ Data & computing everywhere: edge, IoT, cloud, on-prem Every industry is struggling with these issues Costly, hard to productionize & scale algorithms - will get much worse over the next 5Y The challenge WALLAROO
- 4. WALLAROO Hello Wallaroo! Build and scale streaming data & event-driven AI applications as easily as writing a Python script ON-PREM. EDGE, AWS, GCP, AZURE… . . . DATA ANSWERS BUSINESS ALGORITHMS
- 5. ➜ Python and Go ➜ Dynamic scaling of processing capacity and state ➜ Industrial-grade Implement algorithms as code snippets that run at scale APIs + distributed engine WALLAROO
- 6. Wallaroo brings the benefits of serverless... “Serverless on steroids” WALLAROO WALLAROO ➜ Free up time ➜ Get-to-market fast ➜ Scale easily ➜ Real-time response ➜ Handle data bursts ➜ Reduce TCO
- 7. ... together with the benefits of big data to many more types of applications “Serverless on steroids” WALLAROO WALLAROO ➜ High-performance ➜ Portable ➜ Always on ➜ Stateful ➜ Complex workflows ➜ Resiliency
- 8. WALLAROO Medical cybersecurity – device & anomaly detection Hedge fund – parallel trading, scalable signals and ETL Voice e-commerce – scalable workflow Sentiment analysis – financial data Use-cases
- 9. WALLAROO Infra. monitoring at scale – alerts & dashboards Gaming – real-time matchmaking, toxic chat Electric Fleets – Power level monitoring and mgmt. Advertizing– bidding, click-stream analysis, fraud Payments – fraud, real-time analysis ... and many more
- 10. WALLAROO Single data pipeline with 3 steps Example – image analysis Images Result
- 11. WALLAROO Tweets Sentiment Single data pipeline with 4 steps Example – trending sentiment
- 12. WALLAROO Two data pipelines with shared state Example – trading limits Prices Orders Allow / Halt
- 13. WALLAROO Multiple data pipelines with shared state Example – communications Chat Email Issue Issue Issue Tweets Issue
- 14. WALLAROO Streaming Wordcount prototype Python code runs on a single CPU A few dozen lines of code but doesn’t scale.
- 15. WALLAROO Production-ready with Wallaroo Run at any scale and adjust capacity on-demand A few extra lines of code gets you a lot!
- 16. WALLAROO Define pipeline, implement steps in code, run at scale Streaming Wordcount at scale
- 17. WALLAROO Streaming Wordcount at scale Define data pipeline # Instantiate application ab = wallaroo.ApplicationBuilder("Word Count Application") # Name and configure data pipeline ab.new_pipeline("Split and Count", wallaroo.TCPSourceConfig(in_host, in_port, decoder)) # Parallelize data and computation ab.to_parallel(split) # Define stateful computation ab.to_state_partition(count_word, WordTotals, "word totals", partition, word_partitions) # Output results ab.to_sink(wallaroo.TCPSinkConfig(out_host, out_port, encoder))
- 18. WALLAROO Streaming Wordcount at scale Implement steps in code # Implement step to split sentences into words # Implement step to update word count state # get_count is an instance of WordCount containing a word & its count https://docs.wallaroolabs.com/book/python/word-count.html
- 19. WALLAROO Streaming Wordcount at scale 1. Run on one worker China ➜ 1,274 Congress ➜ 414 iPhone ➜ 676 Mexico ➜ 3,229 Walmart ➜ 999 Wallaroo ➜ 6,978 CNNCNNCNN CNN CNN iPHONE WALMART MEXICO
- 20. WALLAROO Streaming Wordcount at scale 2. Another worker joins with the first China ➜ 1,274 Congress ➜ 414 iPhone ➜ 676 Mexico ➜ 3,229 Walmart ➜ 999 Wallaroo ➜ 6,978 CNNCNNCNN CNN CNN iPHONE WALMART MEXICO
- 21. WALLAROO Streaming Wordcount at scale 3. State (word counts) automatically distributed China ➜ 1,384 Congress ➜ 466 CNN ➜ 253 Election ➜ 753 iPhone ➜ 753 Japan ➜ 1,729 Mexico ➜ 3,729 Oprah ➜ 899 Starbucks ➜ 474 Virginia ➜ 3,129 Walmart ➜ 1,001 Wallaroo ➜ 7,991 CNNCNNCNN iPHONE CHINA JAPAN CNN CNNVIRGINI A OPRAH WALMART MEXICO
- 22. WALLAROO Streaming Wordcount at scale 4. Keep going… China ➜ 1,440 Congress ➜ 515 CNN ➜ 753 Election ➜ 793 Facebook ➜ 353 Grain ➜ 76 iPhone ➜ 799 Japan ➜ 3,001 Mexico ➜ 3,888 Navy ➜ 1,013 Oprah ➜ 911 Starbucks ➜ 512 Trump ➜ 1,011 Under ➜ 12 Voter ➜ 431 Virginia ➜ 3,669 Walmart ➜ 1,223 Wallaroo ➜ 9,554 CNNCNN GRAIN CHINA ELECTION NAVY VIRGINI A OPRAH iPHIONE MEXICO CNNUNDERVOTER WALMART VIRGINIA
- 23. WALLAROO Streaming Wordcount at scale 5. Shrink cluster China ➜ 1,440 Congress ➜ 515 CNN ➜ 753 Election ➜ 793 Facebook ➜ 353 Grain ➜ 76 iPhone ➜ 799 Japan ➜ 3,001 Mexico ➜ 3,888 Navy ➜ 1,013 Oprah ➜ 911 Starbucks ➜ 512 Trump ➜ 1,011 Under ➜ 12 Voter ➜ 431 Virginia ➜ 3,669 Walmart ➜ 1,223 Wallaroo ➜ 9,554 CNNCNN GRAIN CHINA ELECTION NAVY VIRGINI A OPRAH iPHIONE MEXICO
- 24. Wallaroo combines the benefits of serverless and big data The future of serverless & big data WALLAROO Serverless Wallaroo Run anywhere High-performance In-memory state Complex workflow Big data Modern languages Easy-to-scale Sub-millisec response WALLAROO