ODSC West TidalScale Keynote Slides
•Download as PPTX, PDF•
0 likes•546 views
This document discusses software-defined servers as a game-changing approach for data scientists. Traditional approaches to scaling workloads like machine learning are inefficient due to limitations of sharding data across servers and the inability to access all data. Software-defined servers can span physical servers and be scaled up and down flexibly using standard hardware. A technology called TidalScale creates a single large virtual machine across multiple servers with its HyperKernel software, enabling in-memory performance at large scale for analytics workloads. Benchmarks show TidalScale outperforming other solutions for retail analytics and machine learning workloads on public healthcare data. Experts agree this represents the future of server architecture.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to ODSC West TidalScale Keynote Slides
Similar to ODSC West TidalScale Keynote Slides (20)
Recently uploaded
Recently uploaded (20)
ODSC West TidalScale Keynote Slides
- 1. Software-Defined Servers A Game Changer for Data Scientists Gary Smerdon CEO TidalScale, Inc.
- 2. NVMe Flash 150 μs $150,000 Flash Array 1 ms $1,500,000 Hard Drive 5 ms $7,500,000 TCP packet retransmit 2 s $2,000,000,000 The Problem In-Memory Computing is Key Operation Processing Latency In Human Terms L1-L3 Cache 1-13 ns $1 DRAM 50 ns $50 Variety & Volume: Data growing at 62% CAGR GB TB PB EB ZB DataVolumes Time Velocity: Data value declines with age BusinessValue Age of Data (seconds)
- 3. Current Approaches - Lower prediction accuracy Algorithm - Lower prediction accuracy Sub Sample Shard 4 Shard 3 Shard 2 Shard 1 - Lower prediction accuracy - Time & Money
- 4. Single Huge Server Easy to discover relationships Hard to discover relationships Sharded Department A Department B Department C Department D Department E Department F Department G Product A Product B Product C Product D Seeing All the Data Uncovers Relationships
- 5. Recent Data is more Predictive 2005-2007 mortgage data would have predicted the 2008 mortgage crisis…but analysts used data only from 2004 Five-year Modeled Default Frequency Rate by Deal Vintage Year Year Five-yearCumulativeDefaultFrequencyRate 2002 2003 2004 2005 2006 2007 0% 5% 10% 15% 20% 25% 30% 35% Actual Default Frequency Rate Model with data through 2004 Model with data through 2007
- 6. Model Accuracy Needs RAM: Decision Trees • Decision Trees model error rates decline with data size & tree depth • Learning time decreases with tree depth • More data & greater tree depth consumes more RAM Number of Observations: Prediction Error Rate by Data Set Size & Decision Tree Depth
- 7. What If Servers were Software-Defined? • In-memory performance at scale • As many cores as needed • Self optimizing • Everything just works • Uses standard hardware Software-Defined Servers
- 8. Traditional Virtualization VirtualPhysical Multiple virtual machines share a single physical server Virtual Machine Virtual Machine Virtual Machine Application Operating System 100%, bit-for-bit unmodified Application Operating System Application Operating System
- 9. Single virtual machine spans multiple physical servers TidalScale: Software-Defined Servers Application Operating System … HyperKernel HyperKernel HyperKernel TidalScale Virtual Machine 100%, bit-for-bit unmodified
- 10. HyperKernel … HyperKernel HyperKernel HyperKernel HyperKernel Application Operating System TidalScale Software-Defined Server Flexible – Scales Up or Down Quickly Seamless Scalability
- 11. HyperKernel … HyperKernel HyperKernel HyperKernel HyperKernel Uses patented machine learning to transparently align resources Application Operating System TidalScale Software-Defined Server Machine Learning Driven Self-Optimization
- 12. Applications Operating Systems Virtual Machine If it runs, it runs on a TidalScale Software-Defined Server TidalScale Software-Defined Server HyperKernel … HyperKernel HyperKernel HyperKernel HyperKernel 100% Compatible Containers
- 13. Use Case: Retail Analytics on TidalScale Performance Comparisons (TPC-H “Powertest” in Minutes) Workload Size in GB MinutestoProcess 100 0 10 20 30 40 50 70 100 Amazon EC2 0 200 300 400 500 600 700 800 900 1,000 60 80 90 69.1 TEST FAILS TidalScale Software-Defined Server 22.0 33.7
- 14. Benchmark: Open Source R on TidalScale • Version: Revolution R Open 8.0.3 with pryr, dplyr, mgcv, rpart, randomForest, FNN, Matrix, doparallel & foreach • Data: CMS Public Use Dataset • In-memory footprints: 32GB-680GB • Operations timed: • Load • Join • GAM linear regression • GLM linear regression • Decision Tree • Random Forest (fixed seed) • K Nearest Neighbors Open Source R Performance Comparisons TotalExecutionTime(Minutes) 100 0 100,000 200,000 400,000 700,000 - 200 300 400 500 600 300,000 500,000 600,000 Bare Metal Server (128GB) 158 days TidalScale Software-Defined Server (5 x 128GB nodes) 1,325 3,787 https://github.com/TidalScale/R_benchmark_test Workload Size in GB
- 15. Tomorrow’s Servers Today: A Game-Changer “Software-defined Servers make it easy to run memory-intensive applications like data mining, machine learning and simulation.” Marc Jones, Director & Distinguished Engineer, IBM
- 16. “This is the way all servers will be built in the future.” Gordon Bell Industry legend & 1st outside investor in TidalScale
Editor's Notes
- Slide 1: Software-Defined Servers: A Game Changer for Data Scientists | Hi I'm Gary Smerdon, CEO of TidalScale. It's my pleasure to be here today. "Game Changer" is a powerful word conjuring the likes of the iPhone and the Internet. But Software-Defined Servers are a game changer for Data Scientists and I'm here today to share why.
- Everyone here is intimately familiar with the problem. Data *Volume* and *Variety* is growing explosively at 62% a year. Simultaneously, the *Velocity* of data change is increasing rapidly such that the business value of that data decreases exponentially by its age in seconds. Getting a handle on this requires that computing be done *in-memory* because the alternative, paging from storage, is so incredibly slow. In dollar terms its the difference between buying your weekly groceries for $50 versus for $150,000. Such outsize costs would have a big negative impact on your annual food budget! We call this latency difference the *Memory Cliff* and it has a similar negative impact on application performance.
- You are all familiar with the solutions available for managing large data problems today. 1. Take a sample of the data, 2. Use a clever algorithm that makes some compressed representation of the data space, or 3. Shard the data set to fit in the memory of multiple small servers. All of these approaches have the disadvantages of lowering prediction accuracy, requiring time and effort and generally just getting in the way of doing data science. It's like having to do the plumbing when you just want to cook dinner.
- There are many sides to the data analysis picture. One thing we know is that having complete data is often critical to data analysis. A famous example of this is the Challenger disaster where the management chain filtered out the complete data picture from upper management. The filtered view seems to indicate that the odds of a problem occurring were low. The complete data set view shows absolutely otherwise.
- More data allows you to see relationships not otherwise visible. A customer in the retail space struggles today with seeing relationships between products in different store departments because the data has been silo-ed across multiple physical machines. If they could have just a single large system it would be trivial to explore and discover these relationships.
- Timely data absolutely influences prediction accuracy. A famous example is the underestimation of mortgage default rates leading up to the 2008 financial crisis. The model was fine but the mortgage industry had been so stagnant for so long that the 2005-2007 predictions were made off 2004 data and thus missed the increasing failure rates. As you can see here, the same model updated with the real 2005-2007 data accurately predicts the actual failure rates seen.
- More data with finer granularity absolutely improves prediction accuracy. The more detail you can assign to a particular outcome, the more reliably you can predict. Larger data sets take more RAM, of course, but did you know that finer granularity features also require RAM? For a given data set, adding tree levels to a Random Forest algorithm explodes its RAM requirements.
- What if the servers you used for data science were themselves software-defined? You could simply do your data science cookery without having to worry about the plumbing! To continue the thought experiment, imagine these servers could be assembled from ordinary servers, tuned themselves automagically, delivered in-memory performance at any scale and didn't require any change to software applications or operating systems. If such a software-defined server existed, it would deliver a missing piece in today's data centers where virtually _every_ other component _is_ software-defined. Today I am happy to share with you that such a technology _does_ exist: *TidalScale*. But to explain it I have to back up a bit and explain Traditional Virtualization.
- Traditional virtualization effectively divvies-up a server into multiple virtual servers. The key fact is that the virtual servers have _no idea_ they aren't actually running on hardware. The applications and operating system is unmodified.
- TidalScale uses virtualization but does it _across_ multiple physical servers, effectively allowing you to treat physical servers like lego building blocks.
- Want a bigger software-defined server? Add more hardware systems underneath.
- Under the covers, TidalScale uses machine learning to automatically co-locate compute entity with the resources it needs. And just like ordinary virtualization it does all of this transparently - the OS and applications have no idea they aren't running on hardware....
- TidalScale can run virtually everything unchanged, and we deliver in-memory performance, for example...
- That retail analytic customer that I mentioned earlier: We were able to demonstrate scalable in-memory performance running TPC-C Power test on their analytics database. Compare that to AWS EC2 where the same workload dies when it hits the memory cliff.
- To document our scalability we wrote and published a benchmark on Open Source R. It measures the time to execute load, join and then the 5 most commonly used data science algorithms on CMS insurance claim data. We ran this benchmakr on a bare metal 128GB server and then on a TidalScale TidalPod composed of five 128GB servers. As you can see on this chart, the Software-defined Server's performance scales linearly.
- So how would it change the way you tackle data science problems to have the ability to deploy the hardware systems of tomorrow today? An example of the kind of systems we can create is the 15TB 400 core TidalPod we deployed at SoftLayer this past June using 20 standard servers. Marc Jones captured it best: “Software-defined Servers make it easy to run memory-intensive applications like data mining, machine learning and simulation.”
- TidalScale Software-Defined Servers are a game changer! They let you explore your data more easily, improve your results and allow you spend more time doing data science instead of data plumbing all while lowering your operational TCO and increasing IT flexibility. This is why one of our early investor's said it best: "This is the way all servers will be built in the future."