SlideShare a Scribd company logo
Introduction to
Software-Defined Servers
Flexible – Fast - Easy
The Problem
Variety &
Volume: Data
growing at 62%
CAGR GB TB
PB
EB
ZB
DataVolumes
Time
Velocity:
Data value declines
with age
BusinessValue
Age of Data (seconds)
Data Growth is
HUGE
The Value of Data
Is Declining Rapidly
NVMe Flash 150 μs $150,000
Flash Array 1 ms $1,500,000
Hard Drive 5 ms $7,500,000
TCP packet retransmit 2 s $2,000,000,000
The Problem
In-Memory Computing is Key
Operation
Processing
Latency
In Human
Terms
L1-L3 Cache 1-13 ns $1
DRAM 50 ns $50
In-Memory Computing
Is Needed
For Performance
Current Approaches
- Lower prediction accuracy
Algorithm
- Lower prediction accuracy
Sub Sample
Do You
Sample
the
Data?
Current Approaches
Shard 4
Shard 3
Shard 2
Shard 1
- Lower prediction accuracy
- Time & Money
Or,
Do You
Shard
the
Data?
What do you do?
Not having the
right Server
adds
Complexity,
increases Risks,
and misses
Opportunities
What If Servers were Software-Defined?
Are your servers
too small?
What If Servers were Software-Defined?
What if your
Servers could
expand to fit all
of your data?
Software-Defined Servers
• Just the right size
• In-memory performance at scale
• As many cores as needed
• Self optimizing
• Everything just works
• Uses standard hardware
Software-Defined Servers
What if you could choose the
size of your servers?
Just the right size.
In-memory performance.
Self-optimizing.
Everything just works.
Using Commodity Servers.
Software-Defined Servers – the Missing Piece
Software-Defined
Datacenters
Storage
Manage
Network
Server
Software-Defined Servers are
the missing piece in the Software-
Defined Datacenter
Traditional VirtualizationVirtualPhysical
Multiple virtual machines
share a single physical server
Virtual
Machine
Virtual
Machine
Virtual
Machine
Application
Operating
System
100%,bit-for-bit
unmodified
Application
Operating
System
Application
Operating
System
Traditional
virtualization cuts
a server into
smaller virtual
servers (VM’s)
TidalScale: Software-Defined Servers
Software-Defined
Servers
are the inverse.
One, two, ten or a
full-rack of servers
operating as one!
Single virtual machine spans multiple physical servers
Application
OperatingSystem
…
HyperKernel HyperKernel HyperKernel
TidalScale
Virtual
Machine
100%,bit-for-bit
unmodified
Machine Learning Driven Self-Optimization
HyperKernel
…
HyperKernel HyperKernel HyperKernel HyperKernel
Uses patented machine learning to transparently align resources
Application
Operating System
TidalScale Software-Defined Server
And it performs.
Machine learning
keeps locality by
constantly &
transparently
optimizing.
100% Compatible
Applications
Operating Systems
Virtual Machine
If it runs, it runs on a TidalScale Software-Defined Server
TidalScale Software-Defined Server
HyperKernel
…
HyperKernel HyperKernel HyperKernel HyperKernel
Containers
Applications and
Operating Systems
just work.
No Changes.
Containers work
great, too!
Benchmark: Open Source R on TidalScale
https://github.com/TidalScale/R_benchmark_test
Open Source R Performance Comparisons
TotalExecutionTime(Minutes)
100
0
100,000
200,000
400,000
700,000
- 200 300 400 500 600
300,000
500,000
600,000
Bare Metal Server (128GB)
158 days
TidalScale Software-Defined Server (5 x 128GB nodes)
1,325 3,787
W orkload Size in GB
A performance
example. Avoid the
memory cliff and
scale the server to
match the size of the
data.
200X Faster
Tomorrow’s Servers Today: A Game-Changer
“Software-defined Servers make it easy to
run memory-intensive applications like
data mining, machine learning and
simulation.”
Marc Jones, Director &
Distinguished Engineer, IBM
Are you interesting in
having the server of
2025?
“This is the way all servers will be built in the future.”
Gordon Bell
Industry legend & 1st outside investor in TidalScale

More Related Content

What's hot

(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
Amazon Web Services
 
"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies
"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies
"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies
Dataconomy Media
 
Intro to databricks delta lake
 Intro to databricks delta lake Intro to databricks delta lake
Intro to databricks delta lake
Mykola Zerniuk
 
Battery Ventures: Simulating and Visualizing Large Scale Cassandra Deployments
Battery Ventures: Simulating and Visualizing Large Scale Cassandra DeploymentsBattery Ventures: Simulating and Visualizing Large Scale Cassandra Deployments
Battery Ventures: Simulating and Visualizing Large Scale Cassandra Deployments
DataStax Academy
 
DataStax: How to Roll Cassandra into Production Without Losing your Health, M...
DataStax: How to Roll Cassandra into Production Without Losing your Health, M...DataStax: How to Roll Cassandra into Production Without Losing your Health, M...
DataStax: How to Roll Cassandra into Production Without Losing your Health, M...
DataStax Academy
 
Introduction to SQream and the IoT environment
Introduction to SQream and the IoT environmentIntroduction to SQream and the IoT environment
Introduction to SQream and the IoT environment
Arnon Shimoni
 
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
In-Memory Computing Summit
 
Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...
Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...
Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...
DataStax Academy
 
Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...
Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...
Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...
Data Con LA
 
Achieving scale and performance using cloud native environment
Achieving scale and performance using cloud native environmentAchieving scale and performance using cloud native environment
Achieving scale and performance using cloud native environment
Rakuten Group, Inc.
 
Data Driven Decisions at Scale
Data Driven Decisions at ScaleData Driven Decisions at Scale
Data Driven Decisions at Scale
Databricks
 
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...Splunk
 
RedisConf18 - Making Real-Time Predictive Decisions with Redis
RedisConf18 - Making Real-Time Predictive Decisions with RedisRedisConf18 - Making Real-Time Predictive Decisions with Redis
RedisConf18 - Making Real-Time Predictive Decisions with Redis
Redis Labs
 
RedisConf18 - The Versatility of Redis - Powering our critical business using...
RedisConf18 - The Versatility of Redis - Powering our critical business using...RedisConf18 - The Versatility of Redis - Powering our critical business using...
RedisConf18 - The Versatility of Redis - Powering our critical business using...
Redis Labs
 
Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...
Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...
Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...
Databricks
 
Building the Next-gen Digital Meter Platform for Fluvius
Building the Next-gen Digital Meter Platform for FluviusBuilding the Next-gen Digital Meter Platform for Fluvius
Building the Next-gen Digital Meter Platform for Fluvius
Databricks
 
How jKool Analyzes Streaming Data in Real Time with DataStax
How jKool Analyzes Streaming Data in Real Time with DataStaxHow jKool Analyzes Streaming Data in Real Time with DataStax
How jKool Analyzes Streaming Data in Real Time with DataStax
DataStax
 
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and BigtopAccelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
In-Memory Computing Summit
 
Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...
Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...
Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...
Spark Summit
 

What's hot (19)

(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
(BDT201) Big Data and HPC State of the Union | AWS re:Invent 2014
 
"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies
"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies
"Democratizing Big Data", Ami Gal, CEO & Co-Founder of SQream Technologies
 
Intro to databricks delta lake
 Intro to databricks delta lake Intro to databricks delta lake
Intro to databricks delta lake
 
Battery Ventures: Simulating and Visualizing Large Scale Cassandra Deployments
Battery Ventures: Simulating and Visualizing Large Scale Cassandra DeploymentsBattery Ventures: Simulating and Visualizing Large Scale Cassandra Deployments
Battery Ventures: Simulating and Visualizing Large Scale Cassandra Deployments
 
DataStax: How to Roll Cassandra into Production Without Losing your Health, M...
DataStax: How to Roll Cassandra into Production Without Losing your Health, M...DataStax: How to Roll Cassandra into Production Without Losing your Health, M...
DataStax: How to Roll Cassandra into Production Without Losing your Health, M...
 
Introduction to SQream and the IoT environment
Introduction to SQream and the IoT environmentIntroduction to SQream and the IoT environment
Introduction to SQream and the IoT environment
 
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
 
Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...
Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...
Microsoft: Building a Massively Scalable System with DataStax and Microsoft's...
 
Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...
Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...
Big Data Day LA 2016/ Use Case Driven track - From Clusters to Clouds, Hardwa...
 
Achieving scale and performance using cloud native environment
Achieving scale and performance using cloud native environmentAchieving scale and performance using cloud native environment
Achieving scale and performance using cloud native environment
 
Data Driven Decisions at Scale
Data Driven Decisions at ScaleData Driven Decisions at Scale
Data Driven Decisions at Scale
 
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
 
RedisConf18 - Making Real-Time Predictive Decisions with Redis
RedisConf18 - Making Real-Time Predictive Decisions with RedisRedisConf18 - Making Real-Time Predictive Decisions with Redis
RedisConf18 - Making Real-Time Predictive Decisions with Redis
 
RedisConf18 - The Versatility of Redis - Powering our critical business using...
RedisConf18 - The Versatility of Redis - Powering our critical business using...RedisConf18 - The Versatility of Redis - Powering our critical business using...
RedisConf18 - The Versatility of Redis - Powering our critical business using...
 
Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...
Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...
Using Azure Databricks, Structured Streaming, and Deep Learning Pipelines to ...
 
Building the Next-gen Digital Meter Platform for Fluvius
Building the Next-gen Digital Meter Platform for FluviusBuilding the Next-gen Digital Meter Platform for Fluvius
Building the Next-gen Digital Meter Platform for Fluvius
 
How jKool Analyzes Streaming Data in Real Time with DataStax
How jKool Analyzes Streaming Data in Real Time with DataStaxHow jKool Analyzes Streaming Data in Real Time with DataStax
How jKool Analyzes Streaming Data in Real Time with DataStax
 
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and BigtopAccelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
 
Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...
Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...
Analytics at the Real-Time Speed of Business: Spark Summit East talk by Manis...
 

Viewers also liked

Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16
Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16
Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16
MLconf
 
Resume
ResumeResume
Designing Good API & Its Importance
Designing Good API & Its ImportanceDesigning Good API & Its Importance
Designing Good API & Its Importance
Imran M Yousuf
 
Software Defined Networking (SDN) for the Datacenter
Software Defined Networking (SDN) for the DatacenterSoftware Defined Networking (SDN) for the Datacenter
Software Defined Networking (SDN) for the Datacenter
POSSCON
 
Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...
Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...
Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...
Ishan Thakkar
 
Perl.Hacks.On.Vim
Perl.Hacks.On.VimPerl.Hacks.On.Vim
Perl.Hacks.On.VimLin Yo-An
 
EVS Advances in VoLTE Networks
EVS Advances in VoLTE NetworksEVS Advances in VoLTE Networks
EVS Advances in VoLTE Networks
IMTC
 
Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...
Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...
Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...
Ishan Thakkar
 
AWS for Semiconductor and Electronics Design | Hsinchu, April 10
AWS for Semiconductor and Electronics Design | Hsinchu, April 10AWS for Semiconductor and Electronics Design | Hsinchu, April 10
AWS for Semiconductor and Electronics Design | Hsinchu, April 10
Amazon Web Services
 
Nordic VMUG User Conference 2014 - Design VMware vCenter Server
Nordic VMUG User Conference 2014 - Design VMware vCenter ServerNordic VMUG User Conference 2014 - Design VMware vCenter Server
Nordic VMUG User Conference 2014 - Design VMware vCenter Server
Andrea Mauro
 
You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...
You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...
You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...Softchoice Corporation
 
Softchoice Webinar Series: VMware vSphere 5.1 Changes
Softchoice Webinar Series: VMware vSphere 5.1 ChangesSoftchoice Webinar Series: VMware vSphere 5.1 Changes
Softchoice Webinar Series: VMware vSphere 5.1 Changes
Softchoice Corporation
 
SQL Server 2012 ile Gelen Yeni Özellikler
SQL Server 2012 ile Gelen Yeni ÖzelliklerSQL Server 2012 ile Gelen Yeni Özellikler
SQL Server 2012 ile Gelen Yeni Özelliklerturgaysahtiyan
 
Cassandra Introduction & Features
Cassandra Introduction & FeaturesCassandra Introduction & Features
Cassandra Introduction & Features
Phil Peace
 
Limewood Event - VMware
Limewood Event - VMware Limewood Event - VMware
Limewood Event - VMware
BlueChipICT
 
System Center 2012 - January Licensing Update
System Center 2012 - January Licensing UpdateSystem Center 2012 - January Licensing Update
System Center 2012 - January Licensing Update
Softchoice Corporation
 
VMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOL
VMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOLVMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOL
VMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOL
gguglie
 
Findability Day 2015 Mattias Ellison - Findwise - Enterprise Search and fin...
Findability Day 2015   Mattias Ellison - Findwise - Enterprise Search and fin...Findability Day 2015   Mattias Ellison - Findwise - Enterprise Search and fin...
Findability Day 2015 Mattias Ellison - Findwise - Enterprise Search and fin...
Findwise
 
Site Recovery Manager - Una visione architetturale
Site Recovery Manager - Una visione architetturaleSite Recovery Manager - Una visione architetturale
Site Recovery Manager - Una visione architetturale
gguglie
 
Docker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken CochraneDocker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken Cochrane
dotCloud
 

Viewers also liked (20)

Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16
Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16
Dr. Ike Nassi, Founder, TidalScale at MLconf NYC - 4/15/16
 
Resume
ResumeResume
Resume
 
Designing Good API & Its Importance
Designing Good API & Its ImportanceDesigning Good API & Its Importance
Designing Good API & Its Importance
 
Software Defined Networking (SDN) for the Datacenter
Software Defined Networking (SDN) for the DatacenterSoftware Defined Networking (SDN) for the Datacenter
Software Defined Networking (SDN) for the Datacenter
 
Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...
Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...
Massed Refresh: An Energy-Efficient Technique to Reduce Refresh Overhead in H...
 
Perl.Hacks.On.Vim
Perl.Hacks.On.VimPerl.Hacks.On.Vim
Perl.Hacks.On.Vim
 
EVS Advances in VoLTE Networks
EVS Advances in VoLTE NetworksEVS Advances in VoLTE Networks
EVS Advances in VoLTE Networks
 
Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...
Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...
Process Variation Aware Crosstalk Mitigation for DWDM based Photonic NoC Arch...
 
AWS for Semiconductor and Electronics Design | Hsinchu, April 10
AWS for Semiconductor and Electronics Design | Hsinchu, April 10AWS for Semiconductor and Electronics Design | Hsinchu, April 10
AWS for Semiconductor and Electronics Design | Hsinchu, April 10
 
Nordic VMUG User Conference 2014 - Design VMware vCenter Server
Nordic VMUG User Conference 2014 - Design VMware vCenter ServerNordic VMUG User Conference 2014 - Design VMware vCenter Server
Nordic VMUG User Conference 2014 - Design VMware vCenter Server
 
You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...
You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...
You voiced your concerns. VMware listened: Major Adjustments to vSphere 5 lic...
 
Softchoice Webinar Series: VMware vSphere 5.1 Changes
Softchoice Webinar Series: VMware vSphere 5.1 ChangesSoftchoice Webinar Series: VMware vSphere 5.1 Changes
Softchoice Webinar Series: VMware vSphere 5.1 Changes
 
SQL Server 2012 ile Gelen Yeni Özellikler
SQL Server 2012 ile Gelen Yeni ÖzelliklerSQL Server 2012 ile Gelen Yeni Özellikler
SQL Server 2012 ile Gelen Yeni Özellikler
 
Cassandra Introduction & Features
Cassandra Introduction & FeaturesCassandra Introduction & Features
Cassandra Introduction & Features
 
Limewood Event - VMware
Limewood Event - VMware Limewood Event - VMware
Limewood Event - VMware
 
System Center 2012 - January Licensing Update
System Center 2012 - January Licensing UpdateSystem Center 2012 - January Licensing Update
System Center 2012 - January Licensing Update
 
VMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOL
VMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOLVMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOL
VMUGIT Meeting Pisa 2015 - SDS secondo VMware: VSAN e VVOL
 
Findability Day 2015 Mattias Ellison - Findwise - Enterprise Search and fin...
Findability Day 2015   Mattias Ellison - Findwise - Enterprise Search and fin...Findability Day 2015   Mattias Ellison - Findwise - Enterprise Search and fin...
Findability Day 2015 Mattias Ellison - Findwise - Enterprise Search and fin...
 
Site Recovery Manager - Una visione architetturale
Site Recovery Manager - Una visione architetturaleSite Recovery Manager - Una visione architetturale
Site Recovery Manager - Una visione architetturale
 
Docker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken CochraneDocker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken Cochrane
 

Similar to Tidal scale short_story_v2

Enhancing Live Migration Process for CPU and/or memory intensive VMs running...
Enhancing Live Migration Process for CPU and/or  memory intensive VMs running...Enhancing Live Migration Process for CPU and/or  memory intensive VMs running...
Enhancing Live Migration Process for CPU and/or memory intensive VMs running...Benoit Hudzia
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
Amazon Web Services
 
Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...
Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...
Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...
OpsRamp
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
Amazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
Amazon Web Services
 
VMWare Wpg Jeff Franz-Lien
VMWare Wpg   Jeff Franz-LienVMWare Wpg   Jeff Franz-Lien
VMWare Wpg Jeff Franz-Lien
jfranzlien
 
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
Amazon Web Services
 
Amazon Kinesis
Amazon KinesisAmazon Kinesis
Amazon Kinesis
Amazon Web Services
 
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Amazon Web Services
 
Tapping the cloud for real time data analytics
 Tapping the cloud for real time data analytics Tapping the cloud for real time data analytics
Tapping the cloud for real time data analytics
Amazon Web Services
 
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
Amazon Web Services
 
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsDay 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Amazon Web Services
 
VMWare Winnipeg Forum - 2011
VMWare Winnipeg Forum - 2011VMWare Winnipeg Forum - 2011
VMWare Winnipeg Forum - 2011
asedha
 
Launching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWSLaunching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWS
Amazon Web Services
 
Xenserver 5 Selling And Positioning
Xenserver 5 Selling And PositioningXenserver 5 Selling And Positioning
Xenserver 5 Selling And PositioningYves Peeters
 
VMworld 2014: Virtualizing Databases
VMworld 2014: Virtualizing DatabasesVMworld 2014: Virtualizing Databases
VMworld 2014: Virtualizing Databases
VMworld
 
Managed NoSQL databases
Managed NoSQL databasesManaged NoSQL databases
Managed NoSQL databases
Amazon Web Services
 
AWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases OptionsAWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases Options
Amazon Web Services
 
Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine
Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice MachineSpark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine
Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine
Data Con LA
 

Similar to Tidal scale short_story_v2 (20)

Enhancing Live Migration Process for CPU and/or memory intensive VMs running...
Enhancing Live Migration Process for CPU and/or  memory intensive VMs running...Enhancing Live Migration Process for CPU and/or  memory intensive VMs running...
Enhancing Live Migration Process for CPU and/or memory intensive VMs running...
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...
Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...
Context is Critical: How Richer Data Yields Richer Results in AIOps | Bhanu S...
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
VMWare Wpg Jeff Franz-Lien
VMWare Wpg   Jeff Franz-LienVMWare Wpg   Jeff Franz-Lien
VMWare Wpg Jeff Franz-Lien
 
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
 
Amazon Kinesis
Amazon KinesisAmazon Kinesis
Amazon Kinesis
 
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
Understanding AWS Database Options (DAT201) | AWS re:Invent 2013
 
Tapping the cloud for real time data analytics
 Tapping the cloud for real time data analytics Tapping the cloud for real time data analytics
Tapping the cloud for real time data analytics
 
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
 
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsDay 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
 
VMWare Winnipeg Forum - 2011
VMWare Winnipeg Forum - 2011VMWare Winnipeg Forum - 2011
VMWare Winnipeg Forum - 2011
 
Launching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWSLaunching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWS
 
Designing virtual infrastructure
Designing virtual infrastructureDesigning virtual infrastructure
Designing virtual infrastructure
 
Xenserver 5 Selling And Positioning
Xenserver 5 Selling And PositioningXenserver 5 Selling And Positioning
Xenserver 5 Selling And Positioning
 
VMworld 2014: Virtualizing Databases
VMworld 2014: Virtualizing DatabasesVMworld 2014: Virtualizing Databases
VMworld 2014: Virtualizing Databases
 
Managed NoSQL databases
Managed NoSQL databasesManaged NoSQL databases
Managed NoSQL databases
 
AWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases OptionsAWS Summit 2013 | Singapore - Understanding Databases Options
AWS Summit 2013 | Singapore - Understanding Databases Options
 
Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine
Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice MachineSpark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine
Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine
 

Recently uploaded

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 

Recently uploaded (20)

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 

Tidal scale short_story_v2

  • 2. The Problem Variety & Volume: Data growing at 62% CAGR GB TB PB EB ZB DataVolumes Time Velocity: Data value declines with age BusinessValue Age of Data (seconds) Data Growth is HUGE The Value of Data Is Declining Rapidly
  • 3. NVMe Flash 150 μs $150,000 Flash Array 1 ms $1,500,000 Hard Drive 5 ms $7,500,000 TCP packet retransmit 2 s $2,000,000,000 The Problem In-Memory Computing is Key Operation Processing Latency In Human Terms L1-L3 Cache 1-13 ns $1 DRAM 50 ns $50 In-Memory Computing Is Needed For Performance
  • 4. Current Approaches - Lower prediction accuracy Algorithm - Lower prediction accuracy Sub Sample Do You Sample the Data?
  • 5. Current Approaches Shard 4 Shard 3 Shard 2 Shard 1 - Lower prediction accuracy - Time & Money Or, Do You Shard the Data?
  • 6. What do you do? Not having the right Server adds Complexity, increases Risks, and misses Opportunities
  • 7. What If Servers were Software-Defined? Are your servers too small?
  • 8. What If Servers were Software-Defined? What if your Servers could expand to fit all of your data?
  • 9. Software-Defined Servers • Just the right size • In-memory performance at scale • As many cores as needed • Self optimizing • Everything just works • Uses standard hardware Software-Defined Servers What if you could choose the size of your servers? Just the right size. In-memory performance. Self-optimizing. Everything just works. Using Commodity Servers.
  • 10. Software-Defined Servers – the Missing Piece Software-Defined Datacenters Storage Manage Network Server Software-Defined Servers are the missing piece in the Software- Defined Datacenter
  • 11. Traditional VirtualizationVirtualPhysical Multiple virtual machines share a single physical server Virtual Machine Virtual Machine Virtual Machine Application Operating System 100%,bit-for-bit unmodified Application Operating System Application Operating System Traditional virtualization cuts a server into smaller virtual servers (VM’s)
  • 12. TidalScale: Software-Defined Servers Software-Defined Servers are the inverse. One, two, ten or a full-rack of servers operating as one! Single virtual machine spans multiple physical servers Application OperatingSystem … HyperKernel HyperKernel HyperKernel TidalScale Virtual Machine 100%,bit-for-bit unmodified
  • 13. Machine Learning Driven Self-Optimization HyperKernel … HyperKernel HyperKernel HyperKernel HyperKernel Uses patented machine learning to transparently align resources Application Operating System TidalScale Software-Defined Server And it performs. Machine learning keeps locality by constantly & transparently optimizing.
  • 14. 100% Compatible Applications Operating Systems Virtual Machine If it runs, it runs on a TidalScale Software-Defined Server TidalScale Software-Defined Server HyperKernel … HyperKernel HyperKernel HyperKernel HyperKernel Containers Applications and Operating Systems just work. No Changes. Containers work great, too!
  • 15. Benchmark: Open Source R on TidalScale https://github.com/TidalScale/R_benchmark_test Open Source R Performance Comparisons TotalExecutionTime(Minutes) 100 0 100,000 200,000 400,000 700,000 - 200 300 400 500 600 300,000 500,000 600,000 Bare Metal Server (128GB) 158 days TidalScale Software-Defined Server (5 x 128GB nodes) 1,325 3,787 W orkload Size in GB A performance example. Avoid the memory cliff and scale the server to match the size of the data. 200X Faster
  • 16. Tomorrow’s Servers Today: A Game-Changer “Software-defined Servers make it easy to run memory-intensive applications like data mining, machine learning and simulation.” Marc Jones, Director & Distinguished Engineer, IBM Are you interesting in having the server of 2025?
  • 17. “This is the way all servers will be built in the future.” Gordon Bell Industry legend & 1st outside investor in TidalScale

Editor's Notes

  1. Slide 1: Software-Defined Servers: A Game Changer for Data Scientists | Hi I'm Gary Smerdon, CEO of TidalScale. It's my pleasure to be here today. "Game Changer" is a powerful word conjuring the likes of the iPhone and the Internet. But Software-Defined Servers are a game changer for Data Scientists and I'm here today to share why.
  2. Everyone here is intimately familiar with the problem. Data *Volume* and *Variety* is growing explosively at 62% a year. Simultaneously, the *Velocity* of data change is increasing rapidly such that the business value of that data decreases exponentially by its age in seconds. Getting a handle on this requires that computing be done *in-memory* because the alternative, paging from storage, is so incredibly slow. In dollar terms its the difference between buying your weekly groceries for $50 versus for $150,000. Such outsize costs would have a big negative impact on your annual food budget! We call this latency difference the *Memory Cliff* and it has a similar negative impact on application performance.
  3. Everyone here is intimately familiar with the problem. Data *Volume* and *Variety* is growing explosively at 62% a year. Simultaneously, the *Velocity* of data change is increasing rapidly such that the business value of that data decreases exponentially by its age in seconds. Getting a handle on this requires that computing be done *in-memory* because the alternative, paging from storage, is so incredibly slow. In dollar terms its the difference between buying your weekly groceries for $50 versus for $150,000. Such outsize costs would have a big negative impact on your annual food budget! We call this latency difference the *Memory Cliff* and it has a similar negative impact on application performance.
  4. You are all familiar with the solutions available for managing large data problems today. 1. Take a sample of the data, 2. Use a clever algorithm that makes some compressed representation of the data space, or 3. Shard the data set to fit in the memory of multiple small servers. All of these approaches have the disadvantages of lowering prediction accuracy, requiring time and effort and generally just getting in the way of doing data science. It's like having to do the plumbing when you just want to cook dinner.
  5. You are all familiar with the solutions available for managing large data problems today. 1. Take a sample of the data, 2. Use a clever algorithm that makes some compressed representation of the data space, or 3. Shard the data set to fit in the memory of multiple small servers. All of these approaches have the disadvantages of lowering prediction accuracy, requiring time and effort and generally just getting in the way of doing data science. It's like having to do the plumbing when you just want to cook dinner.
  6. What if the servers you used for data science were themselves software-defined? You could simply do your data science cookery without having to worry about the plumbing! To continue the thought experiment, imagine these servers could be assembled from ordinary servers, tuned themselves automagically, delivered in-memory performance at any scale and didn't require any change to software applications or operating systems. If such a software-defined server existed, it would deliver a missing piece in today's data centers where virtually _every_ other component _is_ software-defined. Today I am happy to share with you that such a technology _does_ exist: *TidalScale*. But to explain it I have to back up a bit and explain Traditional Virtualization.
  7. What if the servers you used for data science were themselves software-defined? You could simply do your data science cookery without having to worry about the plumbing! To continue the thought experiment, imagine these servers could be assembled from ordinary servers, tuned themselves automagically, delivered in-memory performance at any scale and didn't require any change to software applications or operating systems. If such a software-defined server existed, it would deliver a missing piece in today's data centers where virtually _every_ other component _is_ software-defined. Today I am happy to share with you that such a technology _does_ exist: *TidalScale*. But to explain it I have to back up a bit and explain Traditional Virtualization.
  8. What if the servers you used for data science were themselves software-defined? You could simply do your data science cookery without having to worry about the plumbing! To continue the thought experiment, imagine these servers could be assembled from ordinary servers, tuned themselves automagically, delivered in-memory performance at any scale and didn't require any change to software applications or operating systems. If such a software-defined server existed, it would deliver a missing piece in today's data centers where virtually _every_ other component _is_ software-defined. Today I am happy to share with you that such a technology _does_ exist: *TidalScale*. But to explain it I have to back up a bit and explain Traditional Virtualization.
  9. What if the servers you used for data science were themselves software-defined? You could simply do your data science cookery without having to worry about the plumbing! To continue the thought experiment, imagine these servers could be assembled from ordinary servers, tuned themselves automagically, delivered in-memory performance at any scale and didn't require any change to software applications or operating systems. If such a software-defined server existed, it would deliver a missing piece in today's data centers where virtually _every_ other component _is_ software-defined. Today I am happy to share with you that such a technology _does_ exist: *TidalScale*. But to explain it I have to back up a bit and explain Traditional Virtualization.
  10. Traditional virtualization effectively divvies-up a server into multiple virtual servers. The key fact is that the virtual servers have _no idea_ they aren't actually running on hardware. The applications and operating system is unmodified.
  11. TidalScale uses virtualization but does it _across_ multiple physical servers, effectively allowing you to treat physical servers like lego building blocks.
  12. Under the covers, TidalScale uses machine learning to automatically co-locate compute entity with the resources it needs. And just like ordinary virtualization it does all of this transparently - the OS and applications have no idea they aren't running on hardware....
  13. TidalScale can run virtually everything unchanged, and we deliver in-memory performance, for example...
  14. To document our scalability we wrote and published a benchmark on Open Source R. It measures the time to execute load, join and then the 5 most commonly used data science algorithms on CMS insurance claim data. We ran this benchmakr on a bare metal 128GB server and then on a TidalScale TidalPod composed of five 128GB servers. As you can see on this chart, the Software-defined Server's performance scales linearly.
  15. So how would it change the way you tackle data science problems to have the ability to deploy the hardware systems of tomorrow today? An example of the kind of systems we can create is the 15TB 400 core TidalPod we deployed at SoftLayer this past June using 20 standard servers. Marc Jones captured it best: “Software-defined Servers make it easy to run memory-intensive applications like data mining, machine learning and simulation.”
  16. TidalScale Software-Defined Servers are a game changer! They let you explore your data more easily, improve your results and allow you spend more time doing data science instead of data plumbing all while lowering your operational TCO and increasing IT flexibility. This is why one of our early investor's said it best: "This is the way all servers will be built in the future."