NoSQL databases like MongoDB, Elasticsearch, and Cassandra are synonymous with scalability, search, and developer agility. But there’s a downside...having to give up the ease and comfort of SQL. Or do you? Join this webcast to learn how the newest databases, like CrateDB and CockroachDB deliver the benefits of NoSQL with the ease of SQL by building SQL engines on top of custom NoSQL technology stacks. Database industry veteran Andy Ellicott, who helped launch Vertica, VoltDB, Cloudant, and now with Crate.io, will provide a no-BS view of current DBMS architectures and predictions for the future of data. If you’re a DBMS user, this webcast will help you make sense of a very crowded DBMS market and make better-informed decisions for your new tech stacks.

1. November, 2017
Andy Ellicott, Crate.io
Is the Future of NoSQL…
SQL?

2. Logistics…
• Submit questions at any time via the questions panel
• Slides & recording will be shared via email after the event

3. Agenda
–
• The NoSQL Era, the good, the bad, the ugly
• Post-NoSQL…Distributed SQL renaissance
• “Things Data” is coming
• Other DBMS futures predictions
• Additional reading
• Questions & answers

4. I like databases
25 years in DBMS & software development companies
IMHO…the coolest ways software is changing what’s
possible in life and business…is usually due to some
database changing what’s possible with software.

5. Good, Bad, & Ugly
The NoSQL Era

6. DBMS Timeline
–
2005 2010
SQL - One Size
Fits All era
Distributed
SQL 1.0 era NoSQL era
• Oracle
• DB2
• SQL Server
• MySQL
• PostgreSQL
• Vertica
• Greenplum
• Netezza
• Paraccel
• VoltDB
• …
• MongoDB
• Hadoop
• DynamoDB
• Cassandra
• Redis
• …

7. NoSQL Family Tree
–

8. NoSQL, The Good…
-
• Many, many choices for most any use case
- JSON document stores
- Key-value stores
- Cacheing
- Time series
- Text search
• Easy, Economical, Developer friendly:
- Scalability
- Fault-tolerance
- Dynamic, flexible schemas (JSON)
- Open source

9. Knowing the CAP Theorem Helped…
-
• A partitioned database…
- where data is duplicated across multiple machines
- Access to that data can be EITHER
• Highly Consistent (e.g., MongoDB)
• or Highly available (e.g., DynamoDB)
• We learned the sky doesn’t fall if you forfeit ACID
- “Eventual Consistency”

10. NoSQL, The Bad…
-
• No standards (i.e., literally, no SQL)
- Harder to learn
- Hard to integrate
• Too many choices, hard to differentiate
- MongoDB vs. Rethink?
- CouchDB vs. Couchbase?
• DBA expertise
- Resizing & rebalancing database clusters
• Brute force query optimization, via code
• Polyglot persistence gone wild
- Use multiple specialized databases in a single system
- Over time, duplicate data storage and sync costs can grow out of control

11. NoSQL, The Ugly…
-
• Market “consolidation”
- RethinkDB
- Riak
- FoundationDB

12. Ten Years Ago …
-
Was NoSQL a step backwards
in DBMS technology…or a step
forwards?

13. • Greatly expanded researchers & contributors
• Debunked assumptions about requirements
- SQL access
- ACID / Eventual consistency
• Created open source code and thought leadership on
which next generation of SQL is being built
IMHO, NoSQL has been a step forwards
-

14. Distributed SQL II

15. The Newest Generation of SQL
–
SQL NOSQL

16. Crate Components ( ​ ​Crate ​ ​ Elasticsearch , ​ ​other Open Source)
The CrateDB Open Source Stack
–
1 file to download & install

17. CrateDB - the key inventions
–
Distributed SQL with search, time
series, geospatial, aggregations
Cloud-native architecture
easy scaling via Containers
NoSQL storage & clustering for
horizontal scaling & dynamic schema
Columnar Caches for real-time, in-
memory SQL query performance
shared-nothing architecture

18. Using CrateDB
–
Simple install
Zero-configuration, auto-join
Compatible
ANSI SQL vis Postgres-wire
protocol, JDBC, REST
Real-time performance
Distributed SQL query engine
Dynamic schema
all data (structured + JSON), time
series, geospatial
Distributed SQL query versatility
Aggregations, time series, search,
geospatial…
Simpler scalability
Shared nothing, horizontal scale out
Always on
High availability, replication, self-
healing
Flexible
No lock-in, runs any cloud and on-
premise

19. The next wave of big data
will come from machines
“Things Data”

20. The Next Wave of Big Data
–
“IoT is creating unparalleled information
management and analytics challenges.”
- Jim Hare, Gartner
Every
Step
Every
Lightbulb
Every
Message
Every
Bottle
•Firehose of data
•Complex data
•Real-time
•Edge + Cloud
Millions of data points per second
Instantly actionable - current & large historic data sets
Run anywhere. Cloud. On-premises Containers. Small
footprint or large clusters with 100+ nodes.
Joins, Time Series, Geospatial, JSON, Text search, AI, Blobs

21. • Plastic bottles, caps, lids etc
- e.g. produces the plastic Coca Cola Bottles in USA
• CrateDB enables “real-time factory”
- 1500+ production lines, 900 sensor types
- Improve overall equipment efficiency (OEE)
- Reduce labor costs / Reduce waste
• Millions of data points per minute
• Dozens of realtime charts
- Chart refresh (query speed) reduced from 5 minutes to 0.3
seconds with CrateDB
Bottles/day Factories Worldwide
160M 180
Customer: ALPA USA
$4B global plastic packaging manufacturer
–

22. Customer: Skyhigh
Cloud security, Campbell
–
• Cloud Access Security Broker (CASB)
• Billions of events/day into CrateDB - internet traffic for
40% of Fortune 500
• Realtime dashboards - flag suspicious, risky internet
usage
• CrateDB replaced Elasticsearch & MySQL
- queries were 20x faster with only 25% of hardware
Sekhar Sarukkai
Co-founder & SVP Engineering
“CrateDB’s real-time SQL
performance, simple scaling, and
high availability make it a key
element of our stack”
Billions of
events / day
40% Fortune 500
30M+ Users
Interactive
real-time dashboards

23. “It’s a lot of data, which you need to ingest
very fast and auto-query very fast. That’s
why we brought in CrateDB.”
Mark Sutheran
CEO
Customer:
Automotive, Singapore
–
Sensor Readings / per second / per car
2,000
• IoT-Enabled Vehicle Tracking
• Predictive maintenance
• Data in Crate allows full 3D reconstruction
of accidents

24. CrateDB Traditional SQL Distributed SQL I NoSQL
SQL
(ease of adoption &
integration)
✅ ✅ ✅ ❌
Complex,
dynamic data ✅ ❌ ❌ ✅
Firehose &
Real-Time Queries
✅ ❌ ✅ ✴
Scale out architecture ✅ ❌ ✅ ✅
Open source / Economical ✅ ✴ ❌ ✅
New DBMS Required for “Things Data” Era?
–

25. What’s Next?

26. If You’re Doing Distributed…
–
Gateway
Devices
Servers, Sensors,
Actuators, Machines,
Wearables, Cars etc.
Applications
& PlatformsGateway & DB
Edge Public/Hybrid/Private
shared-nothing architecture
CrateDB enables use-cases at the “edge” and in the cloud, with SQL, horizontal scaling, high availability, and multi-model data
structures. With CrateDB, customers can extract value from realtime data, enabling applications & services not possible before.

27. Will Scale-out Databases Start Eliminating the Need for Middleware?
–
• Message queues were invented to compensate for
DBMS weaknesses
- Downtime
- Slow ingestion
• New databases like CrateDB don’t have those
pitfalls
• Embedding MQTT broker in CrateDB
- Define “Ingestion rules” in CrateDB
• MQTT topic —> Target table for storage
- Stores messages in tables
- Eliminates the need for extra middleware
• Lowers hosting costs, complexity, development time fast ingest. always-on architecture
Embedded MQTT Broker
Message Queue
Devices
MQTT messages MQTT messages
versus
DBMS
slow ingest &
DB downtime
Devices
MQTT Broker
MQTT Consumer/Writer

28. You might be done with NoSQL when…
–
• You find yourself saying “I wish I could just use a join”
• The competition for hiring new experienced users is
slowing your team growth
• You’re using too many different DBs together…”too
specialized”

29. Thank You!
-
• CrateDB
- https://crate.io
• Slides & recording of this will be sent to you shortly, via email
• Ping me any time
- Andy Ellicott
- andy@crate.io

30. Andy Ellicott
andy@crate.io
Thank you