For the past 25 years applications have been getting built using an RDBMS with a predefined schema which forces data to conform with a schema on-write. Many people still think that they must use an RDBMS for applications even though records in their datasets have no relation to one another. Additionally, those databases are optimized for transactional use, and data must be exported for analytics purposes. NoSQL technologies have turned that model on its side to deliver groundbreaking performance improvements. I will walk through a music database with over 100 tables in the schema and show how to convert that model over for use with a NoSQL database. I will show how to handle creating, updating and deleting records, using column families for different types of data (and why).

1. © 2015 MapR Technologies 1© 2015 MapR Technologies

2. © 2015 MapR Technologies 2
Why Does this Matter
• 90%+ of the use cases do not deal with “relational” data
• RDBMS data models are more complex than a single table
– One-to-many relationships require multiple tables
– Creating code to persist data takes time and QA
• Inferred (or removed) keys used without actual foreign keys
– Difficult for others to understand relationships
• Transactional tables never look the same as analytics tables
– OLTP -> ETL -> OLAP
– This takes significant time to build

3. © 2015 MapR Technologies 3
Empowering “as it happens”
businesses by speeding up the
data-to-action cycle

4. © 2015 MapR Technologies 4© 2015 MapR Technologies
Changing Data Models

5. © 2015 MapR Technologies 5
180 Tables
NOT SHOWN!

6. © 2015 MapR Technologies 6
236 tables
to describe 7 kinds of things

7. © 2015 MapR Technologies 7
artist
id
gid
name
sort_name
begin_date
end_date
ended
type
gender
area
being_area
end_area
comment
list<ipi>
list<isni>
list<alias>
list<release_id>
list<recording_id>
artist
id
gid
name
sort_name
begin_date
end_date
ended
type
gender
area
being_area
end_area
comment
list<ipi>
list<isni>
list<alias>

8. © 2015 MapR Technologies 8
artist
id
gid
name
sort_name
begin_date
end_date
ended
type
gender
area
being_area
end_area
comment
list<ipi>
list<isni>
list<alias>
list<release_id>
list<recording_id>

9. © 2015 MapR Technologies 9
Searching for Elvis
// Find discs where Elvis was credited
> SELECT distinct album_id, name FROM
(SELECT id album_id, artist_id, name, FLATTEN(credit) FROM release) albums
join
(SELECT distinct artist_id FROM
(SELECT id artist_id, FLATTEN(alias) FROM artist
where name like 'Elvis%Presley’)
) artists
USING artist_id;

10. © 2015 MapR Technologies 10
Benefits
• Extended relational model allows massive simplification
– On a real example, we see >20x reduction in number of tables
• Simplification drives improved introspection
– This is good
• Apache Drill gives very high performance execution for extended
relational problems
• You can try this out today

11. © 2015 MapR Technologies 11© 2015 MapR Technologies
A New Database for Data

12. © 2015 MapR Technologies 12
Basics of the API
• http://ojai.github.io/
• Entry point to a table - DocumentStore
– insert()
– insertOrReplace()
– find()
– delete()
– replace()
– update()
– increment()

13. © 2015 MapR Technologies 13
Working with JSON in Java
• Step 1 – Create instance of JSON Serializer
Gson gson = new Gson();
• Step 2 – Serialize POJO to JSON
String json = gson.toJson(myObject);
• Step 3 – Deserialize JSON into POJO
MyObject myObject = gson.fromJson(json, MyObject.class);

14. © 2015 MapR Technologies 14
Creating Documents in Java OJAI
• Use static methods on class org.ojai.json.Json
Document doc = Json.newDocument(myObject);
Document doc = Json.newDocument(jsonString);
• Alternatively
– Use builders
– Stream from disk
– Use InputStream

15. © 2015 MapR Technologies 15
Creating New Documents
• DocumentStore.insert(doc)
Done!
• DocumentStore.insertOrReplace(doc)
Done!
Easy right?

16. © 2015 MapR Technologies 16
Updating Existing Documents
• DocumentStore.update(_id, DocumentMutation)
• Mutation methods
– mutation.append(FieldPath, “user visited URL”);
– mutation.set(“field.name”, “What a great example”);
– mutation.increment(“field”, 1);
– mutation.merge(“field”, Map<String, Object>);
– mutation.setOrReplace(…);
– mutation.delete(field);
Yes, these are atomic.

17. © 2015 MapR Technologies 17
Deleting Documents
• DocumentStore.delete(doc);
Done!
• DocumentStore.delete(_id);
Done!
This is easy too, right?

18. © 2015 MapR Technologies 18
Finding Documents
• DocumentStore.find(QueryCondition);
• Query condition setup:
– qc.is(“field”, EQUAL, “blue”)
.and().notExists(“other.field”)
.or().like(“field”, “%purple”)
.or().matches(“another.field”, “regular expression”)

19. © 2015 MapR Technologies 19© 2015 MapR Technologies
Querying JSON Data and More

20. © 2015 MapR Technologies 20
How To Bring SQL to Non-Relational Data Stores?
Familiarity of SQL Agility of NoSQL
• ANSI SQL semantics
• BI (Tableau, MicroStrategy,
etc.)
• Low latency
• No schema management
– HDFS (Parquet, JSON, etc.)
– HBase
– …
• No transformation
– No silos of data
• Ease of use

21. © 2015 MapR Technologies 21
Drill Supports Schema Discovery On-The-Fly
• Fixed schema
• Leverage schema in centralized
repository (Hive Metastore)
• Fixed schema, evolving schema or
schema-less
• Leverage schema in centralized
repository or self-describing data
2Schema Discovered On-The-FlySchema Declared In Advance
SCHEMA ON
WRITE
SCHEMA
BEFORE READ
SCHEMA ON THE
FLY

22. © 2015 MapR Technologies 22
Enabling “As-It-Happens” Business with Instant Analytics
Hadoop data Data modeling Transformation
Data
movement
(optional)
Users
Hadoop data Users
Traditional
approach
Exploratory
approach
New Business questionsSource data evolution
Total time to insight: weeks to months
Total time to insight: minutes

23. © 2015 MapR Technologies 23
Common Use Cases
Raw Data Exploration JSON Analytics DWH offload
Hive HBaseFiles Directories
…
{JSON}, Parquet
Text Files …

24. © 2015 MapR Technologies 24
Reuse Existing SQL Tools and Skills
Leverage SQL-compatible tools
(BI, query builders, etc.) via Drill’s
standard ODBC, JDBC and ANSI
SQL support
Enable business analysts, technical
analysts and data scientists to
explore and analyze large volumes
of real-time data

25. © 2015 MapR Technologies 25© 2015 MapR Technologies
Security Controls

26. © 2015 MapR Technologies 26
Granular Security via Drill Views
Name City State Credit Card #
Dave San Jose CA 1374-7914-3865-4817
John Boulder CO 1374-9735-1794-9711
Raw File (/raw/cards.csv)
Owner
Admins
Permission
Admins
Business Analyst Data Scientist
Name City State Credit Card #
Dave San Jose CA 1374-1111-1111-1111
John Boulder CO 1374-1111-1111-1111
Data Scientist View (/views/maskedcards.csv)
Not a physical data copy
Name City State
Dave San Jose CA
John Boulder CO
Business Analyst View
Owner
Admins
Permission
Business
Analysts
Owner
Admins
Permission
Data
Scientists

27. © 2015 MapR Technologies 27
Ownership Chaining
Combine Self Service Exploration with Data Governance
Name City State Credit Card #
Dave San Jose CA 1374-7914-3865-4817
John Boulder CO 1374-9735-1794-9711
Raw File (/raw/cards.csv)
Name City State Credit Card #
Dave San Jose CA 1374-1111-1111-1111
John Boulder CO 1374-1111-1111-1111
Data Scientist (/views/V_Scientist)
Jane (Read)
John (Owner)
Name City State
Dave San Jose CA
John Boulder CO
Analyst(/views/V_Analyst)
Jack (Read)
Jane(Owner)
RAWFILEV_ScientistV_Analyst
Does Jack have access to V_Analyst? ->YES
Who is the owner of V_Analyst? ->Jane
Drill accesses V_Analyst as Jane (Impersonation hop 1)
Does Jane have access to V_Scientist ? -> YES
Who is the owner of V_Scientist? ->John
Drill accesses V_Scientist as John (Impersonation hop 2)
John(Owner)
Does John have permissions on raw file? -> YES
Who is the owner of raw file? ->John
Drill accesses source file as John (no impersonation here)
Jack queries the view V_Analyst
*Ownership chain length (# hops) is configurable
Ownership
chaining
Access
path

28. © 2015 MapR Technologies 28© 2015 MapR Technologies
Resources

29. © 2015 MapR Technologies 29
Drill is Top-Ranked SQL-on-Hadoop
Source: Gigaom Research, 2015
Key:
• Number indicates companies relative strength across all vectors
• Size of ball indicates company’s relative strength along individual vector
“Drill isn’t just about
SQL-on-Hadoop.
It’s about SQL-on-
pretty-much-
anything,
immediately, and
without formality.”

30. © 2015 MapR Technologies 30

31. © 2015 MapR Technologies 31
OJAI and MapR-DB
Where to find it…
– The source: https://github.com/ojai/ojai
– The site: http://ojai.github.io/
– Python bindings: https://github.com/mapr-demos/python-bindings
– Javascript bindings: https://github.com/mapr-demos/js-bindings
Ready to play with your data?
– Download the sandbox: http://maprdb.io
– Examples:
• Java: https://github.com/mapr-demos/maprdb-ojai-101
• Python: https://github.com/mapr-demos/maprdb_python_examples

32. © 2015 MapR Technologies 32
Drill Walkthrough
• Example queries
• Conversion from relational model to flat JSON model
https://www.mapr.com/blog/drilling-healthy-choices
https://www.mapr.com/blog/evolution-database-schemas-using-sql-
nosql

33. © 2015 MapR Technologies 33
Recommendations for Getting Started with Drill
New to Drill?
– Get started with Free MapR On Demand training
– Test Drive Drill on cloud with AWS
– Learn how to use Drill with Hadoop using MapR sandbox
Ready to play with your data?
– Try out Apache Drill in 10 mins guide on your desktop
– Download Drill for your cluster and start exploration
– Comprehensive tutorials and documentation available
Ask questions
– user@drill.apache.org

34. © 2015 MapR Technologies 34
Q&A
@kingmesal
jscott@mapr.com
Engage with us!
kingmesal