This document provides an overview of topics related to data management, including regular expressions, databases, relational modeling, SQL, NoSQL, and MongoDB. It discusses using regular expressions to match patterns in text, the history of databases from linked lists to relational models, how to model relationships between entities in tables, how to query and manipulate data with SQL, when to use NoSQL solutions like MongoDB, and how to choose appropriate data storage strategies based on needs. Homework assignments are referenced.

1. MAS.500 - Software Module - Rahul Bhargava
Data
Management
2014.11.21

2. Topics
❖ Regular Expressions (online quickstart)
❖ Databases
❖ History
❖ Relational modeling
❖ Sql (mysql quickstart)
❖ Keys/Indexes
❖ No-sql (couchdb quickstart)
❖ Behind the Scenes with Ed Platt
❖ Homework

3. Regular Expressions

4. Regular Expressions
(RegEx/grep)
❖ Match a string of text by defining a pattern
❖ Useful for cleaning up or identifying data
❖ “Find” Demo on http://regexpal.com
❖ “Find/Replace” Demo with
http://www.sugarscript.com/findandreplace/index.php
❖ Interested? Interactive tutorial on http://regexone.com

5. Databases

6. Database History
❖ List-based
❖ Follow link from one record to another (linked-list)
❖ File-system data stores
❖ Based on filenaming convention, limited by file i/o
speeds
❖ Generic data storage and management
❖ Relational modeling or entities and relationships
(ER)

7. Relational Modeling: In
English
❖ A Group has many People
❖ A Person belongs to one Group
❖ A Group has many Projects
❖ A Project belongs to one Group
❖ A Person has many Projects
❖ A Project has many People

8. Relational Modeling: Diagram
many 1
Person Group
Project
1
many
many
many

9. Relational Modeling: Tables
Group:
id
name
url
Person:
id
name
password
group_id
many 1
Project:
id
name
url
1
many
many
many
Membership:
person_id
project_id

10. Relational Modeling: Keys
Group:
id
name
url
Person:
id
name
password
group_id
many 1
Project:
id
name
url
1
many
many
many
Membership:
person_id
project_id
key
Foreign keys
key
key

11. Structured Query Language
(SQL)
❖ Works in lots of database servers
❖ SQLite, MySQL, PostgreSQL, MS SQL Server
❖ Standard way to:
❖ Find subsets of data based on criteria
❖ Merge data in separate tables
❖ Compute aggregate info
❖ Assumptions
❖ Don’t duplicate data (“data normalization”)
❖ Various parts of your data relate to each other
❖ Your metadata/schema (tables/columns) doesn’t change often
❖ Many frameworks will generate SQL for you
❖ Ask about Database Abstraction Layers

12. NoSQL
❖ Sometimes your data isn’t relational and the metadata
changes often
❖ Queuing, document storage, logging, real-time, low-latency,
concurrency
❖ Read this write up for more:
❖ http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

13. Tangent: JavaScript Object Notation
(JSON)
❖ A human-readable data exchange format
❖ CSV, XML, YAML are some others
❖ Example:
❖ http://media.mongodb.org/zips.json
❖ http://mongohub.todayclose.com (for Mac)

14. ❖ sudo mkdir -p /data/db

15. MongoDB: Intro
❖ Demo:
❖ Command Line
❖ MongoHub

16. Indexes
❖ An index tracks keys
❖ Convention: have an “id” column with an index on it
❖ Why all these indexes?
❖ Multiple ways to get at rows quickly
❖ Creating indexes is tricky
❖ Many frameworks include query logging to help you find
slow queries that might need optimizing
❖ Query optimization is a bit of an art
❖ Use the “Explain” command

17. Map-Reduce Instead of SQL
❖ Used to query large datasets
❖ Example: Count words in a document
❖ Map: select the data you need to operate on
❖ “emit” one records for each word in a document,
keyed by the word
❖ Reduce: combine the mapped data
❖ Sum up the uses of each word, “emitting” one
record for each total

18. Picking Data Storage
Strategies
❖ If you just need to dump data and pull it out by some id, use a no-sql
solution (MongoDB is simple)
❖ flexible, easy to start with
❖ If you are modeling an app, a relational database is usually the
right answer (MySQL/PostgreSQL are standard)
❖ Database modeling is REALLY important to get right at the
start of your project, because it is a pain to change later
❖ Names matter – choose your table names carefully
❖ PS: we can try stuff out on Amazon’s cloud services for free

19. Homework
❖ see course outline