6. http://blog.nahurst.com/visual-guide-to-nosql-systems

9. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "abc",
"number" : 3,
"subobj" : {"subA": 1, "subB": { "subsubC": 2 }},
"array" : [1, 2, 3],
"dbref" : [_id1, _id2, _id3]
}

10. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"nickname" : "doryokujin"
},{
"_id" : ObjectId("4dcd3ebc9278000000005159"),
"firstname" : "Takahiro",
"lastname" : "Inoue",
"mail" : "mr.stoicman@gmail.com",
"twitter" : "@doryokujin"
},...

11. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "abc",
"number" : 3,
"subobj" : {"subA": 1, "subB": 2 },
"array" : [1, 2, 3],
padding
}

12. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "def",
"number" : 4,
"subobj" : {"subA": 1, "subB": 2 },
"array" : [1, 2, 3, 4, 5, 6],
"newkey" : "In-place"
}

16. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "abc",
"number" : 3,
"subobj" : {"subA": 1, "subB": 2 },
"array" : [1, 2, 3],
}

26. Cluster
Shard Servers (Data)
config Servers
(Shard Configration) shard1 shard2 shard3
[ a, f ) [ k, n) [ o, t ) Chunk
[ f, k ) [ n, o ) [ t, } )
mongos Servers (Routers)

27. Shard
1
( mongos
primary primary
)
Shard
cinfig Shard
mongos

35. Every Server:
Large HDD(500GB )
Large Memory(16GB )
Slave Delay in Master Data on
Preparation for AmazonS3
User Error
Master Data on S3
Non Sharding,
Replica Set

38. From Text Logs: (Large)
From mySQL: (Small)
From Other NoSQL: (Middle)
Temporary Raw data storage
is HDFS, not MongoDB
We only need result data
to discover new features

39. Reduction
First Second
Aggregation Aggregation

40. ScientificPython

41. Fluent
Structured logging
Pluggable architecture
Reliable forwarding
e Event Collector Service

42. Fluent
Structured logging
Pluggable architecture
Reliable forwarding
e Event Collector Service
Sadayuki Furuhashi
Treasure Data, Inc.
@frsyuki

43. “2011-04-01 host1 myapp: cmessage size=12MB user=me”
2011-04-01 myapp.message {
“on_host”: ”host1”,
2011-04-01 myapp.message {
”combined”: true, “on_host”: ”host1”,
”combined”: true,
“size”: 12000000, “size”: 12000000,
“user”: “me”
“user”: “me” }
}

46. log log log log

48. aggregate aggregate aggregate aggregate
log log log log
key1 key2 key3 shuffle
aggregate aggregate aggregate
aggregate