10. Norikra Queries: (3)
{“name”:”tagomoris”,
“age”:34, “address”:”Tokyo”,
“corp”:”LINE”, “current”:”Tsukuba”}
SELECT age, COUNT(*) as cnt
FROM events.win:time_batch(5 mins)
GROUP BY age
every 5 mins
{”age”:34,”cnt”:3}, {“age”:33,”cnt”:1}, ...
14年1月31日金曜日
11. Norikra Queries: (4)
{“name”:”tagomoris”,
“age”:34, “address”:”Tokyo”,
“corp”:”LINE”, “current”:”Tsukuba”}
SELECT age, COUNT(*) as cnt
FROM
events.win:time_batch(5 mins)
GROUP BY age
SELECT max(age) as max
FROM
events.win:time_batch(5 mins)
{”age”:34,”cnt”:3}, {“age”:33,”cnt”:1}, ...
{“max”:51}
14年1月31日金曜日
every 5 mins
12. Norikra Queries: (5)
{“name”:”tagomoris”,
“user:{“age”:34, “corp”:”LINE”,
“address”:”Tokyo”},
“current”:”Tsukuba”,
“speaker”:true,
“attend”:[true,true,false, ...]
}
SELECT age, COUNT(*) as cnt
FROM events.win:time_batch(5 mins)
GROUP BY age
14年1月31日金曜日
13. Norikra Queries: (5)
{“name”:”tagomoris”,
“user:{“age”:34, “corp”:”LINE”,
“address”:”Tokyo”},
“current”:”Tsukuba”,
“speaker”:true,
“attend”:[true,true,false, ...]
}
SELECT user.age, COUNT(*) as cnt
FROM events.win:time_batch(5 mins)
GROUP BY user.age
14年1月31日金曜日
14. Norikra Queries: (5)
{“name”:”tagomoris”,
“user:{“age”:34, “corp”:”LINE”,
“address”:”Tokyo”},
“current”:”Kyoto”,
“speaker”:true,
“attend”:[true,true,false, ...]
}
SELECT user.age, COUNT(*) as cnt
FROM events.win:time_batch(5 mins)
WHERE current=”Tsukuba”
AND attend.$0 AND attend.$1
GROUP BY user.age
14年1月31日金曜日
15. Norikra and Esper
Esper:
CEP engine library, Java, GPLv2
EPL: Event Processing Language (SQL + window)
Streams: schema-full flat field set
Norikra:
Using Esper internally
Schema-less stream -> schema-full stream
conversion
Rewriting compiled queries
14年1月31日金曜日
16. Norikra query execution
1.accept query
2.parse -> find target / field set
3.(if target is not opened) wait for first event
4.compile query
5.rewrite target name into stream name
6.rewrite field names
7.register query
8.input events
14年1月31日金曜日
17. Target mapping
Ignore unused fields
Field set matching between streams and queries
Generate field set inheritance tree
14年1月31日金曜日
18. automated stream inheritance
of norikra's target
Base fieldset
Query fieldset
Data fieldset
14年1月31日金曜日
b_xxxxxxxxx
minimal fieldset definition:
name: 'string'
id: 'long'
valid: 'boolean'
action_type: 'string'
19. automated stream inheritance
of norikra's target
Base fieldset
b_xxxxxxxxx
Query fieldset
Data fieldset
14年1月31日金曜日
e_xxxxxxxx1
event data fieldset definition:
name: 'string'
id: 'long'
valid: 'boolean'
action_type: 'string'
product_code: 'string'
charge: 'integer'
shop_code: 'long'
20. automated stream inheritance
of norikra's target
Base fieldset
event data fieldset definition:
name: 'string'
id: 'long'
valid: 'boolean'
action_type: 'string'
product_code: 'string'
Query fieldset
charge: 'integer'
shop_code: 'long'
affiliate: 'string'
Data fieldset
14年1月31日金曜日
e_xxxxxxxx1
b_xxxxxxxxx
e_xxxxxxxx2
21. automated stream inheritance
of norikra's target
Base fieldset
b_xxxxxxxxx
new query:
SELECT count(*)
FROM target.win:time_batch(1min)
WHERE affiliate.length() > 0
Query fieldset
Data fieldset
14年1月31日金曜日
e_xxxxxxxx1
e_xxxxxxxx2
22. new query:
SELECT count(*)
FROM target.win:time_batch(1min)
WHERE affiliate.length() > 0
automated stream inheritance
of norikra's target
Base fieldset
b_xxxxxxxxx
Query fieldset
Data fieldset
14年1月31日金曜日
event data fieldset definition:
name: 'string'
id: 'long'
valid: 'boolean'
action_type: 'string'
affiliate: 'string'
q_xxxxxxxx0
e_xxxxxxxx1
e_xxxxxxxx2'
23. Registered
automated stream inheritance EPL:
SELECT count(*)
of norikra's targetFROM q_xxxxxxxx0.win:time_batch(1min)
WHERE affiliate.length() > 0
Base fieldset
b_xxxxxxxxx
Query fieldset
Data fieldset
14年1月31日金曜日
q_xxxxxxxx0
e_xxxxxxxx1
e_xxxxxxxx2'
24. automated stream inheritance
of norikra's target
Base fieldset
b_xxxxxxxxx
Query fieldset
q_xxxxxxxx0
q_xxxxxxxx1
Data fieldset e_xxxxxxxx1'
e_xxxxxxxx2'
e_xxxxxxxx3'
14年1月31日金曜日
30. {“user$age”:34,
“current”:”Kyoto”,
“attend$$0”:true,
“attend$$1”:true,
}
SELECT
user.age, COUNT(*) as cnt
FROM events.win:time_batch(5 mins)
WHERE current=”Tsukuba”
AND attend.$0 AND attend.$1
GROUP BY user.age
Conversions for compiled query object in fact.
SELECT user$age, COUNT(*) as cnt
FROM events.win:time_batch(5 mins)
WHERE current=”Tsukuba”
AND attend$$0 AND attend$$1
GROUP BY user$age
14年1月31日金曜日