Last Updated: Jan. 2014
VP Platform Evangelism
@cobiacomm on Twitter
http://blog.cobia.net/cobiacomm/
Chris Haddad
Tracking a Soccer
Game with Big Data
Big Data Improves Team Performance
๏ Sports Teams
๏ Your Business Team2
Improvement Requires A Feedback Loop
3
Internet of Things
(IoT) Bridges
Physical and Virtual
Worlds
๏ Sensors, actuators everywhere
๏ Internet connectivity
Improvement = IoT, Apps, and APIs
๏ Visualization Apps
๏ Control APIs
DEBS Challenge
• Bridge game play to Internet of Things
• Sensors in shoes, ball, goalie hands
• Analyze Soccer Game
• Spatial and Temporal Processing
• Visualize Game Play
• Activity Heat maps
• Recommend Performance
Improvements
http://srinathsview.blogspot.com/2013/05/solving-debs-2013-grand-challenge-with.html
Bridging Game Play to IoT
6
Temporal Tracking
Spatial Tracking
• Each event includes the
location (x,y,z), time stamp,
velocity and acceleration
• Throughput 15,000 events per
second
Analyze Game
7
๏ BALL POSSESSION
STATE DIAGRAM
๏ SHOT ON GOAL
STATE DIAGRAM
Analyze Game
8
๏ RUNNING LOCATION STATES
9
Solution Architecture
Batch Analytic Foundation
10
Business Activity Monitor
Realizing Real-time Analytics
๏ Processing Data on the fly, while
storing a minimal amount of
information and responding fast
(from <1 ms to few seconds)
๏ Idea of Event streams
๏ A series of events in time
๏ Enabling technologies
๏ Stream Processing (Storm)
๏ Complex Event processing
(Siddhi)
Complex Event Processing
Complex Event
Processing
Stream Processing
•SQL like language
•Supports powerful
temporal operators
(e.g. windows, event
patterns)
•Focus on speed
•Harder to scale
•e.g. WSO2 CEP,
Streambase, Esper
•Operators connected
in a network, but you
have to write the logic
•Distributed by design
•Focus on reliability (do
not loose messages),
has transactions
•e.g. Storm, S4
Complex Event Processing Operators
๏ Filters or transformations (process a single event)
๏ from Ball[v>10] select .. insert into ..
๏ Windows + aggregation (track window of events: time, length)
๏ from Ball#window.time(30s) select avg(v) ..
๏ Joins (join two event streams to one)
๏ from Ball#window.time(30s) as b join Players as p on p.v
< b.v
๏ Patterns (state machine implementation)
๏ from Ball[v>10], Ball[v<10]*,Ball[v>10] select ..
๏ Event tables (map a database as an event stream)
๏ Define table HitV (v double) using .. db info ..
Usecase 1: Running Analysis
๏ Detect when speed crosses threshold limits
define partition player by Players .id;
from s = Players [v <= 1 or v > 11] ,
t = Players [v > 1 and v <= 11]+ ,
e = Players [v <= 1 or v > 11]
select s.ts as tsStart , e.ts as tsStop ,s.id as
playerId ,
‘‘trot" as intensity , t [0].v as instantSpeed
,
(e.ts - s.ts )/1000000000 as unitPeriod
insert into RunningStats partition by player;
Usecase 2: Ball possession
๏ Ball possession
๏ Defined as possessing the ball from time you hit it until someone else hit it or ball
leaves the ground)
Usecase 3: Heatmap of Activity
๏ Show where actions happened (via cells defined by a grid of
64X100 etc.), need updates once every second
๏ Can solved via cell change boundaries, but does not work if one player stays more than
1 sec in the same cell. So need to join with a timer.
Usecase 4: Detect kicks on goal
๏ Main Idea: Detect kicks on
the ball, calculate direction
after 1m, and keep giving
updates as long as it is in
right direction
Results for DEBS Scenarios
Big Data and IoT Changes the
Game
๏ Real-time Status Dashboard
๏ Player game play
๏ Momentum indicators
๏ Predictions about the next move
๏ Improvement Dashboard
๏ Study of game and players effectiveness
๏ Monitor player health and body functions
Success Target
Photo by John Trainoron Flickr
http://www.flickr.com/photos/trainor/2902023575/, Licensed
Situational Context: Traffic Example
{incidents: [
{impacting: false,
eventCode: 0,
iconURL: "http://...",
lat: 38.743999,
shortDesc: "Exit ramp closed on I 95...",
type: 1,
severity: 0,
ullDesc: "In NEWINGTON exit ramp closed
on I-95 ...",
startTime: "2010-02-21T00:14:07",
lng: -77.188004,
id: "368598263",
endTime: "2010-02-27T05:04:19"
},
Image source: http://www.directoryofnewyorkcity.com/blog/2009/05/how-to-find-parking-in-new-york-
city/
Accelerate
interactions
inside and
outside the
organization
Reduce
interaction
friction and cost
Increase
engagement and
enhance
productivity
Sense
business activity
and
automatically
adapt
Become a More Connected Business
http://wso2.com/landing/enabling-the-connected-business
Connected Business
Reference Architecture
Conceptual Architecture
CEP High Availability
Siddhi Storm Bolt (next Major Release)
๏ We have written a Siddhi bolt that would let users run
distributed Siddhi Queries using Storm
SiddhiBolt siddhiBolt1 = new SiddhiBolt( .. siddhi queries .. );
SiddhiBolt siddhiBolt2 = new SiddhiBolt( .. siddhi queries .. );
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("source", new PlayStream(), 1);
builder.setBolt("node1", siddhiBolt1, 1)
.shuffleGrouping("source", "PlayStream1");
..
builder.setBolt("LeafEacho", new EchoBolt(), 1)
.shuffleGrouping("node1", "LongAdvanceStream");
..
cluster.submitTopology("word-count", conf, builder.createTopology());
30
Contact us !
32
About the Presenter
๏ Chris Haddad
๏ VP Platform Evangelism
๏ Learn more about me
๏ www.linkedin.com/in/cobiacomm/
๏ Follow me
๏ @cobiacomm on Twitter
๏ http://blog.cobia.net/cobiacomm
๏ On Google+ too

Big data streams, Internet of Things, and Complex Event Processing Improve Soccer Team Performance

  • 1.
    Last Updated: Jan.2014 VP Platform Evangelism @cobiacomm on Twitter http://blog.cobia.net/cobiacomm/ Chris Haddad Tracking a Soccer Game with Big Data
  • 2.
    Big Data ImprovesTeam Performance ๏ Sports Teams ๏ Your Business Team2
  • 3.
    Improvement Requires AFeedback Loop 3
  • 4.
    Internet of Things (IoT)Bridges Physical and Virtual Worlds ๏ Sensors, actuators everywhere ๏ Internet connectivity Improvement = IoT, Apps, and APIs ๏ Visualization Apps ๏ Control APIs
  • 5.
    DEBS Challenge • Bridgegame play to Internet of Things • Sensors in shoes, ball, goalie hands • Analyze Soccer Game • Spatial and Temporal Processing • Visualize Game Play • Activity Heat maps • Recommend Performance Improvements http://srinathsview.blogspot.com/2013/05/solving-debs-2013-grand-challenge-with.html
  • 6.
    Bridging Game Playto IoT 6 Temporal Tracking Spatial Tracking • Each event includes the location (x,y,z), time stamp, velocity and acceleration • Throughput 15,000 events per second
  • 7.
    Analyze Game 7 ๏ BALLPOSSESSION STATE DIAGRAM ๏ SHOT ON GOAL STATE DIAGRAM
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
    Realizing Real-time Analytics ๏Processing Data on the fly, while storing a minimal amount of information and responding fast (from <1 ms to few seconds) ๏ Idea of Event streams ๏ A series of events in time ๏ Enabling technologies ๏ Stream Processing (Storm) ๏ Complex Event processing (Siddhi)
  • 13.
  • 15.
    Complex Event Processing Stream Processing •SQLlike language •Supports powerful temporal operators (e.g. windows, event patterns) •Focus on speed •Harder to scale •e.g. WSO2 CEP, Streambase, Esper •Operators connected in a network, but you have to write the logic •Distributed by design •Focus on reliability (do not loose messages), has transactions •e.g. Storm, S4
  • 16.
    Complex Event ProcessingOperators ๏ Filters or transformations (process a single event) ๏ from Ball[v>10] select .. insert into .. ๏ Windows + aggregation (track window of events: time, length) ๏ from Ball#window.time(30s) select avg(v) .. ๏ Joins (join two event streams to one) ๏ from Ball#window.time(30s) as b join Players as p on p.v < b.v ๏ Patterns (state machine implementation) ๏ from Ball[v>10], Ball[v<10]*,Ball[v>10] select .. ๏ Event tables (map a database as an event stream) ๏ Define table HitV (v double) using .. db info ..
  • 17.
    Usecase 1: RunningAnalysis ๏ Detect when speed crosses threshold limits define partition player by Players .id; from s = Players [v <= 1 or v > 11] , t = Players [v > 1 and v <= 11]+ , e = Players [v <= 1 or v > 11] select s.ts as tsStart , e.ts as tsStop ,s.id as playerId , ‘‘trot" as intensity , t [0].v as instantSpeed , (e.ts - s.ts )/1000000000 as unitPeriod insert into RunningStats partition by player;
  • 18.
    Usecase 2: Ballpossession ๏ Ball possession ๏ Defined as possessing the ball from time you hit it until someone else hit it or ball leaves the ground)
  • 19.
    Usecase 3: Heatmapof Activity ๏ Show where actions happened (via cells defined by a grid of 64X100 etc.), need updates once every second ๏ Can solved via cell change boundaries, but does not work if one player stays more than 1 sec in the same cell. So need to join with a timer.
  • 20.
    Usecase 4: Detectkicks on goal ๏ Main Idea: Detect kicks on the ball, calculate direction after 1m, and keep giving updates as long as it is in right direction
  • 21.
  • 22.
    Big Data andIoT Changes the Game ๏ Real-time Status Dashboard ๏ Player game play ๏ Momentum indicators ๏ Predictions about the next move ๏ Improvement Dashboard ๏ Study of game and players effectiveness ๏ Monitor player health and body functions
  • 23.
    Success Target Photo byJohn Trainoron Flickr http://www.flickr.com/photos/trainor/2902023575/, Licensed
  • 24.
    Situational Context: TrafficExample {incidents: [ {impacting: false, eventCode: 0, iconURL: "http://...", lat: 38.743999, shortDesc: "Exit ramp closed on I 95...", type: 1, severity: 0, ullDesc: "In NEWINGTON exit ramp closed on I-95 ...", startTime: "2010-02-21T00:14:07", lng: -77.188004, id: "368598263", endTime: "2010-02-27T05:04:19" }, Image source: http://www.directoryofnewyorkcity.com/blog/2009/05/how-to-find-parking-in-new-york- city/
  • 25.
    Accelerate interactions inside and outside the organization Reduce interaction frictionand cost Increase engagement and enhance productivity Sense business activity and automatically adapt Become a More Connected Business http://wso2.com/landing/enabling-the-connected-business
  • 26.
  • 27.
  • 28.
  • 29.
    Siddhi Storm Bolt(next Major Release) ๏ We have written a Siddhi bolt that would let users run distributed Siddhi Queries using Storm SiddhiBolt siddhiBolt1 = new SiddhiBolt( .. siddhi queries .. ); SiddhiBolt siddhiBolt2 = new SiddhiBolt( .. siddhi queries .. ); TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("source", new PlayStream(), 1); builder.setBolt("node1", siddhiBolt1, 1) .shuffleGrouping("source", "PlayStream1"); .. builder.setBolt("LeafEacho", new EchoBolt(), 1) .shuffleGrouping("node1", "LongAdvanceStream"); .. cluster.submitTopology("word-count", conf, builder.createTopology());
  • 30.
  • 31.
  • 32.
    32 About the Presenter ๏Chris Haddad ๏ VP Platform Evangelism ๏ Learn more about me ๏ www.linkedin.com/in/cobiacomm/ ๏ Follow me ๏ @cobiacomm on Twitter ๏ http://blog.cobia.net/cobiacomm ๏ On Google+ too

Editor's Notes

  • #2 Teams gain a competitive edge by analyzing Big Data streams. In this session, Chris will describe how complex event processing (CEP) and MapReduce based technologies can improve soccer team performance. Soccer match activity data captured by embedded sensors were streamed and analyzed to understand how player actions impact soccer play.
  • #6 http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails http://srinathsview.blogspot.com/2013/05/solving-debs-2013-grand-challenge-with.html
  • #25 After we know where we are, and what we are looking to find, situational context may further influence API response and consumer actions. For example, on your way into the conference, did you check the traffic via an API and adjust your route? ----- Image source: http://www.directoryofnewyorkcity.com/blog/2009/05/how-to-find-parking-in-new-york-city/ Real-time traffic map: http://www.mapquestapi.com/traffic/
  • #26 A connected business seamlessly integrates people, process, and data across an extended value chain decreases interaction cost automatically adapts business activity in response to market events Integrated Accelerates interactions inside and outside the organization Accessible Reduces interaction friction and cost Collaborative Increases engagement and enhances productivity   Adaptive Senses business activity and automatically adapts