http://www.virtus-it.com
A trusted partner
Software to model &
build Business
Intelligence &
Big Data Solutions
http://www.virtus-it.com
Table of Contents
• Big Data Overview………..………………………………………………………… 3 - 6
• Step by Step approach to...
http://www.virtus-it.com
10101010101010101010101010101010101010101010101ABC010
1010101010101010101010100101010101010101ABC...
http://www.virtus-it.com
Understanding business performance with Big
Data includes two distinct capabilities:
Managing per...
http://www.virtus-it.com
Telco’s Core IMS
Network Data
Data, Voice & Video
Performance Data
Data, Voice & Video
Performanc...
http://www.virtus-it.com
Telco’s Core IMS
Network Data
Data, Voice & Video
Performance Data
Data, Voice & Video
Performanc...
http://www.virtus-it.com 7
Steps it takes to build
powerful Big Data
solutions!
Solution
Modeling
Model
Deployment
Data
Ac...
http://www.virtus-it.com 8
Solution Modeling
•Logical Data Model
design
•Data standardization
& transformation
modeling
•K...
http://www.virtus-it.com
9
Solution
Modeling
Model
Deployment
Data
Acquisition
Data
Management
Event
Detection
Discovery &...
http://www.virtus-it.com
Solution
Modeling
Model
Deployment
Data
Acquisition
Data
Management
Event
Detection
Discovery &
A...
http://www.virtus-it.com 11
Introducing
StreamCentral
http://www.virtus-it.com
Solution
Modeling
Model
Deployment
Data
Acquisition
Data
Management
Event
Detection
Discovery &
A...
http://www.virtus-it.com 13
1010101010101010
ABCABCABCABCABC
StreamCentral Workbench:
Solution Designer
StreamCentral
Work...
http://www.virtus-it.com
Database
REST/SOAP
API
LDAP
PUSH
API
Data Processing
Engine
Vertica SQL Server
Correlation Engine...
http://www.virtus-it.com 15
Model Pull
Data
Sources with
strong REST,
SOAP & DB
Support Push Data
API
Data
Transformat
-io...
http://www.virtus-it.com 16
Generating insights from data requires context to be
added to the data. This context is a cont...
http://www.virtus-it.com 17
Auto build
and deploy
DB structure
based on
Workbench
Model
Continuous
Pull with
strong REST,
...
http://www.virtus-it.com
18
Solution
Modeling
Model
Deployment
Data Acquisition
Data
Management
Event Detection
Discovery ...
http://www.virtus-it.com
• Industrial strength, enterprise ready with web scale
characteristics - handles extremely large ...
http://www.virtus-it.com
Why StreamCentral?
• Roadmap to Big Data: StreamCentral is the only solution that enables the evo...
http://www.virtus-it.com 21
Making a business case for leveraging Big Data
just got a whole lot easier with StreamCentral
...
http://www.virtus-it.com 22
No immediate plans to go Big on Data?
Planning to work primarily with
structured data?
But wou...
http://www.virtus-it.com
Traditional Data Warehousing
Interrogation of historical data for trend
analysis. Business Intell...
http://www.virtus-it.com
Reporting:-
What did happen ?
Analysis:-
Why did it happen ?
Happens on previously
stored data
(d...
http://www.virtus-it.com
Reporting:-
What did happen ?
Analysis:-
Why did it happen ?
Happens on previously
stored data
(d...
http://www.virtus-it.com 26
An approach to working
with real-time data -
Operational Intelligence
http://www.virtus-it.com
Data Layer
Interfaces
Data Processing
Real-Time Insights
Business Solutions
Operational (User)
In...
http://www.virtus-it.com
Data Layer
Interfaces
Data Processing
Real-Time Insights
Business Solutions
Operational (User)
In...
http://www.virtus-it.com 29
More details on how
StreamCentral works
http://www.virtus-it.com
StreamCentral Workbench Big Data
Solutions Modeler - Inputs
• Data Sources
• Push/Pull
• Data tra...
http://www.virtus-it.com
StreamCentral Big Data Server - Output
• Database structure automatically created, updated and ma...
http://www.virtus-it.com 32
Sensors
Weather
Enterprise
Applications
Data Visualization
(Reporting, Analytics,
Dashboards)
...
http://www.virtus-it.com 33
builds two distinct types of analytical data marts
360o Data Marts** Real-Time Data Marts
• De...
http://www.virtus-it.com 34
StreamCentral Real-Time Operational Intelligence
• Data Sources
• Import initial data load
• P...
http://www.virtus-it.com 35
StreamCentral 360o Data Aggregation**
• Data Sources
• Import initial data load
• Pull data fr...
http://www.virtus-it.com 36
Data formats supported :
• XML
• JSON
• String
Data Sources supported :
• Database
• Microsoft...
http://www.virtus-it.com 37
Transformation Description
LTRIM Removes all white spaces from the left
RTRIM Removes all whit...
http://www.virtus-it.com 38
StreamCentral
Physical Architecture
http://www.virtus-it.com 39
StreamCentral Collector
Windows Server 2012, .Net Framework 4.5, MSMQ
StreamCentral Stream Pro...
http://www.virtus-it.com 40
1 server for StreamCentral Components:
Collector, Stream Processing Engine, Correlation Engine...
http://www.virtus-it.com
How does StreamCentral
fit within your enterprise
technology architecture?
41
http://www.virtus-it.com
Data Sources Method of Access
StreamCentral -
Read data from
Application
Application - Read data
...
http://www.virtus-it.com
Sensors
Weather
Devices
Traffic
Custom
ApplicationsMainframe
Business Services
Enterprise Service...
http://www.virtus-it.com
Sensors
Weather
Devices
Traffic
ERP
Custom
ApplicationsMainframe
Business Services
Enterprise Ser...
http://www.virtus-it.com
Thank you
for your time
Contact us for a demonstration
Stephen Wells
CEO - Virtus IT Ltd
E: steph...
Upcoming SlideShare
Loading in...5
×

StreamCentral for the IT Professional

433

Published on

This presentation gives an overview of StreamCentral technology targeted for IT professionals. StreamCentral is software to model and build Big Data Solutions. StreamCentral consists of a Big Data Solutions Modeler that not only makes it easy to model traditional BI/DW and Big Data solutions but also auto deploys the model on the latest innovations in Big Data Management solutions (like HP Vertica and SQL Server Parallel Data Warehouse). StreamCentral Big Data Server executes the model definition in real-time. StreamCentral drastically reduces the time to market, risk and cost associated with building traditional BI/DW and Big Data solutions!

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
433
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

StreamCentral for the IT Professional

  1. 1. http://www.virtus-it.com A trusted partner Software to model & build Business Intelligence & Big Data Solutions
  2. 2. http://www.virtus-it.com Table of Contents • Big Data Overview………..………………………………………………………… 3 - 6 • Step by Step approach to building Big Data Solutions……………….. 7 - 10 • StreamCentral Introduction……………………………………..………….... 11 - 21 • Keeping it structured? –Extending current DW/BI investments………. 22 - 25 • An approach to building Operational Intelligence solutions………….... 26 - 28 • StreamCentral additional details……………………………………..………….... 29 - 37 • StreamCentral physical architecture……………………………..………….... 38 - 40 • How StreamCentral fits in an enterprise technology architecture…… 41 - 44 2
  3. 3. http://www.virtus-it.com 10101010101010101010101010101010101010101010101ABC010 1010101010101010101010100101010101010101ABCABCABCABC 1 0 1 0 1 0 1 0 1 0 1 0 3 BIGData Today With Big Data Custom Application ERP • Analysis of structured data from internal applications • Data sets updated using batch processes • Traditional BI & Data Warehousing • Traditional BI and data warehousing extended to include structured and unstructured data from internal and external applications processed in real-time or in batch. • Ability to predict events as well as analyze historical associations in wide sets of data for patterns and trends. Data Management Discovery & Analysis Event Detection Big Data Solution Modeling & Model Deployment Stream Processing & Batch Data Acquisition Blocks for building Big Data Solutions ERP Internal & External Applications Data Stores 1 0 1 0 1 0 Real-Time + Batch Big Data Processing Layer Real-time event data in operational applications Pattern, trend and association analysis Understand the connections in a wide variety of data that impact business performance and use that knowledge to deliver exceptional business results
  4. 4. http://www.virtus-it.com Understanding business performance with Big Data includes two distinct capabilities: Managing performance by analyzing internal and external, structured and unstructured data for patterns and associations collected over time • Customer segmentation based on buying history patterns and finding associations with population, census and twitter data to develop marketing strategy • Web analytics to improve marketing campaigns and relevant content • Sales pipeline analysis compared to industry data to understand the right goals to set • Cash flow analysis to make capital investment decisions considering external variables Managing performance by analyzing real- time data for day to day events – Operational Intelligence • What is the current workload? • Is staff available and working on high priority work? • What factors are impacting customer experience right now? • What processes are taking longer than expected? 4 A few example scenarios: A few example scenarios:
  5. 5. http://www.virtus-it.com Telco’s Core IMS Network Data Data, Voice & Video Performance Data Data, Voice & Video Performance Data Data from Telco Towers Weather Data Traffic IncidentsPopulation Data Data Stream weatherundergrou nd MapquestUSA Today Census data Sources of real time streaming data from networks, devices, services and other internal applications External sources of data that add understanding of what’s happening when events are detected Example Big Data Solutions: Telco Network Test New Service – Investment Planning Adaptive Bit Rate – Video Streaming QoE 360o Customer QoE for 1st Level customer service Video QoE for IPTV Business Solutions 5
  6. 6. http://www.virtus-it.com Telco’s Core IMS Network Data Data, Voice & Video Performance Data Data, Voice & Video Performance Data Data from Telco Towers Weather Data Traffic IncidentsPopulation Data Data Stream weatherunderground INRIXUSA Today Census data Example Big Data Scenario : Utilities - Water Sources of real time data relating to your business Sources of BIG DATA relevant to your business VIBRATION SENSOR ENERGY HARVESTING WATER MAIN PRESSURE SCADA NETWORK 6
  7. 7. http://www.virtus-it.com 7 Steps it takes to build powerful Big Data solutions! Solution Modeling Model Deployment Data Acquisition (streaming or batch, internal or external, structured or unstructured) Data Management Event Detection Discovery & Analysis Big Data Solution Lifecycle Start here
  8. 8. http://www.virtus-it.com 8 Solution Modeling •Logical Data Model design •Data standardization & transformation modeling •Key Performance Indicator modeling via business rules •Dimensional modeling •Historical Data Mart Modeling •Event detection modeling via business rules •Real-time analytics data mart modeling Model Deployment •Physical Design Implementation •Physical deployment of dimensional model •Database deployment •Physical deployment of data marts •Rules deployment Data Acquisition •Data from internal data sources •Data from external sources •Streaming data •Batch data •Structured Data •Unstructured Data •Data transformation •Data standardization Data Management •Structured Data Storage •Unstructured Data Storage •Scalability •Performance Event Detection •Detecting events on streaming data •Alerting •Integration with operational applications Discovery & Analysis •Information Discovery •Data Classification •Analytics •Querying •Visualization Solution Modeling Model Deployment Data Acquisition Data Management Event Detection Discovery& Analysis Big Data Solution Lifecycle – Tasks Detailed
  9. 9. http://www.virtus-it.com 9 Solution Modeling Model Deployment Data Acquisition Data Management Event Detection Discovery & Analysis 1. Hadoop - MapReduce 2. MPP Columnar Databases like Neteeza, Vertica, ParStream 3. NoSQL – MongoDB, Cassandra 4. Evolution of traditional RDBMS to support column indexes SQL Server Big Data Innovations in Data Management Big Data Innovations in Discovery & Analysis Where has the innovation been in Big Data? The last few years have seen lots of innovation in Data Management as well as Discovery and Analysis
  10. 10. http://www.virtus-it.com Solution Modeling Model Deployment Data Acquisition Data Management Event Detection Discovery & Analysis Big Data Lifecycle But, where is the innovation in these areas? • Fragmented, point use or lack of industry strength technology to aid in Design, Model Deployment, Data Acquisition and Event Detection makes it difficult, time consuming and specialist resource intensive to build Big Data Solutions • What is the use of having scalable platforms that can store and manage this data and tools that can deliver incredible visualizations when the effort to get the data right is still a problem as it has always been?
  11. 11. http://www.virtus-it.com 11 Introducing StreamCentral
  12. 12. http://www.virtus-it.com Solution Modeling Model Deployment Data Acquisition Data Management Event Detection Discovery & Analysis Big Data Solution Life cycle 1. StreamCentral Solutions Designer makes it easy to model traditional BI/DW and Big Data solutions 2. Builds and deploys model on HP Vertica or Microsoft SQL Server 3. Adds context by connecting all streaming and static data to time, location and entities 4. StreamCentral Big Data Server, horizontally scalable, executes the model definition in real-time 5. StreamCentral drastically reduces time to market, risk and cost in building Big Data solutions! Software to design & build BI & Big Data SolutionsStreamCentral enables you to quickly move from a blank sheet of paper to a production system, comprehensive and powerful that can be delivered without a large investment in specialist skills.
  13. 13. http://www.virtus-it.com 13 1010101010101010 ABCABCABCABCABC StreamCentral Workbench: Solution Designer StreamCentral Workbench: Model DeploymentData Collection Data Processing Correlation Data Publishing Data Security StreamCentral Big Data Server StreamCentral has three main components: 1. Use the Workbench Designer to define source data, entities, rules for monitoring conditions, events and data correlation, analytical models and knowledgebase 2. Workbench Model Deployment configures, builds and deploys the model on top of HP Vertica or Microsoft SQL Server 3. Big Data Server executes the defined model in real-time 1 2 3
  14. 14. http://www.virtus-it.com Database REST/SOAP API LDAP PUSH API Data Processing Engine Vertica SQL Server Correlation EngineCollector Data Publishing, Access and Security • Capture data • Validate data • Prepare data • Apply transformations • Perform calculations • Determine conditions & KPIs • Identify & build dimensions • Identify alerts • Correlate incoming data based on defined rules • Detect events based on correlated data • Update fact data • Update entity & dimension data • Update analysis collections • Update event collections • Manage data level security Data Acquisition – Push / Pull data from variety of sources Design data transformations Conditions & KPI modeler via rules builder Real-time data correlation Event detection via rules builder Real-time data mart designer 360o data mart designer Define entities and Import Entity Data Dimension modeler Data Security designer StreamCentral Big Data Server StreamCentral Workbench: Big Data Solution Designer Meta Data Create Database Structure Add Context StreamCentral Workbench: Big Data Solution Deployment
  15. 15. http://www.virtus-it.com 15 Model Pull Data Sources with strong REST, SOAP & DB Support Push Data API Data Transformat -ion Model Entities & import static data Dimension modeler Time & Location Standard- ization Conditions & KPI modeler Correlation Modeler Event Detection rules on real-time data Real-time & Historical analytics Data mart modeler • Software targeted to be used by IT and non IT people to design and build Big Data solutions • Can work with batch data (as in traditional Business Intelligence) or real-time streams (as in Operational Intelligence) • Workbench lets analysts model all necessary steps in building a Big Data Solution • Data Pull/Push • Model Transformations • Model Entities (like customers, patients, products), import static entity data and define entity relationships to source data • Shared dimensions across data • Condition modeler via business rules to monitor specific sets of conditions in batch or streaming data • Evaluate different entities with different sets of conditions as data flows in • Specify rules to model how to correlate data streams in real-time • Event detection • Model data marts that aggregate the right data for association and pattern analysis StreamCentral Workbench : Software to design traditional BI/DW & Big Data Solutions Workbench
  16. 16. http://www.virtus-it.com 16 Generating insights from data requires context to be added to the data. This context is a continuous thread that connects all types of data throughout the Big Data Solution lifecycle. Four typical examples of context.. Insight Who (entities like customer, patient) When (time) Where (location) What (streaming & static data correlation) • StreamCentral automatically builds and maintains time and location dimensions • Entities can be created and defined in StreamCentral • All data in StreamCentral is continuously and automatically connected to time, location and defined entities • Resultant real-time events and analytical data marts automatically inherit this context without need for any programming or development work • This increases the impact and value of collected data Converting data to insights by continuously adding context
  17. 17. http://www.virtus-it.com 17 Auto build and deploy DB structure based on Workbench Model Continuous Pull with strong REST, SOAP & DB Support Push Data API for streaming sources Time & Location Standard- ization Monitor conditions Event detection Build data marts & continuously update new data In-Memory Operations Distributed Architecture MPP Support StreamCentral Big Data Server: Software that runs Big Data Solutions • Extends your Business Intelligence strategy by easily incorporating external data sets • Introduces integration of real-time data for event insight to your organization • Auto-builds database schema (facts, dimensions, entities, flat tables and more) • By default, standardizes all incoming data by connecting it to auto created time and location dimensions • Builds event data marts and continuously loads data • Builds real-time data marts to help in understanding associations in data Continuously loads these analysis data marts • Deliver real-time event insights to new or existing operational applications • Significantly reduces IT overhead in building Big Data solutions Big Data Server
  18. 18. http://www.virtus-it.com 18 Solution Modeling Model Deployment Data Acquisition Data Management Event Detection Discovery & Analysis Bringing it together: Building Big Data Solutions with StreamCentral and partner solutions 1. MPP Columnar Databases : Vertica, ParStream 2. Microsoft SQL Server StreamCentral Big Data Server StreamCentral Workbench: Model Deployment StreamCentral Workbench: Big Data Solutions Modeler Tableau Software, Microsoft PowerView StreamCentral Big Data Server
  19. 19. http://www.virtus-it.com • Industrial strength, enterprise ready with web scale characteristics - handles extremely large amounts of data • Uses in-memory processing for high speed • Next generation distributed architecture – allows you to run on any number of commodity hardware • Built in redundancy at every layer for high availability • Easy to use tools to monitor and manage StreamCentral • Built on Microsoft technology that most enterprises already have invested in • Runs on best of breed and latest database technology from Microsoft SQL Server and HP Vertica Choose database from: 19
  20. 20. http://www.virtus-it.com Why StreamCentral? • Roadmap to Big Data: StreamCentral is the only solution that enables the evolution of current practices in Business Intelligence and Data warehousing to now include external data, event monitoring and real-time insights • No programming solution modeler: StreamCentral takes a solution approach – designing and modeling shifts to analysts versus everything being done by developers or programmed from scratch • Continuously adds context to data: Any kind of data that is streamed to StreamCentral, pulled in near real-time or imported via batch is continuously and automatically connected to time, location and defined entities. This significantly reduces risk, time and cost associated with building BI/DW and Big Data solutions • Reduced dependency on specialist skills: No in-depth knowledge needed on HP Vertica or SQL Server development as StreamCentral builds, deploys and maintains all internal structures in those environments automatically • Plays well: Is standards based and agnostic to existing enterprise technologies • Adaptable: Everything created in StreamCentral can be modified. Makes it easy to adapt the Big Data solution to changing needs of the business 20
  21. 21. http://www.virtus-it.com 21 Making a business case for leveraging Big Data just got a whole lot easier with StreamCentral 70% Time taken to build Big Data solutions is drastically reduced by using StreamCentral 60% Cost of building Big Data solutions is drastically reduced by using StreamCentral In addition, StreamCentral reduces risk, data quality issues, specialist skillsets requirements and complexity in building traditional Business Intelligence/Data Warehousing or Big Data solutions
  22. 22. http://www.virtus-it.com 22 No immediate plans to go Big on Data? Planning to work primarily with structured data? But would like to deliver additional insights by enhancing your existing investments in Business Intelligence and Data Warehousing?
  23. 23. http://www.virtus-it.com Traditional Data Warehousing Interrogation of historical data for trend analysis. Business Intelligence applications deliver analytics or reports to management for performance analysis On-Demand Business Intelligence Update Data Warehouse continuously with real-time data. Provides the ability to analyze data updated in real-time Operational Intelligence Allows organizations to monitor fast moving data for key indicators and events and immediately act on these insights, through manual or automated actions Reporting:- What did happen ? Analysis:- Why did it happen ? Happens on previously stored data (data at rest) Happens on real-time streaming data (data in-flight) Solution value to businessLower Higher PerceivedComplexityHigherLower Event Monitoring:- What is happening ? Predictive Analytics:- What will happen ? Traditional Data Warehousing Solutions On-Demand BI Operational Intelligence 23 Keeping it structured – A roadmap to extend current investments in BI/DW
  24. 24. http://www.virtus-it.com Reporting:- What did happen ? Analysis:- Why did it happen ? Happens on previously stored data (data at rest) Happens on real-time streaming data (data in-flight) Solution value to businessLower Higher PerceivedComplexityHigherLower Event Monitoring:- What is happening ? Predictive Analytics:- What will happen ? Traditional Data Warehousing Solutions On-Demand BI Operational Intelligence Most organizations have traditionally invested in this area In most companies, the scope of understanding business performance is limited to historical analysis and rarely includes real-time understanding of key events that impact day to day operational processes Keeping it structured – A roadmap to extend current investments in BI/DW
  25. 25. http://www.virtus-it.com Reporting:- What did happen ? Analysis:- Why did it happen ? Happens on previously stored data (data at rest) Happens on real-time streaming data (data in-flight) Solution value to businessLower Higher PerceivedComplexityHigherLower Event Monitoring:- What is happening ? Predictive Analytics:- What will happen ? Traditional Data Warehousing Solutions On-Demand BI Operational Intelligence Most organizations have traditionally invested in this area StreamCentral’s area of focus 25 Keeping it structured – A roadmap to extend current investments in BI/DW
  26. 26. http://www.virtus-it.com 26 An approach to working with real-time data - Operational Intelligence
  27. 27. http://www.virtus-it.com Data Layer Interfaces Data Processing Real-Time Insights Business Solutions Operational (User) Internal Applications and Data Sets External Data Connections to existing architecture for tapping data & data streams APIs Databases Enterprise Service Bus Messages Push Streaming Data |Pull Data |Format | Standardize | Transform | Measure | Correlate | Event Detection | Rules Engine | In-Memory Processing Real-Time Streaming Analytics Real-Time Event Notification Historical data that supports pattern &trend analytics. New insights are added in real time Customer Experience Continuous Improvement Day to day insights and actions delivered in multiple mediums to many users KPIs Complaints Brand – Protection 1 2 3 4 5 6 ! Access to right information at the right time along with knowledge base of actions to perform Operational Intelligence practices are similar to traditional Data Warehousing practices 27
  28. 28. http://www.virtus-it.com Data Layer Interfaces Data Processing Real-Time Insights Business Solutions Operational (User) Internal Data Sets External Data Connections to existing architecture for tapping data & data streams APIs Databases Enterprise Service Bus Messages Push Streaming Data |Pull Data |Format | Standardize | Transform | Measure | Correlate | Event Detection | Rules Engine | In-Memory Processing Real-Time Streaming Analytics Real-Time Event Notification Historical data supporting pattern &trend analytics. New insights added in real time Customer Experience Continuous Improvement Day to day insights and actions delivered in multiple mediums to many users KPIs Complaints Brand – Protection 1 2 3 4 5 6 ! Access to right information at the right time along with knowledge base of actions to perform Focus of StreamCentral 28
  29. 29. http://www.virtus-it.com 29 More details on how StreamCentral works
  30. 30. http://www.virtus-it.com StreamCentral Workbench Big Data Solutions Modeler - Inputs • Data Sources • Push/Pull • Data transformations • Define and import entity data • Modeling • Rules for monitoring conditions in data • Correlation rules to identify related records across data sources in real-time • Rules for detecting events • Common dimension modeling • Data Mart modelers • Support for Real-time • Correlation rules to identify related records across data sources in real-time • Rules for detecting events • Configure real-time data marts • 360o data aggregation** • Define data relationships across data sources • Configure 360o data marts • Data level security** 30** Coming Q3 2013
  31. 31. http://www.virtus-it.com StreamCentral Big Data Server - Output • Database structure automatically created, updated and managed in Big Data databases like HP Vertica or SQL Server by StreamCentral. • The StreamCentral database automatically builds time and location dimensions, fact tables, other dimension tables, standardizes facts across data sources to the one time and location dimension as well as connects facts to KPIs. StreamCentral also auto-loads this database from various data sources into Big Data databases like HP Vertica or SQL Server • Real-time event notification that can be consumed by operational applications via an API** • Real-time event alerts • Data marts that are automatically created, updated and managed by StreamCentral. The data marts denormalize data into a single table facilitating faster querying and analysis of data • Real-time analytical data marts built that aggregates events and data across data sources to better understand conditions that influence events • Real-time event data marts that bring together all relevant information for a single event • 360o data marts for association and pattern analysis** 31** Coming Q3 2013
  32. 32. http://www.virtus-it.com 32 Sensors Weather Enterprise Applications Data Visualization (Reporting, Analytics, Dashboards) Correlates Data Generates Key Performance Indicators Uncovers Events Consumes real- time or static data OR Pulls data from various data sources and applies transformation and standardization rules Model Deployment Auto-builds database schema Auto-loads database Builds and continuous loads data to event data marts Builds and continuous loads Analysis Collections Publishes event data that can be subscribed by Operational Applications Devices Auto-build Database Schema 360o Data marts and real-time data marts Event Data Marts for every event along with its context as denormalized flat tables StreamCentral Push Push Massively Parallel Processing Systems - Vertica RDBMS – MS SQL Server Publish event data to operational applications – Web, mobile or desktop StreamCentral Workbench – Big Data Solutions Modeler Collate Raw Data (Push/Pull) – Real-Time or Static Model data standardization and transformation rules Define business entities and connect raw data to business entities Model Dimensions Model conditions to monitor across data sources Assign different conditions to different entities Model Correlation Rules Model events and specify context to add to events Model analytical data marts auto built by StreamCentral StreamCentral Big Data Server Enterprise Applications API Traffic API API API
  33. 33. http://www.virtus-it.com 33 builds two distinct types of analytical data marts 360o Data Marts** Real-Time Data Marts • Defined: Easily bring together and aggregate data across data sources to get 360o insight. Analyze associations in data to determine patterns that impact business performance • Define data mart structure by choosing the right set of attributes from data sources, KPIs, attributes from entities, and dimensions in the Workbench • . StreamCentral auto-builds the data mart • Standardize data across time and location • Update data mart at pre-defined intervals StreamCentral Data Marts are denormalized flat tables – Why? • Defined: Aggregate real-time events and bring together data across data sources to analyze conditions that existed when events are detected • Standardize data across time and location • Define data mart structure by choose the right set of attributes from data sources, KPIs, events, attributes from entities, and dimensions . StreamCentral auto-builds the data mart • Once data gets correlated in real-time data mart gets updated with appropriate insights • Technology advancements in columnar data stores, bit map indexes, column indexes make it possible to scan and query large amounts of data in a single table • Takes advantage of distributed architectures to scale out using commodity software • Supports : • SQL Server columnar indexes • Vertica MPP ** Coming Q3 2013
  34. 34. http://www.virtus-it.com 34 StreamCentral Real-Time Operational Intelligence • Data Sources • Import initial data load • Push data to StreamCentral API • Pull data from data sources at defined intervals • Apply transformations on the data in flight • SQL Server, Oracle, My SQL, REST API, SOAP Web Service, LDAP • Auto connects data to time and location dimension • Model entities. Connect data sources to entities • Model measures and KPIs • Model standard dimensions • Model real-time correlation rule (to identify related records across data sources in real time) • Model Events • Events based on real-time correlation rule • Event Data Mart (automatically gets created when event is detected) • Requires real-time correlation • Brings together all data across data sources that were captured at the time the event was detected • Model Real-Time Data Marts • Requires a real-time correlation rule • Update real-time data mart with streaming correlated data • Define attributes that make up the real-time data mart definition. Select subsets of information from : specific attributes from data sources, KPIs, events, entity attributes, dimensions, time and location • Edit real-time data mart definition
  35. 35. http://www.virtus-it.com 35 StreamCentral 360o Data Aggregation** • Data Sources • Import initial data load • Pull data from data sources at defined intervals • Apply transformations on the data • SQL Server, Oracle, My SQL, REST API, SOAP Web Service, LDAP • Auto connects data to time and dimension location • Model entities. Connect data sources to entities • Model measures and KPIs • Model standard dimensions • Model 360o Data Marts • Model 360 view query (define relationships across data sources to aggregate data) • Schedule batch update interval (typically hours) • Define attributes that make up the analysis collection. Select subsets of information from : specific attributes from data sources, KPIs, entity attributes, dimensions, time and location • Edit and update data mart definition • Define data level security ** Coming Q3 2013
  36. 36. http://www.virtus-it.com 36 Data formats supported : • XML • JSON • String Data Sources supported : • Database • Microsoft SQL Server • Oracle • My SQL • REST API • SOAP API • LDAP • Specify transformation rules to data that is applied to data in flight • Specify parameters when calling APIs • Auto fills location parameters based on location data stored in the database about entities • Auto creates tables in the backend database for data source data Pull Data from Applications Push data to StreamCentral • StreamCentral REST API available to stream data to StreamCentral – stream data from agents, sensors, probes, devices • Specify transformation rules to data that are applied to the data in flight • Auto creates the tables in the backend database for source data StreamCentral Databases • Supports Microsoft SQL Server and HP Vertica • Auto creates data structures in the database for source data • Auto creates fact tables, dimensions, flat tables for event analysis and flat tables for pattern and association analysis • Data level security StreamCentral Analytics • Device friendly visualization • Powerful portfolio of visualization tools • Ability to embed in custom applications • In-memory operations for fast querying StreamCentral Reports • Role based security • Subscribe to reports • Ability to embed in custom applications • Export reports to various formats
  37. 37. http://www.virtus-it.com 37 Transformation Description LTRIM Removes all white spaces from the left RTRIM Removes all white spaces from the right Ignore Space Removes all white spaces from left, middle or right Ignore Special Characters Returns string after ignoring all special characters Contains Search for specific characters Substring Extract a substring from a string Left Removes the left part of a character string Right Removes the right part of a character string Replace Replaces specified string with another string Startswith Search for a starting character Endswith Search for an ending character DoesNotContain Search for specific characters Remove Remove specified characters or words from string Range Search for a range RoundOff Rounds off decimal value to a specific length StreamCentral Transformations • Easy to use transformations • Multiple transformations can be executed on one attribute
  38. 38. http://www.virtus-it.com 38 StreamCentral Physical Architecture
  39. 39. http://www.virtus-it.com 39 StreamCentral Collector Windows Server 2012, .Net Framework 4.5, MSMQ StreamCentral Stream Processing Engine Windows Server 2012, .Net Framework 4.5,, MSMQ StreamCentral Stream Correlation Engine Windows Server 2012, .Net Framework 4.5,, MSMQ StreamCentral Data Engine Windows Server 2012, .Net Framework 4.5,, MSMQ All components can run on one machine Every component can run on more than one machine StreamCentral Cache Cluster Windows Server 2012, .Net Framework 4.5, AppFabric StreamCentral Metadata database Windows Server 2012, Microsoft SQL Server2008 R2 or Microsoft SQL Server 2012 StreamCentral Database and data marts Option 1: Windows Server 2012, Microsoft SQL Server2008 R2 or Microsoft SQL Server 2012 or SQL Server Parallel Data Warehouse Option 2 Linux, HP Vertica StreamCentral Analytics Windows Server 2012, Tableau Software StreamCentral Physical Architecture and Software Requirements
  40. 40. http://www.virtus-it.com 40 1 server for StreamCentral Components: Collector, Stream Processing Engine, Correlation Engine, Data Engine Characteristics of this server : Processor dependent therefore the higher the number of cores the better, medium cache and low disk storage Software: Windows Server 2012, .Net Framework 4.5, MSMQ 1 server for cache Hardware characteristics: : Cache dependent therefore more memory the better. Medium CPU and low disk storage Software : Windows Server 2012, .Net Framework 4.5, AppFabric 1 server for StreamCentral Meta Data Database, data mart storage and reporting Hardware Characteristics:: High CPU, High Memory and High Storage Software : Windows Server 2012, SQL Server 1 server for StreamCentral Meta Data Database and reporting Hardware Characteristics:: Medium CPU, Medium Memory and High Storage Software : Windows Server 2012, SQL Server OR 1 server for StreamCentral data marts Hardware Characteristics:: High CPU, High Memory and High Storage Software : Linux, HP Vertica + StreamCentral suggested minimum system configuration
  41. 41. http://www.virtus-it.com How does StreamCentral fit within your enterprise technology architecture? 41
  42. 42. http://www.virtus-it.com Data Sources Method of Access StreamCentral - Read data from Application Application - Read data by subscribing to StreamCentral Real-Time Event API Application - Read data by querying StreamCentral database Enterprise Applications X real-time X Using Web Service or REST API X real-time Using database query X Enterprise Service Bus X real-time X via Web Service or REST API X real-time via subscribing to messages X real-time Enterprise Data Warehouse via database query X X Point databases via database query X X LDAP via database query X External Data Sources via Web Service or REST API X real-time 42
  43. 43. http://www.virtus-it.com Sensors Weather Devices Traffic Custom ApplicationsMainframe Business Services Enterprise Service Bus - Messaging / Mediation / Orchestration / Security Business Process Business Process Business Process Composite Application Composite Application Composite Application Auto-build Database Schema Analysis Collections – Data marts as denormalized flat tables Event Collections – Data Marts for every event along with its context as denormalized flat tables StreamCentral Engine StreamCentral Workbench Collate Raw Data (Push/Pull) Standardize Data Define Business Rules Define Correlation Define events Define analytical data marts auto built by StreamCentral Historical Analysis Real-time event data published to operational applications and dashboards Massively Parallel Processing Systems - Vertica Columnar databases with Bit Map indexes – ParStream RDBMS – MS SQL Server StreamCentral as part of an Enterprise Service Bus architecture API ERP Push / Pull 43
  44. 44. http://www.virtus-it.com Sensors Weather Devices Traffic ERP Custom ApplicationsMainframe Business Services Enterprise Service Bus - Messaging / Mediation / Orchestration / Security Business Process Business Process Business Process Composite Application Composite Application Composite Application Auto-build Database Schema Analysis Collections – Data marts as denormalized flat tables Event Collections – Data Marts for every event along with its context as denormalized flat tables StreamCentral Engine StreamCentral Workbench Collate Raw Data (Push/Pull) Standardize Data Define Business Rules Define Correlation Define events Define analytical data marts auto built by StreamCentral Historical AnalysisEnterprise Business Intelligence System Massively Parallel Processing Systems - Vertica Columnar databases with Bit Map indexes – ParStream RDBMS – MS SQL Server StreamCentral and Enterprise BI as part of an Enterprise Service Bus architecture Real-time event data published to operational applications and dashboardsAPI Push Pull Push / Pull 44
  45. 45. http://www.virtus-it.com Thank you for your time Contact us for a demonstration Stephen Wells CEO - Virtus IT Ltd E: stephen.wells@virtus-it.com M: +44 77 111 30879 Raheel Retiwalla CTO - Virtus IT Ltd E: raheel.retiwalla@virtus-it.com M: +1 617 901 8370 A trusted partner 45
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×