Real- Time Analytics – Process Automation
•
0 likes•345 views
This document discusses real-time analytics using big data. It describes the three V's of big data - volume, velocity, and variety. Examples are given for each V and how real-time analytics could be used for different scenarios involving large amounts and speeds of diverse data types. The document also mentions two additional V's - viability and value. It recommends MongoDB and Syslog-ng as technologies to consider for real-time analytics and envisions how real-time big data analytics could benefit DevOps, agile methodologies, and infrastructure/resource management in the future.
Report
Share
Report
Share
Download to read offline
Recommended
Voxxed days thessaloniki 21/10/2016 - Streaming Engines for Big Data
This document discusses streaming engines for big data and provides a case study on Spark Streaming. It begins with an overview of streaming concepts like streams, stream processing, and time in modern data stream analysis. Next, it covers key design considerations for streaming engines and examples of state-of-the-art stream analysis tools like Apache Flink, Spark Streaming, and Apache Beam. It then focuses on Spark Streaming, describing its DStream and Structured Streaming APIs. Code examples are provided for the DStream API and Structured Streaming. The document concludes with a recommendation to first consider Flink, Spark, or Kafka Streams when choosing a streaming engine.
Bsp media branded_rp_africacom_2013_sub sahara_freecopy
This document discusses Gen C, the mobile video generation, and opportunities for local and relevant content on YouTube mobile platforms. It notes that building mobile capabilities attracts users and examines stimulating more uploads and rich content through maturity of features. The demand for rich content on mobile is here but needs better unlocking to fulfill potential.
Unlock your power with 4 dimensional actionable intelligence
This document summarizes a presentation given by Eric Letellier, VP of Technology Consulting at Tektronix Communications, about optimizing LTE network performance. While network KPIs show good accessibility, retainability, mobility, and integrity, customer trouble tickets reveal issues with video streaming, application freezing, and slow page loads. There is a need to bridge the gap between control plane and user plane metrics to better assess quality of experience from the user perspective. Letellier advocates a new paradigm for LTE optimization focusing on correlating subscriber and network data, collaboration between technical and business teams, and compensation structures that incentivize improving the user experience.
Regulator's role in ensuring successful MNP implementation and management
The document discusses the regulator's role in ensuring successful mobile number portability (MNP) implementation and management in Tanzania. It provides an overview of Tanzania's communications sector, including policies, regulations, licensing frameworks, market growth, and infrastructure development. It then examines available MNP approaches, noting Tanzania opted for direct routing using All Call Query and a centralized clearinghouse operated by an independent third party. The regulator's involvement in MNP includes establishing robust processes, managing numbering resources, ensuring quality of service, and facilitating dispute resolution. Stakeholder collaboration is also key to the success of MNP.
Efficient and simple porting processes make one day porting a reality
This document discusses mobile number portability implementation and management. It focuses on making the porting process efficient and simple to allow for one day porting. The agenda includes examining statistics on porting times across Europe, learning from case studies of fast porting times in other countries, and definitions related to porting time. The overall goal is to analyze how to shorten porting times to improve the customer experience when switching providers.
How to build an awesome mobile APP
This document provides guidance on building a mobile app. It discusses prioritizing functionality, designing for mobile, engineering options like native vs HTML and cloud services, using tools like Asana for project management, and releasing iteratively using TestFlight for user testing before full launch. Contact information is provided for the author, Yossi, an experienced product manager and mobile app consultant.
Carrier Wifi
This document discusses the growing importance and opportunities of carrier Wi-Fi networks. It notes that most tablets are sold without cellular radios, making Wi-Fi networks essential for connectivity. Carriers can offer Wi-Fi services to meet growing user demand and expectations. Wi-Fi also provides carriers with advantages over cellular networks in terms of available spectrum and costs. The document outlines various monetization models for carrier Wi-Fi networks and gives examples of operating models being used. It addresses some technical challenges for carriers in deploying Wi-Fi networks and emphasizes the importance of integration with cellular networks.
Power of Satellite in Africa
This document summarizes how satellite technology can help bridge the digital divide in Africa. It notes that Africa has over 1 billion people across 54 countries, with 50% of Africans expected to live in cities by 2030. Satellite provides an efficient way to connect the 700 million unconnected people across Africa. It can deliver broadband, TV, mobile backhaul, and more to both urban and non-urban areas. SES aims to connect Africa through its satellites and infrastructure, offering media distribution, enterprise solutions, and government services to help advance connectivity.
Recommended
Voxxed days thessaloniki 21/10/2016 - Streaming Engines for Big Data
This document discusses streaming engines for big data and provides a case study on Spark Streaming. It begins with an overview of streaming concepts like streams, stream processing, and time in modern data stream analysis. Next, it covers key design considerations for streaming engines and examples of state-of-the-art stream analysis tools like Apache Flink, Spark Streaming, and Apache Beam. It then focuses on Spark Streaming, describing its DStream and Structured Streaming APIs. Code examples are provided for the DStream API and Structured Streaming. The document concludes with a recommendation to first consider Flink, Spark, or Kafka Streams when choosing a streaming engine.
Bsp media branded_rp_africacom_2013_sub sahara_freecopy
This document discusses Gen C, the mobile video generation, and opportunities for local and relevant content on YouTube mobile platforms. It notes that building mobile capabilities attracts users and examines stimulating more uploads and rich content through maturity of features. The demand for rich content on mobile is here but needs better unlocking to fulfill potential.
Unlock your power with 4 dimensional actionable intelligence
This document summarizes a presentation given by Eric Letellier, VP of Technology Consulting at Tektronix Communications, about optimizing LTE network performance. While network KPIs show good accessibility, retainability, mobility, and integrity, customer trouble tickets reveal issues with video streaming, application freezing, and slow page loads. There is a need to bridge the gap between control plane and user plane metrics to better assess quality of experience from the user perspective. Letellier advocates a new paradigm for LTE optimization focusing on correlating subscriber and network data, collaboration between technical and business teams, and compensation structures that incentivize improving the user experience.
Regulator's role in ensuring successful MNP implementation and management
The document discusses the regulator's role in ensuring successful mobile number portability (MNP) implementation and management in Tanzania. It provides an overview of Tanzania's communications sector, including policies, regulations, licensing frameworks, market growth, and infrastructure development. It then examines available MNP approaches, noting Tanzania opted for direct routing using All Call Query and a centralized clearinghouse operated by an independent third party. The regulator's involvement in MNP includes establishing robust processes, managing numbering resources, ensuring quality of service, and facilitating dispute resolution. Stakeholder collaboration is also key to the success of MNP.
Efficient and simple porting processes make one day porting a reality
This document discusses mobile number portability implementation and management. It focuses on making the porting process efficient and simple to allow for one day porting. The agenda includes examining statistics on porting times across Europe, learning from case studies of fast porting times in other countries, and definitions related to porting time. The overall goal is to analyze how to shorten porting times to improve the customer experience when switching providers.
How to build an awesome mobile APP
This document provides guidance on building a mobile app. It discusses prioritizing functionality, designing for mobile, engineering options like native vs HTML and cloud services, using tools like Asana for project management, and releasing iteratively using TestFlight for user testing before full launch. Contact information is provided for the author, Yossi, an experienced product manager and mobile app consultant.
Carrier Wifi
This document discusses the growing importance and opportunities of carrier Wi-Fi networks. It notes that most tablets are sold without cellular radios, making Wi-Fi networks essential for connectivity. Carriers can offer Wi-Fi services to meet growing user demand and expectations. Wi-Fi also provides carriers with advantages over cellular networks in terms of available spectrum and costs. The document outlines various monetization models for carrier Wi-Fi networks and gives examples of operating models being used. It addresses some technical challenges for carriers in deploying Wi-Fi networks and emphasizes the importance of integration with cellular networks.
Power of Satellite in Africa
This document summarizes how satellite technology can help bridge the digital divide in Africa. It notes that Africa has over 1 billion people across 54 countries, with 50% of Africans expected to live in cities by 2030. Satellite provides an efficient way to connect the 700 million unconnected people across Africa. It can deliver broadband, TV, mobile backhaul, and more to both urban and non-urban areas. SES aims to connect Africa through its satellites and infrastructure, offering media distribution, enterprise solutions, and government services to help advance connectivity.
MBB Networks strategies
This document discusses strategies for mobile broadband data monetization. It notes that mobile data traffic is expected to grow 17 times by 2019, driven by factors like the rise of smartphones and tablets. It emphasizes developing an end-to-end approach to optimize networks for performance and coverage of applications. This involves evolving spectrum usage, densifying networks through small cells and WiFi, and creating a consistent experience across access technologies. The goal is to monetize all traffic and provide a superior user experience on a single network.
Bsp media branded_rp_africacom_2013_fonk_freecopy
This 3 sentence summary provides the high level information from the document:
Touch interaction has become the standard across many digital devices like smartphones and tablets. Designing algorithms to convert visual content into tactile sensations could enrich the user experience. Ivan Poupyrev, director of Disney's Interaction Group, comments on the potential of designing algorithms that can convert visual content into believable tactile sensations.
LTE development and services
This document discusses LTE deployment and services, including content management on LTE networks, voice over LTE using IMS networks, dedicated and default bearers, circuit-switched fallback architecture, GSMA standards for RCS services over LTE, and evidence that LTE encourages increased data usage. It also covers macrocells versus small cells, considerations for small cell rollout, and strategies for successful LTE launches.
Understanding how triple pay is impacting on the African Broadcasters
This document discusses how the triple play model is impacting African broadcasters. It outlines five major shifts in television including time shifting, quality shifting, and engagement shifting. It also discusses how innovation is driving increased differentiation and integration of services like personalized TV and voice/video conferencing. The challenges for service providers in this environment include supporting access across multiple screens, acquiring premium content, and designing networks to guarantee quality of experience while maintaining cost effectiveness.
How do traditional banks fit the digital age?
This document discusses how traditional banks can adapt to the digital age through agent banking and branchless banking. It summarizes that customers now demand simple, convenient banking services through digital channels like internet and mobile due to the disruptive influence of technology. However, cash is still heavily used in Africa. The document proposes that agent banking, which provides banking services through authorized third party agents, can help banks meet customer demands while reducing costs through digital channels, cash transactions, and an expanded network of agents/branches. Regulations are needed but should not stifle the growth of branchless banking in serving more customers.
Transforming Services operations so that CEM becomes an enabler for improving...
This document summarizes key challenges and opportunities for communications service providers (CSPs) related to customer experience management (CEM). It finds that CSPs must address rising customer demands, intense competition, and network upgrade costs. CEM can help CSPs improve customer focus, service differentiation, and revenue growth. The document also reviews CSP drivers for CEM adoption, including customer retention, yield management, and operational efficiencies. It provides recommendations for both CSPs and CEM vendors on implementing holistic CEM strategies.
Tangible magic in digital area.
This 3 sentence summary provides the high level information from the document:
Touch interaction has become the standard across many digital devices like smartphones and tablets. Designing algorithms to convert visual content into tactile sensations could enrich the user experience. Ivan Poupyrev, director of Disney's Interaction Group, believes designing such algorithms has immense potential.
Ensuring a Positive and Efficient Customer Porting Experience – The Key Factors!
This document discusses key factors for ensuring a positive customer porting experience. It evaluates different approaches to implementing number portability and examines the organizational structures and pricing models needed. The objectives include building consensus on number portability service pricing and assessing costs of setting up and managing the portability system.
LTE Core Network
This document provides an overview of the LTE network architecture including:
1) The LTE network introduces a new radio access network called E-UTRAN and a new core network called EPC.
2) Key components of the EPC include the MME for control plane functions, the SGW for user plane functions and mobility, and the PGW which acts as an anchor for the user plane.
3) The architecture supports interworking with other 3GPP networks like UMTS/GPRS and non-3GPP networks like WiFi. It also defines interfaces for connecting to external networks.
4) The network provides IP connectivity and supports different bearer types for implementing Quality
Who is Wizzit
WIZZIT is a global pioneer in innovative banking for the mass market. It has operations in 8 countries, over 5 million customers through bank partners, and a proven track record since 2004. WIZZIT uses innovative, state of the art technology that is device and network agnostic. It has received several awards and invitations to address global forums for its mobile banking model that aims to bank the unbanked population.
Leading the digital revolution in Africa & the Middle East
This document provides an overview and strategic objectives of Orange's international development in Africa and the Middle East between 2011-2015. It outlines Orange's goals to reach 300 million customers worldwide, double turnover in emerging markets to €7 billion by 2015, and be the #1 or #2 provider everywhere in its footprint. It also discusses Orange's cloud computing services including Infrastructure as a Service, Platform as a Service, and Software as a Service and its strategy to provide cloud services across its regions and affiliates through its OneCloud platform.
United Nations E-Government Survey 2012 E-Government for the People
This document summarizes the United Nations E-Government Survey 2012. The key points are:
1) The survey ranks 193 UN member states on their e-government development and analyzes global and regional trends in online service delivery.
2) South Korea, the United Kingdom, the United States, Australia and Canada are the top 5 performers in e-government development. Developing countries are progressing but still lag behind developed nations.
3) Online services are becoming more sophisticated, with many countries improving multichannel access and efforts to bridge digital divides. However, increasing actual usage of services remains a challenge.
4) A whole-of-government approach to integrated service delivery is seen as important but
Spark meetup stream processing use cases
Stream processing is key for enabling real-time insights from big data. It allows continuous processing of data as it flows through a system, enabling instantaneous actions via alerts. This is unlike traditional batch processing where data is analyzed only after it has been stored. Stream processing is becoming more important as organizations need faster decision making and new data sources like sensors produce high volumes of data quickly. It has broad applications across industries like financial services, telecom, online advertising, and IoT. Example use cases include fraud detection, dynamic bidding, infrastructure management, and predictive maintenance. Open source technologies now make stream processing more feasible at large scale for real-time analytics.
Lecture6 introduction to data streams
There are three main points about data streams and stream processing:
1) A data stream is a continuous, ordered sequence of data items that arrives too rapidly to be stored fully. Common sources include sensors, web traffic, and social media.
2) Data stream management systems process continuous queries over streams in real-time using bounded memory. They provide summaries of historical data rather than storing entire streams.
3) Challenges of stream processing include limited memory, complex continuous queries, and unpredictable data rates and characteristics. Approximate query processing techniques like windows, sampling, and load shedding help address these challenges.
Traitement d'événements
Après-midi - Track 2 - S3 - Traitement d'evenements
Cette session vous a plû ? Si c'est le cas, n'hésitez pas à vous inscrire à notre Summit !
Time Series Analytics for Big Fast Data
Mar 2018 talk to SW Data Meetup by Mark O'Mahony, Software Engineer, Kx Systems.
Learn how to use a relational time-series and columnar database as well as a tightly integrated query language capable of doing aggregations and consolidations on billions of streaming and real time historical records.
Kx Systems are the founders of the world’s fastest time series database – Kdb+ – as well as the ‘q’ query language it is written in. kdb+ is a high-performance, high-volume database designed as a solution to Big Data problems.
Kdb+ is widely adopted by financial institutions around the world, including the top ten global investment banks, however as a result in the explosion of big and the internet of things, Kx is now expanding its applicability into new verticals such IoT, Utilities, Telco, Retail, Space and Pharmaceutical industries. Perhaps you would be interested in our most recent work with Aston Martin Red Bull Racing or Airbus.
Data Care, Feeding, and Maintenance
- The document discusses data infrastructure at an online video syndication platform that handles 2-3 million streams per day.
- It describes different tools for data storage, analytics, and real-time processing including Hadoop, Spark, MongoDB, Logstash, Elasticsearch, and Storm.
- It also discusses best practices for data collection, formatting, analysis, and using data to detect issues through a case study on investigating a rapid decline in video streams.
Real Time Insights for Advertising Tech
A leading Ad Tech company serves billions of advertising impressions and collects terabytes of data from several Data centers across the world. In batch data processing, data is collected at different geographic locations and processed at regular intervals. This system brings delay of at least 1 hour before an event is accounted for.
The goal of having real time streaming was to provide publishers, Demand Side Platforms (DSP's) and agencies actionable insights in a few minutes from the time of event generation.
This Ad Tech company uses DataTorrent RTS powered by Apex for:
• Real time reporting
• Resource monitoring
• Real time learning
• Allocation engine
Tushar Gosavi from DataTorrent will take the audience through the architecture, custom operators developed, use cases for real time and the challenges involved in implementing streaming systems at scale where multiple data centers are in play.
Tushar is a Senior Engineer at DataTorrent and has worked in distributed systems and storage domains.
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog
Think you have big data? What about high availability
requirements? At DataDog we process billions of data points every day including metrics and events, as we help the world
monitor the their applications and infrastructure. Being the world’s monitoring system is a big responsibility, and thanks to
Redis we are up to the task. Join us as we discuss how the DataDog team monitors and scales Redis to power our SaaS based monitoring offering. We will discuss our usage and deployment patterns, as well as dive into monitoring best practices for production Redis workloads
AWS APAC Webinar Week - Real Time Data Processing with Kinesis
Extracting real-time information from streaming data generated by mobile devices, sensors, and servers used to require distributed systems skills and writing custom code. This presentation will introduce Kinesis Streams and Kinesis Firehose, the AWS services for real-time streaming big data ingestion and processing.
We’ll provide an overview of the key scenarios and business use cases suitable for real-time processing, and how Kinesis can help customers shift from a traditional batch-oriented processing of data to a continual real-time processing model. We’ll explore the key concepts, attributes, APIs and features of the service, and discuss building a Kinesis-enabled application for real-time processing. This talk will also include key lessons learnt, architectural tips and design considerations in working with Kinesis and building real-time processing applications.
In this webinar, we will also provide an overview of Amazon Kinesis Firehose. We will then walk through a demo showing how to create an Amazon Kinesis Firehose delivery stream, send data to the stream, and configure it to load the data automatically into Amazon S3 and Amazon Redshift.
Operational-Analytics
The document summarizes operational analytics and provides a brief history of analytics from Analytics 1.0 to Analytics 3.0. It then discusses SAP HANA, an in-memory operational analytics database management system. Key features of SAP HANA include its use of columnar storage formats, memory optimization, query execution engines like the join engine, and parallelization at various levels.
IBM Big Data
IBM provides two types of accelerators for big data to speed the development and implementation of specific big data solutions: 1) Analytic accelerators that address specific data types or operations with advanced analytics like text, geospatial, time series, and data mining. 2) Application accelerators that address specific use cases like finance, machine data, social media, and telecommunications event data. The accelerators are packaged software components included with IBM Big Data Platform products at no additional cost to help eliminate complexity and reduce time-to-value for customers' big data deployments.
More Related Content
Viewers also liked
MBB Networks strategies
This document discusses strategies for mobile broadband data monetization. It notes that mobile data traffic is expected to grow 17 times by 2019, driven by factors like the rise of smartphones and tablets. It emphasizes developing an end-to-end approach to optimize networks for performance and coverage of applications. This involves evolving spectrum usage, densifying networks through small cells and WiFi, and creating a consistent experience across access technologies. The goal is to monetize all traffic and provide a superior user experience on a single network.
Bsp media branded_rp_africacom_2013_fonk_freecopy
This 3 sentence summary provides the high level information from the document:
Touch interaction has become the standard across many digital devices like smartphones and tablets. Designing algorithms to convert visual content into tactile sensations could enrich the user experience. Ivan Poupyrev, director of Disney's Interaction Group, comments on the potential of designing algorithms that can convert visual content into believable tactile sensations.
LTE development and services
This document discusses LTE deployment and services, including content management on LTE networks, voice over LTE using IMS networks, dedicated and default bearers, circuit-switched fallback architecture, GSMA standards for RCS services over LTE, and evidence that LTE encourages increased data usage. It also covers macrocells versus small cells, considerations for small cell rollout, and strategies for successful LTE launches.
Understanding how triple pay is impacting on the African Broadcasters
This document discusses how the triple play model is impacting African broadcasters. It outlines five major shifts in television including time shifting, quality shifting, and engagement shifting. It also discusses how innovation is driving increased differentiation and integration of services like personalized TV and voice/video conferencing. The challenges for service providers in this environment include supporting access across multiple screens, acquiring premium content, and designing networks to guarantee quality of experience while maintaining cost effectiveness.
How do traditional banks fit the digital age?
This document discusses how traditional banks can adapt to the digital age through agent banking and branchless banking. It summarizes that customers now demand simple, convenient banking services through digital channels like internet and mobile due to the disruptive influence of technology. However, cash is still heavily used in Africa. The document proposes that agent banking, which provides banking services through authorized third party agents, can help banks meet customer demands while reducing costs through digital channels, cash transactions, and an expanded network of agents/branches. Regulations are needed but should not stifle the growth of branchless banking in serving more customers.
Transforming Services operations so that CEM becomes an enabler for improving...
This document summarizes key challenges and opportunities for communications service providers (CSPs) related to customer experience management (CEM). It finds that CSPs must address rising customer demands, intense competition, and network upgrade costs. CEM can help CSPs improve customer focus, service differentiation, and revenue growth. The document also reviews CSP drivers for CEM adoption, including customer retention, yield management, and operational efficiencies. It provides recommendations for both CSPs and CEM vendors on implementing holistic CEM strategies.
Tangible magic in digital area.
This 3 sentence summary provides the high level information from the document:
Touch interaction has become the standard across many digital devices like smartphones and tablets. Designing algorithms to convert visual content into tactile sensations could enrich the user experience. Ivan Poupyrev, director of Disney's Interaction Group, believes designing such algorithms has immense potential.
Ensuring a Positive and Efficient Customer Porting Experience – The Key Factors!
This document discusses key factors for ensuring a positive customer porting experience. It evaluates different approaches to implementing number portability and examines the organizational structures and pricing models needed. The objectives include building consensus on number portability service pricing and assessing costs of setting up and managing the portability system.
LTE Core Network
This document provides an overview of the LTE network architecture including:
1) The LTE network introduces a new radio access network called E-UTRAN and a new core network called EPC.
2) Key components of the EPC include the MME for control plane functions, the SGW for user plane functions and mobility, and the PGW which acts as an anchor for the user plane.
3) The architecture supports interworking with other 3GPP networks like UMTS/GPRS and non-3GPP networks like WiFi. It also defines interfaces for connecting to external networks.
4) The network provides IP connectivity and supports different bearer types for implementing Quality
Who is Wizzit
WIZZIT is a global pioneer in innovative banking for the mass market. It has operations in 8 countries, over 5 million customers through bank partners, and a proven track record since 2004. WIZZIT uses innovative, state of the art technology that is device and network agnostic. It has received several awards and invitations to address global forums for its mobile banking model that aims to bank the unbanked population.
Leading the digital revolution in Africa & the Middle East
This document provides an overview and strategic objectives of Orange's international development in Africa and the Middle East between 2011-2015. It outlines Orange's goals to reach 300 million customers worldwide, double turnover in emerging markets to €7 billion by 2015, and be the #1 or #2 provider everywhere in its footprint. It also discusses Orange's cloud computing services including Infrastructure as a Service, Platform as a Service, and Software as a Service and its strategy to provide cloud services across its regions and affiliates through its OneCloud platform.
United Nations E-Government Survey 2012 E-Government for the People
This document summarizes the United Nations E-Government Survey 2012. The key points are:
1) The survey ranks 193 UN member states on their e-government development and analyzes global and regional trends in online service delivery.
2) South Korea, the United Kingdom, the United States, Australia and Canada are the top 5 performers in e-government development. Developing countries are progressing but still lag behind developed nations.
3) Online services are becoming more sophisticated, with many countries improving multichannel access and efforts to bridge digital divides. However, increasing actual usage of services remains a challenge.
4) A whole-of-government approach to integrated service delivery is seen as important but
Viewers also liked (12)
Understanding how triple pay is impacting on the African Broadcasters
Understanding how triple pay is impacting on the African Broadcasters
Transforming Services operations so that CEM becomes an enabler for improving...
Transforming Services operations so that CEM becomes an enabler for improving...
Ensuring a Positive and Efficient Customer Porting Experience – The Key Factors!
Ensuring a Positive and Efficient Customer Porting Experience – The Key Factors!
Leading the digital revolution in Africa & the Middle East
Leading the digital revolution in Africa & the Middle East
United Nations E-Government Survey 2012 E-Government for the People
United Nations E-Government Survey 2012 E-Government for the People
Similar to Real- Time Analytics – Process Automation
Spark meetup stream processing use cases
Stream processing is key for enabling real-time insights from big data. It allows continuous processing of data as it flows through a system, enabling instantaneous actions via alerts. This is unlike traditional batch processing where data is analyzed only after it has been stored. Stream processing is becoming more important as organizations need faster decision making and new data sources like sensors produce high volumes of data quickly. It has broad applications across industries like financial services, telecom, online advertising, and IoT. Example use cases include fraud detection, dynamic bidding, infrastructure management, and predictive maintenance. Open source technologies now make stream processing more feasible at large scale for real-time analytics.
Lecture6 introduction to data streams
There are three main points about data streams and stream processing:
1) A data stream is a continuous, ordered sequence of data items that arrives too rapidly to be stored fully. Common sources include sensors, web traffic, and social media.
2) Data stream management systems process continuous queries over streams in real-time using bounded memory. They provide summaries of historical data rather than storing entire streams.
3) Challenges of stream processing include limited memory, complex continuous queries, and unpredictable data rates and characteristics. Approximate query processing techniques like windows, sampling, and load shedding help address these challenges.
Traitement d'événements
Après-midi - Track 2 - S3 - Traitement d'evenements
Cette session vous a plû ? Si c'est le cas, n'hésitez pas à vous inscrire à notre Summit !
Time Series Analytics for Big Fast Data
Mar 2018 talk to SW Data Meetup by Mark O'Mahony, Software Engineer, Kx Systems.
Learn how to use a relational time-series and columnar database as well as a tightly integrated query language capable of doing aggregations and consolidations on billions of streaming and real time historical records.
Kx Systems are the founders of the world’s fastest time series database – Kdb+ – as well as the ‘q’ query language it is written in. kdb+ is a high-performance, high-volume database designed as a solution to Big Data problems.
Kdb+ is widely adopted by financial institutions around the world, including the top ten global investment banks, however as a result in the explosion of big and the internet of things, Kx is now expanding its applicability into new verticals such IoT, Utilities, Telco, Retail, Space and Pharmaceutical industries. Perhaps you would be interested in our most recent work with Aston Martin Red Bull Racing or Airbus.
Data Care, Feeding, and Maintenance
- The document discusses data infrastructure at an online video syndication platform that handles 2-3 million streams per day.
- It describes different tools for data storage, analytics, and real-time processing including Hadoop, Spark, MongoDB, Logstash, Elasticsearch, and Storm.
- It also discusses best practices for data collection, formatting, analysis, and using data to detect issues through a case study on investigating a rapid decline in video streams.
Real Time Insights for Advertising Tech
A leading Ad Tech company serves billions of advertising impressions and collects terabytes of data from several Data centers across the world. In batch data processing, data is collected at different geographic locations and processed at regular intervals. This system brings delay of at least 1 hour before an event is accounted for.
The goal of having real time streaming was to provide publishers, Demand Side Platforms (DSP's) and agencies actionable insights in a few minutes from the time of event generation.
This Ad Tech company uses DataTorrent RTS powered by Apex for:
• Real time reporting
• Resource monitoring
• Real time learning
• Allocation engine
Tushar Gosavi from DataTorrent will take the audience through the architecture, custom operators developed, use cases for real time and the challenges involved in implementing streaming systems at scale where multiple data centers are in play.
Tushar is a Senior Engineer at DataTorrent and has worked in distributed systems and storage domains.
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog
Think you have big data? What about high availability
requirements? At DataDog we process billions of data points every day including metrics and events, as we help the world
monitor the their applications and infrastructure. Being the world’s monitoring system is a big responsibility, and thanks to
Redis we are up to the task. Join us as we discuss how the DataDog team monitors and scales Redis to power our SaaS based monitoring offering. We will discuss our usage and deployment patterns, as well as dive into monitoring best practices for production Redis workloads
AWS APAC Webinar Week - Real Time Data Processing with Kinesis
Extracting real-time information from streaming data generated by mobile devices, sensors, and servers used to require distributed systems skills and writing custom code. This presentation will introduce Kinesis Streams and Kinesis Firehose, the AWS services for real-time streaming big data ingestion and processing.
We’ll provide an overview of the key scenarios and business use cases suitable for real-time processing, and how Kinesis can help customers shift from a traditional batch-oriented processing of data to a continual real-time processing model. We’ll explore the key concepts, attributes, APIs and features of the service, and discuss building a Kinesis-enabled application for real-time processing. This talk will also include key lessons learnt, architectural tips and design considerations in working with Kinesis and building real-time processing applications.
In this webinar, we will also provide an overview of Amazon Kinesis Firehose. We will then walk through a demo showing how to create an Amazon Kinesis Firehose delivery stream, send data to the stream, and configure it to load the data automatically into Amazon S3 and Amazon Redshift.
Operational-Analytics
The document summarizes operational analytics and provides a brief history of analytics from Analytics 1.0 to Analytics 3.0. It then discusses SAP HANA, an in-memory operational analytics database management system. Key features of SAP HANA include its use of columnar storage formats, memory optimization, query execution engines like the join engine, and parallelization at various levels.
IBM Big Data
IBM provides two types of accelerators for big data to speed the development and implementation of specific big data solutions: 1) Analytic accelerators that address specific data types or operations with advanced analytics like text, geospatial, time series, and data mining. 2) Application accelerators that address specific use cases like finance, machine data, social media, and telecommunications event data. The accelerators are packaged software components included with IBM Big Data Platform products at no additional cost to help eliminate complexity and reduce time-to-value for customers' big data deployments.
Ibm big data
IBM provides two types of accelerators for big data to speed the development and implementation of specific big data solutions: 1) Analytic accelerators that address specific data types or operations with advanced analytics; and 2) Application accelerators that address specific use cases and include both industry-specific and cross-industry features. The accelerators are packaged software components that provide business logic, data processing, and visualization capabilities and help eliminate the complexity of building big data applications. Examples of capabilities provided by various accelerators include text analytics, geospatial analysis, time series prediction, data mining, finance analytics, machine data analysis, social media insights, and telecommunications event data processing.
Datastream management system1
Data streams refer to continuous, massive volumes of data that arrive over time. Examples include sensor data, web traffic, and stock prices. A data stream management system processes continuous queries over data streams in real-time using limited memory. It aims to provide approximate answers when necessary due to constraints. Techniques for approximate query answering include using sliding windows, sampling, and maintaining synopses of the data stream.
ClickHouse in Real Life. Case Studies and Best Practices, by Alexander Zaitsev
ClickHouse in Real Life. Case Studies and Best Practices.
By Alexander Zaitsev, Altinity/Lifestreet
Presented at Percona Live 2018
Introduction to Amazon Kinesis Data Streams
This document provides an introduction and overview of Amazon Kinesis Data Streams. It begins with definitions of streaming data and examples. It then introduces Amazon Kinesis Data Streams as a scalable real-time data streaming service. It outlines the basic architecture involving producers, streams, shards, and consumers. It defines key concepts like records, partitions, and capacity modes. It also lists some basic operations on streams like creating, describing, putting records, and getting records. Finally, it concludes with references for further reading.
Next Gen Big Data Analytics with Apache Apex
Next Gen Big Data Analytics with Apache Apex discusses Apache Apex, an open source stream processing framework. It provides an overview of Apache Apex's capabilities for processing continuous, real-time data streams at scale. Specifically, it describes how Apache Apex allows for in-memory, distributed stream processing using a programming model of operators in a directed acyclic graph. It also covers Apache Apex's features for fault tolerance, dynamic scaling, and integration with Hadoop and YARN.
Hadoop Summit SJ 2016: Next Gen Big Data Analytics with Apache Apex
This is an overview of architecture with use cases for Apache Apex, a big data analytics platform. It comes with a powerful stream processing engine, rich set of functional building blocks and an easy to use API for the developer to build real-time and batch applications. Apex runs natively on YARN and HDFS and is used in production in various industries. You will learn more about two use cases: A leading Ad Tech company serves billions of advertising impressions and collects terabytes of data from several data centers across the world every day. Apex was used to implement rapid actionable insights, for real-time reporting and allocation, utilizing Kafka and files as source, dimensional computation and low latency visualization. A customer in the IoT space uses Apex for Time Series service, including efficient storage of time series data, data indexing for quick retrieval and queries at high scale and precision. The platform leverages the high availability, horizontal scalability and operability of Apex.
Wikibon #IoT #HyperConvergence Presentation via @theCUBE
SiliconANGLE Media Research team at Wikibon prepared this presentation to share their findings on a new category called #IoT #HyperConvergence Analytics
Crowd Chat Conversation here:
https://www.crowdchat.net/chat/c3BvdF9vYmpfMTg4Mg==
More and more data is streaming in from many sources in order to drive operations in real-time.
When driving decisions with speed at scale is the norm, the traditional trade-off in analytics between simple but fast and slow but sophisticated has to give way.
Traditionally fast data comes to rest in a database after the simpler in-flight analytics. Only after it is comes to rest can a database perform sophisticated analytics. But in-flight and at rest analytics have to come together in a single, hyper-converged analytic platform.
Hyper-Convergence CrowdChat
The Journet to IoT Systems of Intelligence: Determined by Combination of Tech & Enterprise Capabilities
Dynamics CRM high volume systems - lessons from the field
Three field stories from companies describe their experiences with high volume CRM implementations: a financial institution with 8,000 users and 350GB of data across two implementations; a financial institution with 2,000 users, 2,500GB of data across two implementations; and a financial institution with 1,000 users and over 450GB of data across six implementations, with 50GB added per month for the largest one. The document discusses lessons learned from these implementations regarding infrastructure design, functional design, and performance testing to support high volume systems.
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
This document discusses Amazon Kinesis, a fully managed service for real-time processing of streaming data. It provides an overview of Kinesis and how it can be used to ingest, store, and process streaming data. Examples are given of how companies are using Kinesis for applications like game analytics, digital advertising metrics, and IoT data processing. The key benefits of Kinesis are also summarized such as its ease of use, real-time performance, elastic scalability, integration with other AWS services, and low cost.
Similar to Real- Time Analytics – Process Automation (20)
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDog
AWS APAC Webinar Week - Real Time Data Processing with Kinesis
AWS APAC Webinar Week - Real Time Data Processing with Kinesis
ClickHouse in Real Life. Case Studies and Best Practices, by Alexander Zaitsev
ClickHouse in Real Life. Case Studies and Best Practices, by Alexander Zaitsev
Hadoop Summit SJ 2016: Next Gen Big Data Analytics with Apache Apex
Hadoop Summit SJ 2016: Next Gen Big Data Analytics with Apache Apex
Wikibon #IoT #HyperConvergence Presentation via @theCUBE
Wikibon #IoT #HyperConvergence Presentation via @theCUBE
Dynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the field
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
Recently uploaded
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
A Comprehensive Guide to DeFi Development Services in 2024
DeFi represents a paradigm shift in the financial industry. Instead of relying on traditional, centralized institutions like banks, DeFi leverages blockchain technology to create a decentralized network of financial services. This means that financial transactions can occur directly between parties, without intermediaries, using smart contracts on platforms like Ethereum.
In 2024, we are witnessing an explosion of new DeFi projects and protocols, each pushing the boundaries of what’s possible in finance.
In summary, DeFi in 2024 is not just a trend; it’s a revolution that democratizes finance, enhances security and transparency, and fosters continuous innovation. As we proceed through this presentation, we'll explore the various components and services of DeFi in detail, shedding light on how they are transforming the financial landscape.
At Intelisync, we specialize in providing comprehensive DeFi development services tailored to meet the unique needs of our clients. From smart contract development to dApp creation and security audits, we ensure that your DeFi project is built with innovation, security, and scalability in mind. Trust Intelisync to guide you through the intricate landscape of decentralized finance and unlock the full potential of blockchain technology.
Ready to take your DeFi project to the next level? Partner with Intelisync for expert DeFi development services today!
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
This case study explores designing a scalable e-commerce platform, covering key requirements, system components, and best practices.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
June Patch Tuesday
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Trusted Execution Environment for Decentralized Process Mining
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Artificial Intelligence for XMLDevelopment
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
AWS Cloud Cost Optimization Presentation.pptx
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays, June 2024 with Maria Copot 20
Operating System Used by Users in day-to-day life.pptx
Dive into the realm of operating systems (OS) with Pravash Chandra Das, a seasoned Digital Forensic Analyst, as your guide. 🚀 This comprehensive presentation illuminates the core concepts, types, and evolution of OS, essential for understanding modern computing landscapes.
Beginning with the foundational definition, Das clarifies the pivotal role of OS as system software orchestrating hardware resources, software applications, and user interactions. Through succinct descriptions, he delineates the diverse types of OS, from single-user, single-task environments like early MS-DOS iterations, to multi-user, multi-tasking systems exemplified by modern Linux distributions.
Crucial components like the kernel and shell are dissected, highlighting their indispensable functions in resource management and user interface interaction. Das elucidates how the kernel acts as the central nervous system, orchestrating process scheduling, memory allocation, and device management. Meanwhile, the shell serves as the gateway for user commands, bridging the gap between human input and machine execution. 💻
The narrative then shifts to a captivating exploration of prominent desktop OSs, Windows, macOS, and Linux. Windows, with its globally ubiquitous presence and user-friendly interface, emerges as a cornerstone in personal computing history. macOS, lauded for its sleek design and seamless integration with Apple's ecosystem, stands as a beacon of stability and creativity. Linux, an open-source marvel, offers unparalleled flexibility and security, revolutionizing the computing landscape. 🖥️
Moving to the realm of mobile devices, Das unravels the dominance of Android and iOS. Android's open-source ethos fosters a vibrant ecosystem of customization and innovation, while iOS boasts a seamless user experience and robust security infrastructure. Meanwhile, discontinued platforms like Symbian and Palm OS evoke nostalgia for their pioneering roles in the smartphone revolution.
The journey concludes with a reflection on the ever-evolving landscape of OS, underscored by the emergence of real-time operating systems (RTOS) and the persistent quest for innovation and efficiency. As technology continues to shape our world, understanding the foundations and evolution of operating systems remains paramount. Join Pravash Chandra Das on this illuminating journey through the heart of computing. 🌟
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
dbms calicut university B. sc Cs 4th sem.pdf
Its a seminar ppt on database management system using sql
Recently uploaded (20)
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Operating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptx
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Real- Time Analytics – Process Automation
- 1. This document is offered compliments of BSP Media Group. www.bspmediagroup.com All rights reserved.
- 2. Real-Time Analytics Process Automation Nico Coetzee ncoetzee1@fnb.co.za First National Bank – a division of FirstRand Bank Limited. An Authorised Financial Services and Credit Provider (NCRCP20).
- 3. Big Data • Volume • Amount of data • Velocity • speed of data in and out • Variety • Range of data types and sources
- 6. Volume • Examples: • User profile database • Inventory • Real Time Analytics Scenarios: • Measure changes over time • Assist ERP systems with automation (order stock) • Stock load balancing (automatically redistribute stock from one area to another)
- 8. Velocity • Examples: • POS Transactions (think big national retailers) • Logs (firewall logs) • Real Time Analytics Scenarios • Sudden unexpected patterns (a region experiences an outage) • Attacks and other anomalies that can be picked up from logs
- 10. Variety • Examples: • End-to-end transaction flows through Web logs, Application server logs and database logs • Real Time Analytics Scenarios: • Continues monitor response times (very handy for Cloud type solution where decisions to start more VM’s may be required) • Context required for making intelligent decisions, for example in anti-fraud systems
- 11. The “Missing” V’s • Viability • The secret is uncovering the latent, hidden relationships among these variables. • Value • Remember: Correlation does not mean causation • Realistic scenarios: Was your marketing campaign successful?
- 12. Technologies to Consider • MongoDB • NoSQL DB • Distributed operations (Grid FS) • Built-in Map-Reduce engine • Syslog-ng • Real time decisions on log events • Granular control over log routing • Rules based on regular expressions
- 13. The Future • DevOps and Agile methodologies can benefit from the inputs from real time Big Data analytics • Identified (potential) defects should naturally flow back into the backlog. • Infrastructure and resource management
- 14. Thank You