This document provides an overview of Apache Storm, an open-source distributed real-time computation system. It discusses how Storm can be used for real-time processing of streaming data at massive scales, unlike Hadoop which performs batch processing. The key concepts of Spouts, Bolts, and Topologies in Storm are explained. Examples of a word count topology implemented in Storm using Spouts, Bolts and grouping are also presented through code snippets. Fault tolerance in Storm through worker processes and Zookeeper coordination is summarized.
torm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm is simple, can be used with any programming language, and is a lot of fun to use!
• What is Storm?
• Who use Storm?
• Storm Vs Hadoop
• Storm Components
• Storm Topology
• Storm Primitives
• Why Storm is ideal for Real Time Processing?
torm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm is simple, can be used with any programming language, and is a lot of fun to use!
• What is Storm?
• Who use Storm?
• Storm Vs Hadoop
• Storm Components
• Storm Topology
• Storm Primitives
• Why Storm is ideal for Real Time Processing?
Workshop on "Big Data, Real-time Processing and Storm" at The Fifth Elephant, 2013 Bangalore, India on 11th July, 2013.
http://fifthelephant.in/2013/workshops
Session proposal: https://funnel.hasgeek.com/fifthel2013/652-big-data-real-time-processing-and-storm
Code for the workshop can be found at: https://github.com/P7h
Join this video course on Udemy. Click the below link
https://www.udemy.com/mastering-rtos-hands-on-with-freertos-arduino-and-stm32fx/?couponCode=SLIDESHARE
>> The Complete FreeRTOS Course with Programming and Debugging <<
"The Biggest objective of this course is to demystifying RTOS practically using FreeRTOS and STM32 MCUs"
STEP-by-STEP guide to port/run FreeRTOS using development setup which includes,
1) Eclipse + STM32F4xx + FreeRTOS + SEGGER SystemView
2) FreeRTOS+Simulator (For windows)
Demystifying the complete Architecture (ARM Cortex M) related code of FreeRTOS which will massively help you to put this kernel on any target hardware of your choice.
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareQuantum Leaps, LLC
Embedded software developers from different industries are independently re-discovering patterns for building concurrent software that is safer, more responsive and easier to understand than naked threads of a Real-Time Operating System (RTOS). These best practices universally favor event-driven, asynchronous, non-blocking, encapsulated state machines instead of naked, blocking RTOS threads. This presentation explains the concepts related to this increasingly popular "reactive approach", and specifically how they apply to real-time embedded systems.
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareMiro Samek
Embedded software developers from different industries are independently re-discovering patterns for building concurrent software that is safer, more responsive and easier to understand than naked threads of a Real-Time Operating System (RTOS). These best practices universally favor event-driven, asynchronous, non-blocking, encapsulated active objects with state machines instead of naked, blocking RTOS threads. This presentation explains the concepts related to this increasingly popular "reactive approach", and specifically how they apply to real-time embedded systems.
Porting a Streaming Pipeline from Scala to RustEvan Chan
How we at Conviva ported a streaming data pipeline in months from Scala to Rust. What are the important human and technical factors in our port, and what did we learn?
UKOUG version of a presentation trying to establish the sensible limits of parallelism on a couple of hardware configurations. Detailed white paper is at http://oracledoug.com/px_slaves.pdf
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...Spark Summit
Spark is by its nature very fault tolerant. However, faults, and application failures, can and do happen, in production at scale.
In this talk, we’ll discuss the nuts and bolts of fault tolerance in Spark.
We will begin with a brief overview of the sorts of fault tolerance offered, and lead into a deep dive of the internals of fault tolerance. This will include a discussion of Spark on YARN, scheduling, and resource allocation.
We will then spend some time on a case study and discussing some tools used to find and verify fault tolerance issues. Our case study comes from a customer who experienced an application outage that was root caused to a scheduler bug. We discuss the analysis we did to reach this conclusion and the work that we did to reproduce it locally. We highlight some of the techniques used to simulate faults and find bugs.
At the end, we’ll discuss some future directions for fault tolerance improvements in Spark, such as scheduler and checkpointing changes.
Recruiting in the Digital Age: A Social Media MasterclassLuanWise
In this masterclass, presented at the Global HR Summit on 5th June 2024, Luan Wise explored the essential features of social media platforms that support talent acquisition, including LinkedIn, Facebook, Instagram, X (formerly Twitter) and TikTok.
Workshop on "Big Data, Real-time Processing and Storm" at The Fifth Elephant, 2013 Bangalore, India on 11th July, 2013.
http://fifthelephant.in/2013/workshops
Session proposal: https://funnel.hasgeek.com/fifthel2013/652-big-data-real-time-processing-and-storm
Code for the workshop can be found at: https://github.com/P7h
Join this video course on Udemy. Click the below link
https://www.udemy.com/mastering-rtos-hands-on-with-freertos-arduino-and-stm32fx/?couponCode=SLIDESHARE
>> The Complete FreeRTOS Course with Programming and Debugging <<
"The Biggest objective of this course is to demystifying RTOS practically using FreeRTOS and STM32 MCUs"
STEP-by-STEP guide to port/run FreeRTOS using development setup which includes,
1) Eclipse + STM32F4xx + FreeRTOS + SEGGER SystemView
2) FreeRTOS+Simulator (For windows)
Demystifying the complete Architecture (ARM Cortex M) related code of FreeRTOS which will massively help you to put this kernel on any target hardware of your choice.
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareQuantum Leaps, LLC
Embedded software developers from different industries are independently re-discovering patterns for building concurrent software that is safer, more responsive and easier to understand than naked threads of a Real-Time Operating System (RTOS). These best practices universally favor event-driven, asynchronous, non-blocking, encapsulated state machines instead of naked, blocking RTOS threads. This presentation explains the concepts related to this increasingly popular "reactive approach", and specifically how they apply to real-time embedded systems.
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareMiro Samek
Embedded software developers from different industries are independently re-discovering patterns for building concurrent software that is safer, more responsive and easier to understand than naked threads of a Real-Time Operating System (RTOS). These best practices universally favor event-driven, asynchronous, non-blocking, encapsulated active objects with state machines instead of naked, blocking RTOS threads. This presentation explains the concepts related to this increasingly popular "reactive approach", and specifically how they apply to real-time embedded systems.
Porting a Streaming Pipeline from Scala to RustEvan Chan
How we at Conviva ported a streaming data pipeline in months from Scala to Rust. What are the important human and technical factors in our port, and what did we learn?
UKOUG version of a presentation trying to establish the sensible limits of parallelism on a couple of hardware configurations. Detailed white paper is at http://oracledoug.com/px_slaves.pdf
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...Spark Summit
Spark is by its nature very fault tolerant. However, faults, and application failures, can and do happen, in production at scale.
In this talk, we’ll discuss the nuts and bolts of fault tolerance in Spark.
We will begin with a brief overview of the sorts of fault tolerance offered, and lead into a deep dive of the internals of fault tolerance. This will include a discussion of Spark on YARN, scheduling, and resource allocation.
We will then spend some time on a case study and discussing some tools used to find and verify fault tolerance issues. Our case study comes from a customer who experienced an application outage that was root caused to a scheduler bug. We discuss the analysis we did to reach this conclusion and the work that we did to reproduce it locally. We highlight some of the techniques used to simulate faults and find bugs.
At the end, we’ll discuss some future directions for fault tolerance improvements in Spark, such as scheduler and checkpointing changes.
Recruiting in the Digital Age: A Social Media MasterclassLuanWise
In this masterclass, presented at the Global HR Summit on 5th June 2024, Luan Wise explored the essential features of social media platforms that support talent acquisition, including LinkedIn, Facebook, Instagram, X (formerly Twitter) and TikTok.
Falcon stands out as a top-tier P2P Invoice Discounting platform in India, bridging esteemed blue-chip companies and eager investors. Our goal is to transform the investment landscape in India by establishing a comprehensive destination for borrowers and investors with diverse profiles and needs, all while minimizing risk. What sets Falcon apart is the elimination of intermediaries such as commercial banks and depository institutions, allowing investors to enjoy higher yields.
Enterprise Excellence is Inclusive Excellence.pdfKaiNexus
Enterprise excellence and inclusive excellence are closely linked, and real-world challenges have shown that both are essential to the success of any organization. To achieve enterprise excellence, organizations must focus on improving their operations and processes while creating an inclusive environment that engages everyone. In this interactive session, the facilitator will highlight commonly established business practices and how they limit our ability to engage everyone every day. More importantly, though, participants will likely gain increased awareness of what we can do differently to maximize enterprise excellence through deliberate inclusion.
What is Enterprise Excellence?
Enterprise Excellence is a holistic approach that's aimed at achieving world-class performance across all aspects of the organization.
What might I learn?
A way to engage all in creating Inclusive Excellence. Lessons from the US military and their parallels to the story of Harry Potter. How belt systems and CI teams can destroy inclusive practices. How leadership language invites people to the party. There are three things leaders can do to engage everyone every day: maximizing psychological safety to create environments where folks learn, contribute, and challenge the status quo.
Who might benefit? Anyone and everyone leading folks from the shop floor to top floor.
Dr. William Harvey is a seasoned Operations Leader with extensive experience in chemical processing, manufacturing, and operations management. At Michelman, he currently oversees multiple sites, leading teams in strategic planning and coaching/practicing continuous improvement. William is set to start his eighth year of teaching at the University of Cincinnati where he teaches marketing, finance, and management. William holds various certifications in change management, quality, leadership, operational excellence, team building, and DiSC, among others.
LA HUG - Video Testimonials with Chynna Morgan - June 2024Lital Barkan
Have you ever heard that user-generated content or video testimonials can take your brand to the next level? We will explore how you can effectively use video testimonials to leverage and boost your sales, content strategy, and increase your CRM data.🤯
We will dig deeper into:
1. How to capture video testimonials that convert from your audience 🎥
2. How to leverage your testimonials to boost your sales 💲
3. How you can capture more CRM data to understand your audience better through video testimonials. 📊
Improving profitability for small businessBen Wann
In this comprehensive presentation, we will explore strategies and practical tips for enhancing profitability in small businesses. Tailored to meet the unique challenges faced by small enterprises, this session covers various aspects that directly impact the bottom line. Attendees will learn how to optimize operational efficiency, manage expenses, and increase revenue through innovative marketing and customer engagement techniques.
The world of search engine optimization (SEO) is buzzing with discussions after Google confirmed that around 2,500 leaked internal documents related to its Search feature are indeed authentic. The revelation has sparked significant concerns within the SEO community. The leaked documents were initially reported by SEO experts Rand Fishkin and Mike King, igniting widespread analysis and discourse. For More Info:- https://news.arihantwebtech.com/search-disrupted-googles-leaked-documents-rock-the-seo-world/
Cracking the Workplace Discipline Code Main.pptxWorkforce Group
Cultivating and maintaining discipline within teams is a critical differentiator for successful organisations.
Forward-thinking leaders and business managers understand the impact that discipline has on organisational success. A disciplined workforce operates with clarity, focus, and a shared understanding of expectations, ultimately driving better results, optimising productivity, and facilitating seamless collaboration.
Although discipline is not a one-size-fits-all approach, it can help create a work environment that encourages personal growth and accountability rather than solely relying on punitive measures.
In this deck, you will learn the significance of workplace discipline for organisational success. You’ll also learn
• Four (4) workplace discipline methods you should consider
• The best and most practical approach to implementing workplace discipline.
• Three (3) key tips to maintain a disciplined workplace.
"𝑩𝑬𝑮𝑼𝑵 𝑾𝑰𝑻𝑯 𝑻𝑱 𝑰𝑺 𝑯𝑨𝑳𝑭 𝑫𝑶𝑵𝑬"
𝐓𝐉 𝐂𝐨𝐦𝐬 (𝐓𝐉 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬) is a professional event agency that includes experts in the event-organizing market in Vietnam, Korea, and ASEAN countries. We provide unlimited types of events from Music concerts, Fan meetings, and Culture festivals to Corporate events, Internal company events, Golf tournaments, MICE events, and Exhibitions.
𝐓𝐉 𝐂𝐨𝐦𝐬 provides unlimited package services including such as Event organizing, Event planning, Event production, Manpower, PR marketing, Design 2D/3D, VIP protocols, Interpreter agency, etc.
Sports events - Golf competitions/billiards competitions/company sports events: dynamic and challenging
⭐ 𝐅𝐞𝐚𝐭𝐮𝐫𝐞𝐝 𝐩𝐫𝐨𝐣𝐞𝐜𝐭𝐬:
➢ 2024 BAEKHYUN [Lonsdaleite] IN HO CHI MINH
➢ SUPER JUNIOR-L.S.S. THE SHOW : Th3ee Guys in HO CHI MINH
➢FreenBecky 1st Fan Meeting in Vietnam
➢CHILDREN ART EXHIBITION 2024: BEYOND BARRIERS
➢ WOW K-Music Festival 2023
➢ Winner [CROSS] Tour in HCM
➢ Super Show 9 in HCM with Super Junior
➢ HCMC - Gyeongsangbuk-do Culture and Tourism Festival
➢ Korean Vietnam Partnership - Fair with LG
➢ Korean President visits Samsung Electronics R&D Center
➢ Vietnam Food Expo with Lotte Wellfood
"𝐄𝐯𝐞𝐫𝐲 𝐞𝐯𝐞𝐧𝐭 𝐢𝐬 𝐚 𝐬𝐭𝐨𝐫𝐲, 𝐚 𝐬𝐩𝐞𝐜𝐢𝐚𝐥 𝐣𝐨𝐮𝐫𝐧𝐞𝐲. 𝐖𝐞 𝐚𝐥𝐰𝐚𝐲𝐬 𝐛𝐞𝐥𝐢𝐞𝐯𝐞 𝐭𝐡𝐚𝐭 𝐬𝐡𝐨𝐫𝐭𝐥𝐲 𝐲𝐨𝐮 𝐰𝐢𝐥𝐥 𝐛𝐞 𝐚 𝐩𝐚𝐫𝐭 𝐨𝐟 𝐨𝐮𝐫 𝐬𝐭𝐨𝐫𝐢𝐞𝐬."
Implicitly or explicitly all competing businesses employ a strategy to select a mix
of marketing resources. Formulating such competitive strategies fundamentally
involves recognizing relationships between elements of the marketing mix (e.g.,
price and product quality), as well as assessing competitive and market conditions
(i.e., industry structure in the language of economics).
RMD24 | Debunking the non-endemic revenue myth Marvin Vacquier Droop | First ...BBPMedia1
Marvin neemt je in deze presentatie mee in de voordelen van non-endemic advertising op retail media netwerken. Hij brengt ook de uitdagingen in beeld die de markt op dit moment heeft op het gebied van retail media voor niet-leveranciers.
Retail media wordt gezien als het nieuwe advertising-medium en ook mediabureaus richten massaal retail media-afdelingen op. Merken die niet in de betreffende winkel liggen staan ook nog niet in de rij om op de retail media netwerken te adverteren. Marvin belicht de uitdagingen die er zijn om echt aansluiting te vinden op die markt van non-endemic advertising.
2. Introduction
Hadoop and related technologies have made it
possible
to store and process data at large scales.
Unfortunately, these data processing
technologies are
not realtime systems.
Hadoop does batch processing instead of
realtime
processing.
Apache Storm 2
5. Introduction
E.g. airline
system
Realtime data processing at massive scale is
becoming
more and more of a requirement for businesses.
The lack of a "Hadoop of realtime" has become
the
biggest hole in the data processing ecosystem.
There's no hack that will turn Hadoop into a
realtime
system.
Apache Storm 5
8. Advantages
Free, simple and open source
Can be used with any programming
language
Very fast
Scalabl
e
Fault -
tolerant
Guarantees your data will be
processed
Integrates with any database
technology
Apache Storm 8
10. Storm vs Hadoop
A Storm cluster is superficially similar to a
Hadoop
cluster.
Hadoop runs "MapReduce jobs", while Storm
runs
"topologies".
Apache Storm 10
11. A MapReduce job eventually finishes,
whereas a
topology processes messages forever (or until
you kill
it).
Spouts and Bolts
Spout
s
Bolts
Apache Storm 11
13. Spouts and Bolts
Bolt 1
Bolt 4
Spout 1 Bolt 2
Spout 2 Bolt 3
A stream is an unbounded sequence of
tuples.
A spout is a source of streams.
Apache Storm 13
14. Spouts and Bolts
Bolt 1
Bolt 4
Spout 1 Bolt 2
Spout 2 Bolt 3
For example, a spout may read tuples off of a
queue and
emit them as a stream.
Apache Storm 14
15. Spouts and Bolts
Bolt 1
Bolt 4
Spout 1 Bolt 2
Spout 2 Bolt 3
A bolt consumes any number of input streams,
does
some processing, and possibly emits new streams.
Apache Storm 15
16. Spouts and Bolts
Bolt 1
Bolt 4
Spout 1 Bolt 2
Spout 2 Bolt 3
Each node (spout or bolt) in a Storm topology
executes
in parallel.
Apache Storm 16
17. Architecture
A machine in a storm cluster may run one or more
worker
processes. Worker Process
Each topology has one or more Task Task
worker
processes.
Each worker process
runs
Task
Task
executors (threads) for a specific
topology.
Each executor runs one or more tasks of the
same
component(spout executor or
bolt).
Apache Storm 17
18. Architecture
Supervisor
Supervisor
Supervisor
Supervisor
Supervisor
ZooKeeper
Nimbus ZooKeeper
ZooKeeper
Hadoop v1 Storm
JobTracker Nimbus
(only1)
. distributescode around cluster
. assigns tasks to machines/supervisors
. failure monitoring
TaskTracker Supervisor . listens for work assigned to its machine
(many) . starts and stops worker processes as necessary b
o
a
n
s
e
N
d
i
mbus
ZooKeeper . coordination between Nimbus and the Supervisors
Apache Storm 18
19. Architecture
The Nimbus and Supervisor are
stateless .
All state is kept in Zookeeper .
1 ZK instance per machine
When the Nimbus or Supervisor fails, they'll start
back
up like nothing happened.
storm jar all-my-code.jar org.apache.storm.MyTopology
arg1 arg2
Apache Storm 19
23. Shuffle grouping :
Randomized
round-robin
Fields grouping: all Tuples
with
the same field value(s) are
always
routed to the same task
Direct grouping: producer of
the
tuple decides which task of
the
consumer will receive the tuple
Apache Storm 23
24. A Sample Code of
Configuring
TopologyBuilder topologyBuilder = new TopologyBuilder();
Apache Storm 24