Azure Service Bus
This discussion aims to be an introduction into building scalable, hybrid cloud/on-prem software.
Outline
What is a Service Bus:
Service Messages
Queues
Publisher Subscriber
Extending Legacy Systems
Fault Tolerance
Scalability
Business Flexibility
Reusability
Powerful BI
Leveraging Event Streams
About the speaker:
Chase Q. Aucoin is currently a .Net Beam-Lead at VinSolutions and has had over a decade of experience creating fault-tolerant distributed systems for companies such as Fed-Ex and Microsoft.
3. ABOUT CHASE Q. AUCOIN
Height: 6’10”
Last name pronounced: Oak-Wine
Current Position: Development Lead of Communications at
VinSolutions
Nearly a decade of data and software architecture experience
Let’s connect on LinkedIn
https://www.linkedin.com/in/chaseaucoin
4. HIGH LEVEL OUTLINE
What is a Service Bus
Service Messages
Queues
Publisher Subscriber
Extending Legacy Systems
Fault Tolerance
Scalability
Business Flexibility
Reusability
Powerful BI
Leveraging Event Streams
Questions & Answers / Shameless Self Promotion
6. CHALLENGES IN THE INDUSTRY
Volume of transactions on exponential trend
Businesses want to leverage existing data without wholesale
rebuilding platforms
Businesses want/need analytics closer to “real-time”
Processes need to be able to have intermittent unavailability
7. WHAT IS A SERVICE BUS
“An enterprise service bus (ESB) is a software architecture model used
for designing and implementing communication between mutually
interacting software applications in a service-oriented architecture
(SOA). As a software architectural model for distributed computing it
is a specialty variant of the more general client server model and
promotes agility and flexibility with regard to communication
between applications. Its primary use is in enterprise application
integration (EAI) of heterogeneous and complex landscapes.” -
Wikipedia
8. WHAT THE HECK DOES THAT
MEAN?
Service Bus: A layer of abstraction between clients and services to
relay inputs and outputs.
Inputs and outputs are “Messages”
9. SERVICE MESSAGES
Messages are persisted quickly
Messages are asynchronous by their nature
Messages are still there even if processing goes offline
Messages that can’t be processed can be replayed later
12. LOOSELY COUPLED ENTERPRISE
PLATFORM
How do we keep these systems that are prone to failure “fault
tolerant”?
How do we scale to keep up with the ever increasing demands of our
consumers?
How do we meet the constant demands of business to enrich our data
and deliver new/relevant value when we have systems that are brittle
and complex?
How do we get the most out of the systems we have invested in?
13. SCENARIO 1 – FAULT TOLERANCE
How do we keep these systems that are prone to failure “fault
tolerant”?
14. SAMPLE USE CASE - CUSTOMER
Client
Client
Client
Client
Customer Service
Data Persistence Data Source
15. SERVICE FAILURE – CUSTOMER
Client
Client
Client
Client
Customer Service
Data Persistence Data Source
16. SERVICE FAILURE – CUSTOMER
Client
Client
Client
Client
Customer Service
Data Persistence Data Source
17. WHY DOES THIS SUCK?
Your clients expect quick responses from your services.
Your clients should still be able to operate even when the underlying
service experience intermittent downtime.
18. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
19. SERVICE FAILURE – CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
20. SERVICE FAILURE – CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
21. TO MANY THINGS FROM THE
STUFF!!
How do we handle critical times when our clients need us most?
22. SCENARIO 2 - SCALE
How do we scale to keep up with the ever increasing demands of our
consumers?
23. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
24. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
Data
Persiste
nce
25. Cloud Deployment
SAMPLE USE CASE – CUSTOMER –
CLOUD SCALE
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
Data
Persiste
nce
Data
Persiste
nce
Data
Persiste
nce
27. BUT… WE HAVE INVESTED IN ARC
WELDERS!
Shhhhh… It’ll be ok.
28. On-Premises DeploymentCloud Deployment
SAMPLE USE CASE – CUSTOMER –
HYBRID SCALE
Insert
Update
Insert
Delete
DB
Custome
r
Activities Data
Persiste
nce
Data
Persiste
nce
Worker
Worker
Worker
Worker
30. SCENARIO 3 – EXTENDING
SYSTEMS
How do we meet the constant demands of business to enrich our data
and deliver new/relevant value when we have systems that are brittle
and complex?
31. CAPTAIN PRODUCT SAYS:
Wow.. This thing is running great! Since we “let you” re-architect the
system we have some great feature ideas…
32. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
Data
Persiste
nce
33. CAPTAIN PRODUCT SAYS:
We need to notify people when a customer is added. Just add it to
that new service you wrote.
We’ve got a lot of pressure to get this done. Yesterday. Earlier, if
possible.
35. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persistence
& Notification
Data
Persistence
& Notification
36. CAPTAIN PRODUCT SAYS:
Oh that’s great!!! Now we need to know what happens with these
customers so we can have a audit trail and accountability…
We’ve got a lot of pressure to get this done. Yesterday. Earlier, if
possible.
38. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persistence
& Notification
& Auditing
Data
Persistence
& Notification
& Auditing
39. WHY THIS SUCKS
Your services are back to taking on too much responsibility.
The more responsibility, the more chance for failure and the harder to
fix.
We are now embedding services in our service that could be extended
to support more of our system and provide better value to our
organization as a whole.
40. PRO-TIP
Our goal as technologist is to think about the long term. We are good
at this. Be a good steward of your company and your co-workers.
Don’t let product and short sided goals dictate technology decisions.
Be consultative! Think about their long term needs. Choose
technology & strategy accordingly. Set them up for success, not
failure.
Remember you aren’t responsible for the success of this one project
or task. Really you are responsible for the long term success of your
whole company.
Believe it, or not.
41. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Data
Persiste
nce
Data
Persiste
nce
42. SAMPLE USE CASE - CUSTOMER
Insert
Update
Insert
Delete
DB
Customer
Activities
Customer
Auditing
Customer
Auditing
Data
Persistenc
e
Data
Persistenc
e
All
All
DB
43. SCENARIO 4 – REUSABLE SYSTEMS
How do we keep these systems that are prone to failure “fault
tolerant”?
44. SAMPLE USE CASE – NOTIFICATION
Notification Service:
Gets All Data
From Sales,
Customers, And
Inventory.
Synchronously
Notifies
On Selected
Channels Based on
Changes
Data Source
Email Provider
SMS Provider
Push Notifications Provider
45. WHY DID WE DO THIS
Well at the time, it made sense. We didn’t have very many records
when we created it and we only sent emails. Then over the years we
just kind of added on to it.
47. SAMPLE USE CASE - CUSTOMER
DB
Custom
er
Activitie
s
Notificatio
n
Notificatio
n
Data
Persistenc
e
Data
Persistenc
e
Inserts
All
Notificatio
ns
Email
Text
Push
Notificatio
n
Notificatio
n
Notificatio
n
48. SAMPLE USE CASE - CUSTOMER
Customer
Notificatio
n
Inventory
Notificatio
n
Notificatio
ns
Email
Text
Push
Worker
Worker
Worker
Sales
Notificatio
n
49. ProcessProcess
SAMPLE USE CASE - NOTIFICATION
Customer
Notificatio
n
Inventory
Notificatio
n
Notificatio
ns
Email
Text
Push
Worker
Worker
Worker
Sales
Notificatio
n
50. LEVERAGING EVENT STREAMS
Well… You got the basics. Now it is time to do something amazing
with all of this extensible data.
How about some real time reporting?
51. NOT TODAY :P
Come check out my talk at SQL Saturday
http://www.sqlsaturday.com/444/eventhome.aspx
52. SHAMELESS SELF PROMOTION
Limited availability for consultation, mentorship, and training!
Learn: Patterns, Practices, WCF @ Scale, Cloud Systems, Async
Programming.
Phone/Text: 816-209-0033
https://www.linkedin.com/in/chaseaucoin
chase@cupcakefactory.net
Cupcake Factory – Distributed services made easy. Ask about alpha
access!
http://cupcakefactory.net