SlideShare a Scribd company logo
1 of 17
Download to read offline
An Auto-Documenting
Event-Driven
Architecture Using
Async API and Inngest
Dan Farrelly
@djfarrelly
Why event schemas are important
Why automatically generate event schemas
How we automatically generate event schemas
How you can use it yourself
2
What we’ll cover
Overview
Founder, Inngest.com
Former CTO, Buffer.com
Event-driven architecture and home restoration enthusiast
dan@inngest.com
3
Dan Farrelly
Who am I?
Inngest is an event-driven platform that makes it easy for developers to
build, test, then deploy serverless functions triggered by events — without
worrying about infrastructure, queues, or stateful services.
4
What is Inngest?
Who am I?
“An event-driven architecture in a box”
Features: schema registry, logging, metrics, SDKs and tooling
5
Why build event-driven?
But first,
Event-driven can be beautifully simple
You publish immutable facts & you run code based on those facts
You get flexibility with fan-out - n consumers for a single event
A highly decoupled architecture
What’s the most important
part of an EDA?
<Hint: it’s in the name>
7
Events!
What’s the most important part of an EDA?
If events are facts, they cannot be changed
The system needs to make sure that services (read: teams) are publishing
facts correctly
To accomplish this, we need to document these facts and add
controls to our system
Enter AsyncAPI…
8
How to create AsyncAPI specs?
AsyncAPI
1) Write it manually
2) Automatically parse events in real-time and self-document
9
Automatically parsing events
Why write specs?
Inngest uses HTTP + JSON
Each event received is sent to a worker that parses & generates a schema
Clients can specify a new event version by using a new v field value
Every new version is stored in the registry in a meta format: CUE
10
What is CUE? Why use it?
cuelang.org
Chosen for interoperability
CUE is a data validation format that can
parse, validate and generate JSON, YAML,
Open API spec (JSON Schema), and Go structs.
Our custom parser walks any JSON object,
parses the type of each field and generates
a CUE definition.
11
How it works
Event API
(https://inn.gs/)
Pub/Sub
Event Recorder
Schema Registry
Schema
Validation
CUE Parser
Job Scheduler
Serverless Functions
12
Inngest Schema Registry
13
Self-documenting API + Async API
Match made in heaven
This enables us to fetch JSON schema right from the Inngest API and
build our Async API spec.
Ever user workspace has its own event registry, each event requires a
name and is versioned
Everything is wrapped up into a simple npm package
14
Why translate to Async API?
Async API
We can tap into the Async API ecosystem
Tooling: Code generation, etc.
Documentation: Standard format for sharing system architecture
Future: Use Async API to publish to schema registry; Schema
enforcement tools
15
See it in action!
npm install inngest-cli && inngest login
npx @inngest/asyncapi
16
That’s it!
Questions?
Thanks for listening!
@djfarrelly

More Related Content

Similar to An Auto-Documenting Event-Driven Architecture Using Async API and Inngest - AsyncAPI Conf

apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...
apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...
apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...apidays
 
Spring Boot & Spring Cloud on Pivotal Application Service
Spring Boot & Spring Cloud on Pivotal Application ServiceSpring Boot & Spring Cloud on Pivotal Application Service
Spring Boot & Spring Cloud on Pivotal Application ServiceVMware Tanzu
 
Open Event API
Open Event APIOpen Event API
Open Event APIAvi Aryan
 
Running High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclioRunning High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclioiguazio
 
Serverless Application Development with SAM
Serverless Application Development with SAMServerless Application Development with SAM
Serverless Application Development with SAMAmazon Web Services
 
Using Data Science & Serverless Python to find apartment in Toronto
Using Data Science & Serverless Python to find apartment in TorontoUsing Data Science & Serverless Python to find apartment in Toronto
Using Data Science & Serverless Python to find apartment in TorontoDaniel Zivkovic
 
SHOWDOWN: Threat Stack vs. Red Hat AuditD
SHOWDOWN: Threat Stack vs. Red Hat AuditDSHOWDOWN: Threat Stack vs. Red Hat AuditD
SHOWDOWN: Threat Stack vs. Red Hat AuditDThreat Stack
 
Serverless, oui mais pour quels usages ?
Serverless, oui mais pour quels usages ?Serverless, oui mais pour quels usages ?
Serverless, oui mais pour quels usages ?VMware Tanzu
 
solution Challenge design and flutter day.pptx
solution Challenge design and flutter day.pptxsolution Challenge design and flutter day.pptx
solution Challenge design and flutter day.pptxGoogleDeveloperStude22
 
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...Ajeet Singh Raina
 
Itb 2021 - Bulding Quick APIs by Gavin Pickin
Itb 2021 - Bulding Quick APIs by Gavin PickinItb 2021 - Bulding Quick APIs by Gavin Pickin
Itb 2021 - Bulding Quick APIs by Gavin PickinGavin Pickin
 
Four Scenarios for Using an Integration Service Environment (ISE)
Four Scenarios for Using an Integration Service Environment (ISE)Four Scenarios for Using an Integration Service Environment (ISE)
Four Scenarios for Using an Integration Service Environment (ISE)Daniel Toomey
 
Api-First service design
Api-First service designApi-First service design
Api-First service designStefaan Ponnet
 
Introduction and hacking OpenStack, Pycon India
Introduction and hacking OpenStack,  Pycon IndiaIntroduction and hacking OpenStack,  Pycon India
Introduction and hacking OpenStack, Pycon IndiaAtul Jha
 
Building CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless ApplicationsBuilding CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless ApplicationsDaniel Zivkovic
 
Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)Yan Cui
 
S4: Distributed Stream Computing Platform
S4: Distributed Stream Computing PlatformS4: Distributed Stream Computing Platform
S4: Distributed Stream Computing PlatformAleksandar Bradic
 
Public v1 real world example of azure functions serverless conf london 2016
Public v1 real world example of azure functions serverless conf london 2016 Public v1 real world example of azure functions serverless conf london 2016
Public v1 real world example of azure functions serverless conf london 2016 Yochay Kiriaty
 
Ultimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on KubernetesUltimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on Kuberneteskloia
 
Continuous Delivery - Voxxed Days Cluj-Napoca 2017
Continuous Delivery - Voxxed Days Cluj-Napoca 2017Continuous Delivery - Voxxed Days Cluj-Napoca 2017
Continuous Delivery - Voxxed Days Cluj-Napoca 2017Rafał Leszko
 

Similar to An Auto-Documenting Event-Driven Architecture Using Async API and Inngest - AsyncAPI Conf (20)

apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...
apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...
apidays LIVE London 2021 - Getting started with Event-Driven APIs by Hugo Gue...
 
Spring Boot & Spring Cloud on Pivotal Application Service
Spring Boot & Spring Cloud on Pivotal Application ServiceSpring Boot & Spring Cloud on Pivotal Application Service
Spring Boot & Spring Cloud on Pivotal Application Service
 
Open Event API
Open Event APIOpen Event API
Open Event API
 
Running High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclioRunning High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclio
 
Serverless Application Development with SAM
Serverless Application Development with SAMServerless Application Development with SAM
Serverless Application Development with SAM
 
Using Data Science & Serverless Python to find apartment in Toronto
Using Data Science & Serverless Python to find apartment in TorontoUsing Data Science & Serverless Python to find apartment in Toronto
Using Data Science & Serverless Python to find apartment in Toronto
 
SHOWDOWN: Threat Stack vs. Red Hat AuditD
SHOWDOWN: Threat Stack vs. Red Hat AuditDSHOWDOWN: Threat Stack vs. Red Hat AuditD
SHOWDOWN: Threat Stack vs. Red Hat AuditD
 
Serverless, oui mais pour quels usages ?
Serverless, oui mais pour quels usages ?Serverless, oui mais pour quels usages ?
Serverless, oui mais pour quels usages ?
 
solution Challenge design and flutter day.pptx
solution Challenge design and flutter day.pptxsolution Challenge design and flutter day.pptx
solution Challenge design and flutter day.pptx
 
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
 
Itb 2021 - Bulding Quick APIs by Gavin Pickin
Itb 2021 - Bulding Quick APIs by Gavin PickinItb 2021 - Bulding Quick APIs by Gavin Pickin
Itb 2021 - Bulding Quick APIs by Gavin Pickin
 
Four Scenarios for Using an Integration Service Environment (ISE)
Four Scenarios for Using an Integration Service Environment (ISE)Four Scenarios for Using an Integration Service Environment (ISE)
Four Scenarios for Using an Integration Service Environment (ISE)
 
Api-First service design
Api-First service designApi-First service design
Api-First service design
 
Introduction and hacking OpenStack, Pycon India
Introduction and hacking OpenStack,  Pycon IndiaIntroduction and hacking OpenStack,  Pycon India
Introduction and hacking OpenStack, Pycon India
 
Building CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless ApplicationsBuilding CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless Applications
 
Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)Serverless in production (O'Reilly Software Architecture)
Serverless in production (O'Reilly Software Architecture)
 
S4: Distributed Stream Computing Platform
S4: Distributed Stream Computing PlatformS4: Distributed Stream Computing Platform
S4: Distributed Stream Computing Platform
 
Public v1 real world example of azure functions serverless conf london 2016
Public v1 real world example of azure functions serverless conf london 2016 Public v1 real world example of azure functions serverless conf london 2016
Public v1 real world example of azure functions serverless conf london 2016
 
Ultimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on KubernetesUltimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on Kubernetes
 
Continuous Delivery - Voxxed Days Cluj-Napoca 2017
Continuous Delivery - Voxxed Days Cluj-Napoca 2017Continuous Delivery - Voxxed Days Cluj-Napoca 2017
Continuous Delivery - Voxxed Days Cluj-Napoca 2017
 

Recently uploaded

How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 

Recently uploaded (20)

DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 

An Auto-Documenting Event-Driven Architecture Using Async API and Inngest - AsyncAPI Conf

  • 1. An Auto-Documenting Event-Driven Architecture Using Async API and Inngest Dan Farrelly @djfarrelly
  • 2. Why event schemas are important Why automatically generate event schemas How we automatically generate event schemas How you can use it yourself 2 What we’ll cover Overview
  • 3. Founder, Inngest.com Former CTO, Buffer.com Event-driven architecture and home restoration enthusiast dan@inngest.com 3 Dan Farrelly Who am I?
  • 4. Inngest is an event-driven platform that makes it easy for developers to build, test, then deploy serverless functions triggered by events — without worrying about infrastructure, queues, or stateful services. 4 What is Inngest? Who am I? “An event-driven architecture in a box” Features: schema registry, logging, metrics, SDKs and tooling
  • 5. 5 Why build event-driven? But first, Event-driven can be beautifully simple You publish immutable facts & you run code based on those facts You get flexibility with fan-out - n consumers for a single event A highly decoupled architecture
  • 6. What’s the most important part of an EDA? <Hint: it’s in the name>
  • 7. 7 Events! What’s the most important part of an EDA? If events are facts, they cannot be changed The system needs to make sure that services (read: teams) are publishing facts correctly To accomplish this, we need to document these facts and add controls to our system Enter AsyncAPI…
  • 8. 8 How to create AsyncAPI specs? AsyncAPI 1) Write it manually 2) Automatically parse events in real-time and self-document
  • 9. 9 Automatically parsing events Why write specs? Inngest uses HTTP + JSON Each event received is sent to a worker that parses & generates a schema Clients can specify a new event version by using a new v field value Every new version is stored in the registry in a meta format: CUE
  • 10. 10 What is CUE? Why use it? cuelang.org Chosen for interoperability CUE is a data validation format that can parse, validate and generate JSON, YAML, Open API spec (JSON Schema), and Go structs. Our custom parser walks any JSON object, parses the type of each field and generates a CUE definition.
  • 11. 11 How it works Event API (https://inn.gs/) Pub/Sub Event Recorder Schema Registry Schema Validation CUE Parser Job Scheduler Serverless Functions
  • 13. 13 Self-documenting API + Async API Match made in heaven This enables us to fetch JSON schema right from the Inngest API and build our Async API spec. Ever user workspace has its own event registry, each event requires a name and is versioned Everything is wrapped up into a simple npm package
  • 14. 14 Why translate to Async API? Async API We can tap into the Async API ecosystem Tooling: Code generation, etc. Documentation: Standard format for sharing system architecture Future: Use Async API to publish to schema registry; Schema enforcement tools
  • 15. 15 See it in action! npm install inngest-cli && inngest login npx @inngest/asyncapi