1.
Dan Farrelly
Founder, Inngest
Building a Pluggable,
Cloud-native Event-driven
Serverless Architecture

2.
● What is Inngest?
● What did we set out to build and why?
● Our approach
● Principles of the system design
● Where we are today and where we’re headed
Outline
What we’ll cover

3.
Dan Farrelly
● Founder, Inngest.com
● Former CTO @ Buffer.com
● Interests:
○ Event-driven systems
○ Home restoration
● dan@inngest.com
● @djfarrelly
Who is speaking to you?

4.
About Inngest
● Inngest is an event-driven platform that makes it easy for developers
to build, test, then deploy serverless functions triggered by events —
without worrying about infrastructure, queues, or stateful services.
● “An event-driven architecture in a box”
Features: schema registry, logging, metrics, SDKs and tooling
● Inngest: A Platform + Open Source project
● Our Mission:
“To accelerate the adoption of event-based architecture.”
The backstory

5.
Why this goal?

6.
● We think event-driven can be a beautifully simple way to build
● There currently is a large learning curve
● There is lack of awareness to broad developer community
● We want to introduce event-driven systems to developers in an
approachable way.
Why this goal?
The backstory

7.
High level system design ideas
The backstory
● A fundamentally simple system design
○ Make it approachable
○ Provide an abstraction layer with batteries included
○ Enable reactive applications
● Inngest combines event streams with job scheduling in an easy to
use system
● It should be easy to build applications for
● There shouldn’t be a huge learning curve

8.
● Great Developer Experience (DX)
● A cloud-native solution (Either hosted or Self-hosted)
○ The cloud solution needs to be a pluggable architecture (Open Core)
Our focuses
The backstory

9.
● Developer SDK
○ Define functions, not consumers
○ SDK generates the configuration necessary
and guide the developer while writing code
● Core user needs
○ Consumers are defined as functions a in
standard format
○ Simple to send (publish) events (just JSON)
○ Versioning, throttling, idempotency,
concurrency should all be included
Simple to build
Developer Experience

10.
● Functions should be able to run in different runtime environments
○ HTTP
○ Docker runtimes (local, Kubernetes, etc.)
○ Serverless runtimes (e.g. Lambda)
● SDK can enable write-once-ship-anywhere
● Mix and match - depending on use-case
Runtime Agnostic
Developer Experience

11.
● Automatic event schema parsing and storage
● Type generation from event schemas
● Tight integration with SDK for easy use of Types
● Tooling to share types across teams/services/repos
Event Schema Registry & Types
Developer Experience
$ inngest types ts -o __types__.ts

12.
● Principle: Run on any cloud (or your own machine)
○ Cloud: It should support services that are common and can be spun up on different
cloud providers (e.g. Postgres, Redis, Kafka)
○ Local: It should be able to run in-memory for local dev or end-to-end testing
● Drivers are the way
Platform Agnostic
Pluggable Architecture

13.
High-Level Architecture
Pluggable Architecture

14.
● Event API - Receives events,
handles publishing authn & authz
● Runner - Consumes & records
events, determines functions to
execute (as job).
● Executor - Executes function
steps from job queue on external
runtimes, manages job state.
● Core API - CRUD for system data
(functions, keys, events, logs).
Services
Pluggable Architecture

15.
● Event Stream - e.g. PubSub,
Kafka, NATS.
● DataStore - System data (keys,
functions configuration, history);
e.g. Postgres
● Job Queue - Functions to be
executed; e.g. Redis, SQS
● Job State - Job context, pauses,
retry & failure info; e.g Redis
Swappable Components (Drivers)
Pluggable Architecture

16.
● Function Runtimes - System drivers
can “enable” different runtime
targets.
○ Runtime: Docker / Target: local docker
client, Kubernetes, Nomad, ECS
○ Runtime: HTTP / Target: Your web server
○ Runtime: Serverless / Target: AWS
Lambda, etc.
○ Future? WASM?
Runtimes
Pluggable Architecture

17.
● Single-dependency development environment (inngest dev)
○ Often challenging for developers to emulate production environments on their
machines (e.g. docker compose up -f everything.yml)
○ Avoid costs and awkwardness of shared cloud resources
○ A UI for visual developers
● Enables E2E testing in CI/CD
In-memory Implementations
Pluggable Architecture

18.
● Decoupling points choosing to enable:
○ Customized deployments for special needs
○ Hosted-Self/hosted hybrid setups (BYO Kafka stream or Kubernetes cluster)
● Running multi-cloud or multi-region (e.g. for data sovereignty)
Cloud-Hybrid Configurations
Pluggable Architecture

19.
● Golang
○ You probably could have guessed this.
○ Leaning into packages like Go Cloud
Development Kit (Go CDK) [gocloud.dev]
to leverage fantastic community drivers
for various cloud services.
○ A single binary
● Learning through operating a cloud
service backed by the code
How we’re building it
Pluggable Architecture
Inngest Cloud

20.
● In-memory implementations of all drivers
● Drivers MVP (Postgres, Redis)
● “Cloud stacks” via Terraform
○ AWS Hosted stack (ECS, SQS, Elasticache)
● Seeking feedback
Where we’re at today
Pluggable Architecture
$ npx inngest-cli@latest dev

21.
● Focusing on DX (SDK, local-to-prod
workflow tooling, etc.)
● Step Functions
○ Delays
○ Event coordination
○ Conditional steps/workflows
● Additional SDKs (e.g. Python)
● Additional runtimes (e.g. AWS
Lambda, GCP Cloud Functions)
Roadmap
Pluggable Architecture

22.
● Inngest’s core will be open source
○ We want to build the system that allows developers to build reactive systems that is
powerful and approachable
○ Still more work to both extract and upstream code
● First step taken this year - way more to come!
High Level Vision & Outlook
The Future

23.
● inngest.com
● Project Repo → github.com/inngest/inngest
○ SDK (New) → github.com/inngest/inngest-js
○ AWS Cloud Stack → bit.ly/inngest-aws-stack
● Join the discussion + Provide feedback
○ inngest.com/discord
○ @inngest
● dan@inngest.com
Over to you
The Slide with Links

24.
Thanks!
github.com/inngest/inngest
inngest.com | @inngest | dan@inngest.com | @djfarrelly