SlideShare a Scribd company logo
The Future of Application Development
API Days — Melbourne 2023
David Peterson
Principal Solutions Engineer, APAC
Stream
Processing
ϰ
Kappa Generative
AI
The Future of Application Development - API Days - Melbourne 2023
Grady Booch
“No matter what future we may envision, it relies
on software that has not yet been written”
The Future of Application Development - API Days - Melbourne 2023
Big Data
More == Better
Data in Motion
Timely == Better
Value
of
Data
->
Time ->
Value
of
Data
->
Time ->
ϰ
Kappa Architecture
The Future of Application Development - API Days - Melbourne 2023
Publish /
Subscribe
Store Process
CDC
Connect
MongoDB
Connect
Kafka
MongoDB
Connect
CDC Events Batch Txns
Connect
Schema
Kappa
Schema
MongoDB
Connect
CDC Events Batch Txns
Connect
Kappa
CDC Events Batch Txns
Connect
MongoDB VectorDB
Postgres
Connect
Replay ^^
Kappa
Source of Truth
}
Stream Processing
Old world
Event processor
New world
Stream processor
Event == Fact
☹
Events == Story
"
#
$
% ☹
1. TOPIC
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
{“actor”:”bear”,
“x”:410, “y”:20}
{“actor”:”racoon”,
“x”:380, “y”:20}
{“actor”:”bear”,
“x”:380, “y”:22}
{“actor”:”racoon”,
“x”:350, “y”:22}
2.STREAM
unbounded, continuously updating data set, ordered,replayable, fault-tolerant
The Future of Application Development - API Days - Melbourne 2023
3.TABLE
Exposure Sheet
Changelog stream – immutable events
Stream-table Duality
CDC Events Batch Txns
Connect
MongoDB VectorDB
Postgres
Connect
Kappa
Materialised Views
{
CDC Events Batch Txns
MongoDB VectorDB
Postgres
Connect
Connect
Stream Processing
Consumer and producer ^^
}
Kappa — Stream Processing
Processors
• Kafka Streams
• ksqlDB
• Flink
Context — Customer 360
32
Building our Customer Base
Just in time
Fast Batch
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
CREATE TABLE ORGANISATION_CUSTOMER_CUSTOMERS_360
WITH (
KAFKA_TOPIC = 'ORGANISATION_CUSTOMER_CUSTOMERS_360'
)
AS SELECT
CU.CUSTOMER_ID AS CUSTOMER_ID,
FIRST_NAME AS FIRST_NAME,
LAST_NAME AS LAST_NAME,
STRUCT(
STREET_ADDRESS := CU.ADDRESS->STREET_ADDRESS,
SUBURB := CU.ADDRESS->SUBURB,
POSTCODE := CU.ADDRESS->POSTCODE
) ADDRESS,
STRUCT(
VENDOR := L.VENDOR,
POINTS := L.POINTS,
TRANSACTION_ID := L.TRANSACTION_ID
) LATEST_LOYALTY_TRANSACTION,
STRUCT(
TOTAL_POINTS := L.TOTAL_POINTS,
AVERAGE_POINTS := L.AVERAGE_POINTS
) LOYALTY_POINTS,
STRUCT(
SCORE := CR.CREDIT_SCORE
) CREDIT_RATING
FROM ORGANISATION_CUSTOMER_CUSTOMERS_STATE CU
LEFT OUTER JOIN ORGANISATION_CREDIT_RATING_STATE CR ON ((CU.CUSTOMER_ID = CR.CUSTOMER_ID))
LEFT OUTER JOIN ORGANISATION_LOYALTY_TRANSACTIONS_STATE L ON ((CU.CUSTOMER_ID = L.CUSTOMER_ID))
PARTITION BY CU.CUSTOMER_ID
EMIT CHANGES;
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
Generative AI
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
Kurzweil - 2014
“I am more optimistic and put the date at
2029…”
Ilya Sutskever - 2022
“In the future, it will be obvious that the sole
purpose of science was to build AGI”
/imagine prompt:Street style photo of a woman shot on Kodak
July 2023
July 2022
LLM
Ilya Sutskever
“Shocking degree of understanding about the
world”
The Future of Application Development - API Days - Melbourne 2023
CDC Events Batch Txns
MongoDB VectorDB
Postgres
Connect
Connect
Stream Processing
Consumer and producer ^^
}
Microservice
Generate the prompt.
Powered by C360
Topic
Launch Day
• New / Private data
• Context
• Memory
• Integrating into business
Hard Problems with LLMs / Gen AI
Hard Problem — Private data
{"release_date": "2023-05-02", "shoe_name":"ActivExhale Ultra Light", "press_release": "We're proud to present a groundbreaking shoe crafted with world
class support: the ActivExhale Ultra Light. The shoe's ultra flexible design means your feet are relieved of any stress and adapted for the greatest
comfort. With reinforced soles, shock absorption technology, and its hi-tech mesh pattern, you can walk and move freely in this shoe, no matter the
terrain!"}
{"release_date":"2023-10-28", "shoe_name":"TailorSport Edge", "press_release": "Introducing TailorSport Edge, our newest range of shoes created for
performance and agility. The special woven upper materials adapt to the shape and size of your foot, while the cushioning system provides outstanding
comfort and responsiveness of your feet whilst running or exercising. Its cutting edge technology allows your foot to experience a new level of
performance stylishly.”}
{"release_date":"2023-03-29", "shoe_name":"EasyCloud OG", "press_release": "A fan favorite, the EasyCloud OG, is now available. The OG redefines the
ultimate leisure-style walking shoe thanks to its synthetic cushioning and responsive surface. The unobtrusive materials make walking in this shoe
fast, sleek, and comfortable. The sole hugs your foot perfectly, keeping each stride light yet sure. It's time for effortless style and energetic
running."}
{"release_date":"2023-06-11", "shoe_name":"Smart Grip Trailblazer","press_release": "Designed for trail hikes, the Smart Grip Trailblazer is here! This
all-in-one shoe offers unbeatable traction on uneven surfaces, perfect padding, and fair water resistance - offering you the ideal blend of versatility
and control. Step into a smarter hiking experience: Try the Trailblazer on all kinds of terrains and enjoy comfort from the first step!”}
{"release_date": "2023-07-24", "shoe_name":"Prince Stables Flexy", "press_release": "Welcome the Prince Stables Flexy, designed with comfort,
stability, and full range of motion in mind. Enjoy cornering soles which won’t let you down while relying on shock pads to provide keep your feet safe
from exhaustion. Slip into convenience and agility - With heat forming technology, the practice makes perfection philosophy takes on a whole new
meaning."}
{"release_date":"2023-11-15", "shoe_name":"New Skymax runners", "press_release": "Experience extreme comfort with every step with the new Skymax
runners. Specifically engineered with superior fit, this shoe has lightweight cushioning with ultra high energy return - keeps you powerfully pushing
forward all day. Step up your run as you breeze through long-distances — with the smoothest cushion system and breathable material, get the best of the
best this season and beyond!”}
{"release_date":"2023-02-06", "shoe_name":"FuturiZen FreedomLite", "press_release": "Fusing clean aesthetic with personalized functionary technology,
FuturiZen FreedomLite is a groundbreaking product in footwear with its contouring uppers, featherlight midsole, and laser molded outsole. Walk away in
comfort and confidence - experience extreme lightweight connectivity wherever when you take on everyday life thanks to Versatile Cushion crossovers
within the sole."}
{"release_date":"2023-09-12", "shoe_name":"AiraSaint OnLiners", "press_release": "Beeline into AiraSaint OnLiners for lightweight, supportive running
at all distances. Situated with signature mesh fabrics and forefoot structures, OnLiners is a hybrid style meet first choice that features extreme
breathability but also encourages toe-splaying. Benefit from optimal fit all around your feet as you fasten laces - Infinity Comfort Seams direct your
each movement towards your desired outcome with great bouncy response."}
Hard Problem — Private data
The Future of Application Development - API Days - Melbourne 2023
Hard Problem — Context
The Future of Application Development - API Days - Melbourne 2023
Hard Problem — Long-term Memory
The Future of Application Development - API Days - Melbourne 2023
Hard Problem — Integration
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
Yuval Noah Harari — Sapiens
“Consistency is the playground of dull minds”
The Future of Application Development - API Days - Melbourne 2023
• API != REST, depends on use-case
• Kappa > Kafka — simplify your architecture
• Schema — data contracts
• Source of Truth
• Raw / Enriched data pattern
• Stream Processing — smart pipes
• Events tell a story
Confluent Cloud
Thank you
Q&A

More Related Content

Similar to The Future of Application Development - API Days - Melbourne 2023

(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介
(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介
(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介
Masahiko Ebisuda
 
Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....
Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....
Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....
Alfredo BALLESTER FERNÁNDEZ
 
Casio protrek watches designed for durability
Casio protrek watches   designed for durabilityCasio protrek watches   designed for durability
Casio protrek watches designed for durability
CityWatches UK
 
Casio protrek watches designed for durability
Casio protrek watches   designed for durabilityCasio protrek watches   designed for durability
Casio protrek watches designed for durability
CityWatches UK
 
【IVS CTO Night & Day】Amazon Container Services
【IVS CTO Night & Day】Amazon Container Services【IVS CTO Night & Day】Amazon Container Services
【IVS CTO Night & Day】Amazon Container Services
Amazon Web Services Japan
 
Economies of Scaling Software
Economies of Scaling SoftwareEconomies of Scaling Software
Economies of Scaling Software
Joshua Long
 
Taking your Dev Team on the UX Journey
Taking your Dev Team on the UX JourneyTaking your Dev Team on the UX Journey
Taking your Dev Team on the UX Journey
BSGAfrica
 
Buying Tips Soccer Shoes
Buying Tips Soccer ShoesBuying Tips Soccer Shoes
Buying Tips Soccer Shoes
Tim Michael
 
Cheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m us
Cheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m usCheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m us
Cheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m us
super agency
 
Sports technology
Sports technologySports technology
Sports technology
GarthPereira
 
Big Data Expo 2015 - MapR Impacting Business As It Happens
Big Data Expo 2015 - MapR Impacting Business As It HappensBig Data Expo 2015 - MapR Impacting Business As It Happens
Big Data Expo 2015 - MapR Impacting Business As It Happens
BigDataExpo
 
Blackknight USA
Blackknight USABlackknight USA
Blackknight USA
blackknightusa
 
Black Knight USA
Black Knight USABlack Knight USA
Black Knight USA
Raymond Lindsay
 
SCG Virtual Reality top news q1 2016
SCG  Virtual Reality top news q1 2016SCG  Virtual Reality top news q1 2016
SCG Virtual Reality top news q1 2016
Chris Rigatuso
 
Immersive Web
Immersive WebImmersive Web
Using XML to Create Once - Distribute Everywhere
Using XML to Create Once - Distribute EverywhereUsing XML to Create Once - Distribute Everywhere
Using XML to Create Once - Distribute Everywhere
Jeffrey Stewart
 
Azure for gaming
Azure for gamingAzure for gaming
Azure for gaming
Angelo Gino Varrati
 
Realtà aumentata ed Azure, un binomio imbattibile
Realtà aumentata ed Azure, un binomio imbattibileRealtà aumentata ed Azure, un binomio imbattibile
Realtà aumentata ed Azure, un binomio imbattibile
Alessio Iafrate
 
How Silicon Valley startups are approaching security differently
How Silicon Valley startups are approaching security differentlyHow Silicon Valley startups are approaching security differently
How Silicon Valley startups are approaching security differently
Scott Cressman
 
Storage Visibility for Operations - A Ceph Story
Storage Visibility for Operations - A Ceph StoryStorage Visibility for Operations - A Ceph Story
Storage Visibility for Operations - A Ceph Story
Debojyoti Dutta
 

Similar to The Future of Application Development - API Days - Melbourne 2023 (20)

(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介
(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介
(おそらく)日本最速!_Microsoft Ignite 2019からのHybrid_Edge関連最新トピック紹介
 
Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....
Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....
Fast, portable & low cost 3D foot digitizers: validity & reliability @3DBody....
 
Casio protrek watches designed for durability
Casio protrek watches   designed for durabilityCasio protrek watches   designed for durability
Casio protrek watches designed for durability
 
Casio protrek watches designed for durability
Casio protrek watches   designed for durabilityCasio protrek watches   designed for durability
Casio protrek watches designed for durability
 
【IVS CTO Night & Day】Amazon Container Services
【IVS CTO Night & Day】Amazon Container Services【IVS CTO Night & Day】Amazon Container Services
【IVS CTO Night & Day】Amazon Container Services
 
Economies of Scaling Software
Economies of Scaling SoftwareEconomies of Scaling Software
Economies of Scaling Software
 
Taking your Dev Team on the UX Journey
Taking your Dev Team on the UX JourneyTaking your Dev Team on the UX Journey
Taking your Dev Team on the UX Journey
 
Buying Tips Soccer Shoes
Buying Tips Soccer ShoesBuying Tips Soccer Shoes
Buying Tips Soccer Shoes
 
Cheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m us
Cheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m usCheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m us
Cheaper k swiss womens blade max stable track shoe blackneon pinksilver8.5 m us
 
Sports technology
Sports technologySports technology
Sports technology
 
Big Data Expo 2015 - MapR Impacting Business As It Happens
Big Data Expo 2015 - MapR Impacting Business As It HappensBig Data Expo 2015 - MapR Impacting Business As It Happens
Big Data Expo 2015 - MapR Impacting Business As It Happens
 
Blackknight USA
Blackknight USABlackknight USA
Blackknight USA
 
Black Knight USA
Black Knight USABlack Knight USA
Black Knight USA
 
SCG Virtual Reality top news q1 2016
SCG  Virtual Reality top news q1 2016SCG  Virtual Reality top news q1 2016
SCG Virtual Reality top news q1 2016
 
Immersive Web
Immersive WebImmersive Web
Immersive Web
 
Using XML to Create Once - Distribute Everywhere
Using XML to Create Once - Distribute EverywhereUsing XML to Create Once - Distribute Everywhere
Using XML to Create Once - Distribute Everywhere
 
Azure for gaming
Azure for gamingAzure for gaming
Azure for gaming
 
Realtà aumentata ed Azure, un binomio imbattibile
Realtà aumentata ed Azure, un binomio imbattibileRealtà aumentata ed Azure, un binomio imbattibile
Realtà aumentata ed Azure, un binomio imbattibile
 
How Silicon Valley startups are approaching security differently
How Silicon Valley startups are approaching security differentlyHow Silicon Valley startups are approaching security differently
How Silicon Valley startups are approaching security differently
 
Storage Visibility for Operations - A Ceph Story
Storage Visibility for Operations - A Ceph StoryStorage Visibility for Operations - A Ceph Story
Storage Visibility for Operations - A Ceph Story
 

More from confluent

Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
confluent
 
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
confluent
 
Break data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud ConnectorsBreak data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud Connectors
confluent
 
Building API data products on top of your real-time data infrastructure
Building API data products on top of your real-time data infrastructureBuilding API data products on top of your real-time data infrastructure
Building API data products on top of your real-time data infrastructure
confluent
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
confluent
 
Evolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI EraEvolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI Era
confluent
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
confluent
 
Santander Stream Processing with Apache Flink
Santander Stream Processing with Apache FlinkSantander Stream Processing with Apache Flink
Santander Stream Processing with Apache Flink
confluent
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insights
confluent
 
Workshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con FlinkWorkshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con Flink
confluent
 
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
confluent
 
AWS Immersion Day Mapfre - Confluent
AWS Immersion Day Mapfre   -   ConfluentAWS Immersion Day Mapfre   -   Confluent
AWS Immersion Day Mapfre - Confluent
confluent
 
Eventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalkEventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalk
confluent
 
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent CloudQ&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
confluent
 
Citi TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
confluent
 
Build real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
confluent
 
Q&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
confluent
 
Citi Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
confluent
 
Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
confluent
 
Citi Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
confluent
 

More from confluent (20)

Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
 
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
 
Break data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud ConnectorsBreak data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud Connectors
 
Building API data products on top of your real-time data infrastructure
Building API data products on top of your real-time data infrastructureBuilding API data products on top of your real-time data infrastructure
Building API data products on top of your real-time data infrastructure
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
Evolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI EraEvolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI Era
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
 
Santander Stream Processing with Apache Flink
Santander Stream Processing with Apache FlinkSantander Stream Processing with Apache Flink
Santander Stream Processing with Apache Flink
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insights
 
Workshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con FlinkWorkshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con Flink
 
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
 
AWS Immersion Day Mapfre - Confluent
AWS Immersion Day Mapfre   -   ConfluentAWS Immersion Day Mapfre   -   Confluent
AWS Immersion Day Mapfre - Confluent
 
Eventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalkEventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalk
 
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent CloudQ&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
 
Citi TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
 
Build real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
 
Q&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
 
Citi Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
 
Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
 
Citi Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
 

Recently uploaded

Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
Kief Morris
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
digitalxplive
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
Adam Dunkels
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
Priyanka Aash
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
SubhamMandal40
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
Axel Rennoch
 
Pigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending PlantPigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending Plant
LINUS PROJECTS (INDIA)
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
moinahousna
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc
 
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyyActive Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
RaminGhanbari2
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
Recent Advancements in the NIST-JARVIS Infrastructure
Recent Advancements in the NIST-JARVIS InfrastructureRecent Advancements in the NIST-JARVIS Infrastructure
Recent Advancements in the NIST-JARVIS Infrastructure
KAMAL CHOUDHARY
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
HackersList
 

Recently uploaded (20)

Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
 
Pigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending PlantPigging Unit Lubricant Oil Blending Plant
Pigging Unit Lubricant Oil Blending Plant
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
 
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyyActive Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
Recent Advancements in the NIST-JARVIS Infrastructure
Recent Advancements in the NIST-JARVIS InfrastructureRecent Advancements in the NIST-JARVIS Infrastructure
Recent Advancements in the NIST-JARVIS Infrastructure
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
 

The Future of Application Development - API Days - Melbourne 2023

  • 1. The Future of Application Development API Days — Melbourne 2023 David Peterson Principal Solutions Engineer, APAC
  • 4. Grady Booch “No matter what future we may envision, it relies on software that has not yet been written”
  • 6. Big Data More == Better Data in Motion Timely == Better Value of Data -> Time -> Value of Data -> Time ->
  • 11. MongoDB Connect CDC Events Batch Txns Connect Schema Kappa Schema
  • 12. MongoDB Connect CDC Events Batch Txns Connect Kappa
  • 13. CDC Events Batch Txns Connect MongoDB VectorDB Postgres Connect Replay ^^ Kappa Source of Truth }
  • 15. Old world Event processor New world Stream processor
  • 24. unbounded, continuously updating data set, ordered,replayable, fault-tolerant
  • 28. Changelog stream – immutable events
  • 30. CDC Events Batch Txns Connect MongoDB VectorDB Postgres Connect Kappa Materialised Views {
  • 31. CDC Events Batch Txns MongoDB VectorDB Postgres Connect Connect Stream Processing Consumer and producer ^^ } Kappa — Stream Processing Processors • Kafka Streams • ksqlDB • Flink
  • 38. CREATE TABLE ORGANISATION_CUSTOMER_CUSTOMERS_360 WITH ( KAFKA_TOPIC = 'ORGANISATION_CUSTOMER_CUSTOMERS_360' ) AS SELECT CU.CUSTOMER_ID AS CUSTOMER_ID, FIRST_NAME AS FIRST_NAME, LAST_NAME AS LAST_NAME, STRUCT( STREET_ADDRESS := CU.ADDRESS->STREET_ADDRESS, SUBURB := CU.ADDRESS->SUBURB, POSTCODE := CU.ADDRESS->POSTCODE ) ADDRESS, STRUCT( VENDOR := L.VENDOR, POINTS := L.POINTS, TRANSACTION_ID := L.TRANSACTION_ID ) LATEST_LOYALTY_TRANSACTION, STRUCT( TOTAL_POINTS := L.TOTAL_POINTS, AVERAGE_POINTS := L.AVERAGE_POINTS ) LOYALTY_POINTS, STRUCT( SCORE := CR.CREDIT_SCORE ) CREDIT_RATING FROM ORGANISATION_CUSTOMER_CUSTOMERS_STATE CU LEFT OUTER JOIN ORGANISATION_CREDIT_RATING_STATE CR ON ((CU.CUSTOMER_ID = CR.CUSTOMER_ID)) LEFT OUTER JOIN ORGANISATION_LOYALTY_TRANSACTIONS_STATE L ON ((CU.CUSTOMER_ID = L.CUSTOMER_ID)) PARTITION BY CU.CUSTOMER_ID EMIT CHANGES;
  • 47. Kurzweil - 2014 “I am more optimistic and put the date at 2029…”
  • 48. Ilya Sutskever - 2022 “In the future, it will be obvious that the sole purpose of science was to build AGI”
  • 49. /imagine prompt:Street style photo of a woman shot on Kodak July 2023 July 2022
  • 50. LLM
  • 51. Ilya Sutskever “Shocking degree of understanding about the world”
  • 53. CDC Events Batch Txns MongoDB VectorDB Postgres Connect Connect Stream Processing Consumer and producer ^^ } Microservice Generate the prompt. Powered by C360 Topic
  • 55. • New / Private data • Context • Memory • Integrating into business Hard Problems with LLMs / Gen AI
  • 56. Hard Problem — Private data {"release_date": "2023-05-02", "shoe_name":"ActivExhale Ultra Light", "press_release": "We're proud to present a groundbreaking shoe crafted with world class support: the ActivExhale Ultra Light. The shoe's ultra flexible design means your feet are relieved of any stress and adapted for the greatest comfort. With reinforced soles, shock absorption technology, and its hi-tech mesh pattern, you can walk and move freely in this shoe, no matter the terrain!"} {"release_date":"2023-10-28", "shoe_name":"TailorSport Edge", "press_release": "Introducing TailorSport Edge, our newest range of shoes created for performance and agility. The special woven upper materials adapt to the shape and size of your foot, while the cushioning system provides outstanding comfort and responsiveness of your feet whilst running or exercising. Its cutting edge technology allows your foot to experience a new level of performance stylishly.”} {"release_date":"2023-03-29", "shoe_name":"EasyCloud OG", "press_release": "A fan favorite, the EasyCloud OG, is now available. The OG redefines the ultimate leisure-style walking shoe thanks to its synthetic cushioning and responsive surface. The unobtrusive materials make walking in this shoe fast, sleek, and comfortable. The sole hugs your foot perfectly, keeping each stride light yet sure. It's time for effortless style and energetic running."} {"release_date":"2023-06-11", "shoe_name":"Smart Grip Trailblazer","press_release": "Designed for trail hikes, the Smart Grip Trailblazer is here! This all-in-one shoe offers unbeatable traction on uneven surfaces, perfect padding, and fair water resistance - offering you the ideal blend of versatility and control. Step into a smarter hiking experience: Try the Trailblazer on all kinds of terrains and enjoy comfort from the first step!”} {"release_date": "2023-07-24", "shoe_name":"Prince Stables Flexy", "press_release": "Welcome the Prince Stables Flexy, designed with comfort, stability, and full range of motion in mind. Enjoy cornering soles which won’t let you down while relying on shock pads to provide keep your feet safe from exhaustion. Slip into convenience and agility - With heat forming technology, the practice makes perfection philosophy takes on a whole new meaning."} {"release_date":"2023-11-15", "shoe_name":"New Skymax runners", "press_release": "Experience extreme comfort with every step with the new Skymax runners. Specifically engineered with superior fit, this shoe has lightweight cushioning with ultra high energy return - keeps you powerfully pushing forward all day. Step up your run as you breeze through long-distances — with the smoothest cushion system and breathable material, get the best of the best this season and beyond!”} {"release_date":"2023-02-06", "shoe_name":"FuturiZen FreedomLite", "press_release": "Fusing clean aesthetic with personalized functionary technology, FuturiZen FreedomLite is a groundbreaking product in footwear with its contouring uppers, featherlight midsole, and laser molded outsole. Walk away in comfort and confidence - experience extreme lightweight connectivity wherever when you take on everyday life thanks to Versatile Cushion crossovers within the sole."} {"release_date":"2023-09-12", "shoe_name":"AiraSaint OnLiners", "press_release": "Beeline into AiraSaint OnLiners for lightweight, supportive running at all distances. Situated with signature mesh fabrics and forefoot structures, OnLiners is a hybrid style meet first choice that features extreme breathability but also encourages toe-splaying. Benefit from optimal fit all around your feet as you fasten laces - Infinity Comfort Seams direct your each movement towards your desired outcome with great bouncy response."}
  • 57. Hard Problem — Private data
  • 59. Hard Problem — Context
  • 61. Hard Problem — Long-term Memory
  • 63. Hard Problem — Integration
  • 67. Yuval Noah Harari — Sapiens “Consistency is the playground of dull minds”
  • 69. • API != REST, depends on use-case • Kappa > Kafka — simplify your architecture • Schema — data contracts • Source of Truth • Raw / Enriched data pattern • Stream Processing — smart pipes • Events tell a story