Lessons Learned from Building Enterprise APIs

Lessons learned from
building enterprise APIs
Gustaf Nyman
Apitek Aktiebolag
gustaf.nyman@apitek.se
as presented at the Nordic APIs 2016 Platform Summit
About Enterprise APIs
Enterprises have:
• In-house developed systems
• Acquired system
• External SaaS
• Clients as mobiles, tablets etc.
Most systems communicate and expose APIs
Enterprises has many internal APIs
Enterprises has legacy* APIs using old technology
You need API management
Single point API gateway:
• Route message to backend service
• Sometimes convert or translate messages
• Necessary if using microservice architecture
Administration:
• Keep track of routing configuration and metadata
Monitoring:
• Does it work? If not why?
• Is it fast enough?
Miscellaneous:
• Authentication, throttling, signing, validation, archiving etc.
API gateway
Internal API client Internal API server
External API serverExternal API client
API gateway
Internet
Intranet
Cloud vnet
API gateway
Routing of messages to back-end services
Protocol translation - to support legacy APIs
Keep track of all routing info and metadata
• WSDL/XML Schema, Swagger/JSON Schema etc.
Logging, monitoring and analytics:
• Simplifies monitoring and debugging of connected systems
Performance and reliability
Admin tool
The natural choice for developers and others
Real world: Using legacy API
Task 2002: Access legacy API (monolith) using modern tech?
Legacy API:
• Queue based transport
• Text based message format (structure and value)
• 70 message types
Solution legacy API
Use XML, XML Schema, SOAP and WSDL.
Convert between XML and legacy format using XML Schema
Generate WSDL/XML Schema from repository
API client Legacy API server
XML Schema driven converter
XML
Schema
{getspeakerNordic APIsLesson learned}
<getspeaker>
<conference>Nordic APIs<conference>
<presentation>Lesson learned</presentation>
</getspeaker>
XML Schema
driven converter
Queue adapter
SOAP/HTTP Text/Queue
API gateway
Internal API client
Legacy API server
External API serverExternal API client
API Gateway
Internet
Intranet
Converter
Real World: Using legacy API
Task 2015: Access legacy API using modern tech?
Legacy API:
• SOAP/HTTP based transport
• XML message format described in WSDL/XML Schema
Move to modern technology, Microservices?
Solution: legacy API
Use JSON and Swagger/JSON Schema.
Convert between XML and JSON using existing XML Schema
Generate Swagger from WSDL using mapping rules
Lesson: Use metadata
Description of data, message exchange and contracts:
• WSDL/XML schema and Swagger/JSON schema
Use of metadata:
• Validate data and message exchange
• Generate stubs for client and server code
• Generate documentation, test data, user interfaces
• Data format conversion – with certain constraints
But remember:
• API/contract needs to be the first thing you design
• Think before generating from code or tools
• Make sure schemas are comprehensible!
Real World: B2B communication
Swedish insurance business 2002:
• Documents on paper communicated through surface mail.
Task: Define B2B communication guidelines, a ”standard”
Business requirements:
• Non repudiation
• Confidentiality
• Simple to implement on any platform/language
Solution: SOAP/WS-Security
Use standards, but define how and simplify:
• SOAP/HTTPS and WSDL/XML Schema
• X509 certificates and WS-Security for signing
• X509 client certificates
Add what’s missing:
• Organization identification header and correlation id
• A few standardized message types
10 years later still the natural B2B choice in the Swedish insurance
business.
Lesson: Simplify and select
If you want things to last..
• Do not use ”everything” technologies provide – select the good
stuff!
• Read the specifications – do not rely only on ready-made
toolkits or frameworks.
• Remember: Simple things are easier to migrate and adapt to
new technology
Problem: API failures
All APIs will eventually fail…
Ongoing task: improve analysis and production debugging support
Monitor and log
Log types:
• Processing logs – what happened?
• Data logs – what was the message content at specific point?
• Analytics logs – what is the response times and failure rate?
Logging subsystem:
• Should be always on
• Must be high performance and
• Must handle large amount of logs
• Must correlate and cluster related log data
What to log?
“Logga lagom”, i.e. log just enough information
• Too much and it becomes incomprehensible and unmanageable
• Too little and it is useless
Use experience and an evolutionary approach:
• You debug a difficult problem – improve log to enable easy
identification next time
• Always strive to enable less experienced personnel to effectively
debug using logs
Real world: Logging subsystem
Always on log system logs to disk – remove after 24 hours
On API failure related log files are stored in database
Web site to:
• Access and view error log data
• Automatic classification of errors using patterns
• Searching and reading logs on disk
• View other logs and system parameters
Take care of all your users
Your platform should be the natural choice for developers
• Cater to their needs.
More than developers:
• Many more are interested in what you are doing
• Many users are not computer savvy – they need simple tools
Typical non-technical users:
• People responsible for the business
• Testers with extensive business skills
• Customer service
Problem: Old technology
If we only redesign our system with…
• RPC
• CORBA/MTS
• SOAP/SOA
• REST/HTTP
• Event driven
• Microservices
• [your choice of the next big thing]
We will solve all problems and everything will be much better…
Everything will be legacy
…It is just a matter of time.
If redesign is not an option, put legacy APIs behind converters,
translators and adapters:
• Expose the modern API you prefer
Be prepared…
• Use standards – avoid vendor specific extensions
• Be selective – do not use everything
• Simplify
…mitigate the consequences of systems turning legacy
Summary
Use an API gateway
Use metadata
Simplify and select
Monitor and log
Prepare for systems turning legacy
gustaf.nyman@apitek.se
https://www.apitek.se/
Do not miss my talk tomorrow 14.15:
Build, Deploy and Test APIs and Microservices with Apitek Prisma
1 of 23

Recommended

Apitek Prisma by
Apitek PrismaApitek Prisma
Apitek PrismaGustaf Nyman
176 views13 slides
ChatGPT and the Future of Work - Clark Boyd by
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
28.7K views69 slides
Getting into the tech field. what next by
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
6.7K views22 slides
Google's Just Not That Into You: Understanding Core Updates & Search Intent by
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
7K views99 slides
How to have difficult conversations by
How to have difficult conversations How to have difficult conversations
How to have difficult conversations Rajiv Jayarajah, MAppComm, ACC
5.7K views19 slides
Introduction to Data Science by
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data ScienceChristy Abraham Joy
82.6K views51 slides

More Related Content

Recently uploaded

WITS Deck by
WITS DeckWITS Deck
WITS DeckW.I.T.S.
36 views22 slides
Penetration Testing for Cybersecurity Professionals by
Penetration Testing for Cybersecurity ProfessionalsPenetration Testing for Cybersecurity Professionals
Penetration Testing for Cybersecurity Professionals211 Check
49 views17 slides
ATPMOUSE_융합2조.pptx by
ATPMOUSE_융합2조.pptxATPMOUSE_융합2조.pptx
ATPMOUSE_융합2조.pptxkts120898
35 views70 slides
cis5-Project-11a-Harry Lai by
cis5-Project-11a-Harry Laicis5-Project-11a-Harry Lai
cis5-Project-11a-Harry Laiharrylai126
9 views11 slides
ARNAB12.pdf by
ARNAB12.pdfARNAB12.pdf
ARNAB12.pdfArnabChakraborty499766
5 views83 slides
40th TWNIC Open Policy Meeting: APNIC PDP update by
40th TWNIC Open Policy Meeting: APNIC PDP update40th TWNIC Open Policy Meeting: APNIC PDP update
40th TWNIC Open Policy Meeting: APNIC PDP updateAPNIC
106 views20 slides

Recently uploaded(13)

WITS Deck by W.I.T.S.
WITS DeckWITS Deck
WITS Deck
W.I.T.S.36 views
Penetration Testing for Cybersecurity Professionals by 211 Check
Penetration Testing for Cybersecurity ProfessionalsPenetration Testing for Cybersecurity Professionals
Penetration Testing for Cybersecurity Professionals
211 Check49 views
ATPMOUSE_융합2조.pptx by kts120898
ATPMOUSE_융합2조.pptxATPMOUSE_융합2조.pptx
ATPMOUSE_융합2조.pptx
kts12089835 views
cis5-Project-11a-Harry Lai by harrylai126
cis5-Project-11a-Harry Laicis5-Project-11a-Harry Lai
cis5-Project-11a-Harry Lai
harrylai1269 views
40th TWNIC Open Policy Meeting: APNIC PDP update by APNIC
40th TWNIC Open Policy Meeting: APNIC PDP update40th TWNIC Open Policy Meeting: APNIC PDP update
40th TWNIC Open Policy Meeting: APNIC PDP update
APNIC106 views
The Dark Web : Hidden Services by Anshu Singh
The Dark Web : Hidden ServicesThe Dark Web : Hidden Services
The Dark Web : Hidden Services
Anshu Singh22 views
40th TWNIC OPM: On LEOs (Low Earth Orbits) and Starlink Download by APNIC
40th TWNIC OPM: On LEOs (Low Earth Orbits) and Starlink Download40th TWNIC OPM: On LEOs (Low Earth Orbits) and Starlink Download
40th TWNIC OPM: On LEOs (Low Earth Orbits) and Starlink Download
APNIC112 views
40th TWNIC Open Policy Meeting: A quick look at QUIC by APNIC
40th TWNIC Open Policy Meeting: A quick look at QUIC40th TWNIC Open Policy Meeting: A quick look at QUIC
40th TWNIC Open Policy Meeting: A quick look at QUIC
APNIC109 views
Cracking the Code Decoding Leased Line Quotes for Connectivity Excellence.pptx by LeasedLinesQuote
Cracking the Code Decoding Leased Line Quotes for Connectivity Excellence.pptxCracking the Code Decoding Leased Line Quotes for Connectivity Excellence.pptx
Cracking the Code Decoding Leased Line Quotes for Connectivity Excellence.pptx

Featured

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright... by
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
12.8K views21 slides
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present... by
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Applitools
55.5K views138 slides
12 Ways to Increase Your Influence at Work by
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at WorkGetSmarter
401.7K views64 slides
ChatGPT webinar slides by
ChatGPT webinar slidesChatGPT webinar slides
ChatGPT webinar slidesAlireza Esmikhani
30.5K views36 slides
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G... by
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...DevGAMM Conference
3.6K views12 slides

Featured(20)

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright... by RachelPearson36
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson3612.8K views
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present... by Applitools
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Applitools55.5K views
12 Ways to Increase Your Influence at Work by GetSmarter
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
GetSmarter401.7K views
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G... by DevGAMM Conference
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
DevGAMM Conference3.6K views
Barbie - Brand Strategy Presentation by Erica Santiago
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy Presentation
Erica Santiago25.1K views
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well by Saba Software
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Saba Software25.3K views
Introduction to C Programming Language by Simplilearn
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming Language
Simplilearn8.5K views
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr... by Palo Alto Software
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
Palo Alto Software88.4K views
9 Tips for a Work-free Vacation by Weekdone.com
9 Tips for a Work-free Vacation9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation
Weekdone.com7.2K views
How to Map Your Future by SlideShop.com
How to Map Your FutureHow to Map Your Future
How to Map Your Future
SlideShop.com275.1K views
Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -... by AccuraCast
Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -...Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -...
Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -...
AccuraCast3.4K views
Exploring ChatGPT for Effective Teaching and Learning.pptx by Stan Skrabut, Ed.D.
Exploring ChatGPT for Effective Teaching and Learning.pptxExploring ChatGPT for Effective Teaching and Learning.pptx
Exploring ChatGPT for Effective Teaching and Learning.pptx
Stan Skrabut, Ed.D.57.7K views
How to train your robot (with Deep Reinforcement Learning) by Lucas García, PhD
How to train your robot (with Deep Reinforcement Learning)How to train your robot (with Deep Reinforcement Learning)
How to train your robot (with Deep Reinforcement Learning)
Lucas García, PhD42.5K views
4 Strategies to Renew Your Career Passion by Daniel Goleman
4 Strategies to Renew Your Career Passion4 Strategies to Renew Your Career Passion
4 Strategies to Renew Your Career Passion
Daniel Goleman122K views
The Student's Guide to LinkedIn by LinkedIn
The Student's Guide to LinkedInThe Student's Guide to LinkedIn
The Student's Guide to LinkedIn
LinkedIn88.1K views
Different Roles in Machine Learning Career by Intellipaat
Different Roles in Machine Learning CareerDifferent Roles in Machine Learning Career
Different Roles in Machine Learning Career
Intellipaat12.4K views

Lessons Learned from Building Enterprise APIs

  • 1. Lessons learned from building enterprise APIs Gustaf Nyman Apitek Aktiebolag gustaf.nyman@apitek.se as presented at the Nordic APIs 2016 Platform Summit
  • 2. About Enterprise APIs Enterprises have: • In-house developed systems • Acquired system • External SaaS • Clients as mobiles, tablets etc. Most systems communicate and expose APIs Enterprises has many internal APIs Enterprises has legacy* APIs using old technology
  • 3. You need API management Single point API gateway: • Route message to backend service • Sometimes convert or translate messages • Necessary if using microservice architecture Administration: • Keep track of routing configuration and metadata Monitoring: • Does it work? If not why? • Is it fast enough? Miscellaneous: • Authentication, throttling, signing, validation, archiving etc.
  • 4. API gateway Internal API client Internal API server External API serverExternal API client API gateway Internet Intranet Cloud vnet
  • 5. API gateway Routing of messages to back-end services Protocol translation - to support legacy APIs Keep track of all routing info and metadata • WSDL/XML Schema, Swagger/JSON Schema etc. Logging, monitoring and analytics: • Simplifies monitoring and debugging of connected systems Performance and reliability Admin tool The natural choice for developers and others
  • 6. Real world: Using legacy API Task 2002: Access legacy API (monolith) using modern tech? Legacy API: • Queue based transport • Text based message format (structure and value) • 70 message types
  • 7. Solution legacy API Use XML, XML Schema, SOAP and WSDL. Convert between XML and legacy format using XML Schema Generate WSDL/XML Schema from repository
  • 8. API client Legacy API server XML Schema driven converter XML Schema {getspeakerNordic APIsLesson learned} <getspeaker> <conference>Nordic APIs<conference> <presentation>Lesson learned</presentation> </getspeaker> XML Schema driven converter Queue adapter SOAP/HTTP Text/Queue
  • 9. API gateway Internal API client Legacy API server External API serverExternal API client API Gateway Internet Intranet Converter
  • 10. Real World: Using legacy API Task 2015: Access legacy API using modern tech? Legacy API: • SOAP/HTTP based transport • XML message format described in WSDL/XML Schema Move to modern technology, Microservices?
  • 11. Solution: legacy API Use JSON and Swagger/JSON Schema. Convert between XML and JSON using existing XML Schema Generate Swagger from WSDL using mapping rules
  • 12. Lesson: Use metadata Description of data, message exchange and contracts: • WSDL/XML schema and Swagger/JSON schema Use of metadata: • Validate data and message exchange • Generate stubs for client and server code • Generate documentation, test data, user interfaces • Data format conversion – with certain constraints But remember: • API/contract needs to be the first thing you design • Think before generating from code or tools • Make sure schemas are comprehensible!
  • 13. Real World: B2B communication Swedish insurance business 2002: • Documents on paper communicated through surface mail. Task: Define B2B communication guidelines, a ”standard” Business requirements: • Non repudiation • Confidentiality • Simple to implement on any platform/language
  • 14. Solution: SOAP/WS-Security Use standards, but define how and simplify: • SOAP/HTTPS and WSDL/XML Schema • X509 certificates and WS-Security for signing • X509 client certificates Add what’s missing: • Organization identification header and correlation id • A few standardized message types 10 years later still the natural B2B choice in the Swedish insurance business.
  • 15. Lesson: Simplify and select If you want things to last.. • Do not use ”everything” technologies provide – select the good stuff! • Read the specifications – do not rely only on ready-made toolkits or frameworks. • Remember: Simple things are easier to migrate and adapt to new technology
  • 16. Problem: API failures All APIs will eventually fail… Ongoing task: improve analysis and production debugging support
  • 17. Monitor and log Log types: • Processing logs – what happened? • Data logs – what was the message content at specific point? • Analytics logs – what is the response times and failure rate? Logging subsystem: • Should be always on • Must be high performance and • Must handle large amount of logs • Must correlate and cluster related log data
  • 18. What to log? “Logga lagom”, i.e. log just enough information • Too much and it becomes incomprehensible and unmanageable • Too little and it is useless Use experience and an evolutionary approach: • You debug a difficult problem – improve log to enable easy identification next time • Always strive to enable less experienced personnel to effectively debug using logs
  • 19. Real world: Logging subsystem Always on log system logs to disk – remove after 24 hours On API failure related log files are stored in database Web site to: • Access and view error log data • Automatic classification of errors using patterns • Searching and reading logs on disk • View other logs and system parameters
  • 20. Take care of all your users Your platform should be the natural choice for developers • Cater to their needs. More than developers: • Many more are interested in what you are doing • Many users are not computer savvy – they need simple tools Typical non-technical users: • People responsible for the business • Testers with extensive business skills • Customer service
  • 21. Problem: Old technology If we only redesign our system with… • RPC • CORBA/MTS • SOAP/SOA • REST/HTTP • Event driven • Microservices • [your choice of the next big thing] We will solve all problems and everything will be much better…
  • 22. Everything will be legacy …It is just a matter of time. If redesign is not an option, put legacy APIs behind converters, translators and adapters: • Expose the modern API you prefer Be prepared… • Use standards – avoid vendor specific extensions • Be selective – do not use everything • Simplify …mitigate the consequences of systems turning legacy
  • 23. Summary Use an API gateway Use metadata Simplify and select Monitor and log Prepare for systems turning legacy gustaf.nyman@apitek.se https://www.apitek.se/ Do not miss my talk tomorrow 14.15: Build, Deploy and Test APIs and Microservices with Apitek Prisma