Using Web Services for Third
party integrations to scale
(mandatory cliché business clipart)
Paul Shriner
Twitter: @shrinerp
Phone: 360-643-1403
Email:
paul@audiencepoint.com
Building Rapport with Audience
• 5 KIDS!
• Moved from Small
Washington town, 4
months ago
• Many Web service
integrations
Mandatory Animated GIF
What is a web service
• Pipeline for communication
• A means to integrate to servers
• Often synonymous with an API
• Gateway to Saas
Where can I use it
• Maps
• Social
• Mobile
• User Tracking
• Data exchange
• Automation/Workflow
How do I get started
Google “<BRAND> API Getting Started”
Get all available documentation
• Be greedy about documentation
– Get Everything (Yahoo! Maps)
– Read it
– Learn it
• Identify available metaphors/verbs
REST vs SOAP
REST
• Follows philosophy of
open web
• Easy to test
• Newer and standard
• Multiple return formats
– JSON
– CSV
– XML
• Uses port 80 (Standard)
• Easy to Debug
SOAP
• XML based
• Requires SOAP scaffolding
• XML return format
• Older standard
• Debugging is a harder
• Can handle additional
security/ports
Web Services 101
Favor REST
REST is an acronym for the Representational
State Transfer architectural style for distributed
hypermedia systems.
You will get started faster
Don’t trust the API
• Verify method names
– Input parameters
– Return Values
• Identify all error codes ahead
of time
– Confirm return structure
Handle all error codes
• Many error codes aren’t documented
• Error codes are not priority of API provider
• Missing error codes can break your integration
Write POC application
• Prove the API will work
– Assume that it will not work
– Better to fail implementation early
• POC App should be an inch wide and a mile
deep
Identify Technical Resource
• The Webservice will have secret handshakes,
Assume that everything is not documented.
• WebServices are often the last piece of the
application that is implemented, they are an
after thought.
– Even when part of architecture
FIND A GUIDE
Multi-threaded parallelization
• Web Services are a single call, even batched, it
is call/response.
– GET User
– GET ListOfUsers
• Single call is fast
– Many “single calls” S-L-O-W
• Paging ( still single threaded )
• Get all IDs -> Threaded Gets
Use Batch operations
• Time determines winner
– Multithreaded API calls ( 22 hours )
– Batch operations (20 minutes )
• Flat file
– CSV/TSV
Implement retry logic
• Assume API calls will fail
• Should have retry
threshold
– retry max ‘n’ times
• Retries should have
exponential backoff
• Fail BIG
Create strong feedback loop
• How does implementation break
• Who hears about it and how?
– Email
– GitHub
– User feedback
– SMS
TDD – Testing Driven Development
• APIs tend to change silently
• Write unit tests
• Run them often
Use technical resource
• Suck up to these people
• Get questions answered, don’t feel bad
– It is often their job to evangelize
• They are your gateway to the real help
– Product managers
– Platform programmers
– $$$
• Get code review from technical resources
Tips for Success
• Create mutual value for third party and self
• Regular and ongoing communication
• Send thank you notes/gifts
• Use back channel communication
• Push, but not too hard
Paul Shriner
Twitter: @shrinerp
Phone: 360-643-1403
Email:
paul@audiencepoint.com

Web Planning an Effective Integration with a 3rd party API to Scale

  • 1.
    Using Web Servicesfor Third party integrations to scale (mandatory cliché business clipart)
  • 2.
    Paul Shriner Twitter: @shrinerp Phone:360-643-1403 Email: paul@audiencepoint.com
  • 3.
    Building Rapport withAudience • 5 KIDS! • Moved from Small Washington town, 4 months ago • Many Web service integrations
  • 4.
  • 5.
    What is aweb service • Pipeline for communication • A means to integrate to servers • Often synonymous with an API • Gateway to Saas
  • 6.
    Where can Iuse it • Maps • Social • Mobile • User Tracking • Data exchange • Automation/Workflow
  • 7.
    How do Iget started Google “<BRAND> API Getting Started”
  • 8.
    Get all availabledocumentation • Be greedy about documentation – Get Everything (Yahoo! Maps) – Read it – Learn it • Identify available metaphors/verbs
  • 9.
    REST vs SOAP REST •Follows philosophy of open web • Easy to test • Newer and standard • Multiple return formats – JSON – CSV – XML • Uses port 80 (Standard) • Easy to Debug SOAP • XML based • Requires SOAP scaffolding • XML return format • Older standard • Debugging is a harder • Can handle additional security/ports
  • 10.
  • 11.
    Favor REST REST isan acronym for the Representational State Transfer architectural style for distributed hypermedia systems. You will get started faster
  • 12.
    Don’t trust theAPI • Verify method names – Input parameters – Return Values • Identify all error codes ahead of time – Confirm return structure
  • 13.
    Handle all errorcodes • Many error codes aren’t documented • Error codes are not priority of API provider • Missing error codes can break your integration
  • 14.
    Write POC application •Prove the API will work – Assume that it will not work – Better to fail implementation early • POC App should be an inch wide and a mile deep
  • 15.
    Identify Technical Resource •The Webservice will have secret handshakes, Assume that everything is not documented. • WebServices are often the last piece of the application that is implemented, they are an after thought. – Even when part of architecture FIND A GUIDE
  • 16.
    Multi-threaded parallelization • WebServices are a single call, even batched, it is call/response. – GET User – GET ListOfUsers • Single call is fast – Many “single calls” S-L-O-W • Paging ( still single threaded ) • Get all IDs -> Threaded Gets
  • 17.
    Use Batch operations •Time determines winner – Multithreaded API calls ( 22 hours ) – Batch operations (20 minutes ) • Flat file – CSV/TSV
  • 18.
    Implement retry logic •Assume API calls will fail • Should have retry threshold – retry max ‘n’ times • Retries should have exponential backoff • Fail BIG
  • 19.
    Create strong feedbackloop • How does implementation break • Who hears about it and how? – Email – GitHub – User feedback – SMS
  • 20.
    TDD – TestingDriven Development • APIs tend to change silently • Write unit tests • Run them often
  • 21.
    Use technical resource •Suck up to these people • Get questions answered, don’t feel bad – It is often their job to evangelize • They are your gateway to the real help – Product managers – Platform programmers – $$$ • Get code review from technical resources
  • 22.
    Tips for Success •Create mutual value for third party and self • Regular and ongoing communication • Send thank you notes/gifts • Use back channel communication • Push, but not too hard
  • 23.
    Paul Shriner Twitter: @shrinerp Phone:360-643-1403 Email: paul@audiencepoint.com