This document discusses approaches for handling long-running REST API operations that generate large amounts of data. It describes long-running operations with polling, where the client periodically checks on the status of the operation. It also covers operations with callbacks, where the server calls back the client once complete. For large result sets, it recommends writing the data to a JSON file in an S3 bucket and returning a temporary link to the file.

1. Long-running REST API calls
How to deal with long-running operations with a large
amount data
Tomas Dermisek / NC Group

2. Intro
REST API call = performance, scalability, simplicity

3. Address Lookup & Store Search

4. But how should we
deal with long-running
operations that
generate large
amount data?

5. Placing an Order

6. http://restalk-patterns.org/long-running-operation-polling.html
Long Running Operation with Polling

7. Long Running Operation with Polling
Benefits:
Scalability: The client does not need to keep the
connection open with the server for the entire
duration of the request.
Shared results: The link to the result can be
shared among multiple clients that can retrieve
it without needing the server to recompute it
again for each client.
Request cancellation: An explicit mechanism
consistent with the REST uniform interface is
provided for cancelling requests and thus
avoiding to waste server resources to perform
computations whose results the client is no
longer interested in.

8. Long Running Operation with Polling
Liabilities:
Polling: The client needs to implement polling,
which if done too frequently, may put an
additional burden on the server
Server storage consumption

10. Long Running Operation with Callback
The server calls back the client once the
operation is completed with a link to the final
resource.

11. Long Running Operation with Callback

12. Placing an Order

13. How about the large
amount of data?

14. Large amount of data
Write them into JSON file in S3 bucket
Return link to the file valid only for limited period
of time

15. Sample link
https://menu-api-output-s3.s3.ap-southeast-2.amazonaws.com/a7f46f43-2cfe-4326-9bfc-c3716abdb
75c.json?AWSAccessKeyId=ASIAIEALES2LJVDEJTCA&Expires=1527150464&Signature=B3gNDp%2
B7OwiNLy9y8wCykp9DtO0%3D&x-amz-security-token=FQoDYXdzEFkaDL6i4btP5y1rpDiv2SLpASqhxV
NadBkF0tU7cFOTUS5qz23I3Di8ZXF%2FB8T0gHAIHFPpHYxbCTBNmAkRbeV3dKQQ6YEUib1zDOAKL
eagVx%2BdTEMIVXJ2rw6RI2VmzU7x9E7a%2BGVazQwyO4DEe50JcUe8O4vGJbsexI0MRq2JiYGV0P
KbV32KpVX0eTY8FVLJMipeJqevPyE8IknkduxatB1oJiy4AJOv2EaryVKBZUETW5V9xHtPKPPTk7f%2F
R%2BDiLu%2B2bN9Sb8IH9B98k2Jiabd%2FtcIcnz%2BrHJg2uzBYGgr45gLDPcCx1dDRejBdSfymLUdjn
0L2avpnKIfemdgF

16. Questions!