Be the first to like this
With distributed it-systems somehow and somewhere data shall be collected and made searchable. To be able to do this a company needs to provide and manage infrastructure, may it be on premise or in the cloud.
But there are also solutions to do the whole approach in a serverless way. In AWS there are services like the API-Gateway, Kinesis Firehose, AWS Lambda, S3 (Select), AWS Athena, Quicksight and Route 53 to create one of many of those solutions. Those are all managed services with which the users don’t have to manage any infrastructure on their own. By that the complete process of incoming data, backing them up and analysis of those stored data is meant.
You can decide when to proceed which kind of data. Whether you want to proceed them during transfer or afterwards. The analysis or transformation can be started event based (triggered through savings in S3), manually triggered when it is needed or in another way which fits best.
To store data using REST one can just use the API-Gateway with Kinesis Firehose. To make structured data searchable or achieve storing and transforming them in one step you can use API-Gateway with AWS Lambda in combination. After storing the data in S3 you can use standard SQL with Amazon Athena to analyze them and show the results in Amazon Quicksight.
In any case there are many way to store and process distributed data in a serverless way. We want to showcase a possible solution how to get started with AWS.
Speaker: André Reinecke, Cloud Consultant, tecRacer Consulting GmbH