SlideShare a Scribd company logo
Building Linked Data
Platform with AWS...
...without actually building one
Eugene Morozov
Twitter: @eugenemorozov
Blog: http://emorozov.com
LinkedIn: https://www.linkedin.com/in/emorozov
If you really want to hear about it...
● Dev background
● Financial services
● Some background in building APIs - integration, reporting, etc.
Project experience
● Creating a reporting API that can be used by clients
● Getting away from the assumption that we must actually build something
● Solution to reuse AWS S3 with a thin veneer of custom authentication
Keeping things simple
● One doesn’t need complex infrastructure to host linked data
Exposing data sets with SemanticWeb standards
● Enter Linked Data Platform (https://www.w3.org/TR/ldp)
● Linked Data Platform (LDP) defines a set of rules for HTTP operations on
web resources, some based on RDF, to provide an architecture for read-write
Linked Data on the web.
LDP 101
● Resources
● Containers (Basic, Direct, Indirect)
● Content Types
● Headers
○ Mandates use of Link to specify
resources and containers
○ Use of Accept-Post, Accept-Patch
● REST verbs
○ All of the standard verbs, with
prescribed behaviour for containers
LDP 101
● Use headers to advertise what operations are available on the containers
○ Mandates use of Link to specify LDP resources and containers
○ Use of Accept-Post, Accept-Patch
● Suggests use of well-known terms DC, RDFS
Container Example
Do we really need to deploy stuff ourselves?
● Enter AWS (https://aws.amazon.com)
AWS 101
● Landscape
● https://aws.amazon.com/architecture/
● https://aws.amazon.com/products/
AWS 101 - S3
● Fairly rich API for storage via S3
● But that’s not quite enough to conform to LDP because of how headers are
treated
AWS 101 - API Gateway
● APIGateway is a way to define APIs in Swagger (with some custom
extensions) and deploy it
● Can be used to proxy other AWS services
AWS 101 - Route53
● DNS
AWS 101 - CloudFront CDN
● Use of CDN to proxy requests through to API -> S3
● More control over routing
● Caching
● Lower cost per API request if caching on top of the API
AWS 101 - Cognito
● Rich APIs for access control
● Don’t need to expose IAM roles to the outside world
AWS 101 - CloudFormation
● CloudFormation for provisioning
● Declarative deployment
● Can complete LDP-in-a-box
AWS 101 - LDP in a box
Demo
● Python, AWS
● Picking first two things off the list
○ Flatten content and push it into S3
○ Define subset of the DLP in Swagger
Future work
● Would be great to add Route53, Cognito and CDN
● Would be good to have fully conformant LDP definition in Swagger (doable)
● Perhaps going for a templated container in APIGateway, so we don’t need to
explicitly flatten the containers
● Add billing into LDP API, perhaps via S3 caller pays feature or log analysis
● In-a-box approach with CloudFormation
Some references and inspiration
● MarkLogic case study https://d0.awsstatic.com/whitepapers/marklogic-on-
aws.pdf
● RDF Data Management in Amazon Cloud
http://dl.acm.org/citation.cfm?id=2320790
● Automatic mapping of web APIs http://datalegend.net/assets/paper7.pdf
GitHub
● https://github.com/cadmiumkitty/aws-as-ldp

More Related Content

What's hot

From AWS to GCP, TABLEAPP Architecture Story
From AWS to GCP, TABLEAPP Architecture StoryFrom AWS to GCP, TABLEAPP Architecture Story
From AWS to GCP, TABLEAPP Architecture Story
Yen-Wen Chen
 
Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)
Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)
Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)
Tomasz Tarczyński
 
gRPC Overview
gRPC OverviewgRPC Overview
gRPC Overview
Varun Talwar
 
Introduction to DevOps and the Practical Use Cases at Credit OK
Introduction to DevOps and the Practical Use Cases at Credit OKIntroduction to DevOps and the Practical Use Cases at Credit OK
Introduction to DevOps and the Practical Use Cases at Credit OK
Kriangkrai Chaonithi
 
Kubeflow Pipelines (with Tekton)
Kubeflow Pipelines (with Tekton)Kubeflow Pipelines (with Tekton)
Kubeflow Pipelines (with Tekton)
Animesh Singh
 
CNCF opa
CNCF opaCNCF opa
CNCF opa
Juraj Hantak
 
KFServing and Feast
KFServing and FeastKFServing and Feast
KFServing and Feast
Animesh Singh
 
CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...
CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...
CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...
CodiLime
 
Docker for mac & local developer environment optimization
Docker for mac & local developer environment optimizationDocker for mac & local developer environment optimization
Docker for mac & local developer environment optimization
Radek Baczynski
 
Configuration Management for the Cloud Native world with GitOps and Helm - To...
Configuration Management for the Cloud Native world with GitOps and Helm - To...Configuration Management for the Cloud Native world with GitOps and Helm - To...
Configuration Management for the Cloud Native world with GitOps and Helm - To...
PROIDEA
 
[GID Live] Open-Source Cloud-Native Programming Language
[GID Live] Open-Source Cloud-Native Programming Language[GID Live] Open-Source Cloud-Native Programming Language
[GID Live] Open-Source Cloud-Native Programming Language
Ballerinalang
 
AWS Meetup Paris - Short URL project by Pernod Ricard
AWS Meetup Paris - Short URL project by Pernod RicardAWS Meetup Paris - Short URL project by Pernod Ricard
AWS Meetup Paris - Short URL project by Pernod Ricard
Charles Rapp
 
Logging in The World of DevOps
Logging in The World of DevOps Logging in The World of DevOps
Logging in The World of DevOps
DevOps Indonesia
 
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
WSO2
 
Journey to the cloud
Journey to the cloudJourney to the cloud
Journey to the cloud
Mikhail Advani
 
KFServing and Kubeflow Pipelines
KFServing and Kubeflow PipelinesKFServing and Kubeflow Pipelines
KFServing and Kubeflow Pipelines
Animesh Singh
 
Prometheus loves Grafana
Prometheus loves GrafanaPrometheus loves Grafana
Prometheus loves Grafana
Tobias Schmidt
 
gRPC
gRPCgRPC
Running Flink in Production: The good, The bad and The in Between - Lakshmi ...
Running Flink in Production:  The good, The bad and The in Between - Lakshmi ...Running Flink in Production:  The good, The bad and The in Between - Lakshmi ...
Running Flink in Production: The good, The bad and The in Between - Lakshmi ...
Flink Forward
 
The Next Generation Software Stack: Meteor
The Next Generation Software Stack: MeteorThe Next Generation Software Stack: Meteor
The Next Generation Software Stack: Meteor
MongoDB
 

What's hot (20)

From AWS to GCP, TABLEAPP Architecture Story
From AWS to GCP, TABLEAPP Architecture StoryFrom AWS to GCP, TABLEAPP Architecture Story
From AWS to GCP, TABLEAPP Architecture Story
 
Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)
Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)
Config management for_kubernetes: GitOps + Helm (CfgMgmtCamp 2020)
 
gRPC Overview
gRPC OverviewgRPC Overview
gRPC Overview
 
Introduction to DevOps and the Practical Use Cases at Credit OK
Introduction to DevOps and the Practical Use Cases at Credit OKIntroduction to DevOps and the Practical Use Cases at Credit OK
Introduction to DevOps and the Practical Use Cases at Credit OK
 
Kubeflow Pipelines (with Tekton)
Kubeflow Pipelines (with Tekton)Kubeflow Pipelines (with Tekton)
Kubeflow Pipelines (with Tekton)
 
CNCF opa
CNCF opaCNCF opa
CNCF opa
 
KFServing and Feast
KFServing and FeastKFServing and Feast
KFServing and Feast
 
CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...
CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...
CodiLime Tech Talk - Michał Pawluk: Our production deployment in AWS (HashiCo...
 
Docker for mac & local developer environment optimization
Docker for mac & local developer environment optimizationDocker for mac & local developer environment optimization
Docker for mac & local developer environment optimization
 
Configuration Management for the Cloud Native world with GitOps and Helm - To...
Configuration Management for the Cloud Native world with GitOps and Helm - To...Configuration Management for the Cloud Native world with GitOps and Helm - To...
Configuration Management for the Cloud Native world with GitOps and Helm - To...
 
[GID Live] Open-Source Cloud-Native Programming Language
[GID Live] Open-Source Cloud-Native Programming Language[GID Live] Open-Source Cloud-Native Programming Language
[GID Live] Open-Source Cloud-Native Programming Language
 
AWS Meetup Paris - Short URL project by Pernod Ricard
AWS Meetup Paris - Short URL project by Pernod RicardAWS Meetup Paris - Short URL project by Pernod Ricard
AWS Meetup Paris - Short URL project by Pernod Ricard
 
Logging in The World of DevOps
Logging in The World of DevOps Logging in The World of DevOps
Logging in The World of DevOps
 
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
 
Journey to the cloud
Journey to the cloudJourney to the cloud
Journey to the cloud
 
KFServing and Kubeflow Pipelines
KFServing and Kubeflow PipelinesKFServing and Kubeflow Pipelines
KFServing and Kubeflow Pipelines
 
Prometheus loves Grafana
Prometheus loves GrafanaPrometheus loves Grafana
Prometheus loves Grafana
 
gRPC
gRPCgRPC
gRPC
 
Running Flink in Production: The good, The bad and The in Between - Lakshmi ...
Running Flink in Production:  The good, The bad and The in Between - Lakshmi ...Running Flink in Production:  The good, The bad and The in Between - Lakshmi ...
Running Flink in Production: The good, The bad and The in Between - Lakshmi ...
 
The Next Generation Software Stack: Meteor
The Next Generation Software Stack: MeteorThe Next Generation Software Stack: Meteor
The Next Generation Software Stack: Meteor
 

Similar to Building Linked Data Platform with AWS

Running R on AWS Lambda by Ana-Maria Niculescu
Running R on AWS Lambda by Ana-Maria NiculescuRunning R on AWS Lambda by Ana-Maria Niculescu
Running R on AWS Lambda by Ana-Maria Niculescu
Paris Women in Machine Learning and Data Science
 
Building self service framework
Building self service frameworkBuilding self service framework
Building self service framework
Rovshan Musayev
 
Architecting applications on amazon web services with node.js
Architecting applications on amazon web services with node.jsArchitecting applications on amazon web services with node.js
Architecting applications on amazon web services with node.js
Henry Fougere
 
Back-end (Flask_AWS)
Back-end (Flask_AWS)Back-end (Flask_AWS)
Back-end (Flask_AWS)
GDSC UofT Mississauga
 
Write less (code) and build more with serverless
Write less (code) and build more with serverlessWrite less (code) and build more with serverless
Write less (code) and build more with serverless
Dhaval Nagar
 
Ghost Environment
Ghost EnvironmentGhost Environment
Ghost Environment
PratipD
 
Aws serverless architecture
Aws serverless architectureAws serverless architecture
Aws serverless architecture
genesesoftware
 
Deep dive into cloud security - Jaimin Gohel & Virendra Rathore
Deep dive into cloud security - Jaimin Gohel & Virendra RathoreDeep dive into cloud security - Jaimin Gohel & Virendra Rathore
Deep dive into cloud security - Jaimin Gohel & Virendra Rathore
NSConclave
 
PyConIT 2018 Writing and deploying serverless python applications
PyConIT 2018 Writing and deploying serverless python applicationsPyConIT 2018 Writing and deploying serverless python applications
PyConIT 2018 Writing and deploying serverless python applications
Cesar Cardenas Desales
 
introduction to micro services
introduction to micro servicesintroduction to micro services
introduction to micro services
Spyros Lambrinidis
 
Skillenza Build with Serverless Challenge - Advanced Serverless Concepts
Skillenza Build with Serverless Challenge -  Advanced Serverless ConceptsSkillenza Build with Serverless Challenge -  Advanced Serverless Concepts
Skillenza Build with Serverless Challenge - Advanced Serverless Concepts
Dhaval Nagar
 
Serverless Data Platform
Serverless Data PlatformServerless Data Platform
Serverless Data Platform
Shu-Jeng Hsieh
 
AWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - SlidesAWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - Slides
TobyWilman
 
Demistifying serverless on aws
Demistifying serverless on awsDemistifying serverless on aws
Demistifying serverless on aws
AWS Riyadh User Group
 
Low Cost AWS Services For Application Development in the Cloud
Low Cost AWS Services For Application Development in the CloudLow Cost AWS Services For Application Development in the Cloud
Low Cost AWS Services For Application Development in the Cloud
Dhaval Nagar
 
AWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless RecapAWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless Recap
Daniel Zivkovic
 
Amazon web services (aws) main developer services
Amazon web services (aws)   main developer servicesAmazon web services (aws)   main developer services
Amazon web services (aws) main developer services
Anderson Carvalho
 
PyConIE 2017 Writing and deploying serverless python applications
PyConIE 2017 Writing and deploying serverless python applicationsPyConIE 2017 Writing and deploying serverless python applications
PyConIE 2017 Writing and deploying serverless python applications
Cesar Cardenas Desales
 
Funky serverless features at aws
Funky serverless features at awsFunky serverless features at aws
Funky serverless features at aws
Doug Winter
 
An introduction to cloud systems architecture
An introduction to cloud systems architectureAn introduction to cloud systems architecture
An introduction to cloud systems architecture
Neela Muhil Vannan Mayavannan
 

Similar to Building Linked Data Platform with AWS (20)

Running R on AWS Lambda by Ana-Maria Niculescu
Running R on AWS Lambda by Ana-Maria NiculescuRunning R on AWS Lambda by Ana-Maria Niculescu
Running R on AWS Lambda by Ana-Maria Niculescu
 
Building self service framework
Building self service frameworkBuilding self service framework
Building self service framework
 
Architecting applications on amazon web services with node.js
Architecting applications on amazon web services with node.jsArchitecting applications on amazon web services with node.js
Architecting applications on amazon web services with node.js
 
Back-end (Flask_AWS)
Back-end (Flask_AWS)Back-end (Flask_AWS)
Back-end (Flask_AWS)
 
Write less (code) and build more with serverless
Write less (code) and build more with serverlessWrite less (code) and build more with serverless
Write less (code) and build more with serverless
 
Ghost Environment
Ghost EnvironmentGhost Environment
Ghost Environment
 
Aws serverless architecture
Aws serverless architectureAws serverless architecture
Aws serverless architecture
 
Deep dive into cloud security - Jaimin Gohel & Virendra Rathore
Deep dive into cloud security - Jaimin Gohel & Virendra RathoreDeep dive into cloud security - Jaimin Gohel & Virendra Rathore
Deep dive into cloud security - Jaimin Gohel & Virendra Rathore
 
PyConIT 2018 Writing and deploying serverless python applications
PyConIT 2018 Writing and deploying serverless python applicationsPyConIT 2018 Writing and deploying serverless python applications
PyConIT 2018 Writing and deploying serverless python applications
 
introduction to micro services
introduction to micro servicesintroduction to micro services
introduction to micro services
 
Skillenza Build with Serverless Challenge - Advanced Serverless Concepts
Skillenza Build with Serverless Challenge -  Advanced Serverless ConceptsSkillenza Build with Serverless Challenge -  Advanced Serverless Concepts
Skillenza Build with Serverless Challenge - Advanced Serverless Concepts
 
Serverless Data Platform
Serverless Data PlatformServerless Data Platform
Serverless Data Platform
 
AWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - SlidesAWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - Slides
 
Demistifying serverless on aws
Demistifying serverless on awsDemistifying serverless on aws
Demistifying serverless on aws
 
Low Cost AWS Services For Application Development in the Cloud
Low Cost AWS Services For Application Development in the CloudLow Cost AWS Services For Application Development in the Cloud
Low Cost AWS Services For Application Development in the Cloud
 
AWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless RecapAWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless Recap
 
Amazon web services (aws) main developer services
Amazon web services (aws)   main developer servicesAmazon web services (aws)   main developer services
Amazon web services (aws) main developer services
 
PyConIE 2017 Writing and deploying serverless python applications
PyConIE 2017 Writing and deploying serverless python applicationsPyConIE 2017 Writing and deploying serverless python applications
PyConIE 2017 Writing and deploying serverless python applications
 
Funky serverless features at aws
Funky serverless features at awsFunky serverless features at aws
Funky serverless features at aws
 
An introduction to cloud systems architecture
An introduction to cloud systems architectureAn introduction to cloud systems architecture
An introduction to cloud systems architecture
 

More from EugeneMorozov

Data Provenance and PROV Ontology
Data Provenance and PROV OntologyData Provenance and PROV Ontology
Data Provenance and PROV Ontology
EugeneMorozov
 
Discoverability of Regulatory Rulebooks
Discoverability of Regulatory RulebooksDiscoverability of Regulatory Rulebooks
Discoverability of Regulatory Rulebooks
EugeneMorozov
 
FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...
FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...
FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...
EugeneMorozov
 
Building Ontologies from Concept Maps
Building Ontologies from Concept MapsBuilding Ontologies from Concept Maps
Building Ontologies from Concept Maps
EugeneMorozov
 
Streaming Linked Data to Web UI
Streaming Linked Data to Web UIStreaming Linked Data to Web UI
Streaming Linked Data to Web UI
EugeneMorozov
 
Semantic Web and Micro Services
Semantic Web and Micro ServicesSemantic Web and Micro Services
Semantic Web and Micro Services
EugeneMorozov
 
Documenting Enterprise Architectures Using Ontologies
Documenting Enterprise Architectures Using OntologiesDocumenting Enterprise Architectures Using Ontologies
Documenting Enterprise Architectures Using Ontologies
EugeneMorozov
 
Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...
Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...
Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...
EugeneMorozov
 

More from EugeneMorozov (8)

Data Provenance and PROV Ontology
Data Provenance and PROV OntologyData Provenance and PROV Ontology
Data Provenance and PROV Ontology
 
Discoverability of Regulatory Rulebooks
Discoverability of Regulatory RulebooksDiscoverability of Regulatory Rulebooks
Discoverability of Regulatory Rulebooks
 
FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...
FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...
FIBO and SFTR - reflecting on FIBO workshop at DAS 2018 and applying FIBO to ...
 
Building Ontologies from Concept Maps
Building Ontologies from Concept MapsBuilding Ontologies from Concept Maps
Building Ontologies from Concept Maps
 
Streaming Linked Data to Web UI
Streaming Linked Data to Web UIStreaming Linked Data to Web UI
Streaming Linked Data to Web UI
 
Semantic Web and Micro Services
Semantic Web and Micro ServicesSemantic Web and Micro Services
Semantic Web and Micro Services
 
Documenting Enterprise Architectures Using Ontologies
Documenting Enterprise Architectures Using OntologiesDocumenting Enterprise Architectures Using Ontologies
Documenting Enterprise Architectures Using Ontologies
 
Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...
Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...
Working with RDF in Jupyter Notebooks: some lessons in getting rid of Excel f...
 

Recently uploaded

Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 

Recently uploaded (20)

Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 

Building Linked Data Platform with AWS

  • 1. Building Linked Data Platform with AWS... ...without actually building one Eugene Morozov Twitter: @eugenemorozov Blog: http://emorozov.com LinkedIn: https://www.linkedin.com/in/emorozov
  • 2. If you really want to hear about it... ● Dev background ● Financial services ● Some background in building APIs - integration, reporting, etc.
  • 3. Project experience ● Creating a reporting API that can be used by clients ● Getting away from the assumption that we must actually build something ● Solution to reuse AWS S3 with a thin veneer of custom authentication
  • 4. Keeping things simple ● One doesn’t need complex infrastructure to host linked data
  • 5. Exposing data sets with SemanticWeb standards ● Enter Linked Data Platform (https://www.w3.org/TR/ldp) ● Linked Data Platform (LDP) defines a set of rules for HTTP operations on web resources, some based on RDF, to provide an architecture for read-write Linked Data on the web.
  • 6. LDP 101 ● Resources ● Containers (Basic, Direct, Indirect) ● Content Types ● Headers ○ Mandates use of Link to specify resources and containers ○ Use of Accept-Post, Accept-Patch ● REST verbs ○ All of the standard verbs, with prescribed behaviour for containers
  • 7. LDP 101 ● Use headers to advertise what operations are available on the containers ○ Mandates use of Link to specify LDP resources and containers ○ Use of Accept-Post, Accept-Patch ● Suggests use of well-known terms DC, RDFS
  • 9. Do we really need to deploy stuff ourselves? ● Enter AWS (https://aws.amazon.com)
  • 10. AWS 101 ● Landscape ● https://aws.amazon.com/architecture/ ● https://aws.amazon.com/products/
  • 11. AWS 101 - S3 ● Fairly rich API for storage via S3 ● But that’s not quite enough to conform to LDP because of how headers are treated
  • 12. AWS 101 - API Gateway ● APIGateway is a way to define APIs in Swagger (with some custom extensions) and deploy it ● Can be used to proxy other AWS services
  • 13. AWS 101 - Route53 ● DNS
  • 14. AWS 101 - CloudFront CDN ● Use of CDN to proxy requests through to API -> S3 ● More control over routing ● Caching ● Lower cost per API request if caching on top of the API
  • 15. AWS 101 - Cognito ● Rich APIs for access control ● Don’t need to expose IAM roles to the outside world
  • 16. AWS 101 - CloudFormation ● CloudFormation for provisioning ● Declarative deployment ● Can complete LDP-in-a-box
  • 17. AWS 101 - LDP in a box
  • 18. Demo ● Python, AWS ● Picking first two things off the list ○ Flatten content and push it into S3 ○ Define subset of the DLP in Swagger
  • 19. Future work ● Would be great to add Route53, Cognito and CDN ● Would be good to have fully conformant LDP definition in Swagger (doable) ● Perhaps going for a templated container in APIGateway, so we don’t need to explicitly flatten the containers ● Add billing into LDP API, perhaps via S3 caller pays feature or log analysis ● In-a-box approach with CloudFormation
  • 20. Some references and inspiration ● MarkLogic case study https://d0.awsstatic.com/whitepapers/marklogic-on- aws.pdf ● RDF Data Management in Amazon Cloud http://dl.acm.org/citation.cfm?id=2320790 ● Automatic mapping of web APIs http://datalegend.net/assets/paper7.pdf

Editor's Notes

  1. Hello, welcome to the third Semantic Web London Meetup My name is Eugene Morozov, I co-manage engineering practice at Lab49 We are a strategy, design and technology consulting firm and we specialise in capital markets Fine print - whatever is on the screen is my own work, not a work done for our clients
  2. Other TODO: Look into https://github.com/awslabs/ecs-refarch-cloudformation Serverless architectures (AWS APIGateway + Lambdas) https://github.com/awslabs/aws-big-data-blog/tree/master/aws-blog-titan-graph-database