In this hands on session will make use of the scenario demonstrated previously to show how the same results can be achieved programmatically. The ability to use the services via the API are essential to automate the data management process when dealing with large volumes of data, potentially from many different sources. This will require hands on coding (the demonstrations will be given using python but if users are confident they may choose their own language). By the end of this sessions, attendees should be able to understand how to use the B2 services within their own scientific workflows to allow automated data management.
B2SHARE REST API Hands-on - EUDAT Summer School (Hans van Piggelen, SURFsara)EUDAT
In this hands on session will make use of the scenario demonstrated previously to show how the same results can be achieved programmatically. The ability to use the services via the API are essential to automate the data management process when dealing with large volumes of data, potentially from many different sources. This will require hands on coding (the demonstrations will be given using python but if users are confident they may choose their own language). By the end of this sessions, attendees should be able to understand how to use the B2 services within their own scientific workflows to allow automated data management.
Visit: https://www.eudat.eu/eudat-summer-school
This tutorial introduces Fedora, an open-source content management system that allows users to create digital objects that aggregate content from multiple sources and associate web services to dynamically generate content from those objects. It provides examples of creating digital objects that contain multiple file formats and link to external content. The document is intended for system administrators and developers to learn hands-on how to construct such objects in a Fedora repository.
A Standalone Content Sharing Application for Spontaneous Communities of Mobil...Amir Krifa
This document describes BitHoc, a content sharing application adapted from BitTorrent for use in wireless ad hoc networks. BitHoc includes a peer tracker, BitHoc Tracker, that maintains membership of distributed torrent overlays. It also includes a BitHoc Client that allows users to seed and leech torrents within the mobile ad hoc network. A BitHoc Search Engine allows users to publish, search for, and download torrent files. The architecture uses an OLSR routing daemon and distributed DHT for peer and content discovery within the spontaneous mobile network.
The document provides an overview of web technologies and servlet development environment setup. It discusses HTTP and the request-response model. It also covers servlet lifecycle including initialization, request handling, and destruction. The document explains setting up Java, Tomcat, and configuring the CLASSPATH for servlet development. It outlines the servlet API classes and interfaces like Servlet, ServletRequest, and ServletResponse.
This document provides an overview of the application layer in computer networking. It discusses key concepts like the client-server and peer-to-peer paradigms for application communication. Popular application layer protocols are examined, including HTTP, FTP, SMTP, POP3, IMAP, and DNS. The document also covers programming network applications using sockets and building a web server.
Web services allow programs to call methods on other computers over a network. They are frequently web APIs that can be accessed remotely and executed on another system. Web services consist of method information describing the method being called and scoping information describing required arguments. This information is packaged and sent across the network using various protocols like HTTP, SOAP, and XML-RPC. The internet protocol stack, consisting of layers like application, transport, network and link, is used to break information into packets that can travel over the network to their destination and be reassembled.
The document summarizes a seminar presentation on the World Wide Web (WWW). It discusses the basic client-server architecture of the WWW, with servers hosting documents and clients providing interfaces for users. It also covers the evolution of the WWW to include distributed services beyond just documents. Traditional web systems are described as using simple client-server models with URLs to locate documents on servers. Key aspects like HTTP, document models, and scripting technologies are summarized. Security measures for web transactions like TLS and aspects of caching, replication, and content delivery are also outlined.
This document defines common Internet and web terminology including animated GIFs, anonymous FTP sites, bandwidth, bits, bookmarks, browsers, bytes, cyberspace, domains, firewalls, FTP, GIFs, gigabytes, home pages, hosts, HTML, hyperlinks, IP addresses, JavaScript, JPEG, kilobytes, links, menus, metadata, protocols, search engines, servers, thumbnails, URLs, web pages, and the World Wide Web.
B2SHARE REST API Hands-on - EUDAT Summer School (Hans van Piggelen, SURFsara)EUDAT
In this hands on session will make use of the scenario demonstrated previously to show how the same results can be achieved programmatically. The ability to use the services via the API are essential to automate the data management process when dealing with large volumes of data, potentially from many different sources. This will require hands on coding (the demonstrations will be given using python but if users are confident they may choose their own language). By the end of this sessions, attendees should be able to understand how to use the B2 services within their own scientific workflows to allow automated data management.
Visit: https://www.eudat.eu/eudat-summer-school
This tutorial introduces Fedora, an open-source content management system that allows users to create digital objects that aggregate content from multiple sources and associate web services to dynamically generate content from those objects. It provides examples of creating digital objects that contain multiple file formats and link to external content. The document is intended for system administrators and developers to learn hands-on how to construct such objects in a Fedora repository.
A Standalone Content Sharing Application for Spontaneous Communities of Mobil...Amir Krifa
This document describes BitHoc, a content sharing application adapted from BitTorrent for use in wireless ad hoc networks. BitHoc includes a peer tracker, BitHoc Tracker, that maintains membership of distributed torrent overlays. It also includes a BitHoc Client that allows users to seed and leech torrents within the mobile ad hoc network. A BitHoc Search Engine allows users to publish, search for, and download torrent files. The architecture uses an OLSR routing daemon and distributed DHT for peer and content discovery within the spontaneous mobile network.
The document provides an overview of web technologies and servlet development environment setup. It discusses HTTP and the request-response model. It also covers servlet lifecycle including initialization, request handling, and destruction. The document explains setting up Java, Tomcat, and configuring the CLASSPATH for servlet development. It outlines the servlet API classes and interfaces like Servlet, ServletRequest, and ServletResponse.
This document provides an overview of the application layer in computer networking. It discusses key concepts like the client-server and peer-to-peer paradigms for application communication. Popular application layer protocols are examined, including HTTP, FTP, SMTP, POP3, IMAP, and DNS. The document also covers programming network applications using sockets and building a web server.
Web services allow programs to call methods on other computers over a network. They are frequently web APIs that can be accessed remotely and executed on another system. Web services consist of method information describing the method being called and scoping information describing required arguments. This information is packaged and sent across the network using various protocols like HTTP, SOAP, and XML-RPC. The internet protocol stack, consisting of layers like application, transport, network and link, is used to break information into packets that can travel over the network to their destination and be reassembled.
The document summarizes a seminar presentation on the World Wide Web (WWW). It discusses the basic client-server architecture of the WWW, with servers hosting documents and clients providing interfaces for users. It also covers the evolution of the WWW to include distributed services beyond just documents. Traditional web systems are described as using simple client-server models with URLs to locate documents on servers. Key aspects like HTTP, document models, and scripting technologies are summarized. Security measures for web transactions like TLS and aspects of caching, replication, and content delivery are also outlined.
This document defines common Internet and web terminology including animated GIFs, anonymous FTP sites, bandwidth, bits, bookmarks, browsers, bytes, cyberspace, domains, firewalls, FTP, GIFs, gigabytes, home pages, hosts, HTML, hyperlinks, IP addresses, JavaScript, JPEG, kilobytes, links, menus, metadata, protocols, search engines, servers, thumbnails, URLs, web pages, and the World Wide Web.
This document defines common Internet and web terminology including animated GIFs, anonymous FTP sites, bandwidth, bits, bookmarks, browsers, bytes, cyberspace, domains, firewalls, FTP, GIFs, gigabytes, home pages, hosts, HTML, hyperlinks, Internet service providers, IP addresses, JavaScript, JPEG, kilobytes, links, menus, metadata, ports, protocols, search engines, servers, thumbnails, URLs, uploads, web pages, and the World Wide Web.
https://www.youtube.com/watch?v=lKrbeJ7-J98
HTTP messages are how data is exchanged between a server and a client. There are two types of messages: requests sent by the client to trigger an action on the server, and responses, the answer from the server.
CGI allows a web server to interface with external applications to dynamically generate web pages. It uses environment variables and form data passed via GET or POST requests to run scripts or programs that process input and return output to the server. While CGI provides interactivity, it can be slow and has security issues. Server-side includes and client-side scripting offer alternatives for dynamic content generation.
The document provides an overview of the technical capabilities of the InfoRouter content management system. It describes how InfoRouter is standards-based, scalable, secure, entirely web-based, browser independent, supports multiple document formats and languages, integrates with LDAP directories, and has a distributed architecture. It also discusses features like metadata services, web portals, live updating, and integration with web services APIs.
Overview of HTTP, HTML, WWW and web technologies.
The combo HTTP and HTML is the foundation of the World Wide Web (WWW).
HTML (HyperText Markup Language) defines a text-based format for describing the contents of a web page. HTML is based on tags similar to XML (eXtensible Markup Language), but its definition is less strict.
HTML pages are transported with the HTTP protocol (HyperText Transmission Protocol) over TCP/IP based networks.
The power of the WWW comes with the links based on URLs (Uniform Resource Locators) that connect pages to form a web of content.
Browsers display links as clickable items that, when clicked, trigger the browser to load the web page pointed to by the link.
This statelessness contributed a lot to the stability and scalability of the world wide web where web servers are only tasked with the delivery of web pages while the browser is responsible for the rendering of web pages.
The static nature of the early World Wide Web was soon augmented with the dynamic creation of web pages by web servers or by enriching static web pages with dynamic content.
Technologies like CGI (Common Gateway Interface), JSP (Java Server Pages) or ASP (Active Server Pages) were developed to provide the infrastructure to build dynamic web applications.
These server-side technologies were complemented with client-side technologies like Javascript and AJAX (Asynchronous Javascript And XML).
Web page caching is an important mechanism to reduce latency in loading web pages and reducing network traffic.
HTTP defines different caching control mechanisms. Simpler caching methods are based on web page expiry dates while more complex mechanisms use web page validation.
The document describes the implementation of a peer-to-peer server that allows peers to register, deregister, search for, and download content. The server uses TCP sockets and threads to handle multiple client connections simultaneously. Issues encountered included buffers not clearing properly and thread arrays causing segmentation faults. These were resolved by adding buffer clearing logic and allocating memory for thread indices. The implemented code now meets specifications by allowing peers to share a centralized content registry and download files from each other.
This lecture covered CIDR (Classless Inter-Domain Routing) and subnetting IP addresses. It discussed client-server and peer-to-peer networking models. Common network applications like email, web, and file sharing were mentioned. TCP and UDP transport protocols were summarized. HTTP protocol for web was explained in detail including persistent and non-persistent connections.
The document discusses the application layer and various network applications. It begins by covering principles of network applications like client-server and peer-to-peer architectures. It then covers specific applications including HTTP and the web, email using SMTP, POP3 and IMAP, and DNS. It also discusses socket programming, addressing processes, and factors applications consider when choosing transport protocols. The document closes by focusing on HTTP and the web in more detail, covering HTTP requests and responses, persistent connections, and the use of cookies and caching to improve performance.
The document discusses several application layer protocols used in TCP/IP including HTTP, HTTPS, FTP, and Telnet. HTTP is used to access resources on the world wide web over port 80 and is stateless. HTTPS is a secure version of HTTP that encrypts communications over port 443. FTP is used to transfer files between hosts but sends data and passwords in clear text. Telnet allows users to access programs on remote computers.
The document defines key terms related to web connectivity and communication protocols. It describes concepts like applications, APIs, web services, objects, communication gateways, clients, servers, brokers, proxies, web protocols, firewalls, headers, states, resources, and URIs. The definitions provide explanations of these terms in the context of enabling connectivity and communication between devices, systems, and over the web.
A web service allows for data transfer between platforms or languages. It uses PHP code to perform operations like inserting, deleting, fetching, and updating data in a database. The web service code connects to the database using a connection file that contains login credentials. It then decodes request parameters, fires SQL queries to perform the requested operation, and encodes the response. URLs are used to check the operation by passing parameters to specify things like the table row ID, field names, and new values.
The document discusses the key differences between the Internet and the World Wide Web. It explains that the Internet is a global network of interconnected computer networks, while the Web is a collection of information resources linked by hyperlinks and URLs. It provides details on how information is organized and accessed via the Web using browsers, Web pages, links, and protocols like HTTP. The main components, features, and popular examples of Web browsers are also outlined.
The document discusses various peer-to-peer (P2P) file downloading methods including classic P2P networks like Napster and Kazaa, torrent downloading using BitTorrent clients, and more recent methods like using email mailboxes or file storage servers. It notes that while early P2P networks allowed anonymous sharing and downloading, legal issues have led countries to restrict copyrighted content spreading and internet providers are now required to log user activities for police.
This document provides an overview of the security measures used in ShareFile Enterprise, an enterprise file sync and sharing solution. It describes the three main components: 1) the SaaS application tier which includes web, API and database servers, 2) StorageZones which store customer data either in Citrix-managed cloud storage or customer-managed storage, and 3) clients which access ShareFile across various devices. It then details how each component uses encryption, hashing, access controls and other methods to securely transmit and store customer data and metadata.
EUDAT B2SHARE API - How to store and publish research data using the B2SHARE APIEUDAT
This document provides information about using the B2SHARE REST API to store and publish research data. It describes what the B2SHARE API is, how to authenticate with access tokens, and outlines the full workflow for publishing data, including creating draft records, adding files and metadata, and publishing records. Key concepts discussed include communities, records, metadata schemas, and the different states of records. Examples of API calls are also provided.
B2SHARE REST API - New ppt available at https://www.slideshare.net/EUDAT/euda...EUDAT
This document provides an overview of how to use the B2SHARE REST API to store and publish research data. It describes what the B2SHARE API is, the different types of HTTP requests that can be used, how to authenticate via access tokens, and provides a full example workflow of creating a draft record, adding metadata and files, and publishing the record. Key concepts discussed include communities, records, metadata, and the publication states of draft and published.
The HTTP protocol is an application-level protocol used for distributed, collaborative, hypermedia information systems. It operates as a request-response protocol between clients and servers, with clients making requests using methods like GET and POST and receiving responses with status codes. Requests and responses are composed of text-based headers and messages to communicate metadata and content. Caching and cookies can be used to improve performance and maintain state in this otherwise stateless protocol.
EUDAT B2SHARE Deployment - Deploy your own B2SHARE instanceEUDAT
This document provides instructions for deploying an instance of B2SHARE, an open source research data sharing and publishing platform. It describes how to set up the required services, configure authentication with B2ACCESS, customize communities and metadata schemas, and maintain the instance through updates. The goal of B2SHARE is to provide a user-friendly way for researchers to store and publish research data from different contexts.
ASP.NET Web API is the de facto framework for building HTTP-based services in the .NET ecosystem. With its WCF and MVC lineage, Web API brings to the table better architecture, easier configuration, increased testability, and as always, it's customizable from top to bottom. But to properly use Web API it is not enough to get familiar with its architecture and API, you also need to really understand what HTTP is all about. HTTP is the most common application layer protocol in the world, and yet, not many web developers are familiar with HTTP concepts such as of chunking, caching, and persisted connections. In this full-day tutorial, we will focus on designing and implementing HTTP-based services with ASP.NET Web API, and you will learn how to better use it to implement the features provided by HTTP.
The document discusses updates to Creative Commons' work registration and licensing tools. It summarizes the growth of RDFa adoption and CC Network, and proposes changes to the CC Network specification to simplify verification for clients. It also outlines developments in Science Commons' materials transfer agreements, including moving to a JSON representation accessible via SPARQL queries.
The document provides an overview of a presentation on ASP.NET Web API and HTTP fundamentals. It includes sections on HTTP messages and URIs, routing and verbs in Web API, status codes, error handling, content negotiation, media type formatters, validating user input, and enabling OData queryable actions. The presentation aims to cover HTTP fundamentals through the lens of Web API, cramming attendees' brains with information on how to build HTTP-based web services using ASP.NET Web API.
So, you heard "the Web is Programmable, Internet of Things, Digitalization", but have NO to little programming skills. Nevertheless, this is 2016, and you want to get enough about Web Programming to be part of the some fun and exciting Web challenge, participate in an Hackathon may be …
Well, I am happy we meet. I suggest you take the tour “from ZERO to REST in a hour”: we’ll teach you to forge your own HTTP requests against the Github API. After this tour, you’ll know enough to interact with any RESTful Web APIs. Worth mentionning this presentation is entirely scripted: so give attention to each slide comments.
Did you enjoy the tour ? look forward to learn more ?
Post your comments below about enhancements, and for any subjects you’d like to see covered.
2. Join the Cisco developers community : https://developer.cisco.com/
3. Take a free online Coding Lab (REST, Python, Parsing JSON, RAML, Git…)
https://learninglabs.cisco.com/labs/tags/Coding
4. Meet DevNet teams at a physical event: conferences, hackathons
https://developer.cisco.com/site/devnet/events-contests/events/
This document defines common Internet and web terminology including animated GIFs, anonymous FTP sites, bandwidth, bits, bookmarks, browsers, bytes, cyberspace, domains, firewalls, FTP, GIFs, gigabytes, home pages, hosts, HTML, hyperlinks, Internet service providers, IP addresses, JavaScript, JPEG, kilobytes, links, menus, metadata, ports, protocols, search engines, servers, thumbnails, URLs, uploads, web pages, and the World Wide Web.
https://www.youtube.com/watch?v=lKrbeJ7-J98
HTTP messages are how data is exchanged between a server and a client. There are two types of messages: requests sent by the client to trigger an action on the server, and responses, the answer from the server.
CGI allows a web server to interface with external applications to dynamically generate web pages. It uses environment variables and form data passed via GET or POST requests to run scripts or programs that process input and return output to the server. While CGI provides interactivity, it can be slow and has security issues. Server-side includes and client-side scripting offer alternatives for dynamic content generation.
The document provides an overview of the technical capabilities of the InfoRouter content management system. It describes how InfoRouter is standards-based, scalable, secure, entirely web-based, browser independent, supports multiple document formats and languages, integrates with LDAP directories, and has a distributed architecture. It also discusses features like metadata services, web portals, live updating, and integration with web services APIs.
Overview of HTTP, HTML, WWW and web technologies.
The combo HTTP and HTML is the foundation of the World Wide Web (WWW).
HTML (HyperText Markup Language) defines a text-based format for describing the contents of a web page. HTML is based on tags similar to XML (eXtensible Markup Language), but its definition is less strict.
HTML pages are transported with the HTTP protocol (HyperText Transmission Protocol) over TCP/IP based networks.
The power of the WWW comes with the links based on URLs (Uniform Resource Locators) that connect pages to form a web of content.
Browsers display links as clickable items that, when clicked, trigger the browser to load the web page pointed to by the link.
This statelessness contributed a lot to the stability and scalability of the world wide web where web servers are only tasked with the delivery of web pages while the browser is responsible for the rendering of web pages.
The static nature of the early World Wide Web was soon augmented with the dynamic creation of web pages by web servers or by enriching static web pages with dynamic content.
Technologies like CGI (Common Gateway Interface), JSP (Java Server Pages) or ASP (Active Server Pages) were developed to provide the infrastructure to build dynamic web applications.
These server-side technologies were complemented with client-side technologies like Javascript and AJAX (Asynchronous Javascript And XML).
Web page caching is an important mechanism to reduce latency in loading web pages and reducing network traffic.
HTTP defines different caching control mechanisms. Simpler caching methods are based on web page expiry dates while more complex mechanisms use web page validation.
The document describes the implementation of a peer-to-peer server that allows peers to register, deregister, search for, and download content. The server uses TCP sockets and threads to handle multiple client connections simultaneously. Issues encountered included buffers not clearing properly and thread arrays causing segmentation faults. These were resolved by adding buffer clearing logic and allocating memory for thread indices. The implemented code now meets specifications by allowing peers to share a centralized content registry and download files from each other.
This lecture covered CIDR (Classless Inter-Domain Routing) and subnetting IP addresses. It discussed client-server and peer-to-peer networking models. Common network applications like email, web, and file sharing were mentioned. TCP and UDP transport protocols were summarized. HTTP protocol for web was explained in detail including persistent and non-persistent connections.
The document discusses the application layer and various network applications. It begins by covering principles of network applications like client-server and peer-to-peer architectures. It then covers specific applications including HTTP and the web, email using SMTP, POP3 and IMAP, and DNS. It also discusses socket programming, addressing processes, and factors applications consider when choosing transport protocols. The document closes by focusing on HTTP and the web in more detail, covering HTTP requests and responses, persistent connections, and the use of cookies and caching to improve performance.
The document discusses several application layer protocols used in TCP/IP including HTTP, HTTPS, FTP, and Telnet. HTTP is used to access resources on the world wide web over port 80 and is stateless. HTTPS is a secure version of HTTP that encrypts communications over port 443. FTP is used to transfer files between hosts but sends data and passwords in clear text. Telnet allows users to access programs on remote computers.
The document defines key terms related to web connectivity and communication protocols. It describes concepts like applications, APIs, web services, objects, communication gateways, clients, servers, brokers, proxies, web protocols, firewalls, headers, states, resources, and URIs. The definitions provide explanations of these terms in the context of enabling connectivity and communication between devices, systems, and over the web.
A web service allows for data transfer between platforms or languages. It uses PHP code to perform operations like inserting, deleting, fetching, and updating data in a database. The web service code connects to the database using a connection file that contains login credentials. It then decodes request parameters, fires SQL queries to perform the requested operation, and encodes the response. URLs are used to check the operation by passing parameters to specify things like the table row ID, field names, and new values.
The document discusses the key differences between the Internet and the World Wide Web. It explains that the Internet is a global network of interconnected computer networks, while the Web is a collection of information resources linked by hyperlinks and URLs. It provides details on how information is organized and accessed via the Web using browsers, Web pages, links, and protocols like HTTP. The main components, features, and popular examples of Web browsers are also outlined.
The document discusses various peer-to-peer (P2P) file downloading methods including classic P2P networks like Napster and Kazaa, torrent downloading using BitTorrent clients, and more recent methods like using email mailboxes or file storage servers. It notes that while early P2P networks allowed anonymous sharing and downloading, legal issues have led countries to restrict copyrighted content spreading and internet providers are now required to log user activities for police.
This document provides an overview of the security measures used in ShareFile Enterprise, an enterprise file sync and sharing solution. It describes the three main components: 1) the SaaS application tier which includes web, API and database servers, 2) StorageZones which store customer data either in Citrix-managed cloud storage or customer-managed storage, and 3) clients which access ShareFile across various devices. It then details how each component uses encryption, hashing, access controls and other methods to securely transmit and store customer data and metadata.
EUDAT B2SHARE API - How to store and publish research data using the B2SHARE APIEUDAT
This document provides information about using the B2SHARE REST API to store and publish research data. It describes what the B2SHARE API is, how to authenticate with access tokens, and outlines the full workflow for publishing data, including creating draft records, adding files and metadata, and publishing records. Key concepts discussed include communities, records, metadata schemas, and the different states of records. Examples of API calls are also provided.
B2SHARE REST API - New ppt available at https://www.slideshare.net/EUDAT/euda...EUDAT
This document provides an overview of how to use the B2SHARE REST API to store and publish research data. It describes what the B2SHARE API is, the different types of HTTP requests that can be used, how to authenticate via access tokens, and provides a full example workflow of creating a draft record, adding metadata and files, and publishing the record. Key concepts discussed include communities, records, metadata, and the publication states of draft and published.
The HTTP protocol is an application-level protocol used for distributed, collaborative, hypermedia information systems. It operates as a request-response protocol between clients and servers, with clients making requests using methods like GET and POST and receiving responses with status codes. Requests and responses are composed of text-based headers and messages to communicate metadata and content. Caching and cookies can be used to improve performance and maintain state in this otherwise stateless protocol.
EUDAT B2SHARE Deployment - Deploy your own B2SHARE instanceEUDAT
This document provides instructions for deploying an instance of B2SHARE, an open source research data sharing and publishing platform. It describes how to set up the required services, configure authentication with B2ACCESS, customize communities and metadata schemas, and maintain the instance through updates. The goal of B2SHARE is to provide a user-friendly way for researchers to store and publish research data from different contexts.
ASP.NET Web API is the de facto framework for building HTTP-based services in the .NET ecosystem. With its WCF and MVC lineage, Web API brings to the table better architecture, easier configuration, increased testability, and as always, it's customizable from top to bottom. But to properly use Web API it is not enough to get familiar with its architecture and API, you also need to really understand what HTTP is all about. HTTP is the most common application layer protocol in the world, and yet, not many web developers are familiar with HTTP concepts such as of chunking, caching, and persisted connections. In this full-day tutorial, we will focus on designing and implementing HTTP-based services with ASP.NET Web API, and you will learn how to better use it to implement the features provided by HTTP.
The document discusses updates to Creative Commons' work registration and licensing tools. It summarizes the growth of RDFa adoption and CC Network, and proposes changes to the CC Network specification to simplify verification for clients. It also outlines developments in Science Commons' materials transfer agreements, including moving to a JSON representation accessible via SPARQL queries.
The document provides an overview of a presentation on ASP.NET Web API and HTTP fundamentals. It includes sections on HTTP messages and URIs, routing and verbs in Web API, status codes, error handling, content negotiation, media type formatters, validating user input, and enabling OData queryable actions. The presentation aims to cover HTTP fundamentals through the lens of Web API, cramming attendees' brains with information on how to build HTTP-based web services using ASP.NET Web API.
So, you heard "the Web is Programmable, Internet of Things, Digitalization", but have NO to little programming skills. Nevertheless, this is 2016, and you want to get enough about Web Programming to be part of the some fun and exciting Web challenge, participate in an Hackathon may be …
Well, I am happy we meet. I suggest you take the tour “from ZERO to REST in a hour”: we’ll teach you to forge your own HTTP requests against the Github API. After this tour, you’ll know enough to interact with any RESTful Web APIs. Worth mentionning this presentation is entirely scripted: so give attention to each slide comments.
Did you enjoy the tour ? look forward to learn more ?
Post your comments below about enhancements, and for any subjects you’d like to see covered.
2. Join the Cisco developers community : https://developer.cisco.com/
3. Take a free online Coding Lab (REST, Python, Parsing JSON, RAML, Git…)
https://learninglabs.cisco.com/labs/tags/Coding
4. Meet DevNet teams at a physical event: conferences, hackathons
https://developer.cisco.com/site/devnet/events-contests/events/
1. The document describes a project to build RESTful web services for a social media application like Pinterest. It includes functionalities implemented, high-level architecture with Python web services, CouchDB database, and client testing with cURL.
2. The web services were built with Bottle microframework and exposed CRUD operations as HTTP methods. Data was stored in CouchDB with a flat schema for each user document.
3. While the schema had limitations of returning full documents, it mapped requests to data through URL traversal and operations through HTTP methods as required for REST.
The document provides an overview of REST APIs and how they can be used to build web applications. It discusses HTTP and URLs, defines REST and its architectural constraints, and shows how to implement REST principles using HTTP methods, URLs and response codes. The document also covers REST API design best practices like versioning, limiting fields, using JSON format, and handling errors properly. It provides resources for further reading on REST APIs and their implementation.
This document summarizes a presentation about building RESTful applications using PHP. REST (Representational State Transfer) is a software architectural style that uses HTTP verbs to manipulate resources. The presentation covers the six constraints that define REST - client-server separation, statelessness, cacheability, uniform interface, layered system, and code on demand. It provides tips for implementing REST in PHP like using the header() function, determining the request method, and encoding/decoding JSON. Questions from the audience are answered at the end.
Best Practices for Architecting a Pragmatic Web API.Mario Cardinal
This presentation teach how to design a real-world and pragmatic web API. It draws from the experience Mario Cardinal have gained over the years being involved architecting many Web API. This presentation begins by differencing between a Web and a REST API, and then continue with the design process. We conclude with the core learnings of the session which is a review of the best practices when designing a web API. Armed with skills acquired, you can expect to see significant improvements in your ability to design a pragmatic web API.
shobhit training report (3) (4).pdf reportpoojaranga2911
A report is a nonfiction account that presents and summarizes the facts about a particular event, topic, or issue. The idea is that people who are unfamiliar with the subject can find everything they need to know from a good report. Reports make it easy to catch someone up to speed on a subject, but actually writing a report is anything but easy. So to help you understand what to do, below we present a little report of our own, all about report writing and report format best practices.A report is a nonfiction account that presents and/or summarizes the facts about a particular event, topic, or issue. The idea is that people who are unfamiliar with the subject can find everything they need to know from a good report.Reports make it easy to catch someone up to speed on a subject, but actually writing a report is anything but easy. So to help you understand what to do, below we present a little report of our own, all about report writing and report format best practices.A report is a nonfiction account that presents and/or summarizes the facts about a particular event, topic, or issue. The idea is that people who are unfamiliar with the subject can find everything they need to know from a good report.
Reports make it easy to catch someone up to speed on a subject, but actually writing a report is anything but easy. So to help you understand what to do, below we present a little report of our own, all about report writing and report format best practices.A report is a nonfiction account that presents and/or summarizes the facts about a particular event, topic, or issue. The idea is that people who are unfamiliar with the subject can find everything they need to know from a good report. Reports make it easy to catch someone up to speed on a subject, but actually writing a report is anything but easy. So to help you understand what to do, below we present a little report of our own, all about report writing and report format best practices.A report is a nonfiction account that presents and/or summarizes the facts about a particular event, topic, or issue. The idea is that people who are unfamiliar with the subject can find everything they need to know from a good report. Reports make it easy to catch someone up to speed on a subject, but actually writing a report is anything but easy. So to help you understand what to do, below we present a little report of our own, all about report writing and report format best practices.A report is a nonfiction account that
presents and/or summarizes the facts about a particular event, topic, or issue. The idea is that people who are unfamiliar with the subject can find everything they need to know from a good report. A report is a nonfiction account that presents and/or summarizes the facts about a particular event, topic, or issue. The idea is that people who are unfamiliar with the subject can find everything they need to know from a good report. Reports make it easy to catch someone up to speed A report it
Introduction to data science with H2O-ChicagoSri Ambati
This document provides an overview of H2O.ai, an open source in-memory machine learning platform. It describes H2O.ai's product as an in-memory prediction engine, its team of 37 distributed systems engineers doing machine learning, and its headquarters in Mountain View, CA. It also provides details on how to use H2O with R and Python for scalable machine learning on large datasets across distributed systems.
Introduction to Data Science with H2O- Mountain ViewSri Ambati
This document provides an overview of H2O.ai, an open source in-memory machine learning platform. It discusses that H2O.ai was founded in 2011 and is venture-backed, with a team of 37 people working on distributed systems for machine learning. It also summarizes that H2O provides easy to use APIs for Java, R, Python and other languages, and allows for scalable machine learning on large datasets using distributed algorithms to make full use of data without downsampling. Finally, it highlights how H2O works with other technologies like Spark, Hadoop, and HDFS to enable reading of large datasets for machine learning.
This document provides an overview of distributed web-based systems, including the key components and technologies that enable them. It discusses the World Wide Web and how documents are accessed via URLs. It also describes HTTP and how connections and requests/responses work. Other topics covered include caching, content distribution networks, web services, traditional and multi-tiered web architectures, web server clusters, and web security protocols like SSL.
Primary focus of this presentation is on the hypermedia as the engine of application state (HATEOAS) and how HTTP APIs may benefit from it. Provides sneak peek into HAL media type & gives an overview of hypermedia support in Java tools (JAX-RS / HalBuilder and Spring HATEOAS) along with practical suggestions for server-side design of hypermedia API. Also includes quick overview of Richardson Maturity Model based on a set of examples, current API trends.
GlueCon 2018: Are REST APIs Still Relevant Today?LaunchAny
A look at common API styles available today, a look back at historical API styles, and guidance for selecting the right API styles for your organization. Deep-dive of HTTP, mentioned in the presentation, can be found at: http://bit.ly/power-http
This document discusses REST and ASP.NET WebAPI. It begins with introductions to REST, the Richardson Maturity Model, and ASP.NET WebAPI. It then covers key WebAPI concepts like controllers, routes, content negotiation, and HTTP methods. A large portion discusses hypermedia and the OData protocol for building hypermedia-driven REST services with WebAPI and Entity Framework. It provides examples of OData queries, metadata support, and routing. The document concludes with recommendations for further reading and questions.
With a network of more than 20 European research
organisations, data and computing centres in 14 countries,
the EUDAT Collaborative Data Infrastructure (CDI) is one of
the largest infrastructures of integrated data services and
resources supporting research in Europe.
The EUDAT Collaborative Data Infrastructure (CDI) provides a suite of integrated data services and resources to support research across Europe. It is sustained by over 20 European research organizations and data centers. The CDI includes services like B2SHARE for publishing and sharing research data, B2FIND for discovering data, B2DROP for syncing and exchanging data, B2SAFE for replicating data safely across administrative domains, and more. These services help researchers and communities manage the entire research data lifecycle in a FAIR and trusted manner.
The EUDAT Collaborative Data Infrastructure (CDI) is a growing European research data infrastructure that supports the full lifecycle of research data across borders and disciplines. It provides integrated data services through a collaboration between service providers and research communities. The CDI assists researchers with training and consultancy on research data management and helps build the backbone of the European Open Science Cloud.
B2HANDLE is a distributed service that maintains globally unique persistent identifiers (PIDs) to reliably identify and cite research data objects throughout their lifecycle. It is based on the Handle System and provides a Python library for minting, storing, managing, and resolving PIDs. In the EUDAT ecosystem, services like B2SAFE and B2SHARE use B2HANDLE to create and manage PIDs for hosted data objects, while B2FIND and B2STAGE use PIDs to retrieve and refer to objects.
B2SHARE is a service that allows researchers to store, publish, and share research data in a findable, accessible, interoperable, and reusable (FAIR) manner. It supports metadata descriptions, assigns persistent identifiers like DOIs and Handles, allows versioning and community domains with custom metadata and access rules. Data is stored securely in data centers in Finland and replicas may be made at other trusted centers to ensure long-term preservation.
B2SAFE is a robust service that allows repositories to implement data management policies across administrative domains in a trustworthy manner. It offers an abstraction layer for large-scale heterogeneous storage, protects against data loss, allows optimized access, and enables compute-intensive analysis. B2SAFE is designed to execute auditable policy rules and use persistent identifiers to increase trust in data reuse by ensuring ownership rights and replicating data across sites for safekeeping.
B2FIND is an interdisciplinary discovery portal that allows users to search for and find research data collections from various sources using faceted search options. It harvests and indexes metadata from EUDAT data centers, community repositories, and other sources using various protocols, and provides a unified search interface across data from different scientific domains and metadata standards. Search results can be filtered by location, time period, and other textual facets. Data providers can make their research data collections discoverable on B2FIND by defining mappings between their metadata and the B2FIND schema.
B2ACCESS is a federated authentication and authorization proxy that allows users to log in using credentials from their home institutions or social media accounts to access services in a secure way. It supports standards like SAML, OAuth, and OIDC for service integration and allows single sign-on access for users. B2ACCESS works by authenticating users through external identity providers like university login systems and then authorizes them for access to EUDAT services based on their group memberships.
This document discusses writing effective service documentation for the EUDAT B2 suite of services. It provides an overview of the EUDAT B2 services and research data lifecycle context. It describes the approach to documentation writing, including identifying the end users and leveraging documentation from core technologies. The format and locations of documentation for services like B2ACCESS, B2DROP, B2FIND, B2HANDLE, B2SAFE, and B2SHARE are outlined, including code repositories, descriptions, usage guides, and training materials.
Are you a researcher, citizen scientist, institution or community looking for data storage and value-added services? Do you want access to tools to make your research data more FAIR (findable, accessible, interoperable, and reusable)? Interested in seeing how the future European Open Science Cloud could support research data and practically foster cross-border, cross-disciplinary collaboration? Then this webinar is for you!
This document summarizes a pilot project using B2NOTE to semantically annotate research data stored in EUDAT B2SHARE. The project was conducted by researchers at the University of Porto. B2NOTE allows researchers to directly annotate datasets in B2SHARE using lightweight ontologies. The annotations are stored along with the datasets. This helps promote semantic annotation of research data by those who create it.
The document discusses plans for a task force on research data management (RDM) as part of the OpenAIRE Advance project. The RDM task force will work to establish working groups to examine different elements of the digital research lifecycle. It will gather examples of good RDM practices and standards. The task force will also scope current and future needs of National Open Access Desks to support RDM. The goal is to increase knowledge of RDM among project partners and support open data and FAIR data activities. A separate section discusses plans for open science training through a helpdesk. The helpdesk will provide practical guidance on open science principles and develop training materials and case studies in collaboration with other projects and infrastructures.
European Open Science Cloud - Skills workshopEUDAT
This document outlines the agenda for a workshop on skills for the European Open Science Cloud (EOSC). The workshop will discuss the skills needed to use EOSC services, existing skills frameworks and training provisions from related projects like EOSCpilot, skills mapping between competencies and cloud capabilities, and applying FAIR principles to training resources. Participants will provide input through live polling and breakout groups. The goal is to validate existing work on skills and identify remaining questions, and discuss how EOSC can help close skills gaps for research institutions, libraries and infrastructures.
Linking service capabilities to data stweardship competences for professional...EUDAT
This document discusses the development of a skills framework for the European Open Science Cloud (EOSC) project. It outlines the goals of defining a data stewardship competency framework, analyzing current skills gaps, and making recommendations for training services in EOSC. The document discusses how stewardship ensures data and research objects are managed according to FAIR principles. It also summarizes the analysis of the current complex landscape of skills training providers and the skills gaps identified by EOSC demonstrators and other stakeholders. Next steps include additional workshops to gather feedback on missing competencies and how EOSC can enhance training.
This document discusses applying FAIR data principles to training materials and events to make them more findable, accessible, interoperable, and reusable. It suggests repositories use DOIs and versioning to make materials findable, use standard metadata and formats to make them accessible and interoperable, and provide rich descriptions and licenses to make them reusable. It also poses questions about how EOSC can support training providers and ensure a comprehensive catalogue of high quality training resources.
Training by EOSC-hub - Integrating and Managing services for the European Ope...EUDAT
EOSC-hub receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777536. EOSC-hub will create a federated integration and management system for the European Open Science Cloud to provide services for data, applications and tools, baseline services, a marketplace, AAI, accounting, monitoring and more according to principles of engagement, security regulations, standards, and terms of use. EOSC-hub involves 20 European infrastructures and over 100 partners working to implement the Federated Service Management training and provide domain-specific and generic training services.
The document proposes a draft governance framework for the European Open Science Cloud (EOSC) consisting of three layers:
1) A Steering layer representing users and providers to determine requirements, policies, and evaluate EOSC effectiveness.
2) A Strategic layer representing funders and decision makers to set the strategic vision, objectives, and performance metrics.
3) An Executive layer ensuring EOSC meets objectives by channeling funding as needed.
The framework is intended to be flexible, leveraging existing structures, and applying the European Interoperability Framework to coordinate different stakeholders across borders. Feedback is sought on the proposed model and decision flow between layers.
The document discusses the AARC project, which aims to improve federated identity management (FIM) for researchers across Europe. The project works to address eScience requirements and offer support for global policies around FIM. It has developed a blueprint architecture and set of building blocks to enable authentication and authorization across research collaborations and infrastructures. The AARC project also focuses on engagement with research communities and infrastructure providers to promote adoption of harmonized solutions. It has established several working groups and frameworks to facilitate collaboration and address security, assurance, and policy aspects of integrated identity and access management.
The Ipsos - AI - Monitor 2024 Report.pdfSocial Samosa
According to Ipsos AI Monitor's 2024 report, 65% Indians said that products and services using AI have profoundly changed their daily life in the past 3-5 years.
Learn SQL from basic queries to Advance queriesmanishkhaire30
Dive into the world of data analysis with our comprehensive guide on mastering SQL! This presentation offers a practical approach to learning SQL, focusing on real-world applications and hands-on practice. Whether you're a beginner or looking to sharpen your skills, this guide provides the tools you need to extract, analyze, and interpret data effectively.
Key Highlights:
Foundations of SQL: Understand the basics of SQL, including data retrieval, filtering, and aggregation.
Advanced Queries: Learn to craft complex queries to uncover deep insights from your data.
Data Trends and Patterns: Discover how to identify and interpret trends and patterns in your datasets.
Practical Examples: Follow step-by-step examples to apply SQL techniques in real-world scenarios.
Actionable Insights: Gain the skills to derive actionable insights that drive informed decision-making.
Join us on this journey to enhance your data analysis capabilities and unlock the full potential of SQL. Perfect for data enthusiasts, analysts, and anyone eager to harness the power of data!
#DataAnalysis #SQL #LearningSQL #DataInsights #DataScience #Analytics
Global Situational Awareness of A.I. and where its headedvikram sood
You can see the future first in San Francisco.
Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. Every six months another zero is added to the boardroom plans. Behind the scenes, there’s a fierce scramble to secure every power contract still available for the rest of the decade, every voltage transformer that can possibly be procured. American big business is gearing up to pour trillions of dollars into a long-unseen mobilization of American industrial might. By the end of the decade, American electricity production will have grown tens of percent; from the shale fields of Pennsylvania to the solar farms of Nevada, hundreds of millions of GPUs will hum.
The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word. Along the way, national security forces not seen in half a century will be un-leashed, and before long, The Project will be on. If we’re lucky, we’ll be in an all-out race with the CCP; if we’re unlucky, an all-out war.
Everyone is now talking about AI, but few have the faintest glimmer of what is about to hit them. Nvidia analysts still think 2024 might be close to the peak. Mainstream pundits are stuck on the wilful blindness of “it’s just predicting the next word”. They see only hype and business-as-usual; at most they entertain another internet-scale technological change.
Before long, the world will wake up. But right now, there are perhaps a few hundred people, most of them in San Francisco and the AI labs, that have situational awareness. Through whatever peculiar forces of fate, I have found myself amongst them. A few years ago, these people were derided as crazy—but they trusted the trendlines, which allowed them to correctly predict the AI advances of the past few years. Whether these people are also right about the next few years remains to be seen. But these are very smart people—the smartest people I have ever met—and they are the ones building this technology. Perhaps they will be an odd footnote in history, or perhaps they will go down in history like Szilard and Oppenheimer and Teller. If they are seeing the future even close to correctly, we are in for a wild ride.
Let me tell you what we see.
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...Social Samosa
The Modern Marketing Reckoner (MMR) is a comprehensive resource packed with POVs from 60+ industry leaders on how AI is transforming the 4 key pillars of marketing – product, place, price and promotions.
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
State of Artificial intelligence Report 2023kuntobimo2016
Artificial intelligence (AI) is a multidisciplinary field of science and engineering whose goal is to create intelligent machines.
We believe that AI will be a force multiplier on technological progress in our increasingly digital, data-driven world. This is because everything around us today, ranging from culture to consumer products, is a product of intelligence.
The State of AI Report is now in its sixth year. Consider this report as a compilation of the most interesting things we’ve seen with a goal of triggering an informed conversation about the state of AI and its implication for the future.
We consider the following key dimensions in our report:
Research: Technology breakthroughs and their capabilities.
Industry: Areas of commercial application for AI and its business impact.
Politics: Regulation of AI, its economic implications and the evolving geopolitics of AI.
Safety: Identifying and mitigating catastrophic risks that highly-capable future AI systems could pose to us.
Predictions: What we believe will happen in the next 12 months and a 2022 performance review to keep us honest.
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
End-to-end pipeline agility - Berlin Buzzwords 2024
B2SHARE REST API
1. www.eudat.eu
EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065
B2SHARE REST API hands-on
Hans van Piggelen, hans.vanpiggelen@surfsara.nl
Thursday July 6th, 2017
This work is licensed under the Creative
Commons CC-BY 4.0 licence
2. Today’s hands-on
Theory: (30 min)
The B2SHARE REST API
B2SHARE concepts, variables, metadata schemas
API: making requests, authentication and payloads
Publication workflow
Practice: (1 hour)
Simple examples
Hands-on exercises
Next hands-on: B2FIND
3. What is the B2SHARE REST API?
The B2SHARE REST API is a set of instructions to interact
with a B2SHARE service instance
The B2SHARE REST API:
Allows direct remote interaction with the B2SHARE service
without using a graphical user interface
Allows integration within application or data processing
workflows for automation of publishing tasks
Supports any programming language or operating system
that supports HTTP requests
There is (almost) no limitation on usage and number of calls,
even for unregistered users
To create new or modify existing content, registration is
required
4. Why use the B2SHARE REST API?
Using the B2SHARE API you can automate:
Creation of records
Uploading of files
Adding/changing metadata
Changing publications state
Retrieval of object data
Enable:
Precise replication of metadata into repository
Large file uploads
Implement publishing in your own workflow or application
Direct data ingest
Ease administration and overview of your records
5. What can I do with the B2SHARE API?
List (all) existing records and communities
Search for specific records and communities
Retrieval of community-specific information
Including community metadata schemas
Create new draft records
Upload files to draft records
Add metadata using metadata schemas
Publish draft records
Modification of the metadata of existing records
More to be added in future releases…
6. Important concepts of B2SHARE
Records:
Contain data files and associated metadata
Connected to a community which possibly maintains it
Metadata:
Set of common fixed metadata fields and custom metadata
blocks with additional fields
Governed by fixed and community metadata schemas
Communities:
Curate datasets which are part of the scientific domain or a
research project
Maintain their own metadata schemas and have community
administrators
States:
Current condition of a record, either draft, submitted or published
Can be changed through the API
7
7. B2SHARE request variables
B2SHARE defines several request variables that
function as identifiers for objects in B2SHARE
Used in most HTTP request addresses as part of the
path to access specific objects directly
Most important variables:
COMMUNITY_ID: identifier of a user community
RECORD_ID: identifier for a specific record, in either
state
FILE_BUCKET_ID: identifier for a set of files of a
specific record
8
8. Draft records and versioning
Draft records:
Can be updated with new
files and metadata
Have publication state ‘draft’
Published records:
Cannot have files updated
anymore
Metadata updates possible
but discouraged
B2SHARE supports
versioning of records:
Existing published records
can be updated through their
draft counterpart
Creates new PIDs, bucket
IDs, links
Draft
✔ New files
✔ New records
Published
✗ New files
✗ New records
Draft
9. Community metadata schemas
Every community defines its own metadata schema
using a hierarchical JSON Schema-based structure
Metadata schemas:
Contain descriptions, vocabularies and expected
structure and format of every metadata field,
including optional fields
May contain community-specific fields
Definitions are publically available, e.g. EUDAT:
10
14. Making a request
Requests can be made by using a specific application
directly or by using a programming language that
supports making requests in code
Example applications:
GUI: any file transfer application or web interface
Command line: cURL, wGet
Almost all programming languages support making
requests over HTTP
For more complex operations (like publication), a
dedicated interface or command line application is more
useful
16
15. HTTP requests
A specific call to a service through an API using a method
with address and parameters
An address is a URL:
URL = protocol + hostname + port + path
Protocol: always http:// or https://
Hostname: base address, e.g. b2share.eudat.eu
Port: sometimes required specifically, usually 80 or 443
Path: endpoint specification, e.g. /api/record/1
Optional parameters are additional options given to the
request and are added to the URL
On success the current state of the requested piece of
information is provided
The B2SHARE service accepts payloads along with a HTTP
request, e.g. text, binary data, files
17
16. HTTP request method
Different methods have different meaning, but up to
service on how to process them
Common methods:
GET: requests data from a specified resource
POST: submits data to be processed to a specified
resource
PUT: uploads a representation of the specified URI
PATCH: modify state of specified resource
DELETE: delete a specified resource
17. HTTP responses
Every request returns a status, header and message body, even
when an error occurred
Status line: status code and reason
Header: information on body content
Body: actual response text
Status codes:
1xx: Informational – Request received, continuing process
2xx: Success – Action was successfully received, understood,
and accepted
3xx: Redirection – Further action must be taken in order to
complete the request
4xx: Client Error – Request contains bad syntax or cannot be
fulfilled
5xx: Server Error – Server failed to fulfill an apparently valid
request
HTTP response is pure text, needs interpretation
18. HTTP request overview
20
Browser CLI tool
Your app or
workflow
HTTP response:
- Header
- Status code
- Response text
HTTP request:
- Request method
- Header
- URL & parameters
- Authentication
- Payloads
Server
Client
19. Authentication through the API
B2SHARE does not accept username and password
combination, instead use tokens for authentication!
B2SHARE contains open and restricted data:
Public: all published records and metadata, most files
No access token required
Private: your draft records and files in private records
Only accessible using your access token as parameter
in HTTP request
Access tokens:
Automatically generated unique string of characters
attached to your account in B2SHARE
Only known by the owner, do not share with others!
21
20. Full B2SHARE publication workflow
Publishing in B2SHARE using the API involves multiple
steps:
Identify a target community for your data
Retrieve the metadata schema definition of the
community
The submitted metadata will have to conform to
this schema
Create a draft record:
Upload files into the draft record (one by one)
Add metadata according to schema (possibly in
multiple steps)
Publish the record
22
21. Full publication workflow diagram
23
Create
draft
Add metadata
Add files
POST /api/records
GET /api/records/RECORD_ID/draft
PATCH /api/records/RECORD_ID/draft
PUT
/api/files/FILE_BUCKET_ID/FILE_NAME
Commit
Published
record
PATCH /api/records/RECORD_ID/draft
Draft
record
PID
Checksum
Needs
approval?
Submitted
record
Community
Approve
NO
YES
22. Adding metadata
Metadata is added to a record using the API:
Upon creation of a draft record in a POST request
Or by providing so-called JSON patches in a PATCH
request
Patches modify the current state of the metadata by either
changing, adding or removing fields and values
The structure of the data provided in the patch request must
strictly follow the
metadata schema of the
community
As many patches as necessary
can be applied before
publishing your draft record
24
Add
metadata
Draft
record
23. Adding files
Files can be added during the draft phase of your new
record using a PUT request
Files are uploaded into the file bucket of the draft
record, not the draft record itself
Use the file bucket ID found in the metadata of the
draft record
Files are uploaded one-by-one in separate requests
As many file upload requests
as necessary can be made
before publishing your draft
record
25
Add files
Draft
record
Checksum
24. Publishing your draft record
Draft records are published by altering the value of the
publication state in the metadata
Once your record is published:
The included files can not be changed anymore!
No new files can be added!
Metadata can be changed after publication, but will
create a new version of your published record
Persistent identifiers are automatically added on
commit
26
Commit
Draft
record
Published
record
PID
25. Simple examples
Protocol and host: https://trng-b2share.eudat.eu
Application: python (using command-line interface)
HTTP method: GET
Retrieve all existing records:
List all communities:
Search for specific records of a community:
27
r = requests.get(‘https://trng-b2share.eudat.eu/api/records’)
r = requests.get(‘https://trng-b2share.eudat.eu/api/records?
q=community:COMMUNITY_ID`)
r = requests.get(‘https://trng-b2share.eudat.eu/api/communities’)
26. Complex example
HTTP method: POST
Create draft record ‘My test upload’:
header = {‘Content-Type’: 'application/json'}
metadata = {"titles": [{"title":"My test upload"}],
"community": "e9b9792e-79fb-4b07-b6b4-b9c2bd06d095",
"open_access": True}
r = requests.post('https://trng-b2share.eudat.eu/api/records/', params=parameters,
data=json.dumps(metadata),
headers=header)
parameters = {'access_token': token}
27. JSON patch
A set of operations that alter an existing set of metadata
fields based on another set of fields
Loosely equals the difference between two sets
Operations: add, remove, replace, copy, move, test
Generated by jsonpatch package
Requires PATCH request to apply to record
Metadata
OLD
Metadata
NEW
Metadata
PATCH
29. Today’s hands-on exercises
Get and store your API token
Retrieve single record information
Check metadata and included files
Download files and compare checksum
Retrieve existing communities
Retrieve community metadata schema
Investigate metadata schema structure
Create a new draft record
Upload files and metadata
Update and complete metadata
Publish record
30. General instructions
Create an API token on the B2SHARE training website
Requirements for each request:
Request URL and HTTP method (e.g. GET, PUT)
Optional:
Object identifiers (e.g. record, community)
Additional parameters (e.g. your token)
Data payloads (e.g. files or text)
Use requests package for HTTP requests
Use jsonpatch package to create metadata update
patches
B2SHARE API endpoint: /api
31. General instructions
Log in to 145.100.59.156
ssh <user>@145.100.59.156
Use Python or iPython as interface
Helpful links:
Exercises: https://hdl.handle.net/21.T12996/ESS2017-
B2SHARE-API
Example image:
https://hdl.handle.net/21.T12996/ESS2017-Image.png
Backup token:
https://hdl.handle.net/21.T12996/token.txt
Ask questions anytime!
32. Getting your access token
Log in on B2SHARE and navigate to profile page:
Create a token by entering a new name:
Click on ‘New token’
33. Saving your token to file
Store the token in a file so it can be restored later:
Load the token in Python:
$ echo “<your token>” > token.txt
f = open(‘token.txt’, ‘r’)
token = f.read().strip()
34. Exercise 1a: single record retrieval
Endpoint: /api/records/<RECORD_ID>
Method: GET
Response status code: 200
RECORD_ID: 47077e3c4b9f4852a40709e338ad4620
Steps:
Create the URL
Retrieve the object data
Parse the response text
35. Exercise 1b: process record metadata
Use data of previous record
Steps:
Get the metadata field values
Investigate the file(s) contained
Check if open access and published
Get the file bucket ID, file key(s) and checksum(s)
36. Exercise 1c: download file and check
Endpoint: /api/files/<FILE_BUCKET_ID>/<FILE_KEY>
Method: GET
Response status code: 200
Steps:
Create the URL
Download files
Calculate checksum and compare
37. Exercise 2a: communities retrieval
Endpoint: /api/communities
Method: GET
Response status code: 200
Steps:
Get all communities
Parse the response text
Locate the EUDAT community and its ID
38. Exercise 2b: EUDAT community records
Endpoint:
/api/records?q=community:<COMMUNITY_ID>
Method: GET
Response status code: 200
COMMUNITY_ID: <>
Steps:
Set required parameters for request
Determine number of records
Show first record
39. Exercise 2c:
EUDAT community metadata schema
Endpoint:
/api/communities/<COMMUNITY_ID>/schemas/last
Method: GET
Response status code: 200
COMMUNITY_ID: <>
Steps:
Determine number of metadata fields
Determine required fields
Determine community-specific fields
Determine metadata field structure
40. Exercise 3a: create draft record
Endpoint: /api/records/
Method: POST
Response status code: 201
Steps:
Prepare header and payloads
Get API token
Get draft record ID
Check publication state
Get file bucket ID
41. Exercise 3b: add files
Endpoint: /api/files/<FILE_BUCKET_ID>/<FILE_KEY>
Method: PUT
Response status code: 200
FILE_BUCKET_ID: <draft record’s file bucket ID>
FILE_KEY: <your file name>
Steps:
Prepare header
Open file handle
Send file with request
42. Exercise 3c: add metadata
Endpoint: /api/records/<RECORD_ID>/draft
Method: PATCH
Response status code: 200
RECORD_ID: <draft record ID>
Steps:
Prepare header
Prepare JSON patch
Send patch with request
43. Exercise 3d: publish record
Endpoint: /api/records/<RECORD_ID>/draft
Method: PATCH
Response status code: 200
RECORD_ID: <draft record ID>
Steps:
Prepare header
Prepare JSON patch
Send patch with request
Check publication state of record
Check record in web browser
44. For more info: https://eudat.eu/services/b2share
B2SHARE User Documentation:
https://eudat.eu/services/userdoc/b2share
B2SHARE Training presentations:
https://www.eudat.eu/b2share-training-suite
B2SHARE hands-on training:
https://github.com/EUDAT-Training/B2SHARE-Training 46
45. www.eudat.eu
This work is licensed under the Creative Commons CC-BY 4.0 licence
EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures.
Contract No. 654065
Authors Contributors
Hans van Piggelen, SURFsara
Thank you!
Editor's Notes
Who knows what B2SHARE is?
Who knows what an API is?
Who has used an API before?
Who has accessed an API using Python before?
The next three slides may be common knowledge, so we can skip them if everybody knows about these. API stands for Application Programming Interface, and is a specification of a set of definitions, protocols and tools which allow to interact with a service, possibly from a remote location. An API provides an abstraction layer of the underlying service technologies used by the service itself and external applications to communicate with the technology without exactly knowing how it works. This allows machines to easily interact with the service and correctly handle responses and possible errors.
When it comes to APIs for the Internet, communication is done through HTTP requests which return machine-readable structures of data representing the current state of a piece of information. The API can often used to alter information on the server side as well. The returned data is usually provided in specific formats like XML or JSON. In many cases this can be changed upon request.
Your own browser is using the same mechanisms to get data from a server and present it to the user.
B2SHARE has web-based GUI, but it also offers a REST API. The B2SHARE API is a set of instructions to interact with a B2SHARE service instance, for example the one of EUDAT. It provides direct interaction with the service without using a graphical user interface. Instead a command line tool or an application that integrates the communication is used. For example, the tools used in a research workflow might include automatic upload of final data in order to directly publish it. Most programming language have built-in functionality to interact with a remote service using HTTP requests.
The B2SHARE REST API can be used by any user without registration and without limitations. The only functionality that requires registration is the creation and modification of records and defined communities.
Using the B2SHARE API, the user can do several important operations such as listing all existing records and communities that are currently defined in the service. When a user is looking for specific records and/or communities the query functionality can be used and filtered using several parameters.
Once a community has been chosen, detailed community-specific information, including community metadata schema definitions and other requirements can be retrieved.
New draft records can be created, including the upload of files and the addition of metadata in accordance to the metadata schema defined by a community or project. Once a draft record is completed, it can be published. Furthermore, existing published records can be modified in terms of metadata and published as a new versioned record. The old version will always be available and citable.
There are several important concepts used in B2SHARE which are relevant to using the API.
First there are communities that curate datasets which are part of the scientific domain or a research project. Community administrators maintain the metadata schema of the community. Users have to select the community when creating new records in order to have it connected that community.
Records contain data files and associated metadata and are always connected to a specific community. Communities actively curate records that are published under their name. When publishing under the EUDAT general community, this is not the case.
Metadata are a set of common fixed metadata fields and custom metadata blocks with additional fields. They are governed by fixed and community metadata schemas. When choosing a specific community, there might be additional metadata fields to be filled in.
A state is the current condition of a record, either draft, submitted or published. They can be changed through the API. Only published records are visible in B2SHARE. Draft and submitted records are only visible to the owner and possibly the community administrator under which the record is published.
B2SHARE defines several request variables that function as identifiers for objects in B2SHARE. They are used in most HTTP request addresses as part of the path to access specific objects directly
The most important variables are the community ID which uniquely identifies a community. Record IDs identify specific records, either in draft or published state.
The file bucket ID is used to identify a set of files of a specific record
All records are published under a specific community and have metadata added according to the corresponding community metadata schema in case this is required.
Metadata schemas contain the descriptions, vocabularies and expected structure and format of every metadata field in the schema. Furthermore, they define which fields are mandatory to fill in during the creation of new records.
Community metadata schema definitions are publically available through the API and on the B2SHARE website.
To actually make a request, use a designated tool or application, or integrate it into your own application using a programming language which supports this. Examples are GUI applications like file transfer tools or web interfaces on websites. On the command line typical examples are curl and wget.
Most programming languages support making requests over HTTP, but often require the inclusion of packages which provide this functionality.
For more complex operations (like publication), a dedicated interface or command line application is required and therefore a file transfer tool often doesn’t suffice.
APIs are used by making HTTP requests. These are specific calls to a service using a supported method with address and parameters. The address is structured as a URL which consists of a protocol, hostname, port (often not necessary) and a path which possibly identifies the piece of information requested. The path is often called the API endpoint. Multiple endpoints can exists for a given service.
Parameters are additional options given to the request which can further filter the return information or specify for example the return format.
All requests always return a response, provided that the server is available, even when the request is malformed or some other error occurred. Depending on the success, the request contains a HTTP status code and a response text with further information.
On success the current state of the requested piece of information is provided.
We will now look at some examples of using the API. For many requests you need to authenticate yourself. As APIs are often used directly in applications, you don’t want to provide usernames and passwords for every request you make. Instead an access token is generated on the website which uniquely identifies yourself during a request.
B2SHARE contains public or open access and restricted data. All open access published records and metadata are public and can be accessed by anyone, this mostly holds for the contained files as well. Therefore no access tokens are required. For privately shared records and your draft records and files an access token is required. Depending on the authorization and community settings, you may or may not be able to access these records.
As the access token unique identifies yourself and allows alteration of your published and draft records, do not share it with anyone!
The full publication workflow using the B2SHARE REST API is as follows:
Identify a target community to place your new record under. You need the community ID of that community and depending on the community, only members of that community can publish under their name.
Get the metadata schema definition of that community using the community ID. Now you know the required fields for your publication
Create a draft record and upload your files and add metadata. This can all be done in multiple separate steps and for files one-by-one.
When adding files, checksums will be generated for each file.
Finally publish your record. Depending on the community’s settings, your record may need approval of the community before it will be shown in B2SHARE.
A persistent identifier is added to the record so that it can be uniquely identified.
For clarity, here there is a complete overview of the full publication workflow using the B2SHARE REST API in a diagram. For every step (blue boxes) the corresponding request and HTTP method has been added in which the variables need to be filled in. The only exception is the ‘community approve’ step which can’t be done through the API yet.
A record can be in three states: draft, submitted or published (red boxes). Draft and submitted records are not visible openly, but only to the user or community.
Checksums are calculated for every file added and a PID is added once the draft record is committed (green boxes).
There are two ways of adding metadata to a record using the API: upon creation of a draft record and by providing so-called JSON patches in a request. This can also be done to already published records, but will create a new version of that record with new PIDs and checksums.
JSON patch requests modify the current state of the metadata by either changing, adding or removing fields and values. The structure of the data provided in the patch request must strictly follow the metadata schema of the community.
You can make as many patch requests as necessary before publishing your draft record or new published version.
Files can be added during the draft phase of your new record using a PUT request. Files are uploaded into the file bucket of the draft record, not the draft record itself. Therefore use the file bucket ID found in the metadata of the draft record. All added files have their checksum calculated.
Files are uploaded one-by-one in separate requests. You can upload as many files as necessary before publishing your draft record or published version.
Draft records are published by altering the value of the publication state in the metadata.
Once your record is published, the included files can not be changed anymore and no new files can be added! Metadata can be changed after publication, but will create a new version of your published record.
After your publication request, the persistent identifiers are automatically added to the record
Here are some example GET requests using the B2SHARE training instance and the curl application. No access token is necessary yet, as all this information is publically available.
Examples:
The retrieve all existing records and list all communities examples do not use a variable in the address
To search for specific records of a community, use the community ID in the query parameter q. Community IDs can be found in the listing of all communities
To generate an access token, go to the B2SHARE website, log in and navigate to the profile page.
Create new token by entering a name, followed by a click on the new token button. Note that you will not be able to retrieve this token, so store it safely.
If you loose your token you can create a new one on this page. You can create as many as you like.
Have a look at our website for more information regarding B2SHARE
User documentation is also available here
B2SHARE hands-on training can be found on GitHub. Currently only API access using Python is covered. In the future more modules will be added.