This document discusses stream reasoning and the key achievements in exploring continuous semantics for reasoning over data streams on the Semantic Web. It presents the motivation for stream reasoning to make sense of real-time data streams, and discusses challenges like query languages, reasoning, and dealing with incomplete data. The document outlines research that developed an architecture for a stream reasoner, the concept of RDF streams, and the Continuous SPARQL query language (C-SPARQL) to query RDF streams.
The talk about "Stream Reasoning" for INQUEST -- INnovative QUErying of STreams 2012 -- (http://games.cs.ox.ac.uk/inquest12/) organized in Oxford, United Kingdom, September 25-27 2012.
The talks presents a comprehensive view on "Stream Reasoning" -- reasoning on rapidly flowing information. It illustrates the challenges, presents the achievements of the database group of Politecnico di Milano on the topic, reviews the challenges pointing to results and ongoing work in the Semantic Web community and proposes how to go beyond the current Stream Reasoning concept. It particular, it points out that "orders matters" when processing massive data and it proposes to investigate streaming algorithms for automated reasoning that can be applied not only to data streams that are "naturally" ordered (by recency) but to any sortable data source.
Stream Reasoning: a summary of ten years of research and a vision for the nex...Emanuele Della Valle
Stream reasoning studies the application of inference techniques to data characterised by being highly dynamic. It can find application in several settings, from Smart Cities to Industry 4.0, from Internet of Things to Social Media analytics. This year stream reasoning turns ten, and this talk analyses its growth. In the first part, it traces the main results obtained so far, by presenting the most prominent studies. It starts by an overview of the most relevant studies developed in the context of semantic web, and then it extends the analysis to include contributions from adjacent areas, such as database and artificial intelligence. Looking at the past is useful to prepare for the future: the second part presents a set of open challenges and issues that stream reasoning will face in the next future.
From the semantic interoperability problem to Google's knowledge graph passing from the Semantic Web, Linked Data, Yahoo! search monkey, Facebook Open Graph, and schema.org.
It's a Streaming World! Reasoning upon Rapidly Changing Information (Milano, ...Emanuele Della Valle
Reasoning on rapidly chancing information requires: a) semantic models for representing both data streams and continuous querying/reasoning tasks, and b) reasoning algorithms optimised for continuous reactive query-answering. This talk presents applications cases from which Stream Reasoning requirements were elicited, it briefly covers the findings of 5 year of research, it presents an optimised algorithm for Incremental Reasoning on RDF Streams (IMaRS), and offers an outlook on future research opportunities.
Order Matters! Harnessing a World of Orderings for Reasoning over Massive DataEmanuele Della Valle
More and more applications require real-time processing of massive, dynamically generated, ordered data; order is an essential factor as it reflects recency or relevance. Semantic technologies risk being unable to meet the needs of such applications, as they are not equipped with the appropriate instruments for answering queries over massive, highly dynamic, ordered data sets. This talk argues that some order-aware data management techniques should be exported to the context of semantic technologies, by integrating ordering with reasoning, and by using methods which are inspired by stream and rank-aware data management. This talk systematically explores the problem space, and points both to problems which have been successfully approached and to problems which still need fundamental research, in an attempt to stimulate and guide a paradigm shift in semantic technologies.
The third lecture of the course I'm giving on "Interoperability and Semantic Technologies" at Politecnico di Milano in the academic year 2015-16. It presents an introduction to the Semantic Web taking a brief walk through in this 15 years of research, standardisation and industrial uptake.
The talk about "Stream Reasoning" for INQUEST -- INnovative QUErying of STreams 2012 -- (http://games.cs.ox.ac.uk/inquest12/) organized in Oxford, United Kingdom, September 25-27 2012.
The talks presents a comprehensive view on "Stream Reasoning" -- reasoning on rapidly flowing information. It illustrates the challenges, presents the achievements of the database group of Politecnico di Milano on the topic, reviews the challenges pointing to results and ongoing work in the Semantic Web community and proposes how to go beyond the current Stream Reasoning concept. It particular, it points out that "orders matters" when processing massive data and it proposes to investigate streaming algorithms for automated reasoning that can be applied not only to data streams that are "naturally" ordered (by recency) but to any sortable data source.
Stream Reasoning: a summary of ten years of research and a vision for the nex...Emanuele Della Valle
Stream reasoning studies the application of inference techniques to data characterised by being highly dynamic. It can find application in several settings, from Smart Cities to Industry 4.0, from Internet of Things to Social Media analytics. This year stream reasoning turns ten, and this talk analyses its growth. In the first part, it traces the main results obtained so far, by presenting the most prominent studies. It starts by an overview of the most relevant studies developed in the context of semantic web, and then it extends the analysis to include contributions from adjacent areas, such as database and artificial intelligence. Looking at the past is useful to prepare for the future: the second part presents a set of open challenges and issues that stream reasoning will face in the next future.
From the semantic interoperability problem to Google's knowledge graph passing from the Semantic Web, Linked Data, Yahoo! search monkey, Facebook Open Graph, and schema.org.
It's a Streaming World! Reasoning upon Rapidly Changing Information (Milano, ...Emanuele Della Valle
Reasoning on rapidly chancing information requires: a) semantic models for representing both data streams and continuous querying/reasoning tasks, and b) reasoning algorithms optimised for continuous reactive query-answering. This talk presents applications cases from which Stream Reasoning requirements were elicited, it briefly covers the findings of 5 year of research, it presents an optimised algorithm for Incremental Reasoning on RDF Streams (IMaRS), and offers an outlook on future research opportunities.
Order Matters! Harnessing a World of Orderings for Reasoning over Massive DataEmanuele Della Valle
More and more applications require real-time processing of massive, dynamically generated, ordered data; order is an essential factor as it reflects recency or relevance. Semantic technologies risk being unable to meet the needs of such applications, as they are not equipped with the appropriate instruments for answering queries over massive, highly dynamic, ordered data sets. This talk argues that some order-aware data management techniques should be exported to the context of semantic technologies, by integrating ordering with reasoning, and by using methods which are inspired by stream and rank-aware data management. This talk systematically explores the problem space, and points both to problems which have been successfully approached and to problems which still need fundamental research, in an attempt to stimulate and guide a paradigm shift in semantic technologies.
The third lecture of the course I'm giving on "Interoperability and Semantic Technologies" at Politecnico di Milano in the academic year 2015-16. It presents an introduction to the Semantic Web taking a brief walk through in this 15 years of research, standardisation and industrial uptake.
Wi2015 - Clustering of Linked Open Data - the LODeX toolLaura Po
Presentation of the tool LODeX (http://www.dbgroup.unimore.it/lodex2/testCluster) at the 2015 IEEE/WIC/ACM International Conference on Web Intelligence, Singapore, December 6-8, 2015
Talk at 3th Keystone Training School - Keyword Search in Big Linked Data - Institute for Software Technology and Interactive Systems, TU Wien, Austria, 2017
Knowledge graph embeddings are a mechanism that projects each entity in a knowledge graph to a point in a continuous vector space. It is commonly assumed that those approaches project two entities closely to each other if they are similar and/or related. In this talk, I give a closer look at the roles of similarity and relatedness with respect to knowledge graph embeddings, and discuss how the well-known embedding mechanism RDF2vec can be tailored towards focusing on similarity, relatedness, or both.
Slides of I2G (Poland) for their partner introduction as a new LOD2 partner in the course of the LOD2 project enlargement - presented at the LOD2 plenary meeting in Leuven, Belgium on September 2011
LDQL: A Query Language for the Web of Linked DataOlaf Hartig
I used this slideset to present our research paper at the 14th Int. Semantic Web Conference (ISWC 2015). Find a preprint of the paper here:
http://olafhartig.de/files/HartigPerez_ISWC2015_Preprint.pdf
Overview of the SPARQL-Generate language and latest developmentsMaxime Lefrançois
SPARQL-Generate is an extension of SPARQL 1.1 for querying not only RDF datasets but also documents in arbitrary formats. The solution bindings can then be used to output RDF (SPARQL-Generate) or text (SPARQL-Template)
Anyone familiar with SPARQL can easily learn SPARQL-Generate; Learning SPARQL-Generate helps you learning SPARQL.
The open-source implementation (Apache 2 license) is based on Apache Jena and can be used to execute transformations from a combination of RDF and any kind of documents in XML, JSON, CSV, HTML, GeoJSON, CBOR, streams of messages using WebSocket or MQTT... (easily extensible)
Recent extensions and improvement include:
- heavy refactoring to support parallelization
- more expressive iterators and functions
- simple generation of RDF lists
- support of aggregates
- generation of HDT (thanks Ana for the use case)
- partial implementation of STTL for the generation of Text (https://ns.inria.fr/sparql-template/)
- partial implementation of LDScript (http://ns.inria.fr/sparql-extension/)
- integration of all these types of rules to decouple or compose queries, e.g.:
- call a SPARQL-Generate query in the SPARQL FROM clause
- plug a SPARQL-Generate or a SPARQL-Template query to the output of a SPARQL-
Select function
- a Sublime Text package for local development
The presentation I gave at Semantic Days 2012 (https://www.posccaesar.org/wiki/PCA/SemanticDays2012) about Stream Reasoning. The main goal of the presentation is to give the most up to date comprehensive view on Stream Reasoning.
Wi2015 - Clustering of Linked Open Data - the LODeX toolLaura Po
Presentation of the tool LODeX (http://www.dbgroup.unimore.it/lodex2/testCluster) at the 2015 IEEE/WIC/ACM International Conference on Web Intelligence, Singapore, December 6-8, 2015
Talk at 3th Keystone Training School - Keyword Search in Big Linked Data - Institute for Software Technology and Interactive Systems, TU Wien, Austria, 2017
Knowledge graph embeddings are a mechanism that projects each entity in a knowledge graph to a point in a continuous vector space. It is commonly assumed that those approaches project two entities closely to each other if they are similar and/or related. In this talk, I give a closer look at the roles of similarity and relatedness with respect to knowledge graph embeddings, and discuss how the well-known embedding mechanism RDF2vec can be tailored towards focusing on similarity, relatedness, or both.
Slides of I2G (Poland) for their partner introduction as a new LOD2 partner in the course of the LOD2 project enlargement - presented at the LOD2 plenary meeting in Leuven, Belgium on September 2011
LDQL: A Query Language for the Web of Linked DataOlaf Hartig
I used this slideset to present our research paper at the 14th Int. Semantic Web Conference (ISWC 2015). Find a preprint of the paper here:
http://olafhartig.de/files/HartigPerez_ISWC2015_Preprint.pdf
Overview of the SPARQL-Generate language and latest developmentsMaxime Lefrançois
SPARQL-Generate is an extension of SPARQL 1.1 for querying not only RDF datasets but also documents in arbitrary formats. The solution bindings can then be used to output RDF (SPARQL-Generate) or text (SPARQL-Template)
Anyone familiar with SPARQL can easily learn SPARQL-Generate; Learning SPARQL-Generate helps you learning SPARQL.
The open-source implementation (Apache 2 license) is based on Apache Jena and can be used to execute transformations from a combination of RDF and any kind of documents in XML, JSON, CSV, HTML, GeoJSON, CBOR, streams of messages using WebSocket or MQTT... (easily extensible)
Recent extensions and improvement include:
- heavy refactoring to support parallelization
- more expressive iterators and functions
- simple generation of RDF lists
- support of aggregates
- generation of HDT (thanks Ana for the use case)
- partial implementation of STTL for the generation of Text (https://ns.inria.fr/sparql-template/)
- partial implementation of LDScript (http://ns.inria.fr/sparql-extension/)
- integration of all these types of rules to decouple or compose queries, e.g.:
- call a SPARQL-Generate query in the SPARQL FROM clause
- plug a SPARQL-Generate or a SPARQL-Template query to the output of a SPARQL-
Select function
- a Sublime Text package for local development
The presentation I gave at Semantic Days 2012 (https://www.posccaesar.org/wiki/PCA/SemanticDays2012) about Stream Reasoning. The main goal of the presentation is to give the most up to date comprehensive view on Stream Reasoning.
Reflections on Almost Two Decades of Research into Stream ProcessingKyumars Sheykh Esmaili
This is the slide deck that I used during my tutorial presentation at the ACM DEBS Conference (http://www.debs2017.org/) that was held in Barcelona between June 19 and June 23, 2017.
The tutorial paper itself can be accessed here: http://dl.acm.org/citation.cfm?id=3095110
Given at the annual Open Universiteit Informatics faculty research meeting on March 6, 2012. Video is at http://video.intranet.ou.nl/mediadienst/_website/php/external_video.php?Q=1056|videoID
Leveraging Wikipedia as a Hub for Data Integration: the Remixing Archival Metadata Project (RAMP)
Timothy A. Thompson, Metadata Librarian (Spanish/Portuguese Specialty), Princeton University Library
Towards efficient processing of RDF data streamsAlejandro Llaves
Presentation of short paper submitted to OrdRing workshop, held at ISWC 2014 - http://streamreasoning.org/events/ordring2014.
In the last years, there has been an increase in the amount of real-time data generated. Sensors attached to things are transforming how we interact with our environment. Extracting meaningful information from these streams of data is essential for some application areas and requires processing systems that scale to varying conditions in data sources, complex queries, and system failures. This paper describes ongoing research on the development of a scalable RDF streaming engine.
Towards efficient processing of RDF data streamsAlejandro Llaves
In the last years, there has been an increase in the amount of real-time data generated. Sensors attached to things are transforming how we interact with our environment. Extracting meaningful information from these streams of data is essential for some application areas and requires processing systems that scale to varying conditions in data sources, complex queries, and system failures. This paper describes ongoing research on the development of a scalable RDF streaming engine.
Presented at OrdRing workshop, International Semantic Web Conference 2014.
http://streamreasoning.org/events/ordring2014
Usage of Linked Data: Introduction and Application ScenariosEUCLID project
This presentation introduces the main principles of Linked Data, the underlying technologies and background standards. It provides basic knowledge for how data can be published over the Web, how it can be queried, and what are the possible use cases and benefits. As an example, we use the development of a music portal (based on the MusicBrainz dataset), which facilitates access to a wide range of information and multimedia resources relating to music.
Engaging Information Professionals in the Process of Authoritative Interlinki...Lucy McKenna
Through the use of Linked Data (LD), Libraries, Archives and Museums (LAMs) have the potential to expose their collections to a larger audience and to allow for more efficient user searches. Despite this, relatively few LAMs have invested in LD projects and the majority of these display limited interlinking across datasets and institutions. A survey was conducted to understand Information Professionals' (IPs') position with regards to LD, with a particular focus on the interlinking problem. The survey was completed by 185 librarians, archivists, metadata cataloguers and researchers. Results indicated that, when interlinking, IPs find the process of ontology and property selection to be particularly challenging, and LD tooling to be technologically complex and unsuitable for their needs.
Our research is focused on developing an authoritative interlinking framework for LAMs with a view to increasing IP engagement in the linking process. Our framework will provide a set of standards to facilitate IPs in the selection of link types, specifically when linking local resources to authorities. The framework will include guidelines for authority, ontology and property selection, and for adding provenance data. A user-interface will be developed which will direct IPs through the resource interlinking process as per our framework. Although there are existing tools in this domain, our framework differs in that it will be designed with the needs and expertise of IPs in mind. This will be achieved by involving IPs in the design and evaluation of the framework. A mock-up of the interface has already been tested and adjustments have been made based on results. We are currently working on developing a minimal viable product so as to allow for further testing of the framework. We will present our updated framework, interface, and proposed interlinking solutions.
This paper surveys the landscape of linked open data projects in cultural heritage, exam- ining the work of groups from around the world. Traditionally, linked open data has been ranked using the five star method proposed by Tim Berners-Lee. We found this ranking to be lacking when evaluating how cultural heritage groups not merely develop linked open datasets, but find ways to used linked data to augment user experience. Building on the five-star method, we developed a six-stage life cycle describing both dataset development and dataset usage. We use this framework to describe and evaluate fifteen linked open data projects in the realm of cultural heritage.
Similar to Stream Reasoning - where we got so far 2011.1.18 Oxford Key Note (20)
Data streams take many forms and their velocity is hard to tame. They can be myriads of tiny flows that you can collect to tame with Time-series Databases; continuous massive flows than you cannot stop to tame with Data Stream Management Systems; Continuous numerous flows that can turn into a torrent to tame with Event-based Systems; and myriads of continuous flows of any size and speed that form an immense delta to tame with Event-Driven Architectures. Enjoy this introductory talk!
This is the presentation that I did for PoliMI Data Scientists on Stream Reasoning, an approach to blend Artificial Intelligence and Stream Processing.
While the state of the art in Machine Learning offers practitioners effective tecniques to deal with static data sets, there are only accademic results tailored to data streams. In this presentation for the 4th Stream Reasoning workshop, I report on an effort of Alessio Bernardo (a student of mines) to set up a benchmark enviroment to (i) repeat academic results, (ii) perform studies on real data for confirming the academic results, and (iii) study the research problem of "incremental rebalancing learning on evolving data streams".
HiPPO and Flipism are no longer the only way to take decisions. In the Big Data / Data Science era one can dream of data-driven organization. If the data were "oil", Big Data technologies extract, transport, and store it, while Data Science methods provide the a way to "refine the crude oil". This presentation elaborates on the Ws (What, Why, When, Who and How) of Big Data and Data Science.
La Città dei Balocchi, con le sue luci, è un evento chiave nel panorama dell'offerta turistica Natalizia Lombarda. La presentazione riporta i risultati di un'analisi di chi è venuto e quando.
Realizzato da Fluxedo srl e Olivetti spa per il Consorzio Como Turistica, con la collaborazione di Politecnico di Milano, TIM e Comune di Como, nel contesto del progetto CrowdInsights finanziato da EIT Digital.
Stream reasoning: an approach to tame the velocity and variety dimensions of ...Emanuele Della Valle
Big Data tech can tame volume and velocity. Taming Variety in presence of volume and velocity is the real challenge. I’ve been working on taming variety and velocity simultaneously (Stream Reasoning) for 10 years, now. In this talk, I give you some examples of application domains where this is necessary. I explain where the Stream Reasoning community went so far in theory, applications and products. In particular I focus on my applications and my startup Fluxedo, which is offering real-time social media analytics across social networks. I conclude the talk discussing what comes next: 1) the need to focus on languages and abstractions able to easily capture user needs; 2) the need to find the sweet-spot between scalability and expressive semantics; 3) the need to used semantics to model more than the data access; and 4) the need to get over imperfect data. If you are exited, I did my job for today!
Every body talks about Big Data, but why? Do it create value? Do it enable some paradigmatic shifts in the way we work with data? This talk I did at ComoNext research and technological park cast some light on those questions.
Listening to the pulse of our cities with Stream Reasoning (and few more tech...Emanuele Della Valle
The digital reflection of our cities is sharpening and it is tracking their evolution with a decreasing delay. However, we risk that data piles up without easing decision making. This key note, which I gave at the 12th Semantic Web Summer School, presents how stream reasoning (an approach to tame simultaneously the variety and velocity dimensions of Big Data) and advance visual analytics can support decision makers and discusses the lesson learnt.
The forth lecture of the course I'm giving on "Interoperability and Semantic Technologies" at Politecnico di Milano in the academic year 2015-16. It presents an introduction to RDF. It starts presenting the data model. Then it presents the turtle serialization. It compares XML vs. RDF. Finally, it provides few informations about RDFa and Linked Data.
The second lecture of the course I'm giving on "Interoperability and Semantic Technologies" at Politecnico di Milano in the academic year 2015-16. It discusses interoperability using HL7 v2 and v3 as examples of syntactic and semantic interoperability, respectively.
Stream reasoning: mastering the velocity and the variety dimensions of Big Da...Emanuele Della Valle
More and more applications require real-time processing of heterogeneous data streams. In terms of the “Vs” of Big Data (volume, velocity, variety and veracity), they require addressing velocity and variety at the same time. Big Data solutions able to handle separately velocity and variety have been around for a while, but only Stream Reasoning approaches those two dimensions at once. Current results in the Stream Reasoning field are relevant for application areas that require to: handle massive datasets, process data streams on the fly, cope with heterogeneous incomplete and noisy data, provide reactive answers, support fine-grained information access, and integrate complex domain models. This talk starting from those requirements, frames the problem addressed by Stream Reasoning. It poses the research question and operationalise it with four simpler sub-questions. It describes how the database group of Politecnico di Milano positively answered those sub-questions in the last 7 years of research. It briefly surveys alternative approaches investigated by other research groups world wide and it elaborates on current limitations and open challenges.
The 10 minutes presentation I gave at my PhD defence on 21.9.2015 in Amsterdam. Prof. Frank van Harmelen was my promoter. Prof. Ian Horrocks, prof. Manfred Hauswirth, prof. Geert-Jan Houben, Peter Boncz and prof. Guus Schreiber were my opponents.
Listening to the pulse of our cities fusing Social Media Streams and Call Dat...Emanuele Della Valle
The digital reflection of our cities is sharpening and it is tracking their evolution with a decreasing delay. This happens thanks to the pervasive deployment of sensors, the wide adoption of smart phones, the usage of (location-based) social networks and the availability of datasets about urban environment. So while data becomes every day more abundant, decision makers face the challenge to increase their capability to create value out of the analysis of this data. This key note presents how advance visual analytics, ontology base data access and information flow processing methods can help in making sense of Social Media Streams and Call Data Records from Mobile Network Operators during city scale events. Real-world deployments demonstrate the ability of those methods to advance our ability to feel the pulse of our cities in order to deliver innovative services.
C’è un modo di raccontare un evento che passa attraverso la lettura dei flussi social che genera. Quella traccia digitale che ogni partecipante lascia sui social network quando condivide la sua partecipazione o la sua opinione. E’ possibile fondere e interpretare in tempo reale tali tracce utilizzando tecnologie d’analisi d’avanguardia e modelli avanzati di visualizzazione dei dati. Nel 2014 in collaborazione con StudioLabo e Telecom Italia, il Politecnico di Milano ha realizzato CitySensing, per mostrare l’impronta lasciata dal FuoriSalone sui social network. Focalizzando, in seguito, CitySensing sulle esigenze del gestore dell’evento, il Politecnico di Milano ha mostrato la potenzialità dell’approccio per il Festival della Comunicazione di Camogli e per il Festival delle Letterature di Pescara. La soluzione è ora offerta da Fluxedo.
C'è un modo di racocontare la città che passa attraverso la lettura dei flussi di dati che essa genera. Quelle tracce digitali che ciascuno di noi lascia ogni volta che compie un piccolo gesto quotidiano, come fare una telefonata o inviare un tweet.
In City Data Fusion, il Politecnico di Milano e Telecom Italia raccontano le città fondendo, interpretando e visualizzando i Big Data, ovvero quell'enorme e continuo flusso di tracce digitali che i loro abitanti e visitotori lasciano utilizzando il proprio smartphone o i servizi della città.
Questa presentazione vi introduce all'osservazione alcune città italiane in una prospettiva nuova.
Bi-later integration are a short term approach to business integration, but only standards provide a long term solution. Unfortunately, agreeing on standards is hard and takes time, thus translation between standards is unavoidable. Embracing change is the only way to benefit from short term translation while developing over time comprehensive standards. Semantic technologies are design with flexibility in mind and, therefore, they can help in developing more comprehensive standards and easier to maintain translations.
Big data: why, what, paradigm shifts enabled , tools and market landscapeEmanuele Della Valle
This presentation brings together many contents you may have seen before (reports by McKinsey, Gatner and IBM, and info-graphics by Intel and Go-Globe) are agglomerated in one comprehensive and up-to-date view of Big Data.
City Data Fusion and City Sensing presented at EIT ICT Labs for EXPO 2015Emanuele Della Valle
EIT ICT Labs wants be present at EXPO 2015. The City Data Fusion project proposes to install City Sensing in EXPO Gate to display the pulse of Milano during the EXPO. The idea of City Data Fusion and the installation of City Data Fusion for Milano Design Week 2014 is covered in the slides.
On the effectiveness of a Mobile Puzzle Game UI to Crowdsource Linked Data Ma...Emanuele Della Valle
Linked Data publishing on the Web is a stably growing phenomenon, but its effective usage depends on the ability of consumers to assess the trustworthiness and the relevance of the published data. Pure automatic techniques are often inadequate to this end. Crowdsourcing is often advocated as a valuable solution. In this presentation, we propose WikiFinder – a Games With A Purpose inspired by popular mobile puzzle games – and we report on its effectiveness in solving typical Linked Data Management tasks.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Stream Reasoning - where we got so far 2011.1.18 Oxford Key Note
1. Stream Reasoning
Where We Got So Far
Oxford - 2010.1.18
http://streamreasoning.org
Emanuele Della Valle
DEI - Politecnico di Milano
emanuele.dellavalle@polimi.it
http://emanueledellavalle.org
Joint work with:
Davide Francesco Barbieri, Daniele Braga, Stefano http://wiki.larkc.eu/UrbanComputing
• For more information visit Ceri, and Michael Grossniklaus
2. Agenda
• Motivation
• Running Example
• Background
• Concept
• Achievements
• Retrospective and Conclusions
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 2
3. Motivation
It s a streaming World! [IEEE-IS2009]
• Sensor networks, …
• traffic engineering, …
• social networking, …
• financial markets, …
• generate streams!
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 3
4. Running Example
Real-Time Streams on the Web
• Streams are appearing more and more often on the
Web in sites that distribute and present information in
real-time streams.
• Checkout http://activitystrea.ms/ for a standard API
• E.g.
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 4
5. Running Example
Examples of Questions Users are Asking
• Which topics have my close friends discussed in the
last hour?
• Which book is my friend likely to read next?
• What impact have I been creating with my tweets in
the last day?
• …
• <query> … <time dimension> ?
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 5
6. Motivation
Problem Statement
• Making sense
– in real time
– of gigantic and inevitably noisy data streams
– in order to support the decision process of
extremely large numbers of concurrent user
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 6
7. Background
What are data streams anyway?
• Formally:
– Data streams are unbounded sequences of time-
varying data elements
time
• Less formally:
– an (almost) continuous flow of information
– with the recent information being more relevant as it
describes the current state of a dynamic system
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 7
8. Background
Continuous Semantics
• Processing data streams in the space of
one-time semantics is difficult
because of the very nature of the underlying data
• Innovative* assumption: continuous semantics!
– streams can be consumed on the fly rather than being
stored forever and
– queries are registered and continuously produce
answers
* This innovation arose in DB community in 90s
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 8
9. Background
Stream Processing
• Continuous queries registered over streams that
are observed trough windows
window
input stream Registered
stream of answer
Con-nuous
Query
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 9
10. Background
Data Stream Management Systems (DSMS)
• Research Prototypes
– Amazon/Cougar (Cornell) – sensors
– Aurora (Brown/MIT) – sensor monitoring, dataflow
– Gigascope: AT&T Labs – Network Monitoring
– Hancock (AT&T) – Telecom streams
– Niagara (OGI/Wisconsin) – Internet DBs & XML
– OpenCQ (Georgia) – triggers, view maintenance
– Stream (Stanford) – general-purpose DSMS
– Stream Mill (UCLA) - power & extensibility
– Tapestry (Xerox) – publish/subscribe filtering
– Telegraph (Berkeley) – adaptive engine for sensors
– Tribeca (Bellcore) – network monitoring
• High-tech startups
– Streambase, Coral8, Apama, Truviso
• Major DBMS vendors are all adding stream extensions as well
– Oracle http://www.oracle.com/technology/products/dataint/htdocs/streams_fo.html
– DB2 http://www.eweek.com/c/a/Database/IBM-DB2-Turns-25-and-Prepares-for-New-Life/
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 10
11. Background
Can the Semantic Web process data stream?
• The Semantic Web, the Web of Data is doing fine
– RDF, RDF Schema, SPARQL, OWL, RIF
– well understood theory,
– rapid increase in scalability
• BUT it pretends that the world is static
or at best a low change rate
both in change-volume and change-frequency
– ontology versioning
– belief revision
– time stamps on named graphs
• It sticks to the traditional one-time semantics
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 11
12. Concept
Stream Reasoning [IEEE-IS2010]
• Idea origination
– Can continuous semantics be ported to reasoning?
– This is an unexplored yet high impact research area!
• Stream Reasoning
– Logical reasoning in real time on gigantic and
inevitably noisy data streams in order to support
the decision process of extremely large numbers
of concurrent users.
-- S. Ceri, E. Della Valle, F. van Harmelen and H. Stuckenschmidt, 2010
• Note: making sense of streams necessarily requires
processing them against rich background knowledge
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 12
13. Concept
Research Challenges
• Relation with data-stream systems
– Just as RDF relates to data-base systems?
• Query languages for semantic streams
– Just as SPARQL for RDF but with continuous semantics?
• Reasoning on Streams
– Formal representations for stream reasoning
– Notions of soundness and completeness
– Efficiency
– Scalability
• Dealing with incomplete & noisy data
– Even more so than on the current Web of Data
• Distributed and parallel processing
– Streams are parallel in nature
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 13
14. Achievements
Explored Continuous Semantics for SeWeb
• We investigated
– Architecture of a Stream Reasoner
– RDF streams
• the natural extension of the RDF data model to the new
continuous scenario and
– Continuous SPARQL (or simply C-SPARQL)
• the extension of SPARQL for querying RDF streams.
– Efficient incremental updates of deductive
closures
• specifically considering the nature of data streams
– Effective inductive stream reasoning (joint work
with Siemens - Munich)
• See paper in IEEE IS special issue on Social Media
Analytics
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 14
15. Achievements
Architecture (IEEE-IS2010)
Social
Media
Analytics
Selector Abstracter Deductive C
Window DSMS
. DSMS Reasoner
C C
Abstracter Inductive
Legend Long-‐Term P
data
stream C C-‐SPARQL
query Matrix Reasoner
RDF
stream P SPARQL
with Probability
Abstracter Inductive
RDF
graph Hype P
Matrix Reasoner
• Based on the LarKC conceptual framework
http://www.larkc.eu
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 15
16. Achievements
RDF Stream [WWW2009,EDBT2010,IJSC2010]
• RDF Stream Data Type
– Ordered sequence of pairs, where each pair is made
of an RDF triple and its timestamp t
(< triple >, t)
• E.g.,
(<:Giulia :likes :Twilight >, 2010-02-12T13:34:41)
(<:John :likes :TheLordOfTheRings >, 2010-02-12T13:36:28)
(<:Alice :dislikes :Twilight >, 2010-02-12T13:36:28)
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 16
17. Achievements
C-SPARQL [WWW2009,EDBT2010,IJSC2010]
• We specificied of C-SPARQL syntax
– Incrementally, from existing specifications
• Including windows, grouping, aggregates, timestamping
• We gave the formal semantics of C-SPARQL
– Query registration, handling overloads
– Order of evaluation, pattern matching over time, …
• We investigated efficiency of evaluation
– Defining a suitable algebra
– Applying optimizations
– Efficient materialization of inferred data from streams
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 17
18. Achievements
An Example of C-SPARQL Query
Who are the opinion makers? i.e., the users who are likely to influence
the behavior of other users who follow them
REGISTER STREAM OpinionMakers COMPUTED EVERY 5m AS
CONSTRUCT { ?opinionMaker sd:about ?resource }
FROM STREAM <http://streamingsocialdata.org/interactions>
[RANGE 30m STEP 5m]
WHERE {
?opinionMaker ?opinion ?resource .
?follower sioc:follows ?opinionMaker.
?follower ?opinion ?resource.
FILTER ( cs:timestamp(?follower) >
cs:timestamp(?opinionMaker)
&& ?opinion != sd:accesses )
}
HAVING ( COUNT(DISTINCT ?follower) > 3 )
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 18
19. Achievements
An Example of C-SPARQL Query
Who are the opinion makers? i.e., the users who are likely to influence
Query registration RDF Stream added as
the (for continuous execution) who follow them
behavior of other users new ouput format
REGISTER STREAM OpinionMakers COMPUTED EVERY 5m AS
CONSTRUCT { ?opinionMaker sd:about ?resource }
FROM STREAM <http://streamingsocialdata.org/interactions>
[RANGE 30m STEP 5m] FROM STREAM clause
WHERE {
?opinionMaker ?opinion ?resource . WINDOW
?follower sioc:follows ?opinionMaker. Builtin to
?follower ?opinion ?resource. access
timestamps
FILTER ( cs:timestamp(?follower) >
cs:timestamp(?opinionMaker)
&& ?opinion != sd:accesses ) Aggregates as
in SPARQL 1.1
}
HAVING ( COUNT(DISTINCT ?follower) > 3 )
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 19
20. Achievements
Efficiency of Evaluation 1/3 [IEEE-IS2010]
• Evaluation of Window-based Selection
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 20
21. Achievements
Efficiency of Evaluation 2/3 [EDBT2010]
• Several transformations can be applied to algebraic
representation of C-SPARQL
• some recalling well known results from classical
relational optimization
– push of FILTERs and projections
• some being more specific to the domain of streams.
– push of aggregates.
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 21
22. Achievements
Efficiency of Evaluation 3/3 [EDBT2010]
• Push of filters and projections
125
100
75
ms
50
25
0
10 100 1000 10000 100000
Window Size
None Static Only Streaming Only Both
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 22
23. Achievements
Example of C-SPARQL and Reasoning 1/2
What impact have I been creating with my tweets in the last hour?
Is it positive or negative? Let’s count them …
REGISTER QUERY CountPositiveAndNegativeReactions AS
PREFIX : <http://ex.org/twitterImpactMining#>
SELECT ?t count(?pos) count(?neg)
FROM STREAM <http://ex.org/discussions.trdf>
[RANGE 30m STEP 30s] :discuss a owl:TransitiveProperty .
WHERE { :reply rdfs:subPropertyOf :discuss .
?t a :MonitoredTweet . :retweet rdfs:subPropertyOf :discuss .
{ ?pos :discuss ?t ;
:ProduceReaction [ a :PositiveReaction ] .
} UNION {
?neg :discuss ?t ;
:ProduceReaction [ a :NegativeReaction ] .
}
} GROUP BY ?t
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 23
24. Achievements
Example of C-SPARQL and Reasoning 2/2
discuss
discuss
retweet
reply
retweet
t1
t1-‐1
t1-‐2
t1-‐3
discuss
discuss
discuss
discuss
Monitored
Posi.ve
Nega.ve
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 24
25. Achievements
State-of-the-Art Approach [Ceri1994,Volz2005]
1. Overestimation of deletion: Overestimates deletions
by computing all direct consequences of a deletion.
2. Rederivation: Prunes those estimated deletions for
which alternative derivations (via some other facts
in the program) exist.
3. Insertion: Adds the new derivations that are
consequences of insertions to extensional
predicates.
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 25
26. Achievements
our approach [ESWC2010] 1/2
• Assuption
– Insertions and deletions are triples respectively
entering and exiting the window
– The window size is known
• Therefore
– The time when each triple will expire is known and
determined by the window size
• E.g. if the window is 10s long a triple entering at time t will
exit at time t+10s
– Note: all knowledge can be annotated with an
expiration time
• i.e., background knowledge is annotated with +∞
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 26
27. Achievements
our approach [ESWC2010] 2/2
• The algorithm
1. deletes all triples (asserted or inferred) that have just
expired
2. computes the entailments derived by the inserts,
3. annotates each entailed triple with a expiration time,
and
4. eliminates from the current state all copies of derived
triples except the one with the highest timestamp.
• learn more
– http://www.slideshare.net/emanueledellavalle/incremental-
reasoning-on-streams-andrich-background-knowledge
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 27
28. Achievements
Comparative Evaluation 1/2 [ESWC2010]
• Hypothesis
– Background knowledge do not change and it is fully materialized
– Changes only take place in the window
• An experiment comparing the time required to compute a new
materialization using
– Re-computing from scratch (i.e.,1250 ms in our setting)
– State of the art incremental approach [Volz, 2005]
– Our approach
• Results at increasing % of the materialization changed when
the window slides
10000
1000
ms.
100
10
0,0% 2,0% 4,0% 6,0% 8,0% 10,0% 12,0% 14,0% 16,0% 18,0% 20,0%
• . %
of
t he
m aterialization
changed
when
t he
window
slides
incremental-‐volz incremental-‐stream
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 28
29. Achievements
Comparative Evaluation 2/2
• Comparison of the average time needed to answer a
C-SPARQL query using
– a forward reasoner,
– the naive approach of re-computing the materialization
– our approach
20
15
10
ms.
5
0
forward
reasoning naive
approach incremental-‐stream
query 5,82 1,61 1,61
materialization 0 15,91 0,28
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 29
30. Retrospective and Conclusions
Wrap Up
• RDF Streams
– Notion defined
• C-SPARQL
– Syntax and semantics defined as a SPARQL extension
– Engine designed
– Engine implemented based on the decision to keep stream
management and query evaluation separated
• Experiments with C-SPARQL under simple RDF entailment
regimes
– window based selection of C-SPARQL outperforms the standard
FILTER based selection
– having formally defined C-SPARQL semantics algebraic
optimizations are possible
• Experiment with C-SPARQL under OWL-RL entailment
regimes
– efficient incremental updates of deductive closures investigated
– our approach outperform state-of-the-art when updates comes as
stream
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 30
31. Retrospective and Conclusions
Achievements vs. Research Challenges
• Relation with data-stream systems
– Notion of RDF stream :-|
• Query languages for semantic streams
– C-SPARQL :-D
• Reasoning on Streams
– Formal representations for stream reasoning
• :-P
– Notions of soundness and completeness
• :-P
– Efficient incremental updates of deductive closures
• ESWC 2010 paper :-) ... but much more work is needed!
– How to combine streams and background knowledge
• ESWC 2010 paper :-| ... but a lot needs to be studied ...
• Dealing with incomplete & noisy data
– :-P
• Distributed and parallel processing
– :-P
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 31
32. References
• Vision
[IEEE-IS2009] Emanuele Della Valle, Stefano Ceri, Frank van Harmelen, Dieter Fensel
It's a Streaming World! Reasoning upon Rapidly Changing Information. IEEE Intelligent
Systems 24(6): 83-89 (2009)
• Continuous SPARQL (C-SPARQL)
[EDBT2010] Davide Francesco Barbieri, Daniele Braga, Stefano Ceri and Michael
Grossniklaus. An Execution Environment for C-SPARQL Queries. EDBT 2010
[WWW2009] Davide Francesco Barbieri, Daniele Braga, Stefano Ceri, Emanuele Della Valle,
Michael Grossniklaus: C-SPARQL: SPARQL for continuous querying. WWW 2009:
1061-1062
[IJSC2010] Davide Francesco Barbieri, Daniele Braga, Stefano Ceri, Emanuele Della Valle,
Michael Grossniklaus: C-SPARQL: a Continuous Query Language for RDF Data Streams.
Int. J. Semantic Computing 4(1): 3-25 (2010)
[IEEE-IS2010] Davide Barbieri, Daniele Braga, Stefano Ceri, Emanuele Della Valle, Yi Huang,
Volker Tresp, Achim Rettinger, Hendrik Wermser, "Deductive and Inductive Stream
Reasoning for Semantic Social Media Analytics," IEEE Intelligent Systems, 30 Aug. 2010.
• Stream Reasoning
[ESWC2010] Davide Francesco Barbieri, Daniele Braga, Stefano Ceri, Emanuele Della Valle,
Michael Grossniklaus. Incremental Reasoning on Streams and Rich Background
Knowledge. In. 7th Extended Semantic Web Conference (ESWC 2010)
• Background work
[Ceri1994] Stefano Ceri, Jennifer Widom: Deriving Incremental Production Rules for Deductive
Data. Inf. Syst. 19(6): 467-490 (1994)
[Volz2005] Raphael Volz, Steffen Staab, Boris Motik: Incrementally Maintaining
Materializations of Ontologies Stored in Logic Databases. J. Data Semantics 2: 1-34 (2005)
Oxford, 2011-1-18 Emanuele Della Valle - visit http://streamreasoning.org 32
33. Thank You! Questions?
Much More to Come!
Keep an eye on
http://www.streamreasoning.org
Oxford, 2011-1-18 For more information visit http://www.larkc.eu/ 33