1. Why Event streaming
2. What is Event streaming
3. What is Transmitting Event Streams
4. What are message brokers
5. Types of message brokers available in market
6. Databases and streams
7. Event Sourcing
8.Command vs Events
9.State, Streams and Immutability
10. Immutable Events
11. Stream Events
12. Stream Joins
13. Demo on Azure Event Hub
14. Visualize data anomalies in real-time events sent to Azure Event Hubs
Discuss four pillars of azure architecture - Security, Performance & scalability, Availability & recoverability and Efficiency & Operation. Things you need to consider before architecting in the cloud. This presentation also provides a framework for architectural decisions
Tips & Tricks to build software architecture documentKarthikeyan VK
This slide shows the step by step process on how to build an architecture document that addresses all stakeholders concerns using viewpoints and perspectives.
In the presentation, we talk about 7 design patterns and 5 antipatterns. It also talks about why you should use design patterns for better architecture. Principle based desing is also discussed. To see the video, visit https://youtu.be/h-_Ns6nmWKw
This presentation targets to guiding security expert and developer to protect PaaS deployment to eliminate security threats. This also introduces Threat Modeling.
Building a document e-signing workflow with Azure Durable FunctionsJoonas Westlin
Durable functions offer an interesting programming model for building workflows. Whether you need to sometimes split and do multiple things or wait for user input, a lot of things are possible. They do present some challenges as well, and the limitations of orchestrator functions can make working with Durable seem very complicated.
In this talk we will go through the basics of Durable Functions along with strategies for deploying and monitoring them. A sample application will be presented where users can send documents for electronic signature. A Durable Functions workflow will power the signing process.
Application development and deployment in the traditional datacenter has been a challenge for many organizations primarily due to resource constraints. This has historically led to unfortunate compromises between functionality and security for business applications.
With public cloud providers, we have seen the limitations to technical capabilities fall away; the attainable to the Fortune 500 has become available to organizations of any size.
This yields some exciting new options for the development, deployment and operation of secure applications. Here you will find the presentation deck and recording of webinar.
Discuss four pillars of azure architecture - Security, Performance & scalability, Availability & recoverability and Efficiency & Operation. Things you need to consider before architecting in the cloud. This presentation also provides a framework for architectural decisions
Tips & Tricks to build software architecture documentKarthikeyan VK
This slide shows the step by step process on how to build an architecture document that addresses all stakeholders concerns using viewpoints and perspectives.
In the presentation, we talk about 7 design patterns and 5 antipatterns. It also talks about why you should use design patterns for better architecture. Principle based desing is also discussed. To see the video, visit https://youtu.be/h-_Ns6nmWKw
This presentation targets to guiding security expert and developer to protect PaaS deployment to eliminate security threats. This also introduces Threat Modeling.
Building a document e-signing workflow with Azure Durable FunctionsJoonas Westlin
Durable functions offer an interesting programming model for building workflows. Whether you need to sometimes split and do multiple things or wait for user input, a lot of things are possible. They do present some challenges as well, and the limitations of orchestrator functions can make working with Durable seem very complicated.
In this talk we will go through the basics of Durable Functions along with strategies for deploying and monitoring them. A sample application will be presented where users can send documents for electronic signature. A Durable Functions workflow will power the signing process.
Application development and deployment in the traditional datacenter has been a challenge for many organizations primarily due to resource constraints. This has historically led to unfortunate compromises between functionality and security for business applications.
With public cloud providers, we have seen the limitations to technical capabilities fall away; the attainable to the Fortune 500 has become available to organizations of any size.
This yields some exciting new options for the development, deployment and operation of secure applications. Here you will find the presentation deck and recording of webinar.
Blockchain is one of the most popular talking points in the IT industry today, with thousands of technical proofs, use-cases and experiments being planned and delivered to show the value of this technology. For all of this, when asked about production deployments of blockchain for the real enterprises of today, most of the proponents of this value will demur when it comes to talking production. Bridging this gap between science experiment and an enterprise quality platform for the modern business requires a great deal of work, much of which is done by the growing community of vendors who are collaborating to deliver not just the technical requirement and use-case solutions for blockchain, but also the practical tools to use blockchain in a meaningful way within a business. join this session to hear about the 5 key qualities that any practically usable enterprise blockchain will need to deliver and how the collaboration of enterprise vendors is making the practical use of Blockchain a reality for organisations in 2018.
Azure Operational Insights is a Cloud based machine data collection, storage, analysis service fully managed and serviced by microsoft. Operational Insights uses data from servers in your on-premise or cloud infrastructure. You can collect machine data from the following sources:
Operations Manager agents
Computers connected directly to Operational Insights
Virtual machine diagnostic data in Azure Storage services
After data is collected, it is sent to the Operational Insights service.
Microsoft Azure is an ever-expanding set of cloud services to help your organization meet your business challenges. It’s the freedom to build, manage, and deploy applications on a massive, global network using your favorite tools and frameworks.
Productive
Reduce time to market, by delivering features faster with over 100 end-to-end services.
Hybrid
Develop and deploy where you want, with the only consistent hybrid cloud on the market. Extend Azure on-premises with Azure Stack.
Intelligent
Create intelligent apps using powerful data and artificial intelligence services.
Trusted
Join startups, governments, and 90 percent of Fortune 500 businesses who run on the Microsoft Cloud today.
I presented this at a user group in Sweden, as a compilation discussion of practical customer experiences with WIndows Azure. The slides led the discussion. Enjoy.
Security Architecture Best Practices for SaaS ApplicationsTechcello
Gartner has predicted 18-20% growth in SaaS market, and expects it to hit US $22.1 billion by the year 2015. They have also measured that SaaS adoption rate has increased many fold in the last few years (almost 71% of enterprises use SaaS solutions).
Enterprise-class security with PostgreSQL - 2Ashnikbiz
For businesses that handle personal data everyday, the security aspect of their database is of utmost importance.
With an increasing number of hack attacks and frauds, organizations want their open source databases to be fully equipped with the top security features.
Enterprise-class security with PostgreSQL - 1Ashnikbiz
For businesses that handle personal data everyday, the security aspect of their database is of utmost importance.
With an increasing number of hack attacks and frauds, organizations want their open source databases to be fully equipped with the top security features.
Modern Data Security for the Enterprises – SQL Server & Azure SQL DatabaseWinWire Technologies Inc
The webinar talked about the layers of data protection, important security features, potential scenarios in which these features can be applied to limit exposure to security threats and best practices for securing business applications and data. We covered following topics on SQL Server 2016 and Azure SQL Database security features
• Access Level Control
• Data Encryption
• Monitoring
In this presentation, I will explain event driven architecture, describe the different types of events, demonstrate how events can be related and orchestrated, and provide a basic understanding of how this method can drive the architecture of enterprise systems. In addition to understanding the concepts of event driven architecture, we will explore a working sample built using an open-source .NET messaging framework called MassTransit.
This presentation walks through the Security and Compliance functionality to customers leveraging Azure as a compute environment. It includes deep-dive references to detailed information on each topic presented.
Search for all with Elastic Enterprise Search Elasticsearch
We reimagined search in the workplace so you can get to the information you need quickly with a unified search experience, out-of-the-box data connectors, and simple search management interfaces. Meet Elastic Enterprise Search.
See the video: https://www.elastic.co/elasticon/tour/2019/washington-dc/search-for-all-with-elastic-enterprise-search
Maintaining consistency in a distributed system is hard. You face a trade-off between consistency and availability, between tight coupling and loose coupling. Events complement commands and queries in microservices to foster loose-coupling and evolvability.
Blockchain is one of the most popular talking points in the IT industry today, with thousands of technical proofs, use-cases and experiments being planned and delivered to show the value of this technology. For all of this, when asked about production deployments of blockchain for the real enterprises of today, most of the proponents of this value will demur when it comes to talking production. Bridging this gap between science experiment and an enterprise quality platform for the modern business requires a great deal of work, much of which is done by the growing community of vendors who are collaborating to deliver not just the technical requirement and use-case solutions for blockchain, but also the practical tools to use blockchain in a meaningful way within a business. join this session to hear about the 5 key qualities that any practically usable enterprise blockchain will need to deliver and how the collaboration of enterprise vendors is making the practical use of Blockchain a reality for organisations in 2018.
Azure Operational Insights is a Cloud based machine data collection, storage, analysis service fully managed and serviced by microsoft. Operational Insights uses data from servers in your on-premise or cloud infrastructure. You can collect machine data from the following sources:
Operations Manager agents
Computers connected directly to Operational Insights
Virtual machine diagnostic data in Azure Storage services
After data is collected, it is sent to the Operational Insights service.
Microsoft Azure is an ever-expanding set of cloud services to help your organization meet your business challenges. It’s the freedom to build, manage, and deploy applications on a massive, global network using your favorite tools and frameworks.
Productive
Reduce time to market, by delivering features faster with over 100 end-to-end services.
Hybrid
Develop and deploy where you want, with the only consistent hybrid cloud on the market. Extend Azure on-premises with Azure Stack.
Intelligent
Create intelligent apps using powerful data and artificial intelligence services.
Trusted
Join startups, governments, and 90 percent of Fortune 500 businesses who run on the Microsoft Cloud today.
I presented this at a user group in Sweden, as a compilation discussion of practical customer experiences with WIndows Azure. The slides led the discussion. Enjoy.
Security Architecture Best Practices for SaaS ApplicationsTechcello
Gartner has predicted 18-20% growth in SaaS market, and expects it to hit US $22.1 billion by the year 2015. They have also measured that SaaS adoption rate has increased many fold in the last few years (almost 71% of enterprises use SaaS solutions).
Enterprise-class security with PostgreSQL - 2Ashnikbiz
For businesses that handle personal data everyday, the security aspect of their database is of utmost importance.
With an increasing number of hack attacks and frauds, organizations want their open source databases to be fully equipped with the top security features.
Enterprise-class security with PostgreSQL - 1Ashnikbiz
For businesses that handle personal data everyday, the security aspect of their database is of utmost importance.
With an increasing number of hack attacks and frauds, organizations want their open source databases to be fully equipped with the top security features.
Modern Data Security for the Enterprises – SQL Server & Azure SQL DatabaseWinWire Technologies Inc
The webinar talked about the layers of data protection, important security features, potential scenarios in which these features can be applied to limit exposure to security threats and best practices for securing business applications and data. We covered following topics on SQL Server 2016 and Azure SQL Database security features
• Access Level Control
• Data Encryption
• Monitoring
In this presentation, I will explain event driven architecture, describe the different types of events, demonstrate how events can be related and orchestrated, and provide a basic understanding of how this method can drive the architecture of enterprise systems. In addition to understanding the concepts of event driven architecture, we will explore a working sample built using an open-source .NET messaging framework called MassTransit.
This presentation walks through the Security and Compliance functionality to customers leveraging Azure as a compute environment. It includes deep-dive references to detailed information on each topic presented.
Search for all with Elastic Enterprise Search Elasticsearch
We reimagined search in the workplace so you can get to the information you need quickly with a unified search experience, out-of-the-box data connectors, and simple search management interfaces. Meet Elastic Enterprise Search.
See the video: https://www.elastic.co/elasticon/tour/2019/washington-dc/search-for-all-with-elastic-enterprise-search
Maintaining consistency in a distributed system is hard. You face a trade-off between consistency and availability, between tight coupling and loose coupling. Events complement commands and queries in microservices to foster loose-coupling and evolvability.
In many database applications we first log data and then, a few hours or days later, we start analyzing it. But in a world that’s moving faster and faster, we sometimes need to analyze what is happening NOW.
Azure Stream Analytics allows you to analyze streams of data via a new Azure service. In this session you will see how to get started using this new service. From event hubs on the input side over temporal SQL queries: the demo’s in this session will show you end to end how to get started with Azure Stream Analytics.
An ode to the underrepresented and underused pattern of events and asynchrony in the design and development of Microservices.
Prepared by Saul Caganoff, and delivered by Saul at Melbourne Microservices, and by Yamen Sader at Sydney Microservices.
Event Sourcing, Stream Processing and Serverless (Benjamin Stopford, Confluen...confluent
In this talk we’ll look at the relationship between three of the most disruptive software engineering paradigms: event sourcing, stream processing and serverless. We’ll debunk some of the myths around event sourcing. We’ll look at the inevitability of event-driven programming in the serverless space and we’ll see how stream processing links these two concepts together with a single ‘database for events’. As the story unfolds we’ll dive into some use cases, examine the practicalities of each approach-particularly the stateful elements-and finally extrapolate how their future relationship is likely to unfold. Key takeaways include: The different flavors of event sourcing and where their value lies. The difference between stream processing at application- and infrastructure-levels. The relationship between stream processors and serverless functions. The practical limits of storing data in Kafka and stream processors like KSQL."
In the Internet of things, data and commands between things and servers are sent as streams of events, which are often aggregated and processed to provide up to date information to end users. Because of this, CQRS and Event Sourcing patterns are a natural fit for IoT applications. In this presentation we provide an overview of these patterns, how they apply to IoT applications and their benefits. A prototype application of Event Sourcing is then demonstrated using the Sense Tecnic FRED platform based on Node-RED - a data flow programming tool for wiring up the internet of things
This webinar by Orkhan Gasimov (Senior Solution Architect, Consultant, GlobalLogic) was delivered at Java Community Webinar #3 on October 16, 2020.
During webinar we had simplified overview of classical and modern architecture patterns and concepts that are used for development of distributed applications during the last decade.
More details and presentation: https://www.globallogic.com/ua/about/events/java-community-webinar-3/
Assessing New Databases– Translytical Use CasesDATAVERSITY
Organizations run their day-in-and-day-out businesses with transactional applications and databases. On the other hand, organizations glean insights and make critical decisions using analytical databases and business intelligence tools.
The transactional workloads are relegated to database engines designed and tuned for transactional high throughput. Meanwhile, the big data generated by all the transactions require analytics platforms to load, store, and analyze volumes of data at high speed, providing timely insights to businesses.
Thus, in conventional information architectures, this requires two different database architectures and platforms: online transactional processing (OLTP) platforms to handle transactional workloads and online analytical processing (OLAP) engines to perform analytics and reporting.
Today, a particular focus and interest of operational analytics includes streaming data ingest and analysis in real time. Some refer to operational analytics as hybrid transaction/analytical processing (HTAP), translytical, or hybrid operational analytic processing (HOAP). We’ll address if this model is a way to create efficiencies in our environments.
Billions of Rows, Millions of Insights, Right NowRob Winters
Presentation from Tableau Customer Conference 2013 on building a real time reporting/analytics platform. Topics discussed include definitions of big data and real time, technology choices and rationale, use cases for real time big data, architecture, and pitfalls to avoid.
A talk through the journey we've been through at Snowplow thinking about event data, starting with our focus on web and then mobile analytics, and exploring our current and future technical and analytic approaches
Similar to Event Streaming Architecture - Deep Dive (20)
AutoGPT is a new AI tool that can automate many of the mundane tasks that take up your time. With AutoGPT, you can focus on the creative and strategic aspects of your work, while the AI takes care of the repetitive and time-consuming tasks.
In this talk, we will discuss how AutoGPT can be used to improve your productivity. We will cover a variety of topics, including:
How to use AutoGPT to automate your tasks
How to integrate AutoGPT into your workflow
How to troubleshoot common problems with AutoGPT
Discover the fascinating world of data science with this beginner-friendly introduction to the field. This presentation covers key concepts and skills, making it perfect for those who are new to data science or looking to refresh their knowledge. You'll learn about the role of a data scientist, the data science process, and popular tools and techniques used in the industry.
Content Overview:
Introduction to Data Science
Key Components and Terminology
Data Types and Sources
Data Cleaning and Preprocessing
Exploratory Data Analysis
Feature Engineering and Selection
Machine Learning Algorithms
Model Evaluation and Validation
Data Visualization Techniques
Essential Tools and Libraries
Real-World Applications and Use Cases
Building a Data Science Career
Tags: #DataScience #MachineLearning #BigData #Analytics #DataVisualization #AI #Python #DataEngineering #Statistics
1. Mindset of an Architect
2. Character of an Architect
3. Habits of an Architect
4. Leader in an Architect
5. Skillset of an Architect
We will also address the following burning questions in every senior software developer.
How to become highly productive.
How to see the big picture when architecting solutions
How to keep me updated on these changing technologies
How to cope with stress in the fast-paced world
How to be successful in spite of heavy competition
Five simple tools that make you ultra-productive.
How to Lead a team without being a manager
How to increase your salary
How to balance your professional and personal life
How not to feel insecure in your office
How to overcome the feeling of stagnation in your career
How to motivate your team without any fake pep talks
How to build a self-motivated team
How to learn core concepts faster
How to implement what you learned in your project
How to overcome procrastination
How to overcome the fear of becoming invalid in your company
How to ignore politics and still climb the corporate ladder.
How to express your ideas to upper management
How to build a high-quality team
How to position yourself as an architect, so you can get opportunities automatically.
How to be an ethical leader
How to sell yourself without coming out as a salesperson
How to motivate your team to new technologies, even though your project does not facilitate new technologies.
How to build your own digital presence.
How to become a demon developer.
How to write simple architectural documents that can be understood by all stakeholders - a pragmatic approach.
5 Simple tools that make you more productive
How to adapt to the new environment and become an architect who has the most influence.
How to build a new team from the ground up.
Blockchain is one of the trending technology that is catching up in the tech space. It solves multiple use cases where the data integrity should be high. In this ppt, the author has worked on explaining the concepts in a simple and elegant way
Learn 14 Antipatterns. Three types of antipatterns. Traps that are applicable to common solutions. Development, Architecture, and Project management pattern. Feud,Golder ,
How to double your productivity as a developerKarthikeyan VK
Learn how to become more productive as a developer. In this presentation, you will learn what you should do, not do and how to change yourself to become more productive
Understanding LUIS and how you can build your own Support team with minimal effort. Learn all the important concepts around QNA Maker and LUIS such as Entity, Intent, and utterance.
Use azure search to build enterprise level search feature into your application. Reference contains link to angular application. Why Azure search, What is azure search are all explained. Cognitive uses are also included
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
3. PROBLEMS WITH CURRENT ANALYTICS
Not Realtime
ETL was created during a
period of monolithic
architectures, data warehouses
and relational databases.
4. PROBLEMS WITH CURRENT MICROSERVICES
ARCHITECTURE
• Client more details about our
service
• Too many details to be
exposed
• Chatty communication
5. PROBLEMS WITH CURRENT MICROSERVICES
ARCHITECTURE
• Gateway can take care of
authentication.
• Rate limiting possible
• Things seems ok.
8. CHANGES ARE STILL HARD
• Adding a new service needs to
communicated to other services of
its presence.
• Chatty communication between
services
• REST+JSON is SLOW
• GRPC better, but not a complete
solution
• Adding services adds new changes
to the other services.
12. ONLINE SHOP
1. No Knowledge of other services
2. Communication at services will.
3. Rest calls reduced.
4. Broadcast your events and can be
ignored by other services or
consumed.
13. ONLINE SHOPPING USING R.E.S.T
1.Order should know about
shipment services
2.Shipment should get details
of the customer from the
customer service
3.What if there is customer
details are changed ?
15. PRODUCT ORDERED - FACT
1.Order submitted
2.Event sent to Kafka
3.Shipment reacts to the order
being placed
4.No REST communication
between order and shipment
5.Product updates itself based
on process result
16. ADDRESS CHANGED - TRIGGER
1.Customer changes address
2.Event sent to Kafka
3.Shipment reacts to changes
4.Order does not have to care
about any changes once it is
placed.
5.Shipment updates its database
6.Other services will not react if it
is not pertained to them,
IGNORE
20. BASICS OF EVENT STREAMING
• Databases vs streams
• Command vs Events
• Immutable Events
• State, Streams and Immutability
• Stream Joins
21. DATABASE VS STREAMS
• Concept wise it is same
• Every action takes places is stored as logs
• Restore happens from reading the logs again and creating
the database
22. COMMAND VS EVENTS VS IMMUTABLE
EVENTS
• Any request is a command.
• After integrity check it becomes an Event.
• Booking an air ticket is command, if there is ticket
available, it becomes an event. It also becomes a fact.
• When the ticket is cancelled, it is still an event, that is
immutable and recorded in our system.
23. STATE, STREAMS AND IMMUTABILITY
• When the ticket is booked, the State is saved.
• When the ticket date is changed and ticket class is
upgraded, there is a Stream of events that changes the
state.
• Ticket can even be cancelled but the history of the ticket
booking will remain and recorded as logs and each event
is Immutable.
24. STREAM JOINS
• Joining different datasets to perform analysis
• Stream-Stream Join
• Stream-Table Join
• Table-Table Join
25. STREAM-STREAM JOINS
• User searches for a book and he clicks the search result.
• You need to map the book searched and clicked book.
• Now the search is recorded as one event and click is
another event.
• If you want to analyse the click through rate you need to
bring together the search and click action, which can be
connected by Session ID
26. STREAM-TABLE JOINS
• User searches for a book
• You need to show some recommendation based on his
search
• Now you need to join the search which is an event and his
search history which is saved as table.
• You can copy the table to local cache and join the search.
But cache can become stale. Better is to subscribe the
cache to the stream of user history