The document describes the ARIES recovery algorithm, which involves three passes - analysis, redo, and undo.
The analysis pass scans the log from the most recent checkpoint and determines which transactions need to be undone, which pages were dirty at the time of crash, and the redo start point. The redo pass repeats history by redoing log records from the redo start point. The undo pass rolls back incomplete transactions by scanning the log backwards and undoing actions of uncommitted transactions.
Gives an overview about Process, PCB, Process States, Process Operations, Scheduling, Schedulers, Interprocess communication, shared memory and message passing systems
Gives an overview about Process, PCB, Process States, Process Operations, Scheduling, Schedulers, Interprocess communication, shared memory and message passing systems
The purpose of types:
To define what the program should do.
e.g. read an array of integers and return a double
To guarantee that the program is meaningful.
that it does not add a string to an integer
that variables are declared before they are used
To document the programmer's intentions.
better than comments, which are not checked by the compiler
To optimize the use of hardware.
reserve the minimal amount of memory, but not more
use the most appropriate machine instructions.
Segmentation topic is presented in a most easy way.
Segmentation is a user view of memory in Operating System. Segmentation is one of the most common ways to achieve memory protection. In a computer system using segmentation, an instruction operand that refers to a memory location includes a value that identifies a segment and an offset within that segment.
This presentation describes about the various memory allocation methods like first fit, best fit and worst fit in memory management and also about fragmentation problem and solution for the problem.
2015 course SPPU SEIT syllabus of subject Processor Architecture and Interfacing (PAI) This covers Introduction to multitasking, Support Registers and Data Structures, Task State Segment (TSS), TSS Descriptor, Task Register, Task Switching via TSS and Task Gate, Task Gate Descriptor,
The topic focuses on different aspects of processor organization and architecture such as architecture models, register organization, instruction formats, addressing modes etc.
The purpose of types:
To define what the program should do.
e.g. read an array of integers and return a double
To guarantee that the program is meaningful.
that it does not add a string to an integer
that variables are declared before they are used
To document the programmer's intentions.
better than comments, which are not checked by the compiler
To optimize the use of hardware.
reserve the minimal amount of memory, but not more
use the most appropriate machine instructions.
Segmentation topic is presented in a most easy way.
Segmentation is a user view of memory in Operating System. Segmentation is one of the most common ways to achieve memory protection. In a computer system using segmentation, an instruction operand that refers to a memory location includes a value that identifies a segment and an offset within that segment.
This presentation describes about the various memory allocation methods like first fit, best fit and worst fit in memory management and also about fragmentation problem and solution for the problem.
2015 course SPPU SEIT syllabus of subject Processor Architecture and Interfacing (PAI) This covers Introduction to multitasking, Support Registers and Data Structures, Task State Segment (TSS), TSS Descriptor, Task Register, Task Switching via TSS and Task Gate, Task Gate Descriptor,
The topic focuses on different aspects of processor organization and architecture such as architecture models, register organization, instruction formats, addressing modes etc.
This presentation is an overview of points discussed in following research paper
Mohan, C., & Jose, S. (1999). Repeating History Beyond ARIES. 25th VLDB Conference, 1–17.
Stibo Systems recently released its in-memory component for our Master Data Management (MDM) platform, giving significant speed-ups in most parts of the system. Our MDM platform provides high volume data management with many concurrent users. This in-memory component is built in-house and this talk is about how and why we did this, including:
MVCC (Multi Version Concurrency Control) aware map, off heap and compact.
- Lock-free MVCC aware indexing.
- Wait-free MVCC aware querying that goes directly on the metal.
- Clustering and MVCC with recovery support.
- Why we built our own in-memory technology, how we integrated it into our existing 200+ man years system and the speed-ups we gained.
This will help you as a developer to navigate the landscape of in-memory products and identify the trade-offs involved helping you choose the right path.
My presentation on a new Oracle Database feature called "Row Pattern Matching". I presented on 9 December 2014 at UKOUG Tech 14. Be sure to download for the animations.
CDC Stream Processing With Apache Flink With Timo Walther | Current 2022HostedbyConfluent
CDC Stream Processing With Apache Flink With Timo Walther | Current 2022
An instant world requires instant decisions at scale. This includes the ability to digest and react to changes in real-time. Thus, event logs such as Apache Kafka can be found in almost every architecture, while databases and similar systems still provide the foundation. Change Data Capture (CDC) has become popular for propagating changes. Nevertheless, integrating all these systems, which often have slightly different semantics, can be a challenge.
In this talk, we highlight what it means for Apache Flink to be a general data processor that acts as a data integration hub. Looking under the hood, we demonstrate Flink's SQL engine as a changelog processor that ships with an ecosystem tailored to processing CDC data and maintaining materialized views. We will discuss the semantics of different data sources and how to perform joins or stream enrichment between them. This talk illustrates how Flink can be used with systems such as Kafka (for upsert logging), Debezium, JDBC, and others.
PowerPoint presentation on Computer architecture,
Signed Multiplication: Booth’s Algorithm.
or
Digital System Design PowerPoint presentation on Booth's algorithm.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
2. ARIES Recovery Algorithm
ARIES recovery involves three passes
Analysis pass: Determines
Which transactions to undo
Which pages were dirty (disk version not up to date) at time of
crash
RedoLSN: LSN from which redo should start
Redo pass:
Repeats history, redoing all actions from RedoLSN
RecLSN and PageLSNs are used to avoid redoing actions already
reflected on page
Undo pass:
Rolls back all incomplete transactions
Transactions whose abort was complete earlier are not undone
Key idea: no need to undo these transactions: earlier undo actions
were logged, and are redone as required
2
3. Aries Recovery: 3 Passes
Analysis, redo and undo passes
Analysis determines where redo should start
Undo has to go back till start of earliest incomplete
transaction
Last checkpoint
Log
Time
End of Log
Analysis pass
Redo pass
Undo pass
3
4. Recovering from a System Crash
Recovering will use WAL & the most recent checkpoint
Write-ahead log
The most recent checkpoint
Compensation Log Records
undoNextLSN: the LSN of the next log record that is to be undone
Transaction table
active (not committed) transactions
lastLSNs: the LSN of the most recent log record for this transaction. (analysis)
Used for undo
Dirty page table
dirty (not written to disk) pages
recLSNs: LSN of the first log record that caused this page to become dirty
Used for redo
4
5. Analysis Phase
Determine three things:
A point in the log to start REDO.
Earliest update log that may not have been written to disk.
Dirty pages in the buffer pool at the time of crash -> restore the
dirty page table to the time of crash.
Active transactions at time of crash for UNDO -> restore the
transaction table to the time of crash.
5
6. Analysis Phase: Algorithm
1. Find the most recent begin_checkpoint log record.
2. Initialize transaction & dirty page tables from the ones
saved in the most recent checkpoint.
3. Scan forward the records from begin_checkpoint log
record to the end of the log. For each log record LSN,
update trans_tab and dirty_page_tab as follows:
If we see an end log record for T, remove T from trans_tab.
If we see a log record for T’ not in trans_tab, add T’ in trans_tab.
If T’ is in the trans_tab, then set T’s lastLSN field to LSN.
If we see an update/CLR log record for page P and P is not in
the dirty page table, add P in dirty page table and set its
recLSN to LSN.
6
7. Analysis Phase: Example (1)
LSN TransID Type PageID
00 T1000 update P500
10 T2000 update P600
20 T2000 update P500
30 T1000 update P505
40 T2000 commit
System Crash
After system crash,
both table are lost.
No previous
checkpointing,
initialize tables to
empty.
7
pageID recLSN
Transaction Table Dirty Page
Table
transID lastLSN
12. Analysis Phase: Example (6)
LSN TransID Type PageID
00 T1000 update P500
10 T2000 update P600
20 T2000 update P500
30 T1000 update P505
40 T2000 Commit
System Crash
Scanning log 40:
Remove T2000 from
transaction table.
We are done!
The redo point starts at
00.
Why?
P500 is the earliest log
that may not have been
written to disk before
crash.
We have restored
transaction table & dirty
page table.
12
pageID recLSN
P500 00
P600 10
P505 30
transID lastLSN
T1000 30
T2000 10
Transaction Table Dirty Page
Table
13. Redo Phase: Algorithm
Scan forward from the redo point (LSN 00).
For each update/CLR-undo log record LSN, perform
redo unless one of the conditions holds:
The affected page is not in the dirty page table
It is not dirty. So no need to redo.
The affected page is in the dirty page table, but recLSN > LSN.
The page’s recLSN (oldest log record causing this page to be dirty) is
after LSN.
pageLSN >= LSN
A later update on this page has been written (pageLSN = the most
recent LSN to update the page on disk).
13
14. Redo Phase: Example (1)
LSN TransID Type PageID
00 T1000 update P500
10 T2000 update P600 (disk)
20 T2000 update P500
30 T1000 update P505
40 T2000 commit
System Crash
Scan forward from the redo
point (LSN 00).
Assume that P600 has been
written to disk.
But it can still be in the dirty
page table.
Scanning 00:
P500 is in the dirty page
table.
00(recLSN) = 00 (LSN)
-10 (pageLSN) < 00 (LSN)
Redo 00
Scanning 10:
14
pageID recLSN
P500 00
P600 10
P505 30
transID lastLSN
T1000 30
Transaction Table Dirty Page
Table
16. Undo Phase: Algorithm
It scans backward in time from the end of the log.
It needs to undo all actions from active (not committed)
transactions. They are also called loser transactions.
Same as aborting them.
Analysis phase gives the set of loser transactions, called
ToUndo set.
Repeatedly choose the record with the largest LSN value in
this set and processes it, until ToUndo is empty.
If it is a CLR and undoNextLSN value is not null, use undoNextLSN
value in ToUndo. If undoNextLSN is null, this transaction is completely
undo.
If it is an update record, a CLR is written and restore the data record
value to before-image. Use prevLSN value in ToUndo.
16
17. Undo Phase: Example (1)
LSN TransID Type PageID
00 T1000 update P500
10 T2000 update P600 (disk)
20 T2000 update P500
30 T1000 update P505
40 T2000 commit
System Crash
The only loser transaction is
T1000.
ToUndo set is {T1000:30}
17
pageID recLSN
P500 00
P600 10
P505 30
transID lastLSN
T1000 30
Transaction Table Dirty Page
Table
18. Undo Phase: Example (2)
The only loser transaction is T1000.
ToUndo set is {T1000:30}
Undoing LSN:30
Write CLR:undo record log.
ToUndo becomes {T1000:00}
Undoing LSN:00
Write CLR:undo record log.
ToUndo becomes null.
We are done.
LSN TransID Type PageID
00 T1000 update P500
10 T2000 update P600 (disk)
20 T2000 update P500
30 T1000 update P505
40 T2000 commit
System Crash
50 T1000 CLR:undo:30 P505
60 T1000 CLR:undo:00 P500
18
undoNextLSN
19. Crashes During Restart (1)
After T1 aborts, undo actions from T1.
Undo LSN #10: write CLR:undo record
log for LSN #10.
Dirty pages:
P1 (recLSN=50)
P3(20)
P5(10).
Loser transaction:
T2(lastLSN=60)
T3(50)
Redo phases starts at 10.
Undo LSN #60.
LSN LOG
00, 05 begin_checkpoint,
end_checkpoint
10 Update: T1 write P5
20 Update: T2 writes P3
30 T1 aborts
40, 45 CLR: Undo T1 LSN 10, T1 end
50 Update: T3 writes P1
60 Update: T2 writes P5
CRASH, RESTART
70 CLR: Undo T2 LSN 60
80, 85 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART
90,95 CLR: Undo T2 LSN 20, T2 end
19
undoNextLSN
20. Crashes During Restart (2)
Undo LSN #50: write CLR: undo record
log.
T3 is completely undone.
LSN #85,80,70 are written to stable
storage.
Crash occurs after restart.
Loser transaction is T2.
Read LSN #70, set ToUndo to #20.
Undo #20: write another CLR.
Done
LSN LOG
00, 05 begin_checkpoint,
end_checkpoint
10 Update: T1 write P5
20 Update: T2 writes P3
30 T1 aborts
40, 45 CLR: Undo T1 LSN 10, T1 end
50 Update: T3 writes P1
60 Update: T2 writes P5
CRASH, RESTART
70 CLR: Undo T2 LSN 60
80, 85 CLR: Undo T3 LSN 50, T3 end
CRASH, RESTART
90,95 CLR: Undo T2 LSN 20, T2 end
20
undoNextLSN