The document discusses ways to make applications more resilient to crashes. It suggests using caches instead of master copies to avoid data loss on crashes. Database transactions can maintain consistency even if the database crashes. Atomic operations like file renames can be used to update counts reliably. Recovery is also important - inconsistent states after crashes should be detected and resolved. The document advocates designing systems with crashes in mind through techniques like logging, versioning, and watchdog processes.
We present a novel dynamic program analysis that builds a semantic view of program executions. These views reflect program abstractions and aspects; however, views are not simply projections of execution traces, but are linked to each other to capture semantic interactions among abstractions at different levels of granularity in a scalable manner.
We describe our approach in the context of Java and demonstrate its utility to improve regression analysis. We first formalize a subset of Java and a grammar for traces generated at program execution. We then introduce several types of views used to analyze regression bugs along with a novel, scalable technique for semantic differencing of traces from different versions of the same program. Benchmark results on large open-source Java programs demonstrate that semantic-aware trace differencing can identify precise and useful details about the underlying cause for a regression, even in programs that use reflection, multithreading, or dynamic code generation, features that typically confound other analysis techniques.
We present a novel dynamic program analysis that builds a semantic view of program executions. These views reflect program abstractions and aspects; however, views are not simply projections of execution traces, but are linked to each other to capture semantic interactions among abstractions at different levels of granularity in a scalable manner.
We describe our approach in the context of Java and demonstrate its utility to improve regression analysis. We first formalize a subset of Java and a grammar for traces generated at program execution. We then introduce several types of views used to analyze regression bugs along with a novel, scalable technique for semantic differencing of traces from different versions of the same program. Benchmark results on large open-source Java programs demonstrate that semantic-aware trace differencing can identify precise and useful details about the underlying cause for a regression, even in programs that use reflection, multithreading, or dynamic code generation, features that typically confound other analysis techniques.
Introduce airflow and then deep dive to lower levels.
Explain some customized operational settings.
Please inform me if any incorrect informations in the slide.
Threads Advance in System Administration with LinuxSoumen Santra
Threads Advance in System Administration with Linux
Process Descriptor Handling
Kernel Stack
Pid_hash Table and Chained Lists
PID Hash Table Handling Functions and Macros
Wait Queues
Process Resource Limits
Task State Segment
System Calls
Pthread Operations
POSIX threads on GNU/Linux
Programs on Thread in C
In this session we will cover wide area replica sets and using tags for backup. Attendees should be well versed in basic replication and familiar with concepts in the morning's basic replication talk. No beginner topics will be covered in this session
go-git is a 100% Go libray used to interact with git repositories. Even if it already supports most of the functionality it still lags a bit in performance when compared with the git CLI or some other libraries. I'll explain some of the problems that we face when dealing with git repos and some examples of performance improvements done to the library.
JDK Flight Recorder introduced in OpenJDK 11.
This feature is low overhead of profiling and be able to used on production environment.
High Performance recording engine is embedded to Hotspot VM.
One of the strongest points for using a NoSQL database is their focus on distribution — both for replication and sharding. This talks takes a short look at what replication is, why you should use it, and what is so difficult about it. We then take a look at MongoDB’s implementation in general and finally focus on what can go wrong. In a practical demo you see how to find the right balance between performance versus data safety and how to use it in your Java application.
JEP349 (JFR Event Streaming) will be integrated into JDK14.
Provide an API for the continuous consumption of JFR data on disk, both for in-process and out-of-process applications.
I will introduce the points in using the JFR Event Streaming.
Introduce airflow and then deep dive to lower levels.
Explain some customized operational settings.
Please inform me if any incorrect informations in the slide.
Threads Advance in System Administration with LinuxSoumen Santra
Threads Advance in System Administration with Linux
Process Descriptor Handling
Kernel Stack
Pid_hash Table and Chained Lists
PID Hash Table Handling Functions and Macros
Wait Queues
Process Resource Limits
Task State Segment
System Calls
Pthread Operations
POSIX threads on GNU/Linux
Programs on Thread in C
In this session we will cover wide area replica sets and using tags for backup. Attendees should be well versed in basic replication and familiar with concepts in the morning's basic replication talk. No beginner topics will be covered in this session
go-git is a 100% Go libray used to interact with git repositories. Even if it already supports most of the functionality it still lags a bit in performance when compared with the git CLI or some other libraries. I'll explain some of the problems that we face when dealing with git repos and some examples of performance improvements done to the library.
JDK Flight Recorder introduced in OpenJDK 11.
This feature is low overhead of profiling and be able to used on production environment.
High Performance recording engine is embedded to Hotspot VM.
One of the strongest points for using a NoSQL database is their focus on distribution — both for replication and sharding. This talks takes a short look at what replication is, why you should use it, and what is so difficult about it. We then take a look at MongoDB’s implementation in general and finally focus on what can go wrong. In a practical demo you see how to find the right balance between performance versus data safety and how to use it in your Java application.
JEP349 (JFR Event Streaming) will be integrated into JDK14.
Provide an API for the continuous consumption of JFR data on disk, both for in-process and out-of-process applications.
I will introduce the points in using the JFR Event Streaming.
Strategies for Friendly English and Successful LocalizationJohn Collins
This slideshow was designed for a 20-minute progression session at the 2014 Society for Technical Communication Summit, presented on Tuesday, May 20, 2014. It's a significantly shortened version of a 45-minute session I'll be giving at Information Development World.
Companies are starting to distinguish themselves with a unique, natural English voice and tone, and many companies also realize there’s a growth potential in localizing their product to reach international markets. That leaves a tension for writers of the English content that will be translated for the international markets. Do the writers focus on tone or on writing easily translated content? Those two goals may seem mutually exclusive, but actually, they’re a healthy combination. We’ll look at what localization is and how to create content that’s good for your English-speaking users and well-suited for translation.
These are the slides of my talk about Rails 2.2 i18n (internationalization) which I gave at the Fosdem '09 Ruby and Rails DevRoom on February 8th 2009.
See http://fosdem.org/2009/schedule/events/ror_i18n_rails_2_2 for more details.
Here we talk about designing across, and for, multiple touchscreen platforms (Nokia, iPhone, iPad and Windows Phone 7) using Ribot’s recent suite of Tesco apps as a case study.
How do different form factors, operating systems, and interaction paradigms inform the design of real I-want-to-use-it-every-day apps?
How do you take the constraints (and opportunities) of differing mobile devices and design interfaces that, for the user, feel like they belong on the device and as part of their life?
(Download the presentation for full transcript)
Putting Out Fires with Content Strategy (STC Academic SIG)John Collins
You’ve probably heard – or said – something like “All I did today at work was put out fires.” We’ve all been there. We don’t want fires, but they happen. So, let’s see how content strategy helps put out fires in software development, and what you can do to prepare for a content strategy career.
Presented to the Society for Technical Communication Special Interest Group as part of the pre-conference activities for the CPTSC (Council for Programs in Technical and Scientific Communication) Conference in September 2014.
The ruby on rails i18n core api-Neeraj KumarThoughtWorks
Across the world, natural or regional languages differ in many ways, (e.g. in pluralization rules). Therefore, Internationalization became a complex problem and it is hard to provide tools for solving all problems at once. Sven Fuchs focused to provide an extensible framework and easy to use gem that is Ruby I18n (internationalization) gem.
The Ruby I18n gem is mainly designed for translating your application to a single custom language other than English or for providing multi-language support for your ruby on rails application. The pivotal point of the new I18n api in Rails is the I18n module which is provided as a gem and shipped with Rails (starting from Rails 2.2) in ActiveSupport’s vendor directory.
Therefore, during my presentation I will try to go over some of the advanced optional features and architecture of I18n gem. Besides, I will also try to cover begin with I18n gem, setup, benefits, the work flow, what's in? and what's not? Etc.
Building Quality Experiences for Users in Any LanguageJohn Collins
Companies are striving for a friendly tone with their content and also taking it to other cultures. Those two content goals seem to be at odds, but they don’t have to be. How do you reconcile more casual writing for content that will be translated into other languages? We’ll look at how to craft quality content for all users, regardless of their language. We’ll cover pitfalls to avoid and tactics to use for creating friendly English that yields pleasing translations.
Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)John Collins
More and more companies are striving for a friendly tone with their content. Many of those same companies are taking their content to other cultures with localized content. Those two content goals seem to be at odds.
This slideshow, presented at Information Development World in October of 2014, looks at how to accomplish both goals.
Putting Out Fires with Content Strategy (InfoDevDC meetup)John Collins
You’ve probably heard – or said – something like “All I did today at work was put out fires.” We’ve all been there. We don’t want fires, but they happen. So, let’s see how content strategy helps put out fires in software development, and what you can do to transition from a technical writing role to a content strategy role.
Presented at the InfoDevDC meetup on Dec. 9, 2014 (http://www.meetup.com/InfoDevDC/events/212733712/)
Presentación de la charla impartida en el meetup de Python Madrid sobre Asincronía en Python https://www.meetup.com/es-ES/python-madrid/events/268111847/
Think Async: Asynchronous Patterns in NodeJSAdam L Barrett
JavaScript is single threaded, so understanding the async patterns available in the language is critical to creating maintainable NodeJS applications with good performance. In order to master “thinking in async”, we’ll explore the async patterns available in node and JavaScript including standard callbacks, promises, thunks/tasks, the new async/await, the upcoming asynchronous iteration features, streams, CSP and ES Observables.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
4. ...except it does
C bugs, untrapped exception, infinite loops,
blocking calls, thread dead-lock, inconsistent
resident state
5. Recovery is important
"[S]ystem failure can usually be considered to
be the result of two program errors[...] the
second, in the recovery routine[...]"
10. Example: Counting
def update_counter():
fp = file("counter.txt")
s = fp.read()
counter = int(s.strip())
counter += 1
# If there is a crash before this point,
# no changes have been done.
fp = file("counter.txt.tmp", 'w')
print >>fp, counter
fp.close()
# If there is a crash before this point,
# only a temp file has been modified
# The following is an atomic operation
os.rename("counter.txt.tmp", "counter.txt")
19. Watchdog: Heartbeats
## In a Twisted process
def beat():
file('beats/my-name', 'a').close()
task.LoopingCall(beat).start(30)
20. Watchdog: Get time-outs
def getTimeout()
timeout = dict()
now = time.time()
for heart in glob.glob('hearts/*'):
beat = int(file(heart).read().strip())
timeout[heart] = now-beat
return timeout
21. Watchdog: Mark problems
def markProblems():
timeout = getTimeout()
for heart in glob.glob('beats/*'):
mtime = os.path.getmtime(heart)
problem = 'problems/'+heart
if (mtime<timeout[heart] and
not os.path.isfile(problem)):
fp = file('problems/'+heart, 'w')
fp.write('watchdog')
fp.close()
22. Watchdog: check solutions
def checkSolutions():
now = time.time()
problemTimeout = now-30
for problem in glob.glob('problems/*'):
mtime = os.path.getmtime(problem)
if mtime<problemTimeout:
subprocess.call(['restart-system'])
30. Example: Counting on Windows
def update_counter():
fp = file("counter.txt")
s = fp.read()
counter = int(s.strip())
counter += 1
# If there is a crash before this point,
# no changes have been done.
fp = file("counter.txt.tmp", 'w')
print >>fp, counter
fp.close()
# If there is a crash before this point,
# only a temp file has been modified
os.remove("counter.txt")
# At this point, the state is inconsistent*
# The following is an atomic operation
32. Example: Counting on Windows
(Recovery)
def recover():
if not os.path.exists("counter.txt"):
# The permanent file has been removed
# Therefore, the temp file is valid
os.rename("counter.txt.tmp",
"counter.txt")
33. Example: Counting with versions
def update_counter():
files = [int(name.split('.')[-1])
for name in os.listdir('.')
if name.startswith('counter.')]
last = max(files)
counter = int(file('counter.%s' % last
).read().strip())
counter += 1
# If there is a crash before this point,
# no changes have been done.
fp = file("tmp.counter", 'w')
print >>fp, counter
fp.close()
# If there is a crash before this point,
# only a temp file has been modified
35. Example: Counting with versions
(cleanup)
# This is not a recovery routine, but a cleanup
# routine.
# Even in its absence, the state is consistent
def cleanup():
files = [int(name.split('.')[-1])
for name in os.listdir('.')
if name.startswith('counter.')]
files.sort()
files.pop()
for n in files:
os.remove('counter.%d' % n)
if os.path.exists('tmp.counter'):
os.remove('tmp.counter')
36. Correct ordering
def activate_due():
scheduled = rs.smembers('scheduled')
now = time.time()
for el in scheduled:
due = int(rs.get(el+':due'))
if now<due:
continue
rs.sadd('activated', el)
rs.delete(el+':due')
rs.sremove('scheduled', el)
37. Correct ordering (recovery)
def recover():
inconsistent = rs.sinter('activated',
'scheduled')
for el in inconsistent:
rs.delete(el+':due') #*
rs.sremove('scheduled', el)
40. Example: Key/value stores (utility
functions)
## Get the level of a file
def getLevel(s)
return int(s.split('.')[0])
## Get all files of a given type
def getType(tp):
return [(getLevel(s), s)
for s in files if s.endswith(tp)]
41. Example: Key/value stores
(classifying files)
## Get all relevant files
def relevant(d):
files = os.listdir(d):
mlevel, master = max(getType('.master'))
logs = getType('.log')
logs.sort()
return master+[log for llevel, log in logs
if llevel>mlevel]
42. Example: Key/value stores (reading)
## Read in a single file
def update(result, fp):
for line in fp:
val = json.loads(line)
if val[0] == 'add':
result[val[1]] = val[2]
else:
del result[val[1]]
## Read in several files
def read(files):
result = dict()
for fname in files:
try:
update(result, file(fname))
46. Example: Key/value stores (storage
class)
## The actual data store abstraction.
class Store(object):
def __init__(self):
files = relevant(d)
self.result = read(files)
level = getLevel(files[-1])
self.writer = Writer(level)
def get(self, key):
return self.result[key]
def add(self, key, value):
self.writer.write(['add', key, value])
def remove(self, key):
self.writer.write(['remove', key])
47. Example: Key/value stores
(compression code)
## This should be run periodically
# from a different thread
def compress(d):
files = relevant(d)[:-1]
if len(files)<2:
return
result = read(files)
master = getLevel(files[-1])+1
fp = file('%3d.master.tmp' % master, 'w')
for key, value in result.iteritems():
towrite = ['add', key, value])
print >>fp, json.dumps(towrite)
fp.close()
48. Vertical splitting: Example
def forking_server():
s = socket.socket()
s.bind(('', 8080))
s.listen(5)
while True:
client = s.accept()
newpid = os.fork()
if newpid:
f = client.makefile()
f.write("Sunday, May 22, 1983 "
"18:45:59-PST")
f.close()
os._exit()
49. Horizontal splitting: front-end
## Process one
class SchedulerResource(resource.Resource):
isLeaf = True
def __init__(self, filepath):
resource.Resource.__init__(self)
self.filepath = filepath
def render_PUT(self, request):
uuid, = request.postpath
content = request.content.read()
child = self.filepath.child(uuid)
child.setContent(content)
fp = filepath.FilePath("things")
r = SchedulerResource(fp)
s = server.Site(r)
reactor.listenTCP(8080, s)
50. Horizontal splitting: scheduler
## Process two
rs = redis.Redis(host='localhost',
port=6379, db=9)
while True:
for fname in os.listdir("things"):
when = int(file(fname).read().strip())
rs.set(uuid+':due', when)
rs.sadd('scheduled', uuid)
os.remove(fname)
time.sleep(1)
51. Horizontal splitting: runner
## Process three
rs = redis.Redis(host='localhost',
port=6379, db=9)
recover()
while True:
activate_due()
time.sleep(1)
53. Horizontal splitting: message
queues: sender
## Process four
rs = redis.Redis(host='localhost',
port=6379, db=9)
params = pika.ConnectionParameters('localhost')
conn = pika.BlockingConnection(params)
channel = conn.channel()
channel.queue_declare(queue='active')
while True:
activated = rs.smembers('activated')
finished = set(rs.smembers('finished'))
for el in activated:
if el in finished:
continue
55. Horizontal splitting: message
queues: receiver
## Process five
# It is possible to get "dups" of bodies.
# Application logic should deal with that
params = pika.ConnectionParameters('localhost')
conn = pika.BlockingConnection(params)
channel = conn.channel()
channel.queue_declare(queue='active')
def callback(ch, method, properties, el):
syslog.syslog('Activated %s' % el)
channel.basic_consume(callback, queue='hello', no_ack=True)
channel.start_consuming()