Kamaelia is a concurrency toolkit created by the BBC for building scalable networked applications. It uses generators and message passing between components to allow for concurrent yet sequential processing. Key aspects of Kamaelia include Axon, which provides the core component infrastructure based on communicating generators, and a collection of components that use Axon to implement things like TCP clients and servers. The document discusses the motivations for creating Kamaelia, provides examples of how components are built and linked together, and suggests that the use of generators makes concurrency programming easier for novices compared to other approaches. It also describes testing Kamaelia's ease of use with a pre-university student who was able to build a useful system after a short learning
Slides of the OSLC Webcast of the 16th of October, 2012 presenting the OSLC-CM Perl modules in Eclipse Lyo.
Video of the webcast: http://vimeo.com/51541598
Hyper-Parameter Tuning Across the Entire AI Pipeline GPU Tech Conference San ...Chris Fregly
Chris Fregly, Founder @ PipelineAI, will walk you through a real-world, complete end-to-end Pipeline-optimization example. We highlight hyper-parameters - and model pipeline phases - that have never been exposed until now.
While most Hyperparameter Optimizers stop at the training phase (ie. learning rate, tree depth, ec2 instance type, etc), we extend model validation and tuning into a new post-training optimization phase including 8-bit reduced precision weight quantization and neural network layer fusing - among many other framework and hardware-specific optimizations.
Next, we introduce hyperparameters at the prediction phase including request-batch sizing and chipset (CPU v. GPU v. TPU).
Lastly, we determine a PipelineAI Efficiency Score of our overall Pipeline including Cost, Accuracy, and Time. We show techniques to maximize this PipelineAI Efficiency Score using our massive PipelineDB along with the Pipeline-wide hyper-parameter tuning techniques mentioned in this talk.
Bio
Chris Fregly is Founder and Applied AI Engineer at PipelineAI, a Real-Time Machine Learning and Artificial Intelligence Startup based in San Francisco.
He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High Performance TensorFlow in Production with Kubernetes and GPUs."
Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco.
Slides of the OSLC Webcast of the 16th of October, 2012 presenting the OSLC-CM Perl modules in Eclipse Lyo.
Video of the webcast: http://vimeo.com/51541598
Hyper-Parameter Tuning Across the Entire AI Pipeline GPU Tech Conference San ...Chris Fregly
Chris Fregly, Founder @ PipelineAI, will walk you through a real-world, complete end-to-end Pipeline-optimization example. We highlight hyper-parameters - and model pipeline phases - that have never been exposed until now.
While most Hyperparameter Optimizers stop at the training phase (ie. learning rate, tree depth, ec2 instance type, etc), we extend model validation and tuning into a new post-training optimization phase including 8-bit reduced precision weight quantization and neural network layer fusing - among many other framework and hardware-specific optimizations.
Next, we introduce hyperparameters at the prediction phase including request-batch sizing and chipset (CPU v. GPU v. TPU).
Lastly, we determine a PipelineAI Efficiency Score of our overall Pipeline including Cost, Accuracy, and Time. We show techniques to maximize this PipelineAI Efficiency Score using our massive PipelineDB along with the Pipeline-wide hyper-parameter tuning techniques mentioned in this talk.
Bio
Chris Fregly is Founder and Applied AI Engineer at PipelineAI, a Real-Time Machine Learning and Artificial Intelligence Startup based in San Francisco.
He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High Performance TensorFlow in Production with Kubernetes and GPUs."
Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco.
From the Cocoon GetTogether 2005:
When flowscript came up it was not only the powerful idea of continuations that helped making it a big success. In combination with its scripting nature it provided a much quicker development cycle.Soon people wished to have such a short turnaround in their java based development environments. And that's what we have in the latest Cocoon trunk - today! Auto-compilation of javaflow (the java based alternative to flowscript) and instant reloading of components helps to cut down development times tremendously. Without further need of restarting your servlet container after every little change, java development has finally become as easy as it should be.
This session will demonstrates on how to use features like reloading or auto-compilation of the current Cocoon trunk. It will also try to demystify the magic behind javaflow and will provide an overview about the current status and limitations.
Verified CKAD Exam Questions and Answersdalebeck957
Really impressed by the brilliant exam practise software. Highly recommended to all candidates for the CKAD exam preparations. I got 93% in the first attempt.
EMC World 2016 - code.09 Introduction to the Docker Platform{code}
History is repeating itself with disruptive software infrastructure platforms taking over in the data center. This session will cover the Docker platform, reviewing each Docker project focused on incremental innovation and providing developers and operations the ability to run, deploy, manage and monitor containers. Learn all about Docker Engine, Machine, Compose, Swarm, Hub, Trusted Registry and more! Demos of each product will be provided as well as how each tie into EMC II technology.
Join this workshop and accelerate your journey to production-ready Kubernetes by learning the practical techniques for reliably operating your software lifecycle using the GitOps pattern. The Weaveworks team will be running a full-day workshop, sharing their expertise as users and contributors of Kubernetes and Prometheus, as well as followers of GitOps (operations by pull request) practices.
Using a combination of instructor led demonstrations and hands-on exercises, the workshop will enable the attendee to go into detail on the following topics:
• Developing and operating your Kubernetes microservices at scale
• DevOps best practices and the movement towards a “GitOps” approach
• Building with Kubernetes in production: caring for your apps, implementing CI/CD best practices, and utilizing the right metrics, monitoring tools, and automated alerts
• Operating Kubernetes in production: Upgrading and managing Kubernetes, managing incident response, and adhering to security best practices for Kubernetes
EMC World 2016 - code.05 Automating your Physical Data Center with RackHD{code}
RackHD is an open source project started with the goal of providing a consistent and clear mechanism to perform hardware inventory and firmware upgrades to commodity white-box servers. With an event-based workflow engine that interacts with existing protocols and mechanisms such as PXE, BMC, and IPMI it can create workflows of tasks, boot scripts, and interactions to achieve full system automation. Come see it in action in this session showing useful workflows integrating with Docker, OpenStack, and Cloud Foundry.
From the Cocoon GetTogether 2005:
When flowscript came up it was not only the powerful idea of continuations that helped making it a big success. In combination with its scripting nature it provided a much quicker development cycle.Soon people wished to have such a short turnaround in their java based development environments. And that's what we have in the latest Cocoon trunk - today! Auto-compilation of javaflow (the java based alternative to flowscript) and instant reloading of components helps to cut down development times tremendously. Without further need of restarting your servlet container after every little change, java development has finally become as easy as it should be.
This session will demonstrates on how to use features like reloading or auto-compilation of the current Cocoon trunk. It will also try to demystify the magic behind javaflow and will provide an overview about the current status and limitations.
Verified CKAD Exam Questions and Answersdalebeck957
Really impressed by the brilliant exam practise software. Highly recommended to all candidates for the CKAD exam preparations. I got 93% in the first attempt.
EMC World 2016 - code.09 Introduction to the Docker Platform{code}
History is repeating itself with disruptive software infrastructure platforms taking over in the data center. This session will cover the Docker platform, reviewing each Docker project focused on incremental innovation and providing developers and operations the ability to run, deploy, manage and monitor containers. Learn all about Docker Engine, Machine, Compose, Swarm, Hub, Trusted Registry and more! Demos of each product will be provided as well as how each tie into EMC II technology.
Join this workshop and accelerate your journey to production-ready Kubernetes by learning the practical techniques for reliably operating your software lifecycle using the GitOps pattern. The Weaveworks team will be running a full-day workshop, sharing their expertise as users and contributors of Kubernetes and Prometheus, as well as followers of GitOps (operations by pull request) practices.
Using a combination of instructor led demonstrations and hands-on exercises, the workshop will enable the attendee to go into detail on the following topics:
• Developing and operating your Kubernetes microservices at scale
• DevOps best practices and the movement towards a “GitOps” approach
• Building with Kubernetes in production: caring for your apps, implementing CI/CD best practices, and utilizing the right metrics, monitoring tools, and automated alerts
• Operating Kubernetes in production: Upgrading and managing Kubernetes, managing incident response, and adhering to security best practices for Kubernetes
EMC World 2016 - code.05 Automating your Physical Data Center with RackHD{code}
RackHD is an open source project started with the goal of providing a consistent and clear mechanism to perform hardware inventory and firmware upgrades to commodity white-box servers. With an event-based workflow engine that interacts with existing protocols and mechanisms such as PXE, BMC, and IPMI it can create workflows of tasks, boot scripts, and interactions to achieve full system automation. Come see it in action in this session showing useful workflows integrating with Docker, OpenStack, and Cloud Foundry.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Kamaelia-ACCU-20050422
1. Kamaelia - Networking
Using Generators
Michael Sparks
BBC Research & Development
ACCU Python 2005, Oxford
2. Kamaelia
• Project to explore long term systems for large
scale media delivery
• Forms a concurrency toolkit, focussed mainly on experimenting
with network protocols.
• 2 key portions:
• Axon - Core Component infrastructure, based on communicating
generators
• Kamaelia - Collection of components that use Axon.
• Aim: Scalable, easy & safe concurrent systems
(c) 2005 BBC R&D
3. Kamaelia Status
• Released as open source:
• http://kamaelia.sourceforge.net/
• Axon is at version 1.0.3, and considered feature
stable.
• Runs on Linux, Windows (variety), Mac OS X
• Specialised distribution for Nokia Series 60 mobiles
• Kamaelia is at version 0.1.2, and growing
• Ability to write TCP & Multicast clients and servers
• Variety of simple servers, clients and protocols included
(c) 2005 BBC R&D
4. Kamaelia Status
• Kamaelia 0.1.2:
• Tested on Linux, Windows (variety), Mac OS X
• Subset on Nokia Series 60 mobiles
• Ease of use hypothesis has been tested with 1 pre-
university trainee, looks promising
(c) 2005 BBC R&D
5. Kamaelia Motivations
• Large Scale Streaming
• Several million streams per day
• Big events have tens of thousands of concurrent viewers
• Want to scale to handling millions of concurrent viewers
• Since this could happen.
(c) 2005 BBC R&D
6. Kamaelia Motivations
• What If 10 years from now...
• the BBC opened the entire archive?
• Creative Archive is NOT that ambitious! (AFAIK)
• the entire UK got broadband?
• Instantly hit long tail problems
• 20 million homes?
• 20 million different things?
• Not like 20 million people watching BBC1 !
(c) 2005 BBC R&D
7. Kamaelia Motivations
• Key Problems:
• RTP was originally concieved for A/V conferencing/telephony
• Aspects don’t scale well for large scale unidirectional streaming
• Need a platform for designing, implementing and testing new open
standards to scale in this way.
• Scalability and ability to experiment often conflict.
• Large scale means highly parallel
• Scalable concurrency often has a high barrier to entry
• Limits new ideas, collaboration
(c) 2005 BBC R&D
8. Axon
• Kamaelia's Core Concurrency System
• Key aims:
• Scalable appoach
• Reusable
• Simple - easy enough for novice programmer to pick up and
produce useful systems.
• Novices see possibilities, not problems
• Safe - it should be possible to write programs without worrying
about race hazards
• Non locking if possible
(c) 2005 BBC R&D
9. Scaling Concurrency
• quot;Threads are for people who cant program state
machines.quot; -- Alan Cox (http://tinyurl.com/a66es)
• Processes/Threads/Build your own
• Processes and threads are well known to be not scalable cross
platform.
• Build your own:
• Normally means state machines
• What about people who quot;cant program state machinesquot; ?
• (Not a dig at Alan !)
(c) 2005 BBC R&D
10. Scalability : State machines
• Hard to get 100% right - especially for novices
• Debugging someone else’s - twice as hard
• State machine is a piece of sequential processing that
can release control half way and be restarted retaining
state
• Twisted - at it’s heart very state machine based.
• Provides a very good framework for this and provides lots of
high quality assistance
• Still has this barrier to entry (my personal opinion,YMMV)
(c) 2005 BBC R&D
11. Scalability or ease ?
Do we really have to choose?
• Consider:
• A state machine is a piece of sequential processing that can
release control half way and be restarted
• A generator is a piece of sequential processing that can release
control half way and be restarted
• Twisted also recognises this: twisted.flow
• Takes a different approach to composition
• Kamaelia uses generators
• Hypothesised this would be easier for novices
(c) 2005 BBC R&D
12. Kamaelia vs Twisted?
• NO!
• Kamaelia could be integrated into twisted (or vice versa) - we just
haven't looked at that yet
• Twisted is stable, mature and usable for production
systems
• Kamaelia isn't mature or suitable for production systems at
present
• Won’t always be that way, but even when it isn’t we’d rather
collaborate rather than compete.
• Lengthy answer in Kamaelia’s blog
(c) 2005 BBC R&D
13. Concurrency is Easy ?
• Concurrency is hard
• ... so why do we let sys admins do it?
• Think unix pipelines:
• find -type f | egrep -v '/build/|^./MANIFEST' |while read i ;
do cp ../Source/$i $i done
• This has 4 logically concurrent units!
• Do unix sys admins think of themselves concurrent programmers?
• Do you think of it that way?
(c) 2005 BBC R&D
14. Unix Pipelines
• Concurrent sequential processes - linear
• Items don't know what's next in the pipeline
• Simply communicate with local file handles
• Often forgotten “hidden” details:
• How data passes between processes
• The system environment
(c) 2005 BBC R&D
15. Axon - Key classes
• Components - self pausing sequential objects that
send data to local interfaces
• Linkages - a facility for joining interfaces, allowing
system composition
• Scheduler - gives components CPU time
• Postman - The facility for tracking linkages, and
handling data transferral
• Co-ordinating Assistant/Tracker (cat) - Provides
environmental facilities akin to a Linda tuple space
(c) 2005 BBC R&D
16. Axon Components
• Classes with a generator method called quot;mainquot;
• Augmented by:
• List of Inboxes - defaults: inbox, control
• List of Outboxes - defaults: outbox, signal
• class Echo(component):
def main(self):
while 1:
if self.dataReady(quot;inboxquot;):
data = self.recv(quot;inboxquot;)
self.send(data,quot;outboxquot;)
yield 1
(c) 2005 BBC R&D
17. Axon Scheduler
• Operation
• Holds a run queue containing activated components
• Calls the generator for each component sequentially
• Component Activation
• If the return value is a newComponent object the components
contained are activated (essentially their main() method is called,
and the resulting generator stored)
• Component Deactivation
• If the return value is false, the component is removed from the run
queue
(c) 2005 BBC R&D
18. Linkages
• Normally join outboxes to inboxes between
components
• out-out and in-in also allowed between parent and nest
components
• Linkages can only be create inside a component
• Inboxes and outboxes designed for connection to subcomponents
are considered private and have the naming convention of a
leading underscore
• Encourages composition and reuse
(c) 2005 BBC R&D
19. Linkage Example
• class SimpleStreamingClient(component):
def main(self):
client=TCPClient(quot;127.0.0.1quot;,1500)
decoder = VorbisDecode()
player = AOAudioPlaybackAdaptor()
self.link((client,quot;outboxquot;), (decoder,quot;inboxquot;)
self.link((decoder,quot;outboxquot;), (player,quot;inboxquot;))
self.addChildren(decoder, player, client)
yield newComponent(decoder, player, client)
while 1:
self.pause()
yield 1
(c) 2005 BBC R&D
20. Linkage Example 2
def AdHocFileProtocolHandler(filename):
class klass(Kamaelia.ReadFileAdaptor.ReadFileAdaptor):
def __init__(self,*argv,**argd):
self.__super.__init__(filename, readmode=quot;bitratequot;, bitrate=400000)
return klass
class SimpleStreamingServer(component):
def main(self):
server = SimpleServer(protocol=AdHocFileProtocolHandler (quot;foo.oggquot;),
port=clientServerTestPort)
self.addChildren(server)
yield _Axon.Ipc.newComponent(*(self.children))
while 1:
self.pause()
yield 1
(c) 2005 BBC R&D
22. Co-ordinating Assistant Tracker
• Tracking Services
• This allows for the concept of services
• A service is a mapping of name to (component, inbox) tuple
• Only ever quot;needquot; one 'select' statement in a program for example.
(want is a different matter!)
• The Kamaelia.Internet.Selector component offers a quot;selectorquot;
service
• Tracking Values
• Provides look up and modification of values for keys
• Use case: to enable stats collection in servers
(c) 2005 BBC R&D
23. Howto: Example Component
• MIME/RFC2822 type objects are common in
network protocols
• Email, web, usenet, etc..
• Essentially serialised key/value pairs - much like a
dict.
• Create a “MIME Dict” component.
• Accepts dict like objects, but translates them to MIME-like
messages
• Accepts MIME-like messages, and converts them to dicts.
(c) 2005 BBC R&D
24. MimeDictComponent
• How it was written
• First of all a class that could be a quot;MIME dictquot; was written
• Subclasses dict
• Always adds a __BODY__ key
• Replaces __str__ with something that displays the dict as an
RFC2822/MIME style message
• Adds a staticmethod quot;fromStringquot; as a factory method.
• Written entirely test first without a view
to being used as a component
(c) 2005 BBC R&D
25. MimeDictComponent 2
• Wanted a component thus:
• control - on which we may receive a shutdown message
• signal - one which we will send shutdown messages
• demarshall - an inbox to which you send strings for turning into
dicts
• marshall - an inbox to which you send objects for turning into
strings
• demarshalled - an outbox which spits out parsed strings as
dicts
• marshalled = an outbox which spits out translated dicts as
strings
(c) 2005 BBC R&D
26. MimeDictComponent 3
• Turned out to be simpler to write a generic marshalling
component instead, main loop looked like this:
while 1:
self.pause()
if self.dataReady(quot;controlquot;):
data = self.recv(quot;controlquot;)
if isinstance(data, Axon.Ipc.producerFinished)
self.send(Axon.Ipc.producerFinished(), quot;signalquot;)
return
if self.dataReady(quot;marshallquot;):
data = self.recv(quot;marshallquot;)
self.send(str(data),quot;marshalledquot;)
if self.dataReady(quot;demarshallquot;):
data = self.recv(quot;demarshallquot;)
self.send(self.klass.fromString(data),quot;demarshalledquot;)
yield 1
(c) 2005 BBC R&D
27. MimeDictComponent 4
• Subclassing approach:
• class MimeDictMarshaller(MarshallComponent):
def __init__(self,*argv,**argd):
self.__super.__init__(MimeDict, *argv,**argd)
• Class decoration approach:
• def MarshallerFactory(klass):
class newclass(MarshallComponent):
def __init__(self,*argv,**argd):
self.__super.__init__(klass, *argv,**argd)
return newclass
MimeDictMarshaller=MarshallerFactory(MimeDict)
(c) 2005 BBC R&D
28. Summary: New Components
• Longer tutorial based around a multicast
transceiver on the website.
• Same approach:
• Don't worry about concurrency, write single threaded
• When code works, then convert to components
• Change control methods into inboxes/outboxes
(c) 2005 BBC R&D
29. Ease of use?
• Tested on Ciaran Eaton, a pre-university trainee
• Happy to let me call him a novice programmer (triple checked)
• Previous experience: A-Level computer studies - small amount of
Visual Basic programming and Access
• 3 Month placement with our group
• Started off learning python & axon (2 weeks)
• Created a “learning system” based around parsing a Shakespeare
play:
• Performs filtering, character identification, demultiplexing etc
• Used pygame for display, stopped short of using pyTTS...
(c) 2005 BBC R&D
30. Ease of use? 2
• Ciaran’s project:
• Created a simplistic low bandwidth video streamer
• Server has an MPEG video, and takes a frame as JPEG every n
seconds
• This is sent to the client over a framing protocol Ciaran designed
and implemented
• The client then displays the images as they arrive
• On a PC this uses pygame
• On a series 60 mobile this uses the native image display calls
• The idea is this simulates previewing PVR content on a mobile
(c) 2005 BBC R&D
31. Ease of use? 3
• Project was successful, Ciaran achieved the goals
• Ciaran wrote all the components for every part of
the description.
• Relied on a “SimpleServer” and simple “TCPclient”
components - but these only provide reliable data
transfer over the network.
• He’s noted that it was a fun experience
• I find it interesting it was not frustrating given his background.
(c) 2005 BBC R&D
32. CSP vs State Machines
• Is this approach inherently worse or better?
• We would suggest neither.
• State machine systems often have intermediate
buffers (even single variables) for handoff between
state machines
• This is akin to outboxes and inboxes. If they are
collapsed into one, as planned, this is equivalent
• If we do collapse outboxes into inboxes when we create linkages,
then the system should be as efficient as frameworks like twisted.
• This is currently hypothetical.
(c) 2005 BBC R&D
33. Integration with other systems
• Default component provides a default main, which
calls 3 default callbacks.
• Looks like this:
• def main(self):
result = self.initialiseComponent()
if not result:
result = 1
yield result
while(result):
result = self.mainBody()
if result:
yield result
yield self.closeDownComponent()
(c) 2005 BBC R&D
34. Integration: 2
• Purpose of the 3 callback form is for 2 main
reasons
• For those who find callback forms easier to work with
• To allow these methods to be overridden by classes written in:
• Pyrex
• C
• C++
• ie optimisation of components
(c) 2005 BBC R&D
35. Futures
• C++ Version.
• Simple “miniaxon” version including C++ based generators
working. see: cvs:/Code/CPP/Scratch/miniaxon.cpp
• Python Axon will be optimised
• Syntactic Sugar will be added
• Automated component distribution over clusters
• Kamaelia Component Repository
• More protocols, experimental servers:
• RTSP/RTP initially. New protocols to follow!
(c) 2005 BBC R&D
36. Finally: Collaboration
• If you’re interested in working with us, please do
• If you find the code looks vaguely interesting, please use and give
us feedback
• We’re very open to exploring changes to the system and willing to
give people CVS commit access in order to try their ideas.
• Anyone working with twisted is very welcome to come and
criticise and suggest new ideas - integration would be very nice!
• Contacts, project blog:
• michaels@rd.bbc.co.uk, kamaelia-list@lists.sourceforge.net
• http://kamaelia.sourceforge.net/cgi-bin/blog/blog.cgi
(c) 2005 BBC R&D