Beginners: Bandwidth, Throughput, Latency & Jitter in mobile networks3G4G
A short presentation & video looking at what is meant by Bandwidth, Throughput, Latency & Jitter in mobile networks. We will look at simple examples to explain the concepts and also look at definitions from viewpoint of standards bodies like 3GPP, ITU, etc.
Beginners: Bandwidth, Throughput, Latency & Jitter in mobile networks3G4G
A short presentation & video looking at what is meant by Bandwidth, Throughput, Latency & Jitter in mobile networks. We will look at simple examples to explain the concepts and also look at definitions from viewpoint of standards bodies like 3GPP, ITU, etc.
UMTS/3G RAN Capacity Management Guideline Part-02 (Sectorization))Md Mustafizur Rahman
UMTS RAN capacity management is one of the Key activities to maintain the good QoS & stability of the 3G system. A relatively high load can affect the accessibility, coverage and QoS of established services. The capacity, coverage and QoS of the WCDMA system are mutually affected. Hence, for a stable & good quality 3G network, UMTS capacity need to be monitored, expanded & manage regularly.
In the first part of document (Part-1) UMTS soft capacity management along with the correlation between capacity vs accessibility, UMTS expansion threshold, counters & effective mechanism of UMTS Soft capacity management have been depicted. This part of the Document is focused on UMTS Hard Expansion Methodology. 3G Capacity management by Sectorization has been explained thoroughly.
After completing this, you will be able to gain knowledge on 3G Expansion Parameters, 3G Expansion Threshold, Capacity Gain, Cost vs Capacity Gain by Sectorization. Moreover, you will be familiar with Planning, Optimization & Post Performance activities of Sectorization.
In 5-2020, I and Minh have a sharing session about Sync, Async, and Multi-threading in C#.
We believe this is one of the most important things when working with .Net technology.
This sharing is aiming to Junior Dev, or even Sr Dev or anyone interested in Sync, Async, and Multi-threading.
I hope it will bring you some values.
The presentation covers the basics of packet forwarding and simplified architecture of the router. Additionally it explains what problem Cisco Express Forwarding (CEF) solves and how. At the end static routing is covered.
Delivered by Dmitry Figol, CCIE R&S #53592.
Presentació duta a terme el 30 de juny de 2017 al CSUC a càrrec de Walt Wollny, d’Hurricane Electric, on s'han comentat les eines per a l’estudi de paràmetres BGP d’Hurricane Electric, un dels membres del Punt Neutre d’Internet a Catalunya (CATNIX).
How does the Facebook Messenger app achieve phone-to-phone messaging latency in the order of milliseconds instead of seconds? Answer: It uses the MQTT protocol. And so can you.
In this session we look at the MQTT protocol and explain why it in many cases is a much better choice than HTTP or push notification for your mobile communication needs. Using the MQTT protocol your mobile app can achieve secure, reliable two-way communication without killing battery or wasting precious bandwidth. And it’s open source!
UMTS/3G RAN Capacity Management Guideline Part-02 (Sectorization))Md Mustafizur Rahman
UMTS RAN capacity management is one of the Key activities to maintain the good QoS & stability of the 3G system. A relatively high load can affect the accessibility, coverage and QoS of established services. The capacity, coverage and QoS of the WCDMA system are mutually affected. Hence, for a stable & good quality 3G network, UMTS capacity need to be monitored, expanded & manage regularly.
In the first part of document (Part-1) UMTS soft capacity management along with the correlation between capacity vs accessibility, UMTS expansion threshold, counters & effective mechanism of UMTS Soft capacity management have been depicted. This part of the Document is focused on UMTS Hard Expansion Methodology. 3G Capacity management by Sectorization has been explained thoroughly.
After completing this, you will be able to gain knowledge on 3G Expansion Parameters, 3G Expansion Threshold, Capacity Gain, Cost vs Capacity Gain by Sectorization. Moreover, you will be familiar with Planning, Optimization & Post Performance activities of Sectorization.
In 5-2020, I and Minh have a sharing session about Sync, Async, and Multi-threading in C#.
We believe this is one of the most important things when working with .Net technology.
This sharing is aiming to Junior Dev, or even Sr Dev or anyone interested in Sync, Async, and Multi-threading.
I hope it will bring you some values.
The presentation covers the basics of packet forwarding and simplified architecture of the router. Additionally it explains what problem Cisco Express Forwarding (CEF) solves and how. At the end static routing is covered.
Delivered by Dmitry Figol, CCIE R&S #53592.
Presentació duta a terme el 30 de juny de 2017 al CSUC a càrrec de Walt Wollny, d’Hurricane Electric, on s'han comentat les eines per a l’estudi de paràmetres BGP d’Hurricane Electric, un dels membres del Punt Neutre d’Internet a Catalunya (CATNIX).
How does the Facebook Messenger app achieve phone-to-phone messaging latency in the order of milliseconds instead of seconds? Answer: It uses the MQTT protocol. And so can you.
In this session we look at the MQTT protocol and explain why it in many cases is a much better choice than HTTP or push notification for your mobile communication needs. Using the MQTT protocol your mobile app can achieve secure, reliable two-way communication without killing battery or wasting precious bandwidth. And it’s open source!
Using Mikko Koppanen's PHP ZMQ extension we will look at how you can easily distribute work to background processes, provide flexible service brokering for your next service oriented architecture, and manage caches efficiently and easily with just PHP and the ZeroMQ libraries. Whether the problem is asynchronous communication, message distribution, process management or just about anything, ZeroMQ can help you build an architecture that is more resilient, more scalable and more flexible, without introducing unnecessary overhead or requiring a heavyweight queue manager node.
Redis & ZeroMQ: How to scale your applicationrjsmelo
Presented at #PHPLX 11 July 2013
When you need to do some heavy processing how do you scale you application?
You can use Redis and ZeroMQ to leverage the heavy work for you!
With this presentation we will know more about this two technologies and how they can be used to help solve problems with the performance and scalability of your application.
Some slides demonstrating what ZeroMQ is and how it can be used from Scala, with the native Scala-ZeroMQ binding or the Akka-zeromq module.
Acually, code examples are on a GitHub repository here: https://github.com/fanf/scala_zeromq.
The presentation was given on 2013-08-21 at the Paris Scala User Group.
ZeroMQ e Redis: soluzioni open source per l'integrazione di componentiMatteo Fortini
Realizzare software complessi con componenti semplici che interagiscono in forme ben definite è da sempre il modo per ottenere architetture robuste ed efficienti.
ZeroMQ e Redis permettono di implementare questo paradigma ed estenderlo alle reti, ai diversi linguaggi, alle diverse architetture hardware.
ZMQ socket based networking library, which can also be molded as broker-less messaging framework. The slides talks about how zmq been woven to work with openstack services.
Slides used at CNCF Paris Meetup 02/15/18.
This covers how we setup Prometheus at Deezer and his architecture. We also give some configuration examples and tweaks.
0507 057 01 98 * Adana Klima Servisleri, Adana Klima Servisi, Adana Klima Servisleri, Arçelik Klima Servisleri Adana, Beko Klima Servisleri Adana, Demirdöküm Klima Servisleri Adana, Vestel Klima Servisleri Adana, Aeg Klima Servisleri Adana, Bosch Klima Servisleri Adana, Ariston Klima Servisleri Adana, Samsung Klima Servisleri Adana, Siemens Klima Servisleri Adana, Profilo Klima Servisleri Adana, Fujitsu Klima Servisleri Adana, Baymak Klima Servisleri Adana, Sharp Klima Servisleri Adana, Mitsubishi Klima Servisleri Adana, Alaska Klima Servisleri Adana, Aura Klima Servisleri Adana, Adana Çukurova Klima Servisleri, Adana Yüreğir Klima Servisleri, Adana Seyhan Klima Servisleri
Kernel Recipes 2014 - NDIV: a low overhead network traffic diverterAnne Nicolas
NDIV is a young, very simple, yet efficient network traffic diverter. Its purpose is to help build network applications that intercept packets at line rate with a very low processing overhead. A first example application is a stateless HTTP server reaching line rate on all packet sizes.
Willy Tarreau, HaproxyTech
Because this system is web application (partially)
Because we based (100%) on FOSS (open-source)
Because security matters
Because OWASP people cares about security and can affect recruitment processes (hopefully) ;)
Linux offers an extensive selection of programmable and configurable networking components from traditional bridges, encryption, to container optimized layer 2/3 devices, link aggregation, tunneling, several classification and filtering languages all the way up to full SDN components. This talk will provide an overview of many Linux networking components covering the Linux bridge, IPVLAN, MACVLAN, MACVTAP, Bonding/Team, OVS, classification & queueing, tunnel types, hidden routing tricks, IPSec, VTI, VRF and many others.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
2. Outline
● Who we are
● Quick tour of ZeroRPC features
● Implementation overview
3. Introduction: why?
● We are running a PAAS:
we deploy, monitor, and scale your apps
(in the Cloud!)
● Many moving parts
● ... On a large distributed cluster
4. API endpoint Users (auth)
CLI
Formation
(analyzes stack)
Uploader
(code store)
Containers
(intances)
Deployments
(services)
Cluster Node
Cluster Node
Cluster Node
Cluster Node Host Info
VHosts
(HTTP)
NATS
(TCP/UDP)
Load
Balancer
Load
Balancer
Logger
Metrics
Dashboard
Internal tools
dotCloud architecture (simplified) — components, servers, things receiving data from everything else, me
5. Our requirements (easy)
● We want to be able to expose arbitrary code,
with minimal modification (if any)
If we can do import foo; foo.bar(42),
we want to be able to do foo=RemoteService(...) ;
foo.bar(42)
● We want a self-documented system
We want to see methods, signatures, docstrings, without
opening the code of the remote service, or relying on
(always out-dated) documentation
6. Our requirements (harder)
● We want to propagate exceptions properly
● We want to be language-agnostic
● We want to be brokerless, highly available,
fast, and support fan-in/fan-out topologies
(Not necessarily everything at the same time!)
● We want to trace & profile nested calls
Which call initiated the subcall which raised SomeException?
Which subcall of this complex call causes it to take
forever?
7. Why not {x} ?
● x=HTTP
o too much overhead for some use-cases
(logs and metrics)
o continuous stream of updates requires extra work
(chunked encoding, websockets...)
o asynchronous notification requires extra work
(long polling, websockets...)
o fan-out topology requires extra work
(implementing some kind of message queue)
● x=AMQP (or similar)
o too much overhead for some use-cases
8. What we came up with
ZeroRPC!
● Based on ZeroMQ and MessagePack
● Supports everything we needed!
● Multiple implementations
o Internal "reference" implementation in Python
o Public "alternative" implementation with gevents
o Internal Node.js implementation (so-so)
9. Example: unmodified
code
Expose the "urllib" module over RPC:
$ zerorpc-client --server --bind tcp://0:1234 urllib
Yup. That's it.
There is now something listening on TCP port
1234, and exposing the Python "urllib"
module.
10. Example: calling code
From the command line (for testing):
$ zerorpc-client tcp://0:1234 quote "hello pycon"
"hello%20pycon"
From Python code:
>>> import zerorpc
>>> remote_urllib = zerorpc.Client( )
>>> remote_urllib.connect('tcp://0:1234')
>>> remote_urllib.quote('hello pycon')
'hello%20pycon'
11. Example: introspection
We can list methods:
$ zerorpc-client tcp://localhost:1234 | grep ^q
quote quote('abc def') -> 'abc%20def'
quote_plus Quote the query fragment of a URL; replacing ' ' with '+'
We can see signatures and docstrings:
$ zerorpc-client tcp://localhost:1234 quote_plus - ?
quote_plus(s, safe='')
Quote the query fragment of a URL; replacing ' ' with '+'
12. Example: exceptions
$ zerorpc-client tcp://localhost:1234 quote_plus
Traceback (most recent call last): File
"/home/jpetazzo/.virtualenvs/dotcloud_develop/bin/zerorpc-client", line 131, in <module>
main()
File "/home/jpetazzo/.virtualenvs/dotcloud_develop/bin/zerorpc-client", line 127, in main
pprint(client(args.command,*parameters))
File "/home/jpetazzo/Work/DOTCLOUD/dotcloud/zerorpc/zerorpc.py",line 362, in __call__
return self.recv()
File "/home/jpetazzo/Work/DOTCLOUD/dotcloud/zerorpc/zerorpc.py",line 243, in recv
ret = self.handler(headers, method, *args)
File "/home/jpetazzo/Work/DOTCLOUD/dotcloud/zerorpc/zerorpc.py",line 323, in handle_result
raise eTypeError: quote( ) takes at least 1 argument (0 given)
13. Example: load balancing
Start a load balancing hub:
$ cat foo.yml
in: "tcp://*:1111"
out: "tcp://*:2222"
type: queue
$ zerohub.py foo.yml
Start (at least) one worker:
$ zerorpc-client --server tcp://localhost:2222 urllib
Now connect to the "in" side of the hub:
$ zerorpc-client tcp://localhost:1111
14. Example: high availability
Start a local HAProxy in TCP mode, dispatching requests
to 2 or more remote services or hubs:
$ cat haproxy.cfg
listen zerorpc 0.0.0.0:1111
mode tcp
server backend_a localhost:2222 check
server backend_b localhost:3333 check
$ haproxy -f haproxy.cfg
Start (at least) one backend:$ zerorpc-client --server --bind
tcp://0:2222 urllib
Now connect to HAProxy:$ zerorpc-client tcp://localhost:1111
15. Non-Example: PUB/SUB
(not in public repo—yet)
● Broadcast a message to a group of nodes
o But if a node leaves and rejoins, he'll lose messages
● Send a continuous stream of information
o But if a speaker or listener leaves and rejoins...
You generally don't want to do this!
Better pattern: ZeroRPC streaming with
gevents
16. Example: streaming
● Server code returns an list iterator
● Client code gets an list iterator
● Small messages, high latency? No problem!
o Server code will pre-push elements
o Client code will notify server if pipeline runs low
● Huge messages? No problem!
o Big data sets can be nicely chunked
o They don't have to fit entirely in memory
o Don't worry about timeouts anymore
● Also supports long polling
18. Implementation details
This will be useful if...
● You think you might want to use ZeroRPC
● You think you might want to hack ZeroRPC
● You want to reimplement something similar
● You just happen to love distributed systems
19. ØMQ
● Sockets on
steroidshttp://zguide.zeromq.org/page:all
● Handles (re)connections for us
● Works over regular TCP
● Also has superfast ipc:// and inproc://
● Different patterns:
o REQ/REP (basic, synchronous RPC call +
response)
o PUB/SUB (shout and listen to streams of events)
o PUSH/PULL (load balance or collect messages)
o DEALER/ROUTER (REQ/REP with routing)
20. Serialization:
MessagePack
In our tests, msgpack is more efficient than
JSON, BSON, YAML:
● 20-50x faster
● serialized output is 2x smaller or better
$ pip install msgpack-python
>>> import msgpack
>>> bytes = msgpack.dumps(data)
21. Wire format
Request: (headers, method_name, args)
● headers dict
o no mandatory header
o carries the protocol version number
o used for tracing in our in-house version
● args
o list of arguments
o no named parameters
Response: (headers, ERR|OK|STREAM, value)
22. Timeouts
● ØMQ does not detect disconnections
(or rather, it works hard to hide them)
● You can't know when the remote is gone
● Original implementation: 30s timeout
● Published implementation: heartbeat
23. Introspection
● Expose a few special calls:
o _zerorpc_list to list calls
o _zerorpc_name to know who you're talking to
o _zerorpc_ping (redundant with the previous one)
o _zerorpc_help to retrieve the docstring of a call
o _zerorpc_args to retrieve the argspec of a call
o _zerorpc_inspect to retrieve everything at once
● Introspection + service discovery = WIN
24. Naming
● Published implementation does not include
any kind of naming/discovery
● In-house version uses a flat YAML file,
mapping service names to ØMQ addresses
and socket types
● In progress: use DNS records
o SRV for host+port
o TXT for ØMQ socket type (not sure about this!)
● In progress: registration of services
o Majordomo protocol
25. Security: there is none
● No security at all in ØMQ
o assumes that you are on a private, internal network
● If you need to run "in the wild", use SSL:
o bind ØMQ socket on localhost
o run stunnel (with client cert verification)
● In progress: authentication layer
● dotCloud API is actually ZeroRPC,
exposed through a HTTP/ZeroRPC gateway
● In progress: standardization of this gateway
26. Tracing (not published
yet)
● Initial implementation during a hack day
o bonus: displays live latency and request rates,
using http://projects.nuttnet.net/hummingbird/
o bonus: displays graphical call flow,
using http://raphaeljs.com/
o bonus: send exceptions to airbrake/sentry
● Refactoring in progress, to "untie" it from the
dotCloud infrastructure and Open Source it
How it works: all calls and responses are
logged to a central place, along with a
trace_id unique to each sequence of calls.
27. Tracing: trace_id
● Each call has a trace_id
● The trace_id is propagated to subcalls
● The trace_id is bound to a local context
(think thread local storage)
● When making a call:
o if there is a local trace_id, use it
o if there is none ("root call"), generate one (GUID)
● trace_id is passed in all calls and responses
Note: this is not (yet) in the github repository
28. Tracing: trace collection
● If a message (sent or received) has a
trace_id, we send out the following things:
o trace_id
o call name (or, for return values, OK|ERR+exception)
o current process name and hostname
o timestamp
Internal details: the collection is built on top of
the standard logging module.
29. Tracing: trace storage
● Traces are sent to a Redis key/value store
o each trace_id is associated with a list of traces
o we keep some per-service counters
o Redis persistence is disabled
o entries are given a TTL so they expire automatically
o entries were initially JSON (for easy debugging)
o ... then "compressed" with msgpack to save space
o approximately 16 GB of traces per day
Internal details: the logging handler does not talk directly to
Redis; it sends traces to a collector (which itself talks to
Redis).
30. The problem with being
synchronous
● Original implementation was synchronous
● Long-running calls blocked the server
● Workaround: multiple workers and a hub
● Wastes resources
● Does not work well for very long calls
o Deployment and provisioning of new cluster nodes
o Deployment and scaling of user apps
Note: this is not specific to ZeroRPC
31. First shot at
asynchronicity
● Send asynchronous events & setup
callbacks
● "Please do foo(42) and send the result to this
other place once you're done"
● We tried this. We failed.
o distributed spaghetti code
o trees falling in the forest with no one to hear them
● Might have worked better if we had...
o better support in the library
o better naming system
o something to make sure that we don't lose calls
32. Gevent to the rescue!
● Write synchronous code
(a.k.a. don't rewrite your services)
● Uses coroutines to achieve concurrency
● No fork, no threads, no problems
● Monkey patch of the standard library
(to replace blocking calls with async
versions)
● Achieve "unlimited" concurrency server-side
The version published on github uses gevent.
33. Show me the code!
https://github.com/dotcloud/zerorpc-python
$ pip install git+git://github.com/dotcloud/zerorpc-python.git
Has: zerorpc module, zerorpc-client helper,
exception propagation, gevent integration
Doesn't have: tracing, naming, helpers for
PUB/SUB & PUSH/PULL, authentication