This document describes carepo, a content addressable repository for Named Data Networking (NDN) that improves content distribution. Carepo identifies identical content that appears under different names by computing hashes of content chunks. It maintains a hash index at repositories and publishes hash lists from producers. This allows consumers to find identical chunks cached on nearby nodes, reducing download times by up to 38% compared to NDN in tests with Linux ISO files. Carepo increases publishing time by around 4x to generate chunk hashes and indexes but this is a one-time cost outweighed by faster distribution to multiple consumers.
Named Data Networking, for Computer Communications course presentation
pictures are cropped from that slides:
http://www.slideshare.net/wanderer_from/named-date?qid=1abab327-219a-4b69-a114-46e7f1634d42&v=qf1
http://www.slideshare.net/haroonrashidlone/named-data-networking?qid=bb7c7b7b-ee1b-4c2f-8df5-c4194282e8e2&v=qf1
http://named-data.net/content-centric-networking-video/
https://hal.inria.fr/file/index/docid/785298/filename/AIMS12_tutorial_CCN.pdf
he Named Data Networking (NDN) project proposed an evolution of the IP architecture that generalizes the role of this thin waist, such that packets can name objects other than communication endpoints. More specifically, NDN changes the semantics of network service from delivering the packet to a given destination address to fetching data identified by a given name. The name in an NDN packet can name anything – an endpoint, a data chunk in a movie or a book, a command to turn on some lights, etc. The hope is that this conceptually simple change allows NDN networks to apply almost all of the Internet’s well-tested engineering properties to broader range of problems beyond end-to-end communications.
Named Data Networking, for Computer Communications course presentation
pictures are cropped from that slides:
http://www.slideshare.net/wanderer_from/named-date?qid=1abab327-219a-4b69-a114-46e7f1634d42&v=qf1
http://www.slideshare.net/haroonrashidlone/named-data-networking?qid=bb7c7b7b-ee1b-4c2f-8df5-c4194282e8e2&v=qf1
http://named-data.net/content-centric-networking-video/
https://hal.inria.fr/file/index/docid/785298/filename/AIMS12_tutorial_CCN.pdf
he Named Data Networking (NDN) project proposed an evolution of the IP architecture that generalizes the role of this thin waist, such that packets can name objects other than communication endpoints. More specifically, NDN changes the semantics of network service from delivering the packet to a given destination address to fetching data identified by a given name. The name in an NDN packet can name anything – an endpoint, a data chunk in a movie or a book, a command to turn on some lights, etc. The hope is that this conceptually simple change allows NDN networks to apply almost all of the Internet’s well-tested engineering properties to broader range of problems beyond end-to-end communications.
How to connect FIWARE to Robots ? We discuss how the FIWARE enablers can connect to ROS2, a de facto standard for robotic frameworks, using Fast RTPS and KIARA.
Implementation is the stage of the project when the theoretical design is turned out into a working system. Thus it can be considered to be the most critical stage in achieving a successful new system and in giving the user, confidence that the new system will work and be effective.
The implementation stage involves careful planning, investigation of the existing system and it’s constraints on implementation, designing of methods to achieve changeover and evaluation of changeover methods.
Main Modules:-
1. User Module:
In this module, Users are having authentication and security to access the detail which is presented in the ontology system. Before accessing or searching the details user should have the account in that otherwise they should register first.
2. Secure DeDuplication System:
To support authorized deduplication, the tag of a file F will be determined by the file F and the privilege. To show the difference with traditional notation of
tag, we call it file token instead. To support authorized access, a secret key kp will be bounded with a privilege p to generate a file token. Let ϕ′ F;p = TagGen(F, kp) denote the token of F that is only allowed to access by user with privilege p. In another word, the token ϕ′ F;p could only be computed by the users with privilege p. As a result, if a file has been uploaded by a user with a duplicate token ϕ′
F;p, then a duplicate check sent from another user will be successful if and only if he also has the file F and privilege p. Such a token generation function could be
easily implemented as H(F, kp), where H(_) denotes a cryptographic hash function.
3. Security Of Duplicate Check Token :
We consider several types of privacy we need protect, that is, i) unforgeability of duplicate-check token: There are two types of adversaries, that is, external adversary and internal adversary. As shown below, the external adversary
can be viewed as an internal adversary without any privilege. If a user has privilege p, it requires that the adversary cannot forge and output a valid duplicate token with any other privilege p′ on any file F, where p does not match p′. Furthermore, it also requires that if the adversary does not make a request of token with its own privilege from private cloud server, it cannot forge and output a valid duplicate token with p on any F that has been queried.
4. Send Key:
Once the key request was received, the sender can send the key or he can decline it. With this key and request id which was generated at the time of sending key request the receiver can decrypt the message.
This document contains study of Peer to Peer Distributed system.Three Models of Distributed system.Such as Centralizes,Decentralized,Hybird Model and Pros and cons of these models. Skpye and Bit torrent architecture is also discussed.This tutorial can be very help full for those who are beginners.
Abstract: File replication is an effective way to enhance file availability and reduce file querying delay. It creates replicas for a file to improve its probability of being encountered by requests. Here, we introduce a new concept of resource for file replication, which considers both node storage and node meeting ability. We theoretically study the influence of resource allocation on the average querying delay and derive an optimal file replication rule (OFRR) that allocates resources to each file based on its popularity and size. We then propose a file replication protocol based on the rule, which approximates the minimum global querying delay in a fully distributed manner.
APNIC Chief Scientist, Geoff Huston, gives a presentation on DOH and the changing nature of the DNS as infrastructure at NZNOG 2020 in Christchurch, New Zealand, from 28 to 31 January 2020.
LAN Technologies
Ethernet and IEEE Token Ring
IEEE Token Ring
Message transfer in OSI reference Model
Example of Message Transfer in the OSI Model
Application Layer Protocols.
A constructive review of in network caching a core functionality of icn slidesAnshuman Kalla
In-Network Caching in Information Centric Networking (ICN) with Content Centric Networking (CCN) as key design architecture. Aim is to make an exhaustive review of work related to in-network caching in ICN.
How to connect FIWARE to Robots ? We discuss how the FIWARE enablers can connect to ROS2, a de facto standard for robotic frameworks, using Fast RTPS and KIARA.
Implementation is the stage of the project when the theoretical design is turned out into a working system. Thus it can be considered to be the most critical stage in achieving a successful new system and in giving the user, confidence that the new system will work and be effective.
The implementation stage involves careful planning, investigation of the existing system and it’s constraints on implementation, designing of methods to achieve changeover and evaluation of changeover methods.
Main Modules:-
1. User Module:
In this module, Users are having authentication and security to access the detail which is presented in the ontology system. Before accessing or searching the details user should have the account in that otherwise they should register first.
2. Secure DeDuplication System:
To support authorized deduplication, the tag of a file F will be determined by the file F and the privilege. To show the difference with traditional notation of
tag, we call it file token instead. To support authorized access, a secret key kp will be bounded with a privilege p to generate a file token. Let ϕ′ F;p = TagGen(F, kp) denote the token of F that is only allowed to access by user with privilege p. In another word, the token ϕ′ F;p could only be computed by the users with privilege p. As a result, if a file has been uploaded by a user with a duplicate token ϕ′
F;p, then a duplicate check sent from another user will be successful if and only if he also has the file F and privilege p. Such a token generation function could be
easily implemented as H(F, kp), where H(_) denotes a cryptographic hash function.
3. Security Of Duplicate Check Token :
We consider several types of privacy we need protect, that is, i) unforgeability of duplicate-check token: There are two types of adversaries, that is, external adversary and internal adversary. As shown below, the external adversary
can be viewed as an internal adversary without any privilege. If a user has privilege p, it requires that the adversary cannot forge and output a valid duplicate token with any other privilege p′ on any file F, where p does not match p′. Furthermore, it also requires that if the adversary does not make a request of token with its own privilege from private cloud server, it cannot forge and output a valid duplicate token with p on any F that has been queried.
4. Send Key:
Once the key request was received, the sender can send the key or he can decline it. With this key and request id which was generated at the time of sending key request the receiver can decrypt the message.
This document contains study of Peer to Peer Distributed system.Three Models of Distributed system.Such as Centralizes,Decentralized,Hybird Model and Pros and cons of these models. Skpye and Bit torrent architecture is also discussed.This tutorial can be very help full for those who are beginners.
Abstract: File replication is an effective way to enhance file availability and reduce file querying delay. It creates replicas for a file to improve its probability of being encountered by requests. Here, we introduce a new concept of resource for file replication, which considers both node storage and node meeting ability. We theoretically study the influence of resource allocation on the average querying delay and derive an optimal file replication rule (OFRR) that allocates resources to each file based on its popularity and size. We then propose a file replication protocol based on the rule, which approximates the minimum global querying delay in a fully distributed manner.
APNIC Chief Scientist, Geoff Huston, gives a presentation on DOH and the changing nature of the DNS as infrastructure at NZNOG 2020 in Christchurch, New Zealand, from 28 to 31 January 2020.
LAN Technologies
Ethernet and IEEE Token Ring
IEEE Token Ring
Message transfer in OSI reference Model
Example of Message Transfer in the OSI Model
Application Layer Protocols.
A constructive review of in network caching a core functionality of icn slidesAnshuman Kalla
In-Network Caching in Information Centric Networking (ICN) with Content Centric Networking (CCN) as key design architecture. Aim is to make an exhaustive review of work related to in-network caching in ICN.
About the author: Priya Autee is software engineer at Intel working on various leading edge IA features and Intel(R) RDT expert. She is focused on prototyping and researching open source APIs like DPDK, Intel(R) RDT etc. to support NFV/compute sensitive requirements on Intel Architecture. She holds Masters in Computer Science from Arizona State University, Arizona.
LinuxCon2009: 10Gbit/s Bi-Directional Routing on standard hardware running Linuxbrouer
This talk my 2009 updates on the progress of doing 10Gbit/s routing on standard hardware running Linux. The results are good, BUT to achieve these results, a lot of tuning is required of hardware queues, MSI interrupts and SMP affinity, together with some (now) submitted patches. I\'ll explain the concept of network hardware queues and why interrupt and SMP tuning is essential. I\'ll present results from different hardware both 10GbE netcards and CPUs (current CPUs under test is AMD phenom and Core i7). Many future challenges still exists, especially in the area of more easy tuning. A high knowledge level about the Linux kernel is required to follow all the details.
An overview of Haystack's security features for low power IoT networks. Unlike most IoT stacks, when Haystack invented DASH7, security was an a priori principle and led to the most secure networking stack available in the low power, wide area networking (LPWAN) space today.
Introduction to Real Application Cluster
RAC - Savior of DBA
Oracle Clusterware (Platform on Platform)
RAC Startup sequence
RAC Architecture
RAC Components
Single Instance on RAC
Node Eviction
Important Log directories in RAC.
Tips to monitor and improve the RAC environment.
Faraz Ahmad and T.N. Vijaykumar, Joint Optimization of Idle and Cooling Power in Data Centers While Maintaining Response Time
presented in Green Computing class, 05NOV2012
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
2. Background: Named Data Networking
Today’s Internet is primarily used for content distribution
Named Data Networking (NDN), an emerging future
Internet architecture, makes Data the first class entity
NDN has a receiver-driven communication model
Consumer sends Interest packet (request)
Producer replies Data packet (response)
Interest
Interest
Interest
Data
Data
Data
3. NDN universal caching
Router opportunistically caches Data packets
Cached Data packets are used to satisfy future
Interests with the same Name
Data packet crosses each link only once
Every Data packet carries a signature
so it could be verified regardless of whether it’s from
producer or from a cache
Interest
Data from cache
4. Caching relies on naming
cached: Linux Mint 15 MATE 64-bit DVD, segment 0
request 1: Linux Mint 15 MATE 64-bit DVD, segment 0
OK, satisfy from cache
request 2: Linux Mint Olivia MATE 64-bit DVD, segment 0
codename of Linux Mint 15
Router does not know they are the same
5. Problem: same payload under
different Names
numeric version vs codename
slightly updated file: different version marker, most
chunks unchanged
tape archive (TAR) vs individual files
web content: HTML / XML / plain text
6. Scenario
People in a local area network download files from a
remote repository
Identical payload appears in those files under
different Names
We want to identify identical payload in Data packets
in order to shorten download completion time, and
save bandwidth
7. Solution
Producer
publish file chunks as Data packets
publish a hash list
Repository
index Data packets by Name
index Data packets by payload hash
Consumer
fetch the hash list, and search local and nearby
repositories for Data packets with same payload
download unfulfilled segments from remote repository
8. server
Internet
local area network
client
hash list
0: 4004 octets,
1: 2100 octets,
2: 4200 octets,
3: 2100 octets,
hash1
hash2
hash3
hash2
need 3 unique chunks
0: 4004 octets, hash1
1,3: 2100 octets, hash2
2: 4200 octets, hash3
SHA256 hash collision is unlikely.
If two Data packets have the same
payload hash, we assume they
have identical payload.
name request(s)
0 1 2 3
hash1? hash request(s)
hash2?
hash index
hash3?
hash1 hash3
9. Hash request & Name request
Hash request
/%C1.R.SHA256/hash
neighbor scope (1-hop),
multicast to local area
network
Name request
/repo/filename/version
/segment
global scope, forward
toward remote repository
concurrency: 30
concurrency: 10
timeout: 500ms
timeout: 4000ms
no retry, send Name
request after timeout
retry twice
10. Chunking
We want to maximize number of identical chunks
Fixed chunking is not resistant to insertions
A
R
I
Z
O
9FB3313F
C
S
.
B8858AB9
A
R
I
N
A
.
C1ED0864
Z
17229319
O
N
A
E
D
U
CC868CDF
.
9163767A
E
D
U
363F6587
This illustration shows the first 32 bits of MD5 hash. carepo uses stronger SHA256 hash.
11. This is a simplification.
The actual Rabin fingerprint
chunking calculates a rolling
hash for every 31-octet window,
and claims a boundary when
the hash ends with several zeros.
Rabin fingerprint chunking
Rabin fingerprint chunking selects chunk boundary
according to content, not offset
Let’s claim end of chunk on every period
A
R
I
Z
O
N
A
.
D9318D04
C
S
.
3B630D26
A
R
I
Z
O
D9318D04
E
D
U
CC868CDF
N
A
.
E
D
U
CC868CDF
This illustration shows the first 32 bits of MD5 hash. carepo uses stronger SHA256 hash.
12. Chunk size is not arbitrary in network
Chunks are enclosed in Data packets
packet too large: inefficient or infeasible to transmit
packet too small: higher overhead in network
Rabin configuration
average chunk size: 4096 octets
min/max chunk size: [1024,8192] octets
13. Trust model
In NDN, every Data packet must carry a signature
Publisher only needs to RSA-sign the hash list
Chunks don’t need strong signatures, because they can
be verified by hash
hash list
0: 4004 octets,
1: 2100 octets,
2: 4200 octets,
3: 2100 octets,
hash1
hash2
hash3
hash2
20. CCNx inter-file similarity
Client has ALL prior versions: need to download 55.3%
chunks
Client has ONE immediate prior version: need to
download 60.3% chunks
Duplicate chunk percentage varies with each version
21. What about compressed TAR.GZ?
intra-file similarity: NONE
DEFLATE algorithm has duplicate string elimination
inter-file similar - client has ALL prior versions: need to
download 98.2% chunks
22. Linux Mint ‘Olivia’
MATE 64-bit
MATE no-codecs 64-bit
filename
linuxmint-15-mate-dvd64bit.iso
linuxmint-15-mate-dvdnocodecs-64bit.iso
size
1000MB
981MB
media
DVD
DVD
package base
Ubuntu Raring
Ubuntu Raring
desktop
MATE
MATE
video playback
included
not included
23. Linux Mint analysis
MATE 64-bit
number of chunks
chunk
size
MATE no-codecs 64-bit
238436
233852
average
4398
4399
standard
deviation
2460
2460
235509
231270
intra-file unique chunks
inter-file unique chunks
254276
If a client already has MATE 64-bit locally, only 18767 chunks need to
be downloaded in order to construct MATE no-codecs 64-bit.
25. Deployment on virtual machines
slow link
–2.5Mbps, 20ms delay
0.5Mbps, 20ms delay–
local area network
fast links
simulated by NetEm
server
gateway
clients
28. Download time: Linux Mint
1.
2.
download MATE 64-bit (1000MB) onto client1
download MATE no-codecs 64-bit (981MB) onto client2
download time (s)
0
500
1000
1500
2000
2500
3000
3500
carepo
ndn
MATE 64-bit
MATE no-codecs 64-bit
total download time for two files: carepo is 38% less than ndn
4000
4500
30. Publishing time
MATE no-codecs 64-bit
0
100
200
300
400
MATE 64-bit
500
600
700
800
900
1000
ndnputfile->ndnr
overhead of Rabin chunking
caput(signed)->ndnr
benefit of omitting strong signatures
caput->ndnr
overhead of computing hash again
at repo, and maintaining hash index
caput->car
not a big problem
• server: publish once, serve many clients
• client: file is available on download completion; publish to help neighbors
32. Conclusion
NDN universal caching relies on Naming, but identical
payload may appear under different Names
identify identical payload by hash
Repository maintains hash index;
Producer publishes hash list;
Client finds identical payload on nearby nodes by hash
Download time is reduced by 38% for two DVD images
Publishing time is increased to 3.8x