This document discusses noise, information theory, and entropy as they relate to communication systems. It introduces key concepts such as additive noise channels, random processes like white Gaussian noise, signal-to-noise ratio, analog vs. digital signals, information theory measures like entropy and channel capacity, and coding techniques including Huffman coding. Huffman coding is presented as an optimal variable-length coding scheme that assigns codes based on symbol probabilities to achieve an average code length close to the entropy limit.
Here is the presentation for Data Link Layer Numericals from the book Andrew S. Tanenbaum (Computer Networks) and B A Forouzan ( Data Communication and Networking)
By reading this you can enhance your knowledge about Data Communication Network and Redundancy check used for it for error detection. It only Detect the error and discard it from the sequence given in that codes.
Here is the presentation for Data Link Layer Numericals from the book Andrew S. Tanenbaum (Computer Networks) and B A Forouzan ( Data Communication and Networking)
By reading this you can enhance your knowledge about Data Communication Network and Redundancy check used for it for error detection. It only Detect the error and discard it from the sequence given in that codes.
Asynchronous Transfer Mode ATM is the cell relay protocol designed by ATM Forum and adopted by the ITU-T. Cell, a small fixed size block of information with asynchronous TDM ensures high speed real time transmission with efficient and cheaper technology. Instead of user addresses, it uses virtual circuit identifier and virtual path identifier, which can be repeated at unrelated locations. This technology ensures connectivity to much more users than normal packet switching networks.
ATM and ISDN-B combination allows high-speed interconnection of world's network.
In multicast communication, there is one source and a group of destination.
In multicasting, the router may forward the received packet through several of its interfaces.
The source address is a unicast address, but destination address is a group address.
In Digital Communications, a Turbo Equalizer is a type of receiver used to receive a message corrupted by a communication channel with intersymbol interference (ISI).
The sender initializes the checksum to 0 and adds all data items and the checksum. However, 36 cannot be expressed in 4 bits. The extra two bits are wrapped and added with the sum to create the wrapped sum value 6. The sum is then complemented, resulting in the checksum value 9 (15 − 6 = 9).
This presentation enables users to understand basics of Information Theory, Entropy, Binary channels, channel capacity and error condition in easy and detailed manner. Concepts are explained properly using derivations and examples.
Question paper of Pre-University Examination of Analog Communication held at Guru Nanak Education Trust Group of Institution, Roorkee in EVEN Semester Session: 2012-13
Asynchronous Transfer Mode ATM is the cell relay protocol designed by ATM Forum and adopted by the ITU-T. Cell, a small fixed size block of information with asynchronous TDM ensures high speed real time transmission with efficient and cheaper technology. Instead of user addresses, it uses virtual circuit identifier and virtual path identifier, which can be repeated at unrelated locations. This technology ensures connectivity to much more users than normal packet switching networks.
ATM and ISDN-B combination allows high-speed interconnection of world's network.
In multicast communication, there is one source and a group of destination.
In multicasting, the router may forward the received packet through several of its interfaces.
The source address is a unicast address, but destination address is a group address.
In Digital Communications, a Turbo Equalizer is a type of receiver used to receive a message corrupted by a communication channel with intersymbol interference (ISI).
The sender initializes the checksum to 0 and adds all data items and the checksum. However, 36 cannot be expressed in 4 bits. The extra two bits are wrapped and added with the sum to create the wrapped sum value 6. The sum is then complemented, resulting in the checksum value 9 (15 − 6 = 9).
This presentation enables users to understand basics of Information Theory, Entropy, Binary channels, channel capacity and error condition in easy and detailed manner. Concepts are explained properly using derivations and examples.
Question paper of Pre-University Examination of Analog Communication held at Guru Nanak Education Trust Group of Institution, Roorkee in EVEN Semester Session: 2012-13
Application of recording instrument, Servo System, Advantage of interferometric circuit in recorder, Curvilinear and rectilinear ecumenism, Construction and working of galvanometric recorders, Automatic balance in recording instrument. Prepared By: Paras Prateek Bhatanagar, B.Tech (EEE) 6th Semester (2009-2010), College Of Engineering Roorkee.
Standard Signal Generator (sine wave), AF and square wave generator, Function generator, Spectrum Analyzer, Harmonic Analyzer, Frequency Synthesizer, X-Y Recorders, Digital X-Y Recorders (Plotters), Electro-encephalo-graphy (EEG), Electrocardiogram (ECG) Book Referred: R K Rajput, "Electronic Measurement & Instrumentation", S. Chand Publication.
A Practical Report on Introduction to Filters under labVIEW Environment during Third Semester of M.Tech (ICE) during session 2014-15, ODD Semester at Sant Longowal Institute of Engineering and Technology, Longowal, INDIA.
Signals and Systems
What is a signal?
Signal Basics
Analog / Digital Signals
Real vs Complex
Periodic vs. Aperiodic
Bounded vs. Unbounded
Causal vs. Noncausal
Even vs. Odd
Power vs. Energy
Noise reduction is the process of removing noise from a signal. In this project, two audio files are given: (1) speech.au and (2) noisy_speech.au. The first file contains the original speech signal and the second one contains the noisy version of the first signal. The objective of this project is to reduce the noise from the noisy file
Echo and reverberation effects are used extensively in the music industry. Here we will design a digital filter that will create the echo and reverb effect on audio signals.
Introduction to Channel Capacity | DCNIT-LDTalks-1Arunabha Saha
DCNIT-LDTalks-1
Here I have discussed the channel capacity for noiseless and noisy channels. How Nyquist capacity and Shannon capacity play a key role in the noiseless and noisy channels are discussed in detail. We will see the several expressions of SNR_dB in terms of power and amplitude and try to understand how both the capacity are different from each other. For extreme values of SNR, we will deduce the Shannon capacity formula to understand the bandwidth-limited region and power-limited region. Here I have used a few numerical examples to understand the concept clearly. In the last section of the talk, I have deduced the Shannon capacity formula from scratch to get better exposure and will understand how these ideas contribute to its mathematical framework.
Course: CMS-A-CC-4-8
youtube: https://www.youtube.com/watch?v=1OjlMqHWq6o
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
20 Comprehensive Checklist of Designing and Developing a WebsitePixlogix Infotech
Dive into the world of Website Designing and Developing with Pixlogix! Looking to create a stunning online presence? Look no further! Our comprehensive checklist covers everything you need to know to craft a website that stands out. From user-friendly design to seamless functionality, we've got you covered. Don't miss out on this invaluable resource! Check out our checklist now at Pixlogix and start your journey towards a captivating online presence today.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
National Security Agency - NSA mobile device best practices
Noise info theory and Entrophy
1. Noise, Information Theory,
and Entropy
CS414 – Spring 2007
By Roger Cheng
(Huffman coding slides courtesy
of Brian Bailey)
2. Why study noise?
It’s present in all
systems of interest,
and we have to deal
with it
By knowing its
characteristics, we
can fight it better
Create models to
evaluate analytically
4. The additive noise channel
Transmitted signal s(t)
is corrupted by noise
source n(t), and the
resulting received signal
is r(t)
Noise could result form
many sources, including
electronic components
and transmission
interference
n(t)
+s(t) r(t)
5. Random processes
A random variable is the result of a single
measurement
A random process is a indexed collection of
random variables, or equivalently a non-
deterministic signal that can be described by
a probability distribution
Noise can be modeled as a random process
6. WGN (White Gaussian Noise)
Properties
At each time instant t = t0, the value of n(t)
is normally distributed with mean 0,
variance σ2
(ie E[n(t0)] = 0, E[n(t0)2
] = σ2
)
At any two different time instants, the
values of n(t) are uncorrelated (ie
E[n(t0)n(tk)] = 0)
The power spectral density of n(t) has
equal power in all frequency bands
7. WGN continued
When an additive noise channel has a white
Gaussian noise source, we call it an AWGN
channel
Most frequently used model in
communications
Reasons why we use this model
It’s easy to understand and compute
It applies to a broad class of physical channels
8. Signal energy and power
Energy is defined as
Power is defined as
Most signals are either finite energy and zero
power, or infinite energy and finite power
Noise power is hard to compute in time domain
Power of WGN is its variance σ2
2
= | ( ) | dtx x tε
∞
−∞
∫
/2
2
/2
1
= lim | ( ) |
T
x
T
T
P x t dt
T→∞
−
∫
9. Signal to Noise Ratio (SNR)
Defined as the ratio of signal power to the
noise power corrupting the signal
Usually more practical to measure SNR on a
dB scale
Obviously, want as high an SNR as possible
10. SNR example - Audio
Original sound file is
CD quality (16 bit,
44.1 kHz sampling
rate)
Original 40 dB
20 dB 10 dB
11. Analog vs. Digital
Analog system
Any amount of noise will create distortion at the
output
Digital system
A relatively small amount of noise will cause no
harm at all
Too much noise will make decoding of received
signal impossible
Both - Goal is to limit effects of noise to a
manageable/satisfactory amount
12. Information theory and
entropy
Information theory tries to
solve the problem of
communicating as much
data as possible over a
noisy channel
Measure of data is
entropy
Claude Shannon first
demonstrated that
reliable communication
over a noisy channel is
possible (jump-started
digital age)
14. Properties of entropy
Can be defined as the expectation of log p(x)
(ie H(X) = E[-log p(x)])
Is not a function of a variable’s values, is a
function of the variable’s probabilities
Usually measured in “bits” (using logs of base
2) or “nats” (using logs of base e)
Maximized when all values are equally likely
(ie uniform distribution)
Equal to 0 when only one value is possible
Cannot be negative
15. Joint and conditional entropy
Joint entropy is the entropy of the
pairing (X,Y)
Conditional entropy is the entropy of X if
the value of Y was known
Relationship between the two
17. Mathematical model of a
channel
Assume that our input to the channel is
X, and the output is Y
Then the characteristics of the channel
can be defined by its conditional
probability distribution p(y|x)
18. Channel capacity and rate
Channel capacity is defined as the
maximum possible value of the mutual
information
We choose the best f(x) to maximize C
For any rate R < C, we can transmit
information with arbitrarily small
probability of error
19. Binary symmetric channel
Correct bit transmitted with probability 1-p
Wrong bit transmitted with probability p
Sometimes called “cross-over probability”
Capacity C = 1 - H(p,1-p)
20. Binary erasure channel
Correct bit transmitted with probability 1-p
“Erasure” transmitted with probability p
Capacity C = 1 - p
21. Coding theory
Information theory only gives us an upper
bound on communication rate
Need to use coding theory to find a practical
method to achieve a high rate
2 types
Source coding - Compress source data to a
smaller size
Channel coding - Adds redundancy bits to make
transmission across noisy channel more robust
22. Source-channel separation
theorem
Shannon showed that when dealing
with one transmitter and one receiver,
we can break up source coding and
channel coding into separate steps
without loss of optimality
Does not apply when there are multiple
transmitters and/or receivers
Need to use network information theory
principles in those cases
23. Huffman Encoding
Use probability distribution to determine
how many bits to use for each symbol
higher-frequency assigned shorter codes
entropy-based, block-variable coding
scheme
24. Huffman Encoding
Produces a code which uses a minimum
number of bits to represent each symbol
cannot represent same sequence using fewer real
bits per symbol when using code words
optimal when using code words, but this may differ
slightly from the theoretical lower limit
Build Huffman tree to assign codes
25. Informal Problem Description
Given a set of symbols from an alphabet and
their probability distribution
assumes distribution is known and stable
Find a prefix free binary code with minimum
weighted path length
prefix free means no codeword is a prefix of any
other codeword
26. Huffman Algorithm
Construct a binary tree of codes
leaf nodes represent symbols to encode
interior nodes represent cumulative probability
edges assigned 0 or 1 output code
Construct the tree bottom-up
connect the two nodes with the lowest probability
until no more nodes to connect
28. Characteristics of Solution
Lowest probability symbol is
always furthest from root
Assignment of 0/1 to children
edges arbitrary
other solutions possible; lengths
remain the same
If two nodes have equal
probability, can select any two
Notes
prefix free code
O(nlgn) complexity
Symbol
(S)
Code
A 11
B 00
C 010
D 011
E 10
30. Entropy (Theoretical Limit)
= -.25 * log2 .25 +
-.30 * log2 .30 +
-.12 * log2 .12 +
-.15 * log2 .15 +
-.18 * log2 .18
H = 2.24 bits
∑=
−=
N
i
ii spspH
1
)(log)( 2
Symbol P(S)
Cod
e
A 0.25 11
B 0.30 00
C 0.12 010
D 0.15 011
E 0.18 10
31. Average Codeword Length
= .25(2) +
.30(2) +
.12(3) +
.15(3) +
.18(2)
L = 2.27 bits
∑=
=
N
i
ii scodelengthspL
1
)()(
Symbol P(S)
Cod
e
A 0.25 11
B 0.30 00
C 0.12 010
D 0.15 011
E 0.18 10
32. Code Length Relative to
Entropy
Huffman reaches entropy limit when all
probabilities are negative powers of 2
i.e., 1/2; 1/4; 1/8; 1/16; etc.
H <= Code Length <= H + 1
∑=
−=
N
i
ii spspH
1
)(log)( 2∑=
=
N
i
ii scodelengthspL
1
)()(
33. Example
H = -.01*log2.01 +
-.99*log2.99
= .08
L = .01(1) +
.99(1)
= 1
Symbol P(S) Code
A 0.01 1
B 0.99 0
34. Limitations
Diverges from lower limit when probability of
a particular symbol becomes high
always uses an integral number of bits
Must send code book with the data
lowers overall efficiency
Must determine frequency distribution
must remain stable over the data set