Information inequalities are very useful and play a fundamental role in the literature of Information Theory. Applicati ons of information inequalities have discussed by well - kn own authors like as Dragomir, Taneja and many researchers etc. In this research paper, we shall consider some new functional information inequalities in the form of generalized information divergence measures. We shall also consider relations between Csiszar’s f - divergence, new f - divergence and other well - known divergence measures using information inequalities. Numerical bounds of information divergence measure have also studied

1. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
DOI : 10.5121/ijit.2015.4204 31
Some Generalized Information Inequalities
Ram Naresh Saraswat
Department of Mathematics, ManipalUniveristy, Jaipur
Rajasthan-303007 INDIA,
ABSTRACT
Information inequalities are very useful and play a fundamental role in the literature of Information
Theory. Applications of information inequalities have discussed by well-known authors like as Dragomir,
Taneja and many researchers etc. In this research paper, we shall consider some new functional
information inequalities in the form of generalized information divergence measures. We shall also
consider relations between Csiszar’s f-divergence, new f-divergence and other well-known divergence
measures using information inequalities. Numerical bounds of information divergence measure have also
studied.
KEYWORDCsiszar’s f-divergence, New f-divergence measure, Relative information of type’s, J-divergence of
type’s, Relative J-divergence of type’setc.
1. INTRODUCTION
Let
1, 2,
1
( ........ ) 0, 1 , 2
n
n n i i
i
P p p p p p n

 
      
 

be the set of all complete finite discrete probability distributions. There are many information and
divergence measures are exist in the literature of Information Theory and Statistics. Csiszar [2] &
[3] introduced a generalized measure of information using f-divergence measure given by
1
( , )
n
i
f i
i i
p
I P Q q f
q
 
  
 
 (1.1)
where :f  R R is a convex function and , nP Q .
As in Csiszar [3], we have interpret undefined expressions by Csiszar’s f-divergence is a general
class of divergence measures that includes several divergences used in measuring the distance or
affinity between two probability distributions. This class is introduced by using a convex function
f, defined on (0, ∞).
Here we give some examples of divergence measures which are the category of Csiszar’s f-
divergence measure like as Bhattacharya divergence [1], Triangular discrimination [4], Relative
J-divergence [5], Hellinger discrimination [6], Chi-square divergence [9], Relative Jensen-
Shannon divergence [13], Relative arithmetic-geometric divergence measure [10], Unified
relative Jensen-Shannon and arithmetic-geometric divergence measure[10].

2. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
32
Now, we give some examples of well-knowngeneralized information divergence measures of
type’s which are obtained from Csiszar’s f-divergence measure.
 Relative information of type s [11]
The following measures and particular cases are introduced by Taneja [11]
 
12 1
1
1
1
( , ) ( 1) 1 , 0,1
( , ) ( , ) log , 0
( , ) log , 1
n
s s
s i i
i
n
i
s i
i i
n
i
i
i i
K P Q s s p q s
q
P Q D Q P q s
p
p
D P Q p s
q
 



  
     
 
  
     
 
     
  



(1.2)
and
1
1
1
1
( 1) ( ) , 1
( , )
( , ) ( )log , 1
s
n
i
i i
i i
s
n
i
i i
i i
p
s p q s
q
P Q
p
J P Q p q s
q





  
    
   
 
   
 


(1.3)
 Relative J-divergence of type s [12]
 
1
1
1
1
( , ) 1 ( ) , 1
2
( , )
( , ) log , 1
s
n
i i
s i i
i i
s
n
i
i
i i
p q
D P Q s p q s
q
P Q
p
J P Q p s
q





  
     
   
 
  
 


(1.4)
2. NEW F-DIVERGENCE MEASURE
In this section we shall consider some properties of a new f-divergence measure [Jain and
Saraswat [7] &[8] and its particular cases which are may be interesting in areas of information
theory is given by
1
( , )
2
n
i i
f i
i i
p q
S P Q q f
q
 
  
 
 (2.1)
Where :f  R R is a convex functionand , nP Q  .
It is shown that using new f-divergence measure we derive some well-known divergence
measures such as Chi-square divergence, Relative J-divergence, Jenson-Shannon’s divergence,
Triangular discrimination, Hellinger discrimination, Bhattacharya divergence, Unified relative
Jensen-Shannon and arithmetic-geometric divergence of type’setc. in this section.
The following propositions are presented by Jain &Saraswat in [7] & [8].

3. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
33
Proposition 2.1 Let :[0, )f   R be convex and , nP Q with 1n nP Q  then we have
the following inequality
 ( , ) 1fS P Q f (2.2)
Equality holds in (2.2) iff
1,2,..,i ip q i n   (2.3)
Corollary 2.1.1 (Non-negativity of new f-divergence measure) Let :[0, )f   R be convex
and normalized, i.e.
(1) 0f  (2.4)
Then for any , nP Q from (2.2) of proposition 2.1 and (2.4), we have the inequality
( , ) 0fS P Q  (2.5)
If f is strictly convex, equality holds in (2.5) iff
 ,2,................i ip q i i n   (2.6)
In particular, if P & Q are probability vectors, then Corollary 2.1.1 shows that for a strictly
convex and normalized :[0, )f   R
( , ) 0fS P Q  and ( , ) 0fS P Q  iff P Q (2.7)
Proposition 2.2 Let 1 2&f f are two convex functions and 1 2g a f b f  then
1 2
( , ) ( , ) ( , )g f fS P Q aS P Q bS P Q  , where , nP Q .
We now give some examples of well-known information divergence measures which are obtained
from new f-divergence measure.
 Chi-square divergence measure: - If  
2
( ) 1f t t  then Chi-square divergence measure is
given by
2
2
1
1 1
( , ) 1 ( , )
4 4
n
i
f
i i
p
S P Q P Q
q


 
   
 
 (2.8)
 Relative Jensen-Shannon divergence measure:-If ( ) logf t t  then relative Jensen-Shannon
divergence measure is given by
1
2
( , ) log ( , )
n
i
f i
i i i
q
S P Q q F Q P
p q
 
  
 
 (2.9)

4. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
34
 Relative arithmetic-geometric divergence measure:-If ( ) logf t t t thenrelative arithmetic-
geometric divergence measure is given by
1
( , ) log ( , )
2 2
n
i i i i
f
i i
p q p q
S P Q G Q P
q
   
   
   
 (2.10)
 Triangular discrimination: - If
2
( 1)
( ) , 0
t
f t t
t

   then Triangular discrimination is given
by
 
2
1
( ) 1
( , ) ( , )
2 2
n
i i
f
i i i
p q
S P Q P Q
p q

  

 (2.11)
 Relative J-divergence measure: - If ( ) ( 1)logf t t t  then Relative J-divergence
measure is given by
1
1
( , ) log ( , )
2 2 2
n
i i i i
f R
i i
p q p q
S P Q J P Q
q
   
   
   
 (2.12)
 Hellinger discrimination: - If ( ) (1 )f t t  then Hellinger discrimination is given by
( , ) 1 , ,
2 2
f
P Q P Q
S P Q B Q h Q
      
      
    
(2.13)
3. INFORMATION INEQUALITIES
The following Theorems 3.1 presented in Taneja&Pranesh Kumar [11] and Theorem 3.2
presented in Jain&Saraswat [7] respectively.
Theorem 3.1Let :f  R R be the differentiable convex function and normalized i.e.
(1) 0f  . Then for all , nP Q we have the following inequality
0 ( , ) ( , )ff II P Q W P Q  (3.1)
where '
1
( , ) ( )f
n
i
I i i
i i
p
W P Q p q f
q
 
   
 

In addition, if we have  0 , 1,2,..............,i
i
p
r R i n
q
       for some r and R with
0 1r R     ,then the followings holds
 
1
( , ) ( , ) ( ) '( ) '( )
4ff II P Q W P Q R r f R f r    (3.2)

5. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
35
 
( 1) ( ) (1 ) ( ) 1
( , ) ( ) '( ) '( )
4
f
R f r r f R
I P Q R r f R f r
R r
  
   

(3.3)
Theorem 3.2:-Let :f I   R R be a differentiable convex mapping on
0
I . If
0
ix I and
.
, nP Q . Then we have the following inequality,
1
1
0 ( , ) (1) ( ) '
2 2
n
i i
f i i
i i
p q
S P Q f p q f
q
 
     
 
 (3.4)
where
1
( , )
2
n
i i
f i
i i
p q
S P Q q f
q
 
  
 

and 'f is derivative of f . If f is strictly convex, and , 0,( 1,2,........ )i ip q i n  then the
equality holds in (3.4) iff 1 2
1 2
................ n
n
pp p
q q q
  .
Corollary 3.1If function f is normalized i.e. (1) 0f  , then we have the following inequality
1
0 ( , ) '
2 2
n
i i i i
f
i i
p q p q
S P Q f
q
   
    
   

'
0 ( , ) ( , )ff SS P Q E P Q  (3.5)
where '
1
( , ) '
2 2f
n
i i i i
S
i i
p q p q
E P Q f
q
   
   
   

4.NEW FUNCTIONAL INFORMATION INEQUALITIES
In this section we shall consider some new functionalinformation inequalities among various f-
divergences. Using these functional information inequalities, we shall establish relations between
well-known generalized information divergence measures and its particular cases.
Theorem 4.1Let :f I   R R be the differentiable convex function and normalized i.e.
(1) 0f  . Then for all , nP Q we have the following inequality
 '
1
( , ) ( , ) ( , ) ( , ) ( ) '( ) '( )
4f ff S f IS P Q E P Q I P Q W P Q R r f R f r      (4.1)
'
( 1) ( ) (1 ) ( )
( , ) ( , ) ( , )ff S f
R f r r f R
S P Q E P Q I P Q
R r
  
  

(4.2)
 '
( 1) ( ) (1 ) ( ) 1
( , ) ( , ) ( , ) ( ) '( ) '( )
4ff S f
R f r r f R
S P Q E P Q I P Q R r f R f r
R r
  
     

(4.3)
Proof: - As f is differential convex on
0
I , then for all
0
,x y I , we have the inequality

6. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
36
( ) '( ) ( ) ( )x y f y f x f y   ,
0
,x y I 
Now we take
1
2
x
y


1 1 1
' ( )
2 2 2
x x x
x f f x f
       
       
     
1 1 1
' ( )
2 2 2
x x x
f f x f
       
      
     
(4.4)
Put i
i
p
x
q

'
2 2 2
i i i i i i i
i i i i
p q p q p p q
f f f
q q q q
         
        
       
Multiplying by iq and taking summation both side then we get
1 1 1
'
2 2 2
n n n
i i i i i i i
i i
i i ii i i
p q p q p p q
f q f q f
q q q  
        
       
       
  
'
( , ) ( , ) ( , )fS f fE P Q I P Q S P Q 
'
( , ) ( , ) ( , )fS f fE P Q S P Q I P Q  (4.5)
Hence we get
'
( , ) ( , )fS fE P Q I P Q (4.6)
From (3.5)& (4.5), we get
'
( , ) ( , ) ( , )ff S fS P Q E P Q I P Q  (4.7)
Form (3.1) & (4.7), we get
'
( , ) ( , ) ( , ) ( , )f ff S f IS P Q E P Q I P Q W P Q   (4.8)
From (3.2) & (4.7) give the result (4.1)and (3.3) & (4.8) give the results (4.2) & (4.3)
respectively.
5. APPLICATIONS IN INFORMATION THEORY
In this section we shall establish relationship between various known generalized information
measures of type’s in the form of inequality using the results (4.3), (4.4) and (4.5) of Theorem

7. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
37
(4.1).In the following Theorem we shall consider the applications of functional information
inequalities for relations between Unified relative Jensen-Shannon and arithmetic-geometric
divergence of type’s, Relative J-divergence of type’s, Relative information of type’s etc.
Theorem: 5.1- Let , nP Q , then we have the following relations
( , )1
( , ) ( , ) ( , ) ( , )
2
r R
s s s s sQ P P Q P Q P Q U       (5.1)
( , )1
( , ) ( , ) ( , )
2
r R
s s s sQ P P Q P Q T     (5.2)
( , ) ( , )1
( , ) ( , ) ( , )
2
r R r R
s s s s sQ P P Q P Q T U      (5.3)
where
( , ) 1 11 ( )
4 ( 1)
r R s s
s
R r
U R r
s
 
   
,  
1( , ) ( 1) ( 1) (1 )( 1)
( 1)
s s
r R
s
R r r R
T s s
R r
     
 

Proof: - Considering the mapping :(0, )f   R
 
1
( ) ( 1) ( 1) 0,1s
sf t s s t if s

    (5.4)
1 1
'( ) [ 1] s
f t s t 
  ,
2
''( ) s
f t t 

"( ) 0, 0f t t   and (1) 0f  , So function f is convex and normalized.
Then
( , ) ( , )f sS P Q Q P  (5.5)
'
1
( , ) ( , )
2fS sE P Q P Q (5.6)
( , ) ( , )f sI P Q P Q  (5.7)
( , ) ( , )fI sW P Q P Q (5.8)
 
1
( ) ( 1) ( 1) 0,1s
sf R s s R if s

    (5.9)
 
1
( ) ( 1) ( 1) 0,1s
sf r s s r if s

    (5.10)
From equation (5.4), (5.9) & (5.10), we get
 
1( , ) ( 1) ( ) (1 ) ( ) ( 1) ( 1) (1 )( 1)
( 1)
s s
r R
s
R f r r f R R r r R
T s s
R r R r
       
  
 
(5.11)
 ( , ) 1 11 ( )
( ) '( ) '( )
4 ( 1)
r R s s
s
R r
U R r f R f r R r
s
 
      
(5.12)

8. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
38
Using equation (4.1), (4.2), (4.3) (4.4), (5.4),(5.5),(5.6),(5.7),(5.8),(5.9),(5.10) (5.11) & (5.12)
give the relation (5.1), (5.2) & (5.3).
In the following corollaries, we shall discuses some cases for particular values of
in corollary (5.1), (5.2), (5.3) and (5.4) respectively.
Corollary 5.1:-If 1s  
2
2 2
2
2 2 2
1 1
( )1 1 1 ( ) ( )
( , ) ( , ) ( )
4 ( ) 2 8
n n
i i i i
i i
i ii i i
p p q p R r R r
P Q P Q q p
p q q R r

 
   
      
  
  (5.13)
2
2
2
1
( )1 1 1 ( 1)(1 ) (1 )(1 )
( , ) ( , )
4 ( ) 2 2 ( )
n
i i i
i i i
p p q r R r r R
P Q P Q
p q r R R r


     
   
 
 (5.14)
2 2
2
2 2 2
1
( )1 1 1 ( 1)(1 ) (1 )(1 ) 1 ( ) ( )
( , ) ( , )
4 ( ) 2 2 ( ) 8
n
i i i
i i i
p p q r R r r R R r R r
P Q P Q
p q r R R r R r


       
    
 
 (5.15)
Corollary 5.2If 0s 
 21 1
( , ) ( , ) ( , ) ( , ) log log
2 4
R r
F P Q P Q D P Q P Q r R R r
Rr


      (5.16)
1
( , ) ( , ) ( , ) ( 1)log ( 1)log
2
F P Q P Q D P Q R r r R       (5.17)
 
1 1
( , ) ( , ) ( , ) ( 1)log ( 1)log log log
2 4
R r
F P Q P Q D P Q R r r R r R R r
Rr

         (5.18)
Corollary 5.3 If 1 2s 
1
12
1 1
4 1 , ( , ) 4 ( , ) 2 ( ) ( )
2 2 2 2
n
i
i i
i i
qP Q P Q r R
B Q Q h P Q q p r R
p Rr


      
          
     

(5.19) 1
2
1 (1 )( 1) (1 )( 1)
4 1 , ( , ) 4 ( , ) 4
2 2 2
P Q P Q R r r R
B Q Q h P Q
R r

      
   

   
        
(5.20)
1
4 1 , ( , ) 4 ( , )
12 2 2
2
P Q P Q
B Q Q h P Q
 
  
  
    
(1 )( 1) (1 )( 1) 1
4 ( )
2
R r r R r R
r R
R r R r
     
  

  
  
   
(5.21)

9. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
39
Corollary 5.4 For 1s 
1 1
( , ) ( , ) ( , ) ( , ) ( )log
2 4
R
R
G P Q J P Q D P Q J P Q R r
r
 
      
 
(5.22)
 
1
( , ) ( , ) ( , ) ( 1) log log
2
RG P Q J P Q D P Q R r r R R     (5.23)
 
1 1
( , ) ( , ) ( , ) ( 1) log log ( )log
2 4
R
R
G P Q J P Q D P Q R r r R R R r
r
 
        
 
(5.24)
6. NUMERICAL ILLUSTRATIONS
In this section we shall consider some numerical bounds of f-divergence measure using functional
information inequalities and binomial distribution.
Example 6.1
Let P be the binomial probability distribution for the random valuable X with parameter (n=8
p=0.5) and Q its approximated normal probability distribution.
Table 6.1 Binomial Probability Distribution
(n=8 p=0.5)
x 0 1 2 3 4 5 6 7 8
p (x) .004 .031 .109 .219 .274 .219 .109 .031 .004
q (x) .005 .030 .104 .220 .282 .220 .104 .030 .005
)x(q
)x(p
.774 1.042 1.0503 .997 .968 .997 1.0503 1.042 .774
It is noted that ,
 
1( , )
(.05) (.77) 1 (.33) (1.05) 1( 1) ( ) (1 ) ( )
( 1)
(.28)
s s
r R
s
R f r r f R
T s s
R r
              

 
(.77,1.05)
5 (.77) 1 33 (1.05) 1
28 ( 1)
s s
sT
s s
        

(.77,1.05)
1 .1533,T  (.77,1.05)
0 0.01047,T  (.77,1.05)
1 2 .20T 
(.77,1.05) 1 11 (1.05 .77)
(1.05) (.77)
4 ( 1)
s s
sU
s
 
   
(.77,1.05)
1 .026819,U  (.77,1.05)
0 .011,U  (.77,1.05)
1 2 .023,U  (.77,1.05)
1 .009U 

10. International Journal on Information Theory (IJIT),Vol.4, No.2, April 2015
40
REFERENCES
[1]. Bhattacharya A., “Some analogues to amount of information and their uses in statistical estimation”,
Sankhya8 (1946) 1-14
[2]. Csiszar I. Information measure,A critical servey.Trans.7th
prague conf. on info. Th. Statist. Decius.
Funct, Random Processes and 8th
European meeting of statist Volume B. Acadmia Prague, 1978, 73-
86.
[3]. Csiszar I., Information-type measures of difference of probability functions and indirect observations.
Studia Sci.Math.hunger.2(1961).299-318.
[4]. Dacunha-Castella D., “Ecoled’Ete de probabilities de Saint-Flour VII-1977 Berline, Heidelberg”,
New-York:Springer 1978
[5]. Dragomir S.S., V. Gluscevic and C.E.M. Pearce, “Approximation for the Csiszar f –divergence via
mid-point inequalities, in inequality theory and applications- Y. J. Cho, J. K. Kim and S.S. dragomir
(Eds), nova science publishers, inc., Huntington, new York, vol1, 2001, 139-154.
[6]. Hellinger E., “NeueBegrundung der Theorie der
quadratischenFormenVonunendlichenvielenveranderlichen”, J.Rein.Aug.Math,136(1909),210-271
[7]. Jain K. C. and R. N. Saraswat “A New Information Inequality and its Application in Establishing
Relation among various f-Divergence Measures”, Journal of Applied Mathematics, Statistics and
Informatics, 8(1) (2012), 17-32.
[8]. Jain K. C. and R. N. Saraswat“Some bounds of information divergence measure in term of Relative
arithmetic-geometric divergence measure” International Journal of Applied Mathematics and
Statistics, 32,(2) (2013), 48-58.
[9]. Pearson K., “On the criterion that a give system of deviations from the probable in the case of
correlated system of variables in such that it can be reasonable supposed to have arisen from random
sampling”, Phil. Mag., 50(1900),157-172.
[10]. Taneja I.J., “New Developments in generalized information measures”, Chapter in: Advances in
imaging and Electron Physics, Ed. P. W. Hawkes 91 (1995), 37-135.
[11]. Taneja I. J. and Pranesh Kumar, “Relative Information of type’s, Csiszar f-divergence and
information inequalities”, Information Sciences, 166(1-4) (2004), 105-12
[12]. Taneja I. J. and Pranesh Kumar, “Generalized non-symmetric divergence measures and
inequalities”.The Natural Science and Engineering Research Council’s Discovery grant to Pranesh
Kumar
[13]. Sibson R., Information Radius, Z, Wahrs. undverw.geb. (14) (1969), 149-160.
[14]. Wang Yufeng Wang Wendong“A Novel Interest Coverage Method Based On Jensen-Shannon
Divergence In Sensor Networks” 24(4) Journal of Electronics (China) July 2007.