11th International Conference COMPUTER DATA ANALYSIS & MODELING 2016 Theoretical & Applied Stochastics September, 6-10, 2016 Minsk, Belarus

1. ASSIGNMENT OF ARBITRARILY
DISTRIBUTED RANDOM SAMPLES TO THE
FIXED PROBABILITY DISTRIBUTION AND
ITS RISK
E.E. Zhuk, D.D. Dus
Computer Data Analysis & Modeling
Minsk, 2016
1

2. Introduction
1. Let random samples be determined in the observation space :
2. And let some fixed (hypothetical) probability density be determined:
3. The problem is to construct decision rule for assignment one of (1) to (2):
2
N
R
( ) ( )
1{ } : ( ) 0, , 1, ;
( ) 1
i
N
ni i N
t t i
i
R
X x p x x i mR
p x dx
      
 
( ) 0, ;
( ) 1
N
N
R
p x x R
p x dx
  
 
(1)
(2)
(1) ( )
( ,..., ) , {1, , }m
d d X X M M m   
2m 
(3)

3. Maximum likelihood decision rule
• Proposed to use decision rule based on maximum likelihood
principle:
where - fixed “hypothetical” density from (2).
(1) ( ) ( )
( ) ( )
1
( ,..., ) argmax ( );
( ) ( ), ;
i
m i
i M
n
i i
t
t
d d X X P X
P X p x i M


 
  
3
(4)
( )p 

4. Theorem 1:
1. Let the following integrals be finite:
2. Let the only one of values be greater than others:
3. Suppose, that all samples have the same size:
Then following statement is true for decision rule (4):
4
 ln ( ) ( ) , .N iR
p x p x dx i M   
(6)
a.s.(1) ( )
( ,..., ) , .m o
d d X X d n     (8)
 
0
( ( ), ( )) ln ( ) ( ) , ,
argmax .
Ni i iR
i
i M
H H p p p x p x dx i M
d H

     


(5)
, .in n i M   (7)
iH
(1) ( )
,..., m
X X

5. Risk generalization
• Traditional risk generalization for the case of “same-sized”
samples:
5
 
 
(1) ( ) (1) ( )
( ( ,..., )) P{ ( ,..., ) };
arg max ;
( ( ), ( )) ln ( ) ( ) , ,
1,..., .
N
m m o
o
j
j M
i i iR
r r d X X d X X d
d H
H H p p p x p x dx i M
M m

  

     


(9)

6. Two samples of the same size
Maximum likelihood rule can be rewritten as:
And its risk takes form:
6
(1) (2)
(1) (2)
(1) (2)
(2)
(1) (2)
(1)
1
1, if ( , ) 0;
( , )
2, if ( , ) 0;
( )1
( , ) ln ;
( )
n
n
n
t
n
t t
X X
d X X
X X
p x
X X
n p x




 
 

 
  
 

(10)
(1) (2)
1 2
(1) (2)
1 2
1 2
P{ ( , ) 0}, if ;
1-P{ ( , ) 0}, if ;
0 , if .
n
n
X X H H
r X X H H
H H


  

  
 
(11)

7. Two samples of the same size: asymptotical risk
Theorem 2:
1. Let consider the assignment problem of two samples of the
same size.
2. And let for densities (1), (2) the following be true:
Then the risk (11) can be evaluated asymptotically:
7
  
2
2
ln ( ) ( ) ;
0, 1,2.
Ni iR
i i
G p x p x dx
G H i
  
   
 (12)
1 2
2 2
1 2 1 2
, ;
.
( )
r r n
H H
r n
G G H H
   
 
   
    
(13)

8. Asymptotical risk: Fisher model
Suppose the case of Fisher model:
Then:
where – Mahalanobis distance.
And asymptotical risk can be evaluated analytically:
8
( ) ( | , ), 1,2;
( ) ( | , ).
i N i
N
p x n x i
p x n x


   
 
(14)
 
1
22 2
2 2
1
ln 2 ( , ) , 1,2;
2
1
2 ( , ) , 1,2;
2
N
i i
i i i
R N i
G H N i
   
  
  
         
 
    
(15)
(16)
2 2
1 2
2 2
1 2
( , ) ( , )
.
2 ( , ) ( , )
r n
N
     
     
 
   
   
(17)
(x,y)

9. For the case of arbitrary Gaussian densities:
We have:
9
( ) ( | , ), 1,2;
( ) ( | , ).
i N i i
N
p x n x i
p x n x


   
 
(18)
1
1 22 2
1
ln 2 ( ) ( , ) , 1,2;
2
N
i i iH tr i   
   
           
 
(19)
(20)
 
2 2
1
1
( )ln ( ) ln( ( )) , 1,2;
( | , ).
N
K
i i j iR
j
j N i i
G p x p x dx p z G i
K
z n x 

    


1 2
2 2
1 2 1 2
,
( )
H H
r n
G G H H
 
   
    
Asymptotical risk: arbitrary Gaussians

10. Thanks for your
attention!