Neural Networks inNeural Networks in
ECG classificationECG classification
Under the guidance ofUnder the guidance of
Prof. P. BhattacharyaProf. P. Bhattacharya
Nishant ChandraNishant Chandra
Mrigen NegiMrigen Negi
Meru A PatilMeru A Patil
LayoutLayout
 History of Neural networks in medicalHistory of Neural networks in medical
 Need for accurate processingNeed for accurate processing
 Applications of ANN in medicalApplications of ANN in medical
 What is ECG?What is ECG?
 ANN in classification of ArrhythmiasANN in classification of Arrhythmias
and Ischemiaand Ischemia
 ConclusionConclusion
History of Neural Networks inHistory of Neural Networks in
MedicalMedical
 Pioneering work of neural networkPioneering work of neural network
has started since 1943 by McCullochhas started since 1943 by McCulloch
and Pitts.and Pitts.
 Pattern recognition problem wasPattern recognition problem was
introduced by Rosenblatt (1958)introduced by Rosenblatt (1958)
Need for accurate processingNeed for accurate processing
 One of the major goals of observationalOne of the major goals of observational
studies in medicine is to identify patterns instudies in medicine is to identify patterns in
complex data sets.complex data sets.
 Correct classification of heart beats isCorrect classification of heart beats is
fundamental to ECG monitoring systems suchfundamental to ECG monitoring systems such
as an intensive care etc.as an intensive care etc.
 Computers are used to automate signalComputers are used to automate signal
processing.processing.
 ANNs can detect patterns and makeANNs can detect patterns and make
distinctions between different patterns thatdistinctions between different patterns that
may not be apparent to human analysis.may not be apparent to human analysis.
Applications of ANN in medicalApplications of ANN in medical
 It has been successfully applied to variousIt has been successfully applied to various
areas of medicine to solve non-linearareas of medicine to solve non-linear
problems.problems.
 Applications include prediction of diagnosisApplications include prediction of diagnosis
such as:such as:
– CancerCancer
– the onset of diabetes mellitusthe onset of diabetes mellitus
– survival prediction in AIDSsurvival prediction in AIDS
– eating disorders etceating disorders etc
 Applications in signal processing andApplications in signal processing and
interpretation involve ECGs orinterpretation involve ECGs or
electrocardiogramselectrocardiograms
MotivationMotivation
 Cardiovascular Diseases contributeCardiovascular Diseases contribute
29.3% of total deaths in world.29.3% of total deaths in world.
 Online ECG monitoring in ICUs/CCUs.Online ECG monitoring in ICUs/CCUs.
 Acting Specialist in emergency cases.Acting Specialist in emergency cases.
 Each component (P,QRS,T waves)Each component (P,QRS,T waves)
has different frequencies.has different frequencies.
 Each individual is different.Each individual is different.
 Learning by experience.Learning by experience.
What is ElectrocardiogramWhat is Electrocardiogram
(ECG) ?(ECG) ?
 ECG is the graphic recording of electricECG is the graphic recording of electric
potentials generated by the heart.potentials generated by the heart.
 12 lead ECG12 lead ECG
 3 bipolar limb leads – I, II, III3 bipolar limb leads – I, II, III
 3 unipolar augmented limb leads - AVF, AVR,3 unipolar augmented limb leads - AVF, AVR,
AVLAVL
 6 unipolar chest leads – V1 to V6.6 unipolar chest leads – V1 to V6.
Anatomy of Heart and ECG signalAnatomy of Heart and ECG signal
Normal ECG signal
Conducting System of Heart
Posterior
Anterior
Limb leads orientation with
respect to heart
Chest leads orientation with
respect to heart
The 12 Views of the Heart
12 Lead Normal ECG
6 Limb leads 6 Chest leads
RR
ECG and diseasesECG and diseases
Some of the diseases diagnosed bySome of the diseases diagnosed by
ECG are:ECG are:
 Myocardial Ischemia/Infarction.Myocardial Ischemia/Infarction.
 Arrhythmias.Arrhythmias.
 Hypertrophy and enlargement of heart.Hypertrophy and enlargement of heart.
 Conduction Blocks.Conduction Blocks.
 Preexcitation Syndromes.Preexcitation Syndromes.
 Other cardiac disorders.Other cardiac disorders.
Did you know !!Did you know !!
 In heart Transplant Acute heartIn heart Transplant Acute heart
rejection is more likely to happenrejection is more likely to happen
when the heart donor was femalewhen the heart donor was female
regardless of recipient sex.regardless of recipient sex.
 Every 34 seconds, a person diesEvery 34 seconds, a person dies
from Heart Diseases in the Unitedfrom Heart Diseases in the United
States.States.
Myocardial IschemiaMyocardial Ischemia
 Due to lack of adequate blood flow toDue to lack of adequate blood flow to
the myocardium.the myocardium.
 Ischemia is reversible.Ischemia is reversible.
 Changes in ECG:Changes in ECG:
 T wave peakingT wave peaking
 Symmetric T wave inversionSymmetric T wave inversion
 ST segment elevationST segment elevation
Different ECG Signals
Normal Signal ST segment elevated signal
ECG with T wave inversion ECG Signal with peak T waves
Myocardial Ischemia cont..Myocardial Ischemia cont..
ArrhythmiasArrhythmias
 It refers to any disturbance in theIt refers to any disturbance in the
rate, regularity, site of origin, orrate, regularity, site of origin, or
conduction of cardiac electricalconduction of cardiac electrical
impulse.impulse.
 Broadly two types:Broadly two types:
 Tachycardia – Heart Rate beyond 100Tachycardia – Heart Rate beyond 100
bits/minute.bits/minute.
 Bradycardia – Heart Rate below 60Bradycardia – Heart Rate below 60
bits/minute.bits/minute.
Different ECG Signals
Normal ECG Signal
ECG signal of Bradycardia patient
ECG signal of Tachycardia patient
Arrhythmias cont ..
Sensitivity (SE) and Specificity (SP)Sensitivity (SE) and Specificity (SP)
 Helps us to explore the relationshipHelps us to explore the relationship
between a diagnostic test and the (true)between a diagnostic test and the (true)
presence or absence of disease.presence or absence of disease.
 A test which is very sensitive will rarelyA test which is very sensitive will rarely
miss people with the disease.miss people with the disease.
 A specific test will have few false positiveA specific test will have few false positive
results - it will rarely misclassify peopleresults - it will rarely misclassify people
without the disease as being diseased.without the disease as being diseased.
 Classification Rate:Classification Rate:
CC = 100×(TP+TN)/(TN+TP+FN+FP)]
Sensitivity (SE) and Specificity (SP) Cont…Sensitivity (SE) and Specificity (SP) Cont…
ApproachApproach
 Variable attributes considered toVariable attributes considered to
affect the training and generalizationaffect the training and generalization
of the ANNs were identified asof the ANNs were identified as
follows:follows:
– Number of nodes in the hidden layerNumber of nodes in the hidden layer
– Feature Selection method employedFeature Selection method employed
– Number of files in training setNumber of files in training set
– Size of input feature vectorSize of input feature vector
– Number of epochsNumber of epochs
Case StudyCase Study
Feature Extraction:Feature Extraction:
 Fourier TransformFourier Transform
 Principal component analysis (PCA)Principal component analysis (PCA)
– widely used in signal processing,widely used in signal processing,
statistics, and neural computing.statistics, and neural computing.
– basic goal is to reduce the dimension ofbasic goal is to reduce the dimension of
the data.the data.
 Linear Prediction Coding (LPC)Linear Prediction Coding (LPC)
Fourier TransformFourier Transform
 QRS complex is extracted by
applying a window of some time
duration (say 250 ms).
 Each QRS complex is Fourier
transformed and then the power
spectrum is calculated.
 The components generated along
with the temporal vectors give the
feature vector.
QRS spectra of a normal beatQRS spectra of a normal beat
QRS spectra of a Arrhythmia beatQRS spectra of a Arrhythmia beat
PCAPCA
 Step 1: Get some dataStep 1: Get some data
 Step 2: Subtract the meanStep 2: Subtract the mean
 Step 3: Calculate the covarianceStep 3: Calculate the covariance
matrixmatrix
 Step 4: Calculate the eigenvectors andStep 4: Calculate the eigenvectors and
eigenvalues of the covariance matrixeigenvalues of the covariance matrix
 Step 5: Choosing components andStep 5: Choosing components and
forming a feature vectorforming a feature vector
 Step 6: Deriving the new data setStep 6: Deriving the new data set
Linear Prediction Coding (LPC)Linear Prediction Coding (LPC)
 The basic idea of this technique is that
sampled QRS segment can be
approximated as a linear combination of
the past QRS samples.
 a is the i th linear prediction coefficient,
and p is the order of the predictor.
 LPC coefficients can be extracted using
various methods viz Burg’s Method.
Training the NNTraining the NN
 Number of neurons in the input layer is
determined by the number of elements in
the input feature vector.
 The output layer is determined by the
number of classes desired.
 The number of neurons in the hidden layer
varies according to the specific recognition
task and is determined by the complexity
and amount of training data available.
Neural network classifier
architecture
Performance AnalysisPerformance Analysis
 The performance of the neural
classifiers is evaluated by computing
the percentages of:
– sensitivity (SE),
– specificity (SP) and
– correct classification (CC)
ResultsResults
NeuralNeural
ClassifierClassifier
Input LayerInput Layer Hidden LayerHidden Layer
11 1212 55
22 10 3
33 5 2
Results Cont.Results Cont.
NeuralNeural
ClassifierClassifier
(Avrg.)(Avrg.)
CorrectCorrect
classificationclassification
%%
SensitivitySensitivity
%%
SpecificitySpecificity
%%
11 94.8394.83 86.6386.63 94.4294.42
22 91.34 81.33 91.92
33 88.25 76.17 88.95
Results Cont.Results Cont.
 How does ANN based classificationHow does ANN based classification
compare with:compare with:
– Other ECG widely used interpretationOther ECG widely used interpretation
program?program?
Neural networks were 15.5% more sensitiveNeural networks were 15.5% more sensitive
– Expert cardiologistExpert cardiologist
10.5% more sensitive than the cardiologist10.5% more sensitive than the cardiologist
ConclusionConclusion
 Performance of the neural network
strategy has shown higher
performance than other classical
methods (Cox regression models) in
predicting clinical outcomes of the
risk of coronary artery disease.
ReferencesReferences
 [1] M. A. Chikh, F. Bereksi Reguig. Application of
artificial neural networks to identify the
premature ventricular contraction (PVC)
beats,2004
 [2] Costas Papaloukasa, Dimitrios I. Fotiadisb,
Aristidis Likasb, Lampros K. Michalis. An ischemia
detection method based on artificial neural
networks,2002
 [3] C.D. Nugent, J.A.C. Webb, N.D. Black, G.T.H.
Wright, M. McIntyre. An intelligent framework for
the classification of the 12-lead ECG, 1999.
 Introduction to Neural Networks in Healthcare,Introduction to Neural Networks in Healthcare,
Open Clinic, 2002.Open Clinic, 2002.
 [4] M.S. Thaler, The Only EKG Book You’ll Ever[4] M.S. Thaler, The Only EKG Book You’ll Ever
Need 3Need 3rdrd
Edition, Lippincott Williams & Wilkins.Edition, Lippincott Williams & Wilkins.
 P.J Mehta, Understanding ECG, 5P.J Mehta, Understanding ECG, 5thth
Edition, TheEdition, The
National Book Depot.National Book Depot.
Believe it or NOT !!Believe it or NOT !!
 How much blood does your heart pump?How much blood does your heart pump?
– An average heart pumps 2.4 ounces (70An average heart pumps 2.4 ounces (70
milliliters) per heartbeat. An average heartbeatmilliliters) per heartbeat. An average heartbeat
is 72 beats per minute. Therefore an averageis 72 beats per minute. Therefore an average
heart pumps 1.3 gallons (5 Liters) per minute.heart pumps 1.3 gallons (5 Liters) per minute.
In other words it pumps 1,900 gallons (7,200In other words it pumps 1,900 gallons (7,200
Liters) per day, almost 700,000 gallonsLiters) per day, almost 700,000 gallons
(2,628,000 Liters) per year, or 48 million(2,628,000 Liters) per year, or 48 million
gallons (184,086,000 liters) by the timegallons (184,086,000 liters) by the time
someone is 70 years old. That's not bad for asomeone is 70 years old. That's not bad for a
10 ounce pump!10 ounce pump!
 Men suffer heart attacks about 10 yearsMen suffer heart attacks about 10 years
earlier in life than women do.earlier in life than women do.
Ecg

Ecg

  • 1.
    Neural Networks inNeuralNetworks in ECG classificationECG classification Under the guidance ofUnder the guidance of Prof. P. BhattacharyaProf. P. Bhattacharya Nishant ChandraNishant Chandra Mrigen NegiMrigen Negi Meru A PatilMeru A Patil
  • 2.
    LayoutLayout  History ofNeural networks in medicalHistory of Neural networks in medical  Need for accurate processingNeed for accurate processing  Applications of ANN in medicalApplications of ANN in medical  What is ECG?What is ECG?  ANN in classification of ArrhythmiasANN in classification of Arrhythmias and Ischemiaand Ischemia  ConclusionConclusion
  • 3.
    History of NeuralNetworks inHistory of Neural Networks in MedicalMedical  Pioneering work of neural networkPioneering work of neural network has started since 1943 by McCullochhas started since 1943 by McCulloch and Pitts.and Pitts.  Pattern recognition problem wasPattern recognition problem was introduced by Rosenblatt (1958)introduced by Rosenblatt (1958)
  • 4.
    Need for accurateprocessingNeed for accurate processing  One of the major goals of observationalOne of the major goals of observational studies in medicine is to identify patterns instudies in medicine is to identify patterns in complex data sets.complex data sets.  Correct classification of heart beats isCorrect classification of heart beats is fundamental to ECG monitoring systems suchfundamental to ECG monitoring systems such as an intensive care etc.as an intensive care etc.  Computers are used to automate signalComputers are used to automate signal processing.processing.  ANNs can detect patterns and makeANNs can detect patterns and make distinctions between different patterns thatdistinctions between different patterns that may not be apparent to human analysis.may not be apparent to human analysis.
  • 5.
    Applications of ANNin medicalApplications of ANN in medical  It has been successfully applied to variousIt has been successfully applied to various areas of medicine to solve non-linearareas of medicine to solve non-linear problems.problems.  Applications include prediction of diagnosisApplications include prediction of diagnosis such as:such as: – CancerCancer – the onset of diabetes mellitusthe onset of diabetes mellitus – survival prediction in AIDSsurvival prediction in AIDS – eating disorders etceating disorders etc  Applications in signal processing andApplications in signal processing and interpretation involve ECGs orinterpretation involve ECGs or electrocardiogramselectrocardiograms
  • 6.
    MotivationMotivation  Cardiovascular DiseasescontributeCardiovascular Diseases contribute 29.3% of total deaths in world.29.3% of total deaths in world.  Online ECG monitoring in ICUs/CCUs.Online ECG monitoring in ICUs/CCUs.  Acting Specialist in emergency cases.Acting Specialist in emergency cases.  Each component (P,QRS,T waves)Each component (P,QRS,T waves) has different frequencies.has different frequencies.  Each individual is different.Each individual is different.  Learning by experience.Learning by experience.
  • 7.
    What is ElectrocardiogramWhatis Electrocardiogram (ECG) ?(ECG) ?  ECG is the graphic recording of electricECG is the graphic recording of electric potentials generated by the heart.potentials generated by the heart.  12 lead ECG12 lead ECG  3 bipolar limb leads – I, II, III3 bipolar limb leads – I, II, III  3 unipolar augmented limb leads - AVF, AVR,3 unipolar augmented limb leads - AVF, AVR, AVLAVL  6 unipolar chest leads – V1 to V6.6 unipolar chest leads – V1 to V6.
  • 8.
    Anatomy of Heartand ECG signalAnatomy of Heart and ECG signal Normal ECG signal Conducting System of Heart
  • 9.
    Posterior Anterior Limb leads orientationwith respect to heart Chest leads orientation with respect to heart The 12 Views of the Heart
  • 10.
    12 Lead NormalECG 6 Limb leads 6 Chest leads RR
  • 11.
    ECG and diseasesECGand diseases Some of the diseases diagnosed bySome of the diseases diagnosed by ECG are:ECG are:  Myocardial Ischemia/Infarction.Myocardial Ischemia/Infarction.  Arrhythmias.Arrhythmias.  Hypertrophy and enlargement of heart.Hypertrophy and enlargement of heart.  Conduction Blocks.Conduction Blocks.  Preexcitation Syndromes.Preexcitation Syndromes.  Other cardiac disorders.Other cardiac disorders.
  • 12.
    Did you know!!Did you know !!  In heart Transplant Acute heartIn heart Transplant Acute heart rejection is more likely to happenrejection is more likely to happen when the heart donor was femalewhen the heart donor was female regardless of recipient sex.regardless of recipient sex.  Every 34 seconds, a person diesEvery 34 seconds, a person dies from Heart Diseases in the Unitedfrom Heart Diseases in the United States.States.
  • 13.
    Myocardial IschemiaMyocardial Ischemia Due to lack of adequate blood flow toDue to lack of adequate blood flow to the myocardium.the myocardium.  Ischemia is reversible.Ischemia is reversible.  Changes in ECG:Changes in ECG:  T wave peakingT wave peaking  Symmetric T wave inversionSymmetric T wave inversion  ST segment elevationST segment elevation
  • 14.
    Different ECG Signals NormalSignal ST segment elevated signal ECG with T wave inversion ECG Signal with peak T waves Myocardial Ischemia cont..Myocardial Ischemia cont..
  • 15.
    ArrhythmiasArrhythmias  It refersto any disturbance in theIt refers to any disturbance in the rate, regularity, site of origin, orrate, regularity, site of origin, or conduction of cardiac electricalconduction of cardiac electrical impulse.impulse.  Broadly two types:Broadly two types:  Tachycardia – Heart Rate beyond 100Tachycardia – Heart Rate beyond 100 bits/minute.bits/minute.  Bradycardia – Heart Rate below 60Bradycardia – Heart Rate below 60 bits/minute.bits/minute.
  • 16.
    Different ECG Signals NormalECG Signal ECG signal of Bradycardia patient ECG signal of Tachycardia patient Arrhythmias cont ..
  • 17.
    Sensitivity (SE) andSpecificity (SP)Sensitivity (SE) and Specificity (SP)  Helps us to explore the relationshipHelps us to explore the relationship between a diagnostic test and the (true)between a diagnostic test and the (true) presence or absence of disease.presence or absence of disease.  A test which is very sensitive will rarelyA test which is very sensitive will rarely miss people with the disease.miss people with the disease.  A specific test will have few false positiveA specific test will have few false positive results - it will rarely misclassify peopleresults - it will rarely misclassify people without the disease as being diseased.without the disease as being diseased.  Classification Rate:Classification Rate: CC = 100×(TP+TN)/(TN+TP+FN+FP)]
  • 18.
    Sensitivity (SE) andSpecificity (SP) Cont…Sensitivity (SE) and Specificity (SP) Cont…
  • 19.
    ApproachApproach  Variable attributesconsidered toVariable attributes considered to affect the training and generalizationaffect the training and generalization of the ANNs were identified asof the ANNs were identified as follows:follows: – Number of nodes in the hidden layerNumber of nodes in the hidden layer – Feature Selection method employedFeature Selection method employed – Number of files in training setNumber of files in training set – Size of input feature vectorSize of input feature vector – Number of epochsNumber of epochs
  • 20.
    Case StudyCase Study FeatureExtraction:Feature Extraction:  Fourier TransformFourier Transform  Principal component analysis (PCA)Principal component analysis (PCA) – widely used in signal processing,widely used in signal processing, statistics, and neural computing.statistics, and neural computing. – basic goal is to reduce the dimension ofbasic goal is to reduce the dimension of the data.the data.  Linear Prediction Coding (LPC)Linear Prediction Coding (LPC)
  • 21.
    Fourier TransformFourier Transform QRS complex is extracted by applying a window of some time duration (say 250 ms).  Each QRS complex is Fourier transformed and then the power spectrum is calculated.  The components generated along with the temporal vectors give the feature vector.
  • 22.
    QRS spectra ofa normal beatQRS spectra of a normal beat
  • 23.
    QRS spectra ofa Arrhythmia beatQRS spectra of a Arrhythmia beat
  • 24.
    PCAPCA  Step 1:Get some dataStep 1: Get some data  Step 2: Subtract the meanStep 2: Subtract the mean  Step 3: Calculate the covarianceStep 3: Calculate the covariance matrixmatrix  Step 4: Calculate the eigenvectors andStep 4: Calculate the eigenvectors and eigenvalues of the covariance matrixeigenvalues of the covariance matrix  Step 5: Choosing components andStep 5: Choosing components and forming a feature vectorforming a feature vector  Step 6: Deriving the new data setStep 6: Deriving the new data set
  • 25.
    Linear Prediction Coding(LPC)Linear Prediction Coding (LPC)  The basic idea of this technique is that sampled QRS segment can be approximated as a linear combination of the past QRS samples.  a is the i th linear prediction coefficient, and p is the order of the predictor.  LPC coefficients can be extracted using various methods viz Burg’s Method.
  • 26.
    Training the NNTrainingthe NN  Number of neurons in the input layer is determined by the number of elements in the input feature vector.  The output layer is determined by the number of classes desired.  The number of neurons in the hidden layer varies according to the specific recognition task and is determined by the complexity and amount of training data available.
  • 27.
  • 28.
    Performance AnalysisPerformance Analysis The performance of the neural classifiers is evaluated by computing the percentages of: – sensitivity (SE), – specificity (SP) and – correct classification (CC)
  • 29.
    ResultsResults NeuralNeural ClassifierClassifier Input LayerInput LayerHidden LayerHidden Layer 11 1212 55 22 10 3 33 5 2
  • 30.
  • 31.
    Results Cont.Results Cont. How does ANN based classificationHow does ANN based classification compare with:compare with: – Other ECG widely used interpretationOther ECG widely used interpretation program?program? Neural networks were 15.5% more sensitiveNeural networks were 15.5% more sensitive – Expert cardiologistExpert cardiologist 10.5% more sensitive than the cardiologist10.5% more sensitive than the cardiologist
  • 32.
    ConclusionConclusion  Performance ofthe neural network strategy has shown higher performance than other classical methods (Cox regression models) in predicting clinical outcomes of the risk of coronary artery disease.
  • 33.
    ReferencesReferences  [1] M.A. Chikh, F. Bereksi Reguig. Application of artificial neural networks to identify the premature ventricular contraction (PVC) beats,2004  [2] Costas Papaloukasa, Dimitrios I. Fotiadisb, Aristidis Likasb, Lampros K. Michalis. An ischemia detection method based on artificial neural networks,2002  [3] C.D. Nugent, J.A.C. Webb, N.D. Black, G.T.H. Wright, M. McIntyre. An intelligent framework for the classification of the 12-lead ECG, 1999.
  • 34.
     Introduction toNeural Networks in Healthcare,Introduction to Neural Networks in Healthcare, Open Clinic, 2002.Open Clinic, 2002.  [4] M.S. Thaler, The Only EKG Book You’ll Ever[4] M.S. Thaler, The Only EKG Book You’ll Ever Need 3Need 3rdrd Edition, Lippincott Williams & Wilkins.Edition, Lippincott Williams & Wilkins.  P.J Mehta, Understanding ECG, 5P.J Mehta, Understanding ECG, 5thth Edition, TheEdition, The National Book Depot.National Book Depot.
  • 35.
    Believe it orNOT !!Believe it or NOT !!  How much blood does your heart pump?How much blood does your heart pump? – An average heart pumps 2.4 ounces (70An average heart pumps 2.4 ounces (70 milliliters) per heartbeat. An average heartbeatmilliliters) per heartbeat. An average heartbeat is 72 beats per minute. Therefore an averageis 72 beats per minute. Therefore an average heart pumps 1.3 gallons (5 Liters) per minute.heart pumps 1.3 gallons (5 Liters) per minute. In other words it pumps 1,900 gallons (7,200In other words it pumps 1,900 gallons (7,200 Liters) per day, almost 700,000 gallonsLiters) per day, almost 700,000 gallons (2,628,000 Liters) per year, or 48 million(2,628,000 Liters) per year, or 48 million gallons (184,086,000 liters) by the timegallons (184,086,000 liters) by the time someone is 70 years old. That's not bad for asomeone is 70 years old. That's not bad for a 10 ounce pump!10 ounce pump!  Men suffer heart attacks about 10 yearsMen suffer heart attacks about 10 years earlier in life than women do.earlier in life than women do.

Editor's Notes

  • #4 http://www.generation5.org/content/2004/NNinAnaemia.asp
  • #5 http://www.biomedcentral.com/1472-6947/2/1
  • #6 http://www.generation5.org/content/2004/NNinAnaemia.asp
  • #18 http://www.australianprescriber.com/index.php?content=/magazines/vol25no5/bauman.htm
  • #20 http://www.biomedcentral.com/1472-6947/2/1
  • #21 http://www.cis.hut.fi/aapo/papers/NCS99web/node5.html http://www.gseis.ucla.edu/courses/ed230bc1/notes1/var1.html
  • #25 Pca-tutorial.pdf
  • #33 http://www.generation5.org/content/2004/NNinAnaemia.asp