Experimental Neutrino Physics Concepts in Nutshell

Experimental Neutrino
Physics Concepts in Nutshell
Son Cao

(KEK/J-PARC)
i.e A practical guide for
How to THINK/ADDRESS the things as a neutrino experimentalist

!2
To simplify something such as…
(GeV)νE
0.5 1 1.5 2 2.5 3
Osc.Prob
0
0.02
0.04
0.06
0.08
0.1
flux
µ
νOff-axis°2.5
ν, NH,°=0cpδ
ν, NH,°=270cpδ
ν, NH,°=0cpδ
ν, NH,°=270cpδ
eν→µν,eν→µν
Hypothesis
(parameters)
observed data
T2K has made the ﬁrst observation of electron neutrino
appearance in a muon neutrino beam…with
a signiﬁcance of 7.3𝜎 over the the hypothesis of sin2
2θ13 = 0
data-based statement
Experiment setup

!3
…with something like
“neutrino oscillations” from Oyama-san & Boris

!4
Some confessions
• Call me “teaching assistant” or senpai ( 先輩 in Japanese) rather than lecturer

• I shared almost same background as most of Vietnamese theoretical students,
and found lot of difficulties for first years of Ph.D in experimental HEP

• I requested this lecture since I thought it could make bridge between young
theorists and young experimentalist. But it may be very bias…

• My background is with accelerator-based neutrino experiments, might not
enough to generalize for other fields.

• Many materials are borrowed from other talks, but sometime missing citation

• Your feedback is valuable and I’m alway be a student for listening your
feedback

!5
Contents
• Basic steps as scientists
• Ask question, Design, Build experiment, Collect data and Make
statement based on data

• Examples with neutrino experiments

• Neutrino detection principle
• A complicated, interdisciplinary ﬁeld of Particle & Nuclear physics,
Material science, Mechanics, Electronics, Data mining

• Some selected topics (personal choices)
1) Signal and background

2) Hypothesis test

3) Sensitivity &Parameter estimation

4) Neutrino energy reconstruction

5) Systematics

6) Monte Carlo usage
Number of illustrations will be shown

Source code: https://github.com/cvson/nushortcourse

Feel free to download and play!

!6
Basic steps as scientists
Ask a question about Nature
Formulate the hypothesis to test
Experiment(s)
Design
Exp.
Build
Exp.
Collect
data
Make
data-based
statement

!7
Basic steps as scientists
All connected on one foot!
Ask a question about Nature
Formulate the hypothesis to test
Design
Exp.
Build
Exp.
Collect
data
Make
data-based
statement
Experiment(s)

!8
An example: searchνμ → νe
Do muon neutrinos transform into
electron neutrinos at given distance
of travel?
“neutrino oscillations” from Oyama-san & Boris
Why is addressing this question important?
• Conﬁrm non-zero mixing angle, 𝜃13 >0 or
set lower limit for mixing angle 𝜃13 (𝜃13 >𝛼)

• If non-zero, open the door to measure 𝛿CP,
might a source of matter-antimatter
asymmetry in the Universe
What have you already know at the
time question posed?
• Neutrino oscillations conﬁrmed

• Some upper limit on 𝜃13 from reactor

• etc…
https://arxiv.org/abs/hep-ex/0106019
Supported knowledge
at point A at point B
L (distance)

!9
of travel?
Driven by
theoretical models
T2K, NOvA, etc
Goal of your experiments
Level 2: Estimate parameters which
used to describe your model

• Model give good description of our
data?

• Allowed region for sin2 𝜃13 at 68%
C.L. (1σ) or 90% C.L., etc…
Level 1: Test theoretical hypothesis; basically
yes/no question

• At some C.L., we observe the appearance of
electron neutrino

• At some C.L., we reject the hypothesis that
electron neutrino appeared
Hypothesis test and parameter
estimation will be discussed later
may not know yet?

!10
of travel?
T2K, NOvA, etc
In principle, how can we conduct the search?
1. Need source of

2. Put detector at some distance from source

3. Look for appeared from source in detector
Is it simple for you?
νμ
νe νμ
νμ
at point A at point B
L (distance)

!11
In principle, how we can perform the search?
1. Need source of

2. Put detector at some distance from source

3. Look for appeared from source in detector
νμ
νe νμ
νμ
Do muon neutrino transform into
of travel?
T2K, NOvA, etc
(1)
(2) (3)
(4) (5)
(1) How can source be created? How well you understand the
source? (composition, density, energy, timing, etc)
(2) What kind of detector you need? how big it is? Where do you put?
(3) How can you choose distance? Typically your detector can’t move
from place to place.
(4) How can you identify ?
(5) How do you know it coming from source but not others?
νe
νμ

!12
Design experiments
of travel?
T2K, NOvA, etc
Design
Exp.
source? (density, energy, timing, etc)
νe
νμ
You have to address at least below questions when designing experiment. In reality,
there are many more questions to address.

• Think big, make cheap. HEP is really expensive game

• What available facilities to use, ref. T2K

• How do you know you have the best among many possible experiment setups

• Most important: guarantee of success (doesn’t mean you will get signal, but your
experiment should achieve some measurement w/ unprecedented level of precision)
can be conservative

!13
Short example answer: T2K
of travel?
T2K, NOvA, etc
Design
Exp.
νe
νμ
(1)
(2)
(3)
(4)
(5)
GPS, timing synchronize
details from Oyama-san

!14
Short example answer: NOvA
of travel?
T2K, NOvA, etc
Design
Exp.
νe
νμ
(1)
(2)
(3)
(4)
(5)
GPS,timingsynchronize

!15
Design experiments for νμ → νe
of travel?
T2K, NOvA, etc
Design
Exp.
Neutrino Energy (GeV)
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
)eν→µνoreν→µνProb.(
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
2
eV-3
10×|=2.5032
2
m∆|
2
eV
-5
10×|=7.6021
2
m∆|
=1.0023θ22
sin
=0.8712
θ22
sin
=0.0813
θ22
sin
L=295 km
=0
CP
δ, NH,ν
°
=270CP
δ,NH,ν
=0CP
δ, NH,ν
°
=270CP
δ, NH,ν
T2K 𝜈 energy range
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
2
eV-3
10×|=2.5032
2
m∆|
2
eV
-5
10×|=7.6021
2
m∆|
=1.0023θ22
sin
=0.8712
θ22
sin
=0.0813
θ22
sin
L=810 km
=0
CP
δ, NH,ν
°
=270CP
δ,NH,ν
=0CP
δ, NH,ν
°
=270CP
δ, NH,ν
NOvA 𝜈 energy range
At sin22𝜃13 = 0.08,
Prob. is around 5%
(depend on 𝛿CP value;
smaller for anti-neutrino)
Since oscillation probability depends on neutrino
energy, it’s important to know energy of incoming
neutrinos.
https://github.com/cvson/nushortcourse/tree/master/OscCalculatorPMNS
Let’s look at basic quality: probability as function of
energy. Basically this is counting experiment
νμ → νe
T2K baseline NOvA baseline
Nνe
∼ Prob.(νμ → νe)

!16
of travel?
T2K, NOvA, etc
Design
Exp.
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
2
eV-3
10×|=2.5032
2
m∆|
2
eV
-5
10×|=7.6021
2
m∆|
=1.0023θ22
sin
=0.8712
θ22
sin
=0.0413
θ22
sin
L=295 km
=0
CP
δ, NH,ν
°
=270CP
δ,NH,ν
=0CP
δ, NH,ν
°
=270CP
δ, NH,ν
T2K 𝜈 energy range
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
2
eV-3
10×|=2.5032
2
m∆|
2
eV
-5
10×|=7.6021
2
m∆|
=1.0023θ22
sin
=0.8712
θ22
sin
=0.0413
θ22
sin
L=810 km
=0
CP
δ, NH,ν
°
=270CP
δ,NH,ν
=0CP
δ, NH,ν
°
=270CP
δ, NH,ν
NOvA 𝜈 energy range
At sin22𝜃13 = 0.04,
Prob. is around 2.5%
(depend on 𝛿CP value;
smaller for anti-neutrino)
The probability depends on values of sin22𝜃13
(also 𝛿CP). Smaller probability means with the
same amount of neutrino, number of event you
will observe is smaller in average.
https://github.com/cvson/nushortcourse/tree/master/OscCalculatorPMNS
Let’s look at basic quality: probability as function of
energy. Basically this is a counting experiment
νμ → νe
Nνe
∼ Prob.(νμ → νe)
T2K baseline NOvA baseline

!17
of travel?
T2K, NOvA, etc
Design
Exp.
sin2
2θμe=sin2
2θ13.sin2
θ23
Normally, capability (how good/“sensitivity”) of designed
detector much be evaluated for various scenario of
underlying parameters, e.g.:
- Range of parameter in which detector can explore
- At what values of parameter, detector can make
observation, measurement with some precision
hep-ex/0106019

!18
Build experiment
of travel?
T2K, NOvA, etc
Design
Exp.
Build
Exp.
• Typically, neutrino detector is big, really big (with
few exception) since cross section is small

• Normally, detector is located in deep
underground to cancel the noise from cosmic ray

• big MONEY for this, NOvA is on surface
although it is design to be underground

• India controversy on INO building due to
natural conservation

• etc…
Example/ Details in Oyama-san
Many additional considerations,
• How to access it?

• How to monitor it?

• How to maintain it?

!19
Collect data
• Data taking needs time: from year to decade
(ex, Super-K 20 years, T2K 7 years)

• Your detector is NOT at same condition during
data-collecting period

• Detector position can be unintentionally
moved due to, eg, Earthquake

• Some photon detectors can be out-of-
function or mis-behaved

• Light yield (No. of photon per ﬁxed amount
of deposited energy) can be changed due
to water quality, aging of scintillator, etc…

• etc…
of travel?
T2K, NOvA, etc
Design
Exp.
Build
Exp.
Collect
data
You need to take good data. Keep
condition of your detector in control
as much as you can
Atmospheric ! FCFV events

!20
Make statement based on data
• The statement is never simple like “yes” or “no”

• It always goes with level of uncertainty/
conﬁdence

• If an observation is claimed, some parameters
are estimated, if not, a limit is presented

• E.g:
of travel?
T2K, NOvA, etc
Design
Exp.
Build
Exp.
Collect
data
make
statement
10.1103/PhysRevLett.112.061802

!21
Make statement based on data
Hypothesis
(parameters)
Data
Probability
How interpolate data and parameters to hypothesis, and interpolation of

probability for uncertain/random events to happen.

Two schools of thought: Frequentist and Bayesian

Frequentist vs. Bayesian
!22
Frequentist

• Data are a repeatable
random sample, frequency

• Underlying parameters
remains constant

• State of knowledge worlds is
sharp
Bayesian

• Data are observed from the
realized sample, prior
knowledge

• Parameters are unknown and
described probabilistically

• State of knowledge worlds is
always updated
Not going to detail. We can discuss after class
It’s about philosophy of probability for

and uncertain/random event happen,
e.g World cup’s history record play any
role in the result of current/future
soccer match? (in very rude sense)

Frequentist vs. Bayesian
!23
Frequentist uses impeccable logic to deal with an issue of no
interest to anyone
Bayesian addresses the question everyone is interested in, by
using assumptions no-one believes

L. Lyon
Experimentalists tend to make data-based
statements with both approaches
Frequentist
Bayesian
Reach quite similar conclusion:
CP conserving values lies outside of 2σ conﬁdence/credible interval
T2K, Neutrino 2018

!24
Make your own path?
• Neutrinos are Majorana or Dirac particles?
• CP symmetry violated in neutrino oscillation?
• Neutrino mass is inverted or normal?
• Is there 4th generation of neutrino?
• etc…
More questions from other lecturers

!25
Make your own path?
• Neutrinos are Majorana or Dirac particles?
• CP symmetry violated in neutrino oscillation?
• Neutrino mass is inverted or normal?
• Is there 4th generation of neutrino?
• etc…
Some experiments are going to build (DUNE, HYPER-K, JUNO,etc)
Some are waiting for your!
More questions from other lecturers

!26
Neutrino detection principle

!27
Neutrino detection principle
Neutrinos basically can’t see/directly but we know it via their trace when interacting with nucleon/nuclei

with help of photon detectors/sensors
This is just illustration. Many detection technique out there.
Photon detectors
Event reconstruction
Neutrino interactions
Youwanttoknowthisguys
what you observed
nucleon/nuclei
Charged particles

(𝜇, e,𝝅, p,…)
Neutral particles

(n,𝜋0,𝛾,…)
“eyes”
“lot of eyes”
“Pattern of light induced by
neutrino interaction”

!28
Neutrino detection is complicate
PN physics
Material science
Mechanics
Electronic
Data mining
Neutrino detection is a complicate, interdisciplinary ﬁeld

!29
Involved Particle and Nuclear physics
PN physics
Material science
Mechanics
Electronic
Data mining
F. Sanchez, neutrino 2018
• Neutrino-nucleon/nuclei interaction is complicated

• For oscillation analysis, you need, essentially

(1) Particle identity

(2) Neutrino energy
Van Nguyen’s lecture
Charged particles
neutral particles
Based on induced charged particle
in ﬁnal state interaction

!30
Material science in neutrino experiments
PN physics
Material science
Mechanics
Electronic
Data mining
T2K far detector use water; NOvA use liquid scintillator; MINOS
used magnetized steel, OPERA used Emulsion, etc…?

• T2K, NOvA needs to identify both 𝜈 𝜇 and 𝜈e

• MINOS focus on 𝜈 𝜇 and its antineutrino

• OPERA need to see 𝜈 𝝉
Material selection depends on particle you want to detect and
its properties. Also detector size & our understanding of
neutrino interaction on selected material are important factors.
Water
Liquid scintillator
magnetized iron
Emulsion

!31
Mechanics in neutrino detection
PN physics
Material science
Mechanics
Electronic
Data mining
• In Nov 2001, Super-K suﬀered a serious blow, ~700 PMT tubes
exploded (cost $3000 per each) (5000 PMT remain undamaged)

• Cause: one tubes (contain a vacuum) exploded, released
energy, caused shock wave —> chain reaction of explosion

• To mitigate this possibility: Acrylic shield is developed and used
Bare PMT
PMT w/ acrylic shield
Similar structure in aquarium
One example

!32
Electronics in neutrino experiments
PN physics
Material science
Mechanics
Electronic
Data mining
• Number of photon sensor/ “eyes” per each detector is often very
large: 13,000 channels in Super-K, 334,000 channels in NOvA far
detector

• With many “eyes”, you need a “nervous” system to manipulate
and collect data eﬃciently

• “Eyes” don’t not always open, no need and not good for
lifetime of electronics

• “Eyes” actually operate when receiving “trigger” signal, and
often within a predeﬁned time window
Depend on how often your detector
get data; how many events interact in
your detector in a time window, etc…
Ex: NOvA electronics at Near Detector
Ex: INGRID data acquisition

!33
Data mining in neutrino experiments
PN physics
Material science
Mechanics
Electronic
Data mining
• How do you know this is likely due to 𝜈e interaction?

• Basically, you need guidance from simulation

• The method is something like this:

1. Build detector simulation to simulate what you can observe when particles
enter your detector (so called Geant 3,4 and your detector geometry)

2. Simulate neutrinos (you know true info. such as neutrino type, energy,
direction, interaction point in detector)

3. Obtain pattern for simulated neutrino events and store as a library

4. Compare your data pattern with library and see how likely data match with
what simulated events

!34
Neutrino detection is complicate
PN physics
Material science
Mechanics
Electronic
Data mining
Neutrino detection is a complicate, interdisciplinary ﬁeld.
You don’t need to know all of these. Expert in one ﬁeld is probably enough.

!35
Before going to some selected topics, let’s have a
quick digest on Histogram, a conventional way to
visualize your data in HEP
• Taking about experiment is to talk about data.
• To make number less boring, a sexy way to
visualize it is invented, so-call Histogram
Some lecturers showed but not
sure if you understand it.

!36
A histogram is an accurate representation of the
distribution of numerical data. It is an estimate of the
probability distribution of a continuous variable (quantitative
variable) and was ﬁrst introduced by Karl Pearson.
https://en.wikipedia.org/wiki/Histogram
Histogram
Go google image and type: histogram neutrino

!37
Histogram
#entry #value

0 29.9947

1 22.8262

2 28.909

3 24.8497

4 29.1213

5 24.7164

6 20.4956

7 24.6265

8 25.0396

9 22.9462
Values of variable X
10 15 20 25 30 35 40
Numberofentries
0
0.5
1
1.5
2
2.5
3
Histogram
Entries 10
Mean 25.35
RMS 2.923
https://github.com/cvson/nushortcourse/tree/master/basic01

!38
Histogram
#entry #value

0 29.9947

1 22.8262

2 28.909

3 24.8497

4 29.1213

5 24.7164

6 20.4956

7 24.6265

8 25.0396

9 22.9462
10 15 20 25 30 35 40
Numberofentries
0
0.5
1
1.5
2
2.5
3
Histogram
Entries 10
Mean 25.35
RMS 2.923
bin

[20,21)
bin

[29,30)

!39
100 entries in your sample
Histogram
10 15 20 25 30 35 40
Numberofentries
0
2
4
6
8
10
12
14
Histogram
Entries 100
Mean 24.82
RMS 4.83
Can you guess data following
which distribution?

!40
1000 entries in your sample
(as data sample increased)
Histogram
10 15 20 25 30 35 40
Numberofentries
0
10
20
30
40
50
60
70
80
90
Histogram
Entries 1000
Mean 25.14
RMS 5.131
which distribution?

!41
Histogram
10 15 20 25 30 35 40
Numberofentries
0
100
200
300
400
500
600
700
800
Histogram
Entries 10000
Mean 25.03
RMS 4.906
which distribution?
https://github.com/cvson/nushortcourse/tree/master/basic0110000 entries in your sample

!42
Histogram
10 15 20 25 30 35 40
Numberofentries
0
1000
2000
3000
4000
5000
6000
7000
8000
Histogram
Entries 100000
Mean 25
RMS 4.92
which distribution?
100,000 entries in your sample

!43
Histogram
Histogram
Entries 100000
Mean 25
RMS 4.92
/ ndf2
χ 42.97 / 27
Prob 0.02633
Constant 31.6±7985
Mean 0.0±25
Sigma 0.012±4.993
10 15 20 25 30 35 40
Numberofentries
0
1000
2000
3000
4000
5000
6000
7000
8000
Histogram
Entries 100000
Mean 25
RMS 4.92
/ ndf2
χ 42.97 / 27
Prob 0.02633
Constant 31.6±7985
Mean 0.0±25
Sigma 0.012±4.993
Indeed, it generated with Gaussian
distribution with Mean = 25 and RMS =5
100,000 entries in your sample
Your data might be underlying a particular
distribution/pattern but it might not be easy
to reveal if your data sample is not enough.

!44
1) Underlying distribution
Two-dimensional histogram
10 20 30 40
ValuesofvariableY
10
20
30
Histogram
Entries 10000
Mean x 24.99
Mean y 19.94
RMS x 5.005
RMS y 4.01
Histogram
Entries 10000
Mean x 24.99
Mean y 19.94
RMS x 5.005
RMS y 4.01
Histogram_py
Entries 10000
Mean 19.94
RMS 4.01
0 200 400 600 800 1000
ValuesofvariableY
10
20
30
Histogram_px
Entries 10000
Mean 24.99
RMS 5.005
10 20 30 40
0
200
400
600
800
Histogram_px
Entries 10000
Mean 24.99
RMS 5.005
The histogram can be shown in more than
one-dimension

!45
1) Underlying distribution
Two-dimensional histogram
0
10
20
30
40
50
60
70
80
90
10 20 30 40
ValuesofvariableY
10
20
30
Histogram
Entries 10000
Mean x 24.99
Mean y 19.94
RMS x 5.005
RMS y 4.01
0
10
20
30
40
50
60
70
80
90
Histogram
Entries 10000
Mean x 24.99
Mean y 19.94
RMS x 5.005
RMS y 4.01
Histogram_py
Entries 10000
Mean 19.94
RMS 4.01
0 200 400 600 800 1000
ValuesofvariableY
10
20
30
Histogram_px
Entries 10000
Mean 24.99
RMS 5.005
10 20 30 40
0
200
400
600
800
Histogram_px
Entries 10000
Mean 24.99
RMS 5.005
Zoom
-in
bin, ~90 entries
bin width in Y-view
bin width in X-view
Bin width might be varies, not constant

!46
Some selected topics

2) Hypothesis test

3) Sensitivity & Parameter estimation


5) Systematics

6) Monte Carlo usage 101

!47
Signal and Background
Signal: For what you are placing as object to study, e.g. 𝜈e from 𝜈 𝜇 beam

Background: Anything else

Measurement is performed on a selected sample which contains both signal and
background. Background is always there since your sample selection is not perfect.

• It’s important to deﬁne clearly what’s signal. Sometime it’s not straightforward, e.g.

• For oscillation analysis, 𝜈e from 𝜈 𝜇 beam observed at Far Detector is signal but
intrinsic 𝜈e is background
• For understanding neutrino source composition, cross-section at Near
Detector, intrinsic 𝜈e is signal

• In selected sample, ratio of signal to background is matter, not just number of signal
alone. E.g, you want to ﬁnd a Japanese students, which class you should go?

• A class with 10 Japanese students and 1000 non-Japanese

• A class with 5 Japanese students and 5 non-Japanese

!48
Signal vs. Background
Electron with EM activity, look more fuzzy than muon.
NOvA, scintillator technique
Karol, Miura-san’s lecture & training
You need machinery/tool to separate signal from background. The “fuzzy” thing is quantized into
one or multiple variables which is used to build likelihood of data to be signal or to be
background. (Some machine learning can skip the middle steps.)
Super-Kamiokande,
Water-cherenkov technique
This guides your eyes but science need
quantitative thing

!49
Signal vs. Background: some pattern separation
Variable 1
5− 4− 3− 2− 1− 0 1 2 3 4 5
Variable2
5−
4−
3−
2−
1−
0
1
2
3
4
5
Signal
Background
Variable 1
1.5− 1− 0.5− 0 0.5 1 1.5
Variable2
0.6−
0.4−
0.2−
0
0.2
0.4
0.6
Signal
Background
Variable 1
0.8− 0.6− 0.4− 0.2− 0 0.2 0.4 0.6 0.8 1 1.2
Variable2
1−
0.5−
0
0.5
1
Signal
Background
Variable 1
3− 2− 1− 0 1 2 3
Variable2
3−
2−
1−
0
1
2
3 Signal
Background
https://github.com/cvson/nushortcourse/tree/master/datamining

!50
Signal and Background: by eyes
Variable 1
5− 4− 3− 2− 1− 0 1 2 3 4 5
Variable2
5−
4−
3−
2−
1−
0
1
2
3
4
5
Signal
Background
Variable 1
1.5− 1− 0.5− 0 0.5 1 1.5
Variable2
0.6−
0.4−
0.2−
0
0.2
0.4
0.6
Signal
Background
Variable 1
0.8− 0.6− 0.4− 0.2− 0 0.2 0.4 0.6 0.8 1 1.2
Variable2
1−
0.5−
0
0.5
1
Signal
Background
Variable 1
3− 2− 1− 0 1 2 3
Variable2
3−
2−
1−
0
1
2
3 Signal
Background
Decision rule/boundary

!51
Signal Background: by machine learning
Variable 1
5− 4− 3− 2− 1− 0 1 2 3 4 5
Variable2
5−
4−
3−
2−
1−
0
1
2
3
4
5
Signal
Background
Variable 1
1.5− 1− 0.5− 0 0.5 1 1.5
Variable2
0.6−
0.4−
0.2−
0
0.2
0.4
0.6
Signal
Background
Variable 1
0.8− 0.6− 0.4− 0.2− 0 0.2 0.4 0.6 0.8 1 1.2
Variable2
1−
0.5−
0
0.5
1
Signal
Background
Variable 1
3− 2− 1− 0 1 2 3
Variable2
3−
2−
1−
0
1
2
3 Signal
Background
Decision rule/boundary

!52
Signal vs. Background: ID parameter
Partice ID parameter
-10 -8 -6 -4 -2 0 2 4 6 8 10
0
50
100
150
200
250
300
350
Super Kamiokande IV 2166.5 days : Monitoring
e-like muon-like
Numberofevents
• To make selection (or decision rule/boundary),
typically a likelihood of data to be signal/
background is built.

• Background is unavoidable

• Enhance signal and suppress background is
important in HEP analysis, especially in
neutrino experiment where statistics is limited.

• The improvement can be from hardware side
or software side

• It’s (big) money can be saved when you can
improve your selection since it is eﬀectively
equivalent to collecting more data or enlarge
your detector
Example with atmospheric neutrinos observed in Super-Kamiokande
Red curve is what machine learned

Black dots are your data

!53
Fun exercise
We have 19 students, 10 are from Vietnam.

Can you teach computer how to identify them?

!54
Fun exercise
Partice ID parameter
-10 -8 -6 -4 -2 0 2 4 6 8 10
0
50
100
150
200
250
300
350
Super Kamiokande IV 2166.5 days : Monitoring
e-like muon-like
Numberofevents
Nationality ID parameter
Numberofstudents
Vietnamese Non-Vietnamese

!55
Signal and Background
Reconstructed Energy (GeV)ν
0 0.2 0.4 0.6 0.8 1 1.2
NumberofEvents
0
2
4
6
8
10
12
eν→µν
eν→µν
NC
intrinsiceν/eν
intrinsicµν
intrinsicµν
T2K Run1-8 PreliminaryFinal systematics pending
NOvA, Neutrino 2018
T2K, Moriond 2018
Examples on selected sample in real
analyses from neutrino experiments

!56

2) Hypothesis test



5) Systematics


!57
Hypothesis test
Hypothesis H0: 𝜃13 = 0
Errors are alway there. What matter is the probability to
make error and how much you can tolerate the error
In testing a hypothesis H0, there are two kinds of errors:

• Type I: reject H0 although it is true

• i.e. “observe” 𝜃13 ≠ 0 although the true is 𝜃13 = 0

• Type II: accept H0 although it is false

• i.e. “fail to observe” 𝜃13 ≠ 0 although the true is 𝜃13 ≠ 0

!58
Hypothesis test: Error toleration
Hypothesis H0: Train is arrived at train station on-time.
In Japan, toleration (unconventionally?)

is within 1minute

E.g. Train company apologize for 20s-early

departure
In Vietnam, toleration (unconventionally?)

is order of 30 minutes

(Rarely you hear apologies for delay in arrival)

!59
Hypothesis test
When you make statement, it should include two errors ( 𝛼, β).
• (1- 𝛼) (%) is normally mentioned as Conﬁdence Level (C.L.), set at
beginning as toleration level, e.g 0.05 or 95% C.L.
• (1-β) (%) is probability that you make “observation” at (1- 𝛼) (%) C.L.
We care this error especially when e.g. due to statistic ﬂuctuation, you are
very lucky to make observation
Hypothesis H0: 𝜃13 = 0
In testing a hypothesis H0, there are two kinds of errors:

• Type I: reject H0 although it is true

• i.e. “observe” 𝜃13 ≠ 0 although the true is 𝜃13 = 0

• Type II: accept H0 although it is false

• i.e. “fail to observe” 𝜃13 ≠ 0 although the true is 𝜃13 ≠ 0
denoted by 𝛼
denoted by β

!60
Hypothesis test: example from T2K
T2K measurement of “appearance” of

• Background: 6.5 events

• Data: 9 events
Value of toy experiments
0 10 20 30 40 50Numberoftoyexperiments
1−
10
1
10
2
10
3
10
4
10
5
10
6
10
7
10
if you are familiar with ROOT, can try

p-value = 1-ROOT::Math::poisson_cdf(8,6.5);
νe
p-value =
No. of toy exp., value > 9
All of toy exp.
= 0.208
p-value> 0.05: you failed to reject hypothesis H0 and
the result is statistically nonsigniﬁcant.
Hypothesis H0: data agree with background

!61
Hypothesis test: Example from NOvA
NOvA appearance of

• Background: 5.3 events

• Data: 18 events

P value is 5.27e-06
pvalue = 1-ROOT::Math::poisson_cdf(17,5.3);

sigma = TMath::NormQuantile(1-pvalue);
Value of toy experiments
0 10 20 30 40 50
Numberoftoyexperiments
1−
10
1
10
2
10
3
10
4
10
5
10
6
10
7
10
8
10
P value is 5.27e-06
5.41634e-06
4.39985
νe
p-value =
No. of toy exp., value > 9
All of toy exp.
= 5.27e − 6
p-value< 0.05: you reject hypothesis H0 and
the result is statistically signiﬁcant, but not observation yet!
< 5σ (level of discovery)

!62

2) Hypothesis test



5) Systematics


!63
Sensitivity
You might hear, e.g.

• T2K has good “sensitivity” on CP violation

• NOvA has good “sensitivity” on mass hierarchy
“Good” sensitivity means you can reject some
hypothesis, i.e make observation of something, with high
conﬁdence level (1-𝛼)(%) with high probability (1-β)(%)
For sensitivity, normally only quote C.L while keep (1-β)
(%) = 50%
without mentioning
to data

!64
Prediction of event rate in detector
To give some sense on sensitivity and parameter
estimation, we will follow one speciﬁc example
Ni = Φflux × σ × ϵ
[GeV]νE
0 5 10 15 20 25
p.o.t.]21
/50MeV/1x102
Flux[/cm
1−
10
1
10
2
10
3
10
4
10
5
10
6
10
flux at Super-K with 250 kA operationµνT2K
fluxµν
fluxµν
fluxeν
fluxeν
[GeV]µν
True E
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
/GeV/atom]2
[cmν/Eσ
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
36−
10×
O
16
8Cross section on
Total
CCQE
πCC1
CC coherent
CC other
NC
O
16
8Cross section on
Energy [GeV]ν
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
p.o.t]21
Events[/50MeV/1x10
0
1
2
3
4
5
3
10×
Event rate at Near Detector (1ton H20)
Total
CCQE
πCC1
CC coherent
CC other
NC
Event rate at Near Detector (1ton H20)
https://github.com/cvson/nushortcourse/tree/master/eventpred
= x
Detection eﬃciency

assume a constant (<1)
This assumes there is no oscillation

!65
Prediction with oscillation
Neutrino Energy [GeV]
0 0.5 1 1.5 2 2.5 3
NumberofEvents
0
10
20
30
40
50
60
70
80
Data
Unoscillated Prediction
0 0.5 1 1.5 2 2.5 3
Data/Prediction 0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
https://github.com/cvson/nushortcourse/tree/master/sensitivity
⃗o = (Δm2
21, Δm2
31, θ12, θ13, θ23, δCP)
Ni( ⃗o ) = Φflux × σ × ϵ × P( ⃗o )
Here, oscillation is applied to make “fake” data
⃗otrue
P(νμ → νμ) ∼ 1 − sin2
2θ23 sin2
Δm2
31L
4Eν

!66
0 0.5 1 1.5 2 2.5 3
NumberofEvents
0
10
20
30
40
50
60
70
80
Data
=0.52)θ22
=0.00239,sin2
m∆(
=0.95)θ22
=0.00239,sin2
m∆(
0 0.5 1 1.5 2 2.5 3
Data/UnoscillatedPred. 0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
⃗o = (Δm2
21, Δm2
31, θ12, θ13, θ23, δCP)
Ni( ⃗o ) = Φflux × σ × ϵ × P( ⃗o )
⃗otrue
∼sin2
2θ

!67
0 0.5 1 1.5 2 2.5 3
NumberofEvents
0
10
20
30
40
50
60
70
80
Data
=0.52)θ22
=0.00239,sin2
m∆(
=0.95)θ22
=0.00239,sin2
m∆(
=0.95)θ22
=0.00299,sin2
m∆(
0 0.5 1 1.5 2 2.5 3
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
⃗o = (Δm2
21, Δm2
31, θ12, θ13, θ23, δCP)
Ni( ⃗o ) = Φflux × σ × ϵ × P( ⃗o )
⃗otrue
Δm2

!68
0 0.5 1 1.5 2 2.5 3
NumberofEvents
0
10
20
30
40
50
60
70
80
Data
=0.52)θ22
=0.00239,sin2
m∆(
=0.95)θ22
=0.00239,sin2
m∆(
=0.95)θ22
=0.00299,sin2
m∆(
0 0.5 1 1.5 2 2.5 3
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
⃗o = (Δm2
21, Δm2
31, θ12, θ13, θ23, δCP)
Ni( ⃗o ) = Φflux × σ × ϵ × P( ⃗o )
Data is well described by the blue line. But is it
the “best” parameter yet?
⃗otrue

!69
Parameter estimation
log χ2
=
∑
i
2(Nexp. − Nobs.) − 2Nobs. . log(Nexp./Nobs.)
0 0.5 1 1.5 2 2.5 3
NumberofEvents
0
10
20
30
40
50
60
70
80
Data
0 0.5 1 1.5 2 2.5 3
Data/Prediction
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
• When we talk about parameter, we need
predeﬁne a mode

• Given data and model, how do we estimate
parameter?

• you need quantify the diﬀerence
between data and prediction at various
parameter

• Method to quantify is not unique, e.g
Maximum likelihood
χ2
( ⃗ok, ⃗otrue) =
∑
i
χ2
(Ni( ⃗ok), Ni( ⃗otrue))
P(νμ → νμ) ∼ 1 − sin2
2θ23 sin2
Δm2
31L
4Eν
Nexp
Nobs
bin i

!70
What’s the best parameters to describe your data?
Illustration for comparing data with various prediction on the
left with 𝜒2 calculated corresponding on the right. From this,
conﬁdence intervals are extracted (not going to detail now)
χ2

!71
Probability on 2-parameter space
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
θ22
sin
3−
10 2−
10 1−
10 1
)4
/c2
(eV2
m∆
5−
10
4−
10
3−
10
2−
10
1−
10
1
10
2
10
3
10
P(νμ → νx) ∼ sin2
2θ23 sin2
Δm2
31L
4Eν
Probability values

• Essentially, your experiment is measuring oscillation probability with
some uncertainty
• If measured probability is deﬁnitely larger than 0, you will have
allowed region respectively to some C.L.
!72
Excluded and Allowed region
xbest − a < P(νμ → νx) ∼ sin2
2θ23 sin2
Δm2
31L
4Eν
< xbest + b
θ22
sin
3−
10 2−
10 1−
10 1
)4
/c2
(eV2
m∆
5−
10
4−
10
3−
10
2−
10
1−
10
1
10
2
10
3
10
P(νμ→νx)<0.01
P(νμ→νx)>0.4
Excluded region
• If measured probability is not
deﬁnitely larger than 0 but upper
limit is known with some C.L., you
will have excluding region (ref.
sterile neutrino search)
2θ23 sin2
Δm2
31L
4Eν
< xlimit
do you understand this?

!73
Excluded and Allowed region
some uncertainty
2θ23 sin2
Δm2
31L
4Eν
< xbest + b
2θ23 sin2
Δm2
31L
4Eν
< xlimit
Shape of allowed region is quite different. Why?
θ22
sin
3−
10 2−
10 1−
10 1
)4
/c2
(eV2
m∆
5−
10
4−
10
3−
10
2−
10
1−
10
1
10
2
10
3
10
Excluded region
P(νμ→νx)<0.01
P(νμ→νx)>0.4
Allowedregion
smooth the rapid variation

!74
Allowed region
some uncertainty
2θ23 sin2
Δm2
31L
4Eν
< xbest + b
2θ23 sin2
Δm2
31L
4Eν
< xlimit
Shape of allowed region is quite different. Why?

!75

2) Hypothesis test



5) Systematics


!76
Why is it important?
• Oscillation probability depends on energy

• Basically we don’t know precisely neutrino energy in
neutrino experiment —> most challenging for precision
measurement

• Neutrino energy is estimated based on the produced
particles observed in detector, but you might be fooled
and get wrong estimation on energy —> wrong
estimation on underlying parameters
P(νμ → νμ) ∼ 1 − sin2
2θ23 sin2
Δm2
31L
4Eν
0 0.5 1 1.5 2 2.5 3
NumberofEvents
0
10
20
30
40
50
60
70
80
Data
Data w/ energy bias (50+/-5 MeV)
0 0.5 1 1.5 2 2.5 3
Data/Prediction
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
[GeV]νE
0 5 10 15 20 25
p.o.t.]21
/50MeV/1x102
Flux[/cm
1−
10
1
10
2
10
3
10
4
10
5
10
6
10
fluxµν
fluxµν
fluxeν
fluxeν
Here is one example where energy is
added with a Gaussian (50+/- 5MeV)
for the red points. Effect on oscillation
parameter estimation can be followed
(you can do this.)
Oscillation dip changed

!77
How do we reconstruct energy?
• Method 1: Kinematics of out-going particle when neutrino interact with nucleon/
nuclei. selecting the interaction you know how to reconstruct energy, example
CCQE. Sometime they call “CCQE-enhanced” (ref. T2K)

• Method 2: Using hadronic information: sum of all charge in the detector (ref.
NOvA)

Comments on methods:

• On method 1: rely on kinematics, and capability of detector to look at ﬁnal state
particle —> depend on model. For example

• On method 2: neutron basically doesn’t show up in detector; charge sum is
underestimated and you have to correct it with a model.

Dream of neutrino source with well-know energy: yes, we can but not yet
However in the mean time, you have to live with this by learning how your energy
estimation eﬀects on your measured parameters

!78

2) Hypothesis test



5) Systematics


!79
Systematic sources
For neutrino experiments, there are basically three sources of errors
• Neutrino source
• Proton beam condition (use monitors but still error)
• pion/Kaon production when proton hits on target: this is the most
dominant error, external data from other experiments are used
• Current uncertainty level of 10%, but can improved
• Neutrino interaction model
• Statistic is challenging
• nuclear effect
• Final state interaction
• Detector systematics
• Secondary interaction
• Detector response
without quoting error, your result is meaningless

!80
Systematic sources
0
10
20
30
40
50
0.6 0.8 1 1.2 1.4
ValuesofvariableY
0.6
0.8
1
1.2
1.4
Histogram
Entries 10000
Mean x 0.9999
Mean y 0.9986
RMS x 0.1001
RMS y 0.1003
0
10
20
30
40
50
Histogram
Entries 10000
Mean x 0.9999
Mean y 0.9986
RMS x 0.1001
RMS y 0.1003
Histogram_py
Entries 10000
Mean 0.9986
RMS 0.1003
0 200 400 600
ValuesofvariableY
0.6
0.8
1
1.2
1.4
Histogram_px
Entries 10000
Mean 0.9999
RMS 0.1001
0.6 0.8 1 1.2 1.4
0
200
400
600
Histogram_px
Entries 10000
Mean 0.9999
RMS 0.1001
0
10
20
30
40
50
60
70
0.6 0.8 1 1.2 1.4
ValuesofvariableY
0.6
0.8
1
1.2
1.4
Histogram
Entries 10000
Mean x 0.9999
Mean y 0.9988
RMS x 0.1001
RMS y 0.09973
0
10
20
30
40
50
60
70
Histogram
Entries 10000
Mean x 0.9999
Mean y 0.9988
RMS x 0.1001
RMS y 0.09973
Histogram_py
Entries 10000
Mean 0.9988
RMS 0.09973
0 200 400 600
ValuesofvariableY
0.6
0.8
1
1.2
1.4
Histogram_px
Entries 10000
Mean 0.9999
RMS 0.1001
0.6 0.8 1 1.2 1.4
0
200
400
600
Histogram_px
Entries 10000
Mean 0.9999
RMS 0.1001
0
10
20
30
40
50
60
0.6 0.8 1 1.2 1.4
ValuesofvariableY
0.6
0.8
1
1.2
1.4
Histogram
Entries 10000
Mean x 0.9999
Mean y 0.999
RMS x 0.1001
RMS y 0.1006
0
10
20
30
40
50
60
Histogram
Entries 10000
Mean x 0.9999
Mean y 0.999
RMS x 0.1001
RMS y 0.1006
Histogram_py
Entries 10000
Mean 0.999
RMS 0.1006
0 200 400 600
ValuesofvariableY
0.6
0.8
1
1.2
1.4
Histogram_px
Entries 10000
Mean 0.9999
RMS 0.1001
0.6 0.8 1 1.2 1.4
0
200
400
600
Histogram_px
Entries 10000
Mean 0.9999
RMS 0.1001
No covariance btw 2 parameters
Positive covariance btw 2 parameters
Negative covariance btw 2 parameters
Not just systematic value matter
but covariance btw systematics is also important

!81
Systematic sources
No covariance btw 2 parameters
Positive covariance btw 2 parameters
Negative covariance btw 2 parameters
Total systematics effects
0.6 0.8 1 1.2 1.4
0
100
200
300
Histogram_effect
Entries 10000
Mean 0.998
RMS 0.1407
0.6 0.8 1 1.2 1.4
Numberofentries
0
50
100
150
200
Histogram_effect
Entries 10000
Mean 1.002
RMS 0.1751
0.6 0.8 1 1.2 1.4
Numberofentries
0
100
200
300
400
500
Histogram_effect
Entries 10000
Mean 0.9924
RMS 0.08575
Not just systematic value matter
but covariance btw systematics is also important
Because of this, global combination is non-trivial

!82
Systematic sources: T2K example
Flux parameters cross-section para.
Strong covariance
anti-covariance

!83

2) Hypothesis test



5) Systematics

6) Monte Carlo usage

!84
Monte Carlo usage
Pi calculation method
• Random throw points (dots)
• If inside the circle, you count
• Ratio of counts inside the circle to
all throw points is proportional to
area ratio of 1/4 circle and
corresponding square
https://github.com/cvson/nushortcourse/tree/master/mctoy

!85
Monte Carlo usage
• MC integral: to ﬁnd the mass and
central of mass position
mass : 1.05322 theoretical 1.05221
xcm : -0.00182585
ycm : 0.00821742
zcm : -0.266806 theoretical -0.269071 https://github.com/cvson/nushortcourse/tree/master/mctoy

!86
Monte Carlo usage
Neutrino Event generator is one example
• Neutrino follows a distribution —> NO worry, we
just generate a lot of neutrinos with energy follow
that distribution
• There are many possible interactions for a
deﬁnitive energy with different cross section —>
NO worry, there a lot of neutrinos generated, each
of them can go different interactions
• Etc….
Keep in mind: randomly go through all possibilities under predeﬁned rule
[GeV]νE
0 5 10 15 20 25
p.o.t.]21
/50MeV/1x102
Flux[/cm
1−
10
1
10
2
10
3
10
4
10
5
10
6
10
fluxµν
fluxµν
fluxeν
fluxeν
[GeV]µν
True E
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
/GeV/atom]2
[cmν/Eσ 0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
36−
10×
O
16
8Cross section on
Total
CCQE
πCC1
CC coherent
CC other
NC
O
16
8Cross section on
details from H. Van

!88
See-saw or life balances?
Keep your dream big. But keep in mind, you have to live with small things

Does it somehow link to See-saw phenomenon? Ref. Prof. Kayser’s lecture
Does he look familiar to you?

!89
Experiment is important, theory is your guidance
Does he look familiar to you?
Art McDonal, Neutrino 2018

Experimental Neutrino Physics Concepts in Nutshell

Recommended

Recommended

More Related Content

What's hot

What's hot (12)

Similar to Experimental Neutrino Physics Concepts in Nutshell

Similar to Experimental Neutrino Physics Concepts in Nutshell (20)

More from Son Cao

More from Son Cao (19)

Recently uploaded

Recently uploaded (20)

Experimental Neutrino Physics Concepts in Nutshell