1. SCIDB AS A PLATFORM FOR DESI DATA ANALYIS
ANALYSIS of LARGE SCALE SRUCTURE
MATTER POWER SPECTRUM
TWO POINT CORRELATION FUNCTION
STRONG SCALINGWEAK SCALING
SCIDB PERFORMANCE
CONSTRAINING COSMOLOGIES
FUTURE APPLICATION
ACKOWLEDGMENTS
HALO FINDING
PARALLEL TOOLBOX
William P. Comaskey1, Peter Nugent2, Mentor Yushu Yao2
1Florida Institute of Technology, 2Lawrence Berkeley National Laboratory
DARK ENERGY SPECTROSCOPIC INSTRUMENT:
DEVELOPING A HIGH PERFORMANCE PARALLEL TOOLBOX IN SCIDB
DARK ENERGY
ABSTRACT
ACCELERATING UNIVERSE
COSMIC MICROWAVE
BACKGROUND
FiFing the baryon acoustic
oscillation feature
The acoustic peak present in
the two-‐‑point correlation
function can be used to
measure the the ratio of
d i s t a n c e s t o v a r i o u s
redshifts with accuracies of
less than 5%. The full shape
of the two-‐‑ point correlation
function is capable of
determining Ω h2 to nearly
identical values obtained by
the Wilkinson Microwave
Anisotropy Probe (2003) and
the Plank satellite (2013).
Density Fluctuations in a
given region are defined
by,
where the power spectrum
is the Fourier partner of
the correlation function,
and can be used in unison
w i t h t h e t w o -‐‑ p o i n t
correlation function to
beKer measure the dilation
parameter of the baryon
a c o u s t i c o s c i l l a t i o n
feature.
Galactic Clustering -‐‑ A cluster of galaxies is described as an over density in
the projected distribution of galaxies. Determining the size of these clusters
helps to constrain cosmological parameters. By obtaining the number
density of clusters of various sizes at different redshifts the expansion rate
and cosmological parameters of the universe can be inferred.
Correlation Functions
The correlation function,
measures the excess clustering of
galaxies at a separation r. The
correlation function can be used to
determine the epoch at which a
certain cluster size develops or to
predict the number of pairs of
galaxies within two volumes
surrounding separated galaxies.
Dark MaFer Haloes are large clusters of Dark maKer
that play a large role in the Large-‐‑Scale structure.
Friends-‐‑of-‐‑Friends – Uses a linking length in order to
find Haloes by progressively matching objects within
the linking length of an object within a Halo to that
same Halo.
Two-Point correlation function using two different
estimators on a down sampled simulated region of
utilizing SciDB with simulated errors.
Two, Three and Four-point correlation
functions. Photo-credit: www.astroml.org
The galaxy redshift–space correlation
f u n c t i o n f i t t e d v i a d i f f e r e n t
Cosmologies. Photo-credit:
Nature Insight Review: Cosmology from
start to finish – Charles. Bennett
Thanks to the Department of Energy’s Workforce
Development of Teachers and Scientists as well as
Workforce Development & Education at Berkeley Lab
and also the scientists of the Computational
Cosmology Center at LBNL and Professor Hakeem
Oluseyi of Florida Institute of Technology.
The observation that the universe appears to be
expanding was confirmed in 1998, by the observation
of high redshift Type Ia supernovae. This evidence has
since been supported by the observation of baryon
acoustic oscillations and the measurement of the mass
function for clusters of galaxies.
The cosmic microwave background (CMB) is thermal
radiation consisting of the oldest light in the Universe.
The CMB was produced after the epoch of
recombination which allowed light travel
unobstructed. The baryon acoustic oscillations detected
in the cosmic microwave background created shells of
baryonic maKer surrounding Dark maKer at fixed radii,
known as the sound horizon.
I present the validity of utilizing the database SciDB as
the primary framework for storing and analyzing
observational data from the upcoming Dark Energy
Spectroscopic Instrument (DESI) experiments. The
validity of SciDB was determined by analyzing the
databases intrinsic parallel analysis and parallel
loading implementations, the accessibility of the SciDB
interfaces in R and Python, and the Scaling of
computational algorithms performed within SciDB.
SciDB scaled in a nearly ideal manner in both the
strong and weak scaling tests. Terra-bytes of data were
also able to be loaded into SciDB arrays using parallel
and serial loading in an efficient manner. A software
package was developed in both R and Python in order
to serve as a front end for the SciDB framework and
allow for seamless interaction and manipulation of
cosmological data in order to interpret the Large-scale
structure of the Universe. The powerful and functional
nature of SciDB and the constituent packages
developed herein for cosmological analysis prove
SciDB requires further implementation and
development in cosmology and is fully capable of
serving as the framework for DESI data analysis.
δ(⃗r)
δ(⃗r) ≡
ρ(⃗r) − ¯ρ
¯ρ
ρ(r) ¯ρ
δ(⃗k) = d3
⃗rδ(⃗r)e−i⃗k·⃗r
δ(⃗r) =
d3⃗k
(2π)3
δ(⃗k)ei⃗k·⃗r
δ(⃗k) δ(⃗r)
ξ(r)
r
dP12(r) = ¯n2
(1 + ξ(r)) dV1 · dV2
¯n
ξ(r)
ξ(r)
dV1 ¯n1 · dV1
r dV2
ζ(r12, r23, θ)
dP123(r12, r23, θ) = ¯n3
(1 + ξ(r12) + ξ(r23) + ξ(r13(θ)) + ζ(r12, r23, θ)) dV1·dV2·dV3
r12 r23 r13(θ) θ r12 r23
ζ(r12, r23, θ)
ζ(r12, r23, θ) dP123
ξ(r)
ζ(r12, r23, θ)
ζ(r12, r23, θ)
ξ(r)
ξ(r) = ⟨δ(⃗x)δ(⃗x + ⃗r)⟩
|⃗r| r
ζ(r12, r13, θ) = ⟨δ(⃗x)δ(⃗x + ⃗r12)δ(⃗x + ⃗r23)⟩
The Cosmology based toolbox developed as separate
packages within both the Python and R, will allow
scientists who have liKle to no experience with High
Performance (HPC) and Parallel computing to be able
perform massively scalable analytics on Large-‐‑Scale
cosmological problems within an accessible
framework.
A map projection of the full-sky CMB anisotropy
observed by the Planck Satellite in 2013.
Photo-credit: http://www.czechnationalteam.cz
Density fluctuations measured by
various surveys.
Photo-credit: The invisible Universe:
Dark Energy and Dark Matter
A Visual display of the Friends-
of-Friends algorithm with color
coded clusters.
The Friends-of-Friends used to
identify haloes over 250Mpc3
Photo-credit: hipacc.ucsc.edu
Spherical Over-‐‑density Halo finder
Photo-‐‑credit: hKp://www.aanda.org ASOHF: a new adaptive
spherical overdensity halo finder -‐‑ S. Planelles and V. Quilis
The Covariance distance matrix is analyzed while the
amount of work per computer core is held constant.
The Covariance distance matrix is analyzed while the total
amount of work is held constant. (i.e. 503 Data)
The Wiener–Khintchin theorem states that, for a statistically homogeneous
random field, the two–point correlation function is the Fourier transform of
the power–spectrum (§2.4):
ξ(x) = d3⃗kexp(i⃗k ·⃗x)P(k)
P(k) =
1
(2π)3
d3
⃗xexp(−i⃗k ·⃗x)ξ(x). (300)
The power–spectrum is, roughly speaking, proportional to the mean square
amplitude of Fourier modes of the distribution. For power law primordial
spectra with a short range cut-off: P(k) ∝ kn
k < kc, P(k) = 0 k ≥ kc,
ξ(x) ∝ x−(3+n)
(n > −3), x >> k−1
c ; which can be used to deduce the power
spectrum from a knowledge of ξ in regions where ξ can be represented as a
power law. Being actually a spectral density function, however, P(k) must have
units of volume [43]. This simple fact seems to have caused much confusion in
the literature, and many authors insist on inserting unnecessary factors of the
sample volume V into equation (300) to account for this [440,430]. To avoid
Spherical Overdensity – The Dark maKer Haloes are
found by means of identifying isolated density peaks.
Their masses are determined within a series of radii
which encompass the overdensity.
Hamilton
Landy & Szalay
The massive scale of the observational data that will be
received from the telescope on a nightly basis necessitates
the construction of an accessible parallel toolbox to analyze
the incoming data. The SciDB database framework
intrinsically analyzes data in parallel as mathematical N-‐‑
dimensional arrays and is capable of massive parallel
loading of data.
The Dark Energy Spectroscopic Instrument
(DESI) will measure the effect of dark energy
on the expansion of the universe. It will
obtain optical spectra for tens of millions of
galaxies and quasars, constructing a 3-‐‑
dimensional map spanning the nearby
universe to 10 billion light years.
Kitt Peak 4-meter
Mayall Telescope
![SCIDB AS A PLATFORM FOR DESI DATA ANALYIS
ANALYSIS of LARGE SCALE SRUCTURE
MATTER POWER SPECTRUM
TWO POINT CORRELATION FUNCTION
STRONG SCALINGWEAK SCALING
SCIDB PERFORMANCE
CONSTRAINING COSMOLOGIES
FUTURE APPLICATION
ACKOWLEDGMENTS
HALO FINDING
PARALLEL TOOLBOX
William P. Comaskey1, Peter Nugent2, Mentor Yushu Yao2
1Florida Institute of Technology, 2Lawrence Berkeley National Laboratory
DARK ENERGY SPECTROSCOPIC INSTRUMENT:
DEVELOPING A HIGH PERFORMANCE PARALLEL TOOLBOX IN SCIDB
DARK ENERGY
ABSTRACT
ACCELERATING UNIVERSE
COSMIC MICROWAVE
BACKGROUND
FiFing the baryon acoustic
oscillation feature
The acoustic peak present in
the two-‐‑point correlation
function can be used to
measure the the ratio of
d i s t a n c e s t o v a r i o u s
redshifts with accuracies of
less than 5%. The full shape
of the two-‐‑ point correlation
function is capable of
determining Ω h2 to nearly
identical values obtained by
the Wilkinson Microwave
Anisotropy Probe (2003) and
the Plank satellite (2013).
Density Fluctuations in a
given region are defined
by,
where the power spectrum
is the Fourier partner of
the correlation function,
and can be used in unison
w i t h t h e t w o -‐‑ p o i n t
correlation function to
beKer measure the dilation
parameter of the baryon
a c o u s t i c o s c i l l a t i o n
feature.
Galactic Clustering -‐‑ A cluster of galaxies is described as an over density in
the projected distribution of galaxies. Determining the size of these clusters
helps to constrain cosmological parameters. By obtaining the number
density of clusters of various sizes at different redshifts the expansion rate
and cosmological parameters of the universe can be inferred.
Correlation Functions
The correlation function,
measures the excess clustering of
galaxies at a separation r. The
correlation function can be used to
determine the epoch at which a
certain cluster size develops or to
predict the number of pairs of
galaxies within two volumes
surrounding separated galaxies.
Dark MaFer Haloes are large clusters of Dark maKer
that play a large role in the Large-‐‑Scale structure.
Friends-‐‑of-‐‑Friends – Uses a linking length in order to
find Haloes by progressively matching objects within
the linking length of an object within a Halo to that
same Halo.
Two-Point correlation function using two different
estimators on a down sampled simulated region of
utilizing SciDB with simulated errors.
Two, Three and Four-point correlation
functions. Photo-credit: www.astroml.org
The galaxy redshift–space correlation
f u n c t i o n f i t t e d v i a d i f f e r e n t
Cosmologies. Photo-credit:
Nature Insight Review: Cosmology from
start to finish – Charles. Bennett
Thanks to the Department of Energy’s Workforce
Development of Teachers and Scientists as well as
Workforce Development & Education at Berkeley Lab
and also the scientists of the Computational
Cosmology Center at LBNL and Professor Hakeem
Oluseyi of Florida Institute of Technology.
The observation that the universe appears to be
expanding was confirmed in 1998, by the observation
of high redshift Type Ia supernovae. This evidence has
since been supported by the observation of baryon
acoustic oscillations and the measurement of the mass
function for clusters of galaxies.
The cosmic microwave background (CMB) is thermal
radiation consisting of the oldest light in the Universe.
The CMB was produced after the epoch of
recombination which allowed light travel
unobstructed. The baryon acoustic oscillations detected
in the cosmic microwave background created shells of
baryonic maKer surrounding Dark maKer at fixed radii,
known as the sound horizon.
I present the validity of utilizing the database SciDB as
the primary framework for storing and analyzing
observational data from the upcoming Dark Energy
Spectroscopic Instrument (DESI) experiments. The
validity of SciDB was determined by analyzing the
databases intrinsic parallel analysis and parallel
loading implementations, the accessibility of the SciDB
interfaces in R and Python, and the Scaling of
computational algorithms performed within SciDB.
SciDB scaled in a nearly ideal manner in both the
strong and weak scaling tests. Terra-bytes of data were
also able to be loaded into SciDB arrays using parallel
and serial loading in an efficient manner. A software
package was developed in both R and Python in order
to serve as a front end for the SciDB framework and
allow for seamless interaction and manipulation of
cosmological data in order to interpret the Large-scale
structure of the Universe. The powerful and functional
nature of SciDB and the constituent packages
developed herein for cosmological analysis prove
SciDB requires further implementation and
development in cosmology and is fully capable of
serving as the framework for DESI data analysis.
δ(⃗r)
δ(⃗r) ≡
ρ(⃗r) − ¯ρ
¯ρ
ρ(r) ¯ρ
δ(⃗k) = d3
⃗rδ(⃗r)e−i⃗k·⃗r
δ(⃗r) =
d3⃗k
(2π)3
δ(⃗k)ei⃗k·⃗r
δ(⃗k) δ(⃗r)
ξ(r)
r
dP12(r) = ¯n2
(1 + ξ(r)) dV1 · dV2
¯n
ξ(r)
ξ(r)
dV1 ¯n1 · dV1
r dV2
ζ(r12, r23, θ)
dP123(r12, r23, θ) = ¯n3
(1 + ξ(r12) + ξ(r23) + ξ(r13(θ)) + ζ(r12, r23, θ)) dV1·dV2·dV3
r12 r23 r13(θ) θ r12 r23
ζ(r12, r23, θ)
ζ(r12, r23, θ) dP123
ξ(r)
ζ(r12, r23, θ)
ζ(r12, r23, θ)
ξ(r)
ξ(r) = ⟨δ(⃗x)δ(⃗x + ⃗r)⟩
|⃗r| r
ζ(r12, r13, θ) = ⟨δ(⃗x)δ(⃗x + ⃗r12)δ(⃗x + ⃗r23)⟩
The Cosmology based toolbox developed as separate
packages within both the Python and R, will allow
scientists who have liKle to no experience with High
Performance (HPC) and Parallel computing to be able
perform massively scalable analytics on Large-‐‑Scale
cosmological problems within an accessible
framework.
A map projection of the full-sky CMB anisotropy
observed by the Planck Satellite in 2013.
Photo-credit: http://www.czechnationalteam.cz
Density fluctuations measured by
various surveys.
Photo-credit: The invisible Universe:
Dark Energy and Dark Matter
A Visual display of the Friends-
of-Friends algorithm with color
coded clusters.
The Friends-of-Friends used to
identify haloes over 250Mpc3
Photo-credit: hipacc.ucsc.edu
Spherical Over-‐‑density Halo finder
Photo-‐‑credit: hKp://www.aanda.org ASOHF: a new adaptive
spherical overdensity halo finder -‐‑ S. Planelles and V. Quilis
The Covariance distance matrix is analyzed while the
amount of work per computer core is held constant.
The Covariance distance matrix is analyzed while the total
amount of work is held constant. (i.e. 503 Data)
The Wiener–Khintchin theorem states that, for a statistically homogeneous
random field, the two–point correlation function is the Fourier transform of
the power–spectrum (§2.4):
ξ(x) = d3⃗kexp(i⃗k ·⃗x)P(k)
P(k) =
1
(2π)3
d3
⃗xexp(−i⃗k ·⃗x)ξ(x). (300)
The power–spectrum is, roughly speaking, proportional to the mean square
amplitude of Fourier modes of the distribution. For power law primordial
spectra with a short range cut-off: P(k) ∝ kn
k < kc, P(k) = 0 k ≥ kc,
ξ(x) ∝ x−(3+n)
(n > −3), x >> k−1
c ; which can be used to deduce the power
spectrum from a knowledge of ξ in regions where ξ can be represented as a
power law. Being actually a spectral density function, however, P(k) must have
units of volume [43]. This simple fact seems to have caused much confusion in
the literature, and many authors insist on inserting unnecessary factors of the
sample volume V into equation (300) to account for this [440,430]. To avoid
Spherical Overdensity – The Dark maKer Haloes are
found by means of identifying isolated density peaks.
Their masses are determined within a series of radii
which encompass the overdensity.
Hamilton
Landy & Szalay
The massive scale of the observational data that will be
received from the telescope on a nightly basis necessitates
the construction of an accessible parallel toolbox to analyze
the incoming data. The SciDB database framework
intrinsically analyzes data in parallel as mathematical N-‐‑
dimensional arrays and is capable of massive parallel
loading of data.
The Dark Energy Spectroscopic Instrument
(DESI) will measure the effect of dark energy
on the expansion of the universe. It will
obtain optical spectra for tens of millions of
galaxies and quasars, constructing a 3-‐‑
dimensional map spanning the nearby
universe to 10 billion light years.
Kitt Peak 4-meter
Mayall Telescope](https://image.slidesharecdn.com/fd1a1f11-881d-482c-af07-0518b5b55001-150226195934-conversion-gate02/85/Comaskey_William_Poster_SULI_FALL_2014-1-320.jpg)
