This document provides instructions for performing photometric reduction of image data from the Joan Oró Telescope (TJO) at the Observatori Astronòmic del Montsec (OAdM) using Python packages. It describes installing necessary software, selecting targets from Gaia Science Alerts, cleaning image data, applying astrometry with Astrometry.net, running source extraction with SExtractor, and uploading calibrated data to the Cambridge Photometry Calibration Server. The goal is to produce light curves of transient targets for Gaia follow-up observations within 24 hours of observation.

1. PHOTOMETRIC REDUCTION PROCEDURE
WITH TJO OBSERVATIONS FOR GAIA
SCIENCE ALERTS
Umut BURGAZ
burgaz.umut@gmail.com
University of Barcelona
04/2015

2. Revision Sheet
Release No. Date Revision Description
Rev. 0 18/04/2015 Creation of Template & First Draft

3. TABLE OF CONTENTS
1.0 GENERAL INFORMATION...............................................................................................
1.1 System Overview........................................................................................................
1.2 Installation Notes........................................................................................................
1.2.1 Python...............................................................................................................
1.2.2 Astropy.............................................................................................................
1.2.3 CCDProc...........................................................................................................
1.2.4 Astrometry.net.................................................................................................
1.2.5 Sextractor..........................................................................................................
2.0 TARGET SELETIONN.........................................................................................................
3.0 PHOTOMETRIC REDUCTION.........................................................................................
3.1 Cleaning the Data.......................................................................................................
3.2 Applying the Astrometry..........................................................................................
3.3 Applying the Sextractor.............................................................................................
3.4 Uploading the Data to the Calibration Server......................................................

4. 1.0 GENERAL INFORMATION

5. 1.0 GENERAL INFORMATION
This manual explains the photometric reduction of the data received from Joan
Oró Telescope(TJO) at Observatori Astronòmic del Montsec(OAdM) step by step with
all the priori installations for future usage. In this manual photometric reduction will be
done by Astropy.
1.1 System Overview
Before starting the reduction procedure, one should have all the listed programs
and packages installed and ready. If you have already some of the packages installed,
the current versions of python packages can be checked after opening and importing the
one by the command : packagename.__version__
 Python 2.6.5 , 2.7.x , 3.3.x or 3.4.x
 Astropy 1.0
 Numpy 1.6.0 or later
 Matplotlib 1.4.3
 Ccdproc 0.3
 Astrometry.net 0.5
 Scipy 0.13.3 or later
 Sextractor 2.19.5
For all astronomers, Python 2.7 is the most useful and recommended language
version. It is also known that Astropy has bugs and more difficulties with Python 3
versions. All the process that will be explained in this manual is done with Python 2.7.5.

6. 1.2 Installation Notes
For the first time installers for everything, Python 2.6,2.7,3.3,3.4 and around 270
packages(including Astropy, Numpy, Matplotlib, Scipy, ccdproc) that Python uses can
be gained by a simple installation of Anaconda Scientific Python Distribution.
1.2.1 Python
Python can be installed for all the environments with ease. All the versions of
Python for specific installations are available on the official site of Python downloads
page.
Most recommended one for scientific usage is the version 2.7.
1.2.2 Astropy
Astropy is the main package our reduction process requires. Before installing
Astropy, one should already have installed Numpy, Matplotlib and Scipy which can be
easily done from a terminal in Ubuntu & Debian with the following command :
$ sudo apt-get install python-numpy python-scipy python-matplotlib
Upon successful installation of the packages on top, one can install Astropy using
pip with simply calling :
$ pip install --no-deps astropy
Pip is a general easy install tool for python packages. If you don't have pip, it can
be installed before calling the astropy installation by :
$ python get-pip.py
Also make sure that you have the installing compilers such as gcc. This can be
checked and obtained by :
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install build-essential
$ gcc -v
$ make -v

7. If any permission errors occurs during the installation of Astropy, sudo command
can be used to bypass that permission errors.
Astropy can also be installed from the binary installers by downloading the
related version from the page : astropy and then simply using the build and install
commands from the root of the source directory :
$ python setup.py build
$ python setup.py install
If you have installed Anaconda package all you have to do is just run :
$ conda update astropy
1.2.3 CCDProc
To install with pip we simply run :
$ pip install --no-deps ccdproc
Also can be installed from the binary installers by downloading the latest version
from the page : ccdproc and then simply using the build and install commands from the
root of the source directory :
$ python setup.py build
$ python setup.py install
1.2.4 Astrometry.net
Astrometry is what we use to give our data(from OAdM), astronomic coordinates
compared with 2MASS data fits we have(previusly downloaded index files). This is a
fast and automatic procedure. It basically compares two fields and gives the coordinates
for the matched targets.
Installing Astrometry.net is quite easy. There are more than one way to get the
program but simply all we have to do is write these commands on the terminal :
$ wget http://astrometry.net/downloads/astrometry.net-0.46.tar.bz2
$ tar xjf astrometry.net-0.46.tar.bz2
$ cd astrometry.net-0.46
$ make
$ make py
$ make extra

8. $ make install
In order to have full functionality, we will have to check or get if we have all the
required packages ;
 GNU build tools (gcc/clang, make, etc.)
 cairo
 netpbm
 libpng
 libjpeg
 libz
 bzip2
 python
 numpy
We can acquire all this packages by the following command :
$ sudo apt-get install libcairo2-dev libnetpbm10-dev netpbm libpng12-dev libjpeg-dev
python-numpy python-pyfits python-dev zlib1g-dev libbz2-dev swig cfitsio-dev
In order to use Astrometry.net, we have to download several fields that we will
be using. This can be done through the web page : Astrometry
One has to download all between index-4202-00.fits and index-4202-47.fits
After downloading all the index files, we need to put them into the
corresponding folder. Which can be checked from “cfg” file of the program. If we want
to change the current location of the data or need to keep it on some other places other
than the default path, we can change the add_path to the desired location such as :
add_path /home/USERNAME/astrometry/data
autoindex

9. 1.2.5 Sextractor
Sextractor is a software for source extraction. It creates a catalog of objects from
an astronomical image. To obtain Sextractor(for Ubuntu 14.04 or higher) :
$ sudo apt-get install sextractor
Sextractor can also be downloaded from the official web page : Sextractor.
After downloading the Source file we uncompress the tar.gz file and go to that
directory and :
$ ./configure
$ make
$ sudo make install
If we get an error on the part ./configure , we can try using :
$ ./configure --disable-model-fitting
In order to be able to use the Sextractor for our purposes we need the
configuration and parameter files called (default.param and default.sex) and we need to
modify the default.param to :
MAG_AUTO
MAGERR_AUTO
ALPHA_J2000
DELTA_J2000
Also we will need to change the configuration file(default.sex) with :
CATALOG_NAME to TJOtest.cat
CATALOG_TYPE to ASCII_HEAD
PARAMETERS_NAME to /home/umutt/default.param #wherever our
param file is
We can specifically put our file to any location we want in my case I use home
folder and we can call the program while using such as :
sex -c /home/umutt/default.sex IMAGENAME.new

10. 2.0 TARGET SELECTION

11. 2.0 TARGET SELECTION
Target selection for TJO observations are done by daily controlling of the Gaia
Science Alerts web page. Not all the alerts are published as the same date when Gaia
detected the alerts. Generally, alerts come within 5 to 10 days after the alert detection
date since all this process is still manually done by the Gaia Photometric Science Alerts
Team. Also not all the detected alerts are published via the web page. At first Science
Alerts Team performs some criteria to the alerts and the chosen ones are published in
the web page.
Before starting to observe any target I would suggest to go the Cambridge
Photometry Calibration Server and check for the target if any observations were done by
other observatories and etc. This might give some ideas about the current state of the
alert.
Selection of the target is based on 3 steps. First and the most important one is the
magnitude limit. As a starting point anything brighter than V<17.5 magnitude is good
enough to do follow-ups since day by day the target will loose its brightness, the
brighter the starting magnitude the more time we will have to observe the target.
Another important condition is the alert publication date since these transients
loose their magnitudes very quickly, the sooner we start the observations, the more
possibly we will be able to catch a maximum and we will have the opportunity to
observe the target for a longer period of time and will end up with a more complete
light curve.
Last but not least, another important point is the star altitude, we must have
enough time to observe the target during the night. This condition must be checked
considering an airmass below 1.55 (above 40 degrees) in order to keep the atmospheric
air turbulence minimum. In general we use 4 filters and for each filter we ask for 3
exposures each having 300 seconds of exposure time. Therefore, a minimum of 1 hour
and 5 minutes of time is required for these conditions.
After the first observation we would have some ideas about the current state of
the alert like if it is still in the magnitude reach and etc. After a few days of observations
each target must be checked to see if the light curves are stable or not. Stable light curve
would most probably mean that the alert is missed (recovered its original state by the
time our observations started). Also if we start to get very faint results with big errors,
we must stop observing as well. Day by day checking the CPCS page can give more
ideas about each target and how our light curve is getting built. There might be other
observatories contributing for the same target so we might see more data points.

12. 3.0 PHOTOMETRIC REDUCTION

13. 3.0 PHOTOMETRIC REDUCTION
In this part of the manual, I will describe the photometric reduction that we need
to do before submitting the data to the Gaia Consortium. What we will do is basically,
first clean the observation data of TJO from the effects of bias, dark and flats.
After that we will use Astrometry.net to give the astronomic coordinates to the
sources in the images we have. Then, we will use Sextractor to create the final catalog
file that we will submit to the Cambridge Calibration Server page.
3.1 Cleaning the Data
Our first step to start the reduction process is to clean the data we receive from
OAdM. In order to do that we use Astropy and CCDProc. What we basically do is
combine the flats, combine the bias and apply the darks to our scientific raw
images(imr.fits) on the folder.
In order to do that, we will use a python code that can be obtained from here. We
put this code to the folder where our data is. Then we open a terminal and go to that
directory and call ; python clean_data.py
Successful trials always end up with new files named “imc.fits” created. These
files are the cleaned files we need. After having these files, we can delete all the other
outcomes and everything else.
3.2 Applying the Astrometry.net
Upon cleaning the data what we have to do is first call the Astrometry in
order to give the coordinates to the stars we have on our images. This process is
automatically done by comparing the index files we have downloaded and the
new images we have.
We will be using the command solve-field in order to do that. An example
of the full code that we need is :
$ solve-field --use-sextractor --overwrite TJO2456957.61754_R_imc.fits
--ra 43.7 --dec -6.46 --radius 1 --no-plots

14. The code we have includes the followings :
ra : Right Ascension(ICRS, decimal degrees)
dec : Declination (ICRS, decimal degrees)
use-sextractor : Sextractor can be called and used to extract the sources
overwrite : Uses our image to create a same named file with .new format
radius 1 : Matching radius is 1 by default
no-plots : After the process no plots will appear
We have to use this code to each image we have separately because in one
folder there may be more than one targets and for each target we will have to
modify our RA and DEC in our code.
Successful completion of the process will end up with the creation of 8 new
files. What we will need is the one with the format “.new”
Our code will end up something similar like this upon successful
completion :
Fig.1 : Astrometry result on terminal
We can delete all the files created during this process except “imc.new”
If there are any errors, make sure you have entered the correct ra and dec
for the target you are using at that moment. That is essential to do the
comparison purposes with the index files.

15. 3.3 Applying the Sextractor
After successfully obtaining the “imc.new” file with Astrometry.net we need to
extract the sources from that image with the Sextractor in order to create a catalog. This
can simply be done with the terminal by calling :
$ sex -c /home/umutt/default.sex TJO2457127.55590_U_imc.new
or depending on the installation method it can be :
$ sextractor -c /home/umutt/default.sex TJO2457127.55590_U_imc.new
Upon successful completion we will receive a catalog file named TJOtest.cat and
our terminal will look like something similar to the image below :
Fig.2 : Sextractor result on terminal
3.4 Uploading the Data to the Calibration Server
Once we have the catalog file created by the Sextractor, what we need to do is to
upload it to the Cambridge Photometry Calibration Server (CPCS). The main purpose of
the CPCS is to provide homogeneous photometric follow ups within the astronomers
working with Gaia Science Alerts. Our aim is to provide informations and build a light
curve with the targets we observe. For that purposes we should clean and upload the
data we have received within 24 hours.
In order to be able to upload any data to the CPCS we need a hastag to login the
system. Which can be obtained from Dr. Lukasz Wyrzykowski.

16. After logging in, we click on Upload new followup data. There we will have some
informations to be filled :
Event ID : The name of the target we have observed, must begin with
ivo:// such as ivo://Gaia15adb , In case our target does not exist we can create it from
the main page of the CPCS by clicking Enter new event and filling all the necessary
informations
MJD OBS : Modified Julien Data of the observation. Julien Date of the
observation can be obtained from the Header of the image with ds9 or some other
imaging program. Then, MJD = JD - 2400000.5
Exposure time(sec) : Can also be obtained from the Header of the image if
unknown.
Comment(optional) : If we have any comment about uploading the data
Sextractor catalog : We need to upload the TJOtest.cat data here
Matching radius : We choose 1 arcsec
Force filter : The filter of the image we are uploading, can be obtained
from the Header of the image. In case we do not know, No (automatic determination)
can be chosen. When clicked we will see options such as SDSS, APASS, VSTATLAS,
OGLE3 and so on. We will be uploading our data in APASS form for all the bands( B-V-
R-I). Using the automatic determination is not encouraged.
Dry Run : This is a mode that will not store the data on the server
upon uploading. We can use this mode to see which filter we have in case we don't
know it. We can also check how it behaves with other filters and so on.
In case of uploading the data somehow wrong, the data point can be deleted
from the system by going the Delete a followup point from the system from the home
page.
We can access any data uploaded to the server including our data and light curve
from the Lists of alerts page from the home page.
Upon successful upload we can see our data point in the system both from the
list of follow-up data page and from the light curve such as in case of Gaia15adb :

17. Light curve will be currently updated upon uploading new data, not only by us
but also everyone else. There we will be seeing the apparent magnitude of our target we
have uploaded. Which is automatically determined by the field stars upon uploading
the server.
We can also obtain the apparent magnitude of our data by going to the catalog
file and searching for the coordinates of our target and identifying it. There we will be
seeing the instrumental magnitude of our target. What we need to do now is the
differential photometry. We can check the coordinates of the known stars from Vizier
and get their apparent magnitude to determine the factor between our instrumental
magnitude and the apparent magnitude. Then we can apply this factor to our target and
obtain the apparent magnitude of our target.