1. Multidimensional Data in the
Virtual Observatory
Jose Enrique Ruiz del Mazo
Tutora: Dr. Lourdes Verdes-Montenegro
IAA - CSIC
Máster FISYMAT
Trabajo de Investigación Tutelada
Universidad de Granada
Diciembre 2010

2. SUMMARY
Context
• The AMIGA project
• The Virtual Observatory
• Multidimensional Data in Astronomy
• AMIGA VO Contributions
Generic Datasets
• Data Discovery
• Data Access
Generic Dataset Discovery Service
• Input Parameters
• Query Response
• Implementation
Conclusions and Future Work

3. AMIGA
Analysis of the interstellar Medium of Isolated GAlaxies
PI : Dr. Lourdes Verdes-Montenegro
IAA-CSIC, IRAM
http://amiga.iaa.es
Obs. Marseille, Obs. Paris, CfA, ASIAA, MPIfA, IAC,
Univ. Alabama, Mc Donald Observatory, Arcetri, UNAM,
Kapteyn Astronomical Institute
Need of a statistically significant sample of isolated galaxies, in order
to provide a baseline to compare with the behaviour of galaxies in
denser environments
Multiλ analysis ~1000 galaxias
+
Need of intensive and complex analysis of 3D data
2D spatial + 1 Velocity

4. VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way

5. VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way

6. VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way

7. VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way

8. MULTIDIMENSIONAL DATA
Observational Techniques
• Radiointerferometry
• Integral Field Spectroscopy
• Multi Object Spectroscopy
• Fabry-Perot Instruments
• OTF Imaging
Credit
Stephen Todd and Douglas Pierce-Price
Credit
M. Westmoquette

9. AMIGA VO CONTRIBUTIONS
AMIGA Catalog
• ConeSearch Service
• Web Interface
RADAMS
Radio Astronomy Data Model for Single-dish telescopes
Juan de Dios Santader-Vela
Robledo DSS-63 VO Archive
• ConeSearch Service
• SSA Service
• Web Interface
TAPAS
Telescope Archive for Public Access System
IRAM-30m VO Archive
• ConeSearch Service
• Web Interface

10. GENERIC DATASETS
Typed Datasets SIMPLE ACCESS PROTOCOLS
• Tabular Data
• 1D Spectra
• 2D Images
• 3D Cubes
• Time Series

11. GENERIC DATASETS
Typed Datasets SIMPLE ACCESS PROTOCOLS
• Tabular Data
• 1D Spectra
• 2D Images
• 3D Cubes
• Time Series
Generic / MultiTyped Datasets
Complex data associations of different individual types
Survey Field
• Spectral data cube
• 2D projections/extractions of the cube
• Source catalog computed from the 2-D continuum
• Some extracted spectra of objects in the field

12. GENERIC DATASETS
Typed Datasets SIMPLE ACCESS PROTOCOLS
• Tabular Data
• 1D Spectra
• 2D Images
• 3D Cubes
• Time Series
Generic / MultiTyped Datasets
Complex data associations of different individual types
Survey Field
• Spectral data cube
• 2D projections/extractions of the cube
• Source catalog computed from the 2-D continuum
• Some extracted spectra of objects in the field

13. DATA DISCOVERY
Associated Data Collections
• Simple Access Protocols perform discovery of associated data
• Present IVOA models provide full description
• REF-ID mechanism in VOTable allow association of data
• RESOURCE mechanism in VOTable allow metadata extension

14. DATA DISCOVERY
Associated Data Collections
• Simple Access Protocols perform discovery of associated data
• Present IVOA models provide full description
• REF-ID mechanism in VOTable allow association of data
• RESOURCE mechanism in VOTable allow metadata extension
Issues
• Original data products can be very large, worsening transfer rates and latency
• Clients applications do not support all native observatory-dependent formats
• Users are often not interested in the whole product but in a smaller portion

15. DATA DISCOVERY
Associated Data Collections
• Simple Access Protocols perform discovery of associated data
• Present IVOA models provide full description
• REF-ID mechanism in VOTable allow association of data
• RESOURCE mechanism in VOTable allow metadata extension
Issues
• Original data products can be very large, worsening transfer rates and latency
• Clients applications do not support all native observatory-dependent formats
• Users are often not interested in the whole product but in a smaller portion
Virtual Data from Uniformly Sampled Datasets
• Data generated on-the-fly at access time based on user demands
• Discovery implies negotiation with the service for access methods
• WCS metadata needed for most virtual data generation

16. DATA ACCESS
Virtual Data generation may require
asynchronous services deployed on
distributed GRID architectures
Whole dataset
Filtering/Flagging
Spectrum extraction
2D slices extraction
Dimensional reduction
Cutout 3D sub-cube
General 2D projection
General 3D projection
General 2D slices through a 3D cube
Complex transformations

17. GDS DISCOVERY INPUTS
REQUEST=queryData SPECRES
SPATRES
POS TIMERES RESOLUTION
SIZE FLUXLIMIT
BAND SNR PRECISION
TIME VARAMPL
SENSITIVITY
POL FLUXCALIB
COVERAGE
FORMAT WAVECALIB
ASTCALIB
REDSHIFT
REGION PUBID
INTERSECT CREATORID PUBLISHER
COLLECTION
TARGETNAME MTIME
TARGETCLASS TARGET
TYPE TOP
MAXREC
SERVICE
COMPRESS
RUNID

18. GDS DISCOVERY OUTPUT
Query
Association
Access
DataSet: Declaration of Spatial, Time and Polarization Axis CONSISTENCY
DataID
Provenance: BeamMajorAxis, BeamMinorAxis, BeamPositionAngle INSTRUMENTAL
Curation
Target: Velocity
Derived: DerivedVelocity, VelocityStatError, VelocityConfidence PHYSICS
VarAmplStatError, VarAmplConfidence
CoordSys: RedshiftFrameUcd
Char.SpatialAxis
Char.SpectralAxis
Char.TimeAxis
CONSISTENCY
Char.FluxAxis: FluxAverage, FluxMin, FluxSaturation, FluxSupportExtent
Char.PolarizationAxis PHYSICS

19. GDS IMPLEMENTATION

20. GDS IMPLEMENTATION

21. GDS IMPLEMENTATION

22. accessData

23. getCapabilities

24. CONCLUSIONS
• Study of the state of the art of both MultiD data in Astronomy and Protocols in the VO
• Determine the best strategy for discovery and access of complex MultiD datasets
• Propose Discovery Method for a GDS conceived in the less possible intrusive way
• Reuse of existing VO Data Models and VO Protocols with minor modifications
• Implementation of the proposed Discovery Method for a GDS

25. FUTURE WORK
• Virtual Data generation and accesData standards needed
• Achieve final IVOA recommendation for MultiD discovery and access VO protocols
• Upcoming facilities will provide 3D datacubes and services to access and use them
• getCapabilities method is key for interoperability among services
• Conception and development of VO Scientific Workflows for 3D Data Analysis
• EU funded project Wf4Ever
Advanced Workflow Preservation Technologies for Enhanced Sience