SlideShare a Scribd company logo

Web services based workflows to deal with 3D data

1 of 32
Download to read offline
Web services based workflows
    to deal with 3D data!
                    !
     The next generation of archives!

                               Jose Enrique Ruiz del Mazo!
                                              IAA - CSIC!
Abstract!

Index!
•    AMIGA e-Science developments!
•    Archives Infrastructure!
•    Web services to the rescue!
•    Scientific workflows!
•    Wf4Ever: Preserving digital experiments!




!
Who are you ?!

The AMIGA Group!
    Analysis of the interstellar Medium of Isolated Galaxies!
    !
        Statistical baseline of isolated galaxies to compare!
        with the behaviour of galaxies in denser environments!

                  Multi     study of ~1000 galaxies!
!
IAA-CSIC!
Uuiv . Granada, Obs. Marseille, Obs. Paris, NAOJ, !
FCRAO, UNAM, Univ. Edinburgh, IRAM, ESO,!
Kapteyn Astronomical Institute.!
!
P.I. Lourdes Verdes-Montenegro!
http://amiga.iaa.es!
Who are you ?!

The Virtual Observatory!
•    Interoperability and Discovery!
•    Publishing and Accessing Data!
•    Service Oriented Architecture (SoA)!
•    Integration of Software and Data!
•    Distributed resources!
•    Panchromatic science!

•  Data Models!
•  Web Services!
•  Semantics!
!
!
Who are you ?!

The AMIGA VO Catalog!
The Data Provider!
Who are you ?!

    VO Archives Developments
Robledo DSS-63!
•  Madrid Deep Space Communication Complex (MDSCC)!
•  70m single dish in Robledo de Chavela (Madrid)!
•  5% operational time for observations!
•  K band Spectra (18 - 26 GHz)!
•  H2O Masers, methanol, NH3,..!


!
!

                           TAPAS – IRAM 30m!
                           •  Telescope Archive for Public Access System!
                           •  Bolometric observations, maps, spectra!
                           •  Rotational molecular transitions!
                           •  ~200 scientific projects / year, 1TB!

     Radio Astronomy DAta Model for Single-dish telescopes!

Recommended

Wf4Ever: Workflow Preservation
Wf4Ever: Workflow PreservationWf4Ever: Workflow Preservation
Wf4Ever: Workflow PreservationJose Enrique Ruiz
 
Curating and Preserving Collaborative Digital Experiments
Curating and Preserving Collaborative Digital ExperimentsCurating and Preserving Collaborative Digital Experiments
Curating and Preserving Collaborative Digital ExperimentsJose Enrique Ruiz
 
Collaborative Digital Experiments
Collaborative Digital ExperimentsCollaborative Digital Experiments
Collaborative Digital ExperimentsJose Enrique Ruiz
 
Augmenting interoperability across scholarly repositories
Augmenting interoperability across scholarly repositoriesAugmenting interoperability across scholarly repositories
Augmenting interoperability across scholarly repositoriesHerbert Van de Sompel
 
Use of CharDM in an archive of velocity cubes
Use of CharDM in an archive of velocity cubesUse of CharDM in an archive of velocity cubes
Use of CharDM in an archive of velocity cubesJose Enrique Ruiz
 
B0DEGA 3D VO Archive - IVOA 2010 Fall Interop
B0DEGA 3D VO Archive - IVOA 2010 Fall InteropB0DEGA 3D VO Archive - IVOA 2010 Fall Interop
B0DEGA 3D VO Archive - IVOA 2010 Fall InteropJose Enrique Ruiz
 

More Related Content

Similar to Web services based workflows to deal with 3D data

VO web-services-based astronomy workflows
VO web-services-based astronomy workflowsVO web-services-based astronomy workflows
VO web-services-based astronomy workflowsJose Enrique Ruiz
 
Workflows in the Virtual Observatory
Workflows in the Virtual ObservatoryWorkflows in the Virtual Observatory
Workflows in the Virtual ObservatoryJose Enrique Ruiz
 
ApacheCon NA 2013 VFASTR
ApacheCon NA 2013 VFASTRApacheCon NA 2013 VFASTR
ApacheCon NA 2013 VFASTRLucaCinquini
 
Multidimensional Data in the VO
Multidimensional Data in the VOMultidimensional Data in the VO
Multidimensional Data in the VOJose Enrique Ruiz
 
Curation and Characterization of Web Services
Curation and Characterization of Web ServicesCuration and Characterization of Web Services
Curation and Characterization of Web ServicesJose Enrique Ruiz
 
Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...
Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...
Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...Joint ALMA Observatory
 
Claudia Bauzer Medeiros Digital preservation – caring for our data to foster...
Claudia Bauzer Medeiros  Digital preservation – caring for our data to foster...Claudia Bauzer Medeiros  Digital preservation – caring for our data to foster...
Claudia Bauzer Medeiros Digital preservation – caring for our data to foster...Beniamino Murgante
 
Danis biosystematics2011
Danis biosystematics2011Danis biosystematics2011
Danis biosystematics2011Bruno Danis
 
Making your data work for you: Scratchpads, publishing & the biodiversity dat...
Making your data work for you: Scratchpads, publishing & the biodiversity dat...Making your data work for you: Scratchpads, publishing & the biodiversity dat...
Making your data work for you: Scratchpads, publishing & the biodiversity dat...Vince Smith
 
UCD Digital Library: Creating online access to historical and contemporary co...
UCD Digital Library: Creating online access to historical and contemporary co...UCD Digital Library: Creating online access to historical and contemporary co...
UCD Digital Library: Creating online access to historical and contemporary co...UCD Library
 
NECST @ a Glance - A bird’s eye view on the NECSTLab and on its research pro...
NECST @ a Glance - A bird’s eye view on the NECSTLab  and on its research pro...NECST @ a Glance - A bird’s eye view on the NECSTLab  and on its research pro...
NECST @ a Glance - A bird’s eye view on the NECSTLab and on its research pro...NECST Lab @ Politecnico di Milano
 
Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)
Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)
Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)Adrian Friday
 
Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)
Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)
Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)Adrian Friday
 
Datos enlazados BNE and MARiMbA
Datos enlazados BNE and MARiMbADatos enlazados BNE and MARiMbA
Datos enlazados BNE and MARiMbADaniel Vila Suero
 
Wf4Ever: Work!ows for Methodology and Science Preservation
Wf4Ever: Work!ows for Methodology and Science PreservationWf4Ever: Work!ows for Methodology and Science Preservation
Wf4Ever: Work!ows for Methodology and Science PreservationJoint ALMA Observatory
 
Workflows to access and massage VOData
Workflows to access and massage VODataWorkflows to access and massage VOData
Workflows to access and massage VODataJose Enrique Ruiz
 
Scaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data ChallengesScaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data ChallengesMatthew Vaughn
 

Similar to Web services based workflows to deal with 3D data (20)

VO web-services-based astronomy workflows
VO web-services-based astronomy workflowsVO web-services-based astronomy workflows
VO web-services-based astronomy workflows
 
Workflows in the Virtual Observatory
Workflows in the Virtual ObservatoryWorkflows in the Virtual Observatory
Workflows in the Virtual Observatory
 
Workflow Preservation
Workflow PreservationWorkflow Preservation
Workflow Preservation
 
Roberts leiden110213
Roberts leiden110213Roberts leiden110213
Roberts leiden110213
 
ApacheCon NA 2013 VFASTR
ApacheCon NA 2013 VFASTRApacheCon NA 2013 VFASTR
ApacheCon NA 2013 VFASTR
 
Multidimensional Data in the VO
Multidimensional Data in the VOMultidimensional Data in the VO
Multidimensional Data in the VO
 
Curation and Characterization of Web Services
Curation and Characterization of Web ServicesCuration and Characterization of Web Services
Curation and Characterization of Web Services
 
Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...
Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...
Building a National Virtual Observatory: The Case of the Spanish Virtual Obse...
 
Claudia Bauzer Medeiros Digital preservation – caring for our data to foster...
Claudia Bauzer Medeiros  Digital preservation – caring for our data to foster...Claudia Bauzer Medeiros  Digital preservation – caring for our data to foster...
Claudia Bauzer Medeiros Digital preservation – caring for our data to foster...
 
Danis biosystematics2011
Danis biosystematics2011Danis biosystematics2011
Danis biosystematics2011
 
Making your data work for you: Scratchpads, publishing & the biodiversity dat...
Making your data work for you: Scratchpads, publishing & the biodiversity dat...Making your data work for you: Scratchpads, publishing & the biodiversity dat...
Making your data work for you: Scratchpads, publishing & the biodiversity dat...
 
UCD Digital Library: Creating online access to historical and contemporary co...
UCD Digital Library: Creating online access to historical and contemporary co...UCD Digital Library: Creating online access to historical and contemporary co...
UCD Digital Library: Creating online access to historical and contemporary co...
 
NECST @ a Glance - A bird’s eye view on the NECSTLab and on its research pro...
NECST @ a Glance - A bird’s eye view on the NECSTLab  and on its research pro...NECST @ a Glance - A bird’s eye view on the NECSTLab  and on its research pro...
NECST @ a Glance - A bird’s eye view on the NECSTLab and on its research pro...
 
Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)
Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)
Towards Open Pervasive Displays (Keynote at Tekes UbiSummit, May 2011)
 
Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)
Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)
Towards Open Pervasive Displays (Keynote at UbiSummit, Helsinki, May 2011)
 
Datos enlazados BNE and MARiMbA
Datos enlazados BNE and MARiMbADatos enlazados BNE and MARiMbA
Datos enlazados BNE and MARiMbA
 
Wf4Ever: Work!ows for Methodology and Science Preservation
Wf4Ever: Work!ows for Methodology and Science PreservationWf4Ever: Work!ows for Methodology and Science Preservation
Wf4Ever: Work!ows for Methodology and Science Preservation
 
Our World is Socio-technical
Our World is Socio-technicalOur World is Socio-technical
Our World is Socio-technical
 
Workflows to access and massage VOData
Workflows to access and massage VODataWorkflows to access and massage VOData
Workflows to access and massage VOData
 
Scaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data ChallengesScaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data Challenges
 

More from Jose Enrique Ruiz

Jupyter notebooks on steroids
Jupyter notebooks on steroidsJupyter notebooks on steroids
Jupyter notebooks on steroidsJose Enrique Ruiz
 
IPython Notebooks - Hacia los papers ejecutables
IPython Notebooks - Hacia los papers ejecutablesIPython Notebooks - Hacia los papers ejecutables
IPython Notebooks - Hacia los papers ejecutablesJose Enrique Ruiz
 
Implementing a VO archive for datacubes of galaxies
Implementing a VO archive for datacubes of galaxiesImplementing a VO archive for datacubes of galaxies
Implementing a VO archive for datacubes of galaxiesJose Enrique Ruiz
 
Open Science and Executable Papers
Open Science and Executable PapersOpen Science and Executable Papers
Open Science and Executable PapersJose Enrique Ruiz
 
Digital Science: Towards the executable paper
Digital Science: Towards the executable paperDigital Science: Towards the executable paper
Digital Science: Towards the executable paperJose Enrique Ruiz
 
Digital Science: Reproducibility and Visibility in Astronomy
Digital Science: Reproducibility and Visibility in AstronomyDigital Science: Reproducibility and Visibility in Astronomy
Digital Science: Reproducibility and Visibility in AstronomyJose Enrique Ruiz
 
El Observatorio Virtual - eCA
El Observatorio Virtual - eCAEl Observatorio Virtual - eCA
El Observatorio Virtual - eCAJose Enrique Ruiz
 
Wf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science i
Wf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science iWf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science i
Wf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science iJose Enrique Ruiz
 

More from Jose Enrique Ruiz (11)

Jupyter notebooks on steroids
Jupyter notebooks on steroidsJupyter notebooks on steroids
Jupyter notebooks on steroids
 
IPython Notebooks - Hacia los papers ejecutables
IPython Notebooks - Hacia los papers ejecutablesIPython Notebooks - Hacia los papers ejecutables
IPython Notebooks - Hacia los papers ejecutables
 
Velocity cubes of galaxies
Velocity cubes of galaxiesVelocity cubes of galaxies
Velocity cubes of galaxies
 
Implementing a VO archive for datacubes of galaxies
Implementing a VO archive for datacubes of galaxiesImplementing a VO archive for datacubes of galaxies
Implementing a VO archive for datacubes of galaxies
 
Open Science and Executable Papers
Open Science and Executable PapersOpen Science and Executable Papers
Open Science and Executable Papers
 
Digital Science: Towards the executable paper
Digital Science: Towards the executable paperDigital Science: Towards the executable paper
Digital Science: Towards the executable paper
 
Digital Science: Reproducibility and Visibility in Astronomy
Digital Science: Reproducibility and Visibility in AstronomyDigital Science: Reproducibility and Visibility in Astronomy
Digital Science: Reproducibility and Visibility in Astronomy
 
Research Objects in Wf4Ever
Research Objects in Wf4EverResearch Objects in Wf4Ever
Research Objects in Wf4Ever
 
Digital Science
Digital ScienceDigital Science
Digital Science
 
El Observatorio Virtual - eCA
El Observatorio Virtual - eCAEl Observatorio Virtual - eCA
El Observatorio Virtual - eCA
 
Wf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science i
Wf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science iWf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science i
Wf4Ever: Advanced Workflow Preservation Technologies for Enhanced Science i
 

Recently uploaded

A TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTS
A TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTSA TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTS
A TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTSDr.M.Geethavani
 
Plant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptxPlant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptxAKSHAYMAGAR17
 
11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdfAynouraHamidova
 
Mycobacteriology update 2024 Margie Morgan.ppt
Mycobacteriology update 2024 Margie Morgan.pptMycobacteriology update 2024 Margie Morgan.ppt
Mycobacteriology update 2024 Margie Morgan.pptMargie Morgan
 
New Features in the Odoo 17 Sales Module
New Features in  the Odoo 17 Sales ModuleNew Features in  the Odoo 17 Sales Module
New Features in the Odoo 17 Sales ModuleCeline George
 
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in GhanaDecision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in GhanaPrince Armah, PhD
 
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdfAynouraHamidova
 
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?Aggregage
 
BBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptx
BBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptxBBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptx
BBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptxProf. Kanchan Kumari
 
Capitol Doctoral Presentation -Feb 2024.pptx
Capitol Doctoral Presentation -Feb 2024.pptxCapitol Doctoral Presentation -Feb 2024.pptx
Capitol Doctoral Presentation -Feb 2024.pptxCapitolTechU
 
Diploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdf
Diploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdfDiploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdf
Diploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdfSUMIT TIWARI
 
Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...
Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...
Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...Rabiya Husain
 
The basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptxThe basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptxdeputymitchell2
 
2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptx2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptxMaryPotorti1
 
EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...
EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...
EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...liera silvan
 
A Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdf
A Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdfA Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdf
A Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdfOH TEIK BIN
 
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxMaria Lucia Céspedes
 
Biology 152 - Topic Statement - Boolean Search
Biology 152 - Topic Statement - Boolean SearchBiology 152 - Topic Statement - Boolean Search
Biology 152 - Topic Statement - Boolean SearchRobert Tomaszewski
 

Recently uploaded (20)

A TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTS
A TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTSA TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTS
A TEXTBOOK OF INTELLECTUAL ROPERTY RIGHTS
 
Plant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptxPlant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptx
 
Lipids as Biopolymer
Lipids as Biopolymer Lipids as Biopolymer
Lipids as Biopolymer
 
11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 5-2023-Aynura-Hamidova.pdf
 
Mycobacteriology update 2024 Margie Morgan.ppt
Mycobacteriology update 2024 Margie Morgan.pptMycobacteriology update 2024 Margie Morgan.ppt
Mycobacteriology update 2024 Margie Morgan.ppt
 
New Features in the Odoo 17 Sales Module
New Features in  the Odoo 17 Sales ModuleNew Features in  the Odoo 17 Sales Module
New Features in the Odoo 17 Sales Module
 
CLUBE PERLINGUAS .
CLUBE PERLINGUAS                        .CLUBE PERLINGUAS                        .
CLUBE PERLINGUAS .
 
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in GhanaDecision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
 
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
 
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
 
BBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptx
BBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptxBBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptx
BBA 603 FUNDAMENTAL OF E- COMMERCE UNIT 1.pptx
 
Capitol Doctoral Presentation -Feb 2024.pptx
Capitol Doctoral Presentation -Feb 2024.pptxCapitol Doctoral Presentation -Feb 2024.pptx
Capitol Doctoral Presentation -Feb 2024.pptx
 
Diploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdf
Diploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdfDiploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdf
Diploma 2nd yr PHARMACOLOGY chapter 5 part 1.pdf
 
Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...
Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...
Barrow Motor Ability Test - TEST, MEASUREMENT AND EVALUATION IN PHYSICAL EDUC...
 
The basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptxThe basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptx
 
2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptx2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptx
 
EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...
EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...
EmpTech Lesson 7 - Online Creation Tools, Platforms, and Applications for ICT...
 
A Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdf
A Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdfA Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdf
A Free eBook ~ Mental Exercise ...Puzzles to Analyze.pdf
 
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
 
Biology 152 - Topic Statement - Boolean Search
Biology 152 - Topic Statement - Boolean SearchBiology 152 - Topic Statement - Boolean Search
Biology 152 - Topic Statement - Boolean Search
 

Web services based workflows to deal with 3D data

  • 1. Web services based workflows to deal with 3D data! ! The next generation of archives! Jose Enrique Ruiz del Mazo! IAA - CSIC!
  • 2. Abstract! Index! •  AMIGA e-Science developments! •  Archives Infrastructure! •  Web services to the rescue! •  Scientific workflows! •  Wf4Ever: Preserving digital experiments! !
  • 3. Who are you ?! The AMIGA Group! Analysis of the interstellar Medium of Isolated Galaxies! ! Statistical baseline of isolated galaxies to compare! with the behaviour of galaxies in denser environments! Multi study of ~1000 galaxies! ! IAA-CSIC! Uuiv . Granada, Obs. Marseille, Obs. Paris, NAOJ, ! FCRAO, UNAM, Univ. Edinburgh, IRAM, ESO,! Kapteyn Astronomical Institute.! ! P.I. Lourdes Verdes-Montenegro! http://amiga.iaa.es!
  • 4. Who are you ?! The Virtual Observatory! •  Interoperability and Discovery! •  Publishing and Accessing Data! •  Service Oriented Architecture (SoA)! •  Integration of Software and Data! •  Distributed resources! •  Panchromatic science! •  Data Models! •  Web Services! •  Semantics! ! !
  • 5. Who are you ?! The AMIGA VO Catalog! The Data Provider!
  • 6. Who are you ?! VO Archives Developments Robledo DSS-63! •  Madrid Deep Space Communication Complex (MDSCC)! •  70m single dish in Robledo de Chavela (Madrid)! •  5% operational time for observations! •  K band Spectra (18 - 26 GHz)! •  H2O Masers, methanol, NH3,..! ! ! TAPAS – IRAM 30m! •  Telescope Archive for Public Access System! •  Bolometric observations, maps, spectra! •  Rotational molecular transitions! •  ~200 scientific projects / year, 1TB! Radio Astronomy DAta Model for Single-dish telescopes!
  • 7. Who are you ?! The AMIGA Group! Analysis of the interstellar Medium of Isolated Galaxies! ! Statistical baseline of isolated galaxies to compare! with the behaviour of galaxies in denser environments! ! Multi study of ~1000 galaxies! +! Need of intensive and complex analysis of 3D data! 2D spatial + 1 Velocity!
  • 8. Who are you ?! Velocity Datacubes! ! M. Krips – ESO 3D2008 Workshop – Garching!
  • 9. Who are you ?! GIPSY! Groningen Image Processing SYstem! Connectivity ! •  VO Archives ! •  VO Software! ! Accessibility! •  Usability GUI! •  VO Web Services! ! Kapteyn Astronomical Institute! IAA – CSIC!
  • 10. Who are you ?! B0DEGA Below 0 DEgrees GAlaxies! P.I. : D. Espada! Legacy project of Submillimiter Array interferometer (SMA)! http://b0dega.iaa.es! ! IAA-CSIC! CfA (Harvard-Smithsonian Center for Astrophysics)! ASIAA (Institute of Academia Sinica Astronomy and Astrophysics) ! ! Molecular gas properties of a survey of nearby galaxies.! 30 processed and reduced datacubes of galaxies!
  • 11. Who are you ?! The B0DEGA 3D VO Catalog! The Data and Service provider! Aladin VO Software!
  • 12. Infrastructure! The next generation of archives! ! Much wider FoV and spectral coverage! •  Large volumes for an observed datacube! •  Subproducts are Virtual Data generated on-the-fly! Automated surveys ! •  Huge amounts of tabular data! •  Services for Knowledge Discovery in Databases!
  • 13. Infrastructure! Cube sizes! ! ASKAP Cubes! Prof. Kevin Vinsen !
  • 15. Infrastructure! Use of the European Grid Infrastructure May 2010 – May 2011 !
  • 16. Infrastructure! Use of the European Grid Infrastructure May 2010 – May 2011 !
  • 17. Infrastructure! Use of the European Grid Infrastructure May 2010 – May 2011 !
  • 18. A Cloud of Services! The overall picture! ! Distributed, scalable and flexible infrastructure! •  Grid + Cloud may solve storage and processing! •  Bandwidth is the issue! ! Big Data Science performance is highly dependent upon I/O data rates (local and transfer)! ! The data is the infrastructure! •  Interconnected and interoperable archives! •  Distributed, multi-wavelength and multi-facilities! ! Archives speaking Web Services! ALMA, LSST, ASKAP, MeerKAT, LOFAR, Apertif,...!
  • 19. A Cloud of Services! 3D Data Services! ! •  Cutout! •  Resample! •  Spectrum extraction! •  2D slice extraction! •  Dimensional reduction! •  Filtering/Flagging! •  2D Moments! •  Complex transformations! !
  • 20. A Cloud of Services! Exploration services! KDD - Knowledge Discovery in Databases! Understand what information is contained within the data in order to know how we can efficiently extract it ! •  Anomaly detection! •  Cross-matching data! •  Dimensionality reduction! ! Extraction of scientifically ! relevant information from a! multidimensional parameter space.! visIt software !
  • 21. Scientific Workflows! Web-services-based Users oriented workflows ARE NOT PIPELINES! ! •  Archived science-ready data! •  Automation! •  Repeatable results ! •  Reproducible! •  Encourage best practices! •  Modular nature allows ! •  Re-use! •  Re-purpose! •  Expose the scientific method! •  Formative!
  • 22. Scientific Workflows! Taverna! A workflow enactment and management system! ! University of Manchester !
  • 23. WORKFLOWS J.E. Ruiz - IX Reunión Científica de la SEA – Madrid- 15/09/2010
  • 24. WORKFLOWS J.E. Ruiz - IX Reunión Científica de la SEA – Madrid- 15/09/2010
  • 26. Wf4Ever! Curating and preserving collaborative digital experiments 1.  Intelligent Software Components (ISOCO, Spain)! 2.  University of Manchester (UNIMAN, UK)! 2 7 3.  Universidad Politécnica de Madrid (UPM, Spain)! 5! 4! 4.  Poznan Supercomputing and Networking Centre (PSNC, Poland)! 5.  Universisty of Oxford (OXF, UK)! 6.  Instituto Astrofísica Andalucía (IAA-CSIC, Spain)! 1! 3! 7.  Leiden University Medical Centre (LUMC, NL)! 6!
  • 27. Wf4Ever! All components related to the research lifecycle available, preserved and easily retrievable. ! ! ! •  Proposals! •  Data! •  Processes! •  Workflows! •  Publications! ! http://wf4ever-project.org/
  • 28. Wf4Ever! The recipes store! ! •  Find workflows! •  Share workflows and files! •  Find people! •  Build communities! •  Publish packages! •  Tag workflows! •  Score and rate workflows! •  Comment on workflows! •  Write reviews! •  Access rights! Oxford University!
  • 29. Wf4Ever! Astronomy research is entirely digital ! Time has come to go “Beyond the PDF”! •  Preserved experiments! •  Methodology “in action”! •  All data exposed! •  Reproducible! •  Repeatable! •  Re-usable! •  Re-purposeable! •  Collaborative! •  Formative!
  • 30. Wf4Ever! Open questions for Web Services! In the Virtual Observatory! ! •  Curation and preservation (identifiers)! •  Discovery (semantics) of web services! •  Characterization: input, outputs, functionality, etc.! •  Copies (authenticity) or similar used as alternates ! •  Permissions (authentication), licenses, platform, costs,..! •  Metrics for quality: popularity, use stats, logs uptime, etc.! •  Versioning and authoring (referenced and acknowledged)! ! In a cloud of services and data, Web Services should benefit of the same privileges acquired by Data.!
  • 31. Conclusions! ! We are moving into a world where ! •  computing and storage are cheap ! •  data movement is death! ! Archives should evolve from data providers into virtual data and services providers, where web services may help to solve bandwidth issues.! ! Web Services! •  Smaller virtual data subproducts! •  Distributed, multi-archive, multi-wavelength astronomy! •  Workflows as a disruptive working methodology! !