SlideShare a Scribd company logo
1 of 13
Download to read offline
Global Sky Model for LOFAR and
          MSSS catalog

                                       A. Mints
                                  Hamburger Sternwarte


                            September 21, 2012




A. Mints, Global Sky Model for LOFAR and MSSS catalog    1 / 13
Global Sky Model Data Flow




A. Mints, Global Sky Model for LOFAR and MSSS catalog   2 / 13
GSM components
Database
      Storage only (all logic in the pipeline);
      MonetDB or PostgreSQL support;

Pipeline
Python code with unified database binding.

Front-ends (planned)
      Web-interface;
      Python API;
      Direct DB access;
 A. Mints, Global Sky Model for LOFAR and MSSS catalog   3 / 13
Data

Observations (pyBDSM output)
Time, frequency, Stokes parameter, position
(+size/orientation), peak/total flux, errors.

Catalog
runningcatalog Position, spectral parameters,
classification, errors, (+Frequency, Stokes
parameter, size/orientation).
runningcatalog fluxes Frequency, Stokes
parameter, peak/total flux, errors

 A. Mints, Global Sky Model for LOFAR and MSSS catalog   4 / 13
Point and extended sources


Point sources
1 record in positions table and N records in fluxes
table.
Extended sources
N+1 record in positions table and N records in
fluxes table.



 A. Mints, Global Sky Model for LOFAR and MSSS catalog   5 / 13
Matching


Matching criteria for point sources is based on de
Ruiter distance:

 2              (α1 cos δ1 − α2 cos δ2 )2     (δ1 − δ2 )2
RdeRuiter     =                           +
                  (∆α1 )2 + (∆α2 )2         (∆δ1 )2 + (∆δ2 )2




  A. Mints, Global Sky Model for LOFAR and MSSS catalog         6 / 13
Extended source matching


1. Search for a match in the same band;
2. If 1. failed – search for a cross-band match.

       ∗2                (α1 cos δ1 − α2 cos δ2 )2 + (δ1 − δ2 )2
      RdeRuiter        =             2          2
                                   gmajor ,1 + gmajor ,2




A. Mints, Global Sky Model for LOFAR and MSSS catalog              7 / 13
First snapshot

      6
      5
      4
      3
      2
      1
      0
     -1
          -1        0           1          2            3   4   5   6



A. Mints, Global Sky Model for LOFAR and MSSS catalog                   8 / 13
Second snapshot

      6
                                            2
      5
      4
      3
      2
                                                            4           3
      1
                        1
      0
     -1
          -1        0           1          2            3       4   5       6



A. Mints, Global Sky Model for LOFAR and MSSS catalog                           9 / 13
Resolved source
For point sources a “flux splitting” technique is
applied.




For extended sources flux splitting is applied only
at per-band level.

 A. Mints, Global Sky Model for LOFAR and MSSS catalog   10 / 13
Merged source


In this case the nearest source is chosen.
Exception: extended sources are merged if possible:

  A1, 2, 3 + B5, 6, 7 + X 4 = A1, 2, 3, 4, 5, 6, 7       (1)




 A. Mints, Global Sky Model for LOFAR and MSSS catalog         11 / 13
Groups



Assigned a unique group ID
(all detections in the group).




  A. Mints, Global Sky Model for LOFAR and MSSS catalog   12 / 13
Code features

     Coding style: Python PEP8 style rules;
     In-code documentation (docstrings);
     Unit testing (34 tests unified for
     MonetDB/PostgreSQL);
     SQL mini-preprocessor:
         1.    Queries stored in SQL-file, called by unique Id;
         2.    Parameters can be passed;
         3.    Global parameters can be set;
         4.    Arbitrary Python-code can be called and result
               substituted:


A. Mints, Global Sky Model for LOFAR and MSSS catalog            13 / 13

More Related Content

Similar to LOFAR Global Sky Model database

A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...
A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...
A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...MattBethel
 
Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...
Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...
Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...scalaconfjp
 
Stratosphere with big_data_analytics
Stratosphere with big_data_analyticsStratosphere with big_data_analytics
Stratosphere with big_data_analyticsAvinash Pandu
 
Brief introduction on Hadoop,Dremel, Pig, FlumeJava and Cassandra
Brief introduction on Hadoop,Dremel, Pig, FlumeJava and CassandraBrief introduction on Hadoop,Dremel, Pig, FlumeJava and Cassandra
Brief introduction on Hadoop,Dremel, Pig, FlumeJava and CassandraSomnath Mazumdar
 
Blue Waters and Resource Management - Now and in the Future
 Blue Waters and Resource Management - Now and in the Future Blue Waters and Resource Management - Now and in the Future
Blue Waters and Resource Management - Now and in the Futureinside-BigData.com
 
Geographica: A Benchmark for Geospatial RDF Stores
Geographica: A Benchmark for Geospatial RDF StoresGeographica: A Benchmark for Geospatial RDF Stores
Geographica: A Benchmark for Geospatial RDF StoresKostis Kyzirakos
 
"Petascale Genomics with Spark", Sean Owen,Director of Data Science at Cloudera
"Petascale Genomics with Spark", Sean Owen,Director of Data Science at Cloudera"Petascale Genomics with Spark", Sean Owen,Director of Data Science at Cloudera
"Petascale Genomics with Spark", Sean Owen,Director of Data Science at ClouderaDataconomy Media
 
MongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: ShardingMongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: ShardingMongoDB
 
The Fundamentals Guide to HDP and HDInsight
The Fundamentals Guide to HDP and HDInsightThe Fundamentals Guide to HDP and HDInsight
The Fundamentals Guide to HDP and HDInsightGert Drapers
 
Q4 2016 GeoTrellis Presentation
Q4 2016 GeoTrellis PresentationQ4 2016 GeoTrellis Presentation
Q4 2016 GeoTrellis PresentationRob Emanuele
 
Flume-Cassandra Log Processor
Flume-Cassandra Log ProcessorFlume-Cassandra Log Processor
Flume-Cassandra Log ProcessorCLOUDIAN KK
 
Geospatial Querying in Apache Marmotta - Apache Big Data North America 2016
Geospatial Querying in Apache Marmotta -  Apache Big Data North America 2016Geospatial Querying in Apache Marmotta -  Apache Big Data North America 2016
Geospatial Querying in Apache Marmotta - Apache Big Data North America 2016Sergio Fernández
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Safe Software
 
An open source framework for processing daily satellite images (AVHRR) over l...
An open source framework for processing daily satellite images (AVHRR) over l...An open source framework for processing daily satellite images (AVHRR) over l...
An open source framework for processing daily satellite images (AVHRR) over l...Sajid Pareeth
 
The Apache Cassandra ecosystem
The Apache Cassandra ecosystemThe Apache Cassandra ecosystem
The Apache Cassandra ecosystemAlex Thompson
 
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...Databricks
 
Unifying Space Mission Knowledge with NLP & Knowledge Graph
Unifying Space Mission Knowledge with NLP & Knowledge GraphUnifying Space Mission Knowledge with NLP & Knowledge Graph
Unifying Space Mission Knowledge with NLP & Knowledge GraphVaticle
 
Incubating Apache Hivemall
Incubating Apache HivemallIncubating Apache Hivemall
Incubating Apache HivemallMakoto Yui
 

Similar to LOFAR Global Sky Model database (20)

A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...
A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...
A LiDAR Processing Workflow Supporting LAS 1.4 and Testing for USGS NGP Base ...
 
Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...
Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...
Scarab: SAT-based Constraint Programming System in Scala / Scala上で実現された制約プログラ...
 
Stratosphere with big_data_analytics
Stratosphere with big_data_analyticsStratosphere with big_data_analytics
Stratosphere with big_data_analytics
 
Brief introduction on Hadoop,Dremel, Pig, FlumeJava and Cassandra
Brief introduction on Hadoop,Dremel, Pig, FlumeJava and CassandraBrief introduction on Hadoop,Dremel, Pig, FlumeJava and Cassandra
Brief introduction on Hadoop,Dremel, Pig, FlumeJava and Cassandra
 
Blue Waters and Resource Management - Now and in the Future
 Blue Waters and Resource Management - Now and in the Future Blue Waters and Resource Management - Now and in the Future
Blue Waters and Resource Management - Now and in the Future
 
Geographica: A Benchmark for Geospatial RDF Stores
Geographica: A Benchmark for Geospatial RDF StoresGeographica: A Benchmark for Geospatial RDF Stores
Geographica: A Benchmark for Geospatial RDF Stores
 
"Petascale Genomics with Spark", Sean Owen,Director of Data Science at Cloudera
"Petascale Genomics with Spark", Sean Owen,Director of Data Science at Cloudera"Petascale Genomics with Spark", Sean Owen,Director of Data Science at Cloudera
"Petascale Genomics with Spark", Sean Owen,Director of Data Science at Cloudera
 
Apache Spark
Apache SparkApache Spark
Apache Spark
 
MongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: ShardingMongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: Sharding
 
The Fundamentals Guide to HDP and HDInsight
The Fundamentals Guide to HDP and HDInsightThe Fundamentals Guide to HDP and HDInsight
The Fundamentals Guide to HDP and HDInsight
 
Q4 2016 GeoTrellis Presentation
Q4 2016 GeoTrellis PresentationQ4 2016 GeoTrellis Presentation
Q4 2016 GeoTrellis Presentation
 
Flume-Cassandra Log Processor
Flume-Cassandra Log ProcessorFlume-Cassandra Log Processor
Flume-Cassandra Log Processor
 
Geospatial Querying in Apache Marmotta - Apache Big Data North America 2016
Geospatial Querying in Apache Marmotta -  Apache Big Data North America 2016Geospatial Querying in Apache Marmotta -  Apache Big Data North America 2016
Geospatial Querying in Apache Marmotta - Apache Big Data North America 2016
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
An open source framework for processing daily satellite images (AVHRR) over l...
An open source framework for processing daily satellite images (AVHRR) over l...An open source framework for processing daily satellite images (AVHRR) over l...
An open source framework for processing daily satellite images (AVHRR) over l...
 
The Apache Cassandra ecosystem
The Apache Cassandra ecosystemThe Apache Cassandra ecosystem
The Apache Cassandra ecosystem
 
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
 
Unifying Space Mission Knowledge with NLP & Knowledge Graph
Unifying Space Mission Knowledge with NLP & Knowledge GraphUnifying Space Mission Knowledge with NLP & Knowledge Graph
Unifying Space Mission Knowledge with NLP & Knowledge Graph
 
Incubating Apache Hivemall
Incubating Apache HivemallIncubating Apache Hivemall
Incubating Apache Hivemall
 

More from Alexey Mints

Isochrone fitting and Gaia results
Isochrone fitting and Gaia resultsIsochrone fitting and Gaia results
Isochrone fitting and Gaia resultsAlexey Mints
 
N body dynamics (SAGE lecture)
N body dynamics (SAGE lecture)N body dynamics (SAGE lecture)
N body dynamics (SAGE lecture)Alexey Mints
 
Talk for "The X-ray Universe 2014, Dublin"
Talk for "The X-ray Universe 2014, Dublin"Talk for "The X-ray Universe 2014, Dublin"
Talk for "The X-ray Universe 2014, Dublin"Alexey Mints
 
AIP seminar - ARCHES WP6
AIP seminar - ARCHES WP6AIP seminar - ARCHES WP6
AIP seminar - ARCHES WP6Alexey Mints
 
Tab calc presentation
Tab calc presentationTab calc presentation
Tab calc presentationAlexey Mints
 

More from Alexey Mints (9)

Isochrone fitting and Gaia results
Isochrone fitting and Gaia resultsIsochrone fitting and Gaia results
Isochrone fitting and Gaia results
 
Gaia Lecture
Gaia LectureGaia Lecture
Gaia Lecture
 
N body dynamics (SAGE lecture)
N body dynamics (SAGE lecture)N body dynamics (SAGE lecture)
N body dynamics (SAGE lecture)
 
ARCHES ICF
ARCHES ICFARCHES ICF
ARCHES ICF
 
Talk for "The X-ray Universe 2014, Dublin"
Talk for "The X-ray Universe 2014, Dublin"Talk for "The X-ray Universe 2014, Dublin"
Talk for "The X-ray Universe 2014, Dublin"
 
AIP seminar - ARCHES WP6
AIP seminar - ARCHES WP6AIP seminar - ARCHES WP6
AIP seminar - ARCHES WP6
 
PhD exam
PhD examPhD exam
PhD exam
 
Cospar
CosparCospar
Cospar
 
Tab calc presentation
Tab calc presentationTab calc presentation
Tab calc presentation
 

Recently uploaded

Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsHyundai Motor Group
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphNeo4j
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Neo4j
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 

Recently uploaded (20)

Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 

LOFAR Global Sky Model database

  • 1. Global Sky Model for LOFAR and MSSS catalog A. Mints Hamburger Sternwarte September 21, 2012 A. Mints, Global Sky Model for LOFAR and MSSS catalog 1 / 13
  • 2. Global Sky Model Data Flow A. Mints, Global Sky Model for LOFAR and MSSS catalog 2 / 13
  • 3. GSM components Database Storage only (all logic in the pipeline); MonetDB or PostgreSQL support; Pipeline Python code with unified database binding. Front-ends (planned) Web-interface; Python API; Direct DB access; A. Mints, Global Sky Model for LOFAR and MSSS catalog 3 / 13
  • 4. Data Observations (pyBDSM output) Time, frequency, Stokes parameter, position (+size/orientation), peak/total flux, errors. Catalog runningcatalog Position, spectral parameters, classification, errors, (+Frequency, Stokes parameter, size/orientation). runningcatalog fluxes Frequency, Stokes parameter, peak/total flux, errors A. Mints, Global Sky Model for LOFAR and MSSS catalog 4 / 13
  • 5. Point and extended sources Point sources 1 record in positions table and N records in fluxes table. Extended sources N+1 record in positions table and N records in fluxes table. A. Mints, Global Sky Model for LOFAR and MSSS catalog 5 / 13
  • 6. Matching Matching criteria for point sources is based on de Ruiter distance: 2 (α1 cos δ1 − α2 cos δ2 )2 (δ1 − δ2 )2 RdeRuiter = + (∆α1 )2 + (∆α2 )2 (∆δ1 )2 + (∆δ2 )2 A. Mints, Global Sky Model for LOFAR and MSSS catalog 6 / 13
  • 7. Extended source matching 1. Search for a match in the same band; 2. If 1. failed – search for a cross-band match. ∗2 (α1 cos δ1 − α2 cos δ2 )2 + (δ1 − δ2 )2 RdeRuiter = 2 2 gmajor ,1 + gmajor ,2 A. Mints, Global Sky Model for LOFAR and MSSS catalog 7 / 13
  • 8. First snapshot 6 5 4 3 2 1 0 -1 -1 0 1 2 3 4 5 6 A. Mints, Global Sky Model for LOFAR and MSSS catalog 8 / 13
  • 9. Second snapshot 6 2 5 4 3 2 4 3 1 1 0 -1 -1 0 1 2 3 4 5 6 A. Mints, Global Sky Model for LOFAR and MSSS catalog 9 / 13
  • 10. Resolved source For point sources a “flux splitting” technique is applied. For extended sources flux splitting is applied only at per-band level. A. Mints, Global Sky Model for LOFAR and MSSS catalog 10 / 13
  • 11. Merged source In this case the nearest source is chosen. Exception: extended sources are merged if possible: A1, 2, 3 + B5, 6, 7 + X 4 = A1, 2, 3, 4, 5, 6, 7 (1) A. Mints, Global Sky Model for LOFAR and MSSS catalog 11 / 13
  • 12. Groups Assigned a unique group ID (all detections in the group). A. Mints, Global Sky Model for LOFAR and MSSS catalog 12 / 13
  • 13. Code features Coding style: Python PEP8 style rules; In-code documentation (docstrings); Unit testing (34 tests unified for MonetDB/PostgreSQL); SQL mini-preprocessor: 1. Queries stored in SQL-file, called by unique Id; 2. Parameters can be passed; 3. Global parameters can be set; 4. Arbitrary Python-code can be called and result substituted: A. Mints, Global Sky Model for LOFAR and MSSS catalog 13 / 13