Successfully reported this slideshow.                                                                          Upcoming SlideShare
×

# VO Course 11: Spatial indexing

Introduction to the problem of spatial indexing for spherical coordinate systems, as those used in astronomy. Part of the virtual observatory course by Juan de Dios Santander Vela, as imparted for the MTAF (Métodos y Técnicas Avanzadas en Física, Advanced Methods and Techniques in Physics) Master at the University of Granada (UGR).

• Full Name
Comment goes here.

Are you sure you want to Yes No ### VO Course 11: Spatial indexing

1. 1. Spatial IndexingJuan de Dios Santander Vela (IAA-CSIC)
2. 2. Why Spatial Indexing?Spherical coordinates Wrap Around Singularities at the Poles Changing Width
3. 3. Why Spatial Indexing?Spherical coordinates Wrap Around Singularities at the Poles Changing Width
4. 4. Why Spatial Indexing?Spherical coordinates Wrap Around Singularities at the Poles Changing Width
5. 5. Why Spatial Indexing?Query on Arbitrary Shapes Object Contours Instrument Footprints Region IntersectionsRegion operations integral to ADQL!
6. 6. Using partitioningschemesQuery applications Spatial joinsX-Match applicationsDisplay applicationsCore spatial region support
7. 7. Spherical Coordinates x = r sin q cos f y = r sin q sin f z = r cos q x = sin q cos f y = sin q sin f z = cos q
8. 8. Typical Query Case: Cone SearchSELECT * FROM TARGET_TABLE WHERE 2 * ASIN( SQRT(SIN((\$DEC_C-DEC)/2) * SIN((\$DEC_C-DEC)/2) + COS(\$DEC_C) * COS(DEC) * SIN((\$RA_C - RA)/2) * SIN((\$RA_C - RA)/2))) <= \$SR_CSELECT * FROM RASS_PHOTONS WHERE X*\$X_C+Y*\$Y_C+Z*\$Z_C >= COS(\$SR_C)
9. 9. Spatial Indexing ElementsHalf-Space A.K.A NT AI STR CON
10. 10. Spatial Indexing Elements A POLYGON IS A UNION OF CONVEXESConvex
11. 11. Spatial Indexing Elements LEVEL 0 LEVEL 1 LEVEL 2HTM Indexing &Trixels
12. 12. Spatial Indexing ElementsHTM Indexing &Trixels
13. 13. Indexing SchemesUnordered inﬁnite plane versus ordered ﬁniteplaneDiﬀerent coordinate systems needed for diﬀerentthings Cartesian SphericalPlus, discontinuities in both!
14. 14. Indexing SchemesEasy query/look-up of plane or sphericalcoordinate-based objectsHierarchical (suitable for wavelet, storagelocality…)Adaptability to transformations (sphericalharmonics, wavelet, FFT…)
15. 15. Indexing SchemesPartitioning HEALPix HTM Q3C Mostly quad-tree-like schemes
16. 16. HEALPix
17. 17. HEALPixBy Górsky, Hivon, Banday, et al. (ﬁrst ESO, laterJPL)Main concerns: Hierarchical structure Equal area partitioning Isolatitude distribution
19. 19. HEALPix
20. 20. HEALPix Astronomy Astrophysics manuscript no. mhg (DOI: will be inserted by hand later) June 20, 2005 Mapping on the HEALPix grid M. R. Calabretta1 and B. F. Roukema2 1 Australia Telescope National Facility, PO Box 76, Epping, NSW 1710, Australia 2 Toru´ Centre for Astronomy, N. Copernicus University, ul. Gagarina 11, PL-87-100 Toru´ , Poland n n Draft dated 2005/06/15 Abstract. The natural spherical projection associated with the Hierarchical Equal Area and isoLatitude Pixelisation, HEALPix, is described and shown to be one of a hybrid class that combines the cylindrical equal-area and Collignon projections, not previously documented in the cartographic literature. Projection equations are derived for the class in general and it is shown that the HEALPix projection suggests a simple method (a) of storing, and (b) visualising data sampled on the grid of the HEALPix pixelisation, and also suggests an extension of the pixelisation that is better suited for these purposes. Potentially useful properties of other members of the class are described, and new triangular and hexagonal pixelisations are constructed from them. Finally, the formalism is deﬁned for representing the celestial coordinate system for any member of the class in the FITS data format. Key words. astronomical data bases: miscellaneous – cosmic microwave background – cosmology: observations – methods: data analysis, statistical – techniques: image processing 1. Introduction The Hierarchical Equal Area and isoLatitude Pixelisation, HEALPix (G´ rski et al. 2004), o↵ers a scheme for distributing o 12N 2 (N 2 ) points as uniformly as possible over the surface of the unit sphere subject to the constraint that the points lie on a relatively small number (4N 1) of parallels of latitude and are equispaced in longitude on each of these parallels. These 120 0 -120 properties were chosen to optimise spherical harmonic analy- 90 -90 sis and other computations performed on the sphere. 0 In fact, HEALPix goes further than simply deﬁning a dis- 90 tribution of points; it also speciﬁes the boundary between ad- 75 jacent points and does so in such a way that each occupies the 60 45 60 same area. Thus HEALPix is described as an equal area pix- 30 elisation. Pixels at the same absolute value of the latitude have 15 the same shape in the equatorial region, though pixel shape dif- 0 fers between latitudes, and with longitude in the polar regions. -15 -30 The boundaries for N = 1 deﬁne the 12 base-resolution pix- -45 els and higher-order pixelisations are deﬁned by their regular -60 -60 180 90 -75 0 -90 -180 subdivision. Note, however, that although they are four-sided, HEALPix pixels are not spherical quadrilaterals because their 135 45 -45 -135 edges are not great circle arcs. Fig. 1. The HEALPix class of projections for H = 1, 2, 3 rescaled to HEALPix was originally described purely with reference unit area (top), and the nominative case with H = 4 (bottom) at twice to the sphere, the data itself being stored as a one-dimensional the linear scale and with the top left-hand corner of the graticule “cut array in a FITS binary table (Cotton et al. 1995) with either away” to reveal the underlying cylindrical equal-area projection in the equatorial region. Facets are shown as dashed diamonds. ring or nested organisation, the former being suited for spher- ical harmonic analysis and the latter for nearest-neighbour searches. For visualisation purposes the software that imple- Send o↵print requests to: M. Calabretta, ments HEALPix o↵ers a choice of four conventional projection e-mail: mcalabre@atnf.csiro.au types onto which HEALPix data may be regridded.
21. 21. HEALPix
22. 22. HEALPix HEALPix radians measured eastward. sphere are given by the foll North polar cap.— For i Nside , and the pixel-in-r qﬃﬃ i¼I p j¼pþ z¼ ¼ jÀ 2i North equatorial belt.— F i 2Nside , and 1 j 4Ns i ¼ I ð p0 j ¼ ð p0 m z¼ Fig. 3.—Orthographic view of the HEALPix partition of the sphere. The s overplot of equator and meridians illustrates the octahedral symmetry of ¼ j À ; an HEALPix. Light gray shading shows one of the 8 (4 north and 4 south) identical 2 Nside 2
23. 23. HEALPix HEALPix radians measured eastward. sphere are given by the foll North polar cap.— For i Nside , and the pixel-in-r qﬃﬃ i¼I p j¼pþ z¼ ¼ jÀ 2i North equatorial belt.— F i 2Nside , and 1 j 4Ns i ¼ I ð p0 j ¼ ð p0 m z¼ Fig. 3.—Orthographic view of the HEALPix partition of the sphere. The s overplot of equator and meridians illustrates the octahedral symmetry of ¼ j À ; an HEALPix. Light gray shading shows one of the 8 (4 north and 4 south) identical 2 Nside 2
24. 24. HEALPix HEALPix radians measured eastward. sphere are given by the foll North polar cap.— For i Nside , and the pixel-in-r qﬃﬃ i¼I p j¼pþ z¼ ¼ jÀ 2i North equatorial belt.— F i 2Nside , and 1 j 4Ns i ¼ I ð p0 j ¼ ð p0 m z¼ Fig. 3.—Orthographic view of the HEALPix partition of the sphere. The s overplot of equator and meridians illustrates the octahedral symmetry of ¼ j À ; an HEALPix. Light gray shading shows one of the 8 (4 north and 4 south) identical 2 Nside 2
25. 25. HEALPix
26. 26. HEALPix 0011 0111 1011 1111
27. 27. HEALPix 10100 10101 10110 10111
28. 28. HEALPix
29. 29. HEALPix Fig. 4.—Layout of the HEALPix pixels on the sphere in a cylindrical projection and a demonstration of two possible pixel indexations—one running onisolatitude rings, the other arranged hierarchically or in a nested tree fashion. The top two panels correspond to Nside ¼ 2 in ﬁrst ring then nested schemes; the bottomtwo panels are for Nside ¼ 4. refers to the colatitude and to the longitude.
30. 30. HEALPix FITS Astronomy Astrophysics manuscript no. mhg (DOI: will be inserted by hand later) June 20, 2005 Mapping on the HEALPix grid M. R. Calabretta1 and B. F. Roukema2 1 Australia Telescope National Facility, PO Box 76, Epping, NSW 1710, Australia 2 Toru´ Centre for Astronomy, N. Copernicus University, ul. Gagarina 11, PL-87-100 Toru´ , Poland n n Draft dated 2005/06/15 Abstract. The natural spherical projection associated with the Hierarchical Equal Area and isoLatitude Pixelisation, HEALPix, is described and shown to be one of a hybrid class that combines the cylindrical equal-area and Collignon projections, not previously documented in the cartographic literature. Projection equations are derived for the class in general and it is shown that the HEALPix projection suggests a simple method (a) of storing, and (b) visualising data sampled on the grid of the HEALPix pixelisation, and also suggests an extension of the pixelisation that is better suited for these purposes. Potentially useful properties of other members of the class are described, and new triangular and hexagonal pixelisations are constructed from them. Finally, the formalism is deﬁned for representing the celestial coordinate system for any member of the class in the FITS data format. Key words. astronomical data bases: miscellaneous – cosmic microwave background – cosmology: observations – methods: data analysis, statistical – techniques: image processing 1. Introduction The Hierarchical Equal Area and isoLatitude Pixelisation, HEALPix (G´ rski et al. 2004), o↵ers a scheme for distributing o 12N 2 (N 2 ) points as uniformly as possible over the surface of the unit sphere subject to the constraint that the points lie on a relatively small number (4N 1) of parallels of latitude and are equispaced in longitude on each of these parallels. These 120 0 -120 properties were chosen to optimise spherical harmonic analy- 90 -90 sis and other computations performed on the sphere. 0 In fact, HEALPix goes further than simply deﬁning a dis- 90 tribution of points; it also speciﬁes the boundary between ad- 75 jacent points and does so in such a way that each occupies the 60 45 60 same area. Thus HEALPix is described as an equal area pix- 30 elisation. Pixels at the same absolute value of the latitude have 15 the same shape in the equatorial region, though pixel shape dif- 0 fers between latitudes, and with longitude in the polar regions. -15 -30 The boundaries for N = 1 deﬁne the 12 base-resolution pix- -45 els and higher-order pixelisations are deﬁned by their regular -60 -60 180 90 -75 0 -90 -180 subdivision. Note, however, that although they are four-sided, HEALPix pixels are not spherical quadrilaterals because their 135 45 -45 -135 edges are not great circle arcs. Fig. 1. The HEALPix class of projections for H = 1, 2, 3 rescaled to HEALPix was originally described purely with reference unit area (top), and the nominative case with H = 4 (bottom) at twice to the sphere, the data itself being stored as a one-dimensional the linear scale and with the top left-hand corner of the graticule “cut array in a FITS binary table (Cotton et al. 1995) with either away” to reveal the underlying cylindrical equal-area projection in the equatorial region. Facets are shown as dashed diamonds. ring or nested organisation, the former being suited for spher- ical harmonic analysis and the latter for nearest-neighbour searches. For visualisation purposes the software that imple- Send o↵print requests to: M. Calabretta, ments HEALPix o↵ers a choice of four conventional projection e-mail: mcalabre@atnf.csiro.au types onto which HEALPix data may be regridded.
31. 31. HEALPix FITS M. R. Calabretta: Mapping on the HEALPix grid 5 75 is very much an artifact of the distortions inherent in the pro- 60 45 jection. Because the -coordinate varies non-linearly with θ, 30 15 on the sphere they are actually biased to one side of the pixel. 0 Hence some degree of bias is unavoidable. -15 -30 -45 -60 -75 2.3.2. Hexagonal – H = 3 120 0 -120 The division into 3 × 120◦ suggests hexagonal base-resolution √ Fig. 5. HEALPix projection for H = 3 scaled in by 1/ 3 whereupon pixels. Although the familiar “honeycomb” structure shows it becomes three consecutive hexagons. that it is possible to tile the plane with hexagons, neverthe- less there is no bounded tesselation of hexagons by hexagons. That is, a hexagonal region may not be cut out of a honey- by equilateral triangles. This pixelisation may be deﬁned by √ comb tesselation without cutting the individual elements. Thus rescaling the HEALPix projection with H = 6 by 3 in it may seem surprising that a hexagonal pixelisation can be con- so that the half-facets become equilateral triangles. Such a lin- structed from the HEALPix projection for H = 3 with scaled √ ear scaling does not aﬀect the projection’s equal area property. by 1/ 3. The boundary of this projection, as seen in Fig. 5, is What were previously half-facets may now be identiﬁed with reduced to that of three sequential hexagons. This boundary is 36 new, triangular base-resolution pixels that may be subdi- then used conceptually as a “pie-cutter” on a honeycomb tesse- vided in a hierarchical way as for HEALPix, as depicted in lation of the right scale. Pixels that are cut can be made whole Fig. 4. It is interesting to note that this subdivision is natu- again by borrowing from adjacent facets, much as the square rally hierarchical - the number of pixels varies exponentially as pixelisation in Fig. 3 does. √ 36 × 4N−1 where N is the hierarchy level. In the H = 4 pixelisa- Rescaling Eqs. (13) and (14) gives H◦ / 3 = (2.7, 2.5, tion the exponential hierarchy must be engineered by doubling 2.0, 1.5, 1.5, 1.7, 1.8, 1.8) for θ = (0, 15, 30, θ×, 45, 60, 75, 90). N. Thus the rescaled H = 3 projection does not achieve confor- The conformal latitude computed for H = 6 with this mality at any latitude, it does well close to the equator, but ◦ extra -scaling is θ◦ = 30. 98, indicating that the projec- degrades at mid-latitudes. In the polar regions the 30◦ angle tion becomes conformal at the latitude that bisects the hemi- between meridians and parallels along the edge of the facets sphere by area. Applying Eqs. (13) and (14) with the extra √ is further from the ideal of 90◦ than the 45◦ for the unscaled scaling gives 3H◦ = (8.2, 7.6, 6.1, 4.5, 4.6, 5.1, 5.3, 5.4) for projections. θ = (0, 15, 30, θ×, 45, 60, 75, 90), again indicating less distor- tion in the polar regions than H = 4. It also does better in the polar zone away from the centreline because the 60◦ angle 2.4. HPX: HEALPix in FITS along the edge of the polar facets more closely matches the true In this section the HEALPix projections are described in the angle of 90◦ on the sphere. Overall, this pixelisation performs same terms as the projections deﬁned in Calabretta Greisen adequately at low latitudes and does better than the H = 4 pix- (2002). elisations at mid to high latitudes. HEALPix projections will be denoted in FITS with algo- This rescaling of the H = 6 projection is reminiscent of rithm code HPX in the CTYPE ia keywords for the celestial axes. Tegmark’s (1996) icosahedral projection composed of 20 equi- As data storage has become much less of an issue in recent lateral triangles; the problem of indexing the subdivisions of its years we do not consider it necessary to create an analogue of triangular facets was solved in the implementation of the cor- the CUBEFACE keyword to cover HPX. However, if HEALPix responding pixelisation. In the present context the iso-latitude data is repackaged into the pseudo-quadcube layout shown property is still present but modiﬁed somewhat from the dia- in Fig. 3 the CUBEFACE storage mechanism is applicable for mond pixelisation of H = 4. However, if the pixel centres are H = 4 and will be treated properly by  (Calabretta, 1 moved up or down from the centroid by 12 of the height of 1995). the equilateral triangles to the point half-way between the base Since the HEALPix projections are constructed with the and apex then they fall onto a regular grid sampled more fre- origin of the native coordinate system at the reference point, quently in x than y. This provides some of the same beneﬁts as we set the H = 4 double-pixelisation. (φ0 , θ0 )HEALPix = (0, 0). (15) However, although the displacement is small, there is a pos- sibility that it will introduce statistical biases so the full con- The projection equations and their inverses, re-expressed in the sequences should be investigated for a particular application. form required by FITS, are now summarised formally. These potential biases may be minimised by making the pixel In the equatorial zone where | sin θ | ≤ 2/3: size suﬃciently small, and the fact that the bias between ad- x = φ, (16) jacent pairs of pixels is in opposite senses will tend to cancel 270◦ them over a region encompassing a suﬃcient number of pixels. = sin θ, (17) H It should also be remembered that although the pixel locations appear to be at the centre of the pixel boundary in the projec- in the polar zones, where | sin θ | 2/3: tion of the diamond, square, and triangular pixelisations this x = φc + (φ − φc ) σ, (18)
32. 32. HEALPix FITS SIMPLE = T /FITS FORMAT BITPIX = 16 /DUMMY PRIMARY HEADER NAXIS = 0 /NO DATA IS ASSOCIATED WITH THIS HEADER EXTEND = T /EXTENSIONS MAY (WILL!) BE PRESENT RESOLUTN= 9 /RESOLUTION INDEX PIXTYPE = HEALPIX /PIXEL ALGORIGTHM ORDERING= NESTED /ORDERING SCHEME NSIDE = 512 /RESOLUTION PARAMETER FIRSTPIX= 0 /FIRST PIXEL (0 BASED) LASTPIX = 3145727 /LAST PIXEL (0 BASED) COMMENT THIS FILE CONTAINS A WMAP INTENSITY (I) RES 9 SKYMAP FOR THE COMMENT FREQUENCY BAND INDICATED BY THE FREQ KEYWORD. COMMENT FIVE YEARS OF DATA WENT INTO THE MAP. DATE = 2008-01-02T18:32:00 /FILE CREATION DATE (YYYY-MM- DDTHH:MM:SS UT) TELESCOP= WMAP /WILKINSON MICROWAVE ANISOTROPY PROBE REFERENC= WMAP EXPLANATORY SUPPLEMENT: HTTP:// LAMBDA.GSFC.NASA.GOV/ / STOKES = I / FREQ = KA BAND /FREQUENCY BAND RELEASE = DR3 /WMAP RELEASE END
33. 33. HEALPix FITS XTENSION= BINTABLE /BINARY TABLE EXTENSION BITPIX = 8 /8-BIT BYTES NAXIS = 2 /2-DIMENSIONAL BINARY TABLE NAXIS1 = 8 /WIDTH OF TABLE IN BYTES NAXIS2 = 3145728 /NUMBER OF ROWS IN TABLE PCOUNT = 0 /SIZE OF SPECIAL DATA AREA GCOUNT = 1 /ONE DATA GROUP (REQUIRED KEYWORD) TFIELDS = 2 /NUMBER OF FIELDS IN EACH ROW TTYPE1 = TEMPERATURE /LABEL FOR FIELD 1 TFORM1 = E /DATA FORMAT OF FIELD: 4-BYTE REAL TUNIT1 = MK /PHYSICAL UNIT OF FIELD 1 TTYPE2 = N_OBS /LABEL FOR FIELD 2 TFORM2 = E /DATA FORMAT OF FIELD: 4-BYTE REAL TUNIT2 = COUNTS /PHYSICAL UNIT OF FIELD 2 EXTNAME = ARCHIVE MAP TABLE /NAME OF THIS BINARY TABLE EXTENSION PIXTYPE = HEALPIX /PIXEL ALGORIGTHM ORDERING= NESTED /ORDERING SCHEME NSIDE = 512 /RESOLUTION PARAMETER FIRSTPIX= 0 /FIRST PIXEL (0 BASED) LASTPIX = 3145727 /LAST PIXEL (0 BASED) END
34. 34. XTENSION= BINTABLE /BINARY TABLE EXTENSIONBITPIX = 8 /8-BIT BYTESNAXIS = 2 /2-DIMENSIONAL BINARY TABLENAXIS1 = 8 /WIDTH OF TABLE IN BYTESNAXIS2 = 3145728 /NUMBER OF ROWS IN TABLEPCOUNT = 0 /SIZE OF SPECIAL DATA AREAGCOUNT = 1 /ONE DATA GROUP (REQUIRED KEYWORD)TFIELDS = 2 /NUMBER OF FIELDS IN EACH ROWTTYPE1 = TEMPERATURE /LABEL FOR FIELD 1TFORM1 = E /DATA FORMAT OF FIELD: 4-BYTE REALTUNIT1 = MK /PHYSICAL UNIT OF FIELD 1TTYPE2 = N_OBS /LABEL FOR FIELD 2TFORM2 = E /DATA FORMAT OF FIELD: 4-BYTE REALTUNIT2 = COUNTS /PHYSICAL UNIT OF FIELD 2EXTNAME = ARCHIVE MAP TABLE /NAME OF THIS BINARY TABLEEXTENSIONPIXTYPE = HEALPIX /PIXEL ALGORIGTHMORDERING= NESTED /ORDERING SCHEMENSIDE = 512 /RESOLUTION PARAMETERFIRSTPIX= 0 /FIRST PIXEL (0 BASED)LASTPIX = 3145727 /LAST PIXEL (0 BASED)END
35. 35. HTM
36. 36. HTMBy Szalay, Gray, et al.Hierarchical, multiresolution search system withsupport for region operationsBasis for spatial support in MS SQL Server, SDSS
37. 37. Indexing the Sphere with the Hierarchical Triangular MeshHTM Alexander S. Szalay1, Jim Gray2, George Fekete1 Peter Z. Kunszt3, Peter Kukol2, and Ani Thakar1 1. Dept of Physics and Astronomy, The Johns Hopkins University, Baltimore 2. Microsoft Research Bay Area Research Center, San Francisco 3. CERN, Geneva Abstract: We describe a method to subdivide the surface of a sphere into spherical triangles of similar, but not identical, shapes and sizes. The Hierarchical Triangular Mesh (HTM) is a quad-tree that is particularly good at supporting searches at different resolutions, from arc seconds to hemispheres. The subdivision scheme is universal, providing the basis for addressing and for fast lookups. The HTM provides the basis for an efficient geospatial indexing scheme in relational databases where the data have an inherent location on either the celestial sphere or the Earth. The HTM index is superior to cartographical methods using coordinates with singularities at the poles. We also describe a way to specify surface regions that efficiently represent spherical query areas. This article presents the algorithms used to identify the HTM triangles covering such regions. 1. IntroductionBy Szalay, Gray, et al. Many science and commercial applications must catalog and search objects in three-dimensional space. In Earth and Space Science, the coordinate system is usually spherical, so the object’s position is given by its location on the surface of the unit sphere and its distance from the origin. Queries on catalogs of such objects often involve constraints on these coordinates, often in terms of complex regions that imply complicated spherical trigonometry.Hierarchical, multiresolution search system with There is a great interest in a universal, computer-friendly index on the sphere, especially in astronomy, where the ancient index of stellar constellations is still in common use, and in earth sciences, where people use maps having complicated spherical projections. The spatial index presented here transformssupport for region operations regions of the sphere to unique identifiers (IDs). These IDs can be used both as an identifier for an area and as an indexing strategy. The transformation uses only elementary spherical geometry to identify a certain area. It provides universality, which is essential for cross-referencing across different datasets. The comparisons are especially well-suited for computers because they replace transcendental functions with a few multiplications and additions.Basis for spatial support in MS SQL Server, SDSS The technique to subdivide the sphere into spherical triangles presented here is recursive. At each level of recursion, the triangle areas are roughly the same (within a factor of 2), which is a major advantage over the usual spherical coordinate system projections with singularities at the poles. Also, in areas with high data density, the recursion process can be applied to a higher level than in areas where data points are rare. This enables uneven data distributions to be organized into equal-sized bins. A similar scheme for subdividing the sphere was advocated by Barrett . The idea of the current implementation of the HTM was described in Kunszt et al . Short, Fekete et al [3,4,5] used an icosahedron-based mesh for Earth sciences applications. Quad trees and geometrical indexing are discussed in the books of Samet in detail [6,7]. Lee and Samet  use an identical triangulation, but a different numbering scheme. Goodchild [9,10] and Song et al  created a similar triangulation of the sphere, called the Discrete Global Grid, with precisely equal areas, using small circles for the hierarchical boundaries. Gray et al  described linking the HTM to a relational database system. 1
38. 38. ( 0, 1, 0 ) := v2 (2.1) ( −1, 0, 0 ) := v3 ( 0, −1, 0 ) := v4 HTM ( 0, 0, −1 ) := v5The first eight nodes of the HTM index are defined as these eight spherical triangles named by using Ssouth and N for north, and then numbering the faces clockwise: Figure 1 The Hierarchal Triangular Mesh (HTM) is a recursive decomposition of the sphere. The figures above show the sequence of subdivisions, starting from the octahedron, down to level 5 corresponding to 8192 spherical triangles. The triangles have been plotted as planar ones for simplicity.
39. 39. HTM triangles is 15 percent, and the ratio of the average arc length to the canonical size /2n is about 10 10 8 8 frequency 6 6 4 4 2 2 0 0 0.6 0.8 1 1.2 1.4 1.6 0.6 0.8 1 1.2 1.4 1.6 relative area relative arc length Figure 3 The distribution of relative spherical trixel areas and arc lengths around the mean at HTM depth 7. The scatter arises mainly from the difference in size of the center triangle from the corner ones at depth 2. Subsequent subdivisions smear out the ratios some, but the curvature plays a diminishing role at higher depths.efining the GeometryTM index must support intersections with arbitrary spherical regions. Given a region, we want toHTM trixels of a given depth that cover it. We call this set of trixels the region’s HTM cover. This presents the geometry primitives that define spherical areas. These primitives can easily be
40. 40. HTM vs. HEALPix Planck Splitting the sky - HTM HEALPix. Motivation HTM - searches in spherical space HEALPix - spherical images • Index the sphere • Numerical analysis - • support spherical trigono- • convolutions local/global metrics. kernel • Support data binning • Fourier analysis with • Hierarchical for storage spherical harmonics • power spectrum estimation • Topological analysis • extreme searches • neighbour searches • Minkowski functionals • To aid above should have • equal area pixels • iso latitude distribution • Hierarchical for storage William O’Mullane 20/7/00 2/20
41. 41. HTM vs. HEALPix Planck Splitting the sky - HTM HEALPix. Special Considerations Anomalies are inevitable with any tessellation. HEALPix HTM • Pixel shape varies - not a problem with • Pixel shape varies high enough sampling • Pixel area varies - makes many numeri- • need to take account of posi- cal calculations difﬁcult. tion on the sphere (3 rules) • no rings • number of pixels in rings vary • similar neighbour orientation problem • Most pixels have 8 neighbours - 8 have 7 neighbours • Nested X,Y orientation of base pixel neighbours varies These are mainly problems for developers not particularly for users Advanced libraries exist for both schemes although target audiences are different. William O’Mullane 20/7/00 12/20
42. 42. Q3C
43. 43. Q3CBy Kortosov Barkunov (Sternberg AstronomicalInstitute)Implementation of a faster, open-source, HTM-likepartitioning Not equal areaAlternative to pgSphere, not well documented yet
44. 44. Q3C SPHERE PARTITIONING
45. 45. Q3C CONESEARCH PIXELISATION
46. 46. Using partitioningschemesDisplay applicationsQuery applications Spatial joinsX-Match applications
47. 47. Partitioning Cone Search
48. 48. Partitioning Cone SearchSELECT * FROM TARGET_TABLE WHERE 2 * ASIN( SQRT(SIN((\$DEC_C-DEC)/2) * SIN((\$DEC_C-DEC)/2) + COS(\$DEC_C) * COS(DEC) * SIN((\$RA_C - RA)/2) * SIN((\$RA_C - RA)/2))) = \$SR_CSELECT * FROM RASS_PHOTONS WHERE X*\$X_C+Y*\$Y_C+Z*\$Z_C = COS(\$SR_C)
49. 49. Partitioning Cone SearchSELECT * FROM RASS_PHOTONS WHERE PIXEL_ID IN RESULTING_PIXEL_QUERY AND X*\$X_C+Y*\$Y_C+Z*\$Z_C = \$COS_SR
50. 50. Using Spatial Indexing inDatabasesJuan de Dios Santander Vela (IAA-CSIC)
51. 51. Things to show SE BA SYHTM and Transact SQL (MS SQL)PostgreSQL and pgSphereHands on on PostgreSQL
52. 52. HTM SQL API
53. 53. Main Page Classes Class List Class MembersHTM SQL API Spherical Htm.Sql Spherical.Htm Sql Spherical.Htm.Sql Class ReferenceEncapsulates the HTM procedures for use from SQL2005. More...List of all members.Static Public Member Functions static SqlString fHtmVersion () fHtmVersion() returns the version number of this htm library as a string static long fHtmXyz (double x, double y, double z) fHtmXyz(x,y,z) returns the 20-deep HtmID of the given cartesian point. There are no error cases. All vectors are nomalized and 0,0,0 maps to 1,0,0 static long fHtmEq (double ra, double dec) fHtmEq(ra,dec) returns the 20-deep HtmID of the given equatorial point. There are no error cases. all RA, folded to [0..360] and dec to [-90...90] static long fHtmLatLon (double lat, double lon) fHtm(lat,lon) returns the 20-deep HtmID of the given location. There are no error cases. all RA, folded to [0..360] and dec to [0...90] static SqlDouble fDistanceEq (double ra1, double dec1, double ra2, double dec2) double fDistanceEq(ra1,dec1,ra2,dec2) returns distance in ArcMinutes between two points. static SqlDouble fDistanceLatLon (double lat1, double lon1, double lat2, double lon2) double fDistanceLatLon(lat1,lon1,lat2,lon2) returns distance in ArcMinutes between two points. static SqlDouble fDistanceXyz (double x1, double y1, double z1, double x2, double y2, double z2) double fDistanceXyz(x1,y1,z1,x2,y2,z2) returns distance in ArcMinutes between two points. static SqlString fHtmToString (SqlInt64 HtmID) fHtmToString(HtmID) returns varchar(max)a string describing the HtmID static IEnumerable fHtmLatLonToXyz (SqlDouble lat, SqlDouble lon) XyzTable(x,y,z) fHtmLatLonToXyz(lat, lon) converts an lat,lon point to a table with one row containing the cartesian point (x,y,z). static IEnumerable fHtmEqToXyz (SqlDouble ra, SqlDouble dec) XyzTable(x,y,z) fHtmEqToXyz(ra, dec) converts an equitorial point to a table with one row containing the cartesian point (x,y,z). static IEnumerable fHtmXyzToLatLon (SqlDouble x, SqlDouble y, SqlDouble z) LatLonTable(lat,lon) fHtmXyzToLatLon(x,y,z) converts the cartesian point (x,y,z) to a table with one row containing the equivalent lat,lon point. static IEnumerable fHtmXyzToEq (SqlDouble x, SqlDouble y, SqlDouble z) EqTable(ra, dec) fHtmXyzToEq(x,y,z) converts the cartesian point (x,y,z) to a table with one row containing the equivalent equitorial (ra,dec) point. static IEnumerable fHtmToCenterPoint (SqlInt64 HtmID) fHtmToCenterPoint(HtmID) converts an HTM triangle ID to an (x,y,z) vector of the HTM triangle centerpoint. and returns that vector as the only row of a table. static IEnumerable fHtmToCornerPoints (SqlInt64 HtmID) fHtmToCornerPoints(HtmID) converts an HTM triangle ID to an table of three (x,y,z) vectors of the HTM triangle corners. static IEnumerable fHtmCoverCircleLatLon (SqlDouble lat, SqlDouble lon, SqlDouble radiusArcMin) fHtmCoverCircleLatLon(ra,dec,radiusArcMin) returns a trixel table (a list) covering
54. 54. fHtmRegionToNormalFormString(coverspec) converts a region definiton to normal formParameters: coverspec a string satisfying the region syntax regionSpec := REGION {areaSpec}* | areaSpecHTM SQL API circleSpec := CIRCLE J2000 ra dec radArcMin | CIRCLE LATLON lat lon radArcMin | CIRCLE [CARTESIAN ] x y z radArcMin rectSpec := RECT LATLON {lat lon}2 | RECT J2000 {ra dec }2 | RECT [CARTESIAN ] {x y z }2 polySpec := POLY LATLON {lat lon}3+ | POLY J2000 {ra dec }3+ | POLY [CARTESIAN ] {x y z }3+ hullSpec := CHULL LATLON {lat lon}3+ | CHULL J2000 {ra dec }3+ | CHULL [CARTESIAN ] {x y z }3+ convexSpec := CONVEX LATLON {lat lon}* | CONVEX J2000 {ra dec }* | CONVEX [CARTESIAN ]{x y z }* areaSpec := circleSpec | rectSpec | polySpec | hullSpec | convexSpecReturns: region String SqlString region in normal form ( a union of convexes) This looks something like REGION CONVEX 1 0 0 0 0 1 0 0 CONVEX 0 0 1 0 create function fHtmRegionToNormalFormString(@region nvarchar(max)) returns table (HtmID_start bigint, HtmID_end bigint) as external name HTM.Sql.fHtmToNormalForm select dbo.fHtmRegionToNormalFormString(CIRCLE J2000 195 0 1)--------------------------------------------------------------static SqlString Spherical.Htm.Sql.fHtmRegionError ( SqlString coverspec ) [static]fHtmRegionError(coverspec) returns diagnostic message appropriate for a bad region stringParameters: coverspec a string satisfying the region syntaxReturns: diagnostic message: varchar(max) OK if ok else message and a syntax string. create function fHtmRegionError(@region nvarchar(max)) returns varchar(max) as external name HTM.Sql.fHtmRegionError print dbo.fHtmRegionError(CIRCLE LATLON 195 )See also: fHtmCover()the main routine
55. 55. HTM SQL API Main Page Classes Class List Class Members Spherical.Htm.Sql Member List This is the complete list of members for Spherical.Htm.Sql including all inherited members. Spherical.Htm.Sql, fDistanceEq fDistanceEq(double ra1, double dec1, double ra2, double dec2) Spherical.Htm.Sql [static] fDistanceLatLon fDistanceLatLon(double lat1, double lon1, double lat2, double lon2) Spherical.Htm.Sql [static] fDistanceXyz fDistanceXyz(double x1, double y1, double z1, double x2, double y2, double Spherical.Htm.Sql [static] z2) fHtmCoverCircleEq fHtmCoverCircleEq(SqlDouble ra, SqlDouble dec, SqlDouble radiusArcMin) Spherical.Htm.Sql [static] fHtmCoverCircleLatLon fHtmCoverCircleLatLon(SqlDouble lat, SqlDouble lon, SqlDouble Spherical.Htm.Sql [static] radiusArcMin) fHtmCoverCircleXyz fHtmCoverCircleXyz(SqlDouble x, SqlDouble y, SqlDouble z, SqlDouble Spherical.Htm.Sql [static] radiusArcMin) fHtmCoverList fHtmCoverList(SqlString coverspec) Spherical.Htm.Sql [static] fHtmCoverRegion fHtmCoverRegion(SqlString coverspec) Spherical.Htm.Sql [static] fHtmCoverRegionSelect fHtmCoverRegionSelect(SqlString coverspec, SqlString kind) Spherical.Htm.Sql [static] fHtmCoverTypedRegion fHtmCoverTypedRegion(SqlString coverspec) Spherical.Htm.Sql [static] fHtmEq fHtmEq(double ra, double dec) Spherical.Htm.Sql [static] fHtmEqToXyz fHtmEqToXyz(SqlDouble ra, SqlDouble dec) Spherical.Htm.Sql [static] fHtmLatLon fHtmLatLon(double lat, double lon) Spherical.Htm.Sql [static] fHtmLatLonToXyz fHtmLatLonToXyz(SqlDouble lat, SqlDouble lon) Spherical.Htm.Sql [static] fHtmRegionError fHtmRegionError(SqlString coverspec) Spherical.Htm.Sql [static] fHtmRegionToNormalFormString fHtmRegionToNormalFormString(SqlString coverspec) Spherical.Htm.Sql [static] fHtmRegionToTable fHtmRegionToTable(SqlString coverspec) Spherical.Htm.Sql [static] fHtmToCenterPoint fHtmToCenterPoint(SqlInt64 HtmID) Spherical.Htm.Sql [static] fHtmToCornerPoints fHtmToCornerPoints(SqlInt64 HtmID) Spherical.Htm.Sql [static] fHtmToString fHtmToString(SqlInt64 HtmID) Spherical.Htm.Sql [static] fHtmVersion fHtmVersion() Spherical.Htm.Sql [static] fHtmXyz fHtmXyz(double x, double y, double z) Spherical.Htm.Sql [static] fHtmXyzToEq fHtmXyzToEq(SqlDouble x, SqlDouble y, SqlDouble z) Spherical.Htm.Sql [static] fHtmXyzToLatLon fHtmXyzToLatLon(SqlDouble x, SqlDouble y, SqlDouble z) Spherical.Htm.Sql [static] NOW ALL IN C# FOR MS SQL SERVER, BUT…
56. 56. HTM SQL API
57. 57. HTM SQL API
58. 58. HTM Regions HTMSqlRegions.html 20/01/10 8:56 coverSpec = polySpec | rectSpec | circleSpec | regionSpec | hullSpec circleSpec = CIRCLE J2000 [Ln] ra dec radArcMin | CIRCLE CARTESIAN [ln] x y z radArcMin coverSpec = polySpec | rectSpec | circleSpec | regionSpec | hullSpec circleSpec = CIRCLE J2000 [Li] ra dec rad | CIRCLE CARTESIAN [Li] x y z rad rectSpec = RECT J2000 {ra dec}4 | RECT CARTESIAN {x y z }4 polySpec = POLY [J2000] {ra dec} | POLY CARTESIAN {x y z}3+ hullSpec = CHULL J2000 {ra dec}3+ (Cartesian not implemented currently) convexSpec = CONVEX { x y z D}+ regionSpec = REGION {convexSpec}+
59. 59. pgSphere objects
60. 60. pgSphere objects SPOINT FORMAT STRINGS spoint (0.1,-0.2) spoint ( 10.1d, -90d) spoint ( 10d 12m 11.3s, -13d 14m) spoint ( 23h 44m 10s, -1.4321 ) CONSTRUCTOR FUNCTIONS SPOINT(0.1,-0.2) SPOINT(RADIANS(10.1D),RADIANS(-90)) SPOINT(RADIANS(RA_FIELD),RADIANS(DECL_FIELD))