This document discusses the limitations of the shapefile format and promotes the use of the OGC GeoPackage format as an alternative. It outlines 11 issues with shapefiles, including their multi-file structure, limited attribute name lengths, maximum file size and number of attributes. It then introduces GeoPackages as a single-file SQLite-based format that supports both vector and raster data, has a defined schema and is an open OGC standard implemented in many software programs like QGIS and ArcGIS. The document argues that GeoPackages provide a more sustainable and full-featured replacement for shapefiles.

1. Shapefile: a curse
of today’s Geoinformatics
Jáchym Čepický
OpenGeoLabs s.r.o.
GIVS 2018

2. OGC GeoPackage – new
format for data exchange
Jáchym Čepický
OpenGeoLabs s.r.o.
FOSS4G-Europe 2018

3. (ESRI) Shapefile

4. https://www.esri.com/library/whitepapers/pdfs/shapefile.pdf

5. http://octogeo.cz

6. Raster → Vektor

7. Layer “grass” of global landcover, vector,
cca 3 000 000 vector features

8. 2 days of calculation ….
ERROR 1: Failure writing DBF record 31212233.
ERROR 1: Error in psSHP->sHooks.Fwrite() while writing object
to .shp file.
ERROR 1: Terminating translation prematurely after failed

10. http://switchfromshapefile.org

11. https://github.com/OpenGeoLabs/shapefilemustdie/

12. Shapefile - Positives

13. Shapefile - positives
Well supported in all software packages.

14. Shapefile - positives
Even proprietary, the spec is open.

15. Shapefile - positives
For many use cases, it is good enough.

16. Shapefile - positives
Inner index (*.shx) can speed up reading.

17. Shapefile - positives
It’s relatively efficient.

18. The dark side of Shapefile

19. 1. Multiple-file based format

20. Multi-file format
● shp
● shx
● dbf

21. Multi-file format
● .shp
● .shx
● .dbf
● .prj
● .sbn a .sbx
● .fbn and .fbx
● .ain a .aih
● .ixs
● .mxs
● .atx
● .shp.xml
● .cpg

24. 2. Maximal length of attribute name is 10
characters

25. 2. Maximal length of attribute name is 10 characters
RUIAN - register of land identifications and
addresses,
LandRegistry:
● Kod [Code]
● Nazev [Name]
● Nespravny [FalseName]
● ExistujeDigitalniMapa [DigitalMapExists]
● ObecKod [CityCode]
● PlatiOd [ValidSince]
● PlatiDo [ValidTill]
● IdTransakce [TransactionId]
● GlobalniIdNavrhuZmeny
[GlobalChangeRequest]
● RizeniId [ProcedureId]
● MluvnickeCharakteristikyPad2
[GramaticalCharacteristicsPad2]
● MluvnickeCharakteristikyPad3
[GramaticalCharacteristicsPad3]
● MluvnickeCharakteristikyPad4
[GramaticalCharacteristicsPad4]
● MluvnickeCharakteristikyPad6
[GramaticalCharacteristicsPad6]
● MluvnickeCharakteristikyPad7
[GramaticalCharacteristicsPad7]
● DatumVzniku [Origin]

26. 2. Maximálně 10 znaků na jméno atributu
RUIAN - register of land identifications and
addresses,
LandRegistry:
● Kod
● Nazev
● Nespravny
● ExistujeDigitalniMapa → 21
● ObecKod
● PlatiOd
● PlatiDo
● IdTransakce → 11
● GlobalniIdNavrhuZmeny → 21
● RizeniId
● MluvnickeCharakteristikyPad2 → 28
● MluvnickeCharakteristikyPad3 → 28
● MluvnickeCharakteristikyPad4 → 28
● MluvnickeCharakteristikyPad6 → 28
● MluvnickeCharakteristikyPad7 → 28
● DatumVzniku → 11

27. 2. Maximálně 10 znaků na jméno atributu
Warning 6: Normalized/laundered field name: 'ExistujeDigitalniMapa' to 'ExistujeDi'
Warning 6: Normalized/laundered field name: 'IdTransakce' to 'IdTransakc'
Warning 6: Normalized/laundered field name: 'GlobalniIdNavrhuZmeny' to 'GlobalniId'
Warning 6: Normalized/laundered field name: 'MluvnickeCharakteristikyPad2' to 'MluvnickeC'
Warning 6: Normalized/laundered field name: 'MluvnickeCharakteristikyPad3' to 'Mluvnick_1'
Warning 6: Normalized/laundered field name: 'MluvnickeCharakteristikyPad4' to 'Mluvnick_2'
Warning 6: Normalized/laundered field name: 'MluvnickeCharakteristikyPad6' to 'Mluvnick_3'
Warning 6: Normalized/laundered field name: 'MluvnickeCharakteristikyPad7' to 'Mluvnick_4'
Warning 6: Normalized/laundered field name: 'DatumVzniku' to 'DatumVznik'

28. 3. At most 255 attributes in total in table.

29. 4. Flat data structure

30. 5. Bad support for data types

31. Bad support for data types
+ Float
+ Integer
+ Date
+ Character string - 255 characters max.
- big integers
- boolean
- text
- blob
- ...

32. 6. Unknown characterset

33. Unknown characterset

34. 7. Maximal file size is 2GB.

35. 7. Maximal file size is 2GB.

36. 8. Only one geometry type

37. 9. Limitted support for 3D

38. 10. Unconsitent projection definition

39. 11 sins of Shapfile - http://switchfromshapefile.org

42. Shapefile replacement?

43. Shapefile replacement
● Complicated data structures
● Reasonable big
● Reasonable fast
● Maitained
● Open

44. Shapefile replacement
● OGC GeoPackage
● GeoJSON
● OGC GML
● SpatiaLite
● CSV
● OGC KML

45. SpatiaLite
● http://spatialite.org
● Builds on SQLite
○ File SQL database (almost complete SQL-92)
● OGC Simple Features
● No raster data support

46. OGC GML
● http://opegeospatial.org/standards/gml
● XML
● OGC
● INSPIRE

47. GeoJSON
● http://geojson.org
● JSON

48. OGC GeoPackage
● http://geopackage.org
● OGC Standard
● Postavený nad SQLite
● Podporuje i rastry
● Oficiální rozšíření

49. SQLite
● http://sqlite.org
● Souborová databáze
● SQL 92
● Dynamické typování, datový typ je vázaný na buňku, ne na celý sloupec
● Špatná podpora pro paralelní zápis
● fulltext
● Chrome, Opera, Safari, Firefox, Skype, Windows 10, ...

50. GeoPackage vs. Shapefile
http://switchfromshapefile.org/compare.html

51. GeoPackage vs. Shapefile - velikost dat

57. GeoPackage vs. Shapefile - conclusion
● GeoPackage seems slightly more efficient
● It’s relatively slower for simple queries, but it’s not fatal.
● It seems to be significantly faster for complex spatial and database queries.

58. Implementations of OGC GeoPackage
● QGIS
● GDAL
● ArcGIS
● FME
● Hexagon
● …
● http://www.geopackage.org/implementations.html

59. QGIS 3

61. Map Styling Information
https://github.com/pka/qgpkg/blob/master/qgis_geopackage_extension.md
Official extension

62. GeoPackage - finalisation
● http://geopackage.org
● OGC Standard
● Nad SQLite
● File-based
● Raster support
● Extensions
● Already spread around - even you might not notice

65. Jáchym Čepický
jachym.cepicky@opengeolabs.cz
http://opengeolabs.cz
http://switchfromshapefile.org