This document discusses the history and future of database technology. It notes that in 1998, almost all major database systems were proprietary, but PostgreSQL provided an open source alternative. Over time, other open source databases like MySQL emerged and grew in popularity. The document predicts that in the future, more databases will be designed for cloud hosting and very large scale deployments, taking advantage of distributed and non-relational architectures. Major technology companies have also adopted open source databases like PostgreSQL and MySQL.

1. Free Software
and the
Future of Database Technology
Josh Berkus
PostgreSQL Core Team
November 14, 2008

2. Who is Josh?
●
PostgreSQL Core Team
– a leader of 30,000 participant international
community
●
Free software advocate
– Since 2000
– OpenOffice, SuSE Linux, PostgreSQL, SPI, OSI ...
●
Database Engineer & Application Developer
– PostgreSQL, MySQL, Drizzle, MS SQL, Oracle

3. Database Past

4. 1998
1. Oracle
2. Informix
3. Sybase
4. Microsoft SQL Server
5. IBM DB2

5. 1998
1. Oracle
2. Informix
3. Sybase
4. Microsoft SQL Server
5. IBM DB2
100% Proprietary Databases

6. 1998
1. Oracle
2. Informix
3. Sybase
4. Microsoft SQL Server
5. IBM DB2
100% Proprietary Databases
except ...

7. 1998
1. Oracle
2. Informix Illustra
3. Sybase
4. Microsoft SQL Server
5. IBM DB2
100% Proprietary Databases
except ...

8. 1998
1. Oracle
2. Informix Illustra POSTGRES
3. Sybase
4. Microsoft SQL Server
5. IBM DB2
100% Proprietary Databases
except ...

9. 1998
1. Oracle ?
2. Informix Illustra POSTGRES
3. Sybase
4. Microsoft SQL Server
5. IBM DB2
100% Proprietary Databases
except ...

10. PostgreSQL: History
• 1986: POSTGRES at the University of
California, Berkeley
> Michael Stonebraker project
> Successor to INGRES
• 1994: first commecialized
> as Illustra (later merged into Informix)
• 1995: open­sourced
> Ported to SQL
> PostgreSQL Global Development Group
formed

11. MySQL: History
●
MySQL Server development started in 1994,
marketed by TCX DataKonsult AB
●
MySQL AB founded in 1995 by Michael “Monty”
Widenius, David Axmark and Allan Larsson
●
Server development based on requirements for
practical production use: few features, but fast
and stable
●
Frequent releases with small changes
●
Easy to install and use (15­minute rule)

12. 2004
1. MySQL
2. Microsoft SQL Server
3. Oracle
4. IBM DB2
5. PostgreSQL

13. 2004
1. MySQL
2. Microsoft SQL Server
3. Oracle
4. IBM DB2
5. PostgreSQL
6. Sybase

14. Database Present

15. 2008
1. MySQL
2. Microsoft SQL Server
3. Oracle
4. IBM DB2
5. PostgreSQL
6. Sybase

16. Adoption Momentum
Source: Forrester 2006

17.
●
Europe's largest online hotel booking site
– 20 million visitors a month
– 30,000 listed hotels
– 100% growth 2005 ­> 2006
●
Built site using MySQL
– Dozens of servers
– Used Linux­based DRBD replication for high
availability

18.
●
Runs .ORG, .INFO domains
– 3.8 million domain names
– 100% transactional workload
●
Built using PostgreSQL
– custom replication solution for dozens of servers
– free software, low cost let them win the .ORG
hosting bid

19. Database Future

20. 2009
1. SQLite
2. MySQL
3. Microsoft SQL Server
4. Oracle
5. PostgreSQL
6. IBM DB2

21. 2009
1. SQLite What?
2. MySQL
3. Microsoft SQL Server
4. Oracle
5. PostgreSQL
6. IBM DB2

22.
●
Embedded SQL database
– Tiny memory footprint
– Full SQL parser
– Transactions, foreign keys
●
Designed by Dr. Richard Hipp
– In United Kingdom
– Released under a public domain license
– 100+ million installations

25. ZopeDB
memcached
Skytools
MySQL NDB
BerkeleyDB XML
StreamBase
Kickfire
Yahoo! Data Warehouse

26. Scalable Embedded
Specialty
ZopeDB
memcached
Skytools
MySQL NDB
BerkeleyDB XML
Business Intelligence
Streaming
StreamBase
Kickfire
Yahoo! Data Warehouse

27. 1998: Database server

28. 2008: Database servers

29. 2008: Database servers
1. Large RAM
2. Large Disk
3. Multi­core CPU
4. Solid­state drives
5. Virtualization

30.
●
Geographic Information Database
– stores spatial and positional data
●
points, lines, polygons
– based on PostgreSQL
– power your own Google Maps!
●
#1 open source solution
– database for all OpenGeo tools
– works with all GIS applications: MapServer, ESRI,
Ka­Map, deCarta

31. European Fisheries
MapServer Interface
Positions
Position
Statuses
Status
Map
Data

32. European Fisheries

33. Repertorio Nazionale dei Dati
Territoriali
●
Public service of Italy
– operated by CNIPA
– supply national map data for all Italian users
– similar to USGS in United States
– critical for developing Italian geo service industry
●
All free software stack
– PostGIS
– Mapserver

34. 2010: What is a database
server?

35. 2010: What is a database
server?

36. "Cloud" Hosting
Large­scale, managed
anonymous hosting

37. "Cloud" Hosting
●
Large numbers of servers
– rapid deployment of cloned virtual machines
– rapid expansion of capacity
●
Needs:
– no per­installation licensing fees
– low­administration databases
– Free Software

38.
●
"A Microkernel DBMS for High Performance
Scale­Out Applications"
– designed for cloud hosting
– small footprint, simple database, limited features
– easy to administer in large numbers
●
A fork of MySQL
– developed by lead MySQL developers
– not released yet

39.
PHP / Perl / Python / Ruby
memcached

40.
●
A "web scale" clustered database
– based on concept of Google's BigTable
– designed to scale to 1000's of servers
– like Memcached, but with persistence
– uses Hadoop distributed filesystem
●
Non­relational
– only stores key­value pairs
– all operations are single­key

43.
●
#1 Voice­over­IP service
– 350 million users
– 5 million SkypeOut calls per day
– 6% of worldwide international calling
●
SkyTools/PostgreSQL platform
– PostgreSQL allowed them to build custom
clustering framework
– 200 servers

45. Free Software
ZopeDB
memcached
Skytools
MySQL NDB
BerkeleyDB XML
Built with Free Software
StreamBase
Kickfire
Yahoo! Data Warehouse

46. New 100% Non­Free Databases

47. "Building a new software is like
scaling a mountian. Using open
source to build on means that I get
to start 80% of the way to the top."
­Julian Hyde, Chief Architect, SQLStream

48. Where are we going?
●
More diversity
●
More free software databases
– some free databases
– some built on free software databases
●
More innovation
●
More free software

49. Questions?
●
Josh Berkus
– josh@postgresql.org
●
PostgreSQL project
– www.postgresql.org
●
Links
– EU Fisheries: http://fish.jrc.cec.eu.int/
Booking.com: http://www.mysql.com/why­mysql/scaleout/booking.html
Skytools: https://developer.skype.com/SkypeGarage/DbProjects/SkyTools
PostGIS: http://postgis.refractions.net/ Drizzle: http://drizzleproject.org/
Afilias: www.afilias.info Hypertable: http://hypertable.org/
This talk is copyright 2008 Josh Berkus, and is licensed under the creative commons attribution license
All trademarks belong to their respective owners and are used according to principles of fair use here.
Other uses of these trademarks must be authorized by the trademark owner.
Josh Berkus speaks only for himself, and not for any of the companies or open source projects