Good afternoon and welcome to my talk with the bold title „ Scaling up globally: 30 years of FOSS4G development“ I am very happy to be here, and would like to thank the organizers for the kind invitation. In my presentation I'll briefly review 3 decades of Open Source GIS development, compressed to 30 minutes! Hence, omissions are unavoidable. BTW: The term FOSS4G was coined only in 2004 but I use it in the title as an hommage to the community and its ideals. So, let's now look back 30 years – back to the 80 th ...
Well... what's this? Do you recognise it?
Hence we start in 1980: A student named Jim Westervelt was completing his Master thesis. He wrote the LAGRID software which became the core of the later GRASS GIS software. In the early 1980s the U.S. Army Corps of Engineers' Construction Engineering Research Laboratory (USA/CERL) in Champaign, Illinois, began to explore the possibilities of using GIS to conduct environmental research, assessments, monitoring and management of lands under the stewardship of the U.S. Department of Defense. Bill Goran of CERL conducted a survey and discovered that no existing GIS satisfied their needs. Hence CERL hired several programmers, and began to develop a hybrid raster-vector GIS for the VAX UNIX minicomputers (photo) . Lynn Van Warren was the founding software architect of GRASS. Very important: PROJ (today PROJ.4) – the Cartographic Projections Library essential for many projects, was started in 1983 by Gerald Evenden
CERL then upgraded their machines to the latest technology, see what you could get for 33,000 USD.... e.g. including a 80MB disks for 6500 USD. Around 1984 new analytical functionality was available, some command names at least the GRASS users in this room will be familiar with :-) Please note that it took still another almost 10 years to have the World Wide Web being invented...
In the corner photo you see what data exchange meant in those days. Landsat 5 was launched in 1984, GRASS 1.0 was released a year later. Of huge importance for the Free Software development was the publication of the GNU manifesto by Richard Stallman (the “four freedoms” were defined). In 1987 a GRASS video was produced with William Shatner, known as Captain Kirk of Star Trek, being the speaker. And 5.25 inch floppies were becoming fashion in these days for tiny data sets!
And then finally an initial internet release of Open Source GIS software! Still there was almost nobody online in those days, at least not able to download 100 MB of source code... The big spread of internet happened due to the acceptance of TCP/IP as protocol. Information exchange happened via FTP. Today all geeks are sharing code via SVN or git or other online code sharing repositories. But back in the 80th there was no online system like this. Either manual management or at most RCS which is file based.
So, the first decade of Open Source GIS concluded with the online publication of software (yet with a tiny user group being also online). Consider to think back when you went online the first time! Anyway, let us hop into the next decade: the 90th! A lot happened here, new hardware, new software, and way more data! And especially the advent of the internet society was a social revolution.
In 1990 GRASS 4.0 came out, a key release which was the foundation for the GRASS' architecture. On the “social” side, two mailing lists were started, the grass-user and the grass-dev lists. Those are probably the longest, active mailing lists of the internet with more than 2 decades of archive! Still no WWW, but information exchange happened FTP and Gopher. The photo shows core developers: - Michael Shapiro - Jim Westervelt - Bill Goran (the coordinator)
With the availability of new graphics CPUs a completely new way of rendering became possible. OpenGL was designed with hardware acceleration. Like this, fly-thoughs as seen in the left figure as well as 3D and even 4D animations came to life (here, water contamination modeled as voxels which are changing in time). I remember from working as a student at the Hannover CeBIT computer fair, to enjoy the demos of the SGI machines while carrying around newspapers across the fair area.
At the level of organizations many changes occurred: the GIS world became more professional and organized. In 1992 the GRASS Interagency Coordinating Commitee was founded, it was turned into the Open GRASS Foundation which then became the OpenGIS Consortium. The timeline shows the evolution. While Tim Berners-Lee constituted the World Wide Web Consortium, the today's OGC was established.
The rationale behind the establishing of the OGC you find described in an article written by Kenn Gardels. It got published in the “GRASSCLIPPINGS” issue of Fall 1993. Here the fact that interoperability is a core issue is pointed out. BTW: If you are interested, the scan is online, all links in my presentation which I'll upload later, are pointing to the respective documents.
This map shows the internet accessibility in the year 1998 – people being online in percent. As you see, in those days internet was yet restricted to a few countries in terms of accessibility. And, obviously: without internet no distributed source code development nor easy geodata exchange.
In the year 1993 the first Web application came up, the Xerox PARC map viewer [refer to the previous talk of Maria Brovelli] In 1994 the 1.0 version of Linux was released and shortly after GRASS GIS ported to it. This was still a tough job since the compiler tools where not as mature as nowadays. The year 1995 was a key year for the WebGIS community with the start of the Mapserver project. And... wow, a first spam email reached the ML
What happened in the WebGIS sector: Probably less known is the this first “Web Processing“ style service which was published back in 1994. It was an interactive GIS controllable through the WWW, developed by Susan Huse based on her PhD thesis. This system was able to ...
… perform true analysis in a user controllable way using CGI scripts. Of course it is not comparable with anything you can do today but remember the internet accessibility map of 1998 which I have shown earlier. We are still in the “prototype” years and draft WPS specifications were published only almost a decade later.
With the advent of the collaborative internet based development tools , a series of new projects like GeoTools, deegree, GDAL/OGR were started. In 1998 I started the European GRASS GIS server, completely non-authorized of course (except for the ILN landscape architecture institute at the university)... it turned out to be a good idea to take control. BTW: Anyone here remembering the year 2000 bug? It stimulated us to move the GRASS code from manual code management via email to something more sophisticated called CVS. This was a server based code repository and checkins became independent of a single person who had to read emails and act upon.
Well, we happily survived the Year 2000 bug and reached the next decade... Now Communities growing together! This drawing has been done by the older daugther of Venka (Venkatesh Raghavan in Osaka) for the initial 2004 FOSS4G conference in Bangkok which I'll mention shortly
In just a few years a lot of new software projects were started, I don't have the time to illustrate them in details. You see that all adopted the collaborative tools for development. From the demand to get coordination among these projects the idea arose to create an umbrella foundation , OSGeo, as illustrated by Jeff McKenna earlier today.
For the term “FOSS4G” we have to thank Venka once more. He proposed the conference title in 2004 for the conference at Chulalongkorn University. Also Jim Westervelt came to give us an authentic lecture about the early days at this meeting.
The next slides will be much more familiar for you... code sprint photos Here the mapbender team in 2007 and the QGIS hackfest in Pisa in 2010.
For those not familiar with it: - it is a gathering of likeminded people - no need to be a developer - newcomers are there up to core developers Instead of writing 10 emails to discuss an issue, we just discuss it directly, rarely long with some beer And: The outcome of such a week is often impressive. Next chance for you: on Thursday this week!
After these years of community based software development, the emerge of Wikipedia also crowd mapping came up. No need to explain OpenStreetMap here but for sure the Haiti earthquake example is to be mentioned. It is illustrating an outstanding effort of people around the globe gathering virtually together and helping in an emergency situation. The map shows the geodata situation in January 2010 when a magnitude 7 earthquake occurred – almost no geodata available which were urgently needed.
This is the situation a few days later. Based on high resolution satellite data which were made available by the big data providers, the OSM community was able to generate in an extremely short time a detailed coverage of digital data. A similar effort was done after the Great East Japan Earthquake 2011, you will remember it.
The increasing demand for geospatial data in the Web browser and increasing interactivity naturally requires the adoption of new emerging technologies. An interesting new method of fundraining, instead of direct sponsoring by a few is after crowd knowledge (Wikipedia) and crowd mapping (OSM) now crowd funding where till a deadline a certain funding goal should be achieved. Example: OpenLayers 3 funding, aiming at more than 300,000 USD – and they are almost there!
There are also new possibilities for plotting maps: from 2D to 3D using rapid prototyping technologies which have been directly connected to Open Source GIS. Here an example of the polar ice cap of Mars (so, no need to restrict yourself to mother Earth): The processing chain was: from Radar remote sensing data to voxels to 3D plots
Since we like to think big, why to getting our tools on Supercomputers running... Indeed, we are already there: PROJ4, GDAL and GRASS GIS are available even on TOP500 systems. The harder part was to get it compiled on Non-Linux Supercomputers :-) For massive data processing currently job managers are supported. Work is underway for GPU based clusters which require a major source code restructuring.
Since not everybody has a supercomputer reachable, a lot of efforts have been done on processing massive data on even consumer hardware. Here an example from a power user who managed to process 8.5 billion lidar points on his home computer. With this touching base I would like to conclude my quick ride through 30 years of Open Source GIS development and conclude with...
Happy birthday, GRASS GIS And Thanks to all FOSS4G contributors all over the world. Thank you for listening
Transcript of "Scaling up globally: 30 years of FOSS4G development. Keynote at FOSS4G-CEE 2013, Romania"