Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Google Earth Enterprise as an Open Source Project

369 views

Published on

For almost 10 years Google Earth Enterprise allowed developers to build and host their own private maps and 3D globes. GEE Fusion, Server, Client, and Portable products gave users the ability to process, present, and analyze geospatial content in a very intuitive way. After two years in support and sustainment mode in route to deprecation, the need and interest in GEE was overwhelming enough to make GEE open source the ideal solution.

Google released the Google Earth suite under the Apache 2.0 license which frees up the open source community to modify to their heart’s content! With a very large user community still utilizing GEE within critical IT infrastructure, continued support and future enhancements are a boon that will give the community the freedom to operate well into the future.

Emerging map APIs that provide 2D and 3D functionality will open up a new world of possibilities for GEE serving up rich 3D experiences. Although Google will not provide an official SaaS solution around GEE open-source, the possibility of setting one up is very feasible.

A good proxy for the longevity and stability of an open-source project is to measure the strength of the open-source community around it. The good news with GEE is that it’s already off to a great start. The same team that worked with Google to transition GEE to open-source, Thermopylae Sciences & Technology’s (TST), is sustaining the project and growing a vibrant community around it.

Published in: Software
  • Be the first to comment

Google Earth Enterprise as an Open Source Project

  1. 1. •  Good morning and welcome to Foss4G. I’m Andres Terrazas, CTO of Thermopylae Sciences and Technology. •  I’m really excited to be here to share the news of GEE going open source and specifically our journey helping Google bring it to the open source community •  Thermopylae’s experience with GEE dates back almost to 10 years when we first started to build uses cases around GEE for our government customers
  2. 2. Let’s start from the top. What exactly is Google Earth Enterprise? We’ve I’ll heard of Google Earth, but what do I get with this extra “E”? Essentially, GEE is a collection of tools that help organizations run their own earth globe behind the firewall and fuse their own imagery assets. GEE is made up of three main components: •  First, you get Google Earth Enterprise Fusion – Fusion combines all of an organization’s geospatial data into a flyable globe that is made available to users via the Google Earth Enterprise Server. •  Fusion can integrate raster data, including imagery and terrain, vector data, KML, 3D models, and even data stored in traditional models •  To serve the globe you’ll need GEE Server – Server hosts the data and delivers it to the end user •  Finally, GEE Portable - Portable enables users to access portions of their custom globes in the field when no network access is available.
  3. 3. Well that’s great but how does it all work together? •  Step one is to obtain your data (imagery, vector, or terrain data ) from various sources and place it on storage accessible to the GEE Fusion Server •  You select the resources you want to build into your globe by bringing them into your project •  You kick off your build go grab some coffee, builds can take a while, come back and bam you have a Google Earth Map or Globe database •  At this point you are ready to publish that globe or map to the GEE Server for hosting •  You can either access the globe or map directly via a connection from a GEE client, 3D plugin, or thanks to our friends at AGI the - Cesium WebGL client, or with any client that supports WMS •  Or you optionally take a cut of the globe you care about by defining a polygon of interest and download an offline cut •  Once you have the cut in hand you can open the file in Google Earth Enterprise portable and view it offline. This is an excellent use case for
  4. 4. GEE has had some fairly interesting use cases over the years. I’ll highlight some of the use cases from the opengee.org site. There are some additional videos you can watch if you want to learn more. Islington •  The London Borough of Islington is London’s second largest borough. •  Islington used GEE as an informational portal to better help communicate spatial data with their citizens. •  Islington was able to visualize city assets such as: •  Traffic assets •  Road markings •  Bus stops •  Trees •  Roads •  Crime layers •  The Islington Public Realm division used GEE to map crime data provided by the police to help their staff ‘design out’ certain types of crime common to an area. •  With crime layers active in GEE Islington staff could improve lighting or remove foliage that would otherwise be conducive to specific types of crime. Virtual Alabama •  Another use case is Virtual Alabama. The state used Virtual Alabama as a visualization system that provided a common operating picture or COP for the entire state of Alabama. •  It mapped all 67 counties, had over 2100 users, and represented over 550 agencies •  Using Google Earth Enterprise, Virtual Alabama helped the state assemble, display, evaluate and share data with state, county and municipal governments, including emergency responder teams and law enforcement. •  A great example of this involved using historical imagery after a tornado to allow officials to look at before and after views of the state to help respond to emergencies •  Using and loading KML layers Virtual Alabama allowed users to view live video feeds •  Virtual Alabama also allowed the public school system to map floor plans of all schools in the state
  5. 5. The trajectory of GEE to open source actually started almost 13 years ago •  In 2004, the Keyhole company was acquired by Google and with it came the Earth product •  In 2005 Google Earth launched and over the next 10 years GEE evolved bringing with it the Google Earth Plugin, portable, and other features •  In 2015 Google decided that it would deprecate Google Earth Enterprise - boo •  However after realizing that a large community of GEE users still existed, especially in the government space, they decided that it was better to open source and let the community lead it’s direction •  Since 2016 we’ve been working with Google to help them bring GEE to the open source community •  And after lots of hard work we helped Google release GEE on Github for all to enjoy – Hooray!
  6. 6. The  road  to  open  source  was  not  without  it’s  challenges  of  course.    GEE  was  a   proprietary  product  with  over  470K  lines  of  code  and  17  years  running  under   closed  development:     There  were  3  primary  concerns     One,  changes  regarding  compilers,  build  systems,  and  installers  had  to  be  made     Two,  unbundling  of  libraries  and  specifically  removal  of  proprietaries  libs  was  a   consideraFons     And  Three  and  probably  the  easiest,  was  to  move  to  a  new  configuraFon   management  system       My  First  Template  
  7. 7. Bring it !
  8. 8. •  The first decision was to move away from Google’s custom cross-compiler, based on g++ 4.4, although this allowed code to be compiled once and deployed to multiple Linux environments it required all the library dependencies to be brought along. •  Instead now OpenGEE code is built on each target system i.e. Ubuntu 14.04, 16.04, RHEL/ CentOS 7 using the native g++ compiler •  Overall, this improves the quality of source code and removes any custom Google code •  The Scons build system had to be updated in several places. Libraries are in different places on different operating systems, RHEL vs. Ubuntu •  The build and packaging process was previously automated, so the scons targets didn’t necessarily have to be developer friendly. •  As a result, when the automation was removed, there were 5 different scons targets that had to be called from 4 different directories. •  One of our tasks early on was to simplify this process by consolidating the scons targets to just 2 (build and stage_install targets) from a single directory. •  Due to the open source nature of the project, the proprietary InstallAnywhere software would not be compatible. •  Ideally, OS level packages would be the ideal path for installation. However, there are still many libraries that are still bundled with Open GEE, so that is currently impractical. •  As a compromise, bash scripts were written based on the operation of the previous installers. Thankfully, there was good documentation written by previous Google engineers that detailed the steps necessary. •  We now have installers and uninstallers for Google Earth Fusion Pro and Server.
  9. 9. As part of the open-source effort the team decided that removing as many proprietary libraries as possible was a good idea Google internal libraries had to go. For the most part, this was easy to migrate to open source versions, such as gtes Qt - Originally used a commercial version of the library. This had to be migrated to an open source version of the librar One of the recent contributions to OpenGEE is providing JPEG 2000 support for GDAL. The team unbundled and removed Jpeg2000 support and instead started using the open source OpenJPEG. Similar to JPEG2000, MrSID support was provided via the Lizardtech library. Currently we’ve only known MrSID to be available as a proprietary library. However, users can probably get a license to the SDK and compile GDAL themselves. We believe at this point that a read-only license is available for free.
  10. 10. Moved to GIT from Keyhole’s source control Piper Perforce tool And finally Migrate away from Google’s CM system to GitHub
  11. 11. This all sounds great but why do I care. Well there are various benefits of GEE going open source and here are some: •  Continuous support – GEE can now be supported by the open source community. For customers who thought that GEE would be a dead technology can now continue using it knowing the community will support it. •  Quadtree Spec – The quad-tree spec is the internal spec for organizing an calling tiles in GEE. The quad-tree expanded upon in the open source. •  Low barriers to entry (A suite of tools that a couple of years ago costs tens of thousands of dollars is now freely available). A GIS product that maybe was cost prohibitive for smaller companies, state/local agencies, and non- profits is now available. •  New features are possible as the community needs them. •  Flexibility to embed in other systems. Now with the code open sourced embedding components i.e. portable is a possibility •  Automation of processes. With the source code available and extensible ways to expose APIs and automate processes i.e. fusion of imagery is now a reality
  12. 12. Communication Slack •  We needed a way to communicate with our community so we deiced to use a tool that we already use internally to communicate amongst ourselves . •  For group collaboration we use Slack. We make a conscious effort to invite people to slack. •  Anytime someone has an issue we extend the link to them so that they can join as well. FAQ •  At the beginning of the project we were dealing with creating installers and testing on various platforms. In addition, the tutorial documents were a little out of date with the current state of the project. •  We realized an FAQ page would be very helpful in handling the same questions that kept popping up. We’ve found it very useful for providing quick responses (which we just mentioned was important) and addressing complex topics in a concise manner. •  This is one of those organic things that grew out of working with the community. Responsiveness •  There may be no well defined roadmap on how to make a successful open source project but one way to definitely kill one is to not be responsive. •  We’ve made this a big focus and try to respond to questions on Slack and Issues in Github every day. •  Alex, our Community Manager does a sweep every morning and afternoon and makes the team aware of any issues that have not seen any response. We firmly believe this helps show that we are an active and supportive community for both developers and users. Measuring Progress •  We’re are using GitHub’s built in graphs and Pulse to start tracking metrics. Had to determine what Metrics are important to track. Decided on splitting them into three categories: •  Discovery •  Views •  Usage •  Clone and forks •  Retention •  Contributions •  Repeat contributors
  13. 13. Challenges GEE is not without it’s challenges. The NPAPI 3D plugin is being phased out by modern so having a transition plan for 3D support is essential 3D API Luckily, the community has already responded with Cesium providing support for OpenGEE 3D Tile Spec More support for 3D rendering is something we seriously looking at, especially the glTF 3D Tile Spec already supported by Cesium. Fusion Clustering Speeding up the fusion process is something atop of our list of things to do. Multi-threading processes and Fusion clustering is something we are looking at currently. Maybe supporting Fusion in the cloud could be an option
  14. 14. Lots of great things to happening with OpenGEE and we encourage you to come join us. •  If you want to get involved please start by going to opengee.org. There you’ll find a lot of information, news, and a blog about the project. •  From there you’ll want to head over to the github project @google/ enterprise/ Like I mentioned earlier we keep a very active slack channel at slack.opengee.org so if you curious or have questions about how to work or contribute to OpenGEE please jump on. •  Finally shoot is email if you any particular questions or join the google group
  15. 15. •  That’s it for me now I’ll open it for any questions. •  I’ve brought some our OpenGEE team along to help answer any questions you might have. •  Alex Hernandez is our Community Manager and Chris Camp is our Lead Engineer. These are the two guys with the master plan so if you have juicy questions about the internals of GEE these are your guys Questions? Ok, finally if you see us around please come by and say hello. We’d love to chat with you guys Thank you very much and see you on slack

×