Downloaded 22 times
![Global composites with a few lines of code
var composite = ee.Algorithms.Landsat.simpleComposite({
collection: ee.ImageCollection('LANDSAT/LC08/C01/T1'),
asFloat: true
});
Map.addLayer(composite,
{bands: ['B4', 'B3', 'B2'], max: 0.3},
'composite');
https://code.earthengine.google.com/05d2e23206b329dfe696e5ba8e232c3f](https://image.slidesharecdn.com/whatisgoogleearthengine-221209080129-5f9b35d0/85/What-is-Google-Earth-Engine_-pptx-28-320.jpg)
More Related Content
Similar to What is Google Earth Engine_.pptx
What is Google Earth Engine_.pptx
- 1. What is GoogleEarth Engine? https://goo.gl/n5Gh5Q Nick Clinton, nclinton@google.com Earth Engine Developer Relations
- 2.
- 3. Goodchild et al.(2012): “The supply of geographic information from satellite-based and ground- based sensors has expanded rapidly, encouraging belief in a new, fourth, or “big data,” paradigm of science that emphasizes international collaboration, data-intensive analysis, huge computing resources, and high-end visualization.”
- 4.
- 5. "To organize theworld's information and make it universally accessible and useful." https://www.google.com/about/company/ Google Mission Statement
- 6. -Jim Gray (1944-2007) “Oftenit turns out to be more efficient to move the questions than to move the data.”
- 7. > 50 Petabytesof Earth observation data (imagery, weather, etc.)
- 8.
- 9. Powerful JavaScript* API *python too!
- 11.
- 16.
- 17. 33 years Of satellitedata Over 5,000,000 Landsat and Sentinel scenes analyzed 3 Quadrillion Pixels 3,000,000,000,000,000
- 18.
- 19. The Earth EngineData Catalog > 200 public datasets MODIS 250m daily Vector Data WDPA, Tiger Weather & Climate NOAA NCEP, OMI, ... Terrain & Land Cover > 4000 new images every day > 5 million images > 7 petabytes of data Landsat & Sentinel 1, 2 10-30m, weekly ... and upload your own vectors and rasters
- 20.
- 21. BIG data: 40+years of Landsat imagery
- 22.
- 23.
- 24.
- 25.
- 26.
- 27. Data Types andGeospatial Processing Functions ● Image - band math, clip, convolution, neighborhood, selection ... ● Image Collection - map, aggregate, filter, mosaic, sort ... ● Feature - buffer, centroid, intersection, union, transform ... ● Feature Collection - aggregate, filter, flatten, merge, sort … ● Filter - by bounds, within distance, date, day-of-year, metadata ... ● Reducer - mean, linearRegression, percentile, histogram …. ● Join - simple, inner, outer, inverted ... ● Kernel - square, circle, gaussian, sobel, kirsch … ● Machine Learning - CART, random forests, bayes, SVM, kmeans, cobweb … ● Projection - transform, translate, scale … over 1000 data types and operators, and growing!
- 28. Global composites witha few lines of code var composite = ee.Algorithms.Landsat.simpleComposite({ collection: ee.ImageCollection('LANDSAT/LC08/C01/T1'), asFloat: true }); Map.addLayer(composite, {bands: ['B4', 'B3', 'B2'], max: 0.3}, 'composite'); https://code.earthengine.google.com/05d2e23206b329dfe696e5ba8e232c3f
- 29. Computation Platform On-the-Fly Computation Interactively visualizedata, serve queries, and preview analysis results. Batch Computation Large-scale parallel computation at the click of a button.
- 30. The Earth EngineCode Editor Your Scripts & Example Scripts API Docs Your Data Search Your Code Data Inspector Batch Tasks Output Console Drawing Tools Map code.earthengine.google.com
- 31.
- 32.
- 33.
- 34. Image credit: NewYork Times
- 36.
- 37.
- 39.
- 40.
- 43.
- 44. Data Models Feature Line /Point / Polygon List of Properties TNC Ecoregions
- 45. Data Models Feature Image Stack ofGeoreferenced bands Each band has its own: Mask, Projection, Resolution A list of properties, including: Date, Bounding-box
- 46. Data Models Feature Image Collection Bag ofElements Table of Features Directory of Images Filter, Sort, Join, Map, Reduce
- 47. Map Apply a functionto each element of a collection A "map" (for-each) operation Examples ● Compute area of each feature ● Cloud cover of each image ● Mosaic for each month
- 48. Reduce Aggregate everything ina collection "Reduction" Examples ● Summed area over all features ● Median-pixel composite ● Train a classifier
- 49. Reducers in EarthEngine 8 ways to reduce Image.reduce Image.reduceNeighborhood Image.reduceRegion Image.reduceRegions Image.reduceToVectors ImageCollection.reduce FeatureCollection.reduceColumns FeatureCollection.ReduceToImage 40+ reducers Reducer.allNonZero Reducer.and Reducer.anyNonZero Reducer.count Reducer.countEvery Reducer.histogram Reducer.intervalMean Reducer.linearFit Reducer.linearRegression Reducer.max Reducer.mean Reducer.median Reducer.min Reducer.minMax Reducer.mode Reducer.or Reducer.percentile Reducer.product Reducer.sampleStdDev Reducer.sampleVariance Reducer.stdDev Reducer.sum Reducer.toCollection Reducer.toList Reducer.variance
- 50.
- 51.
- 52.
- 53. Reduce Region B1 B2 B3 B4 B5 Dictionary { B1: 8.3, B2:14, B3: 176, B4: 1.6, B5: 7 }
- 54. Reduce Regions B1 B2 B3 B4 B5 B1 B2B3 B4 B5 F1 F2 F3 FeatureCollection F1 F2 F3
- 55. Reduce To Vectors labels B1 B2 B3 B4 B1B2 B3 B4 F1 F2 F3 F4 F5 ... FeatureCollection
- 56.
- 57.
- 58. Tiling Images are tiledduring ingestion
- 59. Tiling Images are tiledduring ingestion
- 60. Tiling Images are tiledduring ingestion Downsampled by averaging
- 61. Tiling Images are tiledduring ingestion Downsampled by averaging
- 62. Tiling Images are tiledduring ingestion Downsampled by averaging During computation
- 63. Images are tiledduring ingestion Downsampled by averaging During computation Compute output tiles Tiling
- 64. Images are tiledduring ingestion Downsampled by averaging During computation Compute output tiles Tiling
- 65. Images are tiledduring ingestion Downsampled by averaging During computation Compute output tiles Find intersecting source tiles Reproject into the output projection Tiling
- 66.
- 67.
Editor's Notes
- #2 Share the presentation as public, and update the shortlink accordingly.
- #3 This section has information about the background and history of Earth Engine.
- #4 Setting the stage. This is what the external community was asking for around the time EE was under development (this paper was written several years before it was published). What you will discover is that Earth Engine delivers exactly what academia needed.
- #5 There are 40+ years of remotely sensed data available from a constellation of satellites and sensors, but many facilities still lack capacity for downloading and analyzing the data.
- #6 Organizing geospatial data is part of Google's mission! On of the fundamental goals of Earth Engine is to organize all that satellite imagery and make it accessible and useful.
- #7 The most efficient way to make the data accessible and useful is to "move the question to the data." Jim Gray elaborated on this idea in the influential book by Hey et al., director of research at Microsoft. Earth Engine implements this plan.
- #8 In order to move the question to the data, Earth Engine hosts a petabyte-scale archive of satellite imagery and other geospatial data on Google infrastructure. At the same time, Earth Engine provides an API in order to perform processing, analysis, visualization of the data, also using Google machines. Data storage and processing are all performed at Google. You connect to the service through a web browser (details coming).
- #9 You need compute power colocated with the data.
- #10 You need an API to be able to implement your geospatial workflows.
- #11 Example. South East Coast of Borneo as shown in Landsat imagery. At the time this image was collected, it was the most cloud free Landsat image of this area. Since the clouds move around over time, by applying a simple algorithm to a stack of such images, it's possible to opportunistically choose clear pixels in order to create a cloud-free composite. For example, the median of each pixel over time gets rid of both cloud shadows (dark) and clouds (bright), leaving surface reflected light (right in the middle). Some of these places had never before been seen from space in their entirety. This was a clue that Earth Engine might be useful.
- #12 Once the concept was demonstrated for individual places, the next logical step was to create a cloud-free composite of the world! This greenest-pixel composite became known as Pretty Earth, because it's not a real representation of Earth's surface. There is not a cloud in the sky anywhere, it's springtime everywhere, but it's a beautiful representation that became Google Maps and Earth satellite basemap.
- #13 Once a create a cloud-free composite of the world was made for a single time period, the next logical step was to make many composites spanning 30 years of the Landsat record and turn those composites into a video! That video became know as TimeLapse (developed in conjunction with Randy Sargent at CMU). It was released on the Time magazine website and won a webby for best use of video on the internet.
- #14 This animation shows that TimeLapse is nearly global in scope. That means you can go to your area of interest and see how places you care about have changed over time. A few compelling examples follow.
- #15 Meandering river in Pucallpa, Peru.
- #16 Urbanization in Suzhou, China.
- #17 Amazon deforestation. Time Lapse is available from the Earth Engine home page. You can pan, zoom and interact with the video to explore places of interest. You can make tours and/or embed the video in other websites.
- #18 Making Time Lapse used a huge amount of data, taking millions of hours of computation. However, when such a job is run in massive parallel on Google infrastructure, it just takes a few days.
- #19 That's the background. Now it's time to talk about what Earth Engine is. At its core is the petabyte-scale data catalog. Here's a brief introduction to the public data hosted in the Earth Engine archive.
- #20 Landsat data. Whatever USGS has, Earth Engine has. Landsats 1-8. Raw, TOA reflectance, surface reflectance. Sentinels 1-2. TOA reflectance. SAR data from Sentinel-1 processed to backscattering coefficients. Most MODIS terrestrial composites. Global DEMS at a variety of resolutions. Multiple land cover datasets. Atmosphere and climate data. And growing daily. There is a one or two day latency between scene acquisition and when it is ingested to the catalog.
- #22 This is the pixel count of Landsats. To make an image like this would be very, very difficult if you had to download all that imagery. In Earth Engine it is just a few lines of code.
- #23 High temporal cadence data. This is a harmonic model fit to 10+ years of MODIS EVI composites. The colors represent the seasonality of max greenness predicted by the model. Another example of a very difficult analysis that can be done with not very much code in Earth Engine. (We teach this example in about half an hour).
- #24 Terrain data. The purpose of this slide is to illustrate that compelling visualizations can be created and exported exactly as they appear on screen. You have control over scale, projection and rendering.
- #25 This image is mean annual ozone from the merged OMI/TOMS dataset. Red is less ozone, blue is more ozone. There are lots of other climate and atmosphere data in the catalog. Use the search tool to explore the catalog.
- #26 A composite of Sentinel-1 backscattering coefficients.
- #27 You've seen a sample of what's in the data catalog. The next section is intended to give a sense of what you can do with those data.
- #28 Image is the fundamental raster data structure in Earth Engine. Bands. Pixels. Metadata. ImaeCollection is stack or time series of images. Feature is the fundamental vector structure in Earth Engine. Geometry and metadata. FeatureCollection is a collection of Features (surprise). Filter is how to limit the scope of analysis temporally, spatially or by metadata. Reducer is the way to aggregate data. Statistics such as min, max, mean, SD, variance, covariance, linear regression, etc. Joins are the way to combine different data sets. Kernel facilitates image processing operations, for example convolution. Machine Learning algorithms support supervised and unsupervised classification. Projection is the way to control scale and appearance of data outputs. And growing in response to user requests.
- #29 Concept: reduce a stack of images to one image. The inputs are the images. The output is the median. Map is just for review. The reducer is repeated. It is evaluated separately for each band. Come to the arrays session to learn another way.
- #30 Interactive mode is the Code Editor. To run big jobs at arbitrary scale and scope, use batch mode.
- #31 Where the magic happens. This is the online IDE at code.earthengine.google.com running inside a Chrome browser. All you need to use Earth Engine is an internet connection. Note the Scripts tab (Git repository and examples to help get started), Docs tab (API reference docs), Assets tab (upload your own data), Inspector tab (query the layers on the map), Console tab (messages), Tasks tab (execute long running tasks), Code Editor (for JavaScript, but there's also a Python API), Map (including layer tools, geometry tools, etc. The image displayed in this slide is from the Image Collection > Linear Fit example), Get Link button (send it to your friends!), Search bar, Help button
- #32 "Cloud computing: it's as if you had access to a supercomputer designed for geospatial analysis." All the heavy lifting is done on Google servers. Getting the result is low bandwidth. How does it work? The code you write in the Code Editor gets turned into an object representing the set of instructions which is then sent to Google for processing. The analysis you requested is then run in parallel on many computers. What you get back in your browser is only what you request, for example a statistic or chart printed to the console or small RGB tiles to display on the Map. This is low bandwidth, but it's as if you have access to a supercomputer for geospatial analysis.
- #33 See this paper for more details. Segue to publications.
- #34 So far has been a little about background, the data catalog and the API. What follows is a brief literature review of papers that have been done using Earth Engine.
- #35 This visualization is from the New York Times article about the Pekel et al. publication: https://www.nytimes.com/interactive/2016/12/09/science/mapping-three-decades-of-global-water-change.html
- #37 Lobell et al. (2016) fit regressions from many simulations of crop growth over a range of meteorological variables. They used relationships between RS variables and model variables (e.g. VI <-> LAI) to apply the equation in each pixel. They compare aggregate predictions to reported yields (with varying success).
- #38 Dong et al. (2016) used a phenology and pixel based algorithm to estimate rice paddy distribution. Flooding signals are based on interaction of vegetation and water indices. LSWI > NDVI for example. LST for growing season. Define rice paddies as areas flooded for at least 10% of the growing season.
- #39 Allred et al. (2016) used data on well location in conjunction with MOD17 NPP to estimate the loss of ecosystem services from oil and natural gas drilling. Well density per sq. km. determined the percentage (up to 33%) by which the NPP is reduced.
- #40 The previous examples were all published studies and what follows represent examples of operationalizing science through online, interactive apps.
- #41 Climate engine allows users to interactively create maps of climate data and trends, again without reading and writing any code. App Engine app. See https://developers.google.com/earth-engine/app_engine_intro
- #42 Screenshot of the eco dash monitor in Vietnam. Cumulative anomalies indicate the trajectory of the project area in terms of recovery from disturbance. An increasing trajectory indicates revegetation, while a decrease indicates lack of revegetation. http://ecodash-servir.adpc.net/ Also an App Engine app.
- #43 Push-button interactive app publishing. See https://developers.google.com/earth-engine/apps
- #44 All this will sound extremely mysterious, but will start to make more sense once you start using Earth Engine.
- #45 Vector data. Geometry + attributes.
- #46 Raster data. Now it's starting to get weird. Images can mix bands of different projection and scale.
- #47 A stack of images is an ImageCollection. A set of vectors is a FeatureCollection.
- #48 Do something to each element in a collection with map().
- #49 Aggregate data (i.e. compute statistics) with reduce().
- #50 See the docs for a definitive reference. Docs are auto-populated by the server, so should always be up to date.
- #51 "Spectral" reduction.
- #52 Convolutions, linear and non-linear.
- #53 Temporal reduction.
- #54 Spatial reduction
- #55 Spatial reductions (plural).
- #56 Raster to vector conversion.
- #57 AKA map algebra.
- #58 Earth Engine helpfully replicates one band images in mathematical operations.
- #59 Parallelization
- #62 Images exist at multiple scales.
- #64 Locate the dataset.
- #65 Identify tiles in the area of interest (e.g. Code Editor map).
- #66 Resample and reproject as necessary according to the output, then do the computation.
- #67 You code does not run at Google. It gets turned into a request object which is sent to Google for processing. See Gorelick et al. for details.
- #68 It's free for non-commercial use!