Recommended
More Related Content
Viewers also liked
Viewers also liked (16)
More from Rahul Rakshit
More from Rahul Rakshit (13)
Recently uploaded
Recently uploaded (20)
OBIA accuracy survey
- 1. A survey of Accuracy assessment measures in OBIA Rahul Rakshit rrakshit@clarku.edu Graduate School of Geography Clark University hero.clarku.edu/holmes Graduate School of Geography, Clark University 1
- 2. Background: Pixel Vs Object based accuracy estimates Non Legend Forest Forest Classification Reference Accuracy Assessment Pixel 1 Pixel 1 Pixel 1 Inaccurate Classification Object 1 Object 1 Object 1 Inaccurate Geometry Object 1 Object 1 Object 1 Inaccurate Classification Graduate School of Geography, Clark University 2
- 3. Background: Selecting the sampling unit - Point Land-cover 1. Impervious 2. Grass 3. Bare Soil 4. Coniferous 5. Deciduous Scale 1:600 1:150 Graduate School of Geography, Clark University 3
- 4. Background: Selecting the sampling unit - Polygon Land-cover 1. Impervious 2. Grass 3. Bare Soil 4. Coniferous 5. Deciduous Graduate School of Geography, Clark University 4
- 5. Background: Positional Accuracy of a GPS Land-cover 1. Impervious 2. Grass 3. Bare Soil 4. Coniferous 5. Deciduous WAAS Wide Area Augmentation System 5 m accuracy Graduate School of Geography, Clark University 5
- 6. Selection Criteria: 100 papers Studies that have used OBIA as a classification tool to create thematic maps 70 69 60 •IJRS No. of Papers 50 •PE&RS 40 •RS of Env 30 24 •Sensors 20 •CJRS 10 7 •Landscape & Planning 0 Conference Proceedings Book Chapters Peer Reviewed Articles Source 30 30 No. of Papers 20 15 14 10 10 7 8 8 4 3 1 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Year Graduate School of Geography, Clark University 6
- 7. Remote sensing accuracy assessment Response Design Method for determining the reference class Sampling Design Method for choosing locations at which reference class will be determined Analysis Agreement between classified Vs reference datasets, results Stehman (1998), Congalton and Green (2008) Graduate School of Geography, Clark University 7
- 8. Survey Attributes Data Description ●Paper Authors Year of Publication ●Accuracy Assessment Performed Classification Design ●Theme Year of Data Sensor Resolution No. of Classes ●Software Segmentation Properties ●Segmentation Algorithm Segmentation Scale Sampling Design ●Sampling Method Total samples Sampling Units No. of samples per class Response Design ●Reference Data Source Independent validation source ●Difference in years between reference and training data Analysis ●Error Matrix Alternate error estimation Overall Accuracy Graduate School of Geography, Clark University 8
- 9. Sample Data AA = accuracy assessment Blank Space = no data available Graduate School of Geography, Clark University 9
- 10. Results: 12% 88% Accuracy assessment permformed Accuracy assessment not permformed • Accuracy estimated as sufficient/reasonable. • Accuracy measured by visually comparing the OBIA outputs with the imagery. • Accuracy assessment to be conducted in the future. Graduate School of Geography, Clark University 10
- 11. Sampling Design Sampling Unit Accessible 1% 37 39 24 Points Polygons No Information No. of Samples 13 50 50 Total no. of samples No Information Sampling Method No. of samples per class Graduate School of Geography, Clark University 11
- 12. Response Design Independent source of validation 34 56 10 Yes No No Information 28 72 Source of reference data Year of Reference Data No Information Graduate School of Geography, Clark University 12
- 13. Analysis Graduate School of Geography, Clark University 13
- 14. Conclusions 1. Most of the studies do not mention the sampling method 2. Both points and polygons are used as sampling units 3. Almost half of the studies do not provide information on total number of samples 4. Most of the studies use field data (GPS) as reference datasets 5. Majority of the studies separate validation data from classification data 6. Majority of the studies do not provide information on year of reference datasets 7. Majority of the studies use the error matrix to show accuracy assessment results 8. Some studies have visually estimated the agreement between classified and reference data Graduate School of Geography, Clark University 14
- 15. Recommendations: • Object geometry accuracy can be quantified by tools such as LIST (Landscape Interpretation Support Tool) and CI (Comparison Index) that use polygon overlay • Data quality: Jarlath O’Neil Dunne http://letters-sal.blogspot.com/2010/08/is-peer-reviewed-literaturethe-best.html • Temporal difference between classification and reference datasets should be kept to minimum • Avoid misregistration (low positional accuracy) • Take spatially well distributed samples (spatial autocorrelation) • Use stratified random sampling • Specify sample size for each stratum • Use thematic maps as reference datasets with caution • Mask out training data from sampling design • Hybrid point and polygon sampling approach (Albert Decatur: Upcoming) Graduate School of Geography, Clark University 15
- 16. Acknowledgements • Prof. Robert Gilmore Pontius, Jr. • Prof. Colin Polsky • Prof. John Rogan • Albert Decatur • Shitij Mehta • Jarlath O’ Neil Dunne More Information: rrakshit@clarku.edu http://hero.clarku.edu/holmes This material is based upon work supported by the National Science Foundation (NSF) under grant Nos. BCS-0709685 (Coupled Natural- Human Systems), OCE-0423565 (Long-Term Ecological Research), SES-0849985 (REU Site), and BCS-0948984 (ULTRA-ex), and by the Clark University O'Connor '78 Endowment. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funders. Graduate School of Geography, Clark University 16
Editor's Notes
- I work in the holmes project in Clark university and we produce land-cover maps. We use object based image analysis to create these maps from images that are 45 cms in resolution. We have mapped more than 4 billion pixels in our study area.We selected OBIA method to produce land-cover maps with considerable accuracy when compared to the pixel based methods.While working on this project we realized that conducting an accuracy assessment of maps produced by OBIA techniques is a little harder and different form the traditional pixel based validation techniques. Therefore a survey was done to find out how different studies in the last 10 years have conducted their validation exercise when they have used OBIAas a classification tool.
- Now let us see how the OBIA method is different from that of thePixel based method.
- The other factor in OBIA accuracy assessment is the choice of the sampling units. The sampling units can be either points or polygons.Traditionally random or stratified random points are generated on the landscape and then the points are visited either on the field or on a high resolution image and then the category of the point is noted at a particular scale as seen in this image. The scale is 1:600.It is a difficult task to assign one of the 5 land-cover categories to the point. Even after zooming in at 4 times the previous scale it is still not clear.
- Now let us take the example of a polygon a the sampling units. Polygons are selected randomly or stratified randomly. These polygons or segments are created using a particular scale. That scale may produce pure of mixed segments even for the same land-cover category. As seen in this example the segment in red is selected randomly but it can consist of more than one category because of the scale factor. Therefore, it is difficult to assign one single category to this polygon.
- One more thing to consider during accuracy assessment is the use of GPS to collect field data. Even with a WAAS enabled GPS that has an enhanced accuracy: the accuracy is till 5 m. In a heterogeneous urban landscape, the accuracy of 5 m can result confusion on which LC category to be assigned to the point when field data is collected.From our personal experience when we went to do the fieldwork, three persons with GPS units stood in the edges of this circle with the same co-ordinates (upto 2 decimals places). Each of them was standing on a different land-cover category.
- On the basis of these points we decided to do a survey of existing literature that have used OBIA as a classification tool to create thematic maps.100 papers are used in the survey.The majority of the studies mapped land-cover, followed by vegetation mapping. Mapping impervious surfaces was also one the major themes in these papers. The papers are seleted from the following sources and their yearwise distribution is as follows.
- Positional accuracy for field data.Only one out of the 100 papers stated that some of the differences in agreement between the classification and the reference data can be attributed to the difference in time(4 years in their case) between the images.