We're in the final stretch of this six-part series on spatial data accuracy – the icing on the cake. I've provided appropriate background, examples and advice, and now we'll focus on actual LANDFIRE accuracy results as seen through my lens.
LANDFIRE has conducted several internal quantitative assessments of program products and posted these results to the "Data Quality" section of the LF program website. Assessments that were conducted by other organizations are also linked there. I encourage you to review what is posted, and if you have questions, suggestions or concerns, please contact us.
I could focus on literally hundreds of items from the assessments, but in the interest of time and space, I'll offer a taste of my personal impressions and highlights. Remember: what is important and useful to me may be of no interest to you, so in my version of STOP, DROP and ROLL I will ask you to VISIT, DOWNLOAD and REVIEW assessment results yourselves. You will only see what you look for…DON'T RELY ON ANYONE, INCLUDING ME, TO DO THAT FOR YOU.
Disclaimer: these impressions are mine and mine alone. They do not represent the official stance of the LANDFIRE Program nor its sponsors and partners.
Existing Vegetation Cover
One assessment of LANDFIRE Existing Vegetation Cover was performed for the LF2008 version (document and presentation). Because of limited available plot data, it was conducted for forest classes only. Mean bias was low, and average absolute differences were approximately one, e.g. 10% forest cover class. As with most models, lower cover plots tended to be over-predicted and higher cover plots tended to be under-predicted. Expect small canopy openings to be missing in the LANDFIRE forest cover data. This is well within my expectations for use at the landscape, regional, and national levels, and perhaps for local forest applications as well after local review.
LF2008 Existing Vegetation Height Class
One assessment of LANDFIRE Existing Vegetation Height Class was performed for the LF2008 version (document and presentation). Again, because of limited available plot data, it was conducted for forest types only. LANDFIRE maps five forest height classes of varying width (shorter height classes are narrower; taller height classes are wider). Overall, LANDFIRE Existing Height Class for forest categories matched plot information 74% of the time, and were within one height class 95% of the time. As with most models, more major errors occurred at the extremes of height. LANDFIRE height information is coarse but useful at the large landscape, regional, and national levels. However, I feel the height data is limited compared to field data for local applications.
LF2010 Existing Vegetation Type
This is the dataset of highest interest to most users because it is "familiar" in concept, and because it is a fundamental product on which other products downstream in the development process depend. I will focus on LF2010 results, because they 1) are the most recent, 2) utilized larger independent assessment data sets, and 3) contain new information that may be more beneficial to users.
- In my view, overall agreement is of limited usefulness, but many still feel the need to know. There is no consistent overall agreement result across GeoAreas, but is generally lower than most people expect for local applications. However, I am uncertain about what the local agreement expectation should be for a national program delivering several hundred different vegetation categories.
- Despite our ability to increase the assessment sample size, field plots that are useful for assessing EVT agreement are severely lacking in many vegetation types, which constrains the applicability of the assessment in some situations.
- Category agreement is extremely variable both geographically and thematically, from very low to very high. It is my impression that the more ecological "extremes" (very dry or very wet) are mapped with lower local agreement than ecological systems in the grand middle of the overall ecological spectrum. If you want to see the numbers, you need to download the documents.
- LANDFIRE maps several hundred vegetation categories, which is more thematic detail than many applications require. Users should understand that what is presented in the LANDFIRE assessment reports is, in effect, a lower bound on estimated agreement. Fewer combined vegetation categories will have higher agreements. For example, this riparian evaluation application of LANDFIRE data found higher local agreements because the legend of interest was highly simplified.
- Users can estimate the accuracy of a collapsed category scheme by appropriately adding rows and columns in the contingency tables provided by the LANDFIRE program.
Accuracy/agreement/product usability is ultimately defined by individual needs and, indeed, individual expectations, but no one would disagree that there is room for improvement in all LANDFIRE Program products. The LANDFIRE Program has demonstrated again and again that it will listen and respond to user feedback. Please join us in this continuous improvement process because we are all reaching toward the same goal…useful information for everyone.
Thanks for hanging with me through all six parts of this blog series. I hope I have inspired thought and -- more importantly -- conversation about map quality. I would love to hear from you.
Contact Jim