Download The Earth-Observation Image Librarian (EOLib): The data mining

The Earth-Observation Image Librarian
(EOLib): The data mining component of
the TerraSAR-X Payload Ground Segment
Daniela Espinoza Molina, Vlad Manilici, Octavian Dumitru, Christoph Reck,
Shiyong Cui, Henry Rotzoll, Mathias Hofmann,Gottfried Schwarz and Mihai Datcu
German Aerospace Center
DLR.de • Chart 1
Motivation
 ENVISAT provided measurements of the
atmosphere, ocean, land, and ice over 10
years of operations generating a data archive
that reaches many petabytes
 TerraSAR-X and TanDEM-X contain an
extensive data archive of more than 100,000
scenes covering the majority of the Earth's
surface.
 Sentinels are and will be contributing with
data for land, ocean and atmosphere
monitoring and storing several petabytes
The data archive
What is the content of the Image archive ?
Motivation
The data access
How to access the data?
How to take advantage of the
data?
How to explain the image
content?
The data archive
EOLib: Earth Observation Image Librarian


Baseline architecture of EOLib system

EOLib is a modular system
composed of several components:
PGS in blue and new EOLib in
orange
EOLib offers mining/search
services for accessing the image
archive
EOLib generates semantic
descriptions of the image content
Payload Ground Segment Components




Operation Tool
Long Term Archive
Ingestion Interface
User Services
Wolfmüller, M., Dietrich, D., E. Sireteanu, S. Kiemle, E. Mikusch, and M. Böttcher, Data Flow and Workflow organization - The
Data Management for the TerraSAR-X Payload Ground Segment, IEEE Transactions on Geoscience and Remote Sensing, vol.
47, no. 1, pp. 44–50, 2009
EOLib: Knowledge Discovery and new components






Data Model Generation
Data Mining DataBase
Query Engine
Visual Data Mining
Knowledge Discovery in
Databases
Epitome Generation
M. Datcu, K. Seidel, 2005, Human Centered Concepts for Exploration and Understanding of Images, IEEE Trans. on
Geoscience and Remote Sensing, Vol. 43, No.3, pp. 601- 609
Data Model Generation
Data Model Generation
TerraSAR-X L1b product
Metadata
Extraction
TerraSAR-X metadata and
image
Image
Tiling
Tiles with different size
Quick –
looks
generation
Primitive
Feature
extraction
Primitive features: Gabor filters
and Weber Local Descriptors
Create the
product
model
Data Mining Database
Data Mining Data Base
• DMDB is a relational database
• Main tables are:
• Metadata
• Image
• Tiles
• Features
• Labels
• DMDB comprises about
• 8 millions of tiles
• 20 thousand metadata
entries.
• 106 semantic labels
Ack: TELEIOS
Query Engine
•
•
•
•
•
Coordinates (lat/lon)
Incidence angles
Acquisition time
Pixel spacing
Number of
columns/rows
• sensor
• Mission
• orbits
Metadata parameters are based on
XML annotation file of TerraSAR-X
L1b products
Semantics
Metadata
Query Engine
• Agriculture
• Cropland
• Rice plantation…..
• Bare ground
• Cliff
• Desert…..
• Urban area
• Commercial areas
• High density residential
areas….
• Forest
• Forest coniferous
• Forest mixed….
Semantic parameters are based on
EO Taxonomy[1]
[1] C. O. Dumitru, S. Cui, G. Schwarz, M. Datcu, 2015, Information Content of Very-High-Resolution SAR Images: Semantics,
Geospatial Context, and Ontologies, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 8,
No. 4, pp. 1635 – 1650.
Query Engine: Examples
Example of query: Storage tanks and Medium density urban area are the query parameters
Visual Data Mining
Visual Data Mining
 Provides a projection of the entire
database based on primitive
feature vectors
 Representation of the data in the
3D space (dimensionality
reduction)
 Interactive exploration and analysis
of very large, high complexity data
sets
 This allows the user:
 To browse the image archive
 To find scenes of interest
 Semantically consistent groups
may appear inside the data
Ack; Terrasigna
Espinoza-Molina, D., Datcu, M., Teleaga, D. and Balint, C., Application of visual data mining for Earth Observation use cases, ESAEUSC-JRC-2104 Image Information Mining Conference: The Sentinels Era, pp. 111–114, 2014
Knowledge Discovery in Databases
Knowledge Discovery in Databases
 KDD is used to define semantic
annotations of the image content.
 Goal is to build a model which performs
the mapping between low-level image
descriptors (primitive features ) and
high-level image concepts (semantics)
 KDD is based on machine learning
methods and relevance feedback
mechanisms.
Ack: CNES-DLR Center of Competence
D. Espinoza-Molina, M Datcu, 2013, Earth-Observation Image Retrieval Based on Content, Semantics, and Metadata. IEEE
Transactions on Geoscience and Remote Sensing, Vol. 51, No. 11, pp. 5145-5159.
KDD: Graphical User Interface
Positive examples in green, negative examples in red. Classification in blue
P. Blanchart, M. Ferecatu, S. Cui, M. Datcu, 2014, Pattern Retrieval in Large Image Databases Using Multiscale Coarse-to-Fine
Cascaded Active Learning, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 7, No. 4, pp.
1127 - 1141
Water bodies
Industrial area
High buildings
Forest
pyramid
Crops
Low density
Urban area
Sport areas
Storage tanks
Buildings
Low density
Urban area
Crops
Bridges
Residential area
KDD application example
The damages in the agriculture can be clearly seen by comparing the classification in pre disaster image
(left figure) with the post disaster image (right figure).
Agriculture
Bridges
Bridges
Aquaculture
Debris
H. Voltage
poles
TerraSAR-X scene before Tsunami
20.10.2010
Flooded areas
H. Voltage
poles
TerraSAR-X scene after Tsunami
12.03.2011
C. O. Dumitru, S. Cui, D. Faur, M. Datcu, 2015, Data Analytics for Rapid Mapping: Case Study of a Flooding Event in Germany and
the Tsunami in Japan Using Very High Resolution SAR Images, IEEE Journal of Selected Topics in Applied Earth Observations and
Remote Sensing, Vol. 8, No. 1, pp. 114 – 129.
Epitome Generation
Epitome and Epitome Browser
Epitome
Metadata
Image Grid Levels: Tiles,
Primitive features
EO Taxonomy
Tile annotations
 Epitome is a summary of the generated
information. It is considered as an enriched
and value-added product that can be
exploited interactively.
 The epitome is a result of image feature
extraction and semantic annotation and can
be delivered independently or together with
a standard EO product.
 Epitome comes with a browser
User Services
User Services
 Web Interface to search for
TerraSAR-X products.
 Advanced searches based
on
metadata
and
semantics
Conclusions
 We introduced the EOLib system which allows us to perform data mining and knowledge
discovery within the TerraSAR-X Payload Ground Segment.
 EOLib is a modular system which serves the next generation of Image Information Mining
systems.
 The main goal of EOLib is to create a communication channel between a Payload
Ground Segment and the end-user who receives the image content enriched with
annotations and metadata as well as coded data in an understandable format associated
with semantic categories that is ready for immediate exploitation in the form of an
epitome.
 EOLib will be interfaced to and operated in DLR’s Multi-Mission Payload Ground
Segment (PGS) of the Remote Sensing Data Center at Oberpfaffenhofen.