Download GEOSS Common Infrastructure (GCI)

Copernicus Climate Data Store
Workshop ECMWF Reading, 3-6
March 2015
GEOSS Common Infrastructure (GCI)
and the
GEO Discovery and Access Broker
(DAB)
Stefano Nativi
National Research Council of Italy (CNR-IIA)
Global Earth Observation Systems
of Systems (GEOSS)
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
The GCI revolutions
2005
2011
2013
2016
1st Revolution
[Service
driven]
Catalog of
catalogs
2nd Revolution
3rd Revolution
[Brokering
th Revolution
4
pattern]
[Big Data]
Brokering
[… .]
Cloudframework based
EcoSystem … .
Discovery
[functionality]
Discovery and
Access
04/03/
04/03/2015
(re-)Use 3
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
[functionality]
[functionality]
Service Providers
Service Users
Service-driven collaborative
Approach
Registry
Copernicus Climate Change Service, ECMWF March 3-6, 2015
04/03/2015
[email protected]
Generally Recognized Barriers
(especially for GEOSS)
PROBLEM: service users need to know the
nature and location of service providers,
 making it difficult to bind and dynamically change the
bindings between users and providers
SOLUTION: The broker pattern separates users
of services (clients) from providers of services (servers)
by inserting an intermediary, called a broker.
 When a client needs a service, it queries a broker via a
service interface. The broker then forwards the client’s service
request to a server, which processes the request
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Service Users
Service Providers
Collaborative-Acknowledged
Management approach
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GEOSS Information System
GEO
Home Page
GEOSS
Portal
Systems
Registry
Semantic
engines
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GEOSS Assets (Nov 2014)
More than 35 brokered Data Providers –
capacities, systems, Communities
.. .
Publish
About 14 Million (2 Million GEOSS Data
Core) Discoverable and potentially Accessible
first level resources (mix of data
collections, datasets and individual images)
Contain [source: data providers]
.. .
.. .
More than 82 Million (more than 50 Million
GEOSS Data Core) Discoverable and potentially
Accessible individual resources
(e.g. satellite scenes, rain gauge records)
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Ongoing interoperability tests:
• IGN
• UK data.gov
• Geoscience Australia
• GBIF
• DigitalGlobe
• FP7 GeoCarbon DBs
• e-GEOS Cosmo Sky Med
• ….




04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Flexible, adaptable, and extensible
interoperability Environment
Discovery and
Evaluation
Semantic
expansion
Access and
Transformation
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Interoperability Standards
Standardization
Federation
Common
Technological and Community/
Semantic baseline
Organization
interoperability
Mediation/Brokering
Cross-domain and Crossorganization
standards mediation
04/03/
04/03/2015
11
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Big Data challenges for GEOSS

Volume:

Variety: Different product types (data, services, models, documents);

Veracity/Validity/Value: Evaluation support, Essential Variables,
Millions of discoverable (small & medium) products; Long
EO time/space series, … .
Data models; Protocols; Semantics; Granularity levels; Organizations;
Maturity level, … .
Discovery Ranking, User Feedbacks, … .
o Velocity:
serve Countries with limited Internet access, Fast Discovery
& Access, Data Transformation and Analytics
o Visualization: Preview, Tiling services, visual exploration, … .
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GEOSS Information System
GEO
Home Page
GEOSS
Portal
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Web
Browser
Mobile
App
Mobile
App
GEO
Home Page
GEOSS Community Portals
GEOSS
Portal
DAB APIs
Processing
Discovery
Evaluation
Tagging
Workflow mangmt
(simple)
Access
Visualization
Mobile
App
Brokering
QoS control
Tiling
Monitoring
Platform
Data Server
Data Server
Infrastructure
Existing e-Infrastructure Services
….
Existing e-Infrastructure Services
04/03/
04/03/2015
14
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GCI (hybrid) IaaS and PaaS
Computing
Storage
Monitoring
Auto Scaling
Load Balancing
Routing
Clustering
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GEO DAB in the Cloud
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Scalability benefits:
Discovery Ranking Metrics
• A weight-based algorithm rewarding
a set of criteria belonging
to four high-level principles
Record score
depends on
Record
4 main Aspects
quality
Data
accessibility
Essential Variables
• The solution
applied by DAB
come first
Metrics is: Configurable and
Textual
Domain
constraints
Matching
constraints
Matching
Flexible
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
DAB Monitoring
and Statistics
• Statistics
• Year, month, week, day
•
•
•
•
Most used discovery constraints
Most used remote data systems
Number of discovery requests
Number of access requests
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GEO DAB Authentication
 Support existing and
well-used users’
credentials (e.g.
Facebook, Google,
Twitter, etc.)
 Support OpenID and
OAuth protocols
 Support GEO DAB
credentials
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GEO Discovery and Access
Broker APIs
• GEOAPI are high
level clientside Open APIs (Application
Program Interface)
• DAB users are typically software
agents, such as web-based or
desktop client applications
Designed and developed in JavaScript to simplify the
development of applications and clients making use of the DAB
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
GCI and DAB Achievements
 (From a “Catalog of Catalogs” to a) multi-disciplinary Brokering
Platform
 (From discoverability to) accessibility and harmonization
services
 (From an single infrastructure to a) cloud-based software
ecosystem
 (From a virtual governance to a) collaborative-acknowledged
governance
04/03/2015
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]
Thank you !
Questions?
04/03/
04/03/2015
22
Copernicus Climate Change Service, ECMWF March 3-6, 2015
[email protected]