Download Slide 1

Exchange of trade data using SDMX
COMTRADE joint UNSD-OECD project
9 January 2007
Comtrade - overview





Provide a coherent source of merchandise trade statistics data
Eliminate duplication in provision and processing of data
Develop common data processing standards and tools
Share development costs
Nearly identical integrated systems for data collection and processing at both
agencies
 OECD processes member country data for OECD
 UNSD processes data for all other countries
 Content of complete databases in each Organisation is synchronised via the
Data Exchange Suite (DES)
DES = web service & synchronisation program (ComtradeTools)
DECISIONS
!!! Use SDMX as data exchange standard for UNSD  OECD data transfers
!!! Use SDMX as data dissemination format
Comtrade - screens
Comtrade - content
•
•
•
•
•
•
•
•
•
UN Comtrade is a main
disseminator of annual commodity
trade statistics - data since 1962
over 150 reporters 1.3billion records
Over 150 reporters per year (>95%
of the World Trade)
On average, more than 2 billion
records downloaded every month
Newly published data synchronized
with other international
organizations such as World Bank,
ECLAC, OECD, ITC, UNCTAD, etc
via Web Services
Imports, Exports, Re-exports, Reimports
Commodity by Trading Partner
Annual data
Value in US dollars
Net weight and secondary quantities
Comtrade – key elements
•
Classifications and Commodity Codes
HS (Harmonized System) and SITC (Standard International Trade Classification)
•
•
•
•
•
•
Reporter country (ISO-3166-1 3 digit numeric)
Partner country (ISO-3166-1 3 digit numeric)
Trade Flow (Export, Import, Re-export, Re-import)
Time (Year)
Unit of measurements (Weight, Currency)
Observations (Net weight, Net value)
•
•
Comtrade uses SDMX cross-sectional format v1.0.
Each commodity classification has a specific Key Family. Therefore, there are 6
key families: HS2002, HS1996, HS1992, SITC Rev.3, SITC Rev.2, and SITC
Rev.1
Explanatory notes are integrated with Comtrade SDMX.
This is a beta version. The SDMX structure can be modified.
Comtrade Web Services are open Site License users.
For further information please go to
http://comtrade.un.org/kb/article.aspx?id=10050
•
•
•
•
Comtrade – key families
•Data series defined in SDMX Key Families element structure:Dimension
Comtrade – data exchange
• Server: Comtrade web services
– Internet-based, HTTP protocol, REST architecture
• Client:
– Can access Comtrade web service via web
browser
– and/or use ComtradeTools – a command-line
program for data transfer/conversion applications
• SDMX cross-sectional format, schemas specific to
each classification - see UN site for more info
• Web service can compress data on the fly (70-80%)
• Access security based on IP address
Comtrade – data transfer
Trade Data Transfer Architecture
United Nations
Comtrade
Tariff Line
Total Trade
Comtrade
Web Services
Web Server Database Server
SDMX-ML
Element based XML
Organization #1
Http protocol
Internet
PC
XML Files
Organization #2
Comtrade Tools
Other Tools
PC
Organization #3
Http protocol
Comtrade Tools
Text Files
XML Files
Other Tools
Database Server
Comtrade – SDMX downloads
Data can be downloaded in SDMX format directly, e.g.:
– Classification: H2 = Harmonized System 2002
– Reporter: 36 = Australia; Partner 251 = France
– Years: 2003 and 2004; Commodity: Total trade
http://unstats.un.org/unsd/comtradedist/getSdmxV1.aspx?px=H2&r
=36&y=2004,2003&cc=TOTAL&p=251&comp=false
– Classification : S2 = SITC2
– Reporter: 36 = Australia; Partner 0 = World
– Year : 2004; Commodity: Total trade
http://unstats.un.org/unsd/comtradedist/getSdmxV1.aspx?px
Comtrade - tools
• Requires Win2000 or higher
• Standalone command-line program for Comtrade
data download; requires additional software to
synchronize SQL database
• Downloads data in SDMX-ML format, can transform
into CSV (or import into SQL Server)
• Available from
http://unstats.un.org/unsd/comtradedist/comtradeTool
ssetup.zip
Comtrade – tools usage
• ComtradeTools /px:s2 /r:36 /p:0 /cc:TOTAL /y:2004
• ComtradeTools /px:H2 /r:251 /cc:10* /y:2004
• ComtradeTools /px:H2 /r:251 /cc:10* /y:2004
/action:DownloadAndConvertToCSV
/outputDirectory:c:\temp\
• ComtradeTools /help
• ComtradeTools /year:2003
/action:DownloadDataSetList
• ComtradeTools /r:360 /y:2003 /px:H1
/source:Comtrade /format:SDMX
/action:DownloadAndImportToSQLDB
Comtrade – conclusions, future
•
•
•
•
•
•
•
•
•
A production application, based on SDMX data exchange format and
web services
SDMX cross-sectional format – standardised V1.0
Rapid - exchange can be compressed by web service
Can convert to CSV format, and upload/synchronise with SQL
database - no specialised SDMX knowledge necessary for end-users
SDMX is becoming “just a part of the plumbing”
UN Comtrade SDMX v.2.0 is in progress
Data collection via SDMX: would need to be simple, via a generic
format to avoid multiple schemas
User-friendly interface for ComtradeTools
Aligning with other trade SDMX trade projects such as CDS (Common
Data Set) from WTO