Download Virtual Met Mast™ verification report

Virtual Met Mast™
verification report:
June 2013
1
Authors:
Alasdair Skea
Karen Walter
Dr Clive Wilson
Leo Hume-Wright
2
Table of contents
Executive summary .................................................................................................... 4
1. Introduction............................................................................................................. 6
2. Verification process ................................................................................................ 6
2.1 Define Data Sources .................................................................................... 6
2.2 Define Verification Statistics......................................................................... 7
2.3 Grouping sites to calculate bias statistics..................................................... 9
3. Verification results summary................................................................................. 10
4. Other Verification Diagnostics .............................................................................. 11
4.1 Comparison of VMM with NOABL and NCIC ............................................. 11
4.2 Diurnal variation ......................................................................................... 13
4.3 Height variation .......................................................................................... 14
5. Virtual Met Mast™ Turbulence Intensity model. ................................................... 15
5.1 Verification of Virtual Met Mast™ Turbulence Intensity model .................. 15
5.2 Performance of Virtual Met Mast™ Turbulence Intensity model ............... 16
Annex A – VMM’s uncertainty: contributing factors .................................................. 18
Appendix B – Complexity index ................................................................................ 19
3
Executive summary
Virtual Met Mast™ is based on the UK Met Office’s operational Numerical Weather Prediction (NWP)
Model and has been designed to produce long term, site and height specific wind climatologies and
associated time series datasets for the onshore and offshore wind energy industry.
Virtual Met Mast™ is typically used to support site search and selection activities prior to the
installation of an on-site monitoring mast and often supports the positioning of these masts on larger
sites. Virtual Met Mast™ is also of benefit during site design development and construction, where the
output can be compared with traditional methods for assessing long term wind resources, thereby
acting as an effective due diligence process and solution.
For lower value feed in tariff (FiT) and other smaller projects, where the costs and timeframes
associated with the installation and collection of data using real masts are prohibitive, it can be used
as the primary source for delivering accurate, long term wind assessments. It will also be of real value
in countries, or specific areas of the world, where the existence of local wind data is either poor or
totally unavailable. Similarly, with offshore developments being undertaken further away from the
coastline, the ability of Virtual Met Mast™ to generate a site specific virtual reference dataset is critical
for establishing the long term wind climatology for these zones.
The inclusion of additional site climatological parameters such as modelled average turbulence
intensity at 15m/s average wind speed, average wind shear exponent, and 50-year return 3-second
gust wind speed, deliver an early indication of turbine class suitability according to standards IEC
61400-1.
A full description of Virtual Met Mast™ solution can found on the Met Office web site at
www.metoffice.gov.uk/renewables/vmm.
For the industry to use Virtual Met Mast™ as a practical decision support solution it is necessary to
define its performance over a range of sites, heights and complexity challenges presented to the
model. This report details the results of comparing and verifying Virtual Met Mast™ science at 89
sites, using up to a total of ~160 site-years of monitored data at a range of heights. Using this
extensive source data we are able to define the bias (model deviation from measurements) in the
mean wind speeds and also, critically, the confidence levels relating to the estimated long-term mean
speeds.
The degree of the downscaling challenge for Virtual Met Mast™ in going from the 4 km grid down to
the particular site in question is closely related to the complexity of the local topography but not
entirely so: larger topographic features some distance away can have an impact, and individual
nearby obstacles (such as large buildings, cliffs, trees or operating wind turbines) are not modelled.
4
th
Confidence in the estimated long-term mean wind speed at a site is expressed as the 90 percentile
(W90) of the distribution of long term mean bias values from sites of similar complexity. Each on-shore
location is assigned a complexity index derived from Virtual Met Mast™ diagnostics. These indices
have been grouped into three categories (Low, Medium and High Complexity) and the overall bias
statistics (mean and W90) calculated are shown in Table 1. All offshore sites, except those that are
near shore and influenced by coastal effects, present the same level of complexity. Note that these
figures are not comparable with statistics derived from the variability in 10-minute or hourly mean
winds, which are a separate and additional consideration.
Mean Bias
Model Complexity
(VMM - Obs)
90th Percentile (W90)
(m/s)
(m/s)
ALL SITES
0.1
0.7
Offshore
-0.1
0.1
Near-Shore
0.2
0.6
Low Complexity
0.1
0.4
Medium Complexity
0.2
0.7
High Complexity
0.0
1.0
Table 1: Bias statistics for the complexity categories defined.
These confidence statistics have been calculated from the Virtual Met Mast™ Verification Database of
189 anemometers at 89 sites. The criteria for the categorisation of sites’ model complexities and bias
statistics calculated for each are subject to refinement as further verification data become available.
This most recent update to Virtual Met Mast™ Verification report has not updated the results from the
retuning and verification of the modelling of Average Turbulence Intensity at 15m/s, TIav(15) provided
in the previous Verification report released in January 2013. This is due to the small additional number
of usable anemometers available since the previous Verification. The inclusion of 89 anemometers at
various heights at various sites where standard deviation of wind speed data were available has
delivered the removal of overall mean bias. The standard error of site specific TIav(15) biases is 0.024.
Based on the quality of results from this comprehensive verification process it may therefore be
concluded that Virtual Met Mast™ may be used with confidence to support larger and smaller projects,
both onshore and offshore. For further information please contact [email protected]
5
1. Introduction
This report describes the Virtual Met Mast™ verification process and presents the results of
comparing 162 site-years of data at 89 sites, covering a range of model complexity types and hub
heights. The process used provides objective and quantified statistics which have been selected as
appropriate descriptors within the Met Office and externally. The report defines the data sources
utilised, the analytical process followed, and a number of key derived statistics.
A description and the results of the most recent (January 2013) verification of Virtual Met Mast™
Turbulence Intensity model are included in Section 5.
2. Verification process
The verification process consists of three major steps as follows:1. Define data sources.
2. Define verification statistics to be produced by the analysis.
3. Group verification sites and calculate bias statistics.
2.1 Define Data Sources
Virtual Met Mast™ produces an hourly wind time series covering the period January 2001 to date. It is
the accuracy of the mean wind speed over this period that is the primary focus of this report.
Two sources of monitored data are used for comparison with Virtual Met Mast™ calculations:
1. Met Mast data collected on prospective sites at heights between 20m and 100m (often there
are anemometers at several heights on a single mast). The data are usually provided as ten
minute means, in which case the mean speed and direction in the ten minute period leading
up to the hour is compared with the wind calculations made on the hour by Virtual Met Mast™.
Careful consideration has been given regarding the inclusion and weighting of data from
observing periods of widely varying length. Various tests are applied to assess the quality of
the observed data, with dubious data removed.
2. Nacelle data collected from turbines in operation. Data are recorded at one height, but several
sources are often available in close proximity from turbines on the same site. The data are
sampled in the same way as Met Mast data and the same weighting and filtering criteria are
applied to data deemed to be of sufficient quality. However, data tend to be of poorer quality
than mast data, and when the number of sample sites has grown sufficiently these cases will
be filtered from the verification.
6
DATA SOURCE
Height
Advantages
(m)
Disadvantages
Quality
Prospective sites’
monitoring masts
25 – 100
Shorter
measurements,
monitoring
Multiple sources
periods
at one location
Turbine nacelle
anemometers
25 - 100
Close locations
Length of
Data freq.
data
means
Varying
widely
Poor
Less than
Data Quality
10 years
Epoch
10 min
2001 onwards
10 min
2005 onwards
Table 2: Summary of features for different data sources.
2.2 Define Verification Statistics
Where observations at more than one height on the same mast are available, the wind speed biases
are averaged for all heights above 20m. When there are several nacelle observations on the same
site, a subjective assessment on their inclusion is made based on spacing, topography and land use.
The following statistics have been selected to verify the performance of Virtual Met Mast™ when
compared with monitored data, and are based on long term mean wind speeds:
•
Biases - defined as the difference in wind speed between Virtual Met Mast™ and monitored
data, over the monitoring period. A bias value is produced for each hour of concomitant data
and the time average bias then calculated for the location, across all measuring heights over
20m. One overall bias figure is produced for each location.
•
Standard deviation of biases – gives the uncertainty of the bias values around the mean. In
this report the standard deviation of the biases in long term mean wind speed is calculated as:
σ=
•
Root Mean Squared error (RMS) – The RMS gives a measure of the combined systematic
error (bias) and random error (standard deviation).
RMS =
•
th
W90 – defined as the 90 percentile of the normally distributed long term mean bias values,
calculated as 1.28 times the standard deviation. Subtracting this figure from the Virtual Met
Mast™ long-term mean wind speed gives the speed that 90% of the sites of the same
population will exceed. See Annex A for more details about the various factors contributing to
VMM uncertainty.
7
When calculating the bias statistics, several weighting functions have been considered to account for
the uncertainty in using various lengths of monitoring periods. Note that the ultimate objective is to
assess the overall error in Virtual Met Mast™ hourly time series from January 2001 to date and not on
each individual hourly observation. The various weighting alternatives considered were:
1. Only use records over six months in length and apply the same weight to them.
2. Linear weighting as a function of record length.
3. Linear weighting as a function of record length up to one year, and same weight when longer.
4. A weighting function derived from the deviation analysis of different length records using the
verification datasets available.
At this time, Option 4 has been chosen (see Figure 1). This will be kept under review.
Figure 1: The weights applied to the bias values depend on the observational time length.
8
2.3 Grouping sites to calculate bias statistics
Bias statistics can be presented for all verification sites taken as a single population. This grouping
produces the most reliable statistics but could mask significant differences in the way Virtual Met
Mast™ is performing in different geographical scenarios.
However there may be various embedded bias populations in the full set, with the most logical split
being to separate offshore sites from those onshore. Offshore, where the ‘land use’ is constant and
with no topography influences, Virtual Met Mast™ should perform well. On-shore sites are assigned a
complexity index which depends on the downscaling challenge for Virtual Met Mast™ in going from
the 4 km grid down to the particular site in question. It is closely related to the complexity of the local
topography but not entirely so; larger topographic features some distance away can have an impact.
Sites are placed into Low, Medium and High complexity groups according to their Complexity Index
(A/S) and bias statistics calculated for each group. A map of the regions of differing complexity
categorisation is presented in Appendix B.
For each verification site the following information is gathered:
•
The bias
•
The Complexity Index
•
The record length of site monitoring – for weighting the bias values when calculating the bias
statistics of each group.
Linear weighted mean and standard deviation are calculated for each group of sites.
9
3. Verification results summary
Table 3 summarises the overall results. The standard error in the last column indicates the uncertainty
in the mean bias as a result of working with small sample sizes.
Number of
20m+ sites
Model
Complexity
N
ALL sites
Bias statistics (from 2001), for heights over 20m (m/s)
St. Error =
Mean
W90
89
0.1
0.7
0.01
Offshore
9
-0.1
0.1
0.01
Near-shore
17
0.2
0.6
0.03
Low Complexity
13
0.1
0.4
0.03
Medium Complexity
21
0.2
0.7
0.03
High Complexity
29
0.0
1.0
0.03
σ
N
Table 3: Derivation of bias mean and W90 for heights above 20m.
It should be noted that, as more samples become available or alternative model complexity related
sub-divisions are identified, some changes to these figures are to be expected.
The complexity categorisation will be kept under review.
Note:
th
The W90 statistics published here are the 90 percentiles of each complexity category’s standard
deviation of biases in long term mean wind speeds. These bias statistics are not comparable with
statistics derived from the variability in 10-minute or hourly mean winds, which are a separate and
additional consideration. Statistics based on a number of sites’ long term mean biases or speed-ups
do not belong to the same population as those based on spot, 10-minute, hourly or monthly biases or
speed-ups.
10
4. Other Verification Diagnostics
4.1 Comparison of VMM with NOABL
NOABL (National Objective Analysis of Boundary Layer) is a commonly used site screening tool,
comprising 1km square mean wind speeds at 10, 25 and 45m of height. The dataset is the result of an
1
air flow model that estimates the effect of topography on wind speed .
VMM performance has been assessed against NOABL by comparing their long-term wind speed
estimates against observations at 68 UK sites, where NOABL mean wind speeds were extrapolated to
observational heights using wind profile laws. For several of the stations observations at more than
one height were available. Data have been grouped by complexity, and by height into 4 15m wide
bins. As shown in Figures 2 and 3, and Tables 4 and 5, across all complexity and height categories,
VMM mean biases are lower than for NOABL. Uncertainties on the mean bias are similar.
Figure 2: VMM performance comparison against NOABL by complexity
Error bars represent the standard deviation of the biases.
1
http://decc.gov.uk/en/windspeed/default.aspx
11
Figure 3: VMM performance comparison against NOABL by height.
Error bars represent the standard deviation of the biases.
Mean Bias
(m/s)
Std Dev Bias
(m/s)
RMS
(m/s)
Model
Complexity
Number of
20m +
heights
VMM
NOABL
VMM
NOABL
VMM
NOABL
ALL
143
0.2
0.9
0.6
0.8
0.6
1.1
HC
48
0.0
1.4
0.8
0.8
0.7
1.5
MC
42
0.4
1.0
0.6
0.5
0.6
1.0
LC
23
0.2
0.3
0.4
0.5
0.4
0.6
NS
30
0.3
0.6
0.5
0.8
0.5
0.8
Table 4: Bias statistics by complexity for NOABL and VMM screening tools.
Mean Bias
(m/s)
Std Dev Bias
(m/s)
RMS
(m/s)
Model
Complexity
Number of
20m +
heights
VMM
NOABL
VMM
NOABL
VMM
NOABL
ALL
143
0.2
0.9
0.6
0.8
0.6
1.1
20-34m
40
0.1
1.2
0.7
0.8
0.6
1.2
35-49m
32
0.4
0.9
0.6
0.7
0.6
1.1
50-64m
50
0.2
0.8
0.6
0.8
0.5
1.0
65-79m
21
0.2
0.5
0.6
0.8
0.6
0.8
Table 5: Bias statistics by height for NOABL and VMM screening tools.
12
4.2 Diurnal variation
There is a tendency for the wind speed to increase during the afternoon and fall away again during the
evening for onshore sites. This arises from the increased turbulent energy in the atmosphere in the
afternoon due to solar heating. Often, on individual days, this effect is masked by synoptic weather
features but comes through as a feature in hourly wind climatology. Figure 3 gives a typical example,
for a low complexity site, of how Virtual Met Mast™ is able to capture the daily cycle. The correlation
between the calculated and observed wind speed values is good with a standard deviation of the
hourly biases less than 0.25 m/s.
Figure 3: Example of VMM modelling the diurnal cycle.
13
4.3 Height variation
Site data are available at a variety of heights above ground/sea level, enabling an assessment of
whether VMM has a propensity to perform differently at different heights. Reasonably, the expectation
might be that it performs better as height increases, as the impacts of local surface features are
reduced.
Figure 4 shows the mean bias and the standard deviation around the mean for the sites-heights
available. Bias values have been binned in 10m steps (left plot) and also all together (right plot). The
variability for each height bin is shown with error bars reflecting the standard deviation on the bias.
The coloured histograms within each height bin show the distribution of sites by the model complexity
for each height range considered – to be compared with the overall distribution (right plot). It can be
seen that above 20m there is no significant bias or trend.
With only the 50m bin approximating to the overall proportions of site types, these figures must be
treated with caution as indicative of there not being a height bias. More verification data at locations
with multiple observing heights are required.
Figure 4: Monthly biases and their standard deviations about the mean and categorised by mast height.
14
5 Virtual Met Mast™ Turbulence Intensity model
Full Virtual Met Mast™ reports deliver an estimate for omni-directional average Turbulence Intensity in
the 1 m/s wide, 15 m/s wind speed bin, for indicative assessments of wind turbine class suitability in
accordance with standards IEC61400-1. These estimates are based on a parameterised model, with
input variables taken from site-specific location and height, orographic and surface roughness
parameters, and Virtual Met Mast™ wind speeds.
While the parameterisation delivers an estimate of average Turbulence Intensity, TIav(15), the nature
of hourly VMM spot wind speeds does not allow for the modelling of its variability; represented by the
standard deviation of Turbulence Intensity readings in the 15 m/s wind speed bin, TIsd(15). Therefore
calculations of Characteristic Turbulence Intensity, CTI(15) [IEC61400-1 Edition 2] and Representative
Turbulence Intensity, RTI(15) [IEC61400-1 Edition 3] are not available.
However, using the approximation that TIsd(15) = 20% TIav(15), the Edition 2 turbulence classification
categories are presented in terms of TIav(15) only:
Turbulence Class
TIav(15) limit
S
> 0.15
A
0.15
B
< 0.133
5.1 Verification of Virtual Met Mast™ Turbulence Intensity model
The parameterisation of Turbulence Intensity within Virtual Met Mast™ has been adjusted to give the
best possible estimate of on-site average turbulence. These estimates have been verified against
observations by 89 individually calibrated cup anemometers where standard deviation of wind speed
data were available, from 20m to 81m height above ground, at 35 sites of varying orographic
ruggedness and land use roughness.
Six of the 89 anemometers were NRG Max#40 instruments and the remainder were either Vector
A100LK/LM or WindSensor P2546A instruments. The averaging period was 10 minutes in all cases
and, where reported, the logger sampling rate was 1 Hz, except one site where it sampled at 0.5 Hz,
and one where it sampled at 3 Hz. These datasets have been included in the verification without
adjustment.
15
Figure 5: Number of anemometers having a given number of 10 minute Turbulence Intensity samples at 15 m/s
In all cases where an anemometer had recorded any samples in the 15 m/s average wind speed bin,
the anemometer was included in the verification.
There was a small but significant number of
anemometers (particularly at low heights above ground, at low wind speed sites, from short monitoring
period datasets) where there were few records from which to generate statistically reliable values for
observed TIav(15). While some smoothing according to results in adjacent wind speed bins could have
been performed, these datasets have been included in the verification without adjustment.
It should be noted also that Virtual Met Mast™ model does not attempt to model the turbulence
generating effects of individual site-specific obstacles in close proximity to the measurement positions.
5.2 Performance of Virtual Met Mast™ Turbulence Intensity model
Figures 6 and 7 present the results of the verification of Virtual Met Mast™ Turbulence Intensity
model. The mean TIav bias from the 89 tests is -0.0012. The standard deviation of TIav biases is
th
0.024, giving a 90 percentile confidence,T90, of 0.031.
Of the 20 cases where Observed data indicated suitability of a Class A wind turbine, the VMM
estimated correctly in 18 cases and was very close with the other two. There was more variability in
the performance of the model where Observations indicated suitability of a Class S (only) or a Class B
turbine.
The model will be refined in publicised VMM version updates as more observational data become
available.
16
Figure 6: VMM vs Observed TIav(15) comparisons at various heights (≥20m), at 35 sites, with IEC61400-1
Turbulence class bandings
Figure 7: TIav(15) Bias (VMM – Obs) distribution from 89 tests
17
Annex A – VMM’s uncertainty: contributing factors
The uncertainty on the VMM’s long-term wind speed has various contributors which are described
2
here. The standard statistical approach for combining independent errors is to add the variances (σ )
of the contributing components that are subject to error or variability. The components impacting on
Virtual Virtual Met Mast™ are described here.
σ2VMM = σ24km + σ2Clim + σ2Ann
Where:
•
2
4km
σ
is the variance of the bias over the period 2001 to date. This term reflects the uncertainty
stemming from the downscaling process needed to interpolate data from the 4 km grid down
to a particular site.
•
2
Clim
σ
is the variance between climatologies back to 1989 and even longer climatologies, some
extending back over 50 years. Of course this component does not apply where only the
climatology of the last 23 years (1989 onwards) is considered to be relevant. It is not currently
included in the VMM uncertainty calculation.
•
2
Ann
σ
is an additional variance that needs to be included in order to present the result as an
annual mean wind speed, which is as favoured by the wind engineering industry. This is
normally represented by a 6% error but it is not currently included.
th
These variances are transformed into the W90 statistic. W90 = 1.28 x σVMM is the 90 percentile of the
bias distribution, assumed to be normal. It makes possible the calculation of the long term wind speed
that there is 90% probability of exceeding at a site. σVMM is defined separately for different site
groupings, where there is evidence that the grouping samples a distinct population of bias errors.
Note:
The W90 is a measure of the confidence in the long term mean wind speed estimated by the VMM, on
the basis of its accuracy in verifications at sites of similar complexity. This should not be confused
with the uncertainty term for the inter-annual variability of winds accounted for in the production of
energy yield estimates’ P90 figures; which is an entirely separate and subsequent assessment.
18
Appendix B – Complexity Index
Figure 5: A/S Complexity Index values within the regional model domain
The silhouette area per unit horizontal area (A/S) is a parameter used to quantify the complexity of a
site. It is a measure of the upslopes encountered divided by the length of the cross section of the
orography.
19
Met Office
FitzRoy Road, Exeter
Devon EX1 3PB
United Kingdom
Tel (UK): 0870 900 0100 (Int) : +44 1392 885680
Fax (UK): 0870 900 5050 (Int) :+44 1392 885681
[email protected]
www.metoffice.gov.uk