ECMWF

From Eecology

Jump to: navigation, search

Contents

ECMWF

Datamodel for the tables in the ECMWF schema.

Data is created by the ECMWF and retrieved from the KNMI ftp site as GRIB files. The original GRIB files can also be retrieved from the Flysafe server from :

ECMWF Data Archive

More information about the parameters in these GRIB files can be retrieved from the ECMWF website.

INVARIABLE

The INVARIABLE table contains data that is invariant over time.

LATITUDE degrees * 100 to make this a integer
LONGITUDE degrees * 100 to make this a integer
Z 129 Z Geopotential surface/model orography
LSM 172 LSM Land sea mask
LOCATION Latitude/longitude as GIS location


The GIS LOCATION is only added to this table. This is because always the same values for latitude and longitude are used and then if the GIS location is required for the ANALYSIS or FORECAST tables it can be retrieved by joining to this table on the LATITUDE, LONGUTUDE fields.

FORECAST and UVA_FORECAST

The FORECAST table contains predicted values in steps of 3 hours till 72 hours ahead. The MODELTS is the timestamp for which this prediction applies. Two times a day new Forecast GRIB files will arrive. The newer FORECAST data that arrives for the same timestamp will overwrite the older data.

The FORECAST (and UVA_FORECAST) table itself is a master table that does not contain any data. The inherited tables FORECAST<YEAR><MONTH> contain the data, but the master FORECAST or UVA_FORECAST table can be used to SELECT the data.

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the FORECAST_UVA table. The FORECAST table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.

MODELTSTime for which this forecast applies = model run + step hours
LATITUDEdegrees * 100 to make this a integer
LONGITUDEdegrees * 100 to make this a integer
STEPmodel step (smaller is more recent forecast)
SSRD169 SSRD surface solar radiation downwards accumulated
TP228 TP total precipitation accumulated
SP134 SP surface pressure
BLH159 BLH boundary layer height
TCC164 TCC total cloud coverage
U10165 10U 10 meters wind U-component m/s
V10166 10V 10 meters wind V-component m/s
T2167 2T 2 metres air temperature
D2168 2D 2 metres dew point temperature
LCC186 LCC low cloud coverage
Pressure level 1000
T_1000130 T temperature
U_1000131 U U-wind component
V_1000132 V V-wind component
GH_1000156 GH geopotential height
R_1000157 R relative humidity
Pressure level 925
T_925 real130 T temperature
U_925 real131 U U-wind component
V_925 real132 V V-wind component
GH_925 real156 GH geopotential height
R_925 real157 R relative humidity
Pressure level 850
T_850130 T temperature
U_850 real131 U U-wind component
V_850 real132 V V-wind component
GH_850 real156 GH geopotential height
R_850 real157 R relative humidity
Pressure level 700
T_700 real130 T temperature
U_700 real131 U U-wind component
V_700 real132 V V-wind component
GH_700 real156 GH geopotential height
R_700 real157 R relative humidity

ANALYSIS and UVA_ANALYSIS

The ANALYSIS and UVA_ANALYSIS table contains values from the Analysis GRIB files at 18, 00, 06 and 12 UTC

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the ANALYSIS_UVA table. The ANALYSIS table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.


MODELTSTime for which this forecast applies = model run
LATITUDEdegrees * 100 to make this a integer
LONGITUDEdegrees * 100 to make this a integer
SP134 SP surface pressure
TCC164 TCC total cloud coverage
U10165 10U 10 meters wind U-component
V10166 10V 10 meters wind V-component
T2167 2T 2 metres air temperature
D2168 2D 2 metres dew point temperature
LCC186 LCC low cloud coverage
Pressure level 1000
T_1000130 T temperature
U_1000131 U U-wind component
V_1000132 V V-wind component
GH_1000156 GH geopotential height
R_1000157 R relative humidity
Pressure level 925
T_925130 T temperature
U_925131 U U-wind component
V_925132 V V-wind component
GH_925156 GH geopotential height
R_925157 R relative humidity
Pressure level 850
T_850130 T temperature
U_850131 U U-wind component
V_850132 V V-wind component
GH_850156 GH geopotential height
R_850157 R relative humidity
Pressure level 700
T_700130 T temperature
U_700131 U U-wind component
V_700132 V V-wind component
GH_700156 GH geopotential height
R_700157 R relative humidity

COMBINED

If you want to use Analysis data if available for a timestimp but otherwise Forecast, the COMBINED VIEW can be used. If a ANALYSIS value is available for a timestamp and latitude/longitude the ANALYSIS value will be used, otherwise the FORECAST value is given.

Historical data

In addition ongoing process of adding ECMWF data since November 11 2007, a historical data set has been loaded. That historical dataset is stored in additional inherited tables FORECASTHISTORY and ANALYSISHISTORY which inherit from FORECAST and ANALYSIS respectively.

The FORECASTHISTORY does not always contain TP (228 TP total precipitation accumulated). Instead, it contains the following values:

LSP142 LSP Large scale precipitation
CP143 CP Convective precipitation

The historical dat sets span the following periods and locations

StartEndAreaGrid
2007-01-012007-11-1562degN 10degW 335degN 20degE0.25x0.25deg
2003-02-152003-06-1562degN 10degW 335degN 20degE0.5x0.5deg
2005-01-012006-12-3162degN 10degW 335degN 20degE0.5x0.5deg

Helpful Functions & Queries

The following functions have been defined in Postgres to select the nearest gridpoint in spacetime for the ECMWF analysis and forecast tables.


These are ecmwf.forecast_nearest and ecmwf.analysis_nearest and they take a latitude, longitude and timestamp as parameter.


They can be used in the following way, which will return all the fields in the relevant ECMWF data:

 SELECT * FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');
  
 SELECT * FROM ecmwf.analysis_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

Note that the model latitude and model longitude are returned as integers and multiples of 100. To return latititude and longitude as a decimal (for example to compare to the latitude and longitude of your GPS data) you need to first convert these fields to decimals:

 SELECT f.latitude/100::decimal, f.longitude/100::decimal
 FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00') f;

These function can for example be used in a join as follows :

 SELECT g.device_info_serial
      , g.date_time
      , g.latitude
      , g.longitude
      , g.speed_2d
      , g.altitude
      , (ecmwf.forecast_nearest( g.latitude, g.longitude,g.date_time)).*
 FROM gps.ee_tracking_speed_limited g
 WHERE g.device_info_serial = 1 
   AND g.latitude <> 0
   AND g.userflag = 0
 ORDER BY g.date_time;


Although these functions were converted to PLPgSQL to speed them up, a faster ways to retrieve the same data is with (thanks to Floris Sluiters):

 SELECT device_info_serial, date_time, latitude, longitude, speed_2d, altitude, (x::ecmwf.forecast).*
 FROM (SELECT 
      g.*
      ,((SELECT f 
           FROM ecmwf.forecast f
          WHERE f.latitude = round(g.latitude * 100 / 25)::integer * 25
            AND f.longitude =  round(g.longitude * 100 / 25)::integer * 25
            AND f.modelts = TIMESTAMP 'epoch' + round(EXTRACT(EPOCH FROM g.date_time) 
                / (60*60*3))::integer * INTERVAL '3 hours' limit 1)::ecmwf.forecast) as x 
       FROM   gps.ee_tracking_speed_limited g
       WHERE g.device_info_serial = 1 
            AND g.latitude <> 0
            AND g.userflag = 0
         ORDER BY g.date_time) t

Below is example query that retrieves data for a model grid point closest to user provided coordinates and for one model time stamp (eg. 12:00) for a series of consecutive days

 SELECT f.modelts, f.latitude, f.longitude, f.blh
 FROM ecmwf.forecast as f
 WHERE f.latitude = round(42.718104*100/25)::integer*25 AND f.longitude = round(11.517727*100/25)::integer*25 and 
   f.modelts >= '2007-08-01 12:00' and f.modelts <= '2008-08-01 12:00' and date_part('hour', f.modelts) = 12
 ORDER BY f.modelts   


Interpolation

The following functions are added so that simple linear interpolation for a specific point in space and time can be done in Postgres.

These are ecmwf.forecast_interpolate and ecmwf.analysis_interpolate and they take a latitude, longitude and timestamp as parameter.

They can be used in the same way:

 select * from ecmwf.forecast_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');
  
 select * from ecmwf.analysis_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

And they can also be used in joins as above. These functions select the 8 surrounding points in latitude, longitude and date_time and interpolate linear in latitude, longitude and date_time successively. The interpolate functions are much slower then the nearest functions.

Personal tools