UvA-BiTS Tracking Data

From Eecology

(Redirected from DB Views 2015)
Jump to: navigation, search

Contents

UvA-BiTS Tracking Data

Note: These views will show no rows for trackers not yet attached to birds.

gps.ee_tracking_speed_limited

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has not changed with respect to our previous database.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serialint4NOSerial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
date_timetimestampNOTime when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
latitudefloat8YESLatitude of the GPS fix, decimal degrees.
longitudefloat8YESLongitude of the GPS fix, decimal degrees.
altitudeint4YESAltitude above sea level measured by GPS tag in meters
pressureint4YESPressure measured by GPS tag sensor in Pascals
temperaturefloat8YESTemperature measured by GPS tag sensor in Celsius
h_accuracyfloat8YESHorizontal accuracy (meters)
v_accuracyfloat8YESvertical accuracy (meters)
x_speedfloat8YESmeters/second
y_speedfloat8YESmeters/second
z_speedfloat8YESmeters/second
gps_fixtimefloat8YESGPS fix time (also known as Time to Fix) in seconds
userflagint4YESAcceptable = 0; Data flagged as unacceptable by user if not equal to 0.
satellites_usedint2YESNumber of satellites used for fix the GPS measurement
positiondopfloat8YESPositional DOP (Dilution of Precision), unitless
speed_accuracyfloat8YES
locationgeometryYESpostgreSQL geometry; to allow GIS operations
vnorthfloat8YES- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
veastfloat8YES- vx * sin(lon) + vy * cos(lon), m/s
vdownfloat8YES- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
speedfloat8YES- vx * sin(lon) + vy * cos(lon)
speed_3dfloat8YES sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
speed_2dfloat8YES Speed on 2D, m/s
directionnumericYES Heading in degrees; 0 is north, 90 is east
altitude_aglfloat8YES Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: Elevation

gps.ee_acceleration_limited

Acceleration is measured in blocks of 1-200 measurements (records). The date_time mentioned in this block is the date and time of the first acceleration measurement. Sequential measurements within a block are numbered with an index 1... (max 200). Every row represents one measurement in 3 directions (X,Y,Z). The order is important within each block, so you should always sort the acceleration measurements by block and index. If you need the exact time for each row, you can calculate it from the index, the sample frequency (which is almost always 20 Hz) and the date_time of the first row in the block.

The date_time of the first block is the time of the GPS measurement preceding the accelerometer measurement. If there is no valid GPS measurement, this time is calculated from the last GPS measurement and the time of the internal clock of the tracker.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time in gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker.


This table structure has not changed with respect to our previous database.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serialint4NOSerial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
date_timetimestampNOTime of the acceleration block that this measurement belongs to (time in UTC, no time zone).
indexint2NOSorting order of the measurement within the acceleration block (each block starts at 0).
x_accelerationint2YESAcceleration measured on the x (or surge) axis (range between -2666 and 2666).
y_accelerationint2YESAcceleration measured on the y (or sway) axis (range between -2666 and 2666).
z_accelerationint2YESAcceleration measured on the z (or heave) axis (range between -2666 and 2666).

gps.ee_acc_start_limited

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

If there is no valid GPS measurement, this time is calculated from the last GPS measurement and the time of the internal clock of the tracker.

To know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.


Added in firmware 1.0.2.7.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serialint4NOSerial number of the tracker that reported the acceleration measurement.
date_timetimestampNOTime to mark the start of the acceleration block.
line_counterint4NONumber of lines associated with this set of accelerations.
timesyncedint2YESThe source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
asciiint4YESAcceleration measurement interval.
accsnint4YESNumber of samples (0-65536).
fint2YESAcceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).

Bird data

gps.ee_individual_limited

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
ring_numbervarcharNORing number.
species_latin_namevarcharNOLatin name of the species of the bird.
colour_ringvarcharYESString for ring colour.
sexvarcharNOSex (M=male, F=female, X=unknown/unspecified)
massnumeric(5,0)YESMass (grams)
start_datetimestampNODate when the bird was first tracked with an UvA-BiTS logger.
end_datetimestampNODate at which the bird is no longer relevant for the tracking project.
remarkstextYESText for user remarks
individual_idint8NOBird unique id. Included just to enable joins.

gps.ee_species_limited

Each row represents a species that you can assign to your birds to include them in your project. No bird in the system can exist without being linked to one of the species from this list. If you are tagging a bird whose species does not exist in the list, please contact us.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
abbrvarcharNOUnique abbreviation for this species.
english_namevarcharYESName of the species in English.
latin_namevarcharNOName of the species in Latin. It is always filled in.
species_idint8NOUnique id. Added to allow joins.

gps.ee_nest_limited

This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
reference_namevarcharNOA string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
latitudenumericYESLatitude of the coordinates where the nest was found, decimal degrees.
longitudenumericYESLongitude of the coordinates where the nest was found, decimal degrees.
start_date_timetimestampNOWhen the nest was spotted.
end_date_timetimestampNOWhen the nest ceases to be relevant.
found_by_whomvarcharYESPlace to credit people involved in finding the nest.
remarkstextYESFree text.
locationbytearrayYESCalculated Geometry of the coordinates where the nest was found.
nest_idint8NOUnique id. Added to allow joins to nest_inhabitant_limited.

gps.ee_nest_inhabitant_limited

This view shows who lives on which nest, for birds you have access to.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
reference_namevarcharNOA string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
ring_numbervarcharNOThe ring number of a bird on a nest.
device_info_serialint8NOFollowing the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
key_namevarcharNOFollowing the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
nest_idint8NOReference to the nest id. Added to allow joins.
individual_idint8NOReference to the individual id. Added to allow joins.
track_session_idint8NOReference to a resolved track session id. Added to allow joins.

Tracker and project data

gps.ee_project_limited

Each row shows the details of each project, for the projects you are allowed to see.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
key_namevarcharNOUnique key name, with the form <species>_<place>
station_namevarcharNOWhere the receiving station is
start_datetimestampNOProject start date
end_datetimestampNODate at which the project is not relevant any more
descriptiontextYESString for free description. Updatable by the user.
project_idint8NOProject unique id. Included just to enable joins.
parent_idint8YESReference to the parent project in the hierarchy.

gps.ee_tracker_limited

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serialint8NOSerial number of the tracker
firmware_versionvarcharYESString for firmware version
massnumeric(4,2)YESMass (in grams)
start_datetimestampNODate before which the device is not relevant
end_datetimestampNODate at which the device is not relevant any more
x_onumeric(30,6)YESAccelerometer calibration: offset on the x (or surge) axis
x_snumeric(30,6)YESAccelerometer calibration: sensitivity on the x (or surge) axis
y_onumeric(30,6)YESAccelerometer calibration: offset on the y (or sway) axis
y_snumeric(30,6)YESAccelerometer calibration: sensitivity on the y (or sway) axis
z_onumeric(30,6)YESAccelerometer calibration: offset on the z (or heave) axis
z_snumeric(30,6)YESAccelerometer calibration: sensitivity on the y (or heave) axis
tracker_idint8NOTracker unique id. Included just to enable joins.

gps.ee_track_session_limited

A track session links a tracker and a bird for a given period of time. Therefore, it is mandatory for projects to supply this piece of metadata before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
key_namevarcharNOThe project that the track session belongs to.
device_info_serialint8NOSerial number of the tracker this track session involves.
ring_numbervarcharNORing number of the bird that this track session involves.
start_datetimestampNODate before which the track session is not relevant
end_datetimestampNODate at which the track session is not relevant any more
remarkstextYESFree text for comments
start_latitudenumeric(11,8)NOLatitude where the bird was tagged, decimal degrees
start_longitudenumeric(11,8)NOLongitude where the bird was tagged, decimal degrees
project_idint8NOReference to the project this track session belongs to. Included just to enable joins.
tracker_idint8NOReference to the tracker this track session involves. Included just to enable joins.
individual_idint8NOReference to the individual this track session involves. Included just to enable joins.
track_session_idint8NOUnique identifier of the track session. Included just to enable joins.

Relation to projects

gps.ee_user_limited

Each row shows a colleague in a project that you belong to, indicating her role and status there.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
key_namevarcharNOProject name
emailvarcharNOe-mail address
login_namevarcharNOLogin name
namevarcharNO(first) name
surnamevarcharNOSurname
rolevarcharNORole in the project
statusvarcharNOStatus of the membership
user_idint8NOUnique identifier of the user. Included to enable joins.

gps.ee_tracker_ownership_limited

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
key_namevarcharNOThe project that the tracker belongs to.
device_info_serialint8NOSerial number of the tracker.
project_idint8NOReference to the project. Included just to enable joins.
tracker_idint8NOReference to the tracker. Included just to enable joins.

gps.ee_individual_ownership_limited

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
key_namevarcharNOThe project that the individual belongs to.
ring_numbervarcharNORing number of the bird.
project_idint8NOReference to the project. Included just to enable joins.
individual_idint8NOReference to the individual. Included just to enable joins.

Engineering data

Note: These views will show no rows for trackers not yet attached to birds.

gps.ee_comm_limited

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serial int4 NO Serial number of the tracker.
date_time timestamp NO timestamp of start communication
date_time_end timestamp YES timestamp of end communication
date_time_utc timestamp YES date_time in utc (if available)
date_time_end_utc timestamp YES date_time_end in utc (if available)
action textYES JOINED / RESCHEDULE  ??
bytes_communicated int4 YES Number of bytes communicated (total of all bytes per DATA line)
status text YES READY, TIMEOUT, LOWPOWER, MISSING
network_join_time int4YESmilliseconds
number_identical_lines int4 YES Number identical lines in data block. Gives indication
non_ascii_char int4 YES Number of non ascii characters (indicated by question-mark ?)
sts_time timestamp YES derived from S0 hh,mi
timesynced int2YES TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
devicemode int2YESDeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
vbat float8 YES Battery voltage in V (0-~4.200V)
logb int4 YES logged bytes, waiting for download (0-4Mbyte)
logp int2 YES Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
gpsii int4 YES GpsIval, current (0-65535)
comii int4YES ComIval, current (0-65535)
accii int4YES AcceleroIval, current (0-65535)
swddd int4YES User switches, current (0-65535)
last_gps_fix timestampYEStimestamp of last fix
latitudefloat8YESLatitude of last fix. Decimal degrees
longitudefloat8YESLongitude of last fix. Decimal degrees

gps.ee_energy_limited

Only for newer firmware.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serialint4NOSerial number of the tracker
date_timetimestampNODate and time of the reported data
timesynced int2YESTimeSynced (0=not synced, 1=user set, 2=Gps synced)
devicemode int2YES(0-99)
vsll float8YESSolar Voltage during charging in mV (0-~2500)
vbat float8YESBattery Voltage in V (0-~4.200)
ssw int2YES Solar Cell Switches ... (0-255) fw1022 and up
YESextended fields if special user switch is on
vsoo float8YES Solar voltage open in V (0-~2.500)
vdrf float8YESreference voltage Dac in V (0-~2.435)
vchr float8YEScharge current voltage
g int2YES Gps battery threshold OK
c int2YESCom battery threshold OK
m int2YESCharge Mode (0,1,2 Com battery threshold OK
pressure int4YES in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
temperature float8YES in celcius degrees

gps.ee_settings_limited

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines. If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added. If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serialint4NO
date_timetimestampNODerived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
timesynced int2 YES
timezone_time int2 YES
swddd int4YES SpecialUserSwitches
gpsnavmode int2YES GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
gpsreceivermode int2 YES GpsReceiverNode 0-3
gpsfixtime int2YES GpsFixTime 0-255
gps_ivaln_on_h int2YES hours Gps IvalN on
gps_ivaln_on_m int2YES minutes Gps IvalN on
gps_ivaln int4YES GpsIvalN (0-65530)
gps_ivale_on_h int2YES hours Gps IvalE on
gps_ivale_on_m int2YES minutes GpS IvalE on
gps_ivale int4YES GpsIvalE
com_ivaln_on_h int2YES hours Com IvalN on
com_ivaln_on_m int2YES minutes Com IvalN on
com_ivaln int4YES ComIvalN (0-65530)
com_ivale_on_h int2YES hours Com IvalE on
com_ivale_on_m int2YES minutes Com IvalE on
com_ivale int4YES ComIvalE
accii int4YES Accelero measurement interval
accbn int4YES Accelero number of blocks
accsn int4YES Accelero number of samples (0-65536)
f int2YESAccelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
pdp int2YES PdopMsk (3-250)
pacint2YES PaccMsk (10-100)
tdp int2YES TdopMsk (3-250)
tac int2YES TaccMsk (30-300)
ga int2YES Additional Fixes GpsIval mode (3-25) before interval fix is logged
aol int2YESAccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
eventnum textYES 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text

gps.ee_trackingfile_limited

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
idint8NOUnique identifier of the row
date_reporteddateNODate that the file arrived
filenamevarcharNOName of the file
last_modified_datetimestampNODate that the file was last modified on
device_info_serialint4YESSerial number of the tracker this file comes from
filesizeint8NOSize of the file in KB
parsed_datetimestampYESDate and time that the file was last attempted to be parsed on
versionint4NOOptimistic locking field. Ignore it as a user.
last_found_datetimestampYESLast time that a process that runs to look for new files saw the file
n_linesint4YESCount of lines found in the file last time it was processed
n_data_linesint4YESCount of actual data lines, out of the total lines
n_tracking_recordsint4YESCount of GPS fixes found during the parsing process
n_communication_recordsint4YESCount of communication records found during the parsing process
n_energy_recordsint4YESCount of energy records found during the parsing process
n_acceleration_recordsint4YESCount of acceleration records found during the parsing process
n_acceleration_start_recordsint4YESCount of acceleration start records found during the parsing process
n_errorsint4YESCount of errors encountered during the parsing process
n_settings_recordsint4YESCount of settings records found during the parsing process
filepathtextNODirectory path of the file on the server

gps.ee_warning_limited

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

 select * from gps.ee_warning_limited
 where warning = 'CRC8 failure'
 order by device_info_serial;
COLUMN_NAME TYPE_NAME IS_NULLABLE DESCRIPTION
device_info_serialint4NOSerial number of the tracker
filenametextNOFile name
linenumberintegerYESline number in file
warningtextYESThe warning message
linetextYESThe actual line contents in the file

Sharings

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

  • Tracking data
    • gps.ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
    • gps.ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
    • gps.ee_shared_acceleration_limited: acceleration information for trackers shared
  • Metadata
    • gps.ee_shared_tracker_limited: information about each shared tracker
    • gps.ee_shared_individual_limited: information about birds with shared trackers
    • gps.ee_shared_track_session_limited: information about track sessions which involve shared trackers
  • Relation to projects
    • gps.ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

Accelerometer calibration

The accelerometer measures the acceleration in 3D (20 times per second). Acceleration is caused by gravity (static component) and by a change in speed (dynamic component). When the logger is static in horizontal position (solar cells up), gravity (green arrow) shows a heave (vertical acceleration, z) of +1 g (1 g = 9.81 m.s-2) while surge and heave are both 0. When the front of the logger is vertically pointing down (forward acceleration, x), surge is +1g and when it is rolled to the right (sidewards acceleration, y) sway is +1g. Due to gravity, the pitch of a logger is thus translated into the values of heave, surge and sway (or z, x, y).

However, the values are measured in mV instead of g. Before delivering the GPS-loggers, they have been calibrated at UvA. The acceleration (A) expressed in g is calculated from the measured value (M) and the calibration factors offset (O) and sensitivity (S).

Thus acceleration for heave (z), surge (x) and sway (y) is calculated as: Az = (Mz-Oz)/Sz ; Ax = (Mx-Ox)/Sx; Ay = (My-Oy)/Sy ;

The values of O (in general ranging from 0-60, sometimes even 300) and S (ranging from 1300-1450) are available in the device table of the data base. If not available, use the factory defaults O = 0 and S = 1365.

Accelero1.pngAccelero2.png|

In the figure above heave (z) is shown in green, surge(x) in red and sway (y) in blue, all values are expressed in g (on the y-axis).

Data processing and filtering

The raw log files are processed before being entered into the database. Each firmware version has a dedicated parser for the task. In certain cases data that is found in the log files is not stored in the database. This can occur for two reasons:

1. The data did not pass the CRC8 checksum (cyclic redundancy check). This is a standard procedure designed to detect accidental changes to raw computer data.

2. If the data has an invalid signature. For example, if a certain data format (eg numeric) or number of values (eg 4 digits) is expected but not returned, this record is not stored in the database.

When a line is deleted, the log file name, line number and the type of error is stored in the table gps.uva_warning102

Information about GPS receivers

The first series of tags used the UBlox LEAS4S chip (~up to tag 541), newer tags use UBlox LEAS6S. The series developed in 2012 will use the Max6G receivers. From 2012 receiver versions will be stored in the database. The UBX protocol is used for programming firmware, packaging and processing data. For more information about the receivers themselves see the ublox documentation. Ublox 4 Protocol specification [1] Ublox 6 Protocol specification [2]

Personal tools