Manual to vtd_apriori_update.py

L. Petrov

Abstract:

Table of contents:

1   Scope

2   Program vtd_apriori_update.py

2.1   What if vtd_apriori_update.py is interrupted

3   vtd_apriori_update.py control file

3.1   vtd_apriori_update.py control file
3.2   SPD_URL:
3.3   BDS_URL:
3.4   IONO_URL:
3.5   LOG_FILE:
3.6   LOG_VERBOSE_FILE:
3.7   VERBOSITY_LEVEL:
3.8   EOP_FILE:
3.9   IONO_DIR:
3.10   LOAD_BDS_DIR:
3.11   SPD_BIN_DIR:
3.12   EOP_SERIES_UPDATE:
3.13   IONO_UPDATE:
3.14   SPD_UPDATE:
3.15   LOAD_UPDATE:
3.16   MERRA2_GEOSFPIT_ATM:
3.17   MERRA2_GEOSFPIT_LWS:

4   Example

4.1   Example of a vtd_apriori_update.py control file
4.2   Example of a corresponding VTD control file

1   Scope

The following a priori values are derived from observations and need be updated on a regular basis: 1) Earth orientation parameters. The source of these data is the Network Earth rotation service. 2) Ionosphere total electron contents maps. The source of these data is the GNSS. 3) Expansion coefficients for slant path delays for VLBI stations. The source of these data is the path delay service. 4) Time series of crustal deformation excreted my mass loading. The source of these data is the International Mass Loading Service.

2   Program vtd_apriori_update.py

Program vtd_apriori_update.py is a part of VTD package. It is written in Python3 language. Usage: vtd_apriori_update.py -c configuration_file -v verbosity level where, configuration_file -- name of the configuration file verbosity_level -- 0 -- silent, 1 -- normal verbosity, 2 -- debugging level of verbosity vtd_apriori_update.py checks whether the local host has a priori files in the directories specified in the control file. If it does not have, vtd_apriori_update.py downloads the apriori from remote hosts specified in the control file. If it has, then vtd_apriori_update.py compares the local files with those at remote hosts. If it finds that the EOP at remote hosts were updated and are newer than the local copies, it initiates the process of updating: it downloads the new portion of a prioris. NB: EOP files is always updated from the remote host. It is recommended to run vtd_apriori_update.py regularly, for instance using cron job. In total, 60--80 Gb space is needed for all a priori data.

2.1   What if vtd_apriori_update.py is interrupted

vtd_apriori_update.py downloads data from remote hosts in temporary files, and then renames them. Depending on speed of the Internet connection download may be long, however renaming is done very quick. If vtd_apriori_update.py was interrupted, temporary files in the output directories may remain. They need to be removed manually. In rare cases when interrupt results in truncation, entire directories can be cleaned. This will force vtd_apriori_update.py to download full copies of apriori files.

3   vtd_apriori_update.py control file

vtd_apriori_update.py control files consists of the lines of variable lines in ascii coding. The first line identifies the format and version of the control file. The parser checks whether the version of the control file is supported by vtd_apriori_update.py. Lines, others than the 1st, that starts with character # are considered as comments and ignored by the parser. Control file supports environment variable in format ${ENV_VAR}, i.e. enclosed in curly parentheses that start with $. Environment variables may be nested. vtd_apriori_update.py control files consists of definitions in the form of keyword: value separated by one or more either blanks or tabulations. Columns should terminate the keyword.

3.1   vtd_apriori_update.py control file

3.2   SPD_URL:

SPD_URL: url_with_slant_path_delay This keyword specifies the URL with the expansion coefficients of slant path delay files in binary bspd format. At the moment, the following URLs provide slant path delays: http://pathdelay.net/spd/bin http://alt.pathdelay.net/spd/bin URL http://pathdelay.gsfc.nasa.gov/spd/bin is planned to be accessible in 2019.

3.3   BDS_URL:

BDS_URL: url_with_mass_loading This keyword specifies the URL with the time series of 3D site position variations caused by mass loading for a number of stations in in binary bds format generated by the International Mass Loading Service. At the moment, the following URLs provide time series of mass loading in bds format: http://massloading.net/imsl/load_bds http://alt.massloading.net/imsl/load_bds URL http://massloading.gsfc.nasa.gov/spd/bin is planned to be accessible in 2019.

3.4   IONO_URL:

IONO_URL: url_with_tec_maps This keyword specifies the URL with the time series of total electron contents (TEC) maps produced from analysis of GNSS data. At the moment, vtd_apriori_update.py supports TEC maps from CODE institution only: http://ftp.aiub.unibe.ch/CODE

3.5   LOG_FILE:

LOG_FILE: short_log_file This keyword specifies the name of the log file with brief information about outcome of apriori updates.

3.6   LOG_VERBOSE_FILE:

LOG_VERBOSE_FILE: long_log_file This keyword specifies the name of the log file with verbose information about outcome of apriori updates. It is needed mainly for tracking problems.

3.7   VERBOSITY_LEVEL:

VERBOSITY_LEVEL: verbosity_level_of_long_log_file This keyword specifies verbosity level of the long log file. Normal verbosity level is 1. Higher level of verbosity provide additional information for problem tracking.

3.8   EOP_FILE:

EOP_FILE: name_of_the_eop_file This keyword specifies the name of the Earth Orientation Parameter file. This file should be writable by the user who runs vtd_apriori_update.py and readable by other users. This file is re-generated when vtd_apriori_update.py runs. vtd_apriori_update.py calls Network Earth Rotation Service (NERS) for regeneration this file.

3.9   IONO_DIR:

IONO_DIR: name_of_tec_directory This keyword specifies the name of the directory tree where vtd_apriori_update.py writes TEC maps. vtd_apriori_update.py will create sub-directories codg_asc/ and codg_bin/ if they do not exist. The latter directory will have a number of files with TEC maps in vio binary format. Each file has continuous time series. vtd_apriori_update.py creates more than one output file to handle discontinuities and change of time resolution. vtd_apriori_update.py stores TEC maps in ascii format in directory codg_asc.

3.10   LOAD_BDS_DIR:

LOAD_BDS_DIR: name_of_massloading_directory Name of the directory tree with 3D site position variations caused by mass loading. vtd_apriori_update.py creates a tree nto/mpiom06 atm/merra2 atm/geosfpit atm/merra2_geosfpit lws/merra2 lws/geosfpit lws/merra2_geosfpit In total, there are over 9000 files in these directories. The files have extension .bds for data in BDS format and .txt for the summaries. vtd_apriori_update.py updates bds files and the summary files for new mass loading files that it discoverers at remote servers.

3.11   SPD_BIN_DIR:

SPD_BIN_DIR: name_of_slant_path_directory Name of the directory tree with coefficients of slant path delay expansions over 3D B-spline basis for a predefined set of space geodesy stations. vtd_apriroi_update.py creates a tree merra geosfpit In total, there are around 1000 files in binary bspd format in these directories. vtd_apriori_update.py updates bds files and the summary files for new slant path delays, present in the remote server and absent in the local directories that it discoverers at remote servers.

3.12   EOP_SERIES_UPDATE:

EOP_SERIES_UPDATE: yes_or_no A flag whether to update the Earth Orientation Parameter series. Supported values: yes or no

3.13   IONO_UPDATE:

IONO_UPDATE: yes_or_no A flag whether to update the with total electron contents maps. Supported values: yes or no

3.14   SPD_UPDATE:

SPD_UPDATE: yes_or_no A flag whether to update the files with expansion of slant path delays in the neutral atmosphere into B-spline basis. Supported values: yes or no

3.15   LOAD_UPDATE:

LOAD_UPDATE: yes_or_no A flag whether to update the mass loading files with 3D site displacements caused by the mass loading. Supported values: yes or no

3.16   MERRA2_GEOSFPIT_ATM:

MERRA2_GEOSFPIT_ATM: yes_or_no A flag whether to update the combined file of the atmospheric pressure loading from MERRA2 and GEOSFPIT numerical whether models. That file has atmospheric pressure loadings using GEOSFPIT model for all epochs that model is available and using MERRA2 model for all epochs that model is not available, i.e. for the interval 1980--2000. Supported values: yes or no

3.17   MERRA2_GEOSFPIT_LWS:

MERRA2_GEOSFPIT_LWS: yes_or_no A flag whether to update the combined file of the land water storage loading from MERRA2 and GEOSFPIT numerical whether models. That file has land water storage loadings using GEOSFPIT model for all epochs that model is available and using MERRA2 model for all epochs that model is not available, i.e. for the interval 1980--2000. Supported values: yes or no

4   Example

4.1   Example of a vtd_apriori_update.py control file

# Configuration of file vtd_apriori update. Format version of 2018.10.31 # # Control file for apriori update at astrogeo # # Last update: 2019.05.25_09:53:03 # # Here we define urls with resources where # # 1) slant path delay, # 2) massloading displaments and # 3) the ionosphere parameters reside # SPD_URL: http://pathdelay.net/spd/bin BDS_URL: http://massloading.net/imsl/load_bds IONO_URL: http://ftp.aiub.unibe.ch/CODE # # Log files at the local computer # LOG_FILE: /s0/vau/pau.log LOG_VERBOSE_FILE: /s0/vau/verbose_pau.log VERBOSITY_LEVEL: 1 # # Files and directories at local computer that holds output a priori data # to be updated # EOP_FILE: /s0/vau/eop_polu.txt IONO_DIR: /s0/vau/iono LOAD_BDS_DIR: /s0/vau/load_bds SPD_BIN_DIR: /s0/vau/spd/bin # # Flags yes/no whether a given a priori file needs be updated # EOP_SERIES_UPDATE: yes IONO_UPDATE: yes SPD_UPDATE: yes LOAD_UPDATE: yes MERRA2_GEOSFPIT_ATM: yes MERRA2_GEOSFPIT_LWS: yes # # End of the vtd_apriori_update.py

4.2   Example of a corresponding VTD control file

Here are definitions in the VTD control file that use the apriori parameters that are updated with vtd_apriori_update.py that runs with the above mentioned control file: EOP_SERIES: /s0/vau/eop_polu.txt ... POSVAR_FIL: 1 /s0/vau/load_bds/atm/merra2_geosfpit/ POSVAR_MOD: 1 TIME_SERIES POSVAR_INT: 1 SPLINE POSVAR_USE: 1 IF_AVAILABLE # POSVAR_FIL: 2 /s0/vau/load_bds/lws/merra2_geosfpit/ POSVAR_MOD: 2 TIME_SERIES POSVAR_INT: 2 SPLINE POSVAR_USE: 2 IF_AVAILABLE # POSVAR_FIL: 3 /s0/vau/load_bds/nto/mpiom06 POSVAR_MOD: 3 TIME_SERIES POSVAR_INT: 3 SPLINE POSVAR_USE: 3 IF_AVAILABLE # POSVAR_FIL: 4 /s0/vau/load_har_list/atm/merra2/atm_merra2_harmod.hps POSVAR_MOD: 4 HARMONIC_MODEL POSVAR_INT: 4 LINEAR POSVAR_USE: 4 IF_AVAILABLE # POSVAR_FIL: 5 /s0/vau/load_har_list/lws/merra2/lws_merra2_harmod.hps POSVAR_MOD: 5 HARMONIC_MODEL POSVAR_INT: 5 LINEAR POSVAR_USE: 5 IF_AVAILABLE # POSVAR_FIL: 6 /s0/vau/load_har_list/toc/fes2014b/toc_fes2014b_harmod.hps POSVAR_MOD: 6 HARMONIC_MODEL POSVAR_INT: 6 LINEAR POSVAR_USE: 6 IF_AVAILABLE # POSVAR_FIL: 7 /s0/vau/load_har_list/toc/equil01/toc_equil01_harmod.hps POSVAR_MOD: 7 HARMONIC_MODEL POSVAR_INT: 7 LINEAR POSVAR_USE: 7 IF_AVAILABLE ... SLANT_PATH_DELAY_MODEL: SPD_3D EXTERNAL_DELAY_DIR: /spd/bin/geosfpit EXTERNAL_DELAY_DIR_2ND: /spd/bin/merra EXTERNAL_DELAY_DIR_3RD: NONE EXTERNAL_DELAY_DIR_4TH: NONE ... IONOSPHERE_MODEL: GNSS_TEC_MAP IONOSPHERE_DATA_FILE: /s0/vau/iono/code_bin/codg_01.vio IONOSPHERE_DATA_FILE_2ND: /s0/vau/iono/code_bin/codg_02.vio IONOSPHERE_DATA_FILE_3RD: /s0/vau/iono/code_bin/codg_03.vio IONOSPHERE_DATA_FILE_4TH: /s0/vau/iono/code_bin/codg_04.vio IONOSPHERE_DATA_FILE_5TH: /s0/vau/iono/code_bin/codg_05.vio IONOSPHERE_DATA_FILE_6TH: /s0/vau/iono/code_bin/codg_06.vio IONOSPHERE_DATA_FILE_7TH: NONE IONOSPHERE_DATA_FILE_8TH: NONE IONOSPHERE_DATA_FILE_9TH: NONE



Questions and comments about this guide should be directed to:

Leonid Petrov ( https://science.gsfc.nasa.gov/sed/bio/leonid.petrov-1 )

Last update: 2019.05.25