AutomaticDownload

This page describes efforts to automatically download Sentinel-1 data.

This command works for one data set (Ubinas). It creates or updates the file $OPERATIONS/stored_date.data. It assumes that all data prior to this data have been downloaded. To test adjust the endDate in the Google Sheet: https://docs.google.com/spreadsheets/d/1BOwq9k6B4JdfSx09uylEiL9ex3KK_UQ2uZMp34Cc00w/edit#gid=1436095665

If there is no endDate given it should

run_operations.py --startssara --stopssara  --sheet_id 1BOwq9k6B4JdfSx09uylEiL9ex3KK_UQ2uZMp34Cc00w

Note: The options of run_operations.py need to be updated to the new process_rsmas.py (future processApp.py) option. It should understand run_operations.py --step download as does process_rsmas.py

Another test file (three datasets) is : https://docs.google.com/spreadsheets/d/1BpDrNFG7Kqh42_PdK3gv9Jv0LxZFujAqL_uF5fC5hyQ/edit#gid=1436095665

run_operations.py --sheet_id 1BpDrNFG7Kqh42_PdK3gv9Jv0LxZFujAqL_uF5fC5hyQ

We run it under LSF using the following jobfile. Alternatively it could be run daily using a cronjob, but some HPC systems don't support cron.

#! /bin/tcsh
#BSUB -J run_operations
#BSUB -N -u jaz101@rsmas.miami.edu
#BSUB -o /nethome/famelung/insarlab/OPERATIONS/LOGS/JOB_SUBMISSION_LOGS/z_autorun_deqing_%J.o
#BSUB -e /nethome/famelung/insarlab/OPERATIONS/LOGS/JOB_SUBMISSION_LOGS/z_autorun_deqing_%J.e
#BSUB -n 1
#BSUB -P insarlab
#BSUB -q general
#BSUB -b 16:10

s.bgood
#nohup run_operations.py --sheet_id 1BpDrNFG7Kqh42_PdK3gv9Jv0LxZFujAqL_uF5fC5hyQ &
nohup run_operations.py --startssara --stopssara  --sheet_id 1BOwq9k6B4JdfSx09uylEiL9ex3KK_UQ2uZMp34Cc00w &

sleep 120

bsub < $OPERATIONS/run_operations_LSF.job