Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

debug in progress (20241115) #378

Closed
wants to merge 51 commits into from
Closed

debug in progress (20241115) #378

wants to merge 51 commits into from

Conversation

sigmafelix
Copy link
Collaborator

  • Per amadeus update, all calc_* functions are renamed to calculate_*
  • Dropped future and future.apply functions in calculate.R
  • Debugged site calculation except for parts utilizing bulk raster files (i.e., NASA products)

Let me know if there needs updates before pulling. Thank you.

mitchellmanware and others added 30 commits September 25, 2024 11:02
@mitchellmanware
common R libraries
2abba7b
@mitchellmanware
initiate development 09/30
f0fccce
@kyle-messier
set up container, def, and archive old bash scripts
fc275d2
@kyle-messier
add dev to branches triggering GH actions
bad244e
@kyle-messier
Merge pull request #368 from NIEHS/container-01
45a2170 
set up container, def, and archive old bash scripts
@mitchellmanware
appendable date range
f602529
@kyle-messier
def files, bash scipts for containers, testing yaml
30b0f89
@kyle-messier
fixing [on] typo
4c42f5a
@kyle-messier
GH action path correction
969565b
@kyle-messier
name change
66c65aa
@kyle-messier
def file was wrong name
27d64e3
@kyle-messier
rel path update for container build
f4db895
@kyle-messier
path fix, fakeroot
89a2f95
@kyle-messier
path change for test functions
8af11cd
@kyle-messier
cache fixing attempt
7afc578
@kyle-messier
path fix attempts
ab3dfb7
@kyle-messier
change order of yaml names to include new R test file in mnt
775c98d
@kyle-messier
remove _targets/ path from mount
691aa16
@kyle-messier
cache updating and check sections
ddee948
@kyle-messier
test cache of sif
ee9da0b
@kyle-messier
Merge branch 'dev' into container-01
401a78a
@kyle-messier
Merge pull request #369 from NIEHS/container-01
e460622 
def files, bash scipts for containers, testing yaml
@mitchellmanware
refactor calc_*_modis
d8f5fad
@kyle-messier
_targets.R, test and run bash scripts
51a6733
@kyle-messier
Merge branch 'dev' into container-01
8790b40
@kyle-messier
Merge pull request #370 from NIEHS/container-01
7d17aba 
_targets.R, test and run bash scripts
@mitchellmanware
commit to pull container dev
2adeadb
@mitchellmanware
Merge branch 'dev' into dev-0930
4c3cd75
@mitchellmanware
query_modis_files function
ec6d7e6
@mitchellmanware
rectify merge overwrite
03d3782
mitchellmanware and others added 21 commits October 2, 2024 19:15
@mitchellmanware
refactoring targets + container run
91619f5
@mitchellmanware
migrate dev/refactor/ -> root/
62c2b9b
@mitchellmanware
successful container run
8c5a79e
@mitchellmanware
refactor + integrate download targets (requires amadeus::download_mod…
bc2c82e 
…is update)
@mitchellmanware
read_locs -> amadeus::process_aqs
fac9aae
@mitchellmanware
update critical targets
2cfbdef
@mitchellmanware
organize; group storage location
a06a12b
@mitchellmanware
restore
3153f90
@mitchellmanware
manually calculate ecoregions
091cfcc
@mitchellmanware
organize
2b5a5c7
@mitchellmanware
organize 2
5364135
@mitchellmanware
/inst/targets/README.md
40847d8
@mitchellmanware
--force; query_modis_files
af7f5c3
@mitchellmanware
drop SLURM from container
7124813
@mitchellmanware
un-future calc_geos_strict
d7118ca
@mitchellmanware
Merge pull request #377 from NIEHS/dev-0930
56f2a3a 
0.4.2
@sigmafelix
debug in progress
7db5cf9 
- Bypassing download steps
- Confirmed that there were no returned objects even if a target was marked as completed
@sigmafelix
debug in progress
acab12a 
- New TRI data have a different column layout
- Shell script for tar_make at one target
@sigmafelix
debug
3197979 
- reduced redundancy
- dropping future.apply dependency
@sigmafelix
small update
cc06cc2 
- TODO: NASA products, GEOS-CF
- HMS calculation error is possibly related to calculate_hms (missing `by` argument)
- Renamed `calc_*` to `calculate_*` following `amadeus` functions
@sigmafelix
HMS calculation fix
c230aa0 
- upstream amadeus function was fixed
- library argument is populated with dev library locations
@kyle-messier
Copy link
Collaborator

  • Per amadeus update, all calc_* functions are renamed to calculate_*
  • Dropped future and future.apply functions in calculate.R
  • Debugged site calculation except for parts utilizing bulk raster files (i.e., NASA products)

Let me know if there needs updates before pulling. Thank you.

Thanks @sigmafelix. Are you working off the dev or dev-1028 branches? Those have the most recent refactoring updates. Although @mitchellmanware is working on more updates to ensure the proper branching and skipping with new data downloads.

One thought here and related to amadeus, perhaps worth an offline discussion with @mitchellmanware are the modis variables. There are a few issues open on amadeus related to modis, but they probably need clarification for you. Basically, the sparsity of time needs to be resolved more cleanly since the download expectation is by a given day.

@sigmafelix
Copy link
Collaborator Author

@kyle-messier This branch is based on dev. I will recheck dev-1028 and make a new PR.

@sigmafelix sigmafelix closed this Nov 15, 2024
@sigmafelix
Copy link
Collaborator Author

@kyle-messier Sure, the modis download function crawls the list of files from the remote server directories and use the file list as it is. Therefore, I think the sparsity of dates should not matter in downloading files. An error may surface if the NASA data are missing somehow during the input period. Exception handling by try or tryCatch would resolve that problem. I will look at this problem. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sigmafelix @kyle-messier @mitchellmanware