Skip to content

Sprint 109: Release Report (sprint-108 - sprint-109)
Compare
Choose a tag to compare
@elopatin-uc3 elopatin-uc3 released this 16 Oct 23:59
· 3 commits to main since this release
sprint-109
5863d96
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Major accomplishments:

  • Queue Redesign release: Ingest, Inventory, mrt-zk
  • Stage collection and admin object cleanup completed
  • Stage automated cleanup implemented for removing regression test content

mrt-doc sprint-108..sprint-109

  • 5863d96 2024-10-16 Update sprint-log.md
  • 7a6e0ec 2024-10-16 Update sprint-log.md
  • 8aa7191 2024-10-03 Update next_steps.md
  • f76d7cc 2024-10-03 dup section
  • bbc9d72 2024-10-03 manual deploy - all 5 services
  • 304e821 2024-10-03 manual deployment notes
  • ff4f4ab 2024-10-03 Update manual-deploy.md
  • e909a5e 2024-10-03 Create manual-deploy.md
  • edf8bcb 2024-10-01 Update next_steps.md

mrt-doc-private sprint-108..sprint-109

  • 92f406f 2024-10-09 Update routine_librato_checks.md
  • 0959f4d 2024-10-08 Update dependency_scans.md

mrt-inventory sprint-108..sprint-109

  • 7a83864 2024-10-14 Merge pull request 34 from CDLUC3/zoofix
  • d84c9b3 2024-10-11 Fix ephemeral zoo locks
  • b03350e 2024-10-03 Merge pull request 33 from CDLUC3/zooInv
  • c529665 2024-10-03 restore back README
  • 6cbb271 2024-10-03 dummy change to force rebuild
  • 7f967fc 2024-10-01 Merge branch 'zooInv' of https://github.com/CDLUC3/mrt-inventory into zooInv
  • 3e3c906 2024-10-01 Fail Job on bad manifest url
  • 2cd0c58 2024-09-23 semantic tag handling
  • 54b0433 2024-09-13 tag determination
  • b73e430 2024-09-13 changes won't upload - Merge branch 'zooInv' of https://github.com/CDLUC3/mrt-inventory into zooInv
  • 5fd8e1e 2024-09-13 Refactor ark lock handling
  • 3e226b2 2024-08-30 build tag calcluation
  • 35ca6d4 2024-08-29 build.content.txt
  • dfe1363 2024-08-28 Add pending status
  • fe02ce7 2024-08-28 tag reportupdate
  • 335f857 2024-08-28 Add tast db handling
  • e88b95c 2024-08-27 pom var
  • 63fa64d 2024-08-27 use ECR_REGISTRY for images
  • 2489eec 2024-08-27 publish conf
  • 4ab04ad 2024-08-26 multiline format
  • ba64a41 2024-08-26 reset war name
  • 11e5224 2024-08-26 war name
  • e687c84 2024-08-19 reponame
  • 15cf127 2024-08-16 cleanup artifacts
  • e0a9f62 2024-08-16 buildspec report
  • 0ca243c 2024-08-16 test resource
  • 4082488 2024-08-12 fix build label
  • 5f62633 2024-08-12 tagging rules
  • e834d01 2024-08-12 tagging rules
  • ba527fc 2024-08-12 rm create ecr
  • 2042768 2024-08-06 buildspec normalize
  • 47a91e0 2024-07-31 Merge pull request 32 from CDLUC3/codebuild2
  • 1d0908d 2024-07-31 zooInv
  • ebdfbc4 2024-07-30 version refs
  • 4ed3908 2024-07-29 3.0-SNAPSHOT
  • f4f0943 2024-07-26 build
  • a6ee6ec 2024-07-25 zk snapshot
  • b9a2620 2024-07-25 buildx
  • a40d74a 2024-07-25 multiplatform
  • afd61cc 2024-07-24 publish invconf
  • 15ef9a9 2024-07-24 eliminate inventoryconf
  • 97c60a9 2024-07-24 build content fix
  • 6ef13cb 2024-07-24 codebuild2
  • 8c2ee6e 2024-07-24 tidy buildspec and jdpa
  • 1ea1381 2024-07-24 INV it tests
  • 1fbe3b3 2024-07-24 pom
  • 873f210 2024-07-24 gitignore
  • 6fb50a7 2024-07-22 refine build
  • 9a30463 2024-07-22 inv buildspec
  • e0f886a 2024-07-18 codebuild
  • 1fa68ac 2024-07-23 Remove references to cdl-zk-queue
  • 1001c95 2024-07-22 refine build
  • 603df21 2024-07-22 inv buildspec
  • a5a1dcd 2024-07-18 codebuild
  • d11782c 2024-06-28 SaveObject zooKeeper close
  • e4ca0a1 2024-06-27 Merge pull request 31 from CDLUC3/zooInv2
  • 952bb1c 2024-06-27 one more cleanup
  • 4f31c70 2024-06-27 Remove System.out.println s
  • 74534f9 2024-06-27 Restore IT test for adding object through zk zookeeper
  • ba33c75 2024-06-26 IT test with ZooKeeper directly
  • 5838549 2024-06-26 better track state
  • 67dd329 2024-06-24 initialize zookeeper nodes
  • c7d493f 2024-06-21 inv new zoo
  • 6aeeebf 2024-06-21 inv new zoo
  • 4f8982c 2024-06-21 inv new zoo

mrt-ingest sprint-108..sprint-109

  • d87dff8 2024-10-16 Merge pull request 107 from CDLUC3/jobCleanup
  • fe467af 2024-10-16 Turn off hourly Job Cleanup
  • 9819ff3 2024-10-15 Merge pull request 106 from CDLUC3/localidNotify
  • 100e594 2024-10-14 - Refresh Job config/ids after processing. - Ensure notification gets latest LocalID data
  • b2db03b 2024-10-07 Fix serious Update/Submit bug!
  • 717c3fc 2024-10-03 Merge pull request 105 from CDLUC3/queue_LIB
  • 11d4309 2024-10-02 #2046
  • 4532d02 2024-10-02 Fix logic error for ZFS usage
  • 73c9d5d 2024-10-02 Support ZFS usage check in Provision Handler
  • 65e31e6 2024-10-01 Trap more Zookeeper exceptions for long running sessions
  • 9a9e079 2024-09-29 - Force a bad Storage manifest for Inventory on demand - Add submissionDate to Job Configuration
  • e78cc12 2024-09-26 Remove deleted jobs
  • 3320f66 2024-09-25 Force failure if file exists. Useful for testing
  • ecbc764 2024-09-25 Be more clear on Cleanup process scheduling
  • a97aa20 2024-09-24 Trap Connection loss and Session Expiration and retry
  • 1be0dbb 2024-09-23 semantic tag handling
  • deaf215 2024-09-23 Ensure that we unlock job when switching to Held state
  • abe5e3c 2024-09-18 Reconnect after session expiration for all Consumer Daemons
  • 1cc73e3 2024-09-13 Cosmetic change. Trigger a rebuild.
  • 1ec5d18 2024-09-13 update tag logic
  • bbd744c 2024-09-13 TAG logic
  • c4d596b 2024-09-12 Fix logic error with queue status in State command
  • d26b25f 2024-09-09 Specify FAIL on Batch and Job errors
  • 6f7d27a 2024-09-05 Control Zookeeper logging with logback
  • 29e1ed0 2024-08-30 Set default LOGLEVEL to warn (from INFO)
  • 0f29c76 2024-08-30 build tag calcluation
  • 44ff474 2024-08-29 build.content.txt
  • 1813c7a 2024-08-28 Fix serviceStartTime within state. Add more IT assertions w/ primaryID
  • 4bb3b88 2024-08-28 Merge branch 'queue_LIB' of github.com:cdluc3/mrt-ingest into queue_LIB
  • fe4ca06 2024-08-28 tag report update
  • 94c3527 2024-08-27 Add LocalID Integration testing w/ ERC/Form/Manifest entry points
  • 0452f8f 2024-08-27 pom var
  • 7028afd 2024-08-27 use ECR_REGISTRY for images
  • fbe982e 2024-08-22 pull through cache
  • f1bf99c 2024-08-22 registry vars
  • f338489 2024-08-22 enable tests
  • 327c93f 2024-08-22 img refs
  • a524017 2024-08-21 fakesmtp image
  • 3a3d526 2024-08-19 reponame
  • 08b1368 2024-08-16 buildspec
  • e1daf70 2024-08-16 clean unlisted
  • 51aeeaf 2024-08-16 format buildspec
  • e809a90 2024-08-16 buildspec
  • 3321dad 2024-08-16 update integ tests and buildspec
  • 69cc8e9 2024-08-16 test driver fix
  • 2edb0a4 2024-08-16 Add Initialization Consumer Daemon
  • 2943c53 2024-08-12 fix 6
  • c20e37d 2024-08-12 fix 5
  • 72530a4 2024-08-12 fix 4
  • 20d45e3 2024-08-12 fix 3
  • 9ddba6c 2024-08-12 fix 2
  • 0ce10d4 2024-08-12 fix
  • d8d5957 2024-08-12 commit date for build info
  • 3858e23 2024-08-12 refine tagging
  • 29017cb 2024-08-12 rm ecr create
  • 156a0e1 2024-08-06 buildspec normalize
  • 4166f4c 2024-07-31 Merge pull request 104 from CDLUC3/codebuild
  • a4071fd 2024-07-31 queue_LIB
  • 697927e 2024-07-30 version refs
  • afe12b5 2024-07-29 3.0-SNAPSHOT
  • 952d001 2024-07-26 build
  • 819fecf 2024-07-25 zk snapshot
  • 542115b 2024-07-25 buildx
  • 391c0a4 2024-07-25 multiplatform
  • 44f2fd2 2024-07-24 docker profiles
  • 982f525 2024-07-23 pom consistency
  • 2c5eb0d 2024-07-22 it
  • fc78329 2024-07-22 docker
  • 73558bf 2024-07-19 dockerfile fix
  • 4ad4e3b 2024-07-19 dockerfile
  • 81762eb 2024-07-19 docker fix
  • 928e00f 2024-07-19 ingest war
  • f8cf4f1 2024-07-19 skip tests
  • 29a7b1f 2024-07-19 refine test driver
  • 9de3f3c 2024-07-18 codebuild fakesmtp
  • c27f610 2024-07-18 use ecr
  • 3103c9e 2024-07-18 use own copy of fakesmtp
  • 9315afe 2024-07-17 settings.xml
  • ac1e821 2024-07-17 ingest buildspec
  • 7bfc3d8 2024-07-08 Ensure that we do not exceed Thread Pool size when acquiring data
  • bbcc58c 2024-07-03 Remove all reference and usage of old ZK library
  • 7818153 2024-07-01 Remove all references of DistributedQueue from old ZK lib
  • a8dde51 2024-06-27 Cosmetic
  • 1939c58 2024-06-27 Fix session exception
  • e4a05c9 2024-06-27 Was too aggressive with ZK closure
  • 9b66944 2024-06-27 - Close all ZK client connections upon exit - Fix Manifest URL for Inventory
  • ddb97e0 2024-06-26 Support collection level freeze with new ZK lib
  • 81d4f3b 2024-06-25 Ingest Queue Hold support with new ZK lib
  • f7c00d7 2024-06-21 Support Locks with new ZK lib
  • 62afc38 2024-06-20 Force notification attachment format of queueing to JSON
  • 00ddd99 2024-06-20 Clean up debug statements
  • 3d1bcd9 2024-06-18 Cosmetic
  • bb45b23 2024-06-14 More fault tolerant to exceptions
  • 38eeb5d 2024-06-12 Fail Batch if Job posting fails Creat job with priority only for Estimation state
  • 84c594d 2024-06-12 Remove concept of Jobs States (plural)
  • 1bcf91b 2024-06-12 Remove JobsState and /status API
  • be984f3 2024-06-12 - Populate Inventory ZK field - More fault tolerant for ZK setStatus
  • 5c17c9f 2024-06-04 use mrt-zk for IT test validation
  • cbd6aa2 2024-06-03 Look for Completed state
  • f51269b 2024-06-03 old zk lib dependency
  • 2b6584b 2024-06-03 test cleanup
  • 107a8b9 2024-06-03 New handlers
  • eed98cb 2024-06-03 it test changes
  • 75df29c 2024-05-31 Use new ZK method to initialize root ZK paths.
  • 2234ca0 2024-05-31 "submit-object" and "update-object" endpoints deprecated
  • 289805a 2024-05-29 Support state command
  • b1188f4 2024-05-28 mrt-json lib brings in version of JSON lib not supporting JSONObject.isEmpty()
  • 16862cb 2024-05-24 Cosmetic
  • 64991c7 2024-05-23 Store metadata and id info in CONFIGURATION JSONObject
  • df09301 2024-05-21 Populate metadata in ZK lib
  • 432fa4a 2024-05-21 Populate and utilize ids in ZK lib
  • 1371715 2024-05-20 - Add Job failure status in Notification attachments and email message
  • d0f4106 2024-05-19 Additional ZK work
  • 1ce12fc 2024-04-20 Add Estimate and Provision daemon/process
  • 5846676 2024-04-11 Process Batch queue
  • e7852c7 2024-04-08 Post Batch data

mrt-zk sprint-108..sprint-109

  • df72336 2024-10-08 Update states.md
  • 83e808d 2024-10-08 batch job count
  • 80fbed2 2024-10-08 list_batches
  • f359186 2024-09-27 Update data.md
  • d93d5dd 2024-09-27 dont set last successful to failed
  • 52a8795 2024-09-27 dont set last successful to failed
  • ebc36d2 2024-09-27 dont set last successful to failed
  • 907ff2e 2024-09-27 provisioning fail ruby
  • b685dcb 2024-09-27 allow provisioning fail java
  • 2464e8d 2024-09-26 rb batch delete fix
  • bd82f4c 2024-09-26 Only process Jobs when cleaning queue (/jobs/states ignore)
  • e028a63 2024-09-26 Add ability to remove DELETED jobs
  • 4579634 2024-09-25 Catch everything for batch cleaning
  • 17de09d 2024-09-25 Catch additional exception when cleaning batches

mrt-dashboard sprint-108..sprint-109

  • 80db48db 2024-10-14 atom retry
  • 33ff6c49 2024-10-14 Active record class
  • 9794b60c 2024-10-14 refine retry failure (with force failure)

mrt-admin-lambda sprint-108..sprint-109

  • a8152f8 2024-10-16 Update ssm.registry.yml-reset inventory Timeout
  • 11b1c29 2024-10-16 Update ssm.registry.yml - add inventory.queueTimeout
  • affa516 2024-10-08 rubocop updates
  • 24ae396 2024-10-08 Del Batch/Update Reporting
  • 73d1402 2024-10-08 queue menu updates - batch support
  • b831db2 2024-10-08 revise menus, improve deploy
  • 7d7b716 2024-10-07 zknode payload limit size
  • 5f77f81 2024-10-07 list zk nodes - keys only
  • 4c99de3 2024-10-07 show jobs by batch count
  • 748770f 2024-10-02 Update ssm.registry.yml
  • af71212 2024-10-01 ldap cert check
  • 326111f 2024-09-27 add validation tests
  • 7cbefa9 2024-09-27 initial zk orphan report
  • 104caa6 2024-09-27 fix legacy date default
  • b672f61 2024-09-27 refine admin behaviors
  • ac2eb6d 2024-09-27 refine confirmation boxes
  • bcb8f86 2024-09-27 doc update, update zk gem
  • 2d0b66b 2024-09-26 dump zookeeper node content
  • 385a278 2024-09-25 fix held item counts

mrt-cron sprint-108..sprint-109

  • ccbb9e5 2024-10-01 additional consistency check

mrt-sceptre sprint-108..sprint-109

  • 4b157c9 2024-10-15 Update index.md
  • 659edf1 2024-10-03 ingest default branch is main
  • 3539d96 2024-10-03 inv default branch is now main
  • 61301b1 2024-10-02 match mrt lambda name pattern
  • 8bcf943 2024-10-02 Merge pull request 1 from CDLUC3/coderev
  • 40eee40 2024-10-02 demo2 delete
  • ee92104 2024-10-02 hard-code project name for dependency purposed
  • 3d80ddf 2024-10-02 shorten names
  • 5145ed3 2024-10-02 additional projects
  • f2ca69c 2024-10-02 eliminate old style workspace config
  • e209179 2024-10-02 scope lambda permissions
  • 2c1c876 2024-10-02 conditionally add ssm
  • cb6f6e3 2024-10-01 code rev comments
  • adc4997 2024-10-01 part 2 review notes
  • 3fc6a59 2024-10-01 notes update
  • 299508f 2024-10-01 simplify templates
  • 63dd9df 2024-10-01 consolidate ecs setup
  • 41e8e1f 2024-09-30 work in prog
  • b7a8860 2024-09-30 Create CodeReview-IngestWorkspace.md
  • 9e445d1 2024-09-30 Update README.md
  • ba20456 2024-09-30 schedule load test

mrt-tomcat-deploy sprint-108..sprint-109

  • e69c443 2024-10-01 fixed bug with restart verification
  • 0602368 2024-09-20 add --version option to cli
  • 40e364e 2024-09-19 delete ROOT and validate restart via http all working
  • ba9bcdf 2024-09-19 Merge branch 'main' into delete_root_dir
  • ee596fe 2024-09-18 add url check to restart task
  • 2b5c67f 2024-09-18 add task to delete webapps/ROOT dir

merritt-docker sprint-108..sprint-109

  • 981a85a 2024-10-09 update commits
  • 02777d6 2024-10-07 app recs
Assets 2
Loading