Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting order and hooks for test role #2374

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

eshulman2
Copy link
Contributor

  • Allow passing hook list directly to hook role
  • Allow setting up stages and hooks for test operator role

Jira: https://issues.redhat.com/browse/OSPRH-10106

Copy link
Contributor

openshift-ci bot commented Sep 24, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cescgina for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

Thanks for the PR! ❤️
I'm marking it as a draft, once your happy with it merging and the PR is passing CI, click the "Ready for review" button below.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/e1ceb9f2cf3d47658a3de25c1ed70de8

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 39m 16s
podified-multinode-edpm-deployment-crc FAILURE in 1h 03m 43s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 23m 33s
cifmw-multinode-tempest FAILURE in 1h 05m 28s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 35s
cifmw-pod-pre-commit FAILURE in 7m 04s
✔️ build-push-container-cifmw-client SUCCESS in 30m 49s
✔️ cifmw-molecule-run_hook SUCCESS in 3m 26s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 31s

@eshulman2 eshulman2 force-pushed the test_operator_stages branch 3 times, most recently from f24ddba to c4de17d Compare September 24, 2024 15:17
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/a25d6668f88443eeb5b2e354691eb8dc

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 39m 51s
podified-multinode-edpm-deployment-crc FAILURE in 1h 02m 18s
cifmw-crc-podified-edpm-baremetal RETRY_LIMIT in 22m 04s
cifmw-multinode-tempest FAILURE in 1h 02m 50s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 49s
cifmw-pod-pre-commit FAILURE in 8m 22s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 50s
✔️ cifmw-architecture-validate-hci SUCCESS in 4m 00s
✔️ build-push-container-cifmw-client SUCCESS in 31m 24s
✔️ cifmw-molecule-run_hook SUCCESS in 3m 25s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 16s

@eshulman2 eshulman2 force-pushed the test_operator_stages branch 2 times, most recently from c175381 to 5efbd8f Compare September 24, 2024 17:26
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ea84a8ceb689406f85a63f9e9f8d1718

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 39m 40s
podified-multinode-edpm-deployment-crc FAILURE in 1h 00m 58s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 27m 32s
cifmw-multinode-tempest FAILURE in 1h 01m 56s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 29s
cifmw-pod-pre-commit FAILURE in 6m 57s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 18s
✔️ cifmw-architecture-validate-hci SUCCESS in 4m 47s
✔️ build-push-container-cifmw-client SUCCESS in 20m 55s
✔️ cifmw-molecule-run_hook SUCCESS in 4m 02s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 54s

@eshulman2 eshulman2 force-pushed the test_operator_stages branch 3 times, most recently from 1c61cd8 to 34ffac9 Compare September 25, 2024 08:19
@eshulman2 eshulman2 marked this pull request as ready for review September 25, 2024 08:24
@eshulman2 eshulman2 force-pushed the test_operator_stages branch 2 times, most recently from c00804c to e6b67aa Compare September 25, 2024 08:28
@eshulman2 eshulman2 requested review from lpiwowar, lewisdenny and cjeanner and removed request for lpiwowar and lewisdenny September 25, 2024 09:48
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/e49808655fc5428d94a305a80ce4e4fa

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 49m 27s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 15m 48s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 20m 16s
cifmw-multinode-tempest FAILURE in 1h 37m 04s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 39s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 52s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 28s
✔️ build-push-container-cifmw-client SUCCESS in 20m 49s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 14s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/b18567a73541443d926209a283624f10

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 58m 22s
podified-multinode-edpm-deployment-crc FAILURE in 1h 05m 16s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 34m 23s
cifmw-multinode-tempest FAILURE in 1h 02m 15s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 52s
cifmw-pod-pre-commit FAILURE in 8m 08s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 53s
✔️ build-push-container-cifmw-client SUCCESS in 36m 42s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 18s

@eshulman2
Copy link
Contributor Author

recheck

@eshulman2 eshulman2 force-pushed the test_operator_stages branch 2 times, most recently from 0e9d6a4 to 9a7fd83 Compare November 14, 2024 14:51
@eshulman2
Copy link
Contributor Author

recheck

@eshulman2 eshulman2 force-pushed the test_operator_stages branch 2 times, most recently from 8c7258b to d11abb3 Compare November 14, 2024 16:43
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/a298fb370d914192b82b183fa6a19dbd

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 17m 49s
podified-multinode-edpm-deployment-crc FAILURE in 2h 09m 58s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 31m 19s
cifmw-multinode-tempest FAILURE in 2h 14m 39s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 40s
cifmw-pod-pre-commit FAILURE in 7m 42s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 41s
✔️ build-push-container-cifmw-client SUCCESS in 21m 56s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 23s

@eshulman2
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/3b7e156e68d64b8fa72a3df62cdb32b1

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 27m 20s
podified-multinode-edpm-deployment-crc FAILURE in 2h 01m 32s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 32m 38s
cifmw-multinode-tempest FAILURE in 2h 14m 38s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 17s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 12s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 11s
✔️ build-push-container-cifmw-client SUCCESS in 36m 47s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 11s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/b68d0a7b3a8443c9abaa8083eea16d36

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 26m 33s
podified-multinode-edpm-deployment-crc FAILURE in 2h 12m 11s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 28m 16s
cifmw-multinode-tempest FAILURE in 2h 14m 15s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 50s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 43s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 26s
✔️ build-push-container-cifmw-client SUCCESS in 36m 35s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 12s

Copy link
Contributor

@lpiwowar lpiwowar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eshulman2
Copy link
Contributor Author

@lpiwowar pushed a fix

lpiwowar added a commit to lpiwowar/test-operator that referenced this pull request Nov 19, 2024
The linked PR for the test-oprator role introduces the concept of
stages. This patch updates the job definition so that it is
consuming the newly defined interface. It is required because by
default only the tempest stage is enabled.

Depends-On: openstack-k8s-operators/ci-framework#2374
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/9a679b468a4e495a967795c895222826

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 42m 04s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 17m 56s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 30m 11s
cifmw-multinode-tempest FAILURE in 1h 41m 16s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 22s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 34s
✔️ cifmw-pod-zuul-files SUCCESS in 3m 51s
✔️ build-push-container-cifmw-client SUCCESS in 41m 16s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 26s

lpiwowar added a commit to lpiwowar/test-operator that referenced this pull request Nov 20, 2024
The linked PR for the test-oprator role introduces the concept of
stages. This patch updates the job definition so that it is
consuming the newly defined interface. It is required because by
default only the tempest stage is enabled.

Depends-On: openstack-k8s-operators/ci-framework#2374
Copy link
Contributor

@lpiwowar lpiwowar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to continue with reviewing this. Until then there are still some things that need to be fixed. We're also waiting for the results of the test-operator job -> here

roles/test_operator/README.md Outdated Show resolved Hide resolved
roles/test_operator/README.md Outdated Show resolved Hide resolved
@@ -216,3 +229,21 @@ other than the default one (e.g., `tempest-tests`, `tobiko-tests`, ...):
```
cifmw_test_operator_tempest_name: "post-update-tempest-tests"
```

### Use test operator stages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): I would add a text that explains the stages. I would try to imagine that I'm explaining this to somebody who has no idea what test operator stages are and does not want to dig through the code of the role.

  • why is this needed?
  • when do I want to use the stages?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a short description

@@ -25,6 +24,20 @@ Execute tests via the [test-operator](https://openstack-k8s-operators.github.io/
* `cifmw_test_operator_delete_logs_pod`: (Boolean) Delete tempest log pod created by the role at the end of the testing. Default value: `false`.
* `cifmw_test_operator_privileged`: (Boolean) Spawn the test pods with `allowPrivilegedEscalation: true` and default linux capabilities. This is required for certain test-operator functionalities to work properly (e.g.: `extraRPMs`, certain set of tobiko tests). Default value: `true`
* `cifmw_test_operator_selinux_level`: (String) Specify SELinux level that should be used for pods spawned with the test-operator. Note, that `cifmw_test_operator_privileged: true` must be set when this parameter has non-empty value. Default value: `s0:c478,c978`
* `cifmw_test_operator_stages`: (List) List of dictionaries defining the stages that should be used in the test operator role. List items options are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (blocking): Missing default value of the cifmw_test_operator_stages variable in the README.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is actually there after the entire section in line 36

ansible.builtin.stat:
path: "{{ cifmw_test_operator_controller_priv_key_file_path }}"
register: private_key_file
- name: Ensure a secret for the cifmw private key file exists
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Do we really have to keep this outside of the block?

@@ -81,47 +81,38 @@
}}}, recursive=true)
}}

- name: Check that cifmw private key file exists
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (blocking): This task can not be removed. We have to check whether the private_key_file exists prior to the execution of the following tasks.

#2484

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

roles/test_operator/defaults/main.yml Outdated Show resolved Hide resolved
NOTE: this patch will require setting order for all jobs running not only
tempest

- Allow passing hook list directly to hook role
- Allow setting up stages and hooks for test operator role

Jira: https://issues.redhat.com/browse/OSPRH-10106
lpiwowar added a commit to lpiwowar/test-operator that referenced this pull request Nov 20, 2024
The linked PR for the test-oprator role introduces the concept of
stages. This patch updates the job definition so that it is
consuming the newly defined interface. It is required because by
default only the tempest stage is enabled.

Depends-On: openstack-k8s-operators/ci-framework#2374
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/eabd473c32a849bb9a67cb39a34b8c4e

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 47m 38s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 16m 24s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 35m 14s
cifmw-multinode-tempest POST_FAILURE in 1h 46m 36s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 25s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 47s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 32s
✔️ build-push-container-cifmw-client SUCCESS in 36m 41s
✔️ cifmw-molecule-test_operator SUCCESS in 4m 03s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants