Run basic lint CI against Ubuntu noble / Python 3.12

[legoktm]

Status

Ready for review

Description of Changes

See commit messages for details.

Refs #7210.

Testing

How should the reviewer test this PR?

• Visual review
• CI passes, including staging job
• Setting up a staging environment using vagrant/libvirt works
• infra has moved "required" label from lint to lint (focal) (intentionally not noble, yet)

Deployment

Any special considerations for deployment? n/a

Checklist

• Linting (make lint) and tests (make test) pass in the development container
• Linting and tests (make -C admin test) pass in the admin development container
• Configuration tests pass
• I have performed a diff review and pasted the contents to the packaging wiki
  • https://github.com/freedomofpress/securedrop-builder/wiki/cffi-1.14.5-to-1.16.0 (by @zenmonkeykstop)
  • pip is exempt as it's maintained by PyPA

[legoktm]

I'm not setup to test "Setting up a staging environment using vagrant/libvirt works" so help would be appreciated with that.

[legoktm]

Oh also I haven't signed the commits yet, waiting to rebase after the uv PR is merged.

[zenmonkeykstop]

libvirt-based make staging is not currently working, likely due to config file format changes from molecule 3.0 to 3.6. Output below:

kog@unsafe-debian:~/projects/securedrop$ make staging
███ Creating staging environment on Ubuntu Focal...
Creating staging environment via 'libvirt-staging-focal'...
Creating /usr/bin/python3 virtualenv in .venv
created virtual environment CPython3.9.2.final.0-64 in 69ms
  creator CPython3Posix(dest=/home/kog/projects/securedrop/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/kog/.local/share/virtualenv)
    added seed packages: pip==20.3.4, pkg_resources==0.0.0, setuptools==44.1.1, wheel==0.34.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
INFO     libvirt-staging-focal scenario test matrix: dependency, create, prepare, converge
INFO     Performing prerun...
INFO     Set ANSIBLE_LIBRARY=/home/kog/.cache/ansible-compat/38acfb/modules:/home/kog/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/home/kog/.cache/ansible-compat/38acfb/collections:/home/kog/.ansible/collections:/usr/share/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/home/kog/.cache/ansible-compat/38acfb/roles:/home/kog/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
INFO     Running libvirt-staging-focal > dependency
WARNING  Skipping, missing the requirements file.
WARNING  Skipping, missing the requirements file.
INFO     Running libvirt-staging-focal > create
WARNING  Skipping, instances already created.
INFO     Running libvirt-staging-focal > prepare
WARNING  Skipping, instances already prepared.
INFO     Running libvirt-staging-focal > converge
[WARNING]:  * Failed to parse /home/kog/.cache/molecule/securedrop/libvirt-
staging-focal/inventory/ansible_inventory.yml with auto plugin: no root
'plugin' key found, '/home/kog/.cache/molecule/securedrop/libvirt-staging-
focal/inventory/ansible_inventory.yml' is not a valid YAML inventory plugin
config file
[WARNING]:  * Failed to parse /home/kog/.cache/molecule/securedrop/libvirt-
staging-focal/inventory/ansible_inventory.yml with yaml plugin: Unable to
retrieve file contents Could not find or access
'/home/kog/projects/securedrop/ansible-override-vars.yml' on the Ansible
Controller. If you are using a module and expect the file to exist on the
remote, see the remote_src option
[WARNING]:  * Failed to parse /home/kog/.cache/molecule/securedrop/libvirt-
staging-focal/inventory/ansible_inventory.yml with ini plugin: Unable to
retrieve file contents Could not find or access
'/home/kog/projects/securedrop/ansible-override-vars.yml' on the Ansible
Controller. If you are using a module and expect the file to exist on the
remote, see the remote_src option
[WARNING]: Unable to parse /home/kog/.cache/molecule/securedrop/libvirt-
staging-focal/inventory/ansible_inventory.yml as an inventory source
[WARNING]: Unable to parse /home/kog/.cache/molecule/securedrop/libvirt-
staging-focal/inventory as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
ERROR! Unable to retrieve file contents
Could not find or access '/home/kog/projects/securedrop/ansible-override-vars.yml' on the Ansible Controller.
If you are using a module and expect the file to exist on the remote, see the remote_src option
WARNING  Retrying execution failure 1 of: ansible-playbook -e @ansible-override-vars.yml --inventory /home/kog/.cache/molecule/securedrop/libvirt-staging-focal/inventory --skip-tags molecule-notest,notest /home/kog/projects/securedrop/install_files/ansible-base/securedrop-staging.yml
CRITICAL Ansible return code was 1, command was: ['ansible-playbook', '-e', '@ansible-override-vars.yml', '--inventory', '/home/kog/.cache/molecule/securedrop/libvirt-staging-focal/inventory', '--skip-tags', 'molecule-notest,notest', '/home/kog/projects/securedrop/install_files/ansible-base/securedrop-staging.yml']
make: *** [Makefile:249: staging] Error 1

I can likely take a look in the morning.

[zenmonkeykstop]

(Also - do we want a make staging-noble or equivalent? Might be a little early but would start testing out some Ansible roles)

[legoktm]

Thanks, I'll start digging through the changelog, etc. The staging CI job is also failing so I can use that for testing and will come back to you.

(Also - do we want a make staging-noble or equivalent? Might be a little early but would start testing out some Ansible roles)

Yep, will get to that after fixing package building.

[legoktm]

With e7c16b8 I'm seeing the same errors in the CI staging job that you pasted...progress!

[legoktm]

@zenmonkeykstop if you want to give this another pass, the molecule stuff now works in the staging CI job (except for the failure fixed in #7246).

[legoktm]

I've rebased on top of #7246 now that it's merged.

[zenmonkeykstop]

Staging env setup is in progress locally, just waiting for it to complete but jusque ici etc

[zenmonkeykstop]

• visual review good
• make staging passes in a libvirt system!
• lint tasks still pending

LGTM! Provisionally approved pending changes to lint job and it passing.