Retrospectives 2018 12 06 SecureDrop Workstation Alpha 0.1.1

SecureDrop Workstation 0.1.1 Alpha Retrospective

Scope: The work on the workstation from approximately March 2018 to the 0.1.1 Alpha Release on November 15, 2018.

Participants: Jen, Conor, Mickael, Erik, Nina, Kevin, Kushal,, heartsucker

What went well?

Erik: We shipped a working alpha to the auditors on time with a very small team, a very ambitious scope, and a very tight deadline.

heartsucker:

• holy crap we wrote an actual Qt client! :D :D :D
• team was able to ID problems to each other and communicate well (+1 +1)
• rapid iteration, responses, and bug closing (+1)
  • like seriously, we dog piled this pretty hard and managed to bang out a lot of good work in a short time without stepping all over each other

Jen:

• This is the biggest architectural change by far in the last ~4-5 years of the SecureDrop project and we were able to implement these huge changes quite rapidly (+1 +1)
  • In implementing tickets for the new architecture, as a byproduct we also resolved issues that can be used over in SecureDrop core/server land (e.g. packaging using dhvirtualenv) (+1)
  • As a distributed team we were able to work quite effectively together
  • The meeting together in SF for the last 1-2 weeks prior to the deadline was really helpful (this was suggested by Erik, so credit to him there) (Get kushal there too next time)
  • The hackathon right before the release was the first example I've seen of the "SecureDrop hackathon" working very well (indicating that for rapidly evolving / prototyping projects we could do that more) - we had a lot of nice little issues implemented and security issues found. (+1)
    • Very focused "ask" from participants, easy tickets prepared well ahead of time
    • Working on well-defined component like the Qt client much more accessible
  • Nina's designs, research and feedback was fantastically useful, helpful and inspiring to developers.

Conor:

• Fantastically effective collaboration between core team, long-running contractors, and totally new contractors
• Work planning lift was significant and incredibly helpful to evaluating progress and avoiding feature creep
• Prototype represents rethinking long-standing problems with current SD, team was excited to deliver improved functionality
• We hit the deadline and delivered solid work (audit results pending! =D)

mickael:

• the client integration to the workstation just worked +1 ("just worked"™ =D)

Nina:

• UX Design and research actually happened, despite not being engaged until rather late! <3

Kushal:

• We managed to understand/figure out enough of Qubes OS to pull this off.

What didn’t go well?

Erik: We over-extended ourselves beyond the team's capacity, and only added additional contractors (Nina, Nick) late in the game.

heartsucker:

• a lot of the patterns we used in the client at first are going to have to be changed significantly or removed altogether, so it's possible the audit will have been done on something that is entirely unlike the beta version we ship (granted this is confounded by a lot of things) (+1)
• separation of SDK from client code made it a little annoying to work on things
• if we do have separate repos, it would be nice to polish the dependency a bit more before making it a requirement for another project
• circular non-progress on replies functionality

Jen:

• We found two major security bugs far too late. (+1) One the day of the release, and one the day after. We don't yet know what other security issues were found until we get the audit results back.
• Team in the last two weeks before the audit was working at a very unsustainable pace, to the point of being less effective and possibly making mistakes due to lack of rest. (kushal: +100, working till 4-5AM is difficult)
• Generally in the month or so before the release we were still working too much.
• We had to accept some technical debt in order to get core functionality completed in time.
• We had to significantly cut scope in order to focus developer attention on completed highest priority core functionality.

Nina:

• Building a pipeline of research participants from scratch in parallel to everything else was a distraction from design and research tasks that needed to be happening to validate build ideas.
• Early synthesis of the research backlog was time intensive, but important because there was value in the findings.
• Not much organization of volunteer ux resources imposed an undue burden on the lone paid contributor.
• Broader organization of SD-wide UX things (that tangentially impacted this project) took time from the lone paid contributor.
• Using a new frontend thingy to code the Alpha (QT) that was so new to everybody made it difficult to communicate UX things.
• Scope whiplash kinda sucked. I lost a lot of time trying to help support things that were then pulled off the table, and it burns everyone out.

mickael:

• insufficient time to properly QA all the changes (many last-minute, major changes)

Kushal:

• Packaging should be a first hand part of the project (means we should think about it from the start). Same goes to how to deliver/update those on the client systems.
• We still dependent of the provision system less than the OS level packaging tool

Kev:

• Assumptions about the state of Qubes workstation were incorrect:
  • Debian-9 template needed to be updated after install but not done by make all target (kushal: all most all templates need update after install :()
  • One auditor's update proxy VM was broken, had to create a fresh one from whonix-gw-14.

Conor:

• Work planning was complicated by a net-new system to most of the team, so surprises frequently arose that delayed delivery times of individual tickets
• Team members worked very long hours during the final few weeks.
• Some folks backed away from the project due to heavy workload (both SDW-related and otherwise)
• Contractors were so effective, should have been in the mix earlier

Jaysinh:

• No functional tests. Unit tests are limiting to test user level/functional tests.

What could we do differently?

Erik: Ensure we secure resources early, and scope projects more realistically.

Conor:

• When requesting an audit, we should have the work-to-be-audited finished. Racing against a deadline to deliver a security-focused product was an unnecessary burden on the team. Next time, let's set the target, scope the work, deliver, tag, then engage the auditors. (+1 from heartsucker, 👍 to the degree possible given grant commitments +1 - also consider other team commitments - ie Xenial migration, which going to have to happen currently with the beta)
• Raise resource commitment (e.g. contractors) well in advance of planned work
• Enforce sane work hours on a regular basis
• Due to planning problems, feed in lots of time buffers, due to unexpected issues and slow testing turnaround

Jen:

• Generally reign ourselves in on over committing. We all want to evolve the project, but not at the cost of burning people out.
• We should move at a moderate pace (not a hasty pace) when we're developing security-critical software so that we don't take on a bunch of technical debt and security issues (i.e. we now need to backtrack a bit to resolve some technical debt, ideally we'd progress at a slower pace so that backtracking is less necessary (obviously some refactoring is a natural part of the development process))

Erik:

• There's a lot of uncertainty in this project. When we learn about new, unexpected requirements, let's re-check whether the feature is still realistic

mickael:

• bigger/infra-level changes (e.g. rpm hosting) should have been front-loaded
• more diligence in performing clean installs during the course of development (+1)

heartsucker:

• don't set deadlines (save very loose ones?) for MVPs because when using new tech there is way too much unknown

kushal:

• Work on the provision story with dummy packages + dependencies, can save many of the current salt file based updates

Nina:

• Keep UX work happening with dedicated resource(s) in an ongoing fashion, so that many of the systemic-support things (such as a research participants base) are already be in place once a project kicks-off and/or resources become available to work on it. Cadence is important in an org, for design work to meaningfully inform processes to deliver the most value for the work invested.
• Rethink/re-document research best practices to encourage immediate synthesis & findings-sharing of all user research efforts, to prevent synthesis backlog.
• Remove dependency on participant review of transcripts before sharing among team, as that is not a standard in research privacy best practices.
• Learn/embrace more standard research participant privacy best practices, and balance with decentralized-team sharing opportunities better.
• Be MUCH more careful with sharing particpant transcripts; these DO NOT need to be shared among a full team, for relevant findings to be meaningfully shared with contributors. <3
• Budget to (modestly) compensate research participants (of non-customer newsrooms). Compensation is not an "incentive" thing, so much as a respect-for-time thing.
• Participant compensation helps remove bias opportunities introduced by depending upon only the more altruistic-spirited and free-time-available participants.
• Participants from under-represented communities really need to be offered compensation, or FPF can risk appearing opportunistic with white/Western privelege.
• Visual Design from-scratch takes time, over time... more than it takes a bundle of hours in a single-plop.
  • Helpful to do over a longer duration in small chunks, to get best results; when a single resource is not burdened with doing #alltethings, this is more sustainably feasible.

Potential future conversations

• Beta deadline (currently April): shall we re-negotiate once more? If not, what do we remove from scope?
• Development environment standardization/expectations -- what do we want to get out of Qubes dev env, and when do we expect folks to use it?