Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Lwt.Syntax and avoid some >>= fun () patterns #197

Merged
merged 10 commits into from
Oct 16, 2024
Merged

Conversation

dinosaure
Copy link
Member

This little commit takes the advantage of let* to replace some parts of the code. In my opinion (feel free to take it in account), the code is more readable than before. About the vifs function, I take the advantage that String.split_on_char can not return an empty list according to the documentation: we can pattern-match with [@warning "-8"] or use List.hd.

@dinosaure dinosaure requested review from hannesm and palainp May 22, 2024 07:43
dao.ml Outdated Show resolved Hide resolved
dao.ml Outdated Show resolved Hide resolved
dao.ml Outdated Show resolved Hide resolved
@hannesm
Copy link
Member

hannesm commented May 22, 2024

looks pretty good, some minor remarks inline

@palainp
Copy link
Member

palainp commented May 22, 2024

Thank you @dinosaure I just tested and it works as before. The CI failure is just about the resulting hashsum mismatch.

I would suggest, if you have other improvements to add them all and change the hashsum in build-with.sh at the end (we can cut a release when merged as the hashsum in build-with.sh in main is checked in the salt script).

dao.ml Outdated Show resolved Hide resolved
@dinosaure
Copy link
Member Author

Just to inform you, I continue to look deeper into the code and see many places where we can improve/upgrade the code 👍. I will continue to propose some PRs about that: don't think that this is the only and last PR 👍.

dao.ml Show resolved Hide resolved
@palainp
Copy link
Member

palainp commented May 22, 2024

Just to inform you, I continue to look deeper into the code and see many places where we can improve/upgrade the code 👍. I will continue to propose some PRs about that: don't think that this is the only and last PR 👍.

Or continue into this one? :D

@dinosaure
Copy link
Member Author

Or continue into this one? :D

As you want if it's clearer for you.

@hannesm
Copy link
Member

hannesm commented May 22, 2024

I think due to the checksum / release stuff, there are two or three options (as I don't quite know about the salt stuff):

  • continue refactorings in this PR (and then maybe squash the existing commits into a single one)
  • merge to a dev branch, which at the end we'll merge into the main branch and cut a release
  • fix the salt script to use the latest release instead of the main branch (if this is what is happening)

It would be great to keep the checksum as produced by CI and the one in the build-with file in sync... so I guess the first option is a good one :)

@hannesm
Copy link
Member

hannesm commented May 22, 2024

Of course, it is as well fine to accumulate commits here and then do a squash-merge :D

@palainp
Copy link
Member

palainp commented May 22, 2024

I think due to the checksum / release stuff, there are two or three options (as I don't quite know about the salt stuff):
* continue refactorings in this PR (and then maybe squash the existing commits into a single one)
* merge to a dev branch, which at the end we'll merge into the main branch and cut a release
* fix the salt script to use the latest release instead of the main branch (if this is what is happening)

I think solution (a) is the easiest so far, and (c) should be done at some point, and maybe it's easier to include the vmlinuz hashsum in the .sha256 file in the release (but TBH I'm unsure checking the vmlinuz is mandatory if the .bz2 hash already matches?, and https://github.com/unman/shaker/tree/main/mirage only check the .bz2 sum).

It would be great to keep the checksum as produced by CI and the one in the build-with file in sync... so I guess the first option is a good one :)

I also agree that having the correct hashsum in build-with.sh is important to avoid weekly CI failures :)

@dinosaure
Copy link
Member Author

My last patch can be a bit controversial and we should stop at this point to see if the unikernel works again. I explained in the commit why I did this change. Let me know if it's ok or not.

@palainp
Copy link
Member

palainp commented May 22, 2024

My last patch can be a bit controversial and we should stop at this point to see if the unikernel works again. I explained in the commit why I did this change. Let me know if it's ok or not.

Good news, at this point, qmf is still working (not a deep test: 3 clients in and out, bandwidth speed is the same as before). I'm probably ok with that change (at least I still understand the code), but I'm interested with others' thoughts :)

dispatcher.ml Outdated Show resolved Hide resolved
let* () =
Lwt.catch (add_vif get_ts vif dns_client dns_servers ~client_ip ~router
~cleanup_tasks qubesDB)
@@ fun exn ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally struggle with @@ and find parenthesis much easier to read (since I'm always unsure how strong the @@ binding strength is.

clients := !clients |> Dao.VifMap.add key cleanup)))
let open Lwt.Syntax in
let clients : Cleanup.t Dao.VifMap.t ref = ref Dao.VifMap.empty in
Dao.watch_clients @@ fun new_set ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see above about @@.

@hannesm
Copy link
Member

hannesm commented May 23, 2024

I like the last commit, thanks @dinosaure.

It would make reviewing easier if you could separate syntactic changes (using Lwt.Syntax, using @@) and semantic changes.

I added some comments, and thanks to @palainp for testing this. It is intricate code where we should ensure to test extensively before merging (with adding and removing clients). Maybe I'm also afraid since I spent quite some time on that code, and was sometimes in stuck states... But as said, I reviewed the code and it looks good.

Should we adapt ocamlformat to this repository, so we don't have to argue about indentation, where to put parenthesis and where to put the in?

@palainp
Copy link
Member

palainp commented May 23, 2024

Unfortunately I don't have yet a deep test protocol, all is done manually, and I may miss something, but I'll try to write a testbed script soon :)

Should we adapt ocamlformat to this repository, so we don't have to argue about indentation, where to put parenthesis and where to put the in?

Yes I think so :)

@palainp
Copy link
Member

palainp commented May 23, 2024

I wrote a simple naive script (using 10 AppVM clients with 2G memory) that returned me some issues (certainly present before this PR). When I change the netvm property I have in the logs (maybe a packet is still pending in the ring when the unikernel tries to connect to a new netvm?):

[2024-05-23 10:31:14] 2024-05-23T08:31:14-00:00: [ERROR] [dispatcher] uncaught exception trying to send to uplink: 
[2024-05-23 10:31:14]                                                 Failure("Shared_page_pool.use after shutdown")

I also got in dom0 console:

2024-05-23 10:27:08.738 qrexec-client[58505]: process_io.c:188:process_io: vchan connection closed early (fds: 1 -1 -1, status: -1 -1)

but that's more related to my script to be to eager when connecting/disconnecting clients AppVMs. I'll have to rewrite that before sharing :)

@palainp palainp mentioned this pull request May 23, 2024
6 tasks
@hannesm
Copy link
Member

hannesm commented Oct 15, 2024

I'd like to ask what to do here. I'm happy to rebase this PR and if @palainp tests we merge it!? It'd esp. be wonderful to have it included before the next release, so we get something out of the nice cleanups here :) Unfortunately it conflicts now that #201 was merged.

dinosaure and others added 3 commits October 15, 2024 21:37
…clients map

We currently try to spawn 2 fibers [qubes_updated] and [listener] per clients
and we already finalise them correctly if the client is disconnected. However,
the Lwt.async is localized into add_client instead of where we attach a
finalisers for these tasks. The first objective of this patch is to be sure that
the Lwt.async is near where we registerd cancellation of these tasks.

The second part is to localize the global clients to avoid the ability to
read/write on it somewhere else. Only Dispatcher.watch_clients uses it - so it
corresponds to a free variable of the Dispatcher.watch_clients closure.
@hannesm
Copy link
Member

hannesm commented Oct 15, 2024

I rebased on top of main, cleanup up a whitespace and revised XXX to NOTE.

The leftover comment is about "@@", but a git grep "@@" results in several hits, so let's leave that coding style for a separate PR.

@palainp
Copy link
Member

palainp commented Oct 16, 2024

Thank you! I need to test it, and think this PR will be good to merge!

Copy link
Member

@palainp palainp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@palainp palainp merged commit 56e66ca into mirage:main Oct 16, 2024
2 checks passed
@dinosaure dinosaure deleted the lint branch October 16, 2024 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants