Next steps to package and distribute the kernel

[jtpio]

Thinking about the next steps once we get the first version working as a federated extension:

1. we take the xeus lua specific parts out
2. we create a https://github.com/jupyterlite/xeus-kernel repo that other kernels like xeus-lua can depend on. This repo would contain most of the logic
3. jupyterlite-xeus-lua would live in its own repo, for example https://github.com/jupyterlite/xeus-lua-kernel, and would depend on the @jupyterlite/xeus-kernel packages. It would also have the tools (Dockerfile) to create the wasm file, and would specify it as a parameter
4. streamline (or at least document) the process to place the files generated by emscripten so they are picked up by the extension on build
5. Clean up @ts-ignore in the code

[jtpio]

Then we could also check this works well with another xeus kernel such as https://github.com/DerThorsten/xeus-wren (if it can be compiled to wasm)

[DerThorsten]

Then we could also check this works well with another xeus kernel such as https://github.com/DerThorsten/xeus-wren (if it can be compiled to wasm)

It should compile to wasm, it is very simple plain C (this is why I choose this lang.)

[DerThorsten]

3. jupyterlite-xeus-lua would live in its own repo, for example https://github.com/jupyterlite/xeus-lua-kernel, and would depend on the @jupyterlite/xeus-kernel packages. It would also have the tools (Dockerfile) to create the wasm file, and would specify it as a parameter

Can you elaborate what you mean with "specify it as a parameter".

So https://github.com/jupyterlite/xeus-lua-kernel and https://github.com/jupyterlite/xeus-wren-kernel would each contain their own Dockerfile but the logic to trigger the Dockerbuild is in https://github.com/jupyterlite/xeus-kernel ?

[DerThorsten]

btw the wren kernel is atm in https://github.com/DerThorsten/jupyterlite_xeus_wren

[jtpio]

If all we need is the name of the path to the wasm file, it could be passed in the instantiation options here:

https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/c9d7419fdc423a41eae86305a316e4a7bc6e2cb7/src/index.ts#L38-L40

[jtpio]

The xeus-lua and xeus-wren packages would then only have to define a plugin like this one:

https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/c9d7419fdc423a41eae86305a316e4a7bc6e2cb7/src/index.ts#L13-L44

[DerThorsten]

unfortunately there is a bit more : the dockerfile is different:

• the dockerfile needs to build lua / wren for wasm and this is different
• xeus-lua needs xwidgets and xproperties in the dockerfile, xeus-wren does not
  Besides that It should really be just import names

[jtpio]

Right, the Dockerfiles would also be different and live in their respective repos.

[jtpio]

Before tackling the steps above, we could actually make a first release of jupyterlite-xeus-lua. The repo is looking pretty good so this might happen very soon.

That would be a good way to make sure we can use the releaser for that, check all files are correctly bundled (#18 already helps with this) and setup the process for the next releases.

@DerThorsten I've added you as owner on https://pypi.org/project/jupyterlite-xeus-lua/

We can use the Jupyter Releaser to publish the extension to PyPI and npm: https://jupyter-releaser.readthedocs.io/en/latest/get_started/making_first_release.html#set-up

[DerThorsten]

@jtpio I'll try to start with moving the common parts of the jupyterlite xeus kernel to https://github.com/jupyterlite/xeus-kernel

[DerThorsten]

@jtpio out of curiosity, how would you approach this?
Start by copy pasting the jupyterlite-xeus-lua code or from a fresh cookiecutter generated instance?

[jtpio]

It would be fine to copy what we have. Also https://github.com/jupyterlite/xeus-kernel will be a JavaScript dependency only so we don't need to copy the Python files.

[DerThorsten]

@jtpio I have a bit trouble setting the things up st I can import the local xeus-kernel-(base) code from jupyterlite-xeus-lua.

I was trying to add the local package to jupyterlite-xeus-lua via

yarn add $HOME/xeus-kernel 

but then the import does not work.

import {XeusServerKernelBase} from 'xeus-kernel'

Any suggestion on setting up the dev environment st. I can use xeus-kernel-(base) in the jupyterlute-xeus-lua are appreciated

[jtpio]

Right that is often an issue when developing locally and the package has not been published yet. iirc handling of linked packages was fixed in a newer version of yarn.

Just to make, is xeus-kernel built? (files generated in lib)

Otherwise a trick is to use a local npm registry like verdaccio, and publish the package there (being careful to setup the .npmrc correctly to not publish to the main npm registry).

[DerThorsten]

@jtpio thanks! indeed the lib was missing (sry for the noob questions)

I guess the next tricky part will be having a dynamic import in the xeus base worker =)

[DerThorsten]

@jtpio https://github.com/jupyterlite/xeus-kernel/tree/experiments this is where the common code is

[jtpio]

@jtpio https://github.com/jupyterlite/xeus-kernel/tree/experiments this is where the common code is

Thanks @DerThorsten for starting this!

We can go ahead and release jupyterlite-xeus-lua as is for now. And then proceed with the refactor later.

Would you mind making me an admin of this repo, and I'll move it to the jupyterlite org: https://github.com/jupyterlite
After the move I plan to do a small pass to fix the links, and then cut the release.

Thanks!

[DerThorsten]

@jtpio you already have access to the repo,...cannot find the menue to give you more premissions.

[DerThorsten]

And I cannot do it "You don’t have the permission to create public repositories on jupyterlite"

[jtpio]

Ah right Admin is probably just for organizations, this repo is under a private account.

@DerThorsten I made you owner of the org so you can do the transfer. Thanks!

[DerThorsten]

done

[jtpio]

Nice, thanks!

I'll proceed with a couple of renaming for consistency. Followed by a release.