Virtual Pythons

I won’t go into much de­tail here, how­ever, for my par­tic­u­lar setup, other than nix I will be us­ing po­etry to keep track of my de­pen­den­cies.

# Start the project
poetry init
# Define basic things
poetry add ipython jupyterhub ipykernel
# Use it
poetry shell

At this stage you should have ob­tained a shell with the ba­sics of jupyterhub.

Managing Kernels

An im­plicit as­sump­tion here is the rather counter-in­tu­itive but prac­ti­cal is­sue of hav­ing mul­ti­ple in­stances of jupyterhub and even of python. So this sec­tion will make per­fect sense un­til every­one comes to their senses and uses nix:

# List kernels
jupyter kernelspec list

Now we can re­move use­less or out­dated ones with a sim­ple com­mand:

jupyter kernelspec uninstall iHateThisKernel

Anyway, we re­ally want to fig­ure out how to get our shiny new jupyterhub in­stal­la­tion to use our nicely en­cap­su­lated poetry en­vi­ron­ment so let’s get to it.

# Go to directory with pyproject.toml
poetry shell
# Now make sure you get this right
which python
# /home/username/.cache/pypoetry/virtualenvs/something-string-py3.8/bin/python
exit
# Back to the system python
which python
# /usr/bin/python3.8 or whatever
# Register
python -m ipykernel install --user --name=something-string-py3.8
# Now we can select it normally from JupyterHub

An ex­pla­na­tion hinges on the fol­low­ing facts:

• Programs like emacs will nor­mally count on your sys­tem-wide python and thus by ex­ten­sion your sys­tem-wide jupyterhub
• This means when you nor­mally run jupyter lab and play around with things, you’re us­ing your SYSTEM-PYTHON
• However the so-called fix es­sen­tially hinges on the fact that you can reg­is­ter your ker­nel from where-ever

The rea­son why this is dumb is prob­a­bly ob­vi­ous. It does­n’t re­ally solve the is­sue of hav­ing mul­ti­ple pack­ages (at any rate you have du­pli­cated the jupyter pack­age and oth­ers). At the mo­ment though, it makes for a cleaner method than work­ing with just the sys­tem-python. We can still man­age the en­vi­ron­ment of our note­books us­ing the stan­dard poetry work­flow which is a plus.

The best way to fix this is to ei­ther man­age the paths such that only the poetry python is ever vis­i­ble to every­thing, or to just set cus­tom shell en­vi­ron­ments, which will be de­tailed later.

Virtual Jupyter

Another ap­proach is to make sure you only use the jupyter bi­nary to han­dle reg­is­tra­tion and de-reg­is­tra­tion:

poetry shell
which jupyter # Should be the local one
# Remember that this time you need the whole path
jupyter lab

This is now your server which you must point every other ac­cess-mech­a­nism to, like ein. This is a cleaner ap­proach in the sense that you no longer need to keep a sys­tem-wide in­stall of jupyterhub but now you have more re­stric­tions on start­ing the server.