Incorrect package versions for environment in Jupyter

I created an environment using the Anaconda prompt with a few packages downgraded to match a previous run of a Jupyter notebook. I then used ipykernel to make this environment a selectable kernel in Jupyter.

When I check the package versions for the environment in Anaconda Navigator, they are correct, but with the kernel selected in my Jupyter notebook, when I run a cell that prints the package versions, it shows the versions in the base environment.

I assume this means my Jupyter notebook is using the package versions from the base environment - if so, what have I missed in my process to make what I see in Navigator match what I get in my notebook?

3 Likes

I run a cell that prints the package versions

It will likely depend on how exactly you do this.

One simple way to ensure you run pip from the same environment as the current python might look like

import pip
pip.main(["list"])

If you want conda’s opinion on packages, then there are a few things at play. Typically, you do not have conda (he application) installed in an environment, it only lives in base. That’s fine when you run from a command line, as an environment will be “activated”. Navigator, in contrast, knows how to tell conda which environment it wants to work on without activating. So to call conda from within a kernel not in the currently active environment, you should explicitly pass the environment:

!conda list -n myenv
2 Likes

Thank you for the info! I was actually only running a few print commands like this:

print(f"Scikit-Learn version {sklearn.version}") (The post formatting is removing the underscores around “version”, but you get what I mean)

When I try your first suggestion, I get the same results as my print statements for those packages. When I run the 2nd with my environment name, I see my intended package versions instead. But that appears to mean that I’m not actually running in my environment, even though I’ve selected that kernel. I guess my real question now is how do I run in that environment if selecting that kernel isn’t enough?

Appreciate your help

You should check the value of sys.executable, to see which python is running. It should be like “…/conda/envs/myenv/bin/python”. This is in posix format, it will look a little different for Windows, but “myenv” should be in there.

1 Like

Hmm, yes, that gives me the location “…\ProgramData\Anaconda3\python.exe”. That Anaconda3 folder contains a subfolder called “envs” which has the environment I’ve created, but apparently the python.exe that’s sitting in there isn’t being used. Not sure why

Yep sorry, beyond suggestions here. I’m not sure how ipykernel finds the python it wants to execute. Perhaps you need nb_conda_kernels

1 Like

Hi Mike, and welcome! I have just tried the steps in this article and was successful. I suspect that maybe you executed the command python -m ipykernel install --user --name=my-env from the base interpreter, rather than from your conda environment.

On my system, I have my base interpreter, and then another environment called my-env. I ran the command above, once from within my environment (i.e. after conda activate my-env) and once from within the base environment.

In the figure below, my-env was created from the conda environment, my-other-env was created from the base environment. Despite having custom names as Jupyter kernels, they are each associated with the version of python used when running the ipykernel command.

I think you can explicitly pass the path to python.exe on Windows to ensure this behavior, but I would first confirm you have used conda activate my-env first in either case.

Hope that helps!

2 Likes

Thank you! This definitely appears to have addressed that issue. It looks like I have a bit more work to do on the environment itself (getting an error when I try to run the very basic command “import pandas as pd” lol), but the package list has all the right versions now. Really appreciate the help!

Glad I could help!

Similar to @MARTIN_DURANT 's suggestion above, you should be able to install dependencies from within that kernel using

!conda install pandas
1 Like