Calling Python from C - error loading shared libraries

Technical Topics
1

Disclaimer: this post is very similar to a question I asked on StackOverflow a week ago.

I’m using Anaconda as a package manager for a project. I’m trying to call a Python function from C, but when I try to run the compiled program, I get the following error:

~/Documents/code/test: error while loading shared libraries: libpython3.9.so.1.0: cannot open shared object file: No such file or directory
[1] + Done                       "/usr/bin/gdb" --interpreter=mi --tty=${DbgTerm} 0<"/tmp/Microsoft-MIEngine-In-cdpcz1c5.vwd" 1>"/tmp/Microsoft-MIEngine-Out-j5b1thxt.2vz"

I’m including the library ${HOME}/anaconda3/envs/myenv/lib and there is a libpython3.9.so.1.0 file within this directory. When I use the command: find ~/anaconda3/envs/myenv/ -name libpython3.9.so.* , I get this result:

~/anaconda3/envs/myenv/lib/libpython3.9.so.1.0

The build command I’m using is:

/usr/bin/gcc -I${HOME}/Documents/git/resist/code \ 
    -I/usr/include/glib-2.0 \
    -I/usr/lib/x86_64-linux-gnu/glib-2.0/include \
    -I${HOME}/anaconda3/envs/myenv/include/python3.9 \
    -I${HOME}/anaconda3/envs/myenv/lib/python3.9/site-packages/numpy/core/include/numpy \
    -Wno-unused-result \
    -Wsign-compare \
    -march=nocona \
    -mtune=haswell \
    -ftree-vectorize \
    -fPIC \
    -fstack-protector-strong \
    -fno-plt \
    -O3 \
    -ffunction-sections \
    -pipe \
    -isystem ${HOME}/anaconda3/envs/myenv/include \
    -fdebug-prefix-map=/opt/conda/conda-bld/python-split_1649141344976/work=/usr/local/src/conda/python-3.9.12 \
    -fdebug-prefix-map=${HOME}/anaconda3/envs/myenv=/usr/local/src/conda-prefix \
    -I${HOME}/anaconda3/envs/myenv/include \
    -DNDEBUG \
    -O3 \
    -Wall ${HOME}/Documents/git/resist/code/test.c \
    -L${HOME}/anaconda3/envs/myenv/lib/python3.9/config-3.9-x86_64-linux-gnu \
    -L${HOME}/anaconda3/envs/myenv/lib \
    -lcrypt \
    -lpthread \
    -ldl \
    -lutil \
    -lm \
    -lpython3.9 \
    -Xlinker \
    -export-dynamic \
    -o ${HOME}/Documents/git/resist/code/test

I can’t tell whether this is a problem with how I’m grabbing the Python library (i.e., -lpython3.9) or something else. I pulled most of these commands from python3.9-config --cflags and python3.9-config --ldflags after I activated myenv:

(base) ~$ conda activate myenv

(myenv) ~$ python3.9-config --cflags
-I${HOME}/anaconda3/envs/myenv/include/python3.9 -I${HOME}/anaconda3/envs/myenv/include/python3.9  -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -ffunction-sections -pipe -isystem ${HOME}/anaconda3/envs/myenv/include -fdebug-prefix-map=/opt/conda/conda-bld/python-split_1649141344976/work=/usr/local/src/conda/python-3.9.12 -fdebug-prefix-map=${HOME}/anaconda3/envs/myenv=/usr/local/src/conda-prefix -I${HOME}/anaconda3/envs/myenv/include -DNDEBUG  -O3 -Wall

(myenv) ~$ python3.9-config --ldflags
-L${HOME}/anaconda3/envs/myenv/lib/python3.9/config-3.9-x86_64-linux-gnu -L${HOME}/anaconda3/envs/myenv/lib  -lcrypt -lpthread -ldl  -lutil -lm -lm
1 Like
2

Hello,

I think you need to set the environment variable ‘LD_LIBRARY_PATH’ in your linux environment so that it can find your python library, since it doesn’t exist in the default location for C libraries.

Try setting: export LD_LIBRARY_PATH=/anaconda3/envs/myenv

You may also need to add to the existing setting of this env variable.

Then try rerunning your code.

2 Likes
3

OK, so I set the LD_LIBRARY_PATH environment variable in my environment.

~$ export LD_LIBRARY_PATH=~/anaconda3/envs/myenv

~$ echo  $LD_LIBRARY_PATH
${HOME}/anaconda3/envs/myenv

I then built the C program as shown in the OP. After it built, I tried to run the program:

~$ ./test
./test: error while loading shared libraries: libpython3.9.so.1.0: cannot open shared object file: No such file or directory

So, it appears that I’m still getting this error. Should I set this variable in my .bashrc file?

1 Like
4

The fix is to use the lib folder within the virtual environment.

~$ export LD_LIBRARY_PATH=~/anaconda3/envs/myenv/lib

This solved the problem (of course, now I’m getting a seg fault…)

1 Like