I have been tasked to help our researchers update the application they developed. Specifically, we’re trying to upgrade python (from python 3.6.8).
Since the current application has been developed directly on a server, I am retro-engineering the creation process.
On the current server, I did activate the conda environment, checked python version and dumped the list of requirements:
source activate myenv
python --version
conda list -e > requirements.txt
On my build machine, I have the following Dockerfile:
FROM continuumio/anaconda3:latest
RUN mkdir /app
WORKDIR /app
COPY . .
RUN conda config --add channels conda-forge
RUN conda config --set channel_priority strict
RUN conda env create --name myenv python=3.6.8 --file requirements.txt
CMD ["sleep", "infinity"]
After a long building time (2000+ seconds), the environment creation process fails with a list of conflicting packages.
My questions are the following:
- how comes conda is not able to reproduce an existing environment?
- is there another way to bundle and move the existing environment onto another machine?
- is there a way to speed up the process?
Thanks in advance,
Emmanuel
I suspect the issue here is a dependency conflict between pulled from ‘continuumio/anaconda3’ and packages you are attempting to install from conda-forge. I would try just installing the packages from conda-forge in a new conda environment and then installing the packages from continuumio/anaconda3 in a separate conda environment - this should work… The issue may also be a incompatibility with the version of python you are trying to install.
If you attach the dependency list that is returned when you try the install in the ‘myenv’ environment here, I should be able to provide you a better answer.
Thanks for your answer. The list of dependencies generated by the export is:
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
_tflow_select=2.3.0=mkl
absl-py=0.7.0=py36_0
asn1crypto=0.24.0=py36_0
astor=0.7.1=py36_0
blas=1.0=mkl
blosc=1.15.0=hd408876_0
bzip2=1.0.6=h14c3975_5
c-ares=1.15.0=h7b6447c_1
ca-certificates=2019.1.23=0
cairo=1.14.12=h8948797_3
certifi=2018.11.29=py36_0
cffi=1.12.1=py36h2e261b9_0
chardet=3.0.4=py36_1
click=7.0=py36_0
cryptography=2.5=py36h1ba5d50_0
cython=0.29.5=py36he6710b0_0
decorator=4.3.2=py36_0
ffmpeg=4.0=hcdf2ecd_0
flask=1.0.2=py36_1
fontconfig=2.13.0=h9420a91_0
freeglut=3.0.0=hf484d3e_5
freetype=2.9.1=h8a8886c_1
gast=0.2.2=py36_0
glib=2.56.2=hd408876_0
graphite2=1.3.13=h23475e2_0
grpcio=1.16.1=py36hf8bcb03_1
h5py=2.8.0=py36h989c5e5_3
harfbuzz=1.8.8=hffaf4a1_0
hdf5=1.10.2=hba1933b_1
icu=58.2=h9c2bf20_1
idna=2.8=py36_0
imageio=2.5.0=py36_0
intel-openmp=2019.1=144
itsdangerous=1.1.0=py36_0
jasper=2.0.14=h07fcdf6_1
jinja2=2.10=py36_0
jpeg=9b=h024ee3a_2
libedit=3.1.20181209=hc058e9b_0
libffi=3.2.1=hd88cf55_4
libgcc-ng=8.2.0=hdf63c60_1
libgfortran-ng=7.3.0=hdf63c60_0
libglu=9.0.0=hf484d3e_1
libopencv=3.4.2=hb342d67_1
libopus=1.3=h7b6447c_0
libpng=1.6.36=hbc83047_0
libprotobuf=3.6.1=hd408876_0
libstdcxx-ng=8.2.0=hdf63c60_1
libtiff=4.0.10=h2733197_2
libuuid=1.0.3=h1bed415_2
libvpx=1.7.0=h439df22_0
libxcb=1.13=h1bed415_1
libxml2=2.9.9=he19cac6_0
llvmlite=0.27.0=py36hd408876_0
markdown=3.0.1=py36_0
markupsafe=1.1.1=py36h7b6447c_0
mkl=2018.0.3=1
mkl_fft=1.0.6=py36h7dd41cf_0
mkl_random=1.0.1=py36h4414c95_1
ncurses=6.1=he6710b0_1
networkx=2.2=py36_1
numba=0.42.0=py36h962f231_0
numpy=1.15.4=py36h1d66e8a_0
numpy-base=1.15.4=py36h81de0dd_0
olefile=0.46=py36_0
opencv=3.4.2=py36h6fd60c2_1
openssl=1.1.1b=h7b6447c_0
pandas=0.24.1=py36he6710b0_0
pcre=8.42=h439df22_0
pillow=5.4.1=py36h34e0f95_0
pip=19.0.3=py36_0
pixman=0.36.0=h7b6447c_0
protobuf=3.6.1=py36he6710b0_0
py-opencv=3.4.2=py36hb342d67_1
pycparser=2.19=py36_0
pyopenssl=19.0.0=py36_0
pysocks=1.6.8=py36_0
python=3.6.8=h0371630_0
python-dateutil=2.8.0=py36_0
pytz=2018.9=py36_0
readline=7.0=h7b6447c_5
requests=2.21.0=py36_0
scikit-learn=0.19.1=py36hedc7406_0
scipy=1.1.0=py36hfa4b5c9_1
setuptools=40.8.0=py36_0
six=1.12.0=py36_0
snappy=1.1.7=hbae5bb6_3
sqlalchemy=1.2.18=py36h7b6447c_0
sqlite=3.26.0=h7b6447c_0
tensorboard=1.10.0=py36hf484d3e_0
tensorflow=1.10.0=mkl_py36hdb377fd_0
tensorflow-base=1.10.0=mkl_py36h3c3e929_0
termcolor=1.1.0=py36_1
tk=8.6.8=hbc83047_0
tqdm=4.31.1=py_0
urllib3=1.24.1=py36_0
werkzeug=0.14.1=py36_0
wheel=0.33.1=py36_0
xz=5.2.4=h14c3975_4
zlib=1.2.11=h7b6447c_3
zstd=1.3.7=h0b5b093_0
Were you able to resolve this issue by installing the packages into different environments?