@davidrpugh Hi!
I am facing hard time installing horovod using yml file
OS:
β¦
```
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal
```
```
which nvcc
/cephfs/local/cuda-11.0/bin/nvcc
```
```
module load cuda/11.0
export CUDA_HOME=/usr/local/
export HOROVOD_CUDA_HOME=$CUDA_HOME
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
export HOROVOD_NCCL_LINK=SHARED
export HOROVOD_NCCL_INCLUDE=/usr/include
export HOROVOD_NCCL_LIB=/usr/lib/x86_64-linux-gnu
export HOROVOD_GPU_OPERATIONS=NCCL
export HOROVOD_WITH_MPI=1 HOROVOD_WITH_MXNET=1 HOROVOD_GPU_OPERATIONS=1 HOROVOD_WITH_TENSORFLOW=1 HOROVOD_WITH_PYTORCH=1 HOROVOD_GPU_OPERATIONS=NCCL HOROVOD_WITH_GLOO=1
export ENV_PREFIX=$PWD/env
conda env create --prefix $ENV_PREFIX --file horovod.yml --force
```
This is the yml file
```
name: adel
channels:
- pytorch
- conda-forge
- defaults
dependencies:
- ccache
- cmake
- cudatoolkit=11.0
- cudnn=8.0
- cxx-compiler
- mpi4py # installs cuda-aware openmpi
- nccl
- nvcc_linux-64=11.0
- openmpi
- jupyterlab
- pip
- pip:
- tensorflow-gpu==2.4.*
- -r requirements.txt
- python=3.8
- tensorboard=2.4
- torchaudio=0.9.0
- torchvision=0.10.0
- numpy
- tqdm
- sentencepiece=0.1.96
- pytorch=1.9
- tokenizers=0.10.3
- prettytable=2.2.1
- einops=0.3.2
- matplotlib
- scikit-learn
- pandas
```
this is the requirements filr requirements.txt
```
horovod==0.22.1
transformers==4.8.2
datasets==1.8.0
jupyterlab-nvdashboard
huggingface-hub==0.0.12
jupyter-tensorboard
--no-binary=horovod
--no-binary=datasets
--no-binary=huggingface-hub
```
I am getting this error:
```
Successfully built bokeh wrapt
Installing collected packages: wrapt, typing-extensions, termcolor, tensorflow-estimator, simpervisor, regex, flatbuffers, xxhash, tqdm, six, pyyaml, pynvml, psutil, numpy, gast, fsspec, filelock, dill,
cloudpickle, sacremoses, pyarrow, opt-einsum, multiprocess, keras-preprocessing, huggingface-hub, horovod, h5py, grpcio, google-pasta, astunparse, absl-py, transformers, bokeh, ipython, datasets, tensorf
low-gpu, jupyter-tensorboard, jupyter-server-proxy, jupyterlab-nvdashboard
Attempting uninstall: typing-extensions
Found existing installation: typing_extensions 4.1.1
Uninstalling typing_extensions-4.1.1:
Successfully uninstalled typing_extensions-4.1.1
Attempting uninstall: tqdm
Found existing installation: tqdm 4.62.3
Uninstalling tqdm-4.62.3:
Successfully uninstalled tqdm-4.62.3
Attempting uninstall: six
Found existing installation: six 1.16.0
Uninstalling six-1.16.0:
Successfully uninstalled six-1.16.0
Attempting uninstall: numpy
Found existing installation: numpy 1.22.2
Uninstalling numpy-1.22.2:
Successfully uninstalled numpy-1.22.2
Running setup.py install for huggingface-hub: started
Running setup.py install for huggingface-hub: finished with status 'done'
Running setup.py install for horovod: started
Running setup.py install for horovod: finished with status 'error'
| Pip subprocess error:
error: subprocess-exited-with-error
Γ Running setup.py install for horovod did not run successfully.
β exit code: 1
β°β> [252 lines of output]
/home/arij/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/horovod
copying horovod/__init__.py -> build/lib.linux-x86_64-3.8/horovod
creating build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/gradient_aggregation_eager.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/compression.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/mpi_ops.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/functions.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/__init__.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/sync_batch_norm.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/util.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/elastic.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
copying horovod/tensorflow/gradient_aggregation.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow
creating build/lib.linux-x86_64-3.8/horovod/data
copying horovod/data/__init__.py -> build/lib.linux-x86_64-3.8/horovod/data
copying horovod/data/data_loader_base.py -> build/lib.linux-x86_64-3.8/horovod/data
creating build/lib.linux-x86_64-3.8/horovod/spark
copying horovod/spark/runner.py -> build/lib.linux-x86_64-3.8/horovod/spark
copying horovod/spark/mpi_run.py -> build/lib.linux-x86_64-3.8/horovod/spark
copying horovod/spark/conf.py -> build/lib.linux-x86_64-3.8/horovod/spark
copying horovod/spark/__init__.py -> build/lib.linux-x86_64-3.8/horovod/spark
copying horovod/spark/gloo_run.py -> build/lib.linux-x86_64-3.8/horovod/spark
creating build/lib.linux-x86_64-3.8/horovod/_keras
copying horovod/_keras/__init__.py -> build/lib.linux-x86_64-3.8/horovod/_keras
copying horovod/_keras/elastic.py -> build/lib.linux-x86_64-3.8/horovod/_keras
copying horovod/_keras/callbacks.py -> build/lib.linux-x86_64-3.8/horovod/_keras
creating build/lib.linux-x86_64-3.8/horovod/torch
copying horovod/torch/compression.py -> build/lib.linux-x86_64-3.8/horovod/torch
copying horovod/torch/mpi_ops.py -> build/lib.linux-x86_64-3.8/horovod/torch
copying horovod/torch/optimizer.py -> build/lib.linux-x86_64-3.8/horovod/torch
copying horovod/torch/functions.py -> build/lib.linux-x86_64-3.8/horovod/torch
copying horovod/torch/__init__.py -> build/lib.linux-x86_64-3.8/horovod/torch
copying horovod/torch/sync_batch_norm.py -> build/lib.linux-x86_64-3.8/horovod/torch
creating build/lib.linux-x86_64-3.8/horovod/runner
copying horovod/runner/mpi_run.py -> build/lib.linux-x86_64-3.8/horovod/runner
copying horovod/runner/task_fn.py -> build/lib.linux-x86_64-3.8/horovod/runner
copying horovod/runner/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner
copying horovod/runner/js_run.py -> build/lib.linux-x86_64-3.8/horovod/runner
copying horovod/runner/run_task.py -> build/lib.linux-x86_64-3.8/horovod/runner
copying horovod/runner/gloo_run.py -> build/lib.linux-x86_64-3.8/horovod/runner
copying horovod/runner/launch.py -> build/lib.linux-x86_64-3.8/horovod/runner
creating build/lib.linux-x86_64-3.8/horovod/keras
copying horovod/keras/__init__.py -> build/lib.linux-x86_64-3.8/horovod/keras
copying horovod/keras/elastic.py -> build/lib.linux-x86_64-3.8/horovod/keras
copying horovod/keras/callbacks.py -> build/lib.linux-x86_64-3.8/horovod/keras
copying horovod/keras/callbacks.py -> build/lib.linux-x86_64-3.8/horovod/keras
creating build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/runner.py -> build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/ray_logger.py -> build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/driver_service.py -> build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/strategy.py -> build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/__init__.py -> build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/worker.py -> build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/utils.py -> build/lib.linux-x86_64-3.8/horovod/ray
copying horovod/ray/elastic.py -> build/lib.linux-x86_64-3.8/horovod/ray
creating build/lib.linux-x86_64-3.8/horovod/mxnet
copying horovod/mxnet/mpi_ops.py -> build/lib.linux-x86_64-3.8/horovod/mxnet
copying horovod/mxnet/functions.py -> build/lib.linux-x86_64-3.8/horovod/mxnet
copying horovod/mxnet/__init__.py -> build/lib.linux-x86_64-3.8/horovod/mxnet
creating build/lib.linux-x86_64-3.8/horovod/common
copying horovod/common/basics.py -> build/lib.linux-x86_64-3.8/horovod/common
copying horovod/common/__init__.py -> build/lib.linux-x86_64-3.8/horovod/common
copying horovod/common/exceptions.py -> build/lib.linux-x86_64-3.8/horovod/common
copying horovod/common/util.py -> build/lib.linux-x86_64-3.8/horovod/common
copying horovod/common/elastic.py -> build/lib.linux-x86_64-3.8/horovod/common
creating build/lib.linux-x86_64-3.8/horovod/tensorflow/keras
copying horovod/tensorflow/keras/__init__.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow/keras
copying horovod/tensorflow/keras/elastic.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow/keras
copying horovod/tensorflow/keras/callbacks.py -> build/lib.linux-x86_64-3.8/horovod/tensorflow/keras
creating build/lib.linux-x86_64-3.8/horovod/spark/lightning
copying horovod/spark/lightning/legacy.py -> build/lib.linux-x86_64-3.8/horovod/spark/lightning
copying horovod/spark/lightning/__init__.py -> build/lib.linux-x86_64-3.8/horovod/spark/lightning
copying horovod/spark/lightning/util.py -> build/lib.linux-x86_64-3.8/horovod/spark/lightning
copying horovod/spark/lightning/estimator.py -> build/lib.linux-x86_64-3.8/horovod/spark/lightning
copying horovod/spark/lightning/remote.py -> build/lib.linux-x86_64-3.8/horovod/spark/lightning
creating build/lib.linux-x86_64-3.8/horovod/spark/driver
copying horovod/spark/driver/mpirun_rsh.py -> build/lib.linux-x86_64-3.8/horovod/spark/driver
copying horovod/spark/driver/job_id.py -> build/lib.linux-x86_64-3.8/horovod/spark/driver
copying horovod/spark/driver/driver_service.py -> build/lib.linux-x86_64-3.8/horovod/spark/driver
copying horovod/spark/driver/host_discovery.py -> build/lib.linux-x86_64-3.8/horovod/spark/driver
copying horovod/spark/driver/__init__.py -> build/lib.linux-x86_64-3.8/horovod/spark/driver
copying horovod/spark/driver/rendezvous.py -> build/lib.linux-x86_64-3.8/horovod/spark/driver
copying horovod/spark/driver/rsh.py -> build/lib.linux-x86_64-3.8/horovod/spark/driver
creating build/lib.linux-x86_64-3.8/horovod/spark/torch
copying horovod/spark/torch/__init__.py -> build/lib.linux-x86_64-3.8/horovod/spark/torch
copying horovod/spark/torch/util.py -> build/lib.linux-x86_64-3.8/horovod/spark/torch
copying horovod/spark/torch/estimator.py -> build/lib.linux-x86_64-3.8/horovod/spark/torch
copying horovod/spark/torch/remote.py -> build/lib.linux-x86_64-3.8/horovod/spark/torch
creating build/lib.linux-x86_64-3.8/horovod/spark/data_loaders
copying horovod/spark/data_loaders/__init__.py -> build/lib.linux-x86_64-3.8/horovod/spark/data_loaders
copying horovod/spark/data_loaders/pytorch_data_loaders.py -> build/lib.linux-x86_64-3.8/horovod/spark/data_loaders
creating build/lib.linux-x86_64-3.8/horovod/spark/keras
copying horovod/spark/keras/optimizer.py -> build/lib.linux-x86_64-3.8/horovod/spark/keras
copying horovod/spark/keras/__init__.py -> build/lib.linux-x86_64-3.8/horovod/spark/keras
copying horovod/spark/keras/bare.py -> build/lib.linux-x86_64-3.8/horovod/spark/keras
copying horovod/spark/keras/tensorflow.py -> build/lib.linux-x86_64-3.8/horovod/spark/keras
copying horovod/spark/task/mpirun_exec_fn.py -> build/lib.linux-x86_64-3.8/horovod/spark/task [115/1673]
copying horovod/spark/task/task_info.py -> build/lib.linux-x86_64-3.8/horovod/spark/task
copying horovod/spark/task/task_service.py -> build/lib.linux-x86_64-3.8/horovod/spark/task
creating build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/backend.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/store.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/__init__.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/cache.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/params.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/util.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/_namedtuple_fix.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/serialization.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/estimator.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
copying horovod/spark/common/constants.py -> build/lib.linux-x86_64-3.8/horovod/spark/common
creating build/lib.linux-x86_64-3.8/horovod/torch/elastic
copying horovod/torch/elastic/state.py -> build/lib.linux-x86_64-3.8/horovod/torch/elastic
copying horovod/torch/elastic/__init__.py -> build/lib.linux-x86_64-3.8/horovod/torch/elastic
copying horovod/torch/elastic/sampler.py -> build/lib.linux-x86_64-3.8/horovod/torch/elastic
creating build/lib.linux-x86_64-3.8/horovod/torch/mpi_lib_impl
copying horovod/torch/mpi_lib_impl/__init__.py -> build/lib.linux-x86_64-3.8/horovod/torch/mpi_lib_impl
creating build/lib.linux-x86_64-3.8/horovod/torch/mpi_lib
copying horovod/torch/mpi_lib/__init__.py -> build/lib.linux-x86_64-3.8/horovod/torch/mpi_lib
creating build/lib.linux-x86_64-3.8/horovod/runner/http
copying horovod/runner/http/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/http
copying horovod/runner/http/http_server.py -> build/lib.linux-x86_64-3.8/horovod/runner/http
copying horovod/runner/http/http_client.py -> build/lib.linux-x86_64-3.8/horovod/runner/http
creating build/lib.linux-x86_64-3.8/horovod/runner/driver
copying horovod/runner/driver/driver_service.py -> build/lib.linux-x86_64-3.8/horovod/runner/driver
copying horovod/runner/driver/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/driver
creating build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/registration.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/rendezvous.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/discovery.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/worker.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/settings.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/constants.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
copying horovod/runner/elastic/driver.py -> build/lib.linux-x86_64-3.8/horovod/runner/elastic
creating build/lib.linux-x86_64-3.8/horovod/runner/util
copying horovod/runner/util/threads.py -> build/lib.linux-x86_64-3.8/horovod/runner/util
copying horovod/runner/util/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/util
copying horovod/runner/util/cache.py -> build/lib.linux-x86_64-3.8/horovod/runner/util
copying horovod/runner/util/network.py -> build/lib.linux-x86_64-3.8/horovod/runner/util
copying horovod/runner/util/streams.py -> build/lib.linux-x86_64-3.8/horovod/runner/util
copying horovod/runner/util/lsf.py -> build/lib.linux-x86_64-3.8/horovod/runner/util
copying horovod/runner/util/remote.py -> build/lib.linux-x86_64-3.8/horovod/runner/util
creating build/lib.linux-x86_64-3.8/horovod/runner/task
copying horovod/runner/task/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/task
copying horovod/runner/task/task_service.py -> build/lib.linux-x86_64-3.8/horovod/runner/task
creating build/lib.linux-x86_64-3.8/horovod/runner/common
copying horovod/runner/common/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/common
creating build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/config_parser.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/tiny_shell_exec.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/codec.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/secret.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/safe_shell_exec.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/env.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/timeout.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/network.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/host_hash.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/settings.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
copying horovod/runner/common/util/hosts.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/util
creating build/lib.linux-x86_64-3.8/horovod/runner/common/service
copying horovod/runner/common/service/driver_service.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/service
copying horovod/runner/common/service/__init__.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/service
copying horovod/runner/common/service/task_service.py -> build/lib.linux-x86_64-3.8/horovod/runner/common/service
running build_ext
-- The CXX compiler identification is GNU 10.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Build architecture flags: -mf16c -mavx -mfma
-- Using command /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/bin/python
-- Found MPI_CXX: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Found CUDA: /usr/local/cuda-11.0 (found version "11.0")
-- Found NCCL: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/include
-- Determining NCCL version from the header file: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/include/nccl.h
-- NCCL_MAJOR_VERSION: 2
-- Found NCCL (include: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/include, library: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/libnccl.so)
-- Found NVTX: /cephfs/local/cuda-11.0/include
-- Found NVTX (include: /cephfs/local/cuda-11.0/include, library: dl)
-- The C compiler identification is GNU 10.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found MPI_C: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- MPI include path: /cephfs/local/mpi/hpcx-v2.5.0-gcc-MLNX_OFED_LINUX-4.7-1.0.0.1-ubuntu18.04-x86_64/hpcx-rebuild/include/cephfs/local/mpi/hpcx-v2.5.0-gcc-MLNX_OFED_LINUX-4.7-1.0.0.1-ubuntu18.04-x
86_64/hpcx-rebuild/include/openmpi/cephfs/local/mpi/hpcx-v2.5.0-gcc-MLNX_OFED_LINUX-4.7-1.0.0.1-ubuntu18.04-x86_64/hpcx-rebuild/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include/cephfs/local/mpi/hpcx
-v2.5.0-gcc-MLNX_OFED_LINUX-4.7-1.0.0.1-ubuntu18.04-x86_64/hpcx-rebuild/include/openmpi/opal/mca/event/libevent2022/libevent/cephfs/local/mpi/hpcx-v2.5.0-gcc-MLNX_OFED_LINUX-4.7-1.0.0.1-ubuntu18.04-x86_6
4/hpcx-rebuild/include/openmpi/opal/mca/event/libevent2022/libevent/include/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/include
-- MPI libraries: /home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/libmpi.so
CMake Error at /cephfs/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Tensorflow (missing: Tensorflow_LIBRARIES) (Required is at
least version "1.15.0")
Call Stack (most recent call first):
/cephfs/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
cmake/Modules/FindTensorflow.cmake:31 (find_package_handle_standard_args)
horovod/tensorflow/CMakeLists.txt:12 (find_package)
-- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-ra65jfix/horovod_7104a469b03946038075e0e695e7f1c9/build/temp.linux-x86_64-3.8/RelWithDebInfo/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-ra65jfix/horovod_7104a469b03946038075e0e695e7f1c9/setup.py", line 155, in <module>
setup(name='horovod',
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/command/install.py", line 68, in run
return orig.install.run(self)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/command/install.py", line 610, in run
self.run_command('build')
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/arij/.local/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 341, in run
self.build_extensions()
File "/tmp/pip-install-ra65jfix/horovod_7104a469b03946038075e0e695e7f1c9/setup.py", line 94, in build_extensions
subprocess.check_call([cmake_bin, self.extensions[0].cmake_lists_dir] + cmake_args,
File "/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/tmp/pip-install-ra65jfix/horovod_7104a469b03946038075e0e695e7f1c9', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELWIT
HDEBINFO=/tmp/pip-install-ra65jfix/horovod_7104a469b03946038075e0e695e7f1c9/build/lib.linux-x86_64-3.8', '-DPYTHON_EXECUTABLE:FILEPATH=/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/bi
n/python']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
Γ Encountered error while trying to install package.
β°β> horovod
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
failed
CondaEnvException: Pip failed
```