Issue
As far as I know, I believe what conda do about python pkgs is not more than
- manage the python pkg installing path and which path to find python pkgs.
~/miniconda/envs/$(env-name)/lib//$(python-version)/site-packages/...
- Automatically install dependencies of python pkgs in these inter-env-seperate pathes
And I believe:
- If you use
~/miniconda/bin/pip
orconda install
to install python pkgs, all the dependencies will be install into~/miniconda/envs/<env-name>/lib/<python-version>
, and~/miniconda/bin/python
can find them - If a python pkg pyfoo depends on a c++ libary libfoo already installed by
apt install
into/usr/lib/<python-version>
,conda install pyfoo
will install libfoo again into~/miniconda/envs/<env-name>/lib/<python-version>
- If you use
apt
to install python pkgs, the install path will be/usr/lib/<python-version>
and conda can't find them.
My question is:
- Am i right?
- Does
apt install
c++ libs conflicts with any of c++ libs installed into~/miniconda/envs/<env-name>/lib/<python-version>
and verse vice? - Can i just use conda to manage python pkgs in different env and
apt
to manage c++ pkgs without worrying about any conflicts? Which means when i build and run a hybird(c++ and python) project, i just need activate the conda env andconda install
all the python dependencies andapt install
all desired c++ deps, everything will work fine?
Solution
What you say is correct: Conda will almost always bring its own dependencies. The exception to this is using a shell package, but I only know this to exist for mpich
.
Otherwise, the answer to the main question, "Can I bring my own shared libs?", is practically no. This is because Conda packages precompile as much as possible and this often leads to specific symbol references to the exact builds of dynamic libraries. Even getting dynamic libraries from different channels can lead to errors of missing symbols.
However, you could just use Conda to manage environments and still use Pip to install libraries. That is, use Conda to create Python environments, then use Pip to install Python packages. Something like:
conda create -n my-env python=3.9 pip
conda activate my-env
pip install pkg1 pkg2 ...
Note, however, that this loses all the redundancy reduction that Conda does under the hood. It also means that many of your packages will still be compiled locally, whereas Conda precompiles everything, which is why installs are so fast. While more Pip packages are on wheels, those are essentially static builds, meaning even more unnecessary duplication of what should be shared libraries.
Answered By - merv Answer Checked By - Marie Seifert (WPSolving Admin)