Monday, April 18, 2022

[SOLVED] Installing local packages with Python virtualenv --system-site-packages

Issue

I know this topic has been beat to death but I have not been able to find a solution to the problem I'm having on SO or elsewhere, so I suspect that there may be a bug somewhere in my system.

I am on an older RHEL 6 platform with Python 3.4. I am developing an application that will run on this platform that uses Qt. I've installed all of the relevant libraries via yum (e.g. qt-devel, pyqt4-devel, etc.) and now want to install my application package as an "editable" package using pip install -e mypkg. I also have a couple of dependency requirements that are not on yum and must be installed via pip.

What I would like to do is create a virtualenv that "inherits" the system packages installed via yum but allows me to pip install my own packages into a virtualenv directory in my home directory.

From my Googling it looks like the best way to do this is to create a virtual env with the system's site packages directory:

$ python3 -m venv --system-site-packages ~/venv

However, when I try to install a package to this virtualenv's site-packages directory, it attempts to install it under /usr/lib and I get a Permission denied error.

So it appears that the --system-site-packages option makes my virtualenv completely share the site-packages directory from my system instead of using it as a "base", where further packages can be layered on top.

This answer states that using pip install -I should do what I want, but that does not appear to be the case:

(venv) $ pip3 install -I bitstring
...
error: could not create '/usr/lib/python3.4/site-packages/bitstring.py': Permission denied

Solution

Create the virtual environment without the --system-site-packages switch. After the environment was created go to the folder the environment was created in. It should have a file pyvenv.cfg. Edit this file. It has (among other text) a line

include-system-site-packages = false

Change this line to:

include-system-site-packages = true

Activate the environment. Module installations will now go to the virtual environment and the system site packages are visible too.



Answered By - rfindeis
Answer Checked By - Marilyn (WPSolving Volunteer)