Issue
I'm having trouble installing the following packages in a new python 3.9.7 virtual environment on Arch Linux.
My requirements.txt file:
joblib
python-dotenv
azure-cli==2.29.2
azureml-core
Steps to reproduce:
python3 -m venv venv
source venv/bin/activate
pip install -U pip
pip install -r requirements.txt
Here's the relevant part of the output of the last command, it errors when trying to install ruamel.yaml
:
Using legacy 'setup.py install' for ruamel.yaml, since package 'wheel' is not installed.
Installing collected packages: pycparser, urllib3, idna, chardet, cffi, certifi, six, requests, PyJWT, oauthlib, cryptography, requests-oauthlib, python-dateutil, isodate, azure-nspkg, wrapt, tabulate, pyyaml, PySocks, pyopenssl, PyNaCl, pygments, portalocker, msrest, msal, MarkupSafe, jmespath, colorama, bcrypt, azure-mgmt-nspkg, azure-core, argcomplete, applicationinsights, adal, websocket-client, vsts, pyparsing, pyasn1, psutil, pkginfo, pathlib2, paramiko, msrestazure, msal-extensions, knack, jinja2, jeepney, invoke, humanfriendly, deprecated, backports.weakref, azure-mgmt-datalake-nspkg, azure-mgmt-core, azure-common, azure-cli-telemetry, xmltodict, sshtunnel, semver, SecretStorage, scp, ruamel.yaml, pytz, PyGithub, pathspec, packaging, ndg-httpsclient, jsonpickle, jsondiff, javaproperties, fabric, docker, distro, contextlib2, backports.tempfile, azure-synapse-spark, azure-synapse-managedprivateendpoints, azure-synapse-artifacts, azure-synapse-accesscontrol, azure-storage-common, azure-multiapi-storage, azure-mgmt-web, azure-mgmt-trafficmanager, azure-mgmt-synapse, azure-mgmt-storage, azure-mgmt-sqlvirtualmachine, azure-mgmt-sql, azure-mgmt-signalr, azure-mgmt-servicefabricmanagedclusters, azure-mgmt-servicefabric, azure-mgmt-servicebus, azure-mgmt-security, azure-mgmt-search, azure-mgmt-resource, azure-mgmt-reservations, azure-mgmt-relay, azure-mgmt-redis, azure-mgmt-redhatopenshift, azure-mgmt-recoveryservicesbackup, azure-mgmt-recoveryservices, azure-mgmt-rdbms, azure-mgmt-privatedns, azure-mgmt-policyinsights, azure-mgmt-network, azure-mgmt-netapp, azure-mgmt-msi, azure-mgmt-monitor, azure-mgmt-media, azure-mgmt-marketplaceordering, azure-mgmt-maps, azure-mgmt-managementgroups, azure-mgmt-managedservices, azure-mgmt-loganalytics, azure-mgmt-kusto, azure-mgmt-keyvault, azure-mgmt-iothubprovisioningservices, azure-mgmt-iothub, azure-mgmt-iotcentral, azure-mgmt-imagebuilder, azure-mgmt-hdinsight, azure-mgmt-extendedlocation, azure-mgmt-eventhub, azure-mgmt-eventgrid, azure-mgmt-dns, azure-mgmt-devtestlabs, azure-mgmt-deploymentmanager, azure-mgmt-datamigration, azure-mgmt-datalake-store, azure-mgmt-datalake-analytics, azure-mgmt-databoxedge, azure-mgmt-cosmosdb, azure-mgmt-containerservice, azure-mgmt-containerregistry, azure-mgmt-containerinstance, azure-mgmt-consumption, azure-mgmt-compute, azure-mgmt-cognitiveservices, azure-mgmt-cdn, azure-mgmt-botservice, azure-mgmt-billing, azure-mgmt-batchai, azure-mgmt-batch, azure-mgmt-authorization, azure-mgmt-applicationinsights, azure-mgmt-appconfiguration, azure-mgmt-apimanagement, azure-mgmt-advisor, azure-loganalytics, azure-keyvault-keys, azure-keyvault-administration, azure-keyvault, azure-identity, azure-graphrbac, azure-functions-devops-build, azure-datalake-store, azure-cosmos, azure-cli-core, azure-batch, azure-appconfiguration, antlr4-python3-runtime, python-dotenv, joblib, azureml-core, azure-cli
Running setup.py install for ruamel.yaml: started
Running setup.py install for ruamel.yaml: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-8y_97owt/install-record.txt --single-version-externally-managed --compile --install-headers /home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include/site/python3.9/ruamel.yaml
cwd: /tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/
Complete output (649 lines):
sys.argv ['/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py', 'install', '--record', '/tmp/pip-record-8y_97owt/install-record.txt', '--single-version-externally-managed', '--compile', '--install-headers', '/home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include/site/python3.9/ruamel.yaml']
test compiling test_ruamel_yaml
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/ruamel
copying .ruamel/__init__.py -> build/lib.linux-x86_64-3.9/ruamel
creating build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./main.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./util.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./tokens.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarfloat.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scanner.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./comments.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./loader.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./serializer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./resolver.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./nodes.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./dumper.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./configobjwalker.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./timestamp.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./error.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./reader.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./cyaml.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./parser.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./__init__.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./representer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./events.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarbool.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./composer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarint.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./emitter.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./anchor.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./compat.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./constructor.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarstring.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./LICENSE -> build/lib.linux-x86_64-3.9/ruamel/yaml
running build_ext
building '_ruamel_yaml' extension
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/ext
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fPIC -I/home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include -I/usr/include/python3.9 -c ext/_ruamel_yaml.c -o build/temp.linux-x86_64-3.9/ext/_ruamel_yaml.o
In file included from ext/_ruamel_yaml.c:523:
ext/_ruamel_yaml.h:10: warning: "PyString_CheckExact" redefined
<a whole bunch of compiling errors & warnings>
The full log file can be found here
Note the line Using legacy 'setup.py install' for ruamel.yaml, since package 'wheel' is not installed.
. If I first install the wheel
package, the same error appears when installing the packages.
Installing ruamel.yaml
manually works fine, but the azure packages still can't be installed.
EDIT: Installing every package one by one installs everything without errors. Why?
Changing step 3 to pip install -U pip wheel ruamel.yaml
doesn't change anything.
Solution
The ruamel.yaml
documentation states that it should be installed using:
pip install ruamel.yaml
so using the legacy [python] setup.py install
should not be used.
Your full log indicates that it is tried to install two versions of ruamel.yaml:
Collecting ruamel.yaml<=0.17.16,>=0.17.10
Using cached ruamel.yaml-0.17.16-py3-none-any.whl (109 kB)
and
Collecting ruamel.yaml<=0.15.89,>=0.15.35
Using cached ruamel.yaml-0.15.89.tar.gz (306 kB)
Apart from a potential problem with two versions being installed, the second one will not succeed as the 0.15 ruamel.yaml series (the last one uploaded to PyPI in July 2019) doesn't support Python 3.9 first released in Oct 2020.
So when you install each package step by step, you get the latest version or ruamel.yaml. Why that step by step installation doesn't at with some package tries to install ruamel.yaml==0.15.89
as a dependency is unclear.
When I install the four packages listed in your requirements.txt
, then ruamel.yaml
doesn't get installed but pipdeptree
indicates that azureml-core==1.36.0.post2
has possible conflicting dependencies.
When I use your requirements.txt
I get the same error.
When I use your requirements.txt
in a Python 3.7 venv
(so that the old ruamel.yaml wheel can be installed) installation works. From that you can see that the package azureml-core==1.14.0
is installed instead of the azureml-core==1.36.0.post2
Answered By - Anthon