python - Two pip-installed modules have the same name, how to select which one is loaded?

Question

I'm writing a python program that relies on a specific module, rtmidi. Thing is, at least two different packages in PyPI have a module with that name, rtmidi and python-rtmidi. They offer almost the same functionalities but have different syntax.

When only the "right" package is installed everything works fine. But if both packages are installed, using import rtmidi loads the "wrong" module and my program crashes. The only way to get it working again is to uninstall both packages then re-install the right one. Of course, since the user might rely on the other module for other programs, I can't expect them to do that.

Trying to identify the module with rtmidi.__name__ gives the same result with both packages.

So my question, how do I go about resolving this name clash problem? Is there a best-practice way to handle this?

question from:https://stackoverflow.com/questions/65901845/two-pip-installed-modules-have-the-same-name-how-to-select-which-one-is-loaded

Answer 1

You can't have them both installed (if relying on default pip behavior). Here I'll demonstrate with a couple simple pure-Python packages that have an import-name conflict: coveralls and python-coveralls.

# in a fresh environment
pip install python-coveralls
python -c "import coveralls; print(dir(coveralls)); print(coveralls.__file__)"
# ['__author__', '__builtins__', '__cached__', '__classifiers__', '__copyright__', 
# '__doc__', '__docformat__', '__file__', '__license__', '__loader__', '__name__', 
# '__package__', '__path__', '__spec__', '__version__', 'parse_args', 'wear']
# /path/to/lib/python3.8/site-packages/coveralls/__init__.py

These are the contents of python-coveralls. Note that the actual __init__.py file is located in the site-packages/coveralls/ directory.

pip install coveralls
python -c "import coveralls; print(dir(coveralls)); print(coveralls.__file__)"
# ['Coveralls', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', 
# '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', 
# 'api', 'exception', 'git', 'reporter', 'version']
# /path/to/lib/python3.8/site-packages/coveralls/__init__.py

These are the contents of coveralls. python-coveralls has been overwritten. Note that this is the same file. Anything in the old __init__.py is gone.

pip uninstall coveralls
python -c "import coveralls; print(dir(coveralls)); print(coveralls.__file__)"
# ['__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', 
# '__spec__']
# None

There's still a ghost of the package there, but its contents are gone (except things that all packages have internally). Notice that the file is now None: there is no __init__.py file for this package.

You can un-bork your environment by also running pip uninstall python-coveralls and then reinstalling whichever you want.

Solution

You do have to require that your users only use the package that is in your requirements, but that's why we use virtual environments. Alternatively, a user that wants to directly use the other package can change the install location (and thus the name used when loading) with the --target option to pip (but that won't help other apps that use the other library).

In practice, it's probably best to think of this as part of your installation process. You either need to (a) tell your users what requirements they need to install; or (b) have some automated process that gets the right requirements.

I'd say best practice is (b). A common way of doing this is to use setuptools, and defining the install_requires argument to the setup function. This refers to the package name on PyPI, not the import name. The setuptools quickstart is a good place to start.