Skip to main content

PyPI Repositories

Introduction

The Python Package Index, or PyPI, is a vast repository of open-source Python packages supplied by the worldwide community of Python developers. The official index is available at https://pypi.org, and the site itself is maintained by the Python Software Foundation.

Both Nexus Repository Manager Pro and Nexus Repository Manager OSS support proxying the Python Package Index. This allows the repository manager to take advantage of the packages in the official Python Package Index without incurring repeated downloads. This will reduce time and bandwidth usage for accessing Python packages.

Also, you can publish your own packages to a private index as a hosted repository on the repository manager, then expose the remote and private repositories as a repository group, which is a repository that merges and exposes the contents of multiple repositories in one convenient URL.

Note

If using pip with the repository manager, you should consider setting up your repository manager to use SSL as documented in Configuring SSL. Otherwise, you will likely need to put --trusted-host additions at the end of many commands or further configure pip to trust your repository manager.

Proxying PyPI Repositories

You can set up a PyPI proxy repository to access a remote package index. To proxy a PyPI package index, you simply create a new pypi (proxy) recipe as documented in Proxy Repository, in detail.

Minimal configuration steps are:

  • Define Name - e.g. pypi-proxy

  • Define URL for Remote storage. The official Python Package Index Remote Storage URL value to enter is https://pypi.org/. Using https://pypi.python.org/ should also work as long as redirects are maintained.

The repository manager can access Python packages and tools from the remote index. The proxy repository for PyPI packages provides a cache of files available on the index making access to components from the Python Package Index more reliable. Users will be able to browse and search assets against the remote, as mentioned in Browsing PyPI Repositories and Searching Packages.

Hosting PyPI Repositories

Creating a PyPI hosted repository allows you to upload packages in the repository manager. The hosted repository acts as an authoritative location for packages fetched from the Python index.

To host a PyPI package, create a new pypi (hosted) recipe as documented in Hosted Repository, in detail.

Minimal configuration steps are:

  • Define Name - e.g. pypi-internal

  • Pick a Blob store for Storage

Grouping PyPI Repositories

A repository group is the recommended way to expose all your PyPI repositories from the repository manager to your users, with minimal additional client side configuration. A repository group allows you to expose the aggregated content of multiple proxy and hosted repositories as well as other repository groups with one URL in tool configuration. PyPI group repositories can be created with the pypi (group) recipe as documented in Repository Group.

Minimal configuration steps are:

  • Define Name - e.g. pypi-all

  • Pick a Blob store for Storage

  • Add PyPI repositories to the Members list in the desired order

Installing PyPI Client Tools

The latest versions of such Linux distributions as CentOS and Ubuntu come packaged with Python and pip, a tool for installing and managing Python packages from the index. For Mac OS X and Microsoft Windows, download and install a Python version compatible with the repository manager from https://www.python.org/downloads/. This should come automatically with pip but you can see https://pip.pypa.io/en/stable/installation/if not.

Note

Nexus Repository only supports specific versions of Python, pip and setuptools. For Python, only the latest two releases of 2 and 3 are supported. For pip the latest two versions are supported. setuptools removed easy_install from their application in newer versions so only 51.3.3 and older will work. Versions not listed may work but issues with them are unlikely to be addressed if found.

Configuring PyPI Client Tools

Note

Depending on your preference for either twine, distutils, pip and setuptools, your proxy and hosted configuration will vary.

Once you have installed all necessary client tools from the Python Package Index, you can create and configure a .pypirc file to reference packages stored in the repository manager. Depending on your Python configuration you can manage your repository groups with pip.conf (or setup.cfg) to have all commands, such as search and install, run against your project.

Upload to a hosted repository using twine

If you are authoring your own packages and want to distribute them to other users in your organization, you have to upload them to a hosted repository on the repository manager using a client tool such as twine. The .pypirc holds your credentials for authentication when hosting a PyPI repository.

In the example .pypirc file below, specify the URL you want to deploy to the target hosted repository in the repository value. Add username and password values to access the repository manager. The .pypirc file contains distutils, a default server used by PyPI that provides upload commands that stores assets and authentication information.

[distutils]
index-servers =
pypi
[pypi]
repository: http://localhost:8081/repository/pypi-internal/
username: admin
password: admin123

Note

If you have multiple hosted repositories, you can add them to the .pypirc file, each with a different name, pointing to the corresponding repository URL.

After this is configured, you can upload packages to the hosted repository, as explained in Uploading PyPI Packages.

Download, search and install packages using pip

To configure pip, create a pip.conf file on a Unix environment or a pip.ini file on Windows.

If you want pip to install or search Python within a group, configure the file to include the repository group URL.

[global]
index = http://localhost:8081/repository/pypi-all/pypi
index-url = http://localhost:8081/repository/pypi-all/simple

If you prefer to configure pip for proxy (e.g. pypi-proxy) or hosted (e.g. pypi-internal) adjust the file accordingly.

Tip

To verify your configuration is correct run "pip config list -v". That will print the configuration being used, and the source configuration files it came from.

Download and install packages using easy_install

Easy Install lets you download, build, install, and update Python packages. Create a setup.cfg file which sets the index-url to the group, proxy or hosted repository from which you want to download packages.

In this example index-url is set for a proxy repository:

[easy_install]
index-url = http://localhost:8081/repository/pypi-proxy/simple

If you prefer to configure easy_install for hosted (pypi-internal) or group (pypi-all) adjust the file accordingly.

To install a package from the repository: easy_install example-package

SSL Usage for PyPI Repositories

You can proxy Python packages over HTTPS to ensure a secure connection with a self-signed certificate. This works for proxy, hosted, and group repositories. To set up the repository manager to serve HTTPS follow the configuration steps in Configuring SSL.

Also, you can set up pip to use the certificate to enable SSL and fetch packages securely. Additional configuration is necessary for the HTTPS client implementation to work. This assumes the repository manager has already been set up to use SSL, so verify your certificate works. Run the following command:

openssl verify <example-cerfificate>

When your certificate is proven to work, update your pip.conf. Here is an example configuration file for a repository group:

[global]
index = https://localhost:8443/repository/pypi-all/pypi
index-url = https://localhost:8443/repository/pypi-all/simple
cert = nexus.pem

Browsing PyPI Repositories and Searching Packages

You can browse PyPI repositories in the user interface inspecting the components and assets and their details, as described in Browsing Repositories and Repository Groups.

Searching for PyPI packages can be performed in the user interface, as described in Searching for Components. It finds all packages that are currently stored in the repository manager, either because they have been pushed to a hosted repository or they have been proxied from an upstream repository and cached in the repository manager.

From the command line you can search available PyPI packages defined in your configuration. This method is limited to pip (pip.conf).

To search, run:

pip search example-package

Note

"pip search" command is deprecated and doesn't work with a proxy repository, but you can still search for packages in hosted repositories.

Uploading PyPI Packages

Note

The steps to upload a PyPI package will vary if your system is configured with twine or setuptools.

After you configure your .pypirc you can upload packages from the index to the repository manager.

In the example below, twine is invoked to tell your repository what server to use when uploading a package. The -r flag is used to find the NXRM server in your .pypirc.

twine upload -r pypi <filename>

Policy-Compliant Component Selection for PyPI

PRO NEW IN 3.61.0

Note

Policy-compliant component selection for PyPI requires IQ Server version 167+.

Note

This functionality requires integration with Sonatype Repository Firewall and a Firewall license.

When a user requests a PyPI package without explicitly specifying a version,the PyPI client relies on the package metadata to select a version that satisfies the version constraints. If the selected version has policy violations and is quarantined by Sonatype Repository Firewall, it will cause a build failure that requires a manual fix of the root cause.

By enabling this option, Sonatype Repository Firewall will remove quarantined versions from the PyPI package metadata to prevent you from selecting a version with policy violations.

Two settings are needed to enable this behavior:

  1. Enable the Firewall Audit and Quarantine capability on the proxy repository.

  2. Check the Remove Quarantined Versions box in the Sonatype Nexus Repository settings page.

Also see the Sonatype Repository Firewall documentation on policy-compliant component selection.