Troubleshooting Pip Installation Issues in CI/CD Pipelines

Problem

Today, I encountered an issue with my Django application's CI/CD pipeline in GitLab. After running smoothly for months, the pipeline suddenly failed during the pip install -r requirements.txt step. The error log pointed to a problem with the installation of Celery==5.1.0:

Requested celery==5.1.0 from https://files.pythonhosted.org/packages/b4/f3/884b0b5bd46bc9d9721076c060b2ea37b2f95fdb6edb01e7fcda3def56dc/celery-5.1.0-py3-none-any.whl (from -r requirements.txt (line 34)) has invalid metadata: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier

    pytz (>dev)

         ~^

Please use pip<24.1 if you need to use this version.

ERROR: Ignored the following yanked versions: 5.0.6, 5.2.5

ERROR: Ignored the following versions that require a different python version: 5.0 Requires-Python >=3.10; 5.0.1 Requires-Python >=3.10; 5.0.2 Requires-Python >=3.10; 5.0.3 Requires-Python >=3.10; 5.0.4 Requires-Python >=3.10; 5.0.5 Requires-Python >=3.10; 5.0.6 Requires-Python >=3.10; 5.0.7 Requires-Python >=3.10; 5.0.8 Requires-Python >=3.10; 5.0a1 Requires-Python >=3.10; 5.0b1 Requires-Python >=3.10; 5.0rc1 Requires-Python >=3.10; 5.1 Requires-Python >=3.10; 5.1a1 Requires-Python >=3.10; 5.1b1 Requires-Python >=3.10; 5.1rc1 Requires-Python >=3.10

This is really weird as Celery version 5.1.0 existed and I have been using it for years.

Root Cause

The issue was caused by an incompatibility between pip version 24.2 and Celery==5.1.0. The error log revealed that pip was unable to parse Celery's metadata correctly due to a version mismatch, resulting in the installation failure.

Ref link: https://github.com/celery/celery/discussions/9112

pip version install issue

The Solution

To resolve this issue, I modified the CI/CD pipeline to install pip version 24.0 instead of upgrading to the latest version (24.2 as of August 16, 2024). This fixed the problem, and the pipeline successfully installed all dependencies, including Celery.

Create venv and upgrade pip:

  stage: build

  rules:

    - if: $CI_COMMIT_BRANCH == "develop"

  when: manual

  tags:

    - test

  script:

    - rm -R venv

    - python3.8 -m venv venv

    - venv/bin/pip3 install -U pip==24.0 --force

    - venv/bin/pip3 install wheel

Lesson Learned

When maintaining a CI/CD pipeline, it's crucial to monitor changes in the tools you're using. In this case, a pip version upgrade introduced an incompatibility with a package that had been stable for a long time. Rolling back to a previous pip version allowed me to maintain the stability of my pipeline while waiting for an upstream fix or preparing to upgrade the package.

For anyone facing similar issues, try locking your pip version to a stable release that works with your existing dependencies. This can prevent unexpected failures and keep your CI/CD pipeline running smoothly.

[TIPS] Troubleshooting Pip Installation Issues in CI/CD Pipelines

Troubleshooting Pip Installation Issues in CI/CD Pipelines

Problem

Root Cause

The Solution

Lesson Learned

Related

[TIPS] Salesforce REST API: The requested resource does not exist

[TIPS] Tig: A Simple Boost for Your Git Workflow

Python Pytest Introduction