Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
- name: Install OS / deb dependencies
run: |
sudo DEBIAN_FRONTEND=noninteractive apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq gcc libvirt-dev
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq gcc libvirt-dev pkg-config

- name: Cache uv
uses: actions/cache@v5
Expand All @@ -56,7 +56,7 @@ jobs:

- name: Install Python Dependencies
run: |
uv sync --extra ci
uv sync --group ci --no-dev
echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"

- name: Run tox target
Expand Down
31 changes: 14 additions & 17 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:
- name: Install OS / deb dependencies
run: |
sudo DEBIAN_FRONTEND=noninteractive apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq gcc libvirt-dev
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq gcc libvirt-dev pkg-config

- name: Cache uv
uses: actions/cache@v5
Expand All @@ -64,7 +64,7 @@ jobs:
${{ runner.os }}-uv-

- name: Install Python Dependencies
run: uv sync --extra ci
run: uv sync --group ci --no-dev

- name: Add .venv to PATH
run: echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"
Expand Down Expand Up @@ -96,7 +96,7 @@ jobs:
- name: Install OS / deb dependencies
run: |
sudo DEBIAN_FRONTEND=noninteractive apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev pkg-config

- name: Cache uv
uses: actions/cache@v5
Expand All @@ -107,7 +107,7 @@ jobs:
${{ runner.os }}-uv-

- name: Install Python Dependencies
run: uv sync --extra ci
run: uv sync --group ci --no-dev

- name: Add .venv to PATH
run: echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"
Expand Down Expand Up @@ -144,7 +144,7 @@ jobs:
- name: Install OS / deb dependencies
run: |
sudo DEBIAN_FRONTEND=noninteractive apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev pkg-config

- name: Cache uv
uses: actions/cache@v5
Expand All @@ -155,7 +155,7 @@ jobs:
${{ runner.os }}-uv-

- name: Install Python Dependencies
run: uv sync --extra ci
run: uv sync --group ci --no-dev

- name: Add .venv to PATH
run: echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"
Expand Down Expand Up @@ -203,7 +203,7 @@ jobs:

- name: Install Python Dependencies
run: |
uv sync --extra ci --extra build
uv sync --group ci --group build --no-dev
echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"

- name: Build Release Artifact
Expand Down Expand Up @@ -291,7 +291,7 @@ jobs:
- name: Install OS / deb dependencies
run: |
sudo DEBIAN_FRONTEND=noninteractive apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq gcc libvirt-dev
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq gcc libvirt-dev pkg-config

- name: Cache uv
uses: actions/cache@v5
Expand All @@ -302,7 +302,7 @@ jobs:
${{ runner.os }}-uv-

- name: Install Python Dependencies
run: uv sync --extra ci
run: uv sync --group dev --group docs

- name: Add .venv to PATH
run: echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"
Expand Down Expand Up @@ -333,13 +333,10 @@ jobs:
- name: Cleanup
run: rm -rf venv/ || true

- name: Export Development Requirements
run: uv export --extra test --extra lint --extra mypy --extra docs --format requirements.txt --no-hashes --no-emit-project --output-file requirements-dev.txt

- name: Run Pip Audit Check On All Development And Test Dependencies
uses: pypa/gh-action-pip-audit@1220774d901786e6f652ae159f7b6bc8fea6d266 # v1.1.0
with:
inputs: requirements-dev.txt
virtual-environment: .venv
# setuptools which we don't install or depend on directly
ignore-vulns: |
GHSA-r9hx-vwmv-q579
Expand Down Expand Up @@ -367,7 +364,7 @@ jobs:
- name: Install OS / deb dependencies
run: |
sudo DEBIAN_FRONTEND=noninteractive apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev pkg-config

- name: Cache uv
uses: actions/cache@v5
Expand All @@ -378,7 +375,7 @@ jobs:
${{ runner.os }}-uv-

- name: Install Python Dependencies
run: uv sync --extra ci
run: uv sync --group ci --no-dev

- name: Add .venv to PATH
run: echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"
Expand Down Expand Up @@ -409,7 +406,7 @@ jobs:
- name: Install OS / deb dependencies
run: |
sudo DEBIAN_FRONTEND=noninteractive apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev
sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq graphviz gcc libvirt-dev pkg-config

- name: Cache uv
uses: actions/cache@v5
Expand All @@ -420,7 +417,7 @@ jobs:
${{ runner.os }}-uv-

- name: Install Python Dependencies
run: uv sync --extra ci
run: uv sync --group ci --no-dev

- name: Add .venv to PATH
run: echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_dev_artifact.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
run: printenv | sort
- name: Install Dependencies
run: |
uv sync --extra build
uv sync --group build --no-dev
echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"

- name: Create Dev Artifacts
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_pricing_to_s3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:

- name: Install Python Dependencies
run: |
uv sync --extra ci
uv sync --group ci --no-dev
echo "${GITHUB_WORKSPACE}/.venv/bin" >> "$GITHUB_PATH"

- name: Generate and publish pricing data
Expand Down
16 changes: 7 additions & 9 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ build:
os: ubuntu-22.04
tools:
python: "3.10"
jobs:
pre_create_environment:
- python -m pip install --upgrade pip uv
create_environment:
- python -m uv venv "${READTHEDOCS_VIRTUALENV_PATH}"
install:
- UV_PROJECT_ENVIRONMENT="${READTHEDOCS_VIRTUALENV_PATH}" python -m uv sync --frozen --group docs --no-dev

# Build documentation in the "docs/" directory with Sphinx
sphinx:
Expand All @@ -20,12 +27,3 @@ formats:

# NOTE: We need to use older version of sphinx otherwise bullet points won't be rendered correctly
# on RTD using rtd theme.
# Optional but recommended, declare the Python requirements required
# to build your documentation
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- method: pip
path: .
extra_requirements:
- rtd
4 changes: 2 additions & 2 deletions docs/committer_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ To run it:
.. sourcecode:: bash

# Install build dependencies
pip install -e ".[build]"
uv sync --group build --no-dev

cd dist
./release.sh -u <yourusername>@apache.org
Expand Down Expand Up @@ -193,7 +193,7 @@ use this token, see:
.. sourcecode:: bash

# Install publish dependencies
pip install -e ".[publish]"
uv sync --group publish --no-dev

cd dist
./deploy.sh
Expand Down
21 changes: 14 additions & 7 deletions docs/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ Getting started with contributing to Libcloud
---------------------------------------------

We use ``uv`` to lock and manage development dependencies. Dependency groups are
defined as extras in ``pyproject.toml`` and the ``uv.lock`` file replaces the
legacy ``requirements-*.txt`` files.
defined under ``[dependency-groups]`` in ``pyproject.toml`` and ``uv.lock`` is
the source of truth for local development and documentation tooling.

To create or update the lockfile, run:

Expand All @@ -30,17 +30,24 @@ To create or update the lockfile, run:
If ``uv lock`` fails while building ``libvirt-python``, install the system
dependencies (``pkg-config`` and ``libvirt-dev``).

To set up a local development environment with common tooling:
To set up a local development environment with common tooling (tests, linting,
type checking, tox):

.. sourcecode:: bash

uv sync --extra test --extra lint --extra docs --extra mypy --extra ci
uv sync --group dev

If you only need a subset, sync just those extras (for example, tests only):
If you only need a subset, sync just those groups (for example, tests only):

.. sourcecode:: bash

uv sync --extra test
uv sync --group test --no-dev

If you also work on documentation, add the docs group:

.. sourcecode:: bash

uv sync --group dev --group docs

Commit ``uv.lock`` whenever dependency versions change.

Expand Down Expand Up @@ -447,7 +454,7 @@ Some examples which show how to handle those cases are described below.
Type annotation for unions
~~~~~~~~~~~~~~~~

In Python 3.9 and below, the pipe character (``|``) cannot be used for union
In Python 3.9 and below, the pipe character (``|``) cannot be used for union
types in type annotations. Instead, you should use ``Union`` from the ``typing``
module:

Expand Down
6 changes: 3 additions & 3 deletions docs/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ If you use ``uv``, you can install all test dependencies in one step:

.. sourcecode:: bash

uv sync --extra test --extra ci
uv sync --group test --group ci --no-dev

Drop ``--extra ci`` if you don't plan to run ``tox``.
Drop ``--group ci`` if you don't plan to run ``tox``.
If ``uv.lock`` is missing or out of date, run ``uv lock`` first.

If you don't use ``uv``, you need to have the following extra dependencies
If you don't use ``uv``, you need to have the following additional dependencies
installed:

* ``tox`` (``pip install tox``) - you only need this library if you want to
Expand Down
2 changes: 1 addition & 1 deletion integration/compute/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Running the API service

.. code-block:: bash

uv sync --extra integration-compute
uv sync --group integration-compute
python -m integration.compute.api

Running the tests
Expand Down
2 changes: 1 addition & 1 deletion integration/storage/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Setting up the test suite

.. code-block:: bash

uv sync --extra integration-storage
uv sync --group integration-storage

Running the tests
-----------------
Expand Down
20 changes: 11 additions & 9 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -71,12 +71,12 @@ Distribution = "https://pypi.org/project/apache-libcloud"
Changelog = "https://github.com/apache/libcloud/blob/trunk/CHANGES.rst"


[project.optional-dependencies]
[dependency-groups]
build = [
"build==1.2.2"
"build==1.2.2",
]
publish = [
"twine==5.1.1"
"twine==5.1.1",
]
test = [
"coverage[toml]==7.2.7; python_version >= '3.10'",
Expand Down Expand Up @@ -121,14 +121,10 @@ mypy = [
"types-urllib3",
]
ci = [
"tox==4.24.1"
]
rtd = [
"sphinx==6.2.1",
"sphinx_rtd_theme==2.0.0",
"tox==4.24.1",
]
integration-compute = [
"bottle"
"bottle",
]
integration-storage = [
"azure-identity",
Expand All @@ -138,6 +134,12 @@ integration-storage = [
"docker",
"requests",
]
dev = [
{include-group = "test"},
{include-group = "lint"},
{include-group = "mypy"},
{include-group = "ci"},
]

[tool.setuptools.packages.find]
where = ["./"]
Expand Down
Loading