Boyang Yue

Software Engineering, Big Data, and the miscellaneous

03 May 2025

Python Distributions, Native Dependencies, and Environment Boundaries

conda became familiar in data science because it handled a problem pip did not. An environment could carry Python, compiled libraries, command-line tools, and sometimes another runtime together. That model mattered when the hard part of a project was the native stack around it, rather than the Python package being installed.

For many projects, this difference was the deciding factor: when PyPI meant source builds and missing headers, conda avoided local compiler failures. The premise behind that model did not disappear, but it became less universal. As wheels and manylinux matured, many compiled Python projects became ordinary package-index installs. That opened a narrower lane for uv, released in 2024. Its Rust implementation and speed both drew attention. The comparison with conda is about scope. uv’s unit is the Python distribution; conda’s is the broader binary environment.

From compiler to wheel

The problem conda solved

In 2012, installing the scientific Python stack from the Python Package Index (PyPI) usually meant compiling it. pip commonly installed source distributions, so a machine needed a C and Fortran toolchain and the right development headers to build NumPy or SciPy against a system linear-algebra library. A failed build was a common first experience. The wheel format was specified around the same time, but PyPI did not yet have a widely adopted portable binary path for the scientific stack.

The conda model framed this as a problem of binary distribution rather than of Python packaging. A conda package is a prebuilt, typically platform-specific archive that can hold almost anything: a Python library, a shared C library, a compiler, an R package, or a standalone executable. The solver installs the interpreter, the libraries it links against, and the tools around it into one environment in a single step, drawn from a channel of such packages. Python is central to how conda is used, but to the solver it is just another package in the graph.

That design made conda a general-purpose binary environment manager with no special tie to Python. It became especially useful where the difficult dependencies are not Python code alone: data science, geospatial work, bioinformatics, and high-performance computing.

conda create -n hpc -c conda-forge python=3.12 mpi4py openmpi
# interpreter, mpi4py, and the Message Passing Interface (MPI) runtime it links against, in one solve

The wheel era

conda’s advantage rested on something missing from Python packaging, and Python packaging eventually supplied much of it: a binary distribution path for Python packages. Wheels gave PyPI a standardized binary package format. The manylinux profiles, beginning with PEP 513 in 2016 and continuing through PEP 600, allowed Linux wheels to bundle permitted shared libraries and install on mainstream glibc-based distributions without a compiler.

NumPy, SciPy, pandas, and much of the scientific stack began publishing wheels. For a large class of projects, pip install quietly started doing what once required conda.

That changed what a Python-focused installer could cover. Once common compiled dependencies were installable from PyPI, a fast Python-package installer could serve many projects without conda’s machinery. What stayed outside the wheel was the rest of conda’s original territory: native libraries and command-line tools that no wheel bundles, and dependencies written in other languages. Even parts of the GPU stack, a common reason to reach for conda, have crossed over. NVIDIA now publishes user-space CUDA libraries as wheels, and PyTorch’s pip installation path can carry CUDA runtime pieces through Python wheels. The kernel driver stays outside any package manager. The gap grew narrower without closing.

uv’s contribution

By the early 2020s the Python-package path worked but was fragmented. Many projects leaned on pyenv for interpreters, venv for environments, pip to install, and pip-tools or Poetry to pin, each a separate tool with its own files. uv folded that set into one program. It resolves and downloads in parallel and keeps a global cache shared across environments.

Beyond speed, the larger change is scope. uv installs interpreters itself, from the python-build-standalone project, since CPython publishes no official relocatable builds for redistribution. Its lockfile, uv.lock, records exact resolved versions and captures platform-conditional dependencies across Linux, macOS, and Windows in one file.

The dependency metadata stays in the standard pyproject.toml, and uv-specific settings live under tool.uv. uv.lock itself is uv’s own format: PEP 751 became Final in March 2025 and defines pylock.toml, which uv can export, though the standard does not yet express everything uv.lock records. Inside a uv project, one program covers the lifecycle from interpreter to lockfile, following Python’s packaging standards where they exist and uv’s own formats elsewhere.

uv python install 3.12   # fetch a standalone interpreter
uv add 'numpy>=2'        # add to pyproject.toml, resolve, update uv.lock

What uv does not do is conda’s founding job. It resolves and installs Python distributions, wheels and sdists alike, and manages the interpreter beside them. It can create an isolated Python build environment for an sdist’s Python build requirements, but it does not provision the non-Python headers, shared libraries, or compilers that build may expect to find already on the machine. That smaller scope is the design.

What stays on each side

Most of the differences follow from the package unit. In Python packaging, a distribution is a packaged release of one Python project, shipped as a source distribution (sdist) or a prebuilt wheel. That is a package unit, not a distribution of Python itself, such as the Anaconda Distribution.

Dimensionuvconda
Package unitPython distributionGeneral binary package
Source distributionsBuilds them when the build prerequisites are already on the machineDoes not consume them directly; can install compilers, headers, and native libraries as packages
Native libraries and command-line toolsManaged only when carried by a Python distributionFirst-class packages in the dependency graph
Other language runtimesOutside the project graphCan be installed beside Python
Default workflowProject-local environment tied to project metadata and a lockfileNamed environment that can span tools and languages
LockingCross-platform uv.lock in the project workflowExplicit lock usually added with a tool such as conda-lock

conda’s remaining ground

uv is at its strongest when the dependency graph can be expressed as Python distributions, whether they come from a package index, a Git repository, a URL, or a local path. Its appeal there is speed, a single cross-platform lockfile, and one workflow instead of several. A package without a wheel does not automatically fall outside uv: uv can build from a source distribution when the non-Python build prerequisites are already present.

conda’s ground begins when the environment itself must supply those prerequisites. The classic case is an sdist that expects HDF5 headers or a Fortran compiler on the machine: the source build fails on a bare system, while conda can install the headers and the compiler into the environment first. The same applies when the environment has to carry R or a command-line tool next to Python.

There is also a deployment boundary. Apache Spark documents conda as one way to pack a complete Python environment, interpreter included, and distribute the archive to executors; the same page covers venv-pack and PEX for environments built without conda. Keeping the Spark driver and executors on the same environment removes a common class of worker-only failures and keeps behavior consistent from local development to the cluster. That is environment distribution rather than dependency resolution, and it stays outside uv’s scope.

Speed, locking, and reproducibility

The conda side changed too. One of the longest-running complaints was the solver, which could take minutes on a large environment. The libmamba solver, built on the C++ mamba project and made conda’s default in version 23.10, released in late 2023, cut that to a fraction and narrowed the speed argument against conda.

Reproducibility differs in defaults more than in ceilings. A hand-written environment.yml records requirements rather than a solved result, and a full environment export is often platform-specific and noisy. Consistent multi-platform resolution usually needs an explicit lock from a tool such as conda-lock. uv makes its project lockfile part of the default workflow: uv.lock captures resolutions across platforms.

A lockfile of either kind reproduces only the dependency graph the tool controls. It does not describe the host kernel, GPU driver, CPU features, or every system library outside the environment. Workloads that depend on those also need documented host requirements or a container image.

Channels and licensing

Channels complicate conda locking in a way a single package index does not. Two channels may publish packages with the same name but different build assumptions, so adding more channels is not always harmless. A reproducible conda workflow keeps its channel set small and its priority order fixed.

One difference has nothing to do with capability and still matters in many organizations. The conda program is open source, but its packages come from channels with their own ownership and terms, so an instruction to “use conda” is incomplete until it also says which channels are allowed.

Many teams standardize on community channels such as conda-forge, an internal mirror, or an installer such as Miniforge that defaults away from Anaconda’s channels. uv installs from PyPI by default; packages there carry their own licenses, but the index itself does not charge for access. The comparison here is between channels and indexes; the tools just point at them.

Hybrid stacks

The two also compose. A project may need conda for a base containing a full CUDA toolkit, a Java runtime, a compiler, or an R runtime, while its Python application layer changes often enough that uv’s speed is worth keeping on top. uv’s pip interface can install into an activated conda environment, leaving the interpreter and native libraries to conda and taking the Python distribution layer itself.

The pip interface sits outside uv’s project workflow, however, and does not apply uv.lock automatically. If the Python layer needs to follow that lock, the resolved requirements can be exported and installed into the active conda environment:

uv export -o requirements.txt
uv pip install -r requirements.txt   # with the conda environment activated

The choice of uv pip install over uv pip sync matters: sync would remove installed Python packages not listed in the file, including any that conda had installed. The conda side should go in first and then stay mostly fixed, because conda does not fully track what pip or uv changes underneath it. If the native base must change later, rebuilding the environment from its declarations is safer than continuing to mutate it in place.

pixi, from prefix.dev, reached the same convergence from the conda side. It launched in 2023 with a Cargo-like project workflow, manages conda-channel packages with a lockfile, and supports PyPI dependencies in the same environment, resolving them with uv’s own libraries. The lesson is that conda’s package coverage still matters, while fast, project-centered tooling has become a common expectation.

Where the line falls now

The history matters because the boundary moved, not because one tool won. conda was built when Python packaging had no reliable binary path for the scientific stack. Wheels handed much of that back to PyPI. uv made the Python-package path fast and coherent. That speed does not help with dependencies that are not Python packages, so conda keeps the broader scope it started with.

The resulting rule of thumb is about dependency shape. The application domain alone does not decide it: PyTorch can fit uv when compatible wheels cover its runtime needs, while a small utility may need conda because one sdist expects a native library the environment must supply.

uv’s side ends at the last usable wheel, or the last source build whose non-Python prerequisites are already on the machine. conda’s side begins where the environment itself has to provide a library, a compiler, or another language’s runtime. Host drivers and operating-system packages stay outside both; each tool was built for a different definition of a package, and both definitions are still in use.