Boyang Yue

Software Engineering, Big Data, and the miscellaneous

09 Aug 2025

Sharing a Confidential Python Library Without PyPI

A quantitative trading system is rarely a single program. A backtester, a live execution engine, and a pile of research notebooks all need the same calculations: volatility scaling, a Sharpe ratio, a handful of instrument models. Copying those functions into each program means fixing every bug several times while the definitions slowly drift apart. A shared library gives them one implementation.

Two constraints shape how this library should be distributed. The strategy code should not go to the public package index, and the Git host should not receive the implementation in readable form.

Confidential has a narrow meaning here. GitHub encrypts source code at rest, but GitHub controls that storage layer and can process readable source to provide features such as search and browser-based diffs. A private repository limits who may access the code. transcrypt, a Git filter that encrypts matched files client-side under a shared symmetric key, adds a further layer: selected source blobs reach the host as ciphertext instead.

That protection ends wherever the key appears:

LocationWhat is readable
Git host without the transcrypt credentialRepository metadata and unprotected files, but not the contents of matched, non-empty source files
Developer or build machine holding the credentialThe complete working tree
Environment using the pure-Python wheelThe packaged Python source

This does not hide an algorithm from a teammate or machine that can run it. If the consumer itself is untrusted, the implementation needs a different boundary, such as an API running on infrastructure the consumer cannot inspect.

A .env file does not solve this problem. python-dotenv loads configuration values from a file that must stay out of version control; it does not protect implementation code.

A private package index, whether a self-hosted devpi or an artifact repository, is a conventional way to distribute internal packages when that service is trusted. It also adds another system to secure, back up, and keep available. A small team can avoid that service by letting uv install a package from Git. Client-side encryption complicates the last step because an unkeyed clone cannot produce a usable installation.

The ordinary uv and Git path

In the confidential workflow, the scaffold remains uncommitted until transcrypt is configured.

Scaffolding the package

The scaffold starts in a working directory such as ~/projects:

# ~/projects
uv init --lib mylib
cd mylib

uv creates ~/projects/mylib, initializes an uncommitted Git repository, and produces this layout:

mylib/
  .gitignore
  .python-version
  README.md
  pyproject.toml
  src/mylib/__init__.py
  src/mylib/py.typed

The src layout keeps the import package out of the project root, so tests and tools exercise the installed package rather than the working copy. The generated pyproject.toml also selects uv’s build backend. The excerpt here omits descriptive and author metadata:

[project]
name = "mylib"
version = "0.1.0"
requires-python = ">=3.13"
dependencies = []

[build-system]
requires = ["uv_build>=0.8.6,<0.9.0"]
build-backend = "uv_build"

The py.typed marker tells downstream tools that the package ships type information, as specified by PEP 561.

Writing and building the package

The scaffold leaves a placeholder __init__.py. Here src/mylib/stats.py holds the shared calculations:

"""Return and risk calculations shared across the trading system."""


def annualized_volatility(daily_std: float, trading_days: int = 252) -> float:
    """Scale daily volatility under the square-root-of-time assumption."""
    if daily_std < 0:
        raise ValueError("daily_std must be non-negative")
    if trading_days <= 0:
        raise ValueError("trading_days must be positive")
    return daily_std * trading_days**0.5


def sharpe_ratio(mean_return: float, volatility: float, risk_free: float = 0.0) -> float:
    """Return excess return per unit of volatility, using consistent periods."""
    if volatility <= 0:
        raise ValueError("volatility must be positive")
    return (mean_return - risk_free) / volatility

Square-root-of-time scaling assumes the daily observations support that model. The return, risk-free rate, and volatility passed to sharpe_ratio must also use the same period. Those assumptions matter in production even though the functions are only packaging examples here.

src/mylib/__init__.py re-exports the public names:

from mylib.stats import annualized_volatility, sharpe_ratio

__all__ = ["annualized_volatility", "sharpe_ratio"]

Runtime dependencies are added with uv add <package>. uv records them in the dependencies field under [project] and locks their resolved versions in uv.lock. This example has none.

uv build creates both a source distribution and a wheel:

# ~/projects/mylib
$ uv build
$ ls dist/
mylib-0.1.0-py3-none-any.whl  mylib-0.1.0.tar.gz

PyPI could serve either artifact. Here the build only confirms that the package is valid; the repository will be the distribution channel.

Publishing and installing readable source

For version control, uv init runs git init but creates no commit. The initial branch follows Git’s init.defaultBranch setting and is commonly master when that setting is absent. The first commit records the package, and git branch -M normalizes the branch name:

# ~/projects/mylib
git add -A
git commit -m "Initial library: volatility and Sharpe helpers"
git branch -M main

The GitHub CLI can create a private remote and push the branch:

gh repo create <owner>/mylib --private --source=. --remote=origin --push

The equivalent manual setup starts with an empty repository created in GitHub’s web interface:

git remote add origin [email protected]:<owner>/mylib.git
git push -u origin main

A tag gives the release a readable name:

git tag v0.1.0
git push origin v0.1.0

A separate backtester can then add the private repository as a Git dependency:

uv init ~/projects/backtester
cd ~/projects/backtester
uv add git+https://github.com/<owner>/[email protected]
# Or authenticate over SSH:
# uv add git+ssh://[email protected]/<owner>/[email protected]

uv add fetches the repository, builds and installs the wheel, and records the resolved commit in the backtester’s uv.lock. It uses the Git credentials already available on the machine, such as an SSH key or an HTTPS credential helper.

The generated main.py can call the library:

from mylib import annualized_volatility, sharpe_ratio


def main() -> None:
    volatility = annualized_volatility(daily_std=0.01)
    print(sharpe_ratio(mean_return=0.08, volatility=volatility))


if __name__ == "__main__":
    main()

uv run checks that the project environment agrees with the lockfile and then executes the program:

# ~/projects/backtester
$ uv run main.py
0.5039526306789697

The lockfile preserves the commit resolved from v0.1.0, but a tag is still a mutable Git reference. Regenerating the lockfile after someone moves the tag can resolve a different commit. A full commit ID is the stronger reproducibility boundary.

Why encrypted source cannot be installed directly

transcrypt configures clean, smudge, diff, and merge drivers in a repository’s local .git/config. Those definitions and the credential do not travel with a clone. The tracked .gitattributes file can say that src/** uses the crypt filter, but a fresh clone does not yet know how to run that filter.

Git treats an undefined filter as a pass-through unless the local configuration marks it as required. This behavior lets an unkeyed user work on unprotected files, but it also means an automated uv clone receives ciphertext under src/. With this pure-Python package, uv can still build and install a wheel containing those encrypted .py files. Importing the package then fails, usually during parsing.

Encryption must happen before the first commit. A consumer then unlocks a trusted local checkout and depends on that checkout rather than on the remote URL.

Configuring transcrypt before the first push

Installing transcrypt

Homebrew supplies transcrypt on macOS:

brew install transcrypt

Elsewhere it is a single script placed on the PATH:

git clone https://github.com/elasticdog/transcrypt.git
sudo ln -s "${PWD}/transcrypt/transcrypt" /usr/local/bin/transcrypt

Generating and storing the credential

The repository needs its transcrypt configuration before the first git add, commit, or push:

# ~/projects/mylib
transcrypt -c aes-256-cbc -y

Without -p, transcrypt generates 30 random bytes and Base64-encodes them for the repository password. That is safer than putting a human-memorable passphrase in the command line, where shell history and process inspection can expose it.

An existing high-entropy password can be supplied explicitly:

transcrypt -c aes-256-cbc -p '<shared-password>' -y

This form is useful when initializing another clone, but the password can enter shell history or appear in process listings. Importing the credential from a GPG-encrypted file avoids passing it as a command-line argument.

The generated credential is stored in plaintext in the local repository’s .git/config. --display prints it for a password-manager backup:

transcrypt --display

The output also includes a reusable initialization command. It is secret material and has no place in a ticket, build log, or chat.

OpenSSL 3 emits this warning while transcrypt stages a non-empty encrypted file:

*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.

The encryption still completes, but the warning is security-relevant. transcrypt invokes OpenSSL with -md MD5 and no PBKDF2, a fixed format that keeps existing repositories decryptable across machines and releases. That makes the randomly generated credential important because a weak passphrase is easier to test offline. OpenSSL’s own enc documentation recommends PBKDF2 for password-based encryption.

Changing transcrypt’s flags in one clone makes it incompatible with the others. A policy requiring an authenticated cipher or a modern password KDF calls for a tool designed around those requirements.

Selecting and verifying encrypted files

The root .gitattributes file selects the protected files:

src/**  filter=crypt diff=crypt merge=crypt

filter=crypt encrypts matched files as Git writes them to the index and decrypts them on checkout. diff=crypt and merge=crypt let a keyed local checkout compare and merge plaintext. transcrypt leaves empty files unchanged, so the empty py.typed marker remains an empty blob. The README, pyproject.toml, license, and other unmatched files also remain readable.

The initial commit records ciphertext for the matched, non-empty files:

git add -A
git commit -m "Initial encrypted library"

Some Git clients bypass clean filters when they write directly to the index. transcrypt installs a pre-commit check that rejects a managed, non-empty file whose staged content lacks the expected OpenSSL prefix. That hook is a safety check, not a security boundary: git commit --no-verify bypasses it, and transcrypt cannot activate it automatically when another pre-commit hook already exists.

Renormalization sends files staged without the filter through the clean filter again:

git add --renormalize .

A raw view of the committed source blob confirms the filter before the first push:

git check-attr filter -- src/mylib/stats.py
git show HEAD:src/mylib/stats.py --no-textconv
# Equivalent convenience command:
transcrypt --show-raw src/mylib/stats.py

A successful filter produces Base64 ciphertext beginning with OpenSSL’s U2FsdGVk prefix rather than Python source. Each attribute pattern needs the same check, especially after a change to .gitattributes.

The private remote receives the encrypted commit and release tag:

gh repo create <owner>/mylib --private --source=. --remote=origin --push
git tag v0.1.0
git push origin v0.1.0
git rev-parse 'v0.1.0^{commit}'

Matched, non-empty source blobs are ciphertext. The repository name, file paths, unprotected files, commit graph, authors, messages, timestamps, and approximate change sizes still reach GitHub in readable form. The trusted release record used by consumers contains the printed commit ID.

Confidentiality is not integrity

transcrypt uses AES-256-CBC without an authentication tag. The project documents the resulting ciphertext malleability. Git’s object hashes can show that fetched bytes match an expected object ID, but the object ID must come from somewhere the attacker cannot rewrite.

A tag fetched from the same host is not enough when a compromised host is in the threat model. A full release commit recorded outside that host provides a trust anchor. A signed tag or commit is another option when its signature is verified before use. The resolved commit in uv.lock helps after the first trusted resolution; it does not authenticate that first resolution by itself.

Deterministic encryption and metadata leakage

Git needs unchanged plaintext to produce unchanged ciphertext. transcrypt derives each file’s salt from an HMAC-SHA256 over its contents, keyed with the file name and repository password. An unchanged file therefore reuses its blob, while a content change produces a different salt. The file name prevents identical contents at two paths from encrypting alike.

This also leaks equality over time. Someone who can read the repository can see that a protected file stayed unchanged or returned to an earlier version, even without the credential. Encryption does not hide the path, approximate plaintext size, or timing of the change.

Sharing the credential

transcrypt --export-gpg <recipient> encrypts the repository password and cipher to a teammate’s GPG public key. The export depends on a verified fingerprint for that key:

transcrypt --export-gpg [email protected]

The command writes the result to .git/crypt/[email protected]. Files under .git are not tracked, so the export does not travel through the repository automatically. Distribution therefore uses an approved channel or another protected location.

A solo developer can keep the generated password in a password manager. For a team, per-recipient GPG exports avoid copying the plaintext command printed by --display.

If plaintext was already pushed

Adding .gitattributes later protects new Git objects but does not erase old plaintext commits. Rewriting history with git-filter-repo and force-pushing is only part of cleanup.

Old objects may remain in clones, forks, pull-request references, or cached GitHub views. GitHub’s sensitive-data removal procedure covers the required coordination and the cases that need GitHub Support.

Source code cannot be rotated like an API key. Once plaintext reaches an untrusted system or person, that version remains disclosed even if the hosted repository is later cleaned.

Consuming the encrypted library

On the backtester developer’s machine, a no-checkout clone keeps the files encrypted while the worktree moves to the full commit ID obtained through the trusted release channel:

git clone --no-checkout [email protected]:<owner>/mylib.git ~/projects/mylib
cd ~/projects/mylib
git checkout --detach <trusted-commit>
git rev-parse HEAD

git rev-parse HEAD prints the expected full commit ID. The matched files remain ciphertext because this clone does not yet have the local filter configuration. The GPG-wrapped configuration enters the clone only after commit verification:

transcrypt --import-gpg /secure/path/[email protected] -y
git status --short

Importing the configuration installs the local filters and forces a checkout, which turns matched files into plaintext. A clean checkout leaves git status --short with no output.

The decrypted checkout becomes a path dependency of the backtester:

cd ~/projects/backtester
uv add ../mylib
# --editable exposes live library edits to the backtester.

uv stores the relative path under [tool.uv.sources] and builds from the trusted checkout. The lockfile records that path, not the checkout’s Git commit, so the release process must verify both the expected commit and a clean tree before every sync or build. The backtester’s main.py then runs unchanged and prints the same result:

$ uv run main.py
0.5039526306789697

Any wheel built from this checkout contains readable Python source. Storing it on a machine or artifact system extends the implementation’s trust boundary to that system.

CI and operational costs

Every process that starts from the encrypted repository and then tests or builds the library needs the credential and a plaintext checkout. A GitHub-hosted runner therefore exposes the implementation to GitHub’s execution infrastructure. A self-hosted runner changes where the code executes, but it still accepts jobs from the GitHub Actions control plane. If a compromised GitHub service is part of the threat model, the CI controller belongs on trusted infrastructure as well.

Under a narrower threat model that trusts the Actions control plane, self-hosted runners need isolation and access limited to selected repositories and reviewed workflows. Untrusted pull-request code cannot safely share a runner with the transcrypt credential. GitHub’s secure-use guidance explains why workflow code that reaches a runner can extract its secrets.

The host also loses the source-aware features that depend on plaintext. Browser search and diffs show ciphertext. Server-side merge tools cannot reason about the Python changes. A keyed local checkout is different: transcrypt’s text-conversion and merge drivers decrypt the base, local, and remote versions, so ordinary local diffs and non-conflicting three-way merges still work.

A keyed local repository creates another exposure. transcrypt stores the password in .git/config, and Git may cache plaintext text-conversion results in its local object database. A compromised developer machine can read both. transcrypt --flush-credentials removes the local credential, returns protected files to ciphertext, and garbage-collects cached plaintext. Because the operation removes the local credential, a separate backup is a prerequisite.

Rekeying encrypts the current files under a new password. Earlier commits remain under the old password, and anyone who previously held that password may keep the source they already decrypted. A shared symmetric password cannot provide retroactive revocation.

If dependency names are sensitive, pyproject.toml can be added to the encrypted patterns, but that also prevents an unkeyed clone from reading the build metadata. Repository names, paths, lockfiles in consumer projects, and other metadata may still reveal the package’s identity.

Choosing the boundary

RequirementSimpler fit
Trusted team and acceptable Git-host access controlsPrivate repository with a commit-pinned Git dependency
Git host must not receive selected source in readable form, while developer and build machines are trustedtranscrypt plus a verified local checkout
Centralized releases, access policy, and artifact retention are neededTrusted private package index or artifact repository
The implementation must remain hidden from the environment that executes itA service boundary rather than a distributed pure-Python package

Skipping the package registry shifts operational work into credential distribution, commit verification, trusted checkouts, and trusted build infrastructure. That trade is justified only when excluding the Git host from the plaintext trust boundary is an actual requirement.