Python - Packages (Archive, Distribution)

About

Code Shipping - Archive (Distribution) in Python

Packages (or better Distribution package) are the result of the packaging of a project that may contains one or more (regular) package

A package is a compressed tarball file that contains:

• system-level libraries,
• Python or other modules,
• executable programs and other components.

Articles Related

Format

Source

Python - Source Distribution (sdists)

Binary

Python

Wheel and Egg are both binary packaging formats created from compiled python file.

• The Egg format was introduced by setuptools in 2004
• The Wheel format was introduced by PEP 427 in 2012.

Wheel is currently considered the standard for built and binary packaging for Python.

Exe

• http://py2exe.org/ - converts Python scripts into executable Windows programs

Management

Install

Doc

First update the tools

python -m pip install --upgrade pip setuptools wheel

In a binary format

You can install your package:

• in the global path from the index with pip and a requirement-specifiers

pip install package_name
pip install "requirement-specifiers"
# Example
pip install "pyspark == 2.2.2"

• with easy_install (deprecated)

easy_install package

From a tarball / source

• with setup.py

cd /rootOfProject
python setup.py install
# or
setup.py install

• from a branch with pip

pip install https://github.com/myName/myPackage/tarball/master 
pip install https://github.com/myName/myPackage/tarball/X.X.X

• by downloading the source

curl --location --output myPackage-X.X.X.tar.gz https://github.com/myName/myPackage/tarball/X.X.X
tar xvfz myPackage-X.X.X.tar.gz
cd myPackage
# as script
python myScript.py args
# as package
pip install .

• From version control

pip install -e git+https://git.repo/some_pkg.git#egg=SomeProject          # from git
pip install -e hg+https://hg.repo/some_pkg.git#egg=SomeProject            # from mercurial
pip install -e svn+svn://svn.repo/some_pkg/trunk/#egg=SomeProject         # from svn
pip install -e git+https://git.repo/some_pkg.git@feature#egg=SomeProject  # from a branch

where: VCS:

• are detected using url prefixes: “git+”, “hg+”, “bzr+”, “svn+”.
• command are required in the path: git, hg, svn, or bzr.
• -e means development mode

Installation Location

Install location

Global

default global site-packages directory

(for instance /usr/lib/python2.7/site-packages)

GLOBALE_HOME/pythonX.X/site-packages/

This is the pip default

Example: install global of myPackage

pip install myPackage

User

The user location :

• Default to ~/.local/, or %APPDATA%\Python on Windows. (See the Python documentation for site.USER_BASE for full details.)
• or via the env PYTHONUSERBASE

The package are then installed at USER_HOME/pythonX.X/site-packages/

To install in a user location, use the --user option of pip

Example:

export PYTHONUSERBASE=$HOME
pip install --user virtualenv

Virtual environment

When working in a virtual environment, the package will be installed in it. See Python - virtualenv (Python Environment / venv)

Update

See the –update option of pip

Example: upgrade the setuptools and wheel package

python3 -m pip install --user --upgrade setuptools wheel

Packaging

Packaging task management, see setup.py cli

Reference / Specification

See Python - Pypa (Python Packaging Authority)

Documentation / Reference

• https://packaging.python.org/glossary/#term-distribution-package
• https://packaging.python.org/discussions/wheel-vs-egg/