How to Package a Python Code and Upload to PyPi

Introduction

This tutorial walks you through how to package your python project. It will show you how to add the necessary files and structure to create the package, build the package, and upload it to the Python Package Index (PyPI). This tutorial doesn’t describe the only way of doing things, merely one specific approach that works well. 

In particular, packages should make it easy:

• To install with pip or easy_install.
• To specify as a dependency for another package.
• For other users to download and run tests.
• For other users to work on and have immediate familiarity with the basic directory structure.
• To add and distribute documentation.

We’ll walk through the basic steps of building a test python package to support these things. You can find the repository here.

An advantage of this distribution method is its well-established ecosystem of tools such as PyPI and pip, making it easy for other developers to download and install your package for casual experiments or as part of large, professional systems.

It is a well-established convention for Python code to be shared this way. If your code isn’t packaged, then it will be harder for other developers to find it and use it as part of their existing process. They will regard such projects with substantial suspicion of being either badly managed or abandoned.

The downside of distributing code like this is that it relies on the recipient understanding how to install the required version of Python and being able and willing to use tools such as pip to install your code’s other dependencies. This is fine when distributing to other developers but makes this method unsuitable for distributing applications to end-users.

Let's dive into the code packaging by following the topics below.

Structure Your Project

It is important to structure your project in a good format to allow easy packaging. You can find the test repository here.

├── app.py
├── package
│   ├── api.py
│   ├── __init__.py
│   └── __version__.py
├── README.md
├── requirements.txt
├── setup.py
└── tests
    ├── configtest.py
    └── test_app.py

Let's break down the project files structure:

• package

• This is your package folder. It contains the entry point api.py to your project. You can add all files here and basically register them in your entry point.

• api.py - This is the entry point to your package.

• __init__.py - Add it to register your code as a package.

• __version__.py - Optionally required for setup.py. If you don't add it, you can just define your package version in the setup.py file.

• app.py - This is just a test file to test the packaged code. 

• from package.api import testfunc
  
  print(testfunc())

• requirements.txt - You should be familiar with this. It just contains your python requirements or dependencies.

• tests - Define your test cases here.

• setup.py - The main file. Let's treat this separately below.

Setup File

Now that your project structure is ready, you need to add basic information about the project -- like its name, the version, and other packages it requires.

That information is provided in the form of `setup.py.` We'll base our setup.py file on Setup.py For Humans.

Read more about setup.py from StackOverflow https://stackoverflow.com/questions/1471994/what-is-setup-py

Add this file to the project root.
 

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import io
import os

from setuptools import find_packages, setup

# Package meta-data.
NAME = "ktechhub"
DESCRIPTION = "ktechhub: A demo package."
EMAIL = "info@ktechhub.com"
AUTHOR = "KtechHub"
REQUIRES_PYTHON = ">=3.8.0"
VERSION = "0.0.1" #package version

# Which packages are required for this module to be executed?
REQUIRED = [
    "requests==2.27.1",
    #add more here
]

# The rest you shouldn't have to touch too much :)
here = os.path.abspath(os.path.dirname(__file__))

# Import the README and use it as the long-description.
# Note: this will only work if 'README.md' is present in your MANIFEST.in file!
try:
    with io.open(os.path.join(here, "README.md"), encoding="utf-8") as f:
        long_description = "\n" + f.read()
except FileNotFoundError:
    long_description = DESCRIPTION

# Load the package's __version__.py module as a dictionary.
about = {}
if not VERSION:
    project_slug = NAME.lower().replace("-", "_").replace(" ", "_")
    with open(os.path.join(here, project_slug, "__version__.py")) as f:
        exec(f.read(), about)
else:
    about["__version__"] = VERSION


# Where the magic happens:
setup(
    name=NAME,
    version=about["__version__"],
    description=DESCRIPTION,
    long_description=long_description,
    long_description_content_type="text/markdown",
    author=AUTHOR,
    author_email=EMAIL,
    python_requires=REQUIRES_PYTHON,
    packages=find_packages(exclude=["tests/"]),
    install_requires=REQUIRED,
    include_package_data=True,
    license="MIT",
    classifiers=[
        "Programming Language :: Python :: 3.8",
    ],
    setup_requires=["wheel"],
)

The only thing that you will need to change is mostly the Package meta-data part.

You shouldn't have to touch the rest. But make sure to take a look at the bottom function called `setup()`.

Building Package

You need to build the archives and wheels. Run:

python setup.py sdist bdist_wheel

Run it and you will see a dist folder created that includes an archive and a wheel file. The new structure will be:

├── app.py
├── build
│   ├── bdist.linux-x86_64
│   └── lib
│       └── package
│           ├── api.py
│           ├── __init__.py
│           └── __version__.py
├── dist
│   ├── ktechhub-0.0.1-py3-none-any.whl
│   └── ktechhub-0.0.1.tar.gz
├── image.png
├── ktechhub.egg-info
│   ├── dependency_links.txt
│   ├── PKG-INFO
│   ├── requires.txt
│   ├── SOURCES.txt
│   └── top_level.txt
├── LICENSE
├── package
│   ├── api.py
│   ├── __init__.py
│   └── __version__.py
├── README.md
├── requirements.txt
├── setup.py
└── tests
    ├── configtest.py
    └── test_app.py

You can distribute the wheel file for installation. You could upload it to an S3 storage or any remote URL and share with others. Example: 

pip3 install ktechhub-0.0.1-py3-none-any.whl

Publishing To PyPi

To do that, you will need an account on PyPI. So, go ahead and register

You will also need a package called twine, which is a utility for publishing Python packages on PyPI:

pip3 install twine

You can run the following command to see if the build were created correctly:

twine check dist/*

If all the checks passed, you can upload the framework to PyPI:

twine upload dist/*

You will be prompted for your username and password.

Once uploaded, navigate to https://pypi.org/project/PACKAGE_NAME/

You can now easily install your package:

pip3 install <PACKAGE_NAME>

Conclusion

Note that publishing a packaged code to PyPi means making your code public for others to use. You should only do that when you want to make your code available publicly to other users.

Mumuni Mohammed

Experienced DevOps Engineer proficient in Python, Docker, Kubernetes, CI/CD, and cloud technologies. Specialized in automating processes and building scalable infrastructure for seamless development and deployment.