Silicon Soul

If you have already gotten started with Python on a GNU/Linux system, you may find yourself in a quandary. You are both using extant Python software and creating your own projects. You have multiple virtual environments, but are having trouble keeping everything organized.

There are many thoughts on how to best address this issue and reasonable people will disagree. If you are not set on a specific solution and are looking for a set of ideas to try out, the following guide may be of use.

Note: If you are not familiar with the GNU/Linux command line interface, or you intend to use a script obtained from this site, review the Conventions page before proceeding.

Projects and Virtual Environments

First, we start with two directories:

1. ${HOME}/venvs/ - Stores virtual environments.
2. ${HOME}/venv_projs/ - Stores project data that is used with a virtual environment in ${HOME}/venvs/.

For example, assume we have three virtual environments in ${HOME}/venvs/:

$ tree -L 1 "${HOME}/venvs/"
/home/amnesia/venvs
|-- pelican
|-- py_prog
`-- sphinx

4 directories, 0 files

• pelican/ contains the software needed to run Pelican projects.
• py_prog/ contains third-party software that should be easily accessible for a variety of custom Python projects used for day-to-day Python programming.
• sphinx/ contains the software needed to run Sphinx projects.

${HOME}/venv_projs/ might look something like this:

$ tree -L 1 "${HOME}/venv_projs"
/home/amnesia/venv_projs
|-- ex_pelican_site
|-- ex_sphinx_site
`-- py_projs

4 directories, 0 files

Above, py_projs/ serves as a general catchall directory for storing custom Python code, while ex_pelican_site/ and ex_sphinx_site/ are used to store data for a Pelican and Sphinx site, respectively.

Each project directory is associated with a virtual environment.

Virtual Environment Creation

Virtual environments are a core component of Python development. Automation can make virtual environment creation a little easier.

For example, after setting a directory to create virtual environments in and a projects directory, the following script:

1. Asks for a virtual environment name
2. Creates the virtual environment in the specified directory
3. Upgrades the virtual environment's dependencies
4. Activates the virtual environment
5. Prompts for a project selection

The full script is available below:

• View
• Download

Virtual Environment Activation

Virtual environment activation can be streamlined by using either aliases or shell scripts.

For example, the following aliases can be used to:

• Activate the py_prog virtual environment and change to the py_projs/ directory.
• Deactivate any virtual environment and return to the user's home directory.

alias pa='source "${HOME}/venvs/py_prog/bin/activate" \
    && cd "${HOME}/venv_projs/py_projs/"'
alias pd='deactivate && cd'

The following act_venv.bash script uses the names of the virtual environments in a directory of virtual environments (e.g., ${HOME}/venvs/) to dynamically create a menu. When you select a virtual environment from the menu, it is automatically activated. Finally, the script prompts you to select a project.

The full script is available below:

• View
• Download

Managing Python Applications With pipx

Some Python third-party software is meant to be used as an end-user application. Often, this is software that is obtained from the Python Package Index (PyPI).

For example, there are Python applications to:

• Provide a unified command line interface to Amazon Web Services (awscli)
• Explore and arrange tabular data (visidata)
• Search and browse Stack Overflow from the terminal (socli)
• Offer a system monitoring tool in the spirit of top (tiptop)

Since these packages are really meant to be used as applications, it would be burdensome to have to create and manage virtual environments for each of them. pipx is a tool that can automatically and transparently handle this for you.

pipx is closely related to pip, but while pip is designed to handle both library and application installations, pipx focuses on the latter. In addition, pipx creates isolated environments for each application and its associated package.

pipx can be installed from the repositories of many GNU/Linux distributions, including both Debian and Fedora:

# apt install pipx (Debian)

# dnf install pipx (Fedora)

Installing an application with pipx can be done like so:

pipx install ex_app

When you install an application with pipx, it automatically creates a virtual environment, installs the package, and adds the package's associated applications (entry points) to a location on your system's PATH. After installation, the ex_app command will be globally available, but the ex_app package will be sandboxed in its own virtual environment.

pipx Commands

Helpful pipx commands include:

pipx install ex_app

Install a package.

pipx upgrade ex_app

Upgrade a package.

pipx upgrade-all

Upgrade all packages.

pipx uninstall ex_app

Uninstall a package.

pipx uninstall-all

Uninstall all packages.

pipx list

List installed packages.

Finding Your Niche

Choosing a system for organizing and managing a Python environment is an important step in becoming a happy and efficient Python user. For more ideas on how to create a productive Python setup, check out Real Python's An Effective Python Environment: Making Yourself at Home.