Page Body

Software Libraries and the Make Tool

Software libraries serve several purposes. They:

  • Collect frequently required functionality
  • Endow functionality with a standardized and well-documented interface
  • Make functionality available to other programs

Most programs on a GNU/Linux system are written in C, either directly or indirectly. For example, some programs are written directly in C, while some are written in languages derived from C (e.g., C++).

Libraries become increasingly important when programming. In the simplest case, a linker is used to combine a program's object code (i.e., the output of the compiler, which translates a program written in a programming language like C to instructions executable on a Central Processing Unit, or CPU) with the object code of the libraries used by this program to yield an executable program, which can be run on a system.

Note: If you are not familiar with the GNU/Linux command line interface, review the Conventions page before proceeding.

Static Libraries

When a whole library is directly incorporated into an executable file, it is referred to as a static library (e.g., gcc -static sqrttest.c -lm -o sqrttest-static). Here, programs and libraries form a permanent combination. However, if a problem is found in the library, all programs using that library need to be re-linked and the executable needs to be recreated. Also, this leads to larger program files.

The names of static libraries end in .a and version numbering does not apply to them.

Dynamic Libraries

The idea behind a dynamic library is that the actual program and libraries are not permanently combined during linking, but are only put together when the program is actually executed (e.g., gcc sqrttest.c -lm -o sqrttest-dynamic). The program and libraries remain independent of each other.

This has several advantages:

  • As long as a library's interface stays the same (i.e., the signatures of the functions concerned, the number and types of their arguments and results do not change), the library can be easily replaced without having to re-link all programs using it. When a program is started, it automatically fetches the new library version.
  • If n programs are using the library, it does not require n times the space on disk. One library file is enough.
  • Usually, it is sufficient to have the library in memory once. All processes can share one copy. This is why we speak of shared libraries.

The downside of shared libraries is that you are paying for these benefits with time. It takes a bit longer to launch a program using dynamic libraries, since the connection between the actual program and its libraries needs to be made at runtime, instead of when the program is compiled to an executable file.

However, while dynamic libraries may carry a runtime penalty, depending on the processor and programming model used, this penalty usually amounts to a low, single-digit percentage and is outweighed by the advantages listed above.

The names of dynamic libraries generally end in .so, usually followed by a version number.

$ ls -l /usr/lib/libdiscover.*
lrwxrwxrwx 1 root root    20 Jan 14  2018 /usr/lib/libdiscover.so.2 -> libdiscover.so.2.0.1
-rw-r--r-- 1 root root 56480 Jan 14  2018 /usr/lib/libdiscover.so.2.0.1

Above, the symbolic link is used to relate actual programs and the library. A program asks for a specific version of a library (in the example above, libdiscover.so.2). It depends on the system whether this resolves to /usr/lib/libdiscover.so.2.0.1 or /usr/lib/libdiscover.so.2.1.2, i.e., the system returns whichever version of the library is installed.

The basic assumption is that all files purporting to be libdiscover.so.2 implement the same interface.

Dynamic Library Version Numbers

Dynamic libraries are assigned version numbers that typically contain up to three dot-separated parts (e.g., libdiscover.so.2.0.1):

  1. The first number after the library name is the major version number. It should be incremented whenever the programming interface offered by the library to other programs changes in an incompatible fashion (e.g., a function that used to accept two arguments now requires three).
  2. The second number is the minor version number. It is incremented when the programming interface is extended without changes to existing calls (e.g., a completely new function might be added). Older programs can still use the library because everything they require is still available in an identical fashion.
  3. The third number is the patch level. It is incremented when there are changes to the library implementation that do not affect the interface (e.g., a bug in the implementation of a function might be fixed).

Replacing a defective library with a new one has no impact on programs that are already running with the defective library, e.g., daemons. When in doubt, such programs must be manually restarted to benefit from the improvements.

Programs always require a certain major number of a library. Therefore, it is possible for a system to simultaneously contain programs requiring multiple major versions of the same library. You can leave the various library files installed and let the dynamic linker find the correct one for each program.

Dynamic Libraries and the Linux Standard Base

A large part of the Linux Standard Base (LSB) is concerned with prescribing the version and content of important dynamic libraries.

The LSB prescribes certain major versions of common libraries and standardizes a programming interface for the library in question. LSB-conforming GNU/Linux distributions must offer these libraries for the benefit of LSB executables, but these libraries do not need to be the ones that the rest of the system is using.

Therefore, it is possible to make a GNU/Linux distribution conform to LSB from the point-of-view of library support by placing a set of the required libraries in a different directory, and to foist this onto LSB executables, instead of the usual system directories for libraries, through dynamic linker manipulation (e.g., use of the LD_LIBRARY_PATH variable).

The same strategy is also useful to third-party software that does not rely on the LSB. As long as the kernel-libc interface does not incompatibly change, a software package can provide its own libc (plus other sorts of libraries) and try to become independent of the library support offered by the underlying GNU/Linux system.

ldconfig, /etc/ld.so.conf, /etc/ld.so.cache, and LD_LIBRARY_PATH

When you start a program, the dynamic linker must find and load the required libraries. The program itself only contains a library's name, not the name of the file it is found in.

To avoid having the dynamic linker search the file system, the /etc/ld.so.cache file contains an index of all known dynamic libraries (i.e., a list of file paths for shared libraries). The dynamic linker only finds the libraries that occur in this index.

The index in /etc/ld.so.cache is created using the ldconfig program (/sbin/ldconfig), which searches all directories listed in /etc/ld.so.conf, as well as the two standard directories, /lib and /usr/lib, for libraries (/lib and /usr/lib are always searched, no matter what is in /etc/ld.so.conf) . All found libraries are entered into the index. ldconfig also takes care of creating the major version number symbolic links for library files.

/etc/ld.so.cache is a binary file and is not viewable as a normal text file. However, its known library paths can be viewed by using ldconfig's -p (--print-cache) option:

$ /sbin/ldconfig -p | head
992 libs found in cache `/etc/ld.so.cache'
    libzvbi.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzvbi.so.0
    libzvbi-chains.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzvbi-chains.so.0
    libzstd.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzstd.so.1
    libzmq.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzmq.so.5
    libzmf-0.0.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzmf-0.0.so.0
    libzeitgeist-2.0.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzeitgeist-2.0.so.0
    libzbar.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzbar.so.0
    libzapojit-0.0.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libzapojit-0.0.so.0
    libz.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libz.so.1

/etc/ld.so.cache is refreshed when the system is initially booted or by the system's package management tool when new dynamic libraries are installed. In addition, /etc/ld.so.cache can be manually rebuilt using ldconfig. This should be done if you have manually updated the /etc/ld.so.conf file (# /sbin/ldconfig).

If your user does not have the access rights to use ldconfig, you can also add a new library file path to the LD_LIBRARY_PATH shell variable, which the dynamic linker will also search. Its syntax corresponds to that of the PATH variable, i.e., directory names separated by colons:

export LD_LIBRARY_PATH="${HOME}/lib:/example/library/path"

The dynamic linker searches for shared library locations in the following order:

  1. LD_LIBRARY_PATH
  2. /etc/ld.so.cache
  3. /lib and /usr/lib

For more information, run man 8 ld-linux.so.

ldd

Dynamic libraries may depend on other dynamic libraries. These kinds of dependencies can be determined with the ldd command, which displays shared object dependencies:

ldd ex_object...

For example, these are the dependencies of the libdiscover.so.2.0.1 dynamic library used above:

$ ldd '/usr/lib/libdiscover.so.2.0.1'
    linux-vdso.so.1 (0x00007ffe7d7c9000)
    libusb-0.1.so.4 => /lib/x86_64-linux-gnu/libusb-0.1.so.4 (0x00007f40c2db5000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f40c2d78000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f40c2bb7000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f40c31e7000)

/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (which is symbolically linked to /lib/x86_64-linux-gnu/ld-2.28.so) is the dynamic linker, i.e., the program that causes dynamically-linked programs to be connected to their dynamic libraries when they are started.

linux.vdso.so.1 is a virtual dynamic library (i.e., it does not actually exist as a file on the file system) that the Linux kernel makes available to all programs. It is used to speed up system calls on modern Intel- and AMD-based 64-bit systems. The linux-gate.so.1 library serves the same purpose on 32-bit systems as linux.vdso.so.1 does on 64-bit systems.

Run man 7 vdso for more information.

file

The file command determines a file's type:

file ex_file...

This command can be used to determine if an executable program is statically or dynamically linked.

$ file '/usr/bin/ping'
/usr/bin/ping: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=fd4f1d5aa1ba5e2e27ffba4b99e60257f1f336e2, stripped

For statically-linked executables, not stripped refers to the fact that the program in question still contains information that is designed to make debugging easier. You can remove this information using the strip command, e.g., strip ex_static_executable.

Run-Time and Development Packages

It is common to distinguish between a run-time package and a development package when dealing with libraries.

Run-time packages contain the files that must be installed so that other programs can use the library (like the dynamically loadable library in a .so file).

Development packages only need to be installed if you need to compile new or existing programs that will use the library. These types of packages contain the information the C compiler needs to use the library (i.e., include files), a statically linkable library for debugging, or documentation about the library's content.

Software Compilation and Installation

Make is a build automation tool that automatically builds executable programs and libraries from source code by reading Makefiles, which specify how to create the target program.

The general process for compiling and installing a program from source code using Make generally goes like this:

  1. Download and extract the source code to an installation directory.
  2. Change to the installation directory and run ./configure, which checks your system to verify compatibility and creates a Makefile.
  3. Run the make command to create the binary executable file from the source code text files using the instructions provided in the Makefile.
  4. Run the make install command to install the executable on the system.

Software installed from source code will use either an uninstall script in the original installation files or an uninstall target in the Makefile that is called using the make uninstall command. You may need to run ./configure once more to generate the Makefile again before you uninstall the software.

On a GNU/Linux system, GNU Autoconf is often the tool used to create configure scripts. It is part of GNU Autotools (also called the GNU Build System). This is the system that is used for the example below.

GNU Build System Example

The following example is going to build the Liferea RSS application from source. Liferea's developer, Lars Windolf, has facilitated this process by providing clear documentation on how to build Liferea yourself.

First, ensure that your system has the required dependencies installed. For example, on a Debian GNU/Linux system, you can run the following command:

# apt install libxml2-dev libxslt1-dev libsqlite3-dev libwebkit2gtk-4.0-dev libjson-glib-dev libgirepository1.0-dev libpeas-dev gsettings-desktop-schemas-dev python3 libtool intltool

Next, download a tarball (e.g., liferea-1.12.9.tar.bz2) and extract it to an installation directory:

$ cd '/var/tmp' &&
    wget -q \
    'https://github.com/lwindolf/liferea/releases/download/v1.12.9/liferea-1.12.9.tar.bz2'
$ mkdir 'build' && cd '/var/tmp/build' &&
    tar -xf '/var/tmp/liferea-1.12.9.tar.bz2' && ls -l
total 4
drwxr-xr-x 13 amnesia amnesia 4096 Jun 13 08:06 liferea-1.12.9

The liferea-x.xx.x directory will contain the files required to build Liferea. Its contents will look something like this:

$ cd '/var/tmp/build/liferea-1.12.9' && ls -l
total 1648
-rw-r--r-- 1 amnesia amnesia 421305 Aug 28  2020 aclocal.m4
-rwxr-xr-x 1 amnesia amnesia   5826 Sep 13  2018 ar-lib
-rw-r--r-- 1 amnesia amnesia   7321 Sep 13  2018 AUTHORS
-rw-r--r-- 1 amnesia amnesia  47845 Aug 28  2020 ChangeLog
-rwxr-xr-x 1 amnesia amnesia   7333 Sep 13  2018 compile
-rwxr-xr-x 1 amnesia amnesia  43940 Sep 13  2018 config.guess
-rw-r--r-- 1 amnesia amnesia   2996 Aug 28  2020 config.h.in
-rwxr-xr-x 1 amnesia amnesia  36339 Sep 13  2018 config.sub
-rwxr-xr-x 1 amnesia amnesia 532795 Aug 28  2020 configure
-rw-r--r-- 1 amnesia amnesia   2637 Aug 28  2020 configure.ac
-rw-r--r-- 1 amnesia amnesia  18093 Sep 13  2018 COPYING
drwxr-xr-x 2 amnesia amnesia   4096 Aug 28  2020 css
-rwxr-xr-x 1 amnesia amnesia  23566 Sep 13  2018 depcomp
drwxr-xr-x 3 amnesia amnesia   4096 Aug 28  2020 doc
drwxr-xr-x 2 amnesia amnesia   4096 Aug 28  2020 dtd
drwxr-xr-x 2 amnesia amnesia   4096 Aug 28  2020 glade
-rw-r--r-- 1 amnesia amnesia   9720 Sep 13  2018 INSTALL
-rwxr-xr-x 1 amnesia amnesia  15155 Sep 13  2018 install-sh
-rw-r--r-- 1 amnesia amnesia  18962 Aug 28  2020 liferea.appdata.xml
-rw-r--r-- 1 amnesia amnesia   2522 Sep 13  2018 liferea.appdata.xml.in
-rw-r--r-- 1 amnesia amnesia   1784 Sep 13  2018 liferea.convert
-rw-r--r-- 1 amnesia amnesia 324412 Sep 13  2018 ltmain.sh
-rw-r--r-- 1 amnesia amnesia   1971 Jun 10  2020 Makefile.am
-rw-r--r-- 1 amnesia amnesia  37311 Aug 28  2020 Makefile.in
drwxr-xr-x 3 amnesia amnesia   4096 Aug 28  2020 man
-rwxr-xr-x 1 amnesia amnesia   6872 Sep 13  2018 missing
-rw-r--r-- 1 amnesia amnesia  11801 Aug 28  2020 net.sf.liferea.gschema.xml
-rw-r--r-- 1 amnesia amnesia  11821 Aug 28  2020 net.sf.liferea.gschema.xml.in
-rw-r--r-- 1 amnesia amnesia   5267 Aug 28  2020 net.sourceforge.liferea.desktop
-rw-r--r-- 1 amnesia amnesia    455 Apr 19  2020 net.sourceforge.liferea.desktop.in
drwxr-xr-x 2 amnesia amnesia   4096 Aug 28  2020 opml
drwxr-xr-x 8 amnesia amnesia   4096 Aug 28  2020 pixmaps
drwxr-xr-x 2 amnesia amnesia   4096 Aug 28  2020 plugins
drwxr-xr-x 2 amnesia amnesia   4096 Aug 28  2020 po
drwxr-xr-x 7 amnesia amnesia   4096 Aug 28  2020 src
drwxr-xr-x 2 amnesia amnesia   4096 Aug 28  2020 xslt

Next, we run the configure shell script to verify the compatibility of our GNU/Linux system. The output will be extensive, but should end like this:

$ ./configure
...
config.status: creating src/fl_sources/Makefile
config.status: creating src/ui/Makefile
config.status: creating src/tests/Makefile
config.status: creating doc/Makefile
config.status: creating doc/html/Makefile
config.status: creating xslt/Makefile
config.status: creating man/Makefile
config.status: creating man/pl/Makefile
config.status: creating pixmaps/Makefile
config.status: creating pixmaps/16x16/Makefile
config.status: creating pixmaps/22x22/Makefile
config.status: creating pixmaps/24x24/Makefile
config.status: creating pixmaps/32x32/Makefile
config.status: creating pixmaps/48x48/Makefile
config.status: creating pixmaps/scalable/Makefile
config.status: creating opml/Makefile
config.status: creating glade/Makefile
config.status: creating po/Makefile.in
config.status: creating src/liferea-add-feed
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing default-1 commands
config.status: executing po/stamp-it commands

liferea 1.12.9

Liferea will be installed in /usr/local/bin.

configure complete, now type 'make'

Our example system looks good to go, but if you see any configure errors, you can address them by consulting the config.log file that is generated in the directory where you run .configure.

Inside the contents of the directory where you successfully run the configure script, a Makefile should be generated:

$ ls -l Makefile
-rw-r--r-- 1 amnesia amnesia 41978 Jun 13 08:01 Makefile

Next, run the make command to create the liferea executable from the source code text files using the instructions provided in the Makefile. The output will likely be prodigious and if you need a more detailed description of what make is doing, you can add the command's -d option.

Here are the last few lines of a successful Liferea make execution:

$ make
...
make[3]: Leaving directory '/var/tmp/build/liferea-1.12.9/src'
make[2]: Leaving directory '/var/tmp/build/liferea-1.12.9/src'
Making all in xslt
make[2]: Entering directory '/var/tmp/build/liferea-1.12.9/xslt'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/var/tmp/build/liferea-1.12.9/xslt'
Making all in glade
make[2]: Entering directory '/var/tmp/build/liferea-1.12.9/glade'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/var/tmp/build/liferea-1.12.9/glade'
make[2]: Entering directory '/var/tmp/build/liferea-1.12.9'
  GEN      net.sourceforge.liferea.service
  GEN      net.sf.liferea.gschema.valid
make[2]: Leaving directory '/var/tmp/build/liferea-1.12.9'
make[1]: Leaving directory '/var/tmp/build/liferea-1.12.9'

If make was able to successfully build the liferea executable on your system, you can install it with the # make install command. The last few lines of a successful # make install command look like this:

# make install
...
make[2]: Nothing to be done for 'install-exec-am'.
 /usr/bin/mkdir -p '/usr/local/share/appdata'
 /usr/bin/install -c -m 644 liferea.appdata.xml '/usr/local/share/appdata'
 /usr/bin/mkdir -p '/usr/local/share/liferea/css'
 /usr/bin/install -c -m 644 css/liferea.css css/user.css css/adblock.css '/usr/local/share/liferea/css'
 /usr/bin/mkdir -p '/usr/local/share/dbus-1/services'
 /usr/bin/install -c -m 644 net.sourceforge.liferea.service '/usr/local/share/dbus-1/services'
 /usr/bin/mkdir -p '/usr/local/share/applications'
 /usr/bin/install -c -m 644 net.sourceforge.liferea.desktop '/usr/local/share/applications'
 /usr/bin/mkdir -p '/usr/local/share/GConf/gsettings'
 /usr/bin/install -c -m 644 liferea.convert '/usr/local/share/GConf/gsettings'
 /usr/bin/mkdir -p '/usr/local/share/liferea/dtd'
 /usr/bin/install -c -m 644 dtd/html.ent '/usr/local/share/liferea/dtd'
 /usr/bin/mkdir -p '/usr/local/lib/liferea/plugins'
 /usr/bin/install -c -m 644 plugins/bold-unread.py plugins/bold-unread.plugin plugins/gnome-keyring.py plugins/gnome-keyring.plugin plugins/headerbar.py plugins/headerbar.plugin plugins/libnotify.py plugins/libnotify.plugin plugins/media-player.py plugins/media-player.plugin plugins/plugin-installer.py plugins/plugin-installer.plugin plugins/trayicon.py plugins/trayicon.plugin '/usr/local/lib/liferea/plugins'
if test -n "net.sf.liferea.gschema.xml"; then \
    test -z "/usr/local/share/glib-2.0/schemas" || /usr/bin/mkdir -p "/usr/local/share/glib-2.0/schemas"; \
    /usr/bin/install -c -m 644 net.sf.liferea.gschema.xml "/usr/local/share/glib-2.0/schemas"; \
    test -n "" || /usr/lib/x86_64-linux-gnu/glib-2.0/glib-compile-schemas /usr/local/share/glib-2.0/schemas; \
fi
make[2]: Leaving directory '/var/tmp/build/liferea-1.12.9'
make[1]: Leaving directory '/var/tmp/build/liferea-1.12.9'

Afterwards, liferea should be available at /usr/local/bin/liferea:

$ which liferea
/usr/local/bin/liferea
Liferea
The Liferea Application

Documentation

For more information on the commands discussed in this post, run:

  • man 8 ldconfig
  • man 8 ld-linux.so
  • man 1 ldd
  • man 7 vdso
  • man 1 file
  • man 1 make

Also, you can refer to the Linux User's Manual online.

Enjoyed this post?

Subscribe to the feed for the latest updates.