Projekt

Allgemein

Profil

Gwbuild

Work-in-Progress

Please note that this build system is still work-in-progress. I use it for quite some time now to build the binary packages for AqFinance and for me it works nicely.

However, it probably still has bugs I didn't encounter yet, since I have a rather fixed workflow for which I use it (mostly compiling/linking projects of the AqBanking family). So using it on other systems with other compiler or linker versions than those I tested with can lead to errors.

This documentation here is mainly an aide for myself and a few friends for maintaining the build files. However, gwbuild is public code so the documentation might as well be public, too.

Preamble

I don't plan to replace the current autotool usage in AqBanking with gwenbuild, I just want to speedup my own development process and circumvent the current problem where I need a complete rebuild of AqBanking as soon as a single source files has changed because otherwise the resulting libaqbanking.so sometimes still contains older objects. The process of "./configure && make" simply takes an inacceptable amount of time to finish which slows down development considerably.

The only member of the AqBanking family where the autotools have been replaced by gwbuild is AqFinance, a finance application using the ultrafast FOX Toolkit, because I am the only developer on that project.

Reasons to Create Gwbuild

The main reason to build Gwbuild was my dissatisfaction with existing build systems.

Some of the build systems were created when it was really difficult to build libraries, especially shared libs.
To compensate for this tools and utilities were developed around this fact (e.g. libtool, dlltool), and the variety of systems and their specialties when it came to compiling and linking had to be accounted for. That's not the fault of those build systems, its just that they had so much variety to handle.

Current compilers and linkers are much easier to use, nowadays it is much easier to create static or dynamic libraries and to link against those which should make writing build systems easier.

Autotools

autotools (automake, autoconf) are really cool and they work in almost every case as expected.
However, they are extremely slow which is a problem with larger projects containing many source files
and convenience libraries. Also, they are hard to debug if you encounter unusual problems, e.g. in AqBanking
we have the problem that sometimes not all sources/convenience libraries are correctly rebuilt after source code
changes and up to now I have no idea why that is.

CMake

A nice solution would be to switch to cmake, especially when using their build target ninja which is
astonishingly fast. cmake is quite easy to setup for simple projects, however, it is not that easy to port existing larger projects
to use it. Also, there is quite a steep learning curve.

Meson

meson is another good alternative since it also uses ninja and is therefore also extremely fast. But I wasn't able to
adapt my projects to use "meson" because some features I needed for e.g. AqFinance or AqBanking required a version of meson newer
than what is provided by the distribution I was at that time working with.

Gwbuild

Of course creating yet another build system might seem silly especially when there are well established systems like cmake. But to me as a free-time programmer that's beside the point.
I love programming especially in C (if only as a way to keep the brain busy with complex stuff), and I like challenges.

So out of curiosity gwbuild was started. I wanted to see how fast a build system can be and how hard it would be to write one.
Gwbuild is a tool inside the library gwenhywfar, which is a base library for all my projects. Therefore it is always available in
the required version for depending projects. Gwenhywfar itself will not be ported to use this build system to avoid the
hen-and-egg-problem.

Even though gwbuild is not targeted towards a wider audience working on it at least led to some improvements and fixes in the gwenhywfar library which I consider a value in itself.

The following requirements were identified before development started:
  • needs to be much faster than autotools/make
  • needs to support parallel building
  • needs to provide library, header and function search functions
  • needs to be able to generate a "config.h" file
  • needs to be able to rewrite files by replacing placeholders in source files (like "sed" does in ./configure)
  • needs to be easy to setup and easy to understand
  • needs to have support for typemaker2 source files
  • needs to fully support strict out-of-source build
  • should be easy to extend
  • needs to handle dependency trees
  • needs to allow for custom build commands to generate future or (currently) unusual file types

Most of these goals have been reached in the current GIT version (as of 2022/08/26), though gwbuild has so far only been tested on Debian-based systems.

Quickstart

AqBanking was the first project using gwenbuild in parallel with autotools. AqBanking is quite a complex project using convenience libraries, shared loadable modules, shared libraries and applications. Therefore it is a good starting point to understand how gwenbuild is used.

To build AqBanking with gwenbuild try the following commands inside the source tree of AqBanking using 14 processes (takes about 9 secs on my build machine):


mkdir build
cd build
gwbuild -s ..
gwbuild -p
gwbuild -Btm2builder
gwbuild -j14

Using Gwbuild

Gwbuild is designed for out-of-source-tree builds. Therefore you should create a folder outside of your source-tree (or inside it in a toplevel folder, e.g. build).
This makes gwbuild extremely usefull when working with GIT repositories because Gwbuild will not modify files inside the source tree nor will it add files to it. All output will strictly go to the build folder.

Gwbuild Commands vs. Autotool Commands as Used in AqBanking

Step Gwbuild Command Autotools Command Description
1 gwbuild -s .. -O OPTION=ARG /configure --option=arg Prepare for Compilation
2 gwbuild -p make typedefs Prepare type definition files with typemaker2 (only needed if typemaker2 is used)
3 gwbuild -Btm2builder make typefiles Initially create source files from typemaker2 input files (only needed if typemaker2 is used)
4 gwbuild -j 22 make -j22 Build the project (in this case using 22 processes in parallel, the -j option can be omitted)
5 gwbuild -i make install Install project
6 gwbuild -d make dist Prepare packaging (gwbuild currently only creates a folder which needs to be tar'ed before distribution

Step 4 can be repeated as often as needed, e.g. after making changes to source files. Gwbuild will rebuild all files necessary using its dependency tracking code. It will also rebuild typemaker2 output files if input files are changed.

Using multiple processes in parallel can extremely speed up the building process.

Tutorials, Documentation

There are some tutorials for Gwbuild containing documentation about 0BUILD files: