JCons does not alter the environment in any way. So the environment an executed command sees, is the same as the one in effect when jcons was started. And JCons does not find compilers and other tools in some magic way. It is the responsibility of the JCons user to have a suitable PATH before invoking JCons. It is of course possible to set the PATH at the start of the construct.py too, the normal Python way:
# construct.py
import os
os.environ["PATH"] += ":/some/path/bin"
e = Cons()
e.program("prog", "main.c", "mod1.c")
When a Cons object is created, it is also given a number of variable settings that affects how things are built.
TODO: write more about evaluation, predefined variables, ....
Several of the methods of a Cons object expect a list of files (e.g. Cons.program()). The list of files can be “nested”. JCons will automatically “flatten” the list. The following examples are all equivalent:
e.program("prog", "f1.c", "f2.c", "f3.c", "f4.c", "f5.c")
e.program("prog", ["f1.c", "f2.c", "f3.c", "f4.c", "f5.c"])
f12 = ["f1.c", "f2.c"]
f45 = ["f4.c", ["f5.c"]]
e.program("prog", f12, ["f3.c"], f45)
JCons flattens file parameters nested as deep as five levels. This is an arbitrary limit, just to avoid “recursive” lists.
The way a file is compiled is affected by the Cons object used when calling a method (e.g. Cons.program()). Sometimes a program may consist of different groups of files that should be compiled differently. Suppose for example that your application consists of two sets of files that should be built in different ways. JCons handles this by allowing nested Cons parameters:
e = Cons()
e_foo = Cons(CFLAGS = "-DFOO")
e_bar = Cons(CFLAGS = "-DBAR")
foo_srcs = ["foo1.c", "foo2.c", ... "foo30.c"]
bar_srcs = ["bar1.c", "bar2.c", ... "bar40.c"]
e.program("prog", [e_foo, foo_srcs], [e_bar, bar_srcs], "mylib.a")
Here the files foo*.c will be built using the -DFOO setting, and the files bar*.c will be built using the -DBAR setting. The link step will use the first created Cons object (the variable e). For each file, the “closest enclosing” and preceding Cons object will be used.
This is the constructor, creating an object of type Cons. Most other methods described below are instance methods on the objects returned by this constructor. The constructor takes an optional hash-argument with settings of configuration variables affecting how things are to be built.
A Cons object is meant to capture a way of building things, e.g. a debug- or release-build, or the use of a specific compiler. You are free to create as many Cons objects as you need. An example demonstrates some typical uses:
e1 = Cons()
e1.program("foo1", "foo1.cpp", "bar1.c")
e2 = Cons(CFLAGS = "-g", CXXFLAGS = "-g")
e2.program("foo2", "foo2.cpp", "bar2.c")
This method creates a copy of an existing object, but modified by the settings given as arguments. A typical use is to make a slight modification of an already exiting object:
e1 = Cons(A="1", B="2", C="3")
e2 = e1.clone(B="22", D="4")
This is almost the same as the following:
e1 = Cons(A="1", B="2", C="3")
e2 = Cons(A="1", B="22", C="3", D="4")
This is the most basic way of describing how a “target” is built from a “source”. The command needed to build the target is specified explicitly:
e = Cons()
e.command("bar.pdf", "bar.ps", "ps2pdf bar.ps bar.pdf")
Instead of specifying the filenames in two places, the symbols %INPUT and %OUTPUT can be used in the command. JCons will replace these symbols automatically with the actual filenames before executing the command:
e = Cons()
e.command("bar.pdf", "bar.ps", "ps2pdf %INPUT %OUTPUT")
Both the target and the source parameters may be an array of files. So a command taking two input files, and producing three output files can be handled:
e = Cons()
e.command(["out1.txt", "out2.txt", "out3.txt"],
["in1.txt", "in2.txt"],
"some_program ...some_parameters...")
The %INPUT and %OUTPUT variables can be used here too, they will be set to a space separated list of the input/output files.
TODO: describe use of ”:uses_cpp”.
TODO: make this section more “reference style”
A common scenario is that a C/C++ program should be built from sources. JCons has a special method named program() for this. The same effect can in principle be accomplished with a number of calls to the command() method but it would be more clumsy. The most basic program() usage looks like:
e = Cons()
e.program("prog", "main.c", "mod1.c")
This will build an executable prog from the two source files main.c and mod1.c. The compiler settings used are the ones given by the Cons object used. In the example above with no parameters to the constructor, the “default” compiler for the platform will be chosen. On Mac OS X it might look like:
$ jcons .
gcc -c main.c -o main.o
gcc -c mod1.c -o mod1.o
gcc -o prog main.o mod1.o
The sources can be specified as individual arguments to the program method, or as an array. If the list of sources is long it might be convenient to use an array:
srcs = ["foo1.cpp", "foo2.cpp", ... "foo99.cpp"]
e.program("prog", srcs)
The “sources” can also be object files or library files:
e.program("prog", "foo.cpp", "bar.o", "mylib.a")
This method returns the “target”, i.e. the program built. This is almost the same as the target parameter, but is affected by the BUILD_* symbols, and on Windows, an ”.exe” file suffix has been added automatically.
This method is affected by the setting of BUILD_TOP, BUILD_SUBDIR and BUILD_SUFFIX (see the Variant builds section).
To build an object file from a source file with explicit control of the object filename, the object() method can be used:
e1 = Cons()
e1.object("foo.o", "foo.c")
e2 = Cons(CFLAGS = "-g")
e2.object("foo-debug.o", "foo.c")
The method does not return anything useful.
The objects() method tells JCons that a number of source files should be built, and returns a list of the object files produced:
e = Cons()
objs = e.objects("foo.c", "bar.c", "frotz.c")
e.program("foo", objs)
The lines above have the same effect as:
e = Cons()
objs = e.program("foo", "foo.c", "bar.c", "frotz.c")
This method is affected by the setting of BUILD_TOP, BUILD_SUBDIR and BUILD_SUFFIX (see the Variant builds section).
The static_library method tells JCons to build a static library from a number of source or object files:
e = Cons()
e.static_library("libfoo", "foo.c", "bar.c", "frotz.c")
TODO: describe ”.a” handling TODO: describe “lib*” handling TODO: example using the library
This method is affected by the setting of BUILD_TOP, BUILD_SUBDIR and BUILD_SUFFIX (see the Variant builds section).
Tell JCons that “target” depends on “source”, even if JCons can’t find this out by itself. This method is only useful as a complement to another method call, e.g. command or program. depends by itself has no way of of telling which command should be executed if the dependency “fires”.
TODO: example
Tell JCons that the program “target” depends on “source” when executed. If another build rule uses “target” as the command to execute, it will need to be rerun if the “source” has changed.
Copy the “source” to “target”.
Copy the “source” to “target_dir”.
Read a conscript.py file in a sub-directory.
JCons “knows” about a number of variables. These are listed here. Variables beginning with an “_” are set by JCons too.
Why are Make-like programs needed at all? The first and most obvious answer is: to save time. A small application can easily be built by a script, performing all steps every time it is invoked:
#!/bin/sh -e
g++ file1.cpp -o file1.o
g++ file2.cpp -o file2.o
g++ file3.cpp -o file3.o
g++ -o prog file1.o file2.o file3.o
Running this script may perhaps take a couple of seconds. The fact that we re-compile all files even if just one file has changed is not a problem. And if we are not sure that prog is up-to-date, we can run the script once again just to be sure to get an up-to-date prog. But what if we have a project with 800 source files? In principle we could build this program too with a simple script. But now the sheer number of files makes this take much longer (perhaps as long as an hour). We can no longer run the script just to be sure the program is up-to-date. And waiting an hour after just changing one source file is not a real option. A Make-like program on the other hand would detect that just one file had changed, and would re-compile that single file and then re-link the application, probably in less than a minute (instead of an hour). This is a huge time saving.
Another need for a Make-like program is: to make sure that all files are built in the right order. In a large project it may not be obvious exactly what order of commands is needed to produce the final program. Some source files may for example be generated by other tools, which in turn may be built as part of the whole build process. Keeping track of all dependencies “by hand” is difficult, and there is an definite risk that the final program will be built incorrectly.
Make-like programs solve the problems described above by specifying declaratively how different target files depend on their source files. It is then up to the Make-like program to decide which commands should be executed and in what order. The dependencies form a DAG (a directed acyclic graph), and there are well-known algorithms to traverse such a graph in the right order.
So if Make solves the build problem, and essentially does it “the right way”, why would there be a need for other tools than Make?
First: lack of global view of all dependencies. A large software project will span a number of different directories. The traditional Make-solution is to have a Makefile in each subdirectory and have the top Makefile orchestrate the build process by recursively calling Make in each subdirectory. Each Make-instance will then only have a partial view of the total dependency tree. This can easily lead to situations where the programmer feels compelled to invoke Make several times, just to make sure everything is updated correctly, in case the Makefiles don’t catch all global dependencies correctly (see Peter Millers paper: ”...”).
Second: Make-syntax is a poor “programming language”. Originally Make had a simple declarative syntax. But as time went by, the need for more “power” have lead to addition of a number of new features. This can be seen for example in GNU Make, the de facto standard in open source projects. GNU Make has got many programming language-like features. This “make on steroids” is very powerful, but the Makefiles often looks awful.
Third: poor file dependency tracking. A basic assumption in Make is that comparing file timestamps is a good way to see if a file need to be rebuilt. It is easy to “fool” Make that a file is up-to-date (just “touch” the file). Or if a source file is accidentally re-written to disk without any actual changes, a whole project may need to be rebuilt because of the changed timestamp.
Fourth: changed command line is not taken into account when deciding if a target file need to be updated. After changing some compiler option in a Makefile, a full rebuild may be needed to make sure all affected object files are updated correctly.
Fifth: “#include” dependencies are not tracked by Make. Sure, there are ways to compensate for this by having artificial entries in the Makefile, calling the compiler asking it for these dependencies. But this needs a lot of boilerplate code in the Makefile.
Sixth: implicit rules are bad ...
For some of the defects in Make there are workarounds today (e.g. #include file tracking). Others could in principle be solved in Make (e.g. using cryptographic checksums instead of timestamps). But the first two are not easily fixed within Make: 1) the poor programing language, and 2) the lack of global dependency view.
One example: on a program consisting of around 800 C++ files I have measured the time to verify that the program is “up-to-date”. I got the following numbers: Cons 42 s, SCons 102 s, JCons 1.03 s. These numbers were measured on an IMac G5 2.0 GHz from 2005 running MAC OS Leopard. In all three cases the build descriptions just use the most basic methods available: Cons.program(), Cons.static_library() and Cons.objects() (named slightly different in the different tools).