diff --git a/doc/source-builder.txt b/doc/source-builder.txt index 6e0b4bc..622f0fc 100644 --- a/doc/source-builder.txt +++ b/doc/source-builder.txt @@ -613,6 +613,17 @@ pays to run the command with the +--jobs=none+ option to get a log that is correctly sequenced. If searching the log file seach for +error:+ and the error should be just above it. +[[_contributing]] +Contributing +------------ + +The RSB is open source and open to contributions. These can be bug fixes, new +features or new configurations. Please break patches down into changes to the +core Python code, configuration changes or new configurations. + +Pleease email me patches via git so I can maintain your commit messages so you +are acknowledged as the contributor. + Project Sets ------------ @@ -1189,6 +1200,328 @@ and given a +-2+ revision number. Any dependent scripts referencing the earlier revision number will not be effected by the change. This locks down a specific configuration over time. +Personal Configurations +~~~~~~~~~~~~~~~~~~~~~~~ + +The RSB supports personal configurations. You can view the RTEMS support in the ++rtems+ directory as a private configuration tree that resides within the RSB +source. There is also the +bare+ set of configurations. You can create your own +configurations away from the RSB source tree yet use all that the RSB provides. + +To create a private configuration change to a suitable directory: + +------------------------------------------------------------- +$ cd ~/work +$ mkdir test +$ cd test +$ mkdir config +------------------------------------------------------------- + +and create a +config+ directory. Here you can add a new configuration or build +set file. The section 'Adding New Configurations' details how to add a new +confguration. + +New Configurations +~~~~~~~~~~~~~~~~~~ + +This section describes how to add a new configuration to the RSB. We will add a +configuration to build the Device Tree Compiler. The Device Tree Compiler or +DTC is part of the Flattened Device Tree project and compiles Device Tree +Source (DTS) files into Device Tree Blobs (DTB). DTB files can be loaded by +operating systems and used to locate the various resources such as base +addresses of devices or interrupt numbers allocated to devices. The Device Tree +Compiler source code can be downloaded from http://www.jdl.com/software. The +DTC is supported in the RSB and you can find the configuration files under the ++bare/config+ tree. I suggest you have a brief look over these files. + +Layering by Including +^^^^^^^^^^^^^^^^^^^^^ + +Configurations can be layered using the +%include+ directive. The user invokes +the outter layers which include inner layers until all the required +configuration is present and the package can be built. The outter layers can +provide high level details such as the version and the release and the inner +layers provide generic configuration details that do not change from one +release to another. Macro variables are used to provide the specific +configuration details. + +Configuration File Numbering +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Configuration files have a number at the end. This is a release number for that +configuraiton and it gives us the ability to track a specific configuration for +a specific version. For example lets say the developers of the DTC package +change the build system from a single makefile to autoconf and automake between +version 1.3.0 and version 1.4.0. The configuration file used to build the +package would change have to change. If we did not number the configuration +files the ability to build 1.1.0, 1.2.0 or 1.3.0 would be lost if we update a +common configuration file to build an autoconf and automake version. For +version 1.2.0 the same build script can be used so we can share the same +configuration file between version 1.1.0 and version 1.2.0. An update to any +previous release lets us still build the package. + +Common Configuration Scripts +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Common configuration scripts that are independent of version, platform and +architecture are useful to everyone. These live in the Source Builder's +configuration directory. Currrently there are scripts to build binutils, expat, +DTC, GCC, GDB and libusb. These files contain the recipes to build these +package without the specific details of the versions or patches being +built. They expect to be wrapped by a confguration file that ties the package +to a specific version and optionally specific patches. + +DTC Example +^^^^^^^^^^^ + +We will be building the DTC for your host rather than a package for RTEMS. We +will create a file called +source-builder/config/dtc-1-1.cfg+. This is a common +script that can be used to build a specific version using a general recipe. The +file name is 'dtc-1-1.cfg' where the 'cfg' extension indicates this is a +configuration file. The first *1* says this is for the major release 1 of the +package and the last *1* is the build configuration version. + +The file starts with some comments that detail the configuration. If there is +anything unsual about the configuration it is a good idea to add something in +the comments here. The comments are followed by a check for the release. In +this case if a release is not provided a default of 1 is used. + +------------------------------------------------------------- +# +# DTC 1.x.x Version 1. +# +# This configuration file configure's, make's and install's DTC. +# + +%if %{release} == %{nil} +%define release 1 +%endif +------------------------------------------------------------- + +The next section defines some information about the package. It does not effect +the build and is used to annotate the reports. It is recommended this +information is kept updated and accurate. + +------------------------------------------------------------- +Name: dtc-%{dtc_version}-%{_host}-%{release} +Summary: Device Tree Compiler v%{dtc_version} for target %{_target} on host %{_host} +Version: %{dtc_version} +Release: %{release} +URL: http://www.jdl.com/software/ +BuildRoot: %{_tmppath}/%{name}-root-%(%{__id_u} -n) +------------------------------------------------------------- + +The next section defines the source and any patches. In this case there is a +single source package and it can be downloaded using the HTTP protocol. The RSB +knows this is GZip'ped tar file. If more than one package package is needed add +them increasing the index. The +gcc-4.8-1.cfg+ configuration contains examples +of more than one source package as well as conditionally including source +packages based on the outter configuration options. + +------------------------------------------------------------- +# +# Source +# +Source0: http://www.jdl.com/software/dtc-v%{dtc_version}.tgz +------------------------------------------------------------- + +The remainder of the script is broken in to the various phases of a build. They +are: + +. Preperation +. Bulding +. Installing, and +. Cleaning + +Preperation is the unpacking of the source, applying any patches as well as any +package specific set ups. This part of the script is a standard Unix shell +script. Becareful with the the use of '%' and '$'. The RSB uses '%' while the +shell scripts use '$'. + +A standard pattern you will observe is the saving of the build's top +directory. This is used instead of changing into a subdirectory and then +changing to the parent when finished. Some hosts will change in a subdirectory +that is a link however changing to the parent does not change back to the +parent of the link rather it changes to the parent of the target of the link +and that is something the RSB nor you can track easily. The RSB configuration +script's are a collection of various subtle issues so please ask if you are +unsure why something is being done a particular way. + +The preparation phase will often include a number of conditional patch +commands. Outter layers can provide patches as needed while being able to use a +common recipe. Users can override the standard build and supply a custom patch +for testing using the user macro command line interface. + +------------------------------------------------------------- +# +# Prepare the source code. +# +%prep + build_top=$(pwd) + + %setup -q -n dtc-v%{dtc_version} + %{?patch0:%patch0 -p1} + %{?patch1:%patch1 -p1} + %{?patch2:%patch2 -p1} + %{?patch3:%patch3 -p1} + %{?patch4:%patch4 -p1} + %{?patch5:%patch5 -p1} + %{?patch6:%patch6 -p1} + %{?patch7:%patch7 -p1} + + cd ${build_top} +------------------------------------------------------------- + +The configuration file 'gcc-common-1.cfg' is a complex example of source +preparation. It contains a number of source packages and patches and it +combines these into a single source tree for building. It uses links to map +source into the GCC source tree so GCC can be built using the _single source +tree_ method. It also shows how to fetch source code from version +control. Newlib is taken directly from its CVS repository. + +Next is the building phase and for the DTC example this is simply a matter of +running +make+. Note the use of the RSB macros for commands. In the case of +'%{\__make}' it maps to the correct make for your host. In the case of BSD +systems we need to use the GNU make and not the GNU make. + +If your package requires a configuration stage you need to run this before the +make stage. Again the GCC common confguration file provides a detailed example. + +------------------------------------------------------------- +%build + build_top=$(pwd) + + cd dtc-v%{dtc_version} + + %{build_build_flags} + + %{__make} PREFIX=%{_prefix} + + cd ${build_top} +------------------------------------------------------------- + +You can invoke make with the macro '%{?_smp_flags}' as a command line +argument. This macro is controlled by the '--jobs' command line option and the +host CPU detection support in the RSB. If you are on a multicore host you can +increase the build speed using this macro. It also lets you disabled bulding on +multicores to aid debugging when tetsing. + +Next is the install phase. This phase is a little more complex because you may +be building a tar file and the end result of the build is never actually +installed into the prefix on the build host and you may not even have +permissions to perform a real install. Most packages install to the +prefix+ +and the prefix is typically supplied via the command to the RSB or the +package's default is used. The default can vary depending on the host's +operating system. To install to a path that is not the prefix the +DESTDIR+ +make variable is used. Most packages should honour the +DISTDIR+ make variables +and you can typically specify it on the command line to make when invoking the +install target. This results in the package being installed to a loction that +is not the prefix but one you can control. The RSB provides a shell variable +called +SB_BUILD_ROOT+ you can use. In a build set where you are building a +number of packages you can collect all the built packages in a single tree that +is captured in the tar file. + +Also note the use of the macro +%{\__rmdir}+. The use of these macros allow the +RSB to vary specific commands based on the host. This can help on hosts like +Windows where bugs can effect the standard commands such as 'rm'. There are +many many macros to help you. You can find these listed in the +defaults.mc+ +file and in the trace output. If you are new to creating and editing +configurations learning these can take a little time. + +------------------------------------------------------------- +%install + build_top=$(pwd) + + %{__rmdir} -rf $SB_BUILD_ROOT + + cd dtc-v%{dtc_version} + %{__make} DESTDIR=$SB_BUILD_ROOT PREFIX=%{_prefix} install + + cd ${build_top} +------------------------------------------------------------- + +Finally there is an optional clean section. The RSB will run this section if ++--no-clean+ has not been provided on the command line. The RSB does clean up +for you. + +Once we have the configuration files we can execute the build using the +`sb-builder` command. The command will perform the build and create a tar file +in the +tar+ directory. + +------------------------------------------------------------- +$ ../source-builder/sb-builder --prefix=/usr/local \ + --log=log_dtc devel/dtc-1.2.0 +RTEMS Source Builder, Package Builder v0.2.0 +config: devel/dtc-1.2.0 +package: dtc-1.2.0-x86_64-freebsd9.1-1 +download: http://www.jdl.com/software/dtc-v1.2.0.tgz -> sources/dtc-v1.2.0.tgz +building: dtc-1.2.0-x86_64-freebsd9.1-1 +$ ls tar +dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2 +------------------------------------------------------------- + +If you want to have the package installed automatically you need to create a +build set. A build set can build one or more packages from their configurations +at once to create a single package. For example the GNU tools is typically seen +as binutils, GCC and GDB and a build set will build each of these packages and +create a single build set tar file or install the tools on the host into the +prefix path. + +The DTC build set file is called +dtc.bset+ and contains: + +------------------------------------------------------------- +# +# Build the DTC. +# + +%define release 1 + +devel/dtc-1.2.0.cfg +------------------------------------------------------------- + +To build this you can use something similar to: + +------------------------------------------------------------- +$ ../source-builder/sb-set-builder --prefix=/usr/local --log=log_dtc \ + --trace --bset-tar-file --no-install dtc +RTEMS Source Builder - Set Builder, v0.2.0 +Build Set: dtc +config: devel/dtc-1.2.0.cfg +package: dtc-1.2.0-x86_64-freebsd9.1-1 +building: dtc-1.2.0-x86_64-freebsd9.1-1 +tarball: tar/x86_64-freebsd9.1-dtc-set.tar.bz2 +cleaning: dtc-1.2.0-x86_64-freebsd9.1-1 +Build Set: Time 0:00:02.865758 +$ ls tar +dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2 x86_64-freebsd9.1-dtc-set.tar.bz2 +------------------------------------------------------------- + +The build is for a FreeBSD host and the prefix is for user installed +packages. In this example I cannot let the source builder perform the install +because I never run the RSB with root priviledges so a build set or bset tar +file is created. This can then be installed using root privildges. + +The command also supplies the --trace option. The output in the log file will +contian all the macros. + +Debugging +^^^^^^^^^ + +New configuration files require debugging. There are two types of +debugging. The first is debugging RSB script bugs. The +--dry-run+ option is +used here. Suppling this option will result in most of the RSB processing to be +performed and suitable output placed in the log file. This with the +--trace+ +option should help you resolve any issues. + +The second type of bug to fix are related to the execution of one of +phases. These are usually a mix of shell script bugs or package set up or +configuration bugs. Here you can use any normal shell script type debug +technique such as +set -x+ to output the commands or +echo+ +statements. Debugging package related issues may require you start a build with +teh RSB and supply +--no-clean+ option and then locate the build directories +and change directory into them and manually run commands until to figure what +the package requires. + Snapshot Testing ~~~~~~~~~~~~~~~~ @@ -1843,7 +2176,6 @@ The +%bconf_without+ macro provides a way to test if the user has passed a specific option on the command line with the +--without-