diff options
Diffstat (limited to 'Documentation/kbuild')
| -rw-r--r-- | Documentation/kbuild/index.rst | 1 | ||||
| -rw-r--r-- | Documentation/kbuild/kbuild.rst | 28 | ||||
| -rw-r--r-- | Documentation/kbuild/kconfig-language.rst | 12 | ||||
| -rw-r--r-- | Documentation/kbuild/kconfig.rst | 5 | ||||
| -rw-r--r-- | Documentation/kbuild/makefiles.rst | 146 | ||||
| -rw-r--r-- | Documentation/kbuild/modules.rst | 27 | ||||
| -rw-r--r-- | Documentation/kbuild/reproducible-builds.rst | 127 |
7 files changed, 206 insertions, 140 deletions
diff --git a/Documentation/kbuild/index.rst b/Documentation/kbuild/index.rst index e323a3f2cc81..0f144fad99a6 100644 --- a/Documentation/kbuild/index.rst +++ b/Documentation/kbuild/index.rst @@ -18,6 +18,7 @@ Kernel Build System headers_install issues + reproducible-builds .. only:: subproject and html diff --git a/Documentation/kbuild/kbuild.rst b/Documentation/kbuild/kbuild.rst index 61b2181ed3ea..f1e5dce86af7 100644 --- a/Documentation/kbuild/kbuild.rst +++ b/Documentation/kbuild/kbuild.rst @@ -105,6 +105,15 @@ The output directory can also be specified using "O=...". Setting "O=..." takes precedence over KBUILD_OUTPUT. +KBUILD_EXTRA_WARN +----------------- +Specify the extra build checks. The same value can be assigned by passing +W=... from the command line. + +See `make help` for the list of the supported values. + +Setting "W=..." takes precedence over KBUILD_EXTRA_WARN. + KBUILD_DEBARCH -------------- For the deb-pkg target, allows overriding the normal heuristics deployed by @@ -241,11 +250,6 @@ To get all available archs you can also specify all. E.g.:: $ make ALLSOURCE_ARCHS=all tags -KBUILD_ENABLE_EXTRA_GCC_CHECKS ------------------------------- -If enabled over the make command line with "W=1", it turns on additional -gcc -W... options for more extensive build-time checking. - KBUILD_BUILD_TIMESTAMP ---------------------- Setting this to a date string overrides the timestamp used in the @@ -258,17 +262,3 @@ KBUILD_BUILD_USER, KBUILD_BUILD_HOST These two variables allow to override the user@host string displayed during boot and in /proc/version. The default value is the output of the commands whoami and host, respectively. - -KBUILD_LDS ----------- -The linker script with full path. Assigned by the top-level Makefile. - -KBUILD_VMLINUX_OBJS -------------------- -All object files for vmlinux. They are linked to vmlinux in the same -order as listed in KBUILD_VMLINUX_OBJS. - -KBUILD_VMLINUX_LIBS -------------------- -All .a "lib" files for vmlinux. KBUILD_VMLINUX_OBJS and KBUILD_VMLINUX_LIBS -together specify all the object files used to link vmlinux. diff --git a/Documentation/kbuild/kconfig-language.rst b/Documentation/kbuild/kconfig-language.rst index 74bef19f69f0..d0111dd26410 100644 --- a/Documentation/kbuild/kconfig-language.rst +++ b/Documentation/kbuild/kconfig-language.rst @@ -196,14 +196,11 @@ applicable everywhere (see syntax). or equal to the first symbol and smaller than or equal to the second symbol. -- help text: "help" or "---help---" +- help text: "help" This defines a help text. The end of the help text is determined by the indentation level, this means it ends at the first line which has a smaller indentation than the first line of the help text. - "---help---" and "help" do not differ in behaviour, "---help---" is - used to help visually separate configuration logic from help within - the file as an aid to developers. - misc options: "option" <symbol>[=<value>] @@ -594,7 +591,8 @@ The two different resolutions for b) can be tested in the sample Kconfig file Documentation/kbuild/Kconfig.recursion-issue-02. Below is a list of examples of prior fixes for these types of recursive issues; -all errors appear to involve one or more select's and one or more "depends on". +all errors appear to involve one or more "select" statements and one or more +"depends on". ============ =================================== commit fix @@ -656,7 +654,7 @@ the use of the xconfig configurator [1]_. Work should be done to confirm if the deduced semantics matches our intended Kconfig design goals. Having well defined semantics can be useful for tools for practical -evaluation of depenencies, for instance one such use known case was work to +evaluation of dependencies, for instance one such case was work to express in boolean abstraction of the inferred semantics of Kconfig to translate Kconfig logic into boolean formulas and run a SAT solver on this to find dead code / features (always inactive), 114 dead features were found in @@ -683,7 +681,7 @@ abstraction the inferred semantics of Kconfig to translate Kconfig logic into boolean formulas and run a SAT solver on it [5]_. Another known related project is CADOS [6]_ (former VAMOS [7]_) and the tools, mainly undertaker [8]_, which has been introduced first with [9]_. The basic concept of undertaker is to -exract variability models from Kconfig, and put them together with a +extract variability models from Kconfig and put them together with a propositional formula extracted from CPP #ifdefs and build-rules into a SAT solver in order to find dead code, dead files, and dead symbols. If using a SAT solver is desirable on Kconfig one approach would be to evaluate repurposing diff --git a/Documentation/kbuild/kconfig.rst b/Documentation/kbuild/kconfig.rst index a9a855f894b3..dce6801d66c9 100644 --- a/Documentation/kbuild/kconfig.rst +++ b/Documentation/kbuild/kconfig.rst @@ -154,11 +154,6 @@ KCONFIG_AUTOCONFIG This environment variable can be set to specify the path & name of the "auto.conf" file. Its default value is "include/config/auto.conf". -KCONFIG_TRISTATE ----------------- -This environment variable can be set to specify the path & name of the -"tristate.conf" file. Its default value is "include/config/tristate.conf". - KCONFIG_AUTOHEADER ------------------ This environment variable can be set to specify the path & name of the diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index f4f0f7ffde2b..0e0eb2c8da7d 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -28,7 +28,6 @@ This document describes the Linux kernel Makefiles. --- 4.3 Using C++ for host programs --- 4.4 Controlling compiler options for host programs --- 4.5 When host programs are actually built - --- 4.6 Using hostprogs-$(CONFIG_FOO) === 5 Kbuild clean infrastructure @@ -297,9 +296,19 @@ more details, with real examples. If CONFIG_EXT2_FS is set to either 'y' (built-in) or 'm' (modular) the corresponding obj- variable will be set, and kbuild will descend down in the ext2 directory. - Kbuild only uses this information to decide that it needs to visit - the directory, it is the Makefile in the subdirectory that - specifies what is modular and what is built-in. + + Kbuild uses this information not only to decide that it needs to visit + the directory, but also to decide whether or not to link objects from + the directory into vmlinux. + + When Kbuild descends into the directory with 'y', all built-in objects + from that directory are combined into the built-in.a, which will be + eventually linked into vmlinux. + + When Kbuild descends into the directory with 'm', in contrast, nothing + from that directory will be linked into vmlinux. If the Makefile in + that directory specifies obj-y, those objects will be left orphan. + It is very likely a bug of the Makefile or of dependencies in Kconfig. It is good practice to use a `CONFIG_` variable when assigning directory names. This allows kbuild to totally skip the directory if the @@ -471,21 +480,6 @@ more details, with real examples. The second argument is optional, and if supplied will be used if first argument is not supported. - cc-ldoption - cc-ldoption is used to check if $(CC) when used to link object files - supports the given option. An optional second option may be - specified if first option are not supported. - - Example:: - - #arch/x86/kernel/Makefile - vsyscall-flags += $(call cc-ldoption, -Wl$(comma)--hash-style=sysv) - - In the above example, vsyscall-flags will be assigned the option - -Wl$(comma)--hash-style=sysv if it is supported by $(CC). - The second argument is optional, and if supplied will be used - if first argument is not supported. - as-instr as-instr checks if the assembler reports a specific instruction and then outputs either option1 or option2 @@ -600,11 +594,11 @@ compilation stage. Two steps are required in order to use a host executable. The first step is to tell kbuild that a host program exists. This is -done utilising the variable hostprogs-y. +done utilising the variable "hostprogs". The second step is to add an explicit dependency to the executable. This can be done in two ways. Either add the dependency in a rule, -or utilise the variable $(always). +or utilise the variable "always-y". Both possibilities are described in the following. 4.1 Simple Host Program @@ -617,7 +611,7 @@ Both possibilities are described in the following. Example:: - hostprogs-y := bin2hex + hostprogs := bin2hex Kbuild assumes in the above example that bin2hex is made from a single c-source file named bin2hex.c located in the same directory as @@ -635,7 +629,7 @@ Both possibilities are described in the following. Example:: #scripts/lxdialog/Makefile - hostprogs-y := lxdialog + hostprogs := lxdialog lxdialog-objs := checklist.o lxdialog.o Objects with extension .o are compiled from the corresponding .c @@ -655,7 +649,7 @@ Both possibilities are described in the following. Example:: #scripts/kconfig/Makefile - hostprogs-y := qconf + hostprogs := qconf qconf-cxxobjs := qconf.o In the example above the executable is composed of the C++ file @@ -667,7 +661,7 @@ Both possibilities are described in the following. Example:: #scripts/kconfig/Makefile - hostprogs-y := qconf + hostprogs := qconf qconf-cxxobjs := qconf.o qconf-objs := check.o @@ -715,7 +709,7 @@ Both possibilities are described in the following. Example:: #drivers/pci/Makefile - hostprogs-y := gen-devlist + hostprogs := gen-devlist $(obj)/devlist.h: $(src)/pci.ids $(obj)/gen-devlist ( cd $(obj); ./gen-devlist ) < $< @@ -723,49 +717,34 @@ Both possibilities are described in the following. $(obj)/gen-devlist is updated. Note that references to the host programs in special rules must be prefixed with $(obj). - (2) Use $(always) + (2) Use always-y When there is no suitable special rule, and the host program - shall be built when a makefile is entered, the $(always) + shall be built when a makefile is entered, the always-y variable shall be used. Example:: #scripts/lxdialog/Makefile - hostprogs-y := lxdialog - always := $(hostprogs-y) + hostprogs := lxdialog + always-y := $(hostprogs) This will tell kbuild to build lxdialog even if not referenced in any rule. -4.6 Using hostprogs-$(CONFIG_FOO) ---------------------------------- - - A typical pattern in a Kbuild file looks like this: - - Example:: - - #scripts/Makefile - hostprogs-$(CONFIG_KALLSYMS) += kallsyms - - Kbuild knows about both 'y' for built-in and 'm' for module. - So if a config symbol evaluates to 'm', kbuild will still build - the binary. In other words, Kbuild handles hostprogs-m exactly - like hostprogs-y. But only hostprogs-y is recommended to be used - when no CONFIG symbols are involved. - 5 Kbuild clean infrastructure ============================= "make clean" deletes most generated files in the obj tree where the kernel is compiled. This includes generated files such as host programs. -Kbuild knows targets listed in $(hostprogs-y), $(hostprogs-m), $(always), -$(extra-y) and $(targets). They are all deleted during "make clean". -Files matching the patterns "*.[oas]", "*.ko", plus some additional files -generated by kbuild are deleted all over the kernel src tree when -"make clean" is executed. +Kbuild knows targets listed in $(hostprogs), $(always-y), $(always-m), +$(always-), $(extra-y), $(extra-) and $(targets). They are all deleted +during "make clean". Files matching the patterns "*.[oas]", "*.ko", plus +some additional files generated by kbuild are deleted all over the kernel +source tree when "make clean" is executed. -Additional files can be specified in kbuild makefiles by use of $(clean-files). +Additional files or directories can be specified in kbuild makefiles by use of +$(clean-files). Example:: @@ -776,23 +755,8 @@ When executing "make clean", the file "crc32table.h" will be deleted. Kbuild will assume files to be in the same relative directory as the Makefile, except if prefixed with $(objtree). -To delete a directory hierarchy use: - - Example:: - - #scripts/package/Makefile - clean-dirs := $(objtree)/debian/ - -This will delete the directory debian in the toplevel directory, including all -subdirectories. - -To exclude certain files from make clean, use the $(no-clean-files) variable. -This is only a special case used in the top level Kbuild file: - - Example:: - - #Kbuild - no-clean-files := $(bounds-file) $(offsets-file) +To exclude certain files or directories from make clean, use the +$(no-clean-files) variable. Usually kbuild descends down in subdirectories due to "obj-* := dir/", but in the architecture makefiles where the kbuild infrastructure @@ -983,18 +947,25 @@ When kbuild executes, the following steps are followed (roughly): From commandline LDFLAGS_MODULE shall be used (see kbuild.txt). - KBUILD_ARFLAGS Options for $(AR) when creating archives + KBUILD_LDS + + The linker script with full path. Assigned by the top-level Makefile. - $(KBUILD_ARFLAGS) set by the top level Makefile to "D" (deterministic - mode) if this option is supported by $(AR). + KBUILD_LDS_MODULE - ARCH_CPPFLAGS, ARCH_AFLAGS, ARCH_CFLAGS Overrides the kbuild defaults + The module linker script with full path. Assigned by the top-level + Makefile and additionally by the arch Makefile. - These variables are appended to the KBUILD_CPPFLAGS, - KBUILD_AFLAGS, and KBUILD_CFLAGS, respectively, after the - top-level Makefile has set any other flags. This provides a - means for an architecture to override the defaults. + KBUILD_VMLINUX_OBJS + All object files for vmlinux. They are linked to vmlinux in the same + order as listed in KBUILD_VMLINUX_OBJS. + + KBUILD_VMLINUX_LIBS + + All .a "lib" files for vmlinux. KBUILD_VMLINUX_OBJS and + KBUILD_VMLINUX_LIBS together specify all the object files used to + link vmlinux. 6.2 Add prerequisites to archheaders ------------------------------------ @@ -1137,23 +1108,6 @@ When kbuild executes, the following steps are followed (roughly): In this example, extra-y is used to list object files that shall be built, but shall not be linked as part of built-in.a. - header-test-y - - header-test-y specifies headers (*.h) in the current directory that - should be compile tested to ensure they are self-contained, - i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled, - this builds them as part of extra-y. - - header-test-pattern-y - - This works as a weaker version of header-test-y, and accepts wildcard - patterns. The typical usage is: - - header-test-pattern-y += *.h - - This specifies all the files that matches to '*.h' in the current - directory, but the files in 'header-test-' are excluded. - 6.7 Commands useful for building a boot image --------------------------------------------- @@ -1298,12 +1252,12 @@ When kbuild executes, the following steps are followed (roughly): Example:: #arch/x86/kernel/Makefile - always := vmlinux.lds + extra-y := vmlinux.lds #Makefile export CPPFLAGS_vmlinux.lds += -P -C -U$(ARCH) - The assignment to $(always) is used to tell kbuild to build the + The assignment to extra-y is used to tell kbuild to build the target vmlinux.lds. The assignment to $(CPPFLAGS_vmlinux.lds) tells kbuild to use the specified options when building the target vmlinux.lds. diff --git a/Documentation/kbuild/modules.rst b/Documentation/kbuild/modules.rst index 24e763482650..69fa48ee93d6 100644 --- a/Documentation/kbuild/modules.rst +++ b/Documentation/kbuild/modules.rst @@ -470,9 +470,12 @@ build. The syntax of the Module.symvers file is:: - <CRC> <Symbol> <module> + <CRC> <Symbol> <Namespace> <Module> <Export Type> - 0x2d036834 scsi_remove_host drivers/scsi/scsi_mod + 0xe1cc2a05 usb_stor_suspend USB_STORAGE drivers/usb/storage/usb-storage EXPORT_SYMBOL_GPL + + The fields are separated by tabs and values may be empty (e.g. + if no namespace is defined for an exported symbol). For a kernel build without CONFIG_MODVERSIONS enabled, the CRC would read 0x00000000. @@ -489,18 +492,16 @@ build. to the symbols from the kernel to check if all external symbols are defined. This is done in the MODPOST step. modpost obtains the symbols by reading Module.symvers from the kernel source - tree. If a Module.symvers file is present in the directory - where the external module is being built, this file will be - read too. During the MODPOST step, a new Module.symvers file - will be written containing all exported symbols that were not - defined in the kernel. + tree. During the MODPOST step, a new Module.symvers file will be + written containing all exported symbols from that external module. ---- 6.3 Symbols From Another External Module +6.3 Symbols From Another External Module +---------------------------------------- Sometimes, an external module uses exported symbols from - another external module. kbuild needs to have full knowledge of + another external module. Kbuild needs to have full knowledge of all symbols to avoid spitting out warnings about undefined - symbols. Three solutions exist for this situation. + symbols. Two solutions exist for this situation. NOTE: The method with a top-level kbuild file is recommended but may be impractical in certain situations. @@ -518,7 +519,7 @@ build. The top-level kbuild file would then look like:: #./Kbuild (or ./Makefile): - obj-y := foo/ bar/ + obj-m := foo/ bar/ And executing:: @@ -540,8 +541,8 @@ build. all symbols defined and not part of the kernel. Use "make" variable KBUILD_EXTRA_SYMBOLS - If it is impractical to copy Module.symvers from - another module, you can assign a space separated list + If it is impractical to add a top-level kbuild file, + you can assign a space separated list of files to KBUILD_EXTRA_SYMBOLS in your build file. These files will be loaded by modpost during the initialization of its symbol tables. diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst new file mode 100644 index 000000000000..503393854e2e --- /dev/null +++ b/Documentation/kbuild/reproducible-builds.rst @@ -0,0 +1,127 @@ +=================== +Reproducible builds +=================== + +It is generally desirable that building the same source code with +the same set of tools is reproducible, i.e. the output is always +exactly the same. This makes it possible to verify that the build +infrastructure for a binary distribution or embedded system has not +been subverted. This can also make it easier to verify that a source +or tool change does not make any difference to the resulting binaries. + +The `Reproducible Builds project`_ has more information about this +general topic. This document covers the various reasons why building +the kernel may be unreproducible, and how to avoid them. + +Timestamps +---------- + +The kernel embeds timestamps in three places: + +* The version string exposed by ``uname()`` and included in + ``/proc/version`` + +* File timestamps in the embedded initramfs + +* If enabled via ``CONFIG_IKHEADERS``, file timestamps of kernel + headers embedded in the kernel or respective module, + exposed via ``/sys/kernel/kheaders.tar.xz`` + +By default the timestamp is the current time and in the case of +``kheaders`` the various files' modification times. This must +be overridden using the `KBUILD_BUILD_TIMESTAMP`_ variable. +If you are building from a git commit, you could use its commit date. + +The kernel does *not* use the ``__DATE__`` and ``__TIME__`` macros, +and enables warnings if they are used. If you incorporate external +code that does use these, you must override the timestamp they +correspond to by setting the `SOURCE_DATE_EPOCH`_ environment +variable. + +User, host +---------- + +The kernel embeds the building user and host names in +``/proc/version``. These must be overridden using the +`KBUILD_BUILD_USER and KBUILD_BUILD_HOST`_ variables. If you are +building from a git commit, you could use its committer address. + +Absolute filenames +------------------ + +When the kernel is built out-of-tree, debug information may include +absolute filenames for the source files. This must be overridden by +including the ``-fdebug-prefix-map`` option in the `KCFLAGS`_ variable. + +Depending on the compiler used, the ``__FILE__`` macro may also expand +to an absolute filename in an out-of-tree build. Kbuild automatically +uses the ``-fmacro-prefix-map`` option to prevent this, if it is +supported. + +The Reproducible Builds web site has more information about these +`prefix-map options`_. + +Generated files in source packages +---------------------------------- + +The build processes for some programs under the ``tools/`` +subdirectory do not completely support out-of-tree builds. This may +cause a later source package build using e.g. ``make rpm-pkg`` to +include generated files. You should ensure the source tree is +pristine by running ``make mrproper`` or ``git clean -d -f -x`` before +building a source package. + +Module signing +-------------- + +If you enable ``CONFIG_MODULE_SIG_ALL``, the default behaviour is to +generate a different temporary key for each build, resulting in the +modules being unreproducible. However, including a signing key with +your source would presumably defeat the purpose of signing modules. + +One approach to this is to divide up the build process so that the +unreproducible parts can be treated as sources: + +1. Generate a persistent signing key. Add the certificate for the key + to the kernel source. + +2. Set the ``CONFIG_SYSTEM_TRUSTED_KEYS`` symbol to include the + signing key's certificate, set ``CONFIG_MODULE_SIG_KEY`` to an + empty string, and disable ``CONFIG_MODULE_SIG_ALL``. + Build the kernel and modules. + +3. Create detached signatures for the modules, and publish them as + sources. + +4. Perform a second build that attaches the module signatures. It + can either rebuild the modules or use the output of step 2. + +Structure randomisation +----------------------- + +If you enable ``CONFIG_GCC_PLUGIN_RANDSTRUCT``, you will need to +pre-generate the random seed in +``scripts/gcc-plgins/randomize_layout_seed.h`` so the same value +is used in rebuilds. + +Debug info conflicts +-------------------- + +This is not a problem of unreproducibility, but of generated files +being *too* reproducible. + +Once you set all the necessary variables for a reproducible build, a +vDSO's debug information may be identical even for different kernel +versions. This can result in file conflicts between debug information +packages for the different kernel versions. + +To avoid this, you can make the vDSO different for different +kernel versions by including an arbitrary string of "salt" in it. +This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``. + +.. _KBUILD_BUILD_TIMESTAMP: kbuild.html#kbuild-build-timestamp +.. _KBUILD_BUILD_USER and KBUILD_BUILD_HOST: kbuild.html#kbuild-build-user-kbuild-build-host +.. _KCFLAGS: kbuild.html#kcflags +.. _prefix-map options: https://reproducible-builds.org/docs/build-path/ +.. _Reproducible Builds project: https://reproducible-builds.org/ +.. _SOURCE_DATE_EPOCH: https://reproducible-builds.org/docs/source-date-epoch/ |
