diff options
Diffstat (limited to 'import-layers/yocto-poky/documentation/ref-manual/usingpoky.xml')
| -rw-r--r-- | import-layers/yocto-poky/documentation/ref-manual/usingpoky.xml | 2091 |
1 files changed, 0 insertions, 2091 deletions
diff --git a/import-layers/yocto-poky/documentation/ref-manual/usingpoky.xml b/import-layers/yocto-poky/documentation/ref-manual/usingpoky.xml deleted file mode 100644 index 13d9ad6ab..000000000 --- a/import-layers/yocto-poky/documentation/ref-manual/usingpoky.xml +++ /dev/null @@ -1,2091 +0,0 @@ -<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" -"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" -[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > - -<chapter id='usingpoky'> -<title>Using the Yocto Project</title> - - <para> - This chapter describes common usage for the Yocto Project. - The information is introductory in nature as other manuals in the Yocto Project - documentation set provide more details on how to use the Yocto Project. - </para> - -<section id='usingpoky-build'> - <title>Running a Build</title> - - <para> - This section provides a summary of the build process and provides information - for less obvious aspects of the build process. - For general information on how to build an image using the OpenEmbedded build - system, see the - "<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>" - section of the Yocto Project Quick Start. - </para> - - <section id='build-overview'> - <title>Build Overview</title> - - <para> - In the development environment you will need to build an image whenever you change hardware - support, add or change system libraries, or add or change services that have dependencies. - </para> - - <mediaobject> - <imageobject> - <imagedata fileref="figures/building-an-image.png" format="PNG" align='center' scalefit='1'/> - </imageobject> - <caption> - <para>Building an Image</para> - </caption> - </mediaobject> - - <para> - The first thing you need to do is set up the OpenEmbedded build - environment by sourcing the environment setup script - (i.e. - <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>). - Here is an example: - <literallayout class='monospaced'> - $ source &OE_INIT_FILE; [<replaceable>build_dir</replaceable>] - </literallayout> - </para> - - <para> - The <replaceable>build_dir</replaceable> argument is optional and specifies the directory the - OpenEmbedded build system uses for the build - - the - <link linkend='build-directory'>Build Directory</link>. - If you do not specify a Build Directory, it defaults to a directory - named <filename>build</filename> in your current working directory. - A common practice is to use a different Build Directory for different targets. - For example, <filename>~/build/x86</filename> for a <filename>qemux86</filename> - target, and <filename>~/build/arm</filename> for a <filename>qemuarm</filename> target. - </para> - - <para> - Once the build environment is set up, you can build a target using: - <literallayout class='monospaced'> - $ bitbake <replaceable>target</replaceable> - </literallayout> - <note> - <para> - If you experience a build error due to resources - temporarily being unavailable and it appears you - should not be having this issue, it might be due - to the combination of a 4.3+ Linux kernel and - <filename>systemd</filename> version 228+ - (i.e. see this - <ulink url='http://unix.stackexchange.com/questions/253903/creating-threads-fails-with-resource-temporarily-unavailable-with-4-3-kernel'>link</ulink> - for information). - </para> - - <para> - To work around this issue, you can try either - of the following: - <itemizedlist> - <listitem><para> - Try the build again. - </para></listitem> - <listitem><para> - Modify the "DefaultTasksMax" - <filename>systemd</filename> parameter - by uncommenting it and setting it to - "infinity". - You can find this parameter in the - <filename>system.conf</filename> file - located in - <filename>/etc/systemd</filename> - on most systems. - </para></listitem> - </itemizedlist> - </para> - </note> - </para> - - <para> - The <replaceable>target</replaceable> is the name of the recipe you want to build. - Common targets are the images in <filename>meta/recipes-core/images</filename>, - <filename>meta/recipes-sato/images</filename>, etc. all found in the - <link linkend='source-directory'>Source Directory</link>. - Or, the target can be the name of a recipe for a specific piece of software such as - BusyBox. - For more details about the images the OpenEmbedded build system supports, see the - "<link linkend="ref-images">Images</link>" chapter. - </para> - - <note> - Building an image without GNU General Public License Version - 3 (GPLv3), or similarly licensed, components is supported for - only minimal and base images. - See the "<link linkend='ref-images'>Images</link>" chapter for more information. - </note> - </section> - - <section id='building-an-image-using-gpl-components'> - <title>Building an Image Using GPL Components</title> - - <para> - When building an image using GPL components, you need to maintain your original - settings and not switch back and forth applying different versions of the GNU - General Public License. - If you rebuild using different versions of GPL, dependency errors might occur - due to some components not being rebuilt. - </para> - </section> -</section> - -<section id='usingpoky-install'> - <title>Installing and Using the Result</title> - - <para> - Once an image has been built, it often needs to be installed. - The images and kernels built by the OpenEmbedded build system are placed in the - <link linkend='build-directory'>Build Directory</link> in - <filename class="directory">tmp/deploy/images</filename>. - For information on how to run pre-built images such as <filename>qemux86</filename> - and <filename>qemuarm</filename>, see the - <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink> - manual. - For information about how to install these images, see the documentation for your - particular board or machine. - </para> -</section> - -<section id='usingpoky-debugging-tools-and-techniques'> - <title>Debugging Tools and Techniques</title> - - <para> - The exact method for debugging build failures depends on the nature of - the problem and on the system's area from which the bug originates. - Standard debugging practices such as comparison against the last - known working version with examination of the changes and the - re-application of steps to identify the one causing the problem are - valid for the Yocto Project just as they are for any other system. - Even though it is impossible to detail every possible potential failure, - this section provides some general tips to aid in debugging. - </para> - - <para> - A useful feature for debugging is the error reporting tool. - Configuring the Yocto Project to use this tool causes the - OpenEmbedded build system to produce error reporting commands as - part of the console output. - You can enter the commands after the build completes - to log error information - into a common database, that can help you figure out what might be - going wrong. - For information on how to enable and use this feature, see the - "<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-error-reporting-tool'>Using the Error Reporting Tool</ulink>" - section in the Yocto Project Development Tasks Manual. - </para> - - <para> - For discussions on debugging, see the - "<ulink url='&YOCTO_DOCS_DEV_URL;#platdev-gdb-remotedebug'>Debugging With the GNU Project Debugger (GDB) Remotely</ulink>" section - in the Yocto Project Development Tasks Manual - and the - "<ulink url='&YOCTO_DOCS_SDK_URL;#adt-eclipse'>Working within Eclipse</ulink>" - section in the Yocto Project Application Development and the - Extensible Software Development Kit (eSDK) manual. - </para> - - <note> - The remainder of this section presents many examples of the - <filename>bitbake</filename> command. - You can learn about BitBake by reading the - <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>. - </note> - - <section id='usingpoky-debugging-viewing-logs-from-failed-tasks'> - <title>Viewing Logs from Failed Tasks</title> - - <para> - You can find the log for a task in the file - <filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}/temp/log.do_</filename><replaceable>taskname</replaceable>. - For example, the log for the - <link linkend='ref-tasks-compile'><filename>do_compile</filename></link> - task of the QEMU minimal image for the x86 machine - (<filename>qemux86</filename>) might be in - <filename>tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile</filename>. - To see the commands - <link linkend='bitbake-term'>BitBake</link> ran - to generate a log, look at the corresponding - <filename>run.do_</filename><replaceable>taskname</replaceable> - file in the same directory. - </para> - - <para> - <filename>log.do_</filename><replaceable>taskname</replaceable> and - <filename>run.do_</filename><replaceable>taskname</replaceable> - are actually symbolic links to - <filename>log.do_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable> - and - <filename>log.run_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable>, - where <replaceable>pid</replaceable> is the PID the task had when - it ran. - The symlinks always point to the files corresponding to the most - recent run. - </para> - </section> - - <section id='usingpoky-debugging-viewing-variable-values'> - <title>Viewing Variable Values</title> - <para> - BitBake's <filename>-e</filename> option is used to display - variable values after parsing. - The following command displays the variable values after the - configuration files (i.e. <filename>local.conf</filename>, - <filename>bblayers.conf</filename>, - <filename>bitbake.conf</filename> and so forth) have been - parsed: - <literallayout class='monospaced'> - $ bitbake -e - </literallayout> - The following command displays variable values after a specific - recipe has been parsed. - The variables include those from the configuration as well: - <literallayout class='monospaced'> - $ bitbake -e recipename - </literallayout> - <note><para> - Each recipe has its own private set of variables (datastore). - Internally, after parsing the configuration, a copy of the - resulting datastore is made prior to parsing each recipe. - This copying implies that variables set in one recipe will - not be visible to other recipes.</para> - - <para>Likewise, each task within a recipe gets a private - datastore based on the recipe datastore, which means that - variables set within one task will not be visible to - other tasks.</para> - </note> - </para> - - <para> - In the output of <filename>bitbake -e</filename>, each variable is - preceded by a description of how the variable got its value, - including temporary values that were later overriden. - This description also includes variable flags (varflags) set on - the variable. - The output can be very helpful during debugging. - </para> - - <para> - Variables that are exported to the environment are preceded by - <filename>export</filename> in the output of - <filename>bitbake -e</filename>. - See the following example: - <literallayout class='monospaced'> - export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86" - </literallayout> - </para> - - <para> - In addition to variable values, the output of the - <filename>bitbake -e</filename> and - <filename>bitbake -e</filename> <replaceable>recipe</replaceable> - commands includes the following information: - <itemizedlist> - <listitem><para> - The output starts with a tree listing all configuration - files and classes included globally, recursively listing - the files they include or inherit in turn. - Much of the behavior of the OpenEmbedded build system - (including the behavior of the - <link linkend='normal-recipe-build-tasks'>normal recipe build tasks</link>) - is implemented in the - <link linkend='ref-classes-base'><filename>base</filename></link> - class and the classes it inherits, rather than being built - into BitBake itself. - </para></listitem> - <listitem><para> - After the variable values, all functions appear in the - output. - For shell functions, variables referenced within the - function body are expanded. - If a function has been modified using overrides or - using override-style operators like - <filename>_append</filename> and - <filename>_prepend</filename>, then the final assembled - function body appears in the output. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='viewing-package-information-with-oe-pkgdata-util'> - <title>Viewing Package Information with <filename>oe-pkgdata-util</filename></title> - - <para> - You can use the <filename>oe-pkgdata-util</filename> command-line - utility to query - <link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link> - and display various package-related information. - When you use the utility, you must use it to view information - on packages that have already been built. - </para> - - <para> - Following are a few of the available - <filename>oe-pkgdata-util</filename> subcommands. - <note> - You can use the standard * and ? globbing wildcards as part of - package names and paths. - </note> - <itemizedlist> - <listitem><para> - <filename>oe-pkgdata-util list-pkgs [</filename><replaceable>pattern</replaceable><filename>]</filename>: - Lists all packages that have been built, optionally - limiting the match to packages that match - <replaceable>pattern</replaceable>. - </para></listitem> - <listitem><para> - <filename>oe-pkgdata-util list-pkg-files </filename><replaceable>package</replaceable><filename> ...</filename>: - Lists the files and directories contained in the given - packages. - <note> - <para> - A different way to view the contents of a package is - to look at the - <filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}/packages-split</filename> - directory of the recipe that generates the - package. - This directory is created by the - <link linkend='ref-tasks-package'><filename>do_package</filename></link> - task and has one subdirectory for each package the - recipe generates, which contains the files stored in - that package.</para> - <para> - If you want to inspect the - <filename>${WORKDIR}/packages-split</filename> - directory, make sure that - <link linkend='ref-classes-rm-work'><filename>rm_work</filename></link> - is not enabled when you build the recipe. - </para> - </note> - </para></listitem> - <listitem><para> - <filename>oe-pkgdata-util find-path </filename><replaceable>path</replaceable><filename> ...</filename>: - Lists the names of the packages that contain the given - paths. - For example, the following tells us that - <filename>/usr/share/man/man1/make.1</filename> - is contained in the <filename>make-doc</filename> - package: - <literallayout class='monospaced'> - $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 - make-doc: /usr/share/man/man1/make.1 - </literallayout> - </para></listitem> - <listitem><para> - <filename>oe-pkgdata-util lookup-recipe </filename><replaceable>package</replaceable><filename> ...</filename>: - Lists the name of the recipes that - produce the given packages. - </para></listitem> - </itemizedlist> - </para> - - <para> - For more information on the <filename>oe-pkgdata-util</filename> - command, use the help facility: - <literallayout class='monospaced'> - $ oe-pkgdata-util ‐‐help - $ oe-pkgdata-util <replaceable>subcommand</replaceable> --help - </literallayout> - </para> - </section> - - <section id='usingpoky-viewing-dependencies-between-recipes-and-tasks'> - <title>Viewing Dependencies Between Recipes and Tasks</title> - - <para> - Sometimes it can be hard to see why BitBake wants to build other - recipes before the one you have specified. - Dependency information can help you understand why a recipe is - built. - </para> - - <para> - To generate dependency information for a recipe, run the following - command: - <literallayout class='monospaced'> - $ bitbake -g <replaceable>recipename</replaceable> - </literallayout> - This command writes the following files in the current directory: - <itemizedlist> - <listitem><para> - <filename>pn-buildlist</filename>: A list of - recipes/targets involved in building - <replaceable>recipename</replaceable>. - "Involved" here means that at least one task from the - recipe needs to run when building - <replaceable>recipename</replaceable> from scratch. - Targets that are in - <link linkend='var-ASSUME_PROVIDED'><filename>ASSUME_PROVIDED</filename></link> - are not listed. - </para></listitem> - <listitem><para> - <filename>task-depends.dot</filename>: A graph showing - dependencies between tasks. - </para></listitem> - </itemizedlist> - </para> - - <para> - The graphs are in - <ulink url='https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29'>DOT</ulink> - format and can be converted to images (e.g. using the - <filename>dot</filename> tool from - <ulink url='http://www.graphviz.org/'>Graphviz</ulink>). - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - DOT files use a plain text format. - The graphs generated using the - <filename>bitbake -g</filename> command are often so - large as to be difficult to read without special - pruning (e.g. with Bitbake's - <filename>-I</filename> option) and processing. - Despite the form and size of the graphs, the - corresponding <filename>.dot</filename> files can still - be possible to read and provide useful information. - </para> - - <para>As an example, the - <filename>task-depends.dot</filename> file contains - lines such as the following: - <literallayout class='monospaced'> - "libxslt.do_configure" -> "libxml2.do_populate_sysroot" - </literallayout> - The above example line reveals that the - <link linkend='ref-tasks-configure'><filename>do_configure</filename></link> - task in <filename>libxslt</filename> depends on the - <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link> - task in <filename>libxml2</filename>, which is a normal - <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link> - dependency between the two recipes. - </para></listitem> - <listitem><para> - For an example of how <filename>.dot</filename> files - can be processed, see the - <filename>scripts/contrib/graph-tool</filename> Python - script, which finds and displays paths between graph - nodes. - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - You can use a different method to view dependency information - by using the following command: - <literallayout class='monospaced'> - $ bitbake -g -u taskexp <replaceable>recipename</replaceable> - </literallayout> - This command displays a GUI window from which you can view - build-time and runtime dependencies for the recipes involved in - building <replaceable>recipename</replaceable>. - </para> - </section> - - <section id='usingpoky-viewing-task-variable-dependencies'> - <title>Viewing Task Variable Dependencies</title> - - <para> - As mentioned in the - "<ulink url='&YOCTO_DOCS_BB_URL;#checksums'>Checksums (Signatures)</ulink>" - section of the BitBake User Manual, BitBake tries to automatically - determine what variables a task depends on so that it can rerun - the task if any values of the variables change. - This determination is usually reliable. - However, if you do things like construct variable names at runtime, - then you might have to manually declare dependencies on those - variables using <filename>vardeps</filename> as described in the - "<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'>Variable Flags</ulink>" - section of the BitBake User Manual. - </para> - - <para> - If you are unsure whether a variable dependency is being picked up - automatically for a given task, you can list the variable - dependencies BitBake has determined by doing the following: - <orderedlist> - <listitem><para> - Build the recipe containing the task: - <literallayout class='monospaced'> - $ bitbake <replaceable>recipename</replaceable> - </literallayout> - </para></listitem> - <listitem><para> - Inside the - <link linkend='var-STAMPS_DIR'><filename>STAMPS_DIR</filename></link> - directory, find the signature data - (<filename>sigdata</filename>) file that corresponds to the - task. - The <filename>sigdata</filename> files contain a pickled - Python database of all the metadata that went into creating - the input checksum for the task. - As an example, for the - <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link> - task of the <filename>db</filename> recipe, the - <filename>sigdata</filename> file might be found in the - following location: - <literallayout class='monospaced'> - ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 - </literallayout> - For tasks that are accelerated through the shared state - (<link linkend='shared-state-cache'>sstate</link>) - cache, an additional <filename>siginfo</filename> file is - written into - <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> - along with the cached task output. - The <filename>siginfo</filename> files contain exactly the - same information as <filename>sigdata</filename> files. - </para></listitem> - <listitem><para> - Run <filename>bitbake-dumpsig</filename> on the - <filename>sigdata</filename> or - <filename>siginfo</filename> file. - Here is an example: - <literallayout class='monospaced'> - $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 - </literallayout> - In the output of the above command, you will find a line - like the following, which lists all the (inferred) variable - dependencies for the task. - This list also includes indirect dependencies from - variables depending on other variables, recursively. - <literallayout class='monospaced'> - Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch'] - </literallayout> - <note> - Functions (e.g. <filename>base_do_fetch</filename>) - also count as variable dependencies. - These functions in turn depend on the variables they - reference. - </note> - The output of <filename>bitbake-dumpsig</filename> also includes - the value each variable had, a list of dependencies for each - variable, and - <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST'><filename>BB_HASHBASE_WHITELIST</filename></ulink> - information. - </para></listitem> - </orderedlist> - </para> - - <para> - There is also a <filename>bitbake-diffsigs</filename> command for - comparing two <filename>siginfo</filename> or - <filename>sigdata</filename> files. - This command can be helpful when trying to figure out what changed - between two versions of a task. - If you call <filename>bitbake-diffsigs</filename> with just one - file, the command behaves like - <filename>bitbake-dumpsig</filename>. - </para> - - <para> - You can also use BitBake to dump out the signature construction - information without executing tasks by using either of the - following BitBake command-line options: - <literallayout class='monospaced'> - ‐‐dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable> - -S <replaceable>SIGNATURE_HANDLER</replaceable> - </literallayout> - <note> - Two common values for - <replaceable>SIGNATURE_HANDLER</replaceable> are "none" and - "printdiff", which dump only the signature or compare the - dumped signature with the cached one, respectively. - </note> - Using BitBake with either of these options causes BitBake to dump - out <filename>sigdata</filename> files in the - <filename>stamps</filename> directory for every task it would have - executed instead of building the specified target package. - </para> - </section> - - <section id='usingpoky-debugging-taskrunning'> - <title>Running Specific Tasks</title> - - <para> - Any given recipe consists of a set of tasks. - The standard BitBake behavior in most cases is: - <filename>do_fetch</filename>, - <filename>do_unpack</filename>, - <filename>do_patch</filename>, <filename>do_configure</filename>, - <filename>do_compile</filename>, <filename>do_install</filename>, - <filename>do_package</filename>, - <filename>do_package_write_*</filename>, and - <filename>do_build</filename>. - The default task is <filename>do_build</filename> and any tasks - on which it depends build first. - Some tasks, such as <filename>do_devshell</filename>, are not part - of the default build chain. - If you wish to run a task that is not part of the default build - chain, you can use the <filename>-c</filename> option in BitBake. - Here is an example: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c devshell - </literallayout> - </para> - - <para> - The <filename>-c</filename> option respects task dependencies, - which means that all other tasks (including tasks from other - recipes) that the specified task depends on will be run before the - task. - Even when you manually specify a task to run with - <filename>-c</filename>, BitBake will only run the task if it - considers it "out of date". - See the - "<link linkend='stamp-files-and-the-rerunning-of-tasks'>Stamp Files and the Rerunning of Tasks</link>" - section for how BitBake determines whether a task is "out of date". - </para> - - <para> - If you want to force an up-to-date task to be rerun (e.g. - because you made manual modifications to the recipe's - <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> - that you want to try out), then you can use the - <filename>-f</filename> option. - <note> - The reason <filename>-f</filename> is never required when - running the - <link linkend='ref-tasks-devshell'><filename>do_devshell</filename></link> - task is because the - <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink><filename>]</filename> - variable flag is already set for the task. - </note> - The following example shows one way you can use the - <filename>-f</filename> option: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop - . - . - make some changes to the source code in the work directory - . - . - $ bitbake matchbox-desktop -c compile -f - $ bitbake matchbox-desktop - </literallayout> - </para> - - <para> - This sequence first builds and then recompiles - <filename>matchbox-desktop</filename>. - The last command reruns all tasks (basically the packaging tasks) - after the compile. - BitBake recognizes that the <filename>do_compile</filename> - task was rerun and therefore understands that the other tasks - also need to be run again. - </para> - - <para> - Another, shorter way to rerun a task and all - <link linkend='normal-recipe-build-tasks'>normal recipe build tasks</link> - that depend on it is to use the <filename>-C</filename> - option. - <note> - This option is upper-cased and is separate from the - <filename>-c</filename> option, which is lower-cased. - </note> - Using this option invalidates the given task and then runs the - <link linkend='ref-tasks-build'><filename>do_build</filename></link> - task, which is the default task if no task is given, and the - tasks on which it depends. - You could replace the final two commands in the previous example - with the following single command: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -C compile - </literallayout> - Internally, the <filename>-f</filename> and - <filename>-C</filename> options work by tainting (modifying) the - input checksum of the specified task. - This tainting indirectly causes the task and its - dependent tasks to be rerun through the normal task dependency - mechanisms. - <note> - BitBake explicitly keeps track of which tasks have been - tainted in this fashion, and will print warnings such as the - following for builds involving such tasks: - <literallayout class='monospaced'> - WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run - </literallayout> - The purpose of the warning is to let you know that the work - directory and build output might not be in the clean state they - would be in for a "normal" build, depending on what actions - you took. - To get rid of such warnings, you can remove the work directory - and rebuild the recipe, as follows: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c clean - $ bitbake matchbox-desktop - </literallayout> - </note> - </para> - - <para> - You can view a list of tasks in a given package by running the - <filename>do_listtasks</filename> task as follows: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c listtasks - </literallayout> - The results appear as output to the console and are also in the - file <filename>${WORKDIR}/temp/log.do_listtasks</filename>. - </para> - </section> - - <section id='usingpoky-debugging-bitbake'> - <title>General BitBake Problems</title> - - <para> - You can see debug output from BitBake by using the <filename>-D</filename> option. - The debug output gives more information about what BitBake - is doing and the reason behind it. - Each <filename>-D</filename> option you use increases the logging level. - The most common usage is <filename>-DDD</filename>. - </para> - - <para> - The output from <filename>bitbake -DDD -v</filename> <replaceable>targetname</replaceable> can reveal why - BitBake chose a certain version of a package or why BitBake - picked a certain provider. - This command could also help you in a situation where you think BitBake did something - unexpected. - </para> - </section> - - <section id='development-host-system-issues'> - <title>Development Host System Issues</title> - - <para> - Sometimes issues on the host development system can cause your - build to fail. - Following are known, host-specific problems. - Be sure to always consult the - <ulink url='&YOCTO_RELEASE_NOTES;'>Release Notes</ulink> - for a look at all release-related issues. - <itemizedlist> - <listitem><para><emphasis><filename>glibc-initial</filename> fails to build</emphasis>: - If your development host system has the unpatched - <filename>GNU Make 3.82</filename>, - the - <link linkend='ref-tasks-install'><filename>do_install</filename></link> - task fails for <filename>glibc-initial</filename> during - the build.</para> - <para>Typically, every distribution that ships - <filename>GNU Make 3.82</filename> as - the default already has the patched version. - However, some distributions, such as Debian, have - <filename>GNU Make 3.82</filename> as an option, which - is unpatched. - You will see this error on these types of distributions. - Switch to <filename>GNU Make 3.81</filename> or patch - your <filename>make</filename> to solve the problem. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='usingpoky-debugging-buildfile'> - <title>Building with No Dependencies</title> - <para> - To build a specific recipe (<filename>.bb</filename> file), - you can use the following command form: - <literallayout class='monospaced'> - $ bitbake -b <replaceable>somepath</replaceable>/<replaceable>somerecipe</replaceable>.bb - </literallayout> - This command form does not check for dependencies. - Consequently, you should use it - only when you know existing dependencies have been met. - <note> - You can also specify fragments of the filename. - In this case, BitBake checks for a unique match. - </note> - </para> - </section> - - <section id='recipe-logging-mechanisms'> - <title>Recipe Logging Mechanisms</title> - <para> - The Yocto Project provides several logging functions for producing - debugging output and reporting errors and warnings. - For Python functions, the following logging functions exist. - All of these functions log to - <filename>${T}/log.do_</filename><replaceable>task</replaceable>, - and can also log to standard output (stdout) with the right - settings: - <itemizedlist> - <listitem><para> - <filename>bb.plain(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes <replaceable>msg</replaceable> as is to the log while - also logging to stdout. - </para></listitem> - <listitem><para> - <filename>bb.note(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "NOTE: <replaceable>msg</replaceable>" to the log. - Also logs to stdout if BitBake is called with "-v". - </para></listitem> - <listitem><para> - <filename>bb.debug(</filename><replaceable>level</replaceable><filename>, </filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "DEBUG: <replaceable>msg</replaceable>" to the log. - Also logs to stdout if the log level is greater than or - equal to <replaceable>level</replaceable>. - See the - "<ulink url='&YOCTO_DOCS_BB_URL;#usage-and-syntax'>-D</ulink>" - option in the BitBake User Manual for more information. - </para></listitem> - <listitem><para> - <filename>bb.warn(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "WARNING: <replaceable>msg</replaceable>" to the log - while also logging to stdout. - </para></listitem> - <listitem><para> - <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "ERROR: <replaceable>msg</replaceable>" to the log - while also logging to stdout. - <note> - Calling this function does not cause the task to fail. - </note> - </para></listitem> - <listitem><para> - <filename>bb.fatal(</filename><replaceable>msg</replaceable><filename>)</filename>: - This logging function is similar to - <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename> - but also causes the calling task to fail. - <note> - <filename>bb.fatal()</filename> raises an exception, - which means you do not need to put a "return" - statement after the function. - </note> - </para></listitem> - </itemizedlist> - </para> - - <para> - The same logging functions are also available in shell functions, - under the names - <filename>bbplain</filename>, <filename>bbnote</filename>, - <filename>bbdebug</filename>, <filename>bbwarn</filename>, - <filename>bberror</filename>, and <filename>bbfatal</filename>. - The - <link linkend='ref-classes-logging'><filename>logging</filename></link> - class implements these functions. - See that class in the - <filename>meta/classes</filename> folder of the - <link linkend='source-directory'>Source Directory</link> - for information. - </para> - - <section id='logging-with-python'> - <title>Logging With Python</title> - <para> - When creating recipes using Python and inserting code that handles build logs, - keep in mind the goal is to have informative logs while keeping the console as - "silent" as possible. - Also, if you want status messages in the log, use the "debug" loglevel. - </para> - - <para> - Following is an example written in Python. - The code handles logging for a function that determines the - number of tasks needed to be run. - See the - "<link linkend='ref-tasks-listtasks'><filename>do_listtasks</filename></link>" - section for additional information: - <literallayout class='monospaced'> - python do_listtasks() { - bb.debug(2, "Starting to figure out the task list") - if noteworthy_condition: - bb.note("There are 47 tasks to run") - bb.debug(2, "Got to point xyz") - if warning_trigger: - bb.warn("Detected warning_trigger, this might be a problem later.") - if recoverable_error: - bb.error("Hit recoverable_error, you really need to fix this!") - if fatal_error: - bb.fatal("fatal_error detected, unable to print the task list") - bb.plain("The tasks present are abc") - bb.debug(2, "Finished figuring out the tasklist") - } - </literallayout> - </para> - </section> - - <section id='logging-with-bash'> - <title>Logging With Bash</title> - <para> - When creating recipes using Bash and inserting code that handles build - logs, you have the same goals - informative with minimal console output. - The syntax you use for recipes written in Bash is similar to that of - recipes written in Python described in the previous section. - </para> - - <para> - Following is an example written in Bash. - The code logs the progress of the <filename>do_my_function</filename> function. - <literallayout class='monospaced'> - do_my_function() { - bbdebug 2 "Running do_my_function" - if [ exceptional_condition ]; then - bbnote "Hit exceptional_condition" - fi - bbdebug 2 "Got to point xyz" - if [ warning_trigger ]; then - bbwarn "Detected warning_trigger, this might cause a problem later." - fi - if [ recoverable_error ]; then - bberror "Hit recoverable_error, correcting" - fi - if [ fatal_error ]; then - bbfatal "fatal_error detected" - fi - bbdebug 2 "Completed do_my_function" - } - </literallayout> - </para> - </section> - </section> - - <section id='usingpoky-debugging-others'> - <title>Other Tips</title> - - <para> - Here are some other tips that you might find useful: - <itemizedlist> - <listitem><para> - When adding new packages, it is worth watching for - undesirable items making their way into compiler command - lines. - For example, you do not want references to local system - files like - <filename>/usr/lib/</filename> or - <filename>/usr/include/</filename>. - </para></listitem> - <listitem><para> - If you want to remove the <filename>psplash</filename> - boot splashscreen, - add <filename>psplash=false</filename> to the kernel - command line. - Doing so prevents <filename>psplash</filename> from loading - and thus allows you to see the console. - It is also possible to switch out of the splashscreen by - switching the virtual console (e.g. Fn+Left or Fn+Right - on a Zaurus). - </para></listitem> - <listitem><para> - Removing - <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> - (usually <filename>tmp/</filename>, within the - <link linkend='build-directory'>Build Directory</link>) - can often fix temporary build issues. - Removing <filename>TMPDIR</filename> is usually a - relatively cheap operation, because task output will be - cached in - <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> - (usually <filename>sstate-cache/</filename>, which is - also in the Build Directory). - <note> - Removing <filename>TMPDIR</filename> might be a - workaround rather than a fix. - Consequently, trying to determine the underlying cause - of an issue before removing the directory is a good - idea. - </note> - </para></listitem> - <listitem><para> - Understanding how a feature is used in practice within - existing recipes can be very helpful. - It is recommended that you configure some method that - allows you to quickly search through files.</para> - - <para>Using GNU Grep, you can use the following shell - function to recursively search through common - recipe-related files, skipping binary files, - <filename>.git</filename> directories, and the - Build Directory (assuming its name starts with - "build"): - <literallayout class='monospaced'> - g() { - grep -Ir \ - --exclude-dir=.git \ - --exclude-dir='build*' \ - --include='*.bb*' \ - --include='*.inc*' \ - --include='*.conf*' \ - --include='*.py*' \ - "$@" - } - </literallayout> - Following are some usage examples: - <literallayout class='monospaced'> - $ g FOO # Search recursively for "FOO" - $ g -i foo # Search recursively for "foo", ignoring case - $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR" - </literallayout> - If figuring out how some feature works requires a lot of - searching, it might indicate that the documentation should - be extended or improved. - In such cases, consider filing a documentation bug using - the Yocto Project implementation of - <ulink url='https://bugzilla.yoctoproject.org/'>Bugzilla</ulink>. - For general information on how to submit a bug against - the Yocto Project, see the Yocto Project Bugzilla - <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>wiki page</ulink>" - or the - <ulink url='&YOCTO_DOCS_DEV_URL;#submitting-a-defect-against-the-yocto-project'>Submitting a Defect Against the Yocto Project</ulink>" - section, which is in the Yocto Project Development Tasks - Manual. - <note> - The manuals might not be the right place to document - variables that are purely internal and have a limited - scope (e.g. internal variables used to implement a - single <filename>.bbclass</filename> file). - </note> - </para></listitem> - </itemizedlist> - </para> - </section> -</section> - -<section id='ref-quick-emulator-qemu'> - <title>Quick EMUlator (QEMU)</title> - - <para> - The Yocto Project uses an implementation of the Quick EMUlator (QEMU) - Open Source project as part of the Yocto Project development "tool - set". - </para> - - <para> - Within the context of the Yocto Project, QEMU is an - emulator and virtualization machine that allows you to run a complete - image you have built using the Yocto Project as just another task - on your build system. - QEMU is useful for running and testing images and applications on - supported Yocto Project architectures without having actual hardware. - Among other things, the Yocto Project uses QEMU to run automated - Quality Assurance (QA) tests on final images shipped with each - release. - <note> - This implementation is not the same as QEMU in general. - </note> - This section provides a brief reference for the Yocto Project - implementation of QEMU. - </para> - - <para> - For official information and documentation on QEMU in general, see the - following references: - <itemizedlist> - <listitem><para> - <emphasis><ulink url='http://wiki.qemu.org/Main_Page'>QEMU Website</ulink>:</emphasis> - The official website for the QEMU Open Source project. - </para></listitem> - <listitem><para> - <emphasis><ulink url='http://wiki.qemu.org/Manual'>Documentation</ulink>:</emphasis> - The QEMU user manual. - </para></listitem> - </itemizedlist> - </para> - - <para> - For information on how to use the Yocto Project implementation of - QEMU, see the - "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-manual-qemu'>Using the Quick EMUlator (QEMU)</ulink>" - chapter in the Yocto Project Development Tasks Manual. - </para> - - <section id='qemu-availability'> - <title>QEMU Availability</title> - - <para> - QEMU is made available with the Yocto Project a number of ways. - One method is to install a Software Development Kit (SDK). - For more information on how to make sure you have - QEMU available, see - "<ulink url='&YOCTO_DOCS_SDK_URL;#the-qemu-emulator'>The QEMU Emulator</ulink>" - section in the Yocto Project Application Development and the - Extensible Software Development Kit (eSDK) manual. - </para> - </section> - - <section id='qemu-performance'> - <title>QEMU Performance</title> - - <para> - Using QEMU to emulate your hardware can result in speed issues - depending on the target and host architecture mix. - For example, using the <filename>qemux86</filename> image in the - emulator on an Intel-based 32-bit (x86) host machine is fast - because the target and host architectures match. - On the other hand, using the <filename>qemuarm</filename> image - on the same Intel-based host can be slower. - But, you still achieve faithful emulation of ARM-specific issues. - </para> - - <para> - To speed things up, the QEMU images support using - <filename>distcc</filename> to call a cross-compiler outside the - emulated system. - If you used <filename>runqemu</filename> to start QEMU, and the - <filename>distccd</filename> application is present on the host - system, any BitBake cross-compiling toolchain available from the - build system is automatically used from within QEMU simply by - calling <filename>distcc</filename>. - You can accomplish this by defining the cross-compiler variable - (e.g. <filename>export CC="distcc"</filename>). - Alternatively, if you are using a suitable SDK image or the - appropriate stand-alone toolchain is present, the toolchain is - also automatically used. - </para> - - <note> - Several mechanisms exist that let you connect to the system - running on the QEMU emulator: - <itemizedlist> - <listitem><para> - QEMU provides a framebuffer interface that makes standard - consoles available. - </para></listitem> - <listitem><para> - Generally, headless embedded devices have a serial port. - If so, you can configure the operating system of the - running image to use that port to run a console. - The connection uses standard IP networking. - </para></listitem> - <listitem><para> - SSH servers exist in some QEMU images. - The <filename>core-image-sato</filename> QEMU image has a - Dropbear secure shell (SSH) server that runs with the root - password disabled. - The <filename>core-image-full-cmdline</filename> and - <filename>core-image-lsb</filename> QEMU images - have OpenSSH instead of Dropbear. - Including these SSH servers allow you to use standard - <filename>ssh</filename> and <filename>scp</filename> - commands. - The <filename>core-image-minimal</filename> QEMU image, - however, contains no SSH server. - </para></listitem> - <listitem><para> - You can use a provided, user-space NFS server to boot - the QEMU session using a local copy of the root - filesystem on the host. - In order to make this connection, you must extract a - root filesystem tarball by using the - <filename>runqemu-extract-sdk</filename> command. - After running the command, you must then point the - <filename>runqemu</filename> - script to the extracted directory instead of a root - filesystem image file. - See the - "<ulink url='&YOCTO_DOCS_DEV_URL;#qemu-running-under-a-network-file-system-nfs-server'>Running Under a Network File System (NFS) Server</ulink>" - section in the Yocto Project Development Tasks Manual for - more information. - </para></listitem> - </itemizedlist> - </note> - </section> - - <section id='qemu-command-line-syntax'> - <title>QEMU Command-Line Syntax</title> - - <para> - The basic <filename>runqemu</filename> command syntax is as - follows: - <literallayout class='monospaced'> - $ runqemu [<replaceable>option</replaceable> ] [...] - </literallayout> - Based on what you provide on the command line, - <filename>runqemu</filename> does a good job of figuring out what - you are trying to do. - For example, by default, QEMU looks for the most recently built - image according to the timestamp when it needs to look for an - image. - Minimally, through the use of options, you must provide either - a machine name, a virtual machine image - (<filename>*wic.vmdk</filename>), or a kernel image - (<filename>*.bin</filename>). - </para> - - <para> - Following is the command-line help output for the - <filename>runqemu</filename> command: - <literallayout class='monospaced'> - $ runqemu --help - - Usage: you can run this script with any valid combination - of the following environment variables (in any order): - KERNEL - the kernel image file to use - ROOTFS - the rootfs image file or nfsroot directory to use - MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified) - Simplified QEMU command-line options can be passed with: - nographic - disable video console - serial - enable a serial console on /dev/ttyS0 - slirp - enable user networking, no root privileges is required - kvm - enable KVM when running x86/x86_64 (VT-capable CPU required) - kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required) - publicvnc - enable a VNC server open to all hosts - audio - enable audio - [*/]ovmf* - OVMF firmware file or base name for booting with UEFI - tcpserial=<port> - specify tcp serial port number - biosdir=<dir> - specify custom bios dir - biosfilename=<filename> - specify bios filename - qemuparams=<xyz> - specify custom parameters to QEMU - bootparams=<xyz> - specify custom kernel parameters during boot - help, -h, --help: print this text - - Examples: - runqemu - runqemu qemuarm - runqemu tmp/deploy/images/qemuarm - runqemu tmp/deploy/images/qemux86/<qemuboot.conf> - runqemu qemux86-64 core-image-sato ext4 - runqemu qemux86-64 wic-image-minimal wic - runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial - runqemu qemux86 iso/hddimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz... - runqemu qemux86 qemuparams="-m 256" - runqemu qemux86 bootparams="psplash=false" - runqemu path/to/<image>-<machine>.wic - runqemu path/to/<image>-<machine>.wic.vmdk - </literallayout> - </para> - </section> - - <section id='runqemu-command-line-options'> - <title><filename>runqemu</filename> Command-Line Options</title> - - <para> - Following is a description of <filename>runqemu</filename> - options you can provide on the command line: - <note><title>Tip</title> - If you do provide some "illegal" option combination or perhaps - you do not provide enough in the way of options, - <filename>runqemu</filename> provides appropriate error - messaging to help you correct the problem. - </note> - <itemizedlist> - <listitem><para> - <replaceable>QEMUARCH</replaceable>: - The QEMU machine architecture, which must be "qemuarm", - "qemuarm64", "qemumips", "qemumips64", "qemuppc", - "qemux86", or "qemux86-64". - </para></listitem> - <listitem><para> - <filename><replaceable>VM</replaceable></filename>: - The virtual machine image, which must be a - <filename>.wic.vmdk</filename> file. - Use this option when you want to boot a - <filename>.wic.vmdk</filename> image. - The image filename you provide must contain one of the - following strings: "qemux86-64", "qemux86", "qemuarm", - "qemumips64", "qemumips", "qemuppc", or "qemush4". - </para></listitem> - <listitem><para> - <replaceable>ROOTFS</replaceable>: - A root filesystem that has one of the following - filetype extensions: "ext2", "ext3", "ext4", "jffs2", - "nfs", or "btrfs". - If the filename you provide for this option uses “nfs”, it - must provide an explicit root filesystem path. - </para></listitem> - <listitem><para> - <replaceable>KERNEL</replaceable>: - A kernel image, which is a <filename>.bin</filename> file. - When you provide a <filename>.bin</filename> file, - <filename>runqemu</filename> detects it and assumes the - file is a kernel image. - </para></listitem> - <listitem><para> - <replaceable>MACHINE</replaceable>: - The architecture of the QEMU machine, which must be one - of the following: "qemux86", "qemux86-64", "qemuarm", - "qemuarm64", "qemumips", “qemumips64", or "qemuppc". - The <replaceable>MACHINE</replaceable> and - <replaceable>QEMUARCH</replaceable> options are basically - identical. - If you do not provide a <replaceable>MACHINE</replaceable> - option, <filename>runqemu</filename> tries to determine - it based on other options. - </para></listitem> - <listitem><para> - <filename>ramfs</filename>: - Indicates you are booting an initial RAM disk (initramfs) - image, which means the <filename>FSTYPE</filename> is - <filename>cpio.gz</filename>. - </para></listitem> - <listitem><para> - <filename>iso</filename>: - Indicates you are booting an ISO image, which means the - <filename>FSTYPE</filename> is - <filename>.iso</filename>. - </para></listitem> - <listitem><para> - <filename>nographic</filename>: - Disables the video console, which sets the console to - "ttys0". - </para></listitem> - <listitem><para> - <filename>serial</filename>: - Enables a serial console on - <filename>/dev/ttyS0</filename>. - </para></listitem> - <listitem><para> - <filename>biosdir</filename>: - Establishes a custom directory for BIOS, VGA BIOS and - keymaps. - </para></listitem> - <listitem><para> - <filename>biosfilename</filename>: - Establishes a custom BIOS name. - </para></listitem> - <listitem><para> - <filename>qemuparams=\"<replaceable>xyz</replaceable>\"</filename>: - Specifies custom QEMU parameters. - Use this option to pass options other than the simple - "kvm" and "serial" options. - </para></listitem> - <listitem><para><filename>bootparams=\"<replaceable>xyz</replaceable>\"</filename>: - Specifies custom boot parameters for the kernel. - </para></listitem> - <listitem><para> - <filename>audio</filename>: - Enables audio in QEMU. - The <replaceable>MACHINE</replaceable> option must be - either "qemux86" or "qemux86-64" in order for audio to be - enabled. - Additionally, the <filename>snd_intel8x0</filename> - or <filename>snd_ens1370</filename> driver must be - installed in linux guest. - </para></listitem> - <listitem><para> - <filename>slirp</filename>: - Enables "slirp" networking, which is a different way - of networking that does not need root access - but also is not as easy to use or comprehensive - as the default. - </para></listitem> - <listitem><para id='kvm-cond'> - <filename>kvm</filename>: - Enables KVM when running "qemux86" or "qemux86-64" - QEMU architectures. - For KVM to work, all the following conditions must be met: - <itemizedlist> - <listitem><para> - Your <replaceable>MACHINE</replaceable> must be either -qemux86" or "qemux86-64". - </para></listitem> - <listitem><para> - Your build host has to have the KVM modules - installed, which are - <filename>/dev/kvm</filename>. - </para></listitem> - <listitem><para> - The build host <filename>/dev/kvm</filename> - directory has to be both writable and readable. - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <filename>kvm-vhost</filename>: - Enables KVM with VHOST support when running "qemux86" - or "qemux86-64" QEMU architectures. - For KVM with VHOST to work, the following conditions must - be met: - <itemizedlist> - <listitem><para> - <link linkend='kvm-cond'>kvm</link> option - conditions must be met. - </para></listitem> - <listitem><para> - Your build host has to have virtio net device, which - are <filename>/dev/vhost-net</filename>. - </para></listitem> - <listitem><para> - The build host <filename>/dev/vhost-net</filename> - directory has to be either readable or writable - and “slirp-enabled”. - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <filename>publicvnc</filename>: - Enables a VNC server open to all hosts. - </para></listitem> - </itemizedlist> - </para> - </section> -</section> - -<section id='maintaining-build-output-quality'> - <title>Maintaining Build Output Quality</title> - - <para> - Many factors can influence the quality of a build. - For example, if you upgrade a recipe to use a new version of an upstream software - package or you experiment with some new configuration options, subtle changes - can occur that you might not detect until later. - Consider the case where your recipe is using a newer version of an upstream package. - In this case, a new version of a piece of software might introduce an optional - dependency on another library, which is auto-detected. - If that library has already been built when the software is building, - the software will link to the built library and that library will be pulled - into your image along with the new software even if you did not want the - library. - </para> - - <para> - The - <link linkend='ref-classes-buildhistory'><filename>buildhistory</filename></link> - class exists to help you maintain - the quality of your build output. - You can use the class to highlight unexpected and possibly unwanted - changes in the build output. - When you enable build history, it records information about the contents of - each package and image and then commits that information to a local Git - repository where you can examine the information. - </para> - - <para> - The remainder of this section describes the following: - <itemizedlist> - <listitem><para>How you can enable and disable - build history</para></listitem> - <listitem><para>How to understand what the build history contains - </para></listitem> - <listitem><para>How to limit the information used for build history - </para></listitem> - <listitem><para>How to examine the build history from both a - command-line and web interface</para></listitem> - </itemizedlist> - </para> - - <section id='enabling-and-disabling-build-history'> - <title>Enabling and Disabling Build History</title> - - <para> - Build history is disabled by default. - To enable it, add the following <filename>INHERIT</filename> - statement and set the - <link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></link> - variable to "1" at the end of your - <filename>conf/local.conf</filename> file found in the - <link linkend='build-directory'>Build Directory</link>: - <literallayout class='monospaced'> - INHERIT += "buildhistory" - BUILDHISTORY_COMMIT = "1" - </literallayout> - Enabling build history as previously described causes the - OpenEmbedded build system to collect build output information and - commit it as a single commit to a local - <link linkend='git'>Git</link> repository. - <note> - Enabling build history increases your build times slightly, - particularly for images, and increases the amount of disk - space used during the build. - </note> - </para> - - <para> - You can disable build history by removing the previous statements - from your <filename>conf/local.conf</filename> file. - </para> - </section> - - <section id='understanding-what-the-build-history-contains'> - <title>Understanding What the Build History Contains</title> - - <para> - Build history information is kept in - <filename>${</filename><link linkend='var-TOPDIR'><filename>TOPDIR</filename></link><filename>}/buildhistory</filename> - in the Build Directory as defined by the - <link linkend='var-BUILDHISTORY_DIR'><filename>BUILDHISTORY_DIR</filename></link> - variable. - The following is an example abbreviated listing: - <imagedata fileref="figures/buildhistory.png" align="center" width="6in" depth="4in" /> - </para> - - <para> - At the top level, there is a <filename>metadata-revs</filename> file - that lists the revisions of the repositories for the layers enabled - when the build was produced. - The rest of the data splits into separate - <filename>packages</filename>, <filename>images</filename> and - <filename>sdk</filename> directories, the contents of which are - described below. - </para> - - <section id='build-history-package-information'> - <title>Build History Package Information</title> - - <para> - The history for each package contains a text file that has - name-value pairs with information about the package. - For example, <filename>buildhistory/packages/i586-poky-linux/busybox/busybox/latest</filename> - contains the following: - <literallayout class='monospaced'> - PV = 1.22.1 - PR = r32 - RPROVIDES = - RDEPENDS = glibc (>= 2.20) update-alternatives-opkg - RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d - PKGSIZE = 540168 - FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \ - /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \ - /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \ - /usr/share/pixmaps /usr/share/applications /usr/share/idl \ - /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers - FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \ - /etc/busybox.links.nosuid /etc/busybox.links.suid - </literallayout> - Most of these name-value pairs correspond to variables used - to produce the package. - The exceptions are <filename>FILELIST</filename>, which is the - actual list of files in the package, and - <filename>PKGSIZE</filename>, which is the total size of files - in the package in bytes. - </para> - - <para> - There is also a file corresponding to the recipe from which the - package came (e.g. - <filename>buildhistory/packages/i586-poky-linux/busybox/latest</filename>): - <literallayout class='monospaced'> - PV = 1.22.1 - PR = r32 - DEPENDS = initscripts kern-tools-native update-rc.d-native \ - virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \ - virtual/libc virtual/update-alternatives - PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \ - busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \ - busybox-staticdev busybox-dev busybox-doc busybox-locale busybox - </literallayout> - </para> - - <para> - Finally, for those recipes fetched from a version control - system (e.g., Git), a file exists that lists source revisions - that are specified in the recipe and lists the actual revisions - used during the build. - Listed and actual revisions might differ when - <link linkend='var-SRCREV'><filename>SRCREV</filename></link> - is set to - <filename>${<link linkend='var-AUTOREV'>AUTOREV</link>}</filename>. - Here is an example assuming - <filename>buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev</filename>): - <literallayout class='monospaced'> - # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" - SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" - # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" - SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" - </literallayout> - You can use the <filename>buildhistory-collect-srcrevs</filename> - command with the <filename>-a</filename> option to - collect the stored <filename>SRCREV</filename> values - from build history and report them in a format suitable for - use in global configuration (e.g., - <filename>local.conf</filename> or a distro include file) to - override floating <filename>AUTOREV</filename> values to a - fixed set of revisions. - Here is some example output from this command: - <literallayout class='monospaced'> - $ buildhistory-collect-srcrevs -a - # i586-poky-linux - SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072" - SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072" - SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a" - SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" - # x86_64-linux - SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa" - SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf" - SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11" - SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072" - SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3" - SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca" - SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a" - SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff" - SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" - # qemux86-poky-linux - SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" - SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f" - # all-poky-linux - SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11" - </literallayout> - <note> - Here are some notes on using the - <filename>buildhistory-collect-srcrevs</filename> command: - <itemizedlist> - <listitem><para>By default, only values where the - <filename>SRCREV</filename> was - not hardcoded (usually when <filename>AUTOREV</filename> - was used) are reported. - Use the <filename>-a</filename> option to see all - <filename>SRCREV</filename> values. - </para></listitem> - <listitem><para>The output statements might not have any effect - if overrides are applied elsewhere in the build system - configuration. - Use the <filename>-f</filename> option to add the - <filename>forcevariable</filename> override to each output line - if you need to work around this restriction. - </para></listitem> - <listitem><para>The script does apply special handling when - building for multiple machines. - However, the script does place a - comment before each set of values that specifies - which triplet to which they belong as shown above - (e.g., <filename>i586-poky-linux</filename>). - </para></listitem> - </itemizedlist> - </note> - </para> - </section> - - <section id='build-history-image-information'> - <title>Build History Image Information</title> - - <para> - The files produced for each image are as follows: - <itemizedlist> - <listitem><para><filename>image-files:</filename> - A directory containing selected files from the root - filesystem. - The files are defined by - <link linkend='var-BUILDHISTORY_IMAGE_FILES'><filename>BUILDHISTORY_IMAGE_FILES</filename></link>. - </para></listitem> - <listitem><para><filename>build-id.txt:</filename> - Human-readable information about the build configuration - and metadata source revisions. - This file contains the full build header as printed - by BitBake.</para></listitem> - <listitem><para><filename>*.dot:</filename> - Dependency graphs for the image that are - compatible with <filename>graphviz</filename>. - </para></listitem> - <listitem><para><filename>files-in-image.txt:</filename> - A list of files in the image with permissions, - owner, group, size, and symlink information. - </para></listitem> - <listitem><para><filename>image-info.txt:</filename> - A text file containing name-value pairs with information - about the image. - See the following listing example for more information. - </para></listitem> - <listitem><para><filename>installed-package-names.txt:</filename> - A list of installed packages by name only.</para></listitem> - <listitem><para><filename>installed-package-sizes.txt:</filename> - A list of installed packages ordered by size. - </para></listitem> - <listitem><para><filename>installed-packages.txt:</filename> - A list of installed packages with full package - filenames.</para></listitem> - </itemizedlist> - <note> - Installed package information is able to be gathered and - produced even if package management is disabled for the final - image. - </note> - </para> - - <para> - Here is an example of <filename>image-info.txt</filename>: - <literallayout class='monospaced'> - DISTRO = poky - DISTRO_VERSION = 1.7 - USER_CLASSES = buildstats image-mklibs image-prelink - IMAGE_CLASSES = image_types - IMAGE_FEATURES = debug-tweaks - IMAGE_LINGUAS = - IMAGE_INSTALL = packagegroup-core-boot run-postinsts - BAD_RECOMMENDATIONS = - NO_RECOMMENDATIONS = - PACKAGE_EXCLUDE = - ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \ - write_image_manifest ; buildhistory_list_installed_image ; \ - buildhistory_get_image_installed ; ssh_allow_empty_password; \ - postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ; - IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ; - IMAGESIZE = 6900 - </literallayout> - Other than <filename>IMAGESIZE</filename>, which is the - total size of the files in the image in Kbytes, the - name-value pairs are variables that may have influenced the - content of the image. - This information is often useful when you are trying to determine - why a change in the package or file listings has occurred. - </para> - </section> - - <section id='using-build-history-to-gather-image-information-only'> - <title>Using Build History to Gather Image Information Only</title> - - <para> - As you can see, build history produces image information, - including dependency graphs, so you can see why something - was pulled into the image. - If you are just interested in this information and not - interested in collecting specific package or SDK information, - you can enable writing only image information without - any history by adding the following to your - <filename>conf/local.conf</filename> file found in the - <link linkend='build-directory'>Build Directory</link>: - <literallayout class='monospaced'> - INHERIT += "buildhistory" - BUILDHISTORY_COMMIT = "0" - BUILDHISTORY_FEATURES = "image" - </literallayout> - Here, you set the - <link linkend='var-BUILDHISTORY_FEATURES'><filename>BUILDHISTORY_FEATURES</filename></link> - variable to use the image feature only. - </para> - </section> - - <section id='build-history-sdk-information'> - <title>Build History SDK Information</title> - - <para> - Build history collects similar information on the contents - of SDKs - (e.g. <filename>bitbake -c populate_sdk imagename</filename>) - as compared to information it collects for images. - Furthermore, this information differs depending on whether an - extensible or standard SDK is being produced. - </para> - - <para> - The following list shows the files produced for SDKs: - <itemizedlist> - <listitem><para><filename>files-in-sdk.txt:</filename> - A list of files in the SDK with permissions, - owner, group, size, and symlink information. - This list includes both the host and target parts - of the SDK. - </para></listitem> - <listitem><para><filename>sdk-info.txt:</filename> - A text file containing name-value pairs with information - about the SDK. - See the following listing example for more information. - </para></listitem> - <listitem><para><filename>sstate-task-sizes.txt:</filename> - A text file containing name-value pairs with information - about task group sizes - (e.g. <filename>do_populate_sysroot</filename> tasks - have a total size). - The <filename>sstate-task-sizes.txt</filename> file - exists only when an extensible SDK is created. - </para></listitem> - <listitem><para><filename>sstate-package-sizes.txt:</filename> - A text file containing name-value pairs with information - for the shared-state packages and sizes in the SDK. - The <filename>sstate-package-sizes.txt</filename> file - exists only when an extensible SDK is created. - </para></listitem> - <listitem><para><filename>sdk-files:</filename> - A folder that contains copies of the files mentioned in - <filename>BUILDHISTORY_SDK_FILES</filename> if the - files are present in the output. - Additionally, the default value of - <filename>BUILDHISTORY_SDK_FILES</filename> is specific - to the extensible SDK although you can set it - differently if you would like to pull in specific files - from the standard SDK.</para> - <para>The default files are - <filename>conf/local.conf</filename>, - <filename>conf/bblayers.conf</filename>, - <filename>conf/auto.conf</filename>, - <filename>conf/locked-sigs.inc</filename>, and - <filename>conf/devtool.conf</filename>. - Thus, for an extensible SDK, these files get copied - into the <filename>sdk-files</filename> directory. - </para></listitem> - <listitem><para>The following information appears under - each of the <filename>host</filename> - and <filename>target</filename> directories - for the portions of the SDK that run on the host and - on the target, respectively: - <note> - The following files for the most part are empty - when producing an extensible SDK because this - type of SDK is not constructed from packages as is - the standard SDK. - </note> - <itemizedlist> - <listitem><para><filename>depends.dot:</filename> - Dependency graph for the SDK that is - compatible with <filename>graphviz</filename>. - </para></listitem> - <listitem><para><filename>installed-package-names.txt:</filename> - A list of installed packages by name only. - </para></listitem> - <listitem><para><filename>installed-package-sizes.txt:</filename> - A list of installed packages ordered by size. - </para></listitem> - <listitem><para><filename>installed-packages.txt:</filename> - A list of installed packages with full package - filenames.</para></listitem> - </itemizedlist> - </para></listitem> - </itemizedlist> - </para> - - <para> - Here is an example of <filename>sdk-info.txt</filename>: - <literallayout class='monospaced'> - DISTRO = poky - DISTRO_VERSION = 1.3+snapshot-20130327 - SDK_NAME = poky-glibc-i686-arm - SDK_VERSION = 1.3+snapshot - SDKMACHINE = - SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs - BAD_RECOMMENDATIONS = - SDKSIZE = 352712 - </literallayout> - Other than <filename>SDKSIZE</filename>, which is the - total size of the files in the SDK in Kbytes, the - name-value pairs are variables that might have influenced the - content of the SDK. - This information is often useful when you are trying to - determine why a change in the package or file listings - has occurred. - </para> - </section> - - <section id='examining-build-history-information'> - <title>Examining Build History Information</title> - - <para> - You can examine build history output from the command line or - from a web interface. - </para> - - <para> - To see any changes that have occurred (assuming you have - <link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT = "1"</filename></link>), - you can simply - use any Git command that allows you to view the history of - a repository. - Here is one method: - <literallayout class='monospaced'> - $ git log -p - </literallayout> - You need to realize, however, that this method does show - changes that are not significant (e.g. a package's size - changing by a few bytes). - </para> - - <para> - A command-line tool called <filename>buildhistory-diff</filename> - does exist, though, that queries the Git repository and prints just - the differences that might be significant in human-readable form. - Here is an example: - <literallayout class='monospaced'> - $ ~/poky/poky/scripts/buildhistory-diff . HEAD^ - Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt): - /etc/anotherpkg.conf was added - /sbin/anotherpkg was added - * (installed-package-names.txt): - * anotherpkg was added - Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt): - anotherpkg was added - packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras" - * PR changed from "r0" to "r1" - * PV changed from "0.1.10" to "0.1.12" - packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%) - * PR changed from "r0" to "r1" - * PV changed from "0.1.10" to "0.1.12" - </literallayout> - <note> - The <filename>buildhistory-diff</filename> tool requires - the <filename>GitPython</filename> package. - Be sure to install it using Pip3 as follows: - <literallayout class='monospaced'> - $ pip3 install GitPython --user - </literallayout> - Alternatively, you can install - <filename>python3-git</filename> using the appropriate - distribution package manager (e.g. - <filename>apt-get</filename>, <filename>dnf</filename>, or - <filename>zipper</filename>). - </note> - </para> - - <para> - To see changes to the build history using a web interface, follow - the instruction in the <filename>README</filename> file here. - <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/'></ulink>. - </para> - - <para> - Here is a sample screenshot of the interface: - <imagedata fileref="figures/buildhistory-web.png" align="center" scalefit="1" width="130%" contentdepth="130%" /> - </para> - </section> - </section> -</section> - -<section id='speeding-up-the-build'> - <title>Speeding Up the Build</title> - - <para> - Build time can be an issue. - By default, the build system uses simple controls to try and maximize - build efficiency. - In general, the default settings for all the following variables - result in the most efficient build times when dealing with single - socket systems (i.e. a single CPU). - If you have multiple CPUs, you might try increasing the default - values to gain more speed. - See the descriptions in the glossary for each variable for more - information: - <itemizedlist> - <listitem><para> - <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename>:</link> - The maximum number of threads BitBake simultaneously executes. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename>:</ulink> - The number of threads BitBake uses during parsing. - </para></listitem> - <listitem><para> - <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename>:</link> - Extra options passed to the <filename>make</filename> command - during the - <link linkend='ref-tasks-compile'><filename>do_compile</filename></link> - task in order to specify parallel compilation on the - local build host. - </para></listitem> - <listitem><para> - <link linkend='var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename>:</link> - Extra options passed to the <filename>make</filename> command - during the - <link linkend='ref-tasks-install'><filename>do_install</filename></link> - task in order to specify parallel installation on the - local build host. - </para></listitem> - </itemizedlist> - As mentioned, these variables all scale to the number of processor - cores available on the build system. - For single socket systems, this auto-scaling ensures that the build - system fundamentally takes advantage of potential parallel operations - during the build based on the build machine's capabilities. - </para> - - <para> - Following are additional factors that can affect build speed: - <itemizedlist> - <listitem><para> - File system type: - The file system type that the build is being performed on can - also influence performance. - Using <filename>ext4</filename> is recommended as compared - to <filename>ext2</filename> and <filename>ext3</filename> - due to <filename>ext4</filename> improved features - such as extents. - </para></listitem> - <listitem><para> - Disabling the updating of access time using - <filename>noatime</filename>: - The <filename>noatime</filename> mount option prevents the - build system from updating file and directory access times. - </para></listitem> - <listitem><para> - Setting a longer commit: - Using the "commit=" mount option increases the interval - in seconds between disk cache writes. - Changing this interval from the five second default to - something longer increases the risk of data loss but decreases - the need to write to the disk, thus increasing the build - performance. - </para></listitem> - <listitem><para> - Choosing the packaging backend: - Of the available packaging backends, IPK is the fastest. - Additionally, selecting a singular packaging backend also - helps. - </para></listitem> - <listitem><para> - Using <filename>tmpfs</filename> for - <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> - as a temporary file system: - While this can help speed up the build, the benefits are - limited due to the compiler using - <filename>-pipe</filename>. - The build system goes to some lengths to avoid - <filename>sync()</filename> calls into the - file system on the principle that if there was a significant - failure, the - <link linkend='build-directory'>Build Directory</link> - contents could easily be rebuilt. - </para></listitem> - <listitem><para> - Inheriting the - <link linkend='ref-classes-rm-work'><filename>rm_work</filename></link> - class: - Inheriting this class has shown to speed up builds due to - significantly lower amounts of data stored in the data - cache as well as on disk. - Inheriting this class also makes cleanup of - <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> - faster, at the expense of being easily able to dive into the - source code. - File system maintainers have recommended that the fastest way - to clean up large numbers of files is to reformat partitions - rather than delete files due to the linear nature of partitions. - This, of course, assumes you structure the disk partitions and - file systems in a way that this is practical. - </para></listitem> - </itemizedlist> - Aside from the previous list, you should keep some trade offs in - mind that can help you speed up the build: - <itemizedlist> - <listitem><para> - Remove items from - <link linkend='var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></link> - that you might not need. - </para></listitem> - <listitem><para> - Exclude debug symbols and other debug information: - If you do not need these symbols and other debug information, - disabling the <filename>*-dbg</filename> package generation - can speed up the build. - You can disable this generation by setting the - <link linkend='var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></link> - variable to "1". - </para></listitem> - <listitem><para> - Disable static library generation for recipes derived from - <filename>autoconf</filename> or <filename>libtool</filename>: - Following is an example showing how to disable static - libraries and still provide an override to handle exceptions: - <literallayout class='monospaced'> - STATICLIBCONF = "--disable-static" - STATICLIBCONF_sqlite3-native = "" - EXTRA_OECONF += "${STATICLIBCONF}" - </literallayout> - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - Some recipes need static libraries in order to work - correctly (e.g. <filename>pseudo-native</filename> - needs <filename>sqlite3-native</filename>). - Overrides, as in the previous example, account for - these kinds of exceptions. - </para></listitem> - <listitem><para> - Some packages have packaging code that assumes the - presence of the static libraries. - If so, you might need to exclude them as well. - </para></listitem> - </itemizedlist> - </note> - </para></listitem> - </itemizedlist> - </para> -</section> -</chapter> -<!-- -vim: expandtab tw=80 ts=4 ---> |
