diff options
Diffstat (limited to 'import-layers/yocto-poky/documentation/dev-manual')
24 files changed, 16569 insertions, 0 deletions
diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-common-tasks.xml b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-common-tasks.xml new file mode 100644 index 000000000..f926f1d47 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-common-tasks.xml @@ -0,0 +1,10370 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<chapter id='extendpoky'> + +<title>Common Tasks</title> + <para> + This chapter describes fundamental procedures such as creating layers, + adding new software packages, extending or customizing images, + porting work to new hardware (adding a new machine), and so forth. + You will find that the procedures documented here occur often in the + development cycle using the Yocto Project. + </para> + + <section id="understanding-and-creating-layers"> + <title>Understanding and Creating Layers</title> + + <para> + The OpenEmbedded build system supports organizing + <link linkend='metadata'>Metadata</link> into multiple layers. + Layers allow you to isolate different types of customizations from + each other. + You might find it tempting to keep everything in one layer when + working on a single project. + However, the more modular your Metadata, the easier + it is to cope with future changes. + </para> + + <para> + To illustrate how layers are used to keep things modular, consider + machine customizations. + These types of customizations typically reside in a special layer, + rather than a general layer, called a Board Support Package (BSP) + Layer. + Furthermore, the machine customizations should be isolated from + recipes and Metadata that support a new GUI environment, + for example. + This situation gives you a couple of layers: one for the machine + configurations, and one for the GUI environment. + It is important to understand, however, that the BSP layer can + still make machine-specific additions to recipes within the GUI + environment layer without polluting the GUI layer itself + with those machine-specific changes. + You can accomplish this through a recipe that is a BitBake append + (<filename>.bbappend</filename>) file, which is described later + in this section. + </para> + + <para> + </para> + + <section id='yocto-project-layers'> + <title>Layers</title> + + <para> + The <link linkend='source-directory'>Source Directory</link> + contains both general layers and BSP + layers right out of the box. + You can easily identify layers that ship with a + Yocto Project release in the Source Directory by their + folder names. + Folders that represent layers typically have names that begin with + the string <filename>meta-</filename>. + <note> + It is not a requirement that a layer name begin with the + prefix <filename>meta-</filename>, but it is a commonly + accepted standard in the Yocto Project community. + </note> + For example, when you set up the Source Directory structure, + you will see several layers: + <filename>meta</filename>, + <filename>meta-skeleton</filename>, + <filename>meta-selftest</filename>, + <filename>meta-poky</filename>, and + <filename>meta-yocto-bsp</filename>. + Each of these folders represents a distinct layer. + </para> + + <para> + As another example, if you set up a local copy of the + <filename>meta-intel</filename> Git repository + and then explore the folder of that general layer, + you will discover many Intel-specific BSP layers inside. + For more information on BSP layers, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" + section in the Yocto Project Board Support Package (BSP) + Developer's Guide. + </para> + </section> + + <section id='creating-your-own-layer'> + <title>Creating Your Own Layer</title> + + <para> + It is very easy to create your own layers to use with the + OpenEmbedded build system. + The Yocto Project ships with scripts that speed up creating + general layers and BSP layers. + This section describes the steps you perform by hand to create + a layer so that you can better understand them. + For information about the layer-creation scripts, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>" + section in the Yocto Project Board Support Package (BSP) + Developer's Guide and the + "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>" + section further down in this manual. + </para> + + <para> + Follow these general steps to create your layer: + <orderedlist> + <listitem><para><emphasis>Check Existing Layers:</emphasis> + Before creating a new layer, you should be sure someone + has not already created a layer containing the Metadata + you need. + You can see the + <ulink url='http://layers.openembedded.org/layerindex/layers/'><filename>OpenEmbedded Metadata Index</filename></ulink> + for a list of layers from the OpenEmbedded community + that can be used in the Yocto Project. + </para></listitem> + <listitem><para><emphasis>Create a Directory:</emphasis> + Create the directory for your layer. + While not strictly required, prepend the name of the + folder with the string <filename>meta-</filename>. + For example: + <literallayout class='monospaced'> + meta-mylayer + meta-GUI_xyz + meta-mymachine + </literallayout> + </para></listitem> + <listitem><para><emphasis>Create a Layer Configuration + File:</emphasis> + Inside your new layer folder, you need to create a + <filename>conf/layer.conf</filename> file. + It is easiest to take an existing layer configuration + file and copy that to your layer's + <filename>conf</filename> directory and then modify the + file as needed.</para> + <para>The + <filename>meta-yocto-bsp/conf/layer.conf</filename> file + demonstrates the required syntax: + <literallayout class='monospaced'> + # We have a conf and classes directory, add to BBPATH + BBPATH .= ":${LAYERDIR}" + + # We have recipes-* directories, add to BBFILES + BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ + ${LAYERDIR}/recipes-*/*/*.bbappend" + + BBFILE_COLLECTIONS += "yoctobsp" + BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" + BBFILE_PRIORITY_yoctobsp = "5" + LAYERVERSION_yoctobsp = "3" + </literallayout></para> + <para>Here is an explanation of the example: + <itemizedlist> + <listitem><para>The configuration and + classes directory is appended to + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>. + <note> + All non-distro layers, which include all BSP + layers, are expected to append the layer + directory to the + <filename>BBPATH</filename>. + On the other hand, distro layers, such as + <filename>meta-poky</filename>, can choose + to enforce their own precedence over + <filename>BBPATH</filename>. + For an example of that syntax, see the + <filename>layer.conf</filename> file for + the <filename>meta-poky</filename> layer. + </note></para></listitem> + <listitem><para>The recipes for the layers are + appended to + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'>BBFILES</ulink></filename>. + </para></listitem> + <listitem><para>The + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS'>BBFILE_COLLECTIONS</ulink></filename> + variable is then appended with the layer name. + </para></listitem> + <listitem><para>The + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN'>BBFILE_PATTERN</ulink></filename> + variable is set to a regular expression and is + used to match files from + <filename>BBFILES</filename> into a particular + layer. + In this case, + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename> + is used to make <filename>BBFILE_PATTERN</filename> match within the + layer's path.</para></listitem> + <listitem><para>The + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'>BBFILE_PRIORITY</ulink></filename> + variable then assigns a priority to the layer. + Applying priorities is useful in situations + where the same recipe might appear in multiple + layers and allows you to choose the layer + that takes precedence.</para></listitem> + <listitem><para>The + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERVERSION'>LAYERVERSION</ulink></filename> + variable optionally specifies the version of a + layer as a single number.</para></listitem> + </itemizedlist></para> + <para>Note the use of the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename> + variable, which expands to the directory of the current + layer.</para> + <para>Through the use of the <filename>BBPATH</filename> + variable, BitBake locates class files + (<filename>.bbclass</filename>), + configuration files, and files that are included + with <filename>include</filename> and + <filename>require</filename> statements. + For these cases, BitBake uses the first file that + matches the name found in <filename>BBPATH</filename>. + This is similar to the way the <filename>PATH</filename> + variable is used for binaries. + It is recommended, therefore, that you use unique + class and configuration + filenames in your custom layer.</para></listitem> + <listitem><para><emphasis>Add Content:</emphasis> Depending + on the type of layer, add the content. + If the layer adds support for a machine, add the machine + configuration in a <filename>conf/machine/</filename> + file within the layer. + If the layer adds distro policy, add the distro + configuration in a <filename>conf/distro/</filename> + file within the layer. + If the layer introduces new recipes, put the recipes + you need in <filename>recipes-*</filename> + subdirectories within the layer. + <note>In order to be compliant with the Yocto Project, + a layer must contain a + <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout-readme'>README file.</ulink> + </note></para></listitem> + </orderedlist> + </para> + </section> + + <section id='best-practices-to-follow-when-creating-layers'> + <title>Best Practices to Follow When Creating Layers</title> + + <para> + To create layers that are easier to maintain and that will + not impact builds for other machines, you should consider the + information in the following sections. + </para> + + <section id='avoid-overlaying-entire-recipes'> + <title>Avoid "Overlaying" Entire Recipes</title> + + <para> + Avoid "overlaying" entire recipes from other layers in your + configuration. + In other words, do not copy an entire recipe into your + layer and then modify it. + Rather, use an append file (<filename>.bbappend</filename>) + to override + only those parts of the original recipe you need to modify. + </para> + </section> + + <section id='avoid-duplicating-include-files'> + <title>Avoid Duplicating Include Files</title> + + <para> + Avoid duplicating include files. + Use append files (<filename>.bbappend</filename>) + for each recipe + that uses an include file. + Or, if you are introducing a new recipe that requires + the included file, use the path relative to the original + layer directory to refer to the file. + For example, use + <filename>require recipes-core/</filename><replaceable>package</replaceable><filename>/</filename><replaceable>file</replaceable><filename>.inc</filename> + instead of <filename>require </filename><replaceable>file</replaceable><filename>.inc</filename>. + If you're finding you have to overlay the include file, + it could indicate a deficiency in the include file in + the layer to which it originally belongs. + If this is the case, you should try to address that + deficiency instead of overlaying the include file. + For example, you could address this by getting the + maintainer of the include file to add a variable or + variables to make it easy to override the parts needing + to be overridden. + </para> + </section> + + <section id='structure-your-layers'> + <title>Structure Your Layers</title> + + <para> + Proper use of overrides within append files and placement + of machine-specific files within your layer can ensure that + a build is not using the wrong Metadata and negatively + impacting a build for a different machine. + Following are some examples: + <itemizedlist> + <listitem><para><emphasis>Modifying Variables to Support + a Different Machine:</emphasis> + Suppose you have a layer named + <filename>meta-one</filename> that adds support + for building machine "one". + To do so, you use an append file named + <filename>base-files.bbappend</filename> and + create a dependency on "foo" by altering the + <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> + variable: + <literallayout class='monospaced'> + DEPENDS = "foo" + </literallayout> + The dependency is created during any build that + includes the layer + <filename>meta-one</filename>. + However, you might not want this dependency + for all machines. + For example, suppose you are building for + machine "two" but your + <filename>bblayers.conf</filename> file has the + <filename>meta-one</filename> layer included. + During the build, the + <filename>base-files</filename> for machine + "two" will also have the dependency on + <filename>foo</filename>.</para> + <para>To make sure your changes apply only when + building machine "one", use a machine override + with the <filename>DEPENDS</filename> statement: + <literallayout class='monospaced'> + DEPENDS_one = "foo" + </literallayout> + You should follow the same strategy when using + <filename>_append</filename> and + <filename>_prepend</filename> operations: + <literallayout class='monospaced'> + DEPENDS_append_one = " foo" + DEPENDS_prepend_one = "foo " + </literallayout> + As an actual example, here's a line from the recipe + for gnutls, which adds dependencies on + "argp-standalone" when building with the musl C + library: + <literallayout class='monospaced'> + DEPENDS_append_libc-musl = " argp-standalone" + </literallayout> + <note> + Avoiding "+=" and "=+" and using + machine-specific + <filename>_append</filename> + and <filename>_prepend</filename> operations + is recommended as well. + </note></para></listitem> + <listitem><para><emphasis>Place Machine-Specific Files + in Machine-Specific Locations:</emphasis> + When you have a base recipe, such as + <filename>base-files.bb</filename>, that + contains a + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + statement to a file, you can use an append file + to cause the build to use your own version of + the file. + For example, an append file in your layer at + <filename>meta-one/recipes-core/base-files/base-files.bbappend</filename> + could extend + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink> + using + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink> + as follows: + <literallayout class='monospaced'> + FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:" + </literallayout> + The build for machine "one" will pick up your + machine-specific file as long as you have the + file in + <filename>meta-one/recipes-core/base-files/base-files/</filename>. + However, if you are building for a different + machine and the + <filename>bblayers.conf</filename> file includes + the <filename>meta-one</filename> layer and + the location of your machine-specific file is + the first location where that file is found + according to <filename>FILESPATH</filename>, + builds for all machines will also use that + machine-specific file.</para> + <para>You can make sure that a machine-specific + file is used for a particular machine by putting + the file in a subdirectory specific to the + machine. + For example, rather than placing the file in + <filename>meta-one/recipes-core/base-files/base-files/</filename> + as shown above, put it in + <filename>meta-one/recipes-core/base-files/base-files/one/</filename>. + Not only does this make sure the file is used + only when building for machine "one", but the + build process locates the file more quickly.</para> + <para>In summary, you need to place all files + referenced from <filename>SRC_URI</filename> + in a machine-specific subdirectory within the + layer in order to restrict those files to + machine-specific builds.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id='other-recommendations'> + <title>Other Recommendations</title> + + <para> + We also recommend the following: + <itemizedlist> + <listitem><para>Store custom layers in a Git repository + that uses the + <filename>meta-<replaceable>layer_name</replaceable></filename> format. + </para></listitem> + <listitem><para>Clone the repository alongside other + <filename>meta</filename> directories in the + <link linkend='source-directory'>Source Directory</link>. + </para></listitem> + </itemizedlist> + Following these recommendations keeps your Source Directory and + its configuration entirely inside the Yocto Project's core + base. + </para> + </section> + </section> + + <section id='enabling-your-layer'> + <title>Enabling Your Layer</title> + + <para> + Before the OpenEmbedded build system can use your new layer, + you need to enable it. + To enable your layer, simply add your layer's path to the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'>BBLAYERS</ulink></filename> + variable in your <filename>conf/bblayers.conf</filename> file, + which is found in the + <link linkend='build-directory'>Build Directory</link>. + The following example shows how to enable a layer named + <filename>meta-mylayer</filename>: + <literallayout class='monospaced'> + LCONF_VERSION = "6" + + BBPATH = "${TOPDIR}" + BBFILES ?= "" + + BBLAYERS ?= " \ + $HOME/poky/meta \ + $HOME/poky/meta-poky \ + $HOME/poky/meta-yocto-bsp \ + $HOME/poky/meta-mylayer \ + " + </literallayout> + </para> + + <para> + BitBake parses each <filename>conf/layer.conf</filename> file + as specified in the <filename>BBLAYERS</filename> variable + within the <filename>conf/bblayers.conf</filename> file. + During the processing of each + <filename>conf/layer.conf</filename> file, BitBake adds the + recipes, classes and configurations contained within the + particular layer to the source directory. + </para> + </section> + + <section id='using-bbappend-files'> + <title>Using .bbappend Files</title> + + <para> + Recipes used to append Metadata to other recipes are called + BitBake append files. + BitBake append files use the <filename>.bbappend</filename> file + type suffix, while the corresponding recipes to which Metadata + is being appended use the <filename>.bb</filename> file type + suffix. + </para> + + <para> + A <filename>.bbappend</filename> file allows your layer to make + additions or changes to the content of another layer's recipe + without having to copy the other recipe into your layer. + Your <filename>.bbappend</filename> file resides in your layer, + while the main <filename>.bb</filename> recipe file to + which you are appending Metadata resides in a different layer. + </para> + + <para> + Append files must have the same root names as their corresponding + recipes. + For example, the append file + <filename>someapp_&DISTRO;.bbappend</filename> must apply to + <filename>someapp_&DISTRO;.bb</filename>. + This means the original recipe and append file names are version + number-specific. + If the corresponding recipe is renamed to update to a newer + version, the corresponding <filename>.bbappend</filename> file must + be renamed (and possibly updated) as well. + During the build process, BitBake displays an error on starting + if it detects a <filename>.bbappend</filename> file that does + not have a corresponding recipe with a matching name. + See the + <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></ulink> + variable for information on how to handle this error. + </para> + + <para> + Being able to append information to an existing recipe not only + avoids duplication, but also automatically applies recipe + changes in a different layer to your layer. + If you were copying recipes, you would have to manually merge + changes as they occur. + </para> + + <para> + As an example, consider the main formfactor recipe and a + corresponding formfactor append file both from the + <link linkend='source-directory'>Source Directory</link>. + Here is the main formfactor recipe, which is named + <filename>formfactor_0.0.bb</filename> and located in the + "meta" layer at + <filename>meta/recipes-bsp/formfactor</filename>: + <literallayout class='monospaced'> + SUMMARY = "Device formfactor information" + SECTION = "base" + LICENSE = "MIT" + LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=4d92cd373abda3937c2bc47fbc49d690 \ + file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" + PR = "r45" + + SRC_URI = "file://config file://machconfig" + S = "${WORKDIR}" + + PACKAGE_ARCH = "${MACHINE_ARCH}" + INHIBIT_DEFAULT_DEPS = "1" + + do_install() { + # Install file only if it has contents + install -d ${D}${sysconfdir}/formfactor/ + install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/ + if [ -s "${S}/machconfig" ]; then + install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ + fi + } + </literallayout> + In the main recipe, note the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + variable, which tells the OpenEmbedded build system where to + find files during the build. + </para> + + <para> + Following is the append file, which is named + <filename>formfactor_0.0.bbappend</filename> and is from the + Raspberry Pi BSP Layer named + <filename>meta-raspberrypi</filename>. + The file is in <filename>recipes-bsp/formfactor</filename>: + <literallayout class='monospaced'> + FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" + </literallayout> + </para> + + <para> + By default, the build system uses the + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink> + variable to locate files. + This append file extends the locations by setting the + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink> + variable. + Setting this variable in the <filename>.bbappend</filename> + file is the most reliable and recommended method for adding + directories to the search path used by the build system + to find files. + </para> + + <para> + The statement in this example extends the directories to include + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-THISDIR'><filename>THISDIR</filename></ulink><filename>}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>, + which resolves to a directory named + <filename>formfactor</filename> in the same directory + in which the append file resides (i.e. + <filename>meta-raspberrypi/recipes-bsp/formfactor/formfactor</filename>. + This implies that you must have the supporting directory + structure set up that will contain any files or patches you + will be including from the layer. + </para> + + <para> + Using the immediate expansion assignment operator + <filename>:=</filename> is important because of the reference to + <filename>THISDIR</filename>. + The trailing colon character is important as it ensures that + items in the list remain colon-separated. + <note> + <para> + BitBake automatically defines the + <filename>THISDIR</filename> variable. + You should never set this variable yourself. + Using "_prepend" as part of the + <filename>FILESEXTRAPATHS</filename> ensures your path + will be searched prior to other paths in the final + list. + </para> + + <para> + Also, not all append files add extra files. + Many append files simply exist to add build options + (e.g. <filename>systemd</filename>). + For these cases, your append file would not even + use the <filename>FILESEXTRAPATHS</filename> statement. + </para> + </note> + </para> + </section> + + <section id='prioritizing-your-layer'> + <title>Prioritizing Your Layer</title> + + <para> + Each layer is assigned a priority value. + Priority values control which layer takes precedence if there + are recipe files with the same name in multiple layers. + For these cases, the recipe file from the layer with a higher + priority number takes precedence. + Priority values also affect the order in which multiple + <filename>.bbappend</filename> files for the same recipe are + applied. + You can either specify the priority manually, or allow the + build system to calculate it based on the layer's dependencies. + </para> + + <para> + To specify the layer's priority manually, use the + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink> + variable. + For example: + <literallayout class='monospaced'> + BBFILE_PRIORITY_mylayer = "1" + </literallayout> + </para> + + <note> + <para>It is possible for a recipe with a lower version number + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> + in a layer that has a higher priority to take precedence.</para> + <para>Also, the layer priority does not currently affect the + precedence order of <filename>.conf</filename> + or <filename>.bbclass</filename> files. + Future versions of BitBake might address this.</para> + </note> + </section> + + <section id='managing-layers'> + <title>Managing Layers</title> + + <para> + You can use the BitBake layer management tool to provide a view + into the structure of recipes across a multi-layer project. + Being able to generate output that reports on configured layers + with their paths and priorities and on + <filename>.bbappend</filename> files and their applicable + recipes can help to reveal potential problems. + </para> + + <para> + Use the following form when running the layer management tool. + <literallayout class='monospaced'> + $ bitbake-layers <replaceable>command</replaceable> [<replaceable>arguments</replaceable>] + </literallayout> + The following list describes the available commands: + <itemizedlist> + <listitem><para><filename><emphasis>help:</emphasis></filename> + Displays general help or help on a specified command. + </para></listitem> + <listitem><para><filename><emphasis>show-layers:</emphasis></filename> + Shows the current configured layers. + </para></listitem> + <listitem><para><filename><emphasis>show-recipes:</emphasis></filename> + Lists available recipes and the layers that provide them. + </para></listitem> + <listitem><para><filename><emphasis>show-overlayed:</emphasis></filename> + Lists overlayed recipes. + A recipe is overlayed when a recipe with the same name + exists in another layer that has a higher layer + priority. + </para></listitem> + <listitem><para><filename><emphasis>show-appends:</emphasis></filename> + Lists <filename>.bbappend</filename> files and the + recipe files to which they apply. + </para></listitem> + <listitem><para><filename><emphasis>show-cross-depends:</emphasis></filename> + Lists dependency relationships between recipes that + cross layer boundaries. + </para></listitem> + <listitem><para><filename><emphasis>add-layer:</emphasis></filename> + Adds a layer to <filename>bblayers.conf</filename>. + </para></listitem> + <listitem><para><filename><emphasis>remove-layer:</emphasis></filename> + Removes a layer from <filename>bblayers.conf</filename> + </para></listitem> + <listitem><para><filename><emphasis>flatten:</emphasis></filename> + Flattens the layer configuration into a separate output + directory. + Flattening your layer configuration builds a "flattened" + directory that contains the contents of all layers, + with any overlayed recipes removed and any + <filename>.bbappend</filename> files appended to the + corresponding recipes. + You might have to perform some manual cleanup of the + flattened layer as follows: + <itemizedlist> + <listitem><para>Non-recipe files (such as patches) + are overwritten. + The flatten command shows a warning for these + files. + </para></listitem> + <listitem><para>Anything beyond the normal layer + setup has been added to the + <filename>layer.conf</filename> file. + Only the lowest priority layer's + <filename>layer.conf</filename> is used. + </para></listitem> + <listitem><para>Overridden and appended items from + <filename>.bbappend</filename> files need to be + cleaned up. + The contents of each + <filename>.bbappend</filename> end up in the + flattened recipe. + However, if there are appended or changed + variable values, you need to tidy these up + yourself. + Consider the following example. + Here, the <filename>bitbake-layers</filename> + command adds the line + <filename>#### bbappended ...</filename> so that + you know where the following lines originate: + <literallayout class='monospaced'> + ... + DESCRIPTION = "A useful utility" + ... + EXTRA_OECONF = "--enable-something" + ... + + #### bbappended from meta-anotherlayer #### + + DESCRIPTION = "Customized utility" + EXTRA_OECONF += "--enable-somethingelse" + </literallayout> + Ideally, you would tidy up these utilities as + follows: + <literallayout class='monospaced'> + ... + DESCRIPTION = "Customized utility" + ... + EXTRA_OECONF = "--enable-something --enable-somethingelse" + ... + </literallayout></para></listitem> + </itemizedlist></para></listitem> + </itemizedlist> + </para> + </section> + + <section id='creating-a-general-layer-using-the-yocto-layer-script'> + <title>Creating a General Layer Using the yocto-layer Script</title> + + <para> + The <filename>yocto-layer</filename> script simplifies + creating a new general layer. + <note> + For information on BSP layers, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" + section in the Yocto Project Board Specific (BSP) + Developer's Guide. + </note> + The default mode of the script's operation is to prompt you for + information needed to generate the layer: + <itemizedlist> + <listitem><para>The layer priority. + </para></listitem> + <listitem><para>Whether or not to create a sample recipe. + </para></listitem> + <listitem><para>Whether or not to create a sample + append file. + </para></listitem> + </itemizedlist> + </para> + + <para> + Use the <filename>yocto-layer create</filename> sub-command + to create a new general layer. + In its simplest form, you can create a layer as follows: + <literallayout class='monospaced'> + $ yocto-layer create mylayer + </literallayout> + The previous example creates a layer named + <filename>meta-mylayer</filename> in the current directory. + </para> + + <para> + As the <filename>yocto-layer create</filename> command runs, + default values for the prompts appear in brackets. + Pressing enter without supplying anything for the prompts + or pressing enter and providing an invalid response causes the + script to accept the default value. + Once the script completes, the new layer + is created in the current working directory. + The script names the layer by prepending + <filename>meta-</filename> to the name you provide. + </para> + + <para> + Minimally, the script creates the following within the layer: + <itemizedlist> + <listitem><para><emphasis>The <filename>conf</filename> + directory:</emphasis> + This directory contains the layer's configuration file. + The root name for the file is the same as the root name + your provided for the layer (e.g. + <filename><replaceable>layer</replaceable>.conf</filename>). + </para></listitem> + <listitem><para><emphasis>The + <filename>COPYING.MIT</filename> file:</emphasis> + The copyright and use notice for the software. + </para></listitem> + <listitem><para><emphasis>The <filename>README</filename> + file:</emphasis> + A file describing the contents of your new layer. + </para></listitem> + </itemizedlist> + </para> + + <para> + If you choose to generate a sample recipe file, the script + prompts you for the name for the recipe and then creates it + in <filename><replaceable>layer</replaceable>/recipes-example/example/</filename>. + The script creates a <filename>.bb</filename> file and a + directory, which contains a sample + <filename>helloworld.c</filename> source file, along with + a sample patch file. + If you do not provide a recipe name, the script uses + "example". + </para> + + <para> + If you choose to generate a sample append file, the script + prompts you for the name for the file and then creates it + in <filename><replaceable>layer</replaceable>/recipes-example-bbappend/example-bbappend/</filename>. + The script creates a <filename>.bbappend</filename> file and a + directory, which contains a sample patch file. + If you do not provide a recipe name, the script uses + "example". + The script also prompts you for the version of the append file. + The version should match the recipe to which the append file + is associated. + </para> + + <para> + The easiest way to see how the <filename>yocto-layer</filename> + script works is to experiment with the script. + You can also read the usage information by entering the + following: + <literallayout class='monospaced'> + $ yocto-layer help + </literallayout> + </para> + + <para> + Once you create your general layer, you must add it to your + <filename>bblayers.conf</filename> file. + Here is an example where a layer named + <filename>meta-mylayer</filename> is added: + <literallayout class='monospaced'> + BBLAYERS = ?" \ + /usr/local/src/yocto/meta \ + /usr/local/src/yocto/meta-poky \ + /usr/local/src/yocto/meta-yocto-bsp \ + /usr/local/src/yocto/meta-mylayer \ + " + </literallayout> + Adding the layer to this file enables the build system to + locate the layer during the build. + </para> + </section> + </section> + + <section id='usingpoky-extend-customimage'> + <title>Customizing Images</title> + + <para> + You can customize images to satisfy particular requirements. + This section describes several methods and provides guidelines for each. + </para> + + <section id='usingpoky-extend-customimage-localconf'> + <title>Customizing Images Using <filename>local.conf</filename></title> + + <para> + Probably the easiest way to customize an image is to add a + package by way of the <filename>local.conf</filename> + configuration file. + Because it is limited to local use, this method generally only + allows you to add packages and is not as flexible as creating + your own customized image. + When you add packages using local variables this way, you need + to realize that these variable changes are in effect for every + build and consequently affect all images, which might not + be what you require. + </para> + + <para> + To add a package to your image using the local configuration + file, use the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename> + variable with the <filename>_append</filename> operator: + <literallayout class='monospaced'> + IMAGE_INSTALL_append = " strace" + </literallayout> + Use of the syntax is important - specifically, the space between + the quote and the package name, which is + <filename>strace</filename> in this example. + This space is required since the <filename>_append</filename> + operator does not add the space. + </para> + + <para> + Furthermore, you must use <filename>_append</filename> instead + of the <filename>+=</filename> operator if you want to avoid + ordering issues. + The reason for this is because doing so unconditionally appends + to the variable and avoids ordering problems due to the + variable being set in image recipes and + <filename>.bbclass</filename> files with operators like + <filename>?=</filename>. + Using <filename>_append</filename> ensures the operation takes + affect. + </para> + + <para> + As shown in its simplest use, + <filename>IMAGE_INSTALL_append</filename> affects all images. + It is possible to extend the syntax so that the variable + applies to a specific image only. + Here is an example: + <literallayout class='monospaced'> + IMAGE_INSTALL_append_pn-core-image-minimal = " strace" + </literallayout> + This example adds <filename>strace</filename> to the + <filename>core-image-minimal</filename> image only. + </para> + + <para> + You can add packages using a similar approach through the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'>CORE_IMAGE_EXTRA_INSTALL</ulink></filename> + variable. + If you use this variable, only + <filename>core-image-*</filename> images are affected. + </para> + </section> + + <section id='usingpoky-extend-customimage-imagefeatures'> + <title>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and + <filename>EXTRA_IMAGE_FEATURES</filename></title> + + <para> + Another method for customizing your image is to enable or + disable high-level image features by using the + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> + and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> + variables. + Although the functions for both variables are nearly equivalent, + best practices dictate using <filename>IMAGE_FEATURES</filename> + from within a recipe and using + <filename>EXTRA_IMAGE_FEATURES</filename> from within + your <filename>local.conf</filename> file, which is found in the + <link linkend='build-directory'>Build Directory</link>. + </para> + + <para> + To understand how these features work, the best reference is + <filename>meta/classes/core-image.bbclass</filename>. + This class lists out the available + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> + of which most map to package groups while some, such as + <filename>debug-tweaks</filename> and + <filename>read-only-rootfs</filename>, resolve as general + configuration settings. + </para> + + <para> + In summary, the file looks at the contents of the + <filename>IMAGE_FEATURES</filename> variable and then maps + or configures the feature accordingly. + Based on this information, the build system automatically + adds the appropriate packages or configurations to the + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink> + variable. + Effectively, you are enabling extra features by extending the + class or creating a custom class for use with specialized image + <filename>.bb</filename> files. + </para> + + <para> + Use the <filename>EXTRA_IMAGE_FEATURES</filename> variable + from within your local configuration file. + Using a separate area from which to enable features with + this variable helps you avoid overwriting the features in the + image recipe that are enabled with + <filename>IMAGE_FEATURES</filename>. + The value of <filename>EXTRA_IMAGE_FEATURES</filename> is added + to <filename>IMAGE_FEATURES</filename> within + <filename>meta/conf/bitbake.conf</filename>. + </para> + + <para> + To illustrate how you can use these variables to modify your + image, consider an example that selects the SSH server. + The Yocto Project ships with two SSH servers you can use + with your images: Dropbear and OpenSSH. + Dropbear is a minimal SSH server appropriate for + resource-constrained environments, while OpenSSH is a + well-known standard SSH server implementation. + By default, the <filename>core-image-sato</filename> image + is configured to use Dropbear. + The <filename>core-image-full-cmdline</filename> and + <filename>core-image-lsb</filename> images both + include OpenSSH. + The <filename>core-image-minimal</filename> image does not + contain an SSH server. + </para> + + <para> + You can customize your image and change these defaults. + Edit the <filename>IMAGE_FEATURES</filename> variable + in your recipe or use the + <filename>EXTRA_IMAGE_FEATURES</filename> in your + <filename>local.conf</filename> file so that it configures the + image you are working with to include + <filename>ssh-server-dropbear</filename> or + <filename>ssh-server-openssh</filename>. + </para> + + <note> + See the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" + section in the Yocto Project Reference Manual for a complete + list of image features that ship with the Yocto Project. + </note> + </section> + + <section id='usingpoky-extend-customimage-custombb'> + <title>Customizing Images Using Custom .bb Files</title> + + <para> + You can also customize an image by creating a custom recipe + that defines additional software as part of the image. + The following example shows the form for the two lines you need: + <literallayout class='monospaced'> + IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2" + + inherit core-image + </literallayout> + </para> + + <para> + Defining the software using a custom recipe gives you total + control over the contents of the image. + It is important to use the correct names of packages in the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename> + variable. + You must use the OpenEmbedded notation and not the Debian notation for the names + (e.g. <filename>glibc-dev</filename> instead of <filename>libc6-dev</filename>). + </para> + + <para> + The other method for creating a custom image is to base it on an existing image. + For example, if you want to create an image based on <filename>core-image-sato</filename> + but add the additional package <filename>strace</filename> to the image, + copy the <filename>meta/recipes-sato/images/core-image-sato.bb</filename> to a + new <filename>.bb</filename> and add the following line to the end of the copy: + <literallayout class='monospaced'> + IMAGE_INSTALL += "strace" + </literallayout> + </para> + </section> + + <section id='usingpoky-extend-customimage-customtasks'> + <title>Customizing Images Using Custom Package Groups</title> + + <para> + For complex custom images, the best approach for customizing + an image is to create a custom package group recipe that is + used to build the image or images. + A good example of a package group recipe is + <filename>meta/recipes-core/packagegroups/packagegroup-base.bb</filename>. + </para> + + <para> + If you examine that recipe, you see that the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> + variable lists the package group packages to produce. + The <filename>inherit packagegroup</filename> statement + sets appropriate default values and automatically adds + <filename>-dev</filename>, <filename>-dbg</filename>, and + <filename>-ptest</filename> complementary packages for each + package specified in the <filename>PACKAGES</filename> + statement. + <note> + The <filename>inherit packages</filename> should be + located near the top of the recipe, certainly before + the <filename>PACKAGES</filename> statement. + </note> + </para> + + <para> + For each package you specify in <filename>PACKAGES</filename>, + you can use + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'>RDEPENDS</ulink></filename> + and + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'>RRECOMMENDS</ulink></filename> + entries to provide a list of packages the parent task package + should contain. + You can see examples of these further down in the + <filename>packagegroup-base.bb</filename> recipe. + </para> + + <para> + Here is a short, fabricated example showing the same basic + pieces: + <literallayout class='monospaced'> + DESCRIPTION = "My Custom Package Groups" + + inherit packagegroup + + PACKAGES = "\ + packagegroup-custom-apps \ + packagegroup-custom-tools \ + " + + RDEPENDS_packagegroup-custom-apps = "\ + dropbear \ + portmap \ + psplash" + + RDEPENDS_packagegroup-custom-tools = "\ + oprofile \ + oprofileui-server \ + lttng-tools" + + RRECOMMENDS_packagegroup-custom-tools = "\ + kernel-module-oprofile" + </literallayout> + </para> + + <para> + In the previous example, two package group packages are created with their dependencies and their + recommended package dependencies listed: <filename>packagegroup-custom-apps</filename>, and + <filename>packagegroup-custom-tools</filename>. + To build an image using these package group packages, you need to add + <filename>packagegroup-custom-apps</filename> and/or + <filename>packagegroup-custom-tools</filename> to + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>. + For other forms of image dependencies see the other areas of this section. + </para> + </section> + + <section id='usingpoky-extend-customimage-image-name'> + <title>Customizing an Image Hostname</title> + + <para> + By default, the configured hostname (i.e. + <filename>/etc/hostname</filename>) in an image is the + same as the machine name. + For example, if + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + equals "qemux86", the configured hostname written to + <filename>/etc/hostname</filename> is "qemux86". + </para> + + <para> + You can customize this name by altering the value of the + "hostname" variable in the + <filename>base-files</filename> recipe using either + an append file or a configuration file. + Use the following in an append file: + <literallayout class='monospaced'> + hostname="myhostname" + </literallayout> + Use the following in a configuration file: + <literallayout class='monospaced'> + hostname_pn-base-files = "myhostname" + </literallayout> + </para> + + <para> + Changing the default value of the variable "hostname" can be + useful in certain situations. + For example, suppose you need to do extensive testing on an + image and you would like to easily identify the image + under test from existing images with typical default + hostnames. + In this situation, you could change the default hostname to + "testme", which results in all the images using the name + "testme". + Once testing is complete and you do not need to rebuild the + image for test any longer, you can easily reset the default + hostname. + </para> + + <para> + Another point of interest is that if you unset the variable, + the image will have no default hostname in the filesystem. + Here is an example that unsets the variable in a + configuration file: + <literallayout class='monospaced'> + hostname_pn-base-files = "" + </literallayout> + Having no default hostname in the filesystem is suitable for + environments that use dynamic hostnames such as virtual + machines. + </para> + </section> + </section> + + <section id='new-recipe-writing-a-new-recipe'> + <title>Writing a New Recipe</title> + + <para> + Recipes (<filename>.bb</filename> files) are fundamental components + in the Yocto Project environment. + Each software component built by the OpenEmbedded build system + requires a recipe to define the component. + This section describes how to create, write, and test a new + recipe. + <note> + For information on variables that are useful for recipes and + for information about recipe naming issues, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-varlocality-recipe-required'>Required</ulink>" + section of the Yocto Project Reference Manual. + </note> + </para> + + <section id='new-recipe-overview'> + <title>Overview</title> + + <para> + The following figure shows the basic process for creating a + new recipe. + The remainder of the section provides details for the steps. + <imagedata fileref="figures/recipe-workflow.png" width="6in" depth="7in" align="center" scalefit="1" /> + </para> + </section> + + <section id='new-recipe-locate-or-automatically-create-a-base-recipe'> + <title>Locate or Automatically Create a Base Recipe</title> + + <para> + You can always write a recipe from scratch. + However, two choices exist that can help you quickly get a + start on a new recipe: + <itemizedlist> + <listitem><para><emphasis><filename>recipetool</filename>:</emphasis> + A tool provided by the Yocto Project that automates + creation of a base recipe based on the source + files. + </para></listitem> + <listitem><para><emphasis>Existing Recipes:</emphasis> + Location and modification of an existing recipe that is + similar in function to the recipe you need. + </para></listitem> + </itemizedlist> + </para> + + <section id='new-recipe-creating-the-base-recipe-using-recipetool'> + <title>Creating the Base Recipe Using <filename>recipetool</filename></title> + + <para> + <filename>recipetool</filename> automates creation of + a base recipe given a set of source code files. + As long as you can extract or point to the source files, + the tool will construct a recipe and automatically + configure all pre-build information into the recipe. + For example, suppose you have an application that builds + using Autotools. + Creating the base recipe using + <filename>recipetool</filename> results in a recipe + that has the pre-build dependencies, license requirements, + and checksums configured. + </para> + + <para> + To run the tool, you just need to be in your + <link linkend='build-directory'>Build Directory</link> + and have sourced the build environment setup script + (i.e. + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>oe-init-build-env</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>). + Here is the basic <filename>recipetool</filename> syntax: + <note> + Running <filename>recipetool -h</filename> or + <filename>recipetool create -h</filename> produces the + Python-generated help, which presented differently + than what follows here. + </note> + <literallayout class='monospaced'> + recipetool -h + recipetool create [-h] + recipetool [-d] [-q] [--color auto | always | never ] create -o <replaceable>OUTFILE</replaceable> [-m] [-x <replaceable>EXTERNALSRC</replaceable>] <replaceable>source</replaceable> + + -d Enables debug output. + -q Outputs only errors (quiet mode). + --color Colorizes the output automatically, always, or never. + -h Displays Python generated syntax for recipetool. + create Causes recipetool to create a base recipe. The create + command is further defined with these options: + + -o <replaceable>OUTFILE</replaceable> Specifies the full path and filename for the generated + recipe. + -m Causes the recipe to be machine-specific rather than + architecture-specific (default). + -x <replaceable>EXTERNALSRC</replaceable> Fetches and extracts source files from <replaceable>source</replaceable> + and places them in <replaceable>EXTERNALSRC</replaceable>. + <replaceable>source</replaceable> must be a URL. + -h Displays Python-generated syntax for create. + <replaceable>source</replaceable> Specifies the source code on which to base the + recipe. + </literallayout> + </para> + + <para> + Running <filename>recipetool create -o</filename> <replaceable>OUTFILE</replaceable> + creates the base recipe and locates it properly in the + layer that contains your source files. + Following are some syntax examples: + </para> + + <para> + Use this syntax to generate a recipe based on <replaceable>source</replaceable>. + Once generated, the recipe resides in the existing source + code layer: + <literallayout class='monospaced'> + recipetool create -o <replaceable>OUTFILE</replaceable> <replaceable>source</replaceable> + </literallayout> + Use this syntax to generate a recipe using code that you + extract from <replaceable>source</replaceable>. + The extracted code is placed in its own layer defined + by <replaceable>EXTERNALSRC</replaceable>. + <literallayout class='monospaced'> + recipetool create -o <replaceable>OUTFILE</replaceable> -x <replaceable>EXTERNALSRC</replaceable> <replaceable>source</replaceable> + </literallayout> + Use this syntax to generate a recipe based on <replaceable>source</replaceable>. + The options direct <filename>recipetool</filename> to + run in "quiet mode" and to generate debugging information. + Once generated, the recipe resides in the existing source + code layer: + <literallayout class='monospaced'> + recipetool create -o <replaceable>OUTFILE</replaceable> <replaceable>source</replaceable> + </literallayout> + </para> + </section> + + <section id='new-recipe-locating-and-using-a-similar-recipe'> + <title>Locating and Using a Similar Recipe</title> + + <para> + Before writing a recipe from scratch, it is often useful to + discover whether someone else has already written one that + meets (or comes close to meeting) your needs. + The Yocto Project and OpenEmbedded communities maintain many + recipes that might be candidates for what you are doing. + You can find a good central index of these recipes in the + <ulink url='http://layers.openembedded.org'>OpenEmbedded metadata index</ulink>. + </para> + + <para> + Working from an existing recipe or a skeleton recipe is the + best way to get started. + Here are some points on both methods: + <itemizedlist> + <listitem><para><emphasis>Locate and modify a recipe that + is close to what you want to do:</emphasis> + This method works when you are familiar with the + current recipe space. + The method does not work so well for those new to + the Yocto Project or writing recipes.</para> + <para>Some risks associated with this method are + using a recipe that has areas totally unrelated to + what you are trying to accomplish with your recipe, + not recognizing areas of the recipe that you might + have to add from scratch, and so forth. + All these risks stem from unfamiliarity with the + existing recipe space.</para></listitem> + <listitem><para><emphasis>Use and modify the following + skeleton recipe:</emphasis> + If for some reason you do not want to use + <filename>recipetool</filename> and you cannot + find an existing recipe that is close to meeting + your needs, you can use the following structure to + provide the fundamental areas of a new recipe. + <literallayout class='monospaced'> + DESCRIPTION = "" + HOMEPAGE = "" + LICENSE = "" + SECTION = "" + DEPENDS = "" + LIC_FILES_CHKSUM = "" + + SRC_URI = "" + </literallayout> + </para></listitem> + </itemizedlist> + </para> + </section> + </section> + + <section id='new-recipe-storing-and-naming-the-recipe'> + <title>Storing and Naming the Recipe</title> + + <para> + Once you have your base recipe, you should put it in your + own layer and name it appropriately. + Locating it correctly ensures that the OpenEmbedded build + system can find it when you use BitBake to process the + recipe. + </para> + + <itemizedlist> + <listitem><para><emphasis>Storing Your Recipe:</emphasis> + The OpenEmbedded build system locates your recipe + through the layer's <filename>conf/layer.conf</filename> + file and the + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'><filename>BBFILES</filename></ulink> + variable. + This variable sets up a path from which the build system can + locate recipes. + Here is the typical use: + <literallayout class='monospaced'> + BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ + ${LAYERDIR}/recipes-*/*/*.bbappend" + </literallayout> + Consequently, you need to be sure you locate your new recipe + inside your layer such that it can be found.</para> + <para>You can find more information on how layers are + structured in the + "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" + section.</para></listitem> + <listitem><para><emphasis>Naming Your Recipe:</emphasis> + When you name your recipe, you need to follow this naming + convention: + <literallayout class='monospaced'> + <replaceable>basename</replaceable>_<replaceable>version</replaceable>.bb + </literallayout> + Use lower-cased characters and do not include the reserved + suffixes <filename>-native</filename>, + <filename>-cross</filename>, <filename>-initial</filename>, + or <filename>-dev</filename> casually (i.e. do not use them + as part of your recipe name unless the string applies). + Here are some examples: + <literallayout class='monospaced'> + cups_1.7.0.bb + gawk_4.0.2.bb + irssi_0.8.16-rc1.bb + </literallayout></para></listitem> + </itemizedlist> + </section> + + <section id='understanding-recipe-syntax'> + <title>Understanding Recipe Syntax</title> + + <para> + Understanding recipe file syntax is important for + writing recipes. + The following list overviews the basic items that make up a + BitBake recipe file. + For more complete BitBake syntax descriptions, see the + "<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>" + chapter of the BitBake User Manual. + <itemizedlist> + <listitem><para><emphasis>Variable Assignments and Manipulations:</emphasis> + Variable assignments allow a value to be assigned to a + variable. + The assignment can be static text or might include + the contents of other variables. + In addition to the assignment, appending and prepending + operations are also supported.</para> + <para>The following example shows some of the ways + you can use variables in recipes: + <literallayout class='monospaced'> + S = "${WORKDIR}/postfix-${PV}" + CFLAGS += "-DNO_ASM" + SRC_URI_append = " file://fixup.patch" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Functions:</emphasis> + Functions provide a series of actions to be performed. + You usually use functions to override the default + implementation of a task function or to complement + a default function (i.e. append or prepend to an + existing function). + Standard functions use <filename>sh</filename> shell + syntax, although access to OpenEmbedded variables and + internal methods are also available.</para> + <para>The following is an example function from the + <filename>sed</filename> recipe: + <literallayout class='monospaced'> + do_install () { + autotools_do_install + install -d ${D}${base_bindir} + mv ${D}${bindir}/sed ${D}${base_bindir}/sed + rmdir ${D}${bindir}/ + } + </literallayout> + It is also possible to implement new functions that + are called between existing tasks as long as the + new functions are not replacing or complementing the + default functions. + You can implement functions in Python + instead of shell. + Both of these options are not seen in the majority of + recipes.</para></listitem> + <listitem><para><emphasis>Keywords:</emphasis> + BitBake recipes use only a few keywords. + You use keywords to include common + functions (<filename>inherit</filename>), load parts + of a recipe from other files + (<filename>include</filename> and + <filename>require</filename>) and export variables + to the environment (<filename>export</filename>).</para> + <para>The following example shows the use of some of + these keywords: + <literallayout class='monospaced'> + export POSTCONF = "${STAGING_BINDIR}/postconf" + inherit autoconf + require otherfile.inc + </literallayout> + </para></listitem> + <listitem><para><emphasis>Comments:</emphasis> + Any lines that begin with the hash character + (<filename>#</filename>) are treated as comment lines + and are ignored: + <literallayout class='monospaced'> + # This is a comment + </literallayout> + </para></listitem> + </itemizedlist> + </para> + + <para> + This next list summarizes the most important and most commonly + used parts of the recipe syntax. + For more information on these parts of the syntax, you can + reference the + <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink> + chapter in the BitBake User Manual. + <itemizedlist> + <listitem><para><emphasis>Line Continuation: <filename>\</filename></emphasis> - + Use the backward slash (<filename>\</filename>) + character to split a statement over multiple lines. + Place the slash character at the end of the line that + is to be continued on the next line: + <literallayout class='monospaced'> + VAR = "A really long \ + line" + </literallayout> + <note> + You cannot have any characters including spaces + or tabs after the slash character. + </note> + </para></listitem> + <listitem><para><emphasis>Using Variables: <filename>${...}</filename></emphasis> - + Use the <filename>${<replaceable>varname</replaceable>}</filename> syntax to + access the contents of a variable: + <literallayout class='monospaced'> + SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Quote All Assignments: <filename>"<replaceable>value</replaceable>"</filename></emphasis> - + Use double quotes around the value in all variable + assignments. + <literallayout class='monospaced'> + VAR1 = "${OTHERVAR}" + VAR2 = "The version is ${PV}" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Conditional Assignment: <filename>?=</filename></emphasis> - + Conditional assignment is used to assign a value to + a variable, but only when the variable is currently + unset. + Use the question mark followed by the equal sign + (<filename>?=</filename>) to make a "soft" assignment + used for conditional assignment. + Typically, "soft" assignments are used in the + <filename>local.conf</filename> file for variables + that are allowed to come through from the external + environment. + </para> + <para>Here is an example where + <filename>VAR1</filename> is set to "New value" if + it is currently empty. + However, if <filename>VAR1</filename> has already been + set, it remains unchanged: + <literallayout class='monospaced'> + VAR1 ?= "New value" + </literallayout> + In this next example, <filename>VAR1</filename> + is left with the value "Original value": + <literallayout class='monospaced'> + VAR1 = "Original value" + VAR1 ?= "New value" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Appending: <filename>+=</filename></emphasis> - + Use the plus character followed by the equals sign + (<filename>+=</filename>) to append values to existing + variables. + <note> + This operator adds a space between the existing + content of the variable and the new content. + </note></para> + <para>Here is an example: + <literallayout class='monospaced'> + SRC_URI += "file://fix-makefile.patch" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Prepending: <filename>=+</filename></emphasis> - + Use the equals sign followed by the plus character + (<filename>=+</filename>) to prepend values to existing + variables. + <note> + This operator adds a space between the new content + and the existing content of the variable. + </note></para> + <para>Here is an example: + <literallayout class='monospaced'> + VAR =+ "Starts" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Appending: <filename>_append</filename></emphasis> - + Use the <filename>_append</filename> operator to + append values to existing variables. + This operator does not add any additional space. + Also, the operator is applied after all the + <filename>+=</filename>, and + <filename>=+</filename> operators have been applied and + after all <filename>=</filename> assignments have + occurred. + </para> + <para>The following example shows the space being + explicitly added to the start to ensure the appended + value is not merged with the existing value: + <literallayout class='monospaced'> + SRC_URI_append = " file://fix-makefile.patch" + </literallayout> + You can also use the <filename>_append</filename> + operator with overrides, which results in the actions + only being performed for the specified target or + machine: + <literallayout class='monospaced'> + SRC_URI_append_sh4 = " file://fix-makefile.patch" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Prepending: <filename>_prepend</filename></emphasis> - + Use the <filename>_prepend</filename> operator to + prepend values to existing variables. + This operator does not add any additional space. + Also, the operator is applied after all the + <filename>+=</filename>, and + <filename>=+</filename> operators have been applied and + after all <filename>=</filename> assignments have + occurred. + </para> + <para>The following example shows the space being + explicitly added to the end to ensure the prepended + value is not merged with the existing value: + <literallayout class='monospaced'> + CFLAGS_prepend = "-I${S}/myincludes " + </literallayout> + You can also use the <filename>_prepend</filename> + operator with overrides, which results in the actions + only being performed for the specified target or + machine: + <literallayout class='monospaced'> + CFLAGS_prepend_sh4 = "-I${S}/myincludes " + </literallayout> + </para></listitem> + <listitem><para><emphasis>Overrides:</emphasis> - + You can use overrides to set a value conditionally, + typically based on how the recipe is being built. + For example, to set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-KBRANCH'><filename>KBRANCH</filename></ulink> + variable's value to "standard/base" for any target + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>, + except for qemuarm where it should be set to + "standard/arm-versatile-926ejs", you would do the + following: + <literallayout class='monospaced'> + KBRANCH = "standard/base" + KBRANCH_qemuarm = "standard/arm-versatile-926ejs" + </literallayout> + Overrides are also used to separate alternate values + of a variable in other situations. + For example, when setting variables such as + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink> + and + <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink> + that are specific to individual packages produced by + a recipe, you should always use an override that + specifies the name of the package. + </para></listitem> + <listitem><para><emphasis>Indentation:</emphasis> + Use spaces for indentation rather than than tabs. + For shell functions, both currently work. + However, it is a policy decision of the Yocto Project + to use tabs in shell functions. + Realize that some layers have a policy to use spaces + for all indentation. + </para></listitem> + <listitem><para><emphasis>Using Python for Complex Operations: <filename>${@<replaceable>python_code</replaceable>}</filename></emphasis> - + For more advanced processing, it is possible to use + Python code during variable assignments (e.g. + search and replacement on a variable).</para> + <para>You indicate Python code using the + <filename>${@<replaceable>python_code</replaceable>}</filename> + syntax for the variable assignment: + <literallayout class='monospaced'> + SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz + </literallayout> + </para></listitem> + <listitem><para><emphasis>Shell Function Syntax:</emphasis> + Write shell functions as if you were writing a shell + script when you describe a list of actions to take. + You should ensure that your script works with a generic + <filename>sh</filename> and that it does not require + any <filename>bash</filename> or other shell-specific + functionality. + The same considerations apply to various system + utilities (e.g. <filename>sed</filename>, + <filename>grep</filename>, <filename>awk</filename>, + and so forth) that you might wish to use. + If in doubt, you should check with multiple + implementations - including those from BusyBox. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='new-recipe-running-a-build-on-the-recipe'> + <title>Running a Build on the Recipe</title> + + <para> + Creating a new recipe is usually an iterative process that + requires using BitBake to process the recipe multiple times in + order to progressively discover and add information to the + recipe file. + </para> + + <para> + Assuming you have sourced a build environment setup script (i.e. + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>) + and you are in the + <link linkend='build-directory'>Build Directory</link>, + use BitBake to process your recipe. + All you need to provide is the + <filename><replaceable>basename</replaceable></filename> of the recipe as described + in the previous section: + <literallayout class='monospaced'> + $ bitbake <replaceable>basename</replaceable> + </literallayout> + + </para> + + <para> + During the build, the OpenEmbedded build system creates a + temporary work directory for each recipe + (<filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename>) + where it keeps extracted source files, log files, intermediate + compilation and packaging files, and so forth. + </para> + + <para> + The per-recipe temporary work directory is constructed as follows and + depends on several factors: + <literallayout class='monospaced'> + BASE_WORKDIR ?= "${TMPDIR}/work" + WORKDIR = "${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}" + </literallayout> + As an example, assume a Source Directory top-level folder named + <filename>poky</filename>, a default Build Directory at + <filename>poky/build</filename>, and a + <filename>qemux86-poky-linux</filename> machine target system. + Furthermore, suppose your recipe is named + <filename>foo_1.3.0.bb</filename>. + In this case, the work directory the build system uses to + build the package would be as follows: + <literallayout class='monospaced'> + poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 + </literallayout> + Inside this directory you can find sub-directories such as + <filename>image</filename>, <filename>packages-split</filename>, + and <filename>temp</filename>. + After the build, you can examine these to determine how well + the build went. + <note> + You can find log files for each task in the recipe's + <filename>temp</filename> directory (e.g. + <filename>poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp</filename>). + Log files are named <filename>log.<replaceable>taskname</replaceable></filename> + (e.g. <filename>log.do_configure</filename>, + <filename>log.do_fetch</filename>, and + <filename>log.do_compile</filename>). + </note> + </para> + + <para> + You can find more information about the build process in the + "<ulink url='&YOCTO_DOCS_REF_URL;#closer-look'>A Closer Look at the Yocto Project Development Environment</ulink>" + chapter of the Yocto Project Reference Manual. + </para> + + <para> + You can also reference the following variables in the + Yocto Project Reference Manual's glossary for more information: + <itemizedlist> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>: + The top-level build output directory</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-MULTIMACH_TARGET_SYS'><filename>MULTIMACH_TARGET_SYS</filename></ulink>: + The target system identifier</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>: + The recipe name</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-EXTENDPE'><filename>EXTENDPE</filename></ulink>: + The epoch - (if + <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink> + is not specified, which is usually the case for most + recipes, then <filename>EXTENDPE</filename> is blank)</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>: + The recipe version</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>: + The recipe revision</listitem> + </itemizedlist> + </para> + </section> + + <section id='new-recipe-fetching-code'> + <title>Fetching Code</title> + + <para> + The first thing your recipe must do is specify how to fetch + the source files. + Fetching is controlled mainly through the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + variable. + Your recipe must have a <filename>SRC_URI</filename> variable + that points to where the source is located. + For a graphical representation of source locations, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#sources-dev-environment'>Sources</ulink>" + section in the Yocto Project Reference Manual. + </para> + + <para> + The + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink> + task uses the prefix of each entry in the + <filename>SRC_URI</filename> variable value to determine which + fetcher to use to get your source files. + It is the <filename>SRC_URI</filename> variable that triggers + the fetcher. + The + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> + task uses the variable after source is fetched to apply + patches. + The OpenEmbedded build system uses + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESOVERRIDES'><filename>FILESOVERRIDES</filename></ulink> + for scanning directory locations for local files in + <filename>SRC_URI</filename>. + </para> + + <para> + The <filename>SRC_URI</filename> variable in your recipe must + define each unique location for your source files. + It is good practice to not hard-code pathnames in an URL used + in <filename>SRC_URI</filename>. + Rather than hard-code these paths, use + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>, + which causes the fetch process to use the version specified in + the recipe filename. + Specifying the version in this manner means that upgrading the + recipe to a future version is as simple as renaming the recipe + to match the new version. + </para> + + <para> + Here is a simple example from the + <filename>meta/recipes-devtools/cdrtools/cdrtools-native_3.01a20.bb</filename> + recipe where the source comes from a single tarball. + Notice the use of the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> + variable: + <literallayout class='monospaced'> + SRC_URI = "ftp://ftp.berlios.de/pub/cdrecord/alpha/cdrtools-${PV}.tar.bz2" + </literallayout> + </para> + + <para> + Files mentioned in <filename>SRC_URI</filename> whose names end + in a typical archive extension (e.g. <filename>.tar</filename>, + <filename>.tar.gz</filename>, <filename>.tar.bz2</filename>, + <filename>.zip</filename>, and so forth), are automatically + extracted during the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink> + task. + For another example that specifies these types of files, see + the + "<link linkend='new-recipe-autotooled-package'>Autotooled Package</link>" + section. + </para> + + <para> + Another way of specifying source is from an SCM. + For Git repositories, you must specify + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink> + and you should specify + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> + to include the revision with + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>. + Here is an example from the recipe + <filename>meta/recipes-kernel/blktrace/blktrace_git.bb</filename>: + <literallayout class='monospaced'> + SRCREV = "d6918c8832793b4205ed3bfede78c2f915c23385" + + PR = "r6" + PV = "1.0.5+git${SRCPV}" + + SRC_URI = "git://git.kernel.dk/blktrace.git \ + file://ldflags.patch" + </literallayout> + </para> + + <para> + If your <filename>SRC_URI</filename> statement includes + URLs pointing to individual files fetched from a remote server + other than a version control system, BitBake attempts to + verify the files against checksums defined in your recipe to + ensure they have not been tampered with or otherwise modified + since the recipe was written. + Two checksums are used: + <filename>SRC_URI[md5sum]</filename> and + <filename>SRC_URI[sha256sum]</filename>. + </para> + + <para> + If your <filename>SRC_URI</filename> variable points to + more than a single URL (excluding SCM URLs), you need to + provide the <filename>md5</filename> and + <filename>sha256</filename> checksums for each URL. + For these cases, you provide a name for each URL as part of + the <filename>SRC_URI</filename> and then reference that name + in the subsequent checksum statements. + Here is an example: + <literallayout class='monospaced'> + SRC_URI = "${DEBIAN_MIRROR}/main/a/apmd/apmd_3.2.2.orig.tar.gz;name=tarball \ + ${DEBIAN_MIRROR}/main/a/apmd/apmd_${PV}.diff.gz;name=patch + + SRC_URI[tarball.md5sum] = "b1e6309e8331e0f4e6efd311c2d97fa8" + SRC_URI[tarball.sha256sum] = "7f7d9f60b7766b852881d40b8ff91d8e39fccb0d1d913102a5c75a2dbb52332d" + + SRC_URI[patch.md5sum] = "57e1b689264ea80f78353519eece0c92" + SRC_URI[patch.sha256sum] = "7905ff96be93d725544d0040e425c42f9c05580db3c272f11cff75b9aa89d430" + </literallayout> + </para> + + <para> + Proper values for <filename>md5</filename> and + <filename>sha256</filename> checksums might be available + with other signatures on the download page for the upstream + source (e.g. <filename>md5</filename>, + <filename>sha1</filename>, <filename>sha256</filename>, + <filename>GPG</filename>, and so forth). + Because the OpenEmbedded build system only deals with + <filename>sha256sum</filename> and <filename>md5sum</filename>, + you should verify all the signatures you find by hand. + </para> + + <para> + If no <filename>SRC_URI</filename> checksums are specified + when you attempt to build the recipe, or you provide an + incorrect checksum, the build will produce an error for each + missing or incorrect checksum. + As part of the error message, the build system provides + the checksum string corresponding to the fetched file. + Once you have the correct checksums, you can copy and paste + them into your recipe and then run the build again to continue. + <note> + As mentioned, if the upstream source provides signatures + for verifying the downloaded source code, you should + verify those manually before setting the checksum values + in the recipe and continuing with the build. + </note> + </para> + + <para> + This final example is a bit more complicated and is from the + <filename>meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb</filename> + recipe. + The example's <filename>SRC_URI</filename> statement identifies + multiple files as the source files for the recipe: a tarball, a + patch file, a desktop file, and an icon. + <literallayout class='monospaced'> + SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \ + file://xwc.patch \ + file://rxvt.desktop \ + file://rxvt.png" + </literallayout> + </para> + + <para> + When you specify local files using the + <filename>file://</filename> URI protocol, the build system + fetches files from the local machine. + The path is relative to the + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink> + variable and searches specific directories in a certain order: + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink><filename>}</filename>, + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}</filename>, + and <filename>files</filename>. + The directories are assumed to be subdirectories of the + directory in which the recipe or append file resides. + For another example that specifies these types of files, see the + "<link linkend='new-recipe-single-c-file-package-hello-world'>Single .c File Package (Hello World!)</link>" + section. + </para> + + <para> + The previous example also specifies a patch file. + Patch files are files whose names usually end in + <filename>.patch</filename> or <filename>.diff</filename> but + can end with compressed suffixes such as + <filename>diff.gz</filename> and + <filename>patch.bz2</filename>, for example. + The build system automatically applies patches as described + in the + "<link linkend='new-recipe-patching-code'>Patching Code</link>" section. + </para> + </section> + + <section id='new-recipe-unpacking-code'> + <title>Unpacking Code</title> + + <para> + During the build, the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink> + task unpacks the source with + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename> + pointing to where it is unpacked. + </para> + + <para> + If you are fetching your source files from an upstream source + archived tarball and the tarball's internal structure matches + the common convention of a top-level subdirectory named + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}-${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>, + then you do not need to set <filename>S</filename>. + However, if <filename>SRC_URI</filename> specifies to fetch + source from an archive that does not use this convention, + or from an SCM like Git or Subversion, your recipe needs to + define <filename>S</filename>. + </para> + + <para> + If processing your recipe using BitBake successfully unpacks + the source files, you need to be sure that the directory + pointed to by <filename>${S}</filename> matches the structure + of the source. + </para> + </section> + + <section id='new-recipe-patching-code'> + <title>Patching Code</title> + + <para> + Sometimes it is necessary to patch code after it has been + fetched. + Any files mentioned in <filename>SRC_URI</filename> whose + names end in <filename>.patch</filename> or + <filename>.diff</filename> or compressed versions of these + suffixes (e.g. <filename>diff.gz</filename> are treated as + patches. + The + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> + task automatically applies these patches. + </para> + + <para> + The build system should be able to apply patches with the "-p1" + option (i.e. one directory level in the path will be stripped + off). + If your patch needs to have more directory levels stripped off, + specify the number of levels using the "striplevel" option in + the <filename>SRC_URI</filename> entry for the patch. + Alternatively, if your patch needs to be applied in a specific + subdirectory that is not specified in the patch file, use the + "patchdir" option in the entry. + </para> + + <para> + As with all local files referenced in + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + using <filename>file://</filename>, you should place + patch files in a directory next to the recipe either + named the same as the base name of the recipe + (<ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink> + and + <ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink>) + or "files". + </para> + </section> + + <section id='new-recipe-licensing'> + <title>Licensing</title> + + <para> + Your recipe needs to have both the + <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink> + and + <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink> + variables: + <itemizedlist> + <listitem><para><emphasis><filename>LICENSE</filename>:</emphasis> + This variable specifies the license for the software. + If you do not know the license under which the software + you are building is distributed, you should go to the + source code and look for that information. + Typical files containing this information include + <filename>COPYING</filename>, + <filename>LICENSE</filename>, and + <filename>README</filename> files. + You could also find the information near the top of + a source file. + For example, given a piece of software licensed under + the GNU General Public License version 2, you would + set <filename>LICENSE</filename> as follows: + <literallayout class='monospaced'> + LICENSE = "GPLv2" + </literallayout></para> + <para>The licenses you specify within + <filename>LICENSE</filename> can have any name as long + as you do not use spaces, since spaces are used as + separators between license names. + For standard licenses, use the names of the files in + <filename>meta/files/common-licenses/</filename> + or the <filename>SPDXLICENSEMAP</filename> flag names + defined in <filename>meta/conf/licenses.conf</filename>. + </para></listitem> + <listitem><para><emphasis><filename>LIC_FILES_CHKSUM</filename>:</emphasis> + The OpenEmbedded build system uses this variable to + make sure the license text has not changed. + If it has, the build produces an error and it affords + you the chance to figure it out and correct the problem. + </para> + <para>You need to specify all applicable licensing + files for the software. + At the end of the configuration step, the build process + will compare the checksums of the files to be sure + the text has not changed. + Any differences result in an error with the message + containing the current checksum. + For more explanation and examples of how to set the + <filename>LIC_FILES_CHKSUM</filename> variable, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</ulink>" + section in the Yocto Project Reference Manual.</para> + <para>To determine the correct checksum string, you + can list the appropriate files in the + <filename>LIC_FILES_CHKSUM</filename> variable with + incorrect md5 strings, attempt to build the software, + and then note the resulting error messages that will + report the correct md5 strings. + See the + "<link linkend='new-recipe-fetching-code'>Fetching Code</link>" + section for additional information. + </para> + + <para> + Here is an example that assumes the software has a + <filename>COPYING</filename> file: + <literallayout class='monospaced'> + LIC_FILES_CHKSUM = "file://COPYING;md5=xxx" + </literallayout> + When you try to build the software, the build system + will produce an error and give you the correct string + that you can substitute into the recipe file for a + subsequent build. + </para></listitem> + </itemizedlist> + </para> + +<!-- + + <para> + For trying this out I created a new recipe named + <filename>htop_1.0.2.bb</filename> and put it in + <filename>poky/meta/recipes-extended/htop</filename>. + There are two license type statements in my very simple + recipe: + <literallayout class='monospaced'> + LICENSE = "" + + LIC_FILES_CHKSUM = "" + + SRC_URI[md5sum] = "" + SRC_URI[sha256sum] = "" + </literallayout> + Evidently, you need to run a <filename>bitbake -c cleanall htop</filename>. + Next, you delete or comment out the two <filename>SRC_URI</filename> + lines at the end and then attempt to build the software with + <filename>bitbake htop</filename>. + Doing so causes BitBake to report some errors and and give + you the actual strings you need for the last two + <filename>SRC_URI</filename> lines. + Prior to this, you have to dig around in the home page of the + source for <filename>htop</filename> and determine that the + software is released under GPLv2. + You can provide that in the <filename>LICENSE</filename> + statement. + Now you edit your recipe to have those two strings for + the <filename>SRC_URI</filename> statements: + <literallayout class='monospaced'> + LICENSE = "GPLv2" + + LIC_FILES_CHKSUM = "" + + SRC_URI = "${SOURCEFORGE_MIRROR}/htop/htop-${PV}.tar.gz" + SRC_URI[md5sum] = "0d01cca8df3349c74569cefebbd9919e" + SRC_URI[sha256sum] = "ee60657b044ece0df096c053060df7abf3cce3a568ab34d260049e6a37ccd8a1" + </literallayout> + At this point, you can build the software again using the + <filename>bitbake htop</filename> command. + There is just a set of errors now associated with the + empty <filename>LIC_FILES_CHKSUM</filename> variable now. + </para> +--> + + </section> + + <section id='new-recipe-configuring-the-recipe'> + <title>Configuring the Recipe</title> + + <para> + Most software provides some means of setting build-time + configuration options before compilation. + Typically, setting these options is accomplished by running a + configure script with some options, or by modifying a build + configuration file. + <note> + As of Yocto Project Release 7.1, some of the core recipes + that package binary configuration scripts now disable the + scripts due to the scripts previously requiring error-prone + path substitution. + The OpenEmbedded build system uses + <filename>pkg-config</filename> now, which is much more + robust. + You can find a list of the <filename>*-config</filename> + scripts that are disabled list in the + "<ulink url='&YOCTO_DOCS_REF_URL;#migration-1.7-binary-configuration-scripts-disabled'>Binary Configuration Scripts Disabled</ulink>" + section in the Yocto Project Reference Manual. + </note> + </para> + + <para> + A major part of build-time configuration is about checking for + build-time dependencies and possibly enabling optional + functionality as a result. + You need to specify any build-time dependencies for the + software you are building in your recipe's + <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> + value, in terms of other recipes that satisfy those + dependencies. + You can often find build-time or runtime + dependencies described in the software's documentation. + </para> + + <para> + The following list provides configuration items of note based + on how your software is built: + <itemizedlist> + <listitem><para><emphasis>Autotools:</emphasis> + If your source files have a + <filename>configure.ac</filename> file, then your + software is built using Autotools. + If this is the case, you just need to worry about + modifying the configuration.</para> + <para>When using Autotools, your recipe needs to inherit + the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink> + class and your recipe does not have to contain a + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> + task. + However, you might still want to make some adjustments. + For example, you can set + <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></ulink> + to pass any needed configure options that are specific + to the recipe.</para></listitem> + <listitem><para><emphasis>CMake:</emphasis> + If your source files have a + <filename>CMakeLists.txt</filename> file, then your + software is built using CMake. + If this is the case, you just need to worry about + modifying the configuration.</para> + <para>When you use CMake, your recipe needs to inherit + the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink> + class and your recipe does not have to contain a + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> + task. + You can make some adjustments by setting + <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE'><filename>EXTRA_OECMAKE</filename></ulink> + to pass any needed configure options that are specific + to the recipe.</para></listitem> + <listitem><para><emphasis>Other:</emphasis> + If your source files do not have a + <filename>configure.ac</filename> or + <filename>CMakeLists.txt</filename> file, then your + software is built using some method other than Autotools + or CMake. + If this is the case, you normally need to provide a + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> + task in your recipe + unless, of course, there is nothing to configure. + </para> + <para>Even if your software is not being built by + Autotools or CMake, you still might not need to deal + with any configuration issues. + You need to determine if configuration is even a required step. + You might need to modify a Makefile or some configuration file + used for the build to specify necessary build options. + Or, perhaps you might need to run a provided, custom + configure script with the appropriate options.</para> + <para>For the case involving a custom configure + script, you would run + <filename>./configure --help</filename> and look for + the options you need to set.</para></listitem> + </itemizedlist> + </para> + + <para> + Once configuration succeeds, it is always good practice to + look at the <filename>log.do_configure</filename> file to + ensure that the appropriate options have been enabled and no + additional build-time dependencies need to be added to + <filename>DEPENDS</filename>. + For example, if the configure script reports that it found + something not mentioned in <filename>DEPENDS</filename>, or + that it did not find something that it needed for some + desired optional functionality, then you would need to add + those to <filename>DEPENDS</filename>. + Looking at the log might also reveal items being checked for, + enabled, or both that you do not want, or items not being found + that are in <filename>DEPENDS</filename>, in which case + you would need to look at passing extra options to the + configure script as needed. + For reference information on configure options specific to the + software you are building, you can consult the output of the + <filename>./configure --help</filename> command within + <filename>${S}</filename> or consult the software's upstream + documentation. + </para> + </section> + + <section id='new-recipe-compilation'> + <title>Compilation</title> + + <para> + During a build, the <filename>do_compile</filename> task + happens after source is fetched, unpacked, and configured. + If the recipe passes through <filename>do_compile</filename> + successfully, nothing needs to be done. + </para> + + <para> + However, if the compile step fails, you need to diagnose the + failure. + Here are some common issues that cause failures. + <note> + For cases where improper paths are detected for + configuration files or for when libraries/headers cannot + be found, be sure you are using the more robust + <filename>pkg-config</filename>. + See the note in section + "<link linkend='new-recipe-configuring-the-recipe'>Configuring the Recipe</link>" + for additional information. + </note> + <itemizedlist> + <listitem><para><emphasis>Parallel build failures:</emphasis> + These failures manifest themselves as intermittent + errors, or errors reporting that a file or directory + that should be created by some other part of the build + process could not be found. + This type of failure can occur even if, upon inspection, + the file or directory does exist after the build has + failed, because that part of the build process happened + in the wrong order.</para> + <para>To fix the problem, you need to either satisfy + the missing dependency in the Makefile or whatever + script produced the Makefile, or (as a workaround) + set + <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink> + to an empty string: + <literallayout class='monospaced'> + PARALLEL_MAKE = "" + </literallayout></para> + <para> + For information on parallel Makefile issues, see the + "<link linkend='debugging-parallel-make-races'>Debugging Parallel Make Races</link>" + section. + </para></listitem> + <listitem><para><emphasis>Improper host path usage:</emphasis> + This failure applies to recipes building for the target + or <filename>nativesdk</filename> only. + The failure occurs when the compilation process uses + improper headers, libraries, or other files from the + host system when cross-compiling for the target. + </para> + <para>To fix the problem, examine the + <filename>log.do_compile</filename> file to identify + the host paths being used (e.g. + <filename>/usr/include</filename>, + <filename>/usr/lib</filename>, and so forth) and then + either add configure options, apply a patch, or do both. + </para></listitem> + <listitem><para><emphasis>Failure to find required + libraries/headers:</emphasis> + If a build-time dependency is missing because it has + not been declared in + <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>, + or because the dependency exists but the path used by + the build process to find the file is incorrect and the + configure step did not detect it, the compilation + process could fail. + For either of these failures, the compilation process + notes that files could not be found. + In these cases, you need to go back and add additional + options to the configure script as well as possibly + add additional build-time dependencies to + <filename>DEPENDS</filename>.</para> + <para>Occasionally, it is necessary to apply a patch + to the source to ensure the correct paths are used. + If you need to specify paths to find files staged + into the sysroot from other recipes, use the variables + that the OpenEmbedded build system provides + (e.g. + <filename>STAGING_BINDIR</filename>, + <filename>STAGING_INCDIR</filename>, + <filename>STAGING_DATADIR</filename>, and so forth). +<!-- + (e.g. + <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_BINDIR'><filename>STAGING_BINDIR</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_INCDIR'><filename>STAGING_INCDIR</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DATADIR'><filename>STAGING_DATADIR</filename></ulink>, + and so forth). +--> + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='new-recipe-installing'> + <title>Installing</title> + + <para> + During <filename>do_install</filename>, the task copies the + built files along with their hierarchy to locations that + would mirror their locations on the target device. + The installation process copies files from the + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>, + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-B'><filename>B</filename></ulink><filename>}</filename>, + and + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename> + directories to the + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename> + directory to create the structure as it should appear on the + target system. + </para> + + <para> + How your software is built affects what you must do to be + sure your software is installed correctly. + The following list describes what you must do for installation + depending on the type of build system used by the software + being built: + <itemizedlist> + <listitem><para><emphasis>Autotools and CMake:</emphasis> + If the software your recipe is building uses Autotools + or CMake, the OpenEmbedded build + system understands how to install the software. + Consequently, you do not have to have a + <filename>do_install</filename> task as part of your + recipe. + You just need to make sure the install portion of the + build completes with no issues. + However, if you wish to install additional files not + already being installed by + <filename>make install</filename>, you should do this + using a <filename>do_install_append</filename> function + using the install command as described in + the "Manual" bulleted item later in this list. + </para></listitem> + <listitem><para><emphasis>Other (using + <filename>make install</filename>):</emphasis> + You need to define a + <filename>do_install</filename> function in your + recipe. + The function should call + <filename>oe_runmake install</filename> and will likely + need to pass in the destination directory as well. + How you pass that path is dependent on how the + <filename>Makefile</filename> being run is written + (e.g. <filename>DESTDIR=${D}</filename>, + <filename>PREFIX=${D}</filename>, + <filename>INSTALLROOT=${D}</filename>, and so forth). + </para> + <para>For an example recipe using + <filename>make install</filename>, see the + "<link linkend='new-recipe-makefile-based-package'>Makefile-Based Package</link>" + section.</para></listitem> + <listitem><para><emphasis>Manual:</emphasis> + You need to define a + <filename>do_install</filename> function in your + recipe. + The function must first use + <filename>install -d</filename> to create the + directories under + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>. + Once the directories exist, your function can use + <filename>install</filename> to manually install the + built software into the directories.</para> + <para>You can find more information on + <filename>install</filename> at + <ulink url='http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html'></ulink>. + </para></listitem> + </itemizedlist> + </para> + + <para> + For the scenarios that do not use Autotools or + CMake, you need to track the installation + and diagnose and fix any issues until everything installs + correctly. + You need to look in the default location of + <filename>${D}</filename>, which is + <filename>${WORKDIR}/image</filename>, to be sure your + files have been installed correctly. + </para> + + <note><title>Notes</title> + <itemizedlist> + <listitem><para> + During the installation process, you might need to + modify some of the installed files to suit the target + layout. + For example, you might need to replace hard-coded paths + in an initscript with values of variables provided by + the build system, such as replacing + <filename>/usr/bin/</filename> with + <filename>${bindir}</filename>. + If you do perform such modifications during + <filename>do_install</filename>, be sure to modify the + destination file after copying rather than before + copying. + Modifying after copying ensures that the build system + can re-execute <filename>do_install</filename> if + needed. + </para></listitem> + <listitem><para> + <filename>oe_runmake install</filename>, which can be + run directly or can be run indirectly by the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink> + and + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink> + classes, runs <filename>make install</filename> in + parallel. + Sometimes, a Makefile can have missing dependencies + between targets that can result in race conditions. + If you experience intermittent failures during + <filename>do_install</filename>, you might be able to + work around them by disabling parallel Makefile + installs by adding the following to the recipe: + <literallayout class='monospaced'> + PARALLEL_MAKEINST = "" + </literallayout> + See + <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink> + for additional information. + </para></listitem> + </itemizedlist> + </note> + </section> + + <section id='new-recipe-enabling-system-services'> + <title>Enabling System Services</title> + + <para> + If you want to install a service, which is a process that + usually starts on boot and runs in the background, then + you must include some additional definitions in your recipe. + </para> + + <para> + If you are adding services and the service initialization + script or the service file itself is not installed, you must + provide for that installation in your recipe using a + <filename>do_install_append</filename> function. + If your recipe already has a <filename>do_install</filename> + function, update the function near its end rather than + adding an additional <filename>do_install_append</filename> + function. + </para> + + <para> + When you create the installation for your services, you need + to accomplish what is normally done by + <filename>make install</filename>. + In other words, make sure your installation arranges the output + similar to how it is arranged on the target system. + </para> + + <para> + The OpenEmbedded build system provides support for starting + services two different ways: + <itemizedlist> + <listitem><para><emphasis>SysVinit:</emphasis> + SysVinit is a system and service manager that + manages the init system used to control the very basic + functions of your system. + The init program is the first program + started by the Linux kernel when the system boots. + Init then controls the startup, running and shutdown + of all other programs.</para> + <para>To enable a service using SysVinit, your recipe + needs to inherit the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-update-rc.d'><filename>update-rc.d</filename></ulink> + class. + The class helps facilitate safely installing the + package on the target.</para> + <para>You will need to set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PACKAGES'><filename>INITSCRIPT_PACKAGES</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_NAME'><filename>INITSCRIPT_NAME</filename></ulink>, + and + <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PARAMS'><filename>INITSCRIPT_PARAMS</filename></ulink> + variables within your recipe.</para></listitem> + <listitem><para><emphasis>systemd:</emphasis> + System Management Daemon (systemd) was designed to + replace SysVinit and to provide + enhanced management of services. + For more information on systemd, see the systemd + homepage at + <ulink url='http://freedesktop.org/wiki/Software/systemd/'></ulink>. + </para> + <para>To enable a service using systemd, your recipe + needs to inherit the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-systemd'><filename>systemd</filename></ulink> + class. + See the <filename>systemd.bbclass</filename> file + located in your + <link linkend='source-directory'>Source Directory</link>. + section for more information. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='new-recipe-packaging'> + <title>Packaging</title> + + <para> + Successful packaging is a combination of automated processes + performed by the OpenEmbedded build system and some + specific steps you need to take. + The following list describes the process: + <itemizedlist> + <listitem><para><emphasis>Splitting Files</emphasis>: + The <filename>do_package</filename> task splits the + files produced by the recipe into logical components. + Even software that produces a single binary might + still have debug symbols, documentation, and other + logical components that should be split out. + The <filename>do_package</filename> task ensures + that files are split up and packaged correctly. + </para></listitem> + <listitem><para><emphasis>Running QA Checks</emphasis>: + The + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink> + class adds a step to + the package generation process so that output quality + assurance checks are generated by the OpenEmbedded + build system. + This step performs a range of checks to be sure the + build's output is free of common problems that show + up during runtime. + For information on these checks, see the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink> + class and the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-qa-checks'>QA Error and Warning Messages</ulink>" + chapter in the Yocto Project Reference Manual. + </para></listitem> + <listitem><para><emphasis>Hand-Checking Your Packages</emphasis>: + After you build your software, you need to be sure + your packages are correct. + Examine the + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/packages-split</filename> + directory and make sure files are where you expect + them to be. + If you discover problems, you can set + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>, + <filename>do_install(_append)</filename>, and so forth as + needed. + </para></listitem> + <listitem><para><emphasis>Splitting an Application into Multiple Packages</emphasis>: + If you need to split an application into several + packages, see the + "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>" + section for an example. + </para></listitem> + <listitem><para><emphasis>Installing a Post-Installation Script</emphasis>: + For an example showing how to install a + post-installation script, see the + "<link linkend='new-recipe-post-installation-scripts'>Post-Installation Scripts</link>" + section. + </para></listitem> + <listitem><para><emphasis>Marking Package Architecture</emphasis>: + Depending on what your recipe is building and how it + is configured, it might be important to mark the + packages produced as being specific to a particular + machine, or to mark them as not being specific to + a particular machine or architecture at all.</para> + <para>By default, packages apply to any machine with the + same architecture as the target machine. + When a recipe produces packages that are + machine-specific (e.g. the + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + value is passed into the configure script or a patch + is applied only for a particular machine), you should + mark them as such by adding the following to the + recipe: + <literallayout class='monospaced'> + PACKAGE_ARCH = "${MACHINE_ARCH}" + </literallayout></para> + <para>On the other hand, if the recipe produces packages + that do not contain anything specific to the target + machine or architecture at all (e.g. recipes + that simply package script files or configuration + files), you should use the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink> + class to do this for you by adding this to your + recipe: + <literallayout class='monospaced'> + inherit allarch + </literallayout> + Ensuring that the package architecture is correct is + not critical while you are doing the first few builds + of your recipe. + However, it is important in order + to ensure that your recipe rebuilds (or does not + rebuild) appropriately in response to changes in + configuration, and to ensure that you get the + appropriate packages installed on the target machine, + particularly if you run separate builds for more + than one target machine. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='properly-versioning-pre-release-recipes'> + <title>Properly Versioning Pre-Release Recipes</title> + + <para> + Sometimes the name of a recipe can lead to versioning + problems when the recipe is upgraded to a final release. + For example, consider the + <filename>irssi_0.8.16-rc1.bb</filename> recipe file in + the list of example recipes in the + "<link linkend='new-recipe-storing-and-naming-the-recipe'>Storing and Naming the Recipe</link>" + section. + This recipe is at a release candidate stage (i.e. + "rc1"). + When the recipe is released, the recipe filename becomes + <filename>irssi_0.8.16.bb</filename>. + The version change from <filename>0.8.16-rc1</filename> + to <filename>0.8.16</filename> is seen as a decrease by the + build system and package managers, so the resulting packages + will not correctly trigger an upgrade. + </para> + + <para> + In order to ensure the versions compare properly, the + recommended convention is to set + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> + within the recipe to + "<replaceable>previous_version</replaceable>+<replaceable>current_version</replaceable>". + You can use an additional variable so that you can use the + current version elsewhere. + Here is an example: + <literallayout class='monospaced'> + REALPV = "0.8.16-rc1" + PV = "0.8.15+${REALPV}" + </literallayout> + </para> + </section> + + <section id='new-recipe-post-installation-scripts'> + <title>Post-Installation Scripts</title> + + <para> + Post-installation scripts run immediately after installing + a package on the target or during image creation when a + package is included in an image. + To add a post-installation script to a package, add a + <filename>pkg_postinst_PACKAGENAME()</filename> function to + the recipe file (<filename>.bb</filename>) and replace + <filename>PACKAGENAME</filename> with the name of the package + you want to attach to the <filename>postinst</filename> + script. + To apply the post-installation script to the main package + for the recipe, which is usually what is required, specify + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename> + in place of <filename>PACKAGENAME</filename>. + </para> + + <para> + A post-installation function has the following structure: + <literallayout class='monospaced'> + pkg_postinst_PACKAGENAME() { + # Commands to carry out + } + </literallayout> + </para> + + <para> + The script defined in the post-installation function is + called when the root filesystem is created. + If the script succeeds, the package is marked as installed. + If the script fails, the package is marked as unpacked and + the script is executed when the image boots again. + </para> + + <para> + Sometimes it is necessary for the execution of a + post-installation script to be delayed until the first boot. + For example, the script might need to be executed on the + device itself. + To delay script execution until boot time, use the following + structure in the post-installation script: + <literallayout class='monospaced'> + pkg_postinst_PACKAGENAME() { + if [ x"$D" = "x" ]; then + # Actions to carry out on the device go here + else + exit 1 + fi + } + </literallayout> + </para> + + <para> + The previous example delays execution until the image boots + again because the environment variable <filename>D</filename> + points to the directory containing the image when + the root filesystem is created at build time but is unset + when executed on the first boot. + </para> + + <note> + Equivalent support for pre-install, pre-uninstall, and + post-uninstall scripts exist by way of + <filename>pkg_preinst</filename>, + <filename>pkg_prerm</filename>, and + <filename>pkg_postrm</filename>, respectively. + These scrips work in exactly the same way as does + <filename>pkg_postinst</filename> with the exception that they + run at different times. + Also, because of when they run, they are not applicable to + being run at image creation time like + <filename>pkg_postinst</filename>. + </note> + </section> + + <section id='new-recipe-testing'> + <title>Testing</title> + + <para> + The final step for completing your recipe is to be sure that + the software you built runs correctly. + To accomplish runtime testing, add the build's output + packages to your image and test them on the target. + </para> + + <para> + For information on how to customize your image by adding + specific packages, see the + "<link linkend='usingpoky-extend-customimage'>Customizing Images</link>" + section. + </para> + </section> + + <section id='new-recipe-testing-examples'> + <title>Examples</title> + + <para> + To help summarize how to write a recipe, this section provides + some examples given various scenarios: + <itemizedlist> + <listitem><para>Recipes that use local files</para></listitem> + <listitem><para>Using an Autotooled package</para></listitem> + <listitem><para>Using a Makefile-based package</para></listitem> + <listitem><para>Splitting an application into multiple packages</para></listitem> + <listitem><para>Adding binaries to an image</para></listitem> + </itemizedlist> + </para> + + <section id='new-recipe-single-c-file-package-hello-world'> + <title>Single .c File Package (Hello World!)</title> + + <para> + Building an application from a single file that is stored + locally (e.g. under <filename>files</filename>) requires + a recipe that has the file listed in the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> + variable. + Additionally, you need to manually write the + <filename>do_compile</filename> and + <filename>do_install</filename> tasks. + The <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename> + variable defines the directory containing the source code, + which is set to + <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink> + in this case - the directory BitBake uses for the build. + <literallayout class='monospaced'> + SUMMARY = "Simple helloworld application" + SECTION = "examples" + LICENSE = "MIT" + LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" + + SRC_URI = "file://helloworld.c" + + S = "${WORKDIR}" + + do_compile() { + ${CC} helloworld.c -o helloworld + } + + do_install() { + install -d ${D}${bindir} + install -m 0755 helloworld ${D}${bindir} + } + </literallayout> + </para> + + <para> + By default, the <filename>helloworld</filename>, + <filename>helloworld-dbg</filename>, and + <filename>helloworld-dev</filename> packages are built. + For information on how to customize the packaging process, + see the + "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>" + section. + </para> + </section> + + <section id='new-recipe-autotooled-package'> + <title>Autotooled Package</title> + <para> + Applications that use Autotools such as <filename>autoconf</filename> and + <filename>automake</filename> require a recipe that has a source archive listed in + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> and + also inherit the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink> + class, which contains the definitions of all the steps + needed to build an Autotool-based application. + The result of the build is automatically packaged. + And, if the application uses NLS for localization, packages with local information are + generated (one package per language). + Following is one example: (<filename>hello_2.3.bb</filename>) + <literallayout class='monospaced'> + SUMMARY = "GNU Helloworld application" + SECTION = "examples" + LICENSE = "GPLv2+" + LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe" + + SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz" + + inherit autotools gettext + </literallayout> + </para> + + <para> + The variable + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</ulink></filename> + is used to track source license changes as described in the + "<ulink url='&YOCTO_DOCS_REF_URL;#usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</ulink>" section. + You can quickly create Autotool-based recipes in a manner similar to the previous example. + </para> + </section> + + <section id='new-recipe-makefile-based-package'> + <title>Makefile-Based Package</title> + + <para> + Applications that use GNU <filename>make</filename> also require a recipe that has + the source archive listed in + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>. + You do not need to add a <filename>do_compile</filename> step since by default BitBake + starts the <filename>make</filename> command to compile the application. + If you need additional <filename>make</filename> options, you should store them in the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE'>EXTRA_OEMAKE</ulink></filename> + variable. + BitBake passes these options into the GNU <filename>make</filename> invocation. + Note that a <filename>do_install</filename> task is still required. + Otherwise, BitBake runs an empty <filename>do_install</filename> task by default. + </para> + + <para> + Some applications might require extra parameters to be passed to the compiler. + For example, the application might need an additional header path. + You can accomplish this by adding to the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CFLAGS'>CFLAGS</ulink></filename> variable. + The following example shows this: + <literallayout class='monospaced'> + CFLAGS_prepend = "-I ${S}/include " + </literallayout> + </para> + + <para> + In the following example, <filename>mtd-utils</filename> is a makefile-based package: + <literallayout class='monospaced'> + SUMMARY = "Tools for managing memory technology devices" + SECTION = "base" + DEPENDS = "zlib lzo e2fsprogs util-linux" + HOMEPAGE = "http://www.linux-mtd.infradead.org/" + LICENSE = "GPLv2+" + LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \ + file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c" + + # Use the latest version at 26 Oct, 2013 + SRCREV = "9f107132a6a073cce37434ca9cda6917dd8d866b" + SRC_URI = "git://git.infradead.org/mtd-utils.git \ + file://add-exclusion-to-mkfs-jffs2-git-2.patch \ + " + + PV = "1.5.1+git${SRCPV}" + + S = "${WORKDIR}/git/" + + EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'" + + do_install () { + oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} INCLUDEDIR=${includedir} + } + + PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs mtd-utils-misc" + + FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2 ${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool" + FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*" + FILES_mtd-utils-misc = "${sbindir}/nftl* ${sbindir}/ftl* ${sbindir}/rfd* ${sbindir}/doc* ${sbindir}/serve_image ${sbindir}/recv_image" + + PARALLEL_MAKE = "" + + BBCLASSEXTEND = "native" + </literallayout> + </para> + </section> + + <section id='splitting-an-application-into-multiple-packages'> + <title>Splitting an Application into Multiple Packages</title> + + <para> + You can use the variables + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> and + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'>FILES</ulink></filename> + to split an application into multiple packages. + </para> + + <para> + Following is an example that uses the <filename>libxpm</filename> recipe. + By default, this recipe generates a single package that contains the library along + with a few binaries. + You can modify the recipe to split the binaries into separate packages: + <literallayout class='monospaced'> + require xorg-lib-common.inc + + SUMMARY = "Xpm: X Pixmap extension library" + LICENSE = "BSD" + LIC_FILES_CHKSUM = "file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7" + DEPENDS += "libxext libsm libxt" + PE = "1" + + XORG_PN = "libXpm" + + PACKAGES =+ "sxpm cxpm" + FILES_cxpm = "${bindir}/cxpm" + FILES_sxpm = "${bindir}/sxpm" + </literallayout> + </para> + + <para> + In the previous example, we want to ship the <filename>sxpm</filename> + and <filename>cxpm</filename> binaries in separate packages. + Since <filename>bindir</filename> would be packaged into the main + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename> + package by default, we prepend the <filename>PACKAGES</filename> + variable so additional package names are added to the start of list. + This results in the extra <filename>FILES_*</filename> + variables then containing information that define which files and + directories go into which packages. + Files included by earlier packages are skipped by latter packages. + Thus, the main <filename>PN</filename> package + does not include the above listed files. + </para> + </section> + + <section id='packaging-externally-produced-binaries'> + <title>Packaging Externally Produced Binaries</title> + + <para> + Sometimes, you need to add pre-compiled binaries to an + image. + For example, suppose that binaries for proprietary code + exist, which are created by a particular division of a + company. + Your part of the company needs to use those binaries as + part of an image that you are building using the + OpenEmbedded build system. + Since you only have the binaries and not the source code, + you cannot use a typical recipe that expects to fetch the + source specified in + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + and then compile it. + </para> + + <para> + One method is to package the binaries and then install them + as part of the image. + Generally, it is not a good idea to package binaries + since, among other things, it can hinder the ability to + reproduce builds and could lead to compatibility problems + with ABI in the future. + However, sometimes you have no choice. + </para> + + <para> + The easiest solution is to create a recipe that uses + the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-bin-package'><filename>bin_package</filename></ulink> + class and to be sure that you are using default locations + for build artifacts. + In most cases, the <filename>bin_package</filename> class + handles "skipping" the configure and compile steps as well + as sets things up to grab packages from the appropriate + area. + In particular, this class sets <filename>noexec</filename> + on both the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> + and + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink> + tasks, sets + <filename>FILES_${PN}</filename> to "/" so that it picks + up all files, and sets up a + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> + task, which effectively copies all files from + <filename>${S}</filename> to <filename>${D}</filename>. + The <filename>bin_package</filename> class works well when + the files extracted into <filename>${S}</filename> are + already laid out in the way they should be laid out + on the target. + For more information on these variables, see the + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>, + and + <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink> + variables in the Yocto Project Reference Manual's variable + glossary. + </para> + + <para> + If you can't use the <filename>bin_package</filename> + class, you need to be sure you are doing the following: + <itemizedlist> + <listitem><para>Create a recipe where the + <filename>do_configure</filename> and + <filename>do_compile</filename> tasks do nothing: + <literallayout class='monospaced'> + do_configure[noexec] = "1" + do_compile[noexec] = "1" + </literallayout> + Alternatively, you can make these tasks an empty + function. + </para></listitem> + <listitem><para>Make sure your + <filename>do_install</filename> task installs the + binaries appropriately. + </para></listitem> + <listitem><para>Ensure that you set up + <filename>FILES</filename> (usually + <filename>FILES_${PN}</filename>) to point to the + files you have installed, which of course depends + on where you have installed them and whether + those files are in different locations than the + defaults. + </para></listitem> + </itemizedlist> + </para> + </section> + </section> + </section> + + <section id="platdev-newmachine"> + <title>Adding a New Machine</title> + + <para> + Adding a new machine to the Yocto Project is a straightforward + process. + This section describes how to add machines that are similar + to those that the Yocto Project already supports. + <note> + Although well within the capabilities of the Yocto Project, + adding a totally new architecture might require + changes to <filename>gcc/glibc</filename> and to the site + information, which is beyond the scope of this manual. + </note> + </para> + + <para> + For a complete example that shows how to add a new machine, + see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>" + section in the Yocto Project Board Support Package (BSP) Developer's Guide. + </para> + + <section id="platdev-newmachine-conffile"> + <title>Adding the Machine Configuration File</title> + + <para> + To add a new machine, you need to add a new machine + configuration file to the layer's + <filename>conf/machine</filename> directory. + This configuration file provides details about the device + you are adding. + </para> + + <para> + The OpenEmbedded build system uses the root name of the + machine configuration file to reference the new machine. + For example, given a machine configuration file named + <filename>crownbay.conf</filename>, the build system + recognizes the machine as "crownbay". + </para> + + <para> + The most important variables you must set in your machine + configuration file or include from a lower-level configuration + file are as follows: + <itemizedlist> + <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_ARCH'>TARGET_ARCH</ulink></filename> + (e.g. "arm")</para></listitem> + <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</ulink>_virtual/kernel</filename> + </para></listitem> + <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'>MACHINE_FEATURES</ulink></filename> + (e.g. "apm screen wifi")</para></listitem> + </itemizedlist> + </para> + + <para> + You might also need these variables: + <itemizedlist> + <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'>SERIAL_CONSOLES</ulink></filename> + (e.g. "115200;ttyS0 115200;ttyS1")</para></listitem> + <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'>KERNEL_IMAGETYPE</ulink></filename> + (e.g. "zImage")</para></listitem> + <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'>IMAGE_FSTYPES</ulink></filename> + (e.g. "tar.gz jffs2")</para></listitem> + </itemizedlist> + </para> + + <para> + You can find full details on these variables in the reference + section. + You can leverage existing machine <filename>.conf</filename> + files from <filename>meta-yocto-bsp/conf/machine/</filename>. + </para> + </section> + + <section id="platdev-newmachine-kernel"> + <title>Adding a Kernel for the Machine</title> + + <para> + The OpenEmbedded build system needs to be able to build a kernel + for the machine. + You need to either create a new kernel recipe for this machine, + or extend an existing kernel recipe. + You can find several kernel recipe examples in the + Source Directory at + <filename>meta/recipes-kernel/linux</filename> + that you can use as references. + </para> + + <para> + If you are creating a new kernel recipe, normal recipe-writing + rules apply for setting up a + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>. + Thus, you need to specify any necessary patches and set + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename> + to point at the source code. + You need to create a <filename>do_configure</filename> task that + configures the unpacked kernel with a + <filename>defconfig</filename> file. + You can do this by using a <filename>make defconfig</filename> + command or, more commonly, by copying in a suitable + <filename>defconfig</filename> file and then running + <filename>make oldconfig</filename>. + By making use of <filename>inherit kernel</filename> and + potentially some of the <filename>linux-*.inc</filename> files, + most other functionality is centralized and the defaults of the + class normally work well. + </para> + + <para> + If you are extending an existing kernel recipe, it is usually + a matter of adding a suitable <filename>defconfig</filename> + file. + The file needs to be added into a location similar to + <filename>defconfig</filename> files used for other machines + in a given kernel recipe. + A possible way to do this is by listing the file in the + <filename>SRC_URI</filename> and adding the machine to the + expression in + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'>COMPATIBLE_MACHINE</ulink></filename>: + <literallayout class='monospaced'> + COMPATIBLE_MACHINE = '(qemux86|qemumips)' + </literallayout> + For more information on <filename>defconfig</filename> files, + see the + "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>" + section in the Yocto Project Linux Kernel Development Manual. + </para> + </section> + + <section id="platdev-newmachine-formfactor"> + <title>Adding a Formfactor Configuration File</title> + + <para> + A formfactor configuration file provides information about the + target hardware for which the image is being built and information that + the build system cannot obtain from other sources such as the kernel. + Some examples of information contained in a formfactor configuration file include + framebuffer orientation, whether or not the system has a keyboard, + the positioning of the keyboard in relation to the screen, and + the screen resolution. + </para> + + <para> + The build system uses reasonable defaults in most cases. + However, if customization is + necessary, you need to create a <filename>machconfig</filename> file + in the <filename>meta/recipes-bsp/formfactor/files</filename> + directory. + This directory contains directories for specific machines such as + <filename>qemuarm</filename> and <filename>qemux86</filename>. + For information about the settings available and the defaults, see the + <filename>meta/recipes-bsp/formfactor/files/config</filename> file found in the + same area. + </para> + + <para> + Following is an example for "qemuarm" machine: + <literallayout class='monospaced'> + HAVE_TOUCHSCREEN=1 + HAVE_KEYBOARD=1 + + DISPLAY_CAN_ROTATE=0 + DISPLAY_ORIENTATION=0 + #DISPLAY_WIDTH_PIXELS=640 + #DISPLAY_HEIGHT_PIXELS=480 + #DISPLAY_BPP=16 + DISPLAY_DPI=150 + DISPLAY_SUBPIXEL_ORDER=vrgb + </literallayout> + </para> + </section> + </section> + + <section id="platdev-working-with-libraries"> + <title>Working With Libraries</title> + + <para> + Libraries are an integral part of your system. + This section describes some common practices you might find + helpful when working with libraries to build your system: + <itemizedlist> + <listitem><para><link linkend='including-static-library-files'>How to include static library files</link> + </para></listitem> + <listitem><para><link linkend='combining-multiple-versions-library-files-into-one-image'>How to use the Multilib feature to combine multiple versions of library files into a single image</link> + </para></listitem> + <listitem><para><link linkend='installing-multiple-versions-of-the-same-library'>How to install multiple versions of the same library in parallel on the same system</link> + </para></listitem> + </itemizedlist> + </para> + + <section id='including-static-library-files'> + <title>Including Static Library Files</title> + + <para> + If you are building a library and the library offers static linking, you can control + which static library files (<filename>*.a</filename> files) get included in the + built library. + </para> + + <para> + The <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink> + and <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES_*</filename></ulink> + variables in the + <filename>meta/conf/bitbake.conf</filename> configuration file define how files installed + by the <filename>do_install</filename> task are packaged. + By default, the <filename>PACKAGES</filename> variable includes + <filename>${PN}-staticdev</filename>, which represents all static library files. + <note> + Some previously released versions of the Yocto Project + defined the static library files through + <filename>${PN}-dev</filename>. + </note> + Following is part of the BitBake configuration file, where + you can see how the static library files are defined: + <literallayout class='monospaced'> + PACKAGE_BEFORE_PN ?= "" + PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}" + PACKAGES_DYNAMIC = "^${PN}-locale-.*" + FILES = "" + + FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \ + ${sysconfdir} ${sharedstatedir} ${localstatedir} \ + ${base_bindir}/* ${base_sbindir}/* \ + ${base_libdir}/*${SOLIBS} \ + ${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \ + ${datadir}/${BPN} ${libdir}/${BPN}/* \ + ${datadir}/pixmaps ${datadir}/applications \ + ${datadir}/idl ${datadir}/omf ${datadir}/sounds \ + ${libdir}/bonobo/servers" + + FILES_${PN}-bin = "${bindir}/* ${sbindir}/*" + + FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \ + ${datadir}/gnome/help" + SECTION_${PN}-doc = "doc" + + FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}" + FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \ + ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \ + ${datadir}/aclocal ${base_libdir}/*.o \ + ${libdir}/${BPN}/*.la ${base_libdir}/*.la" + SECTION_${PN}-dev = "devel" + ALLOW_EMPTY_${PN}-dev = "1" + RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})" + + FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a" + SECTION_${PN}-staticdev = "devel" + RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})" + </literallayout> + </para> + </section> + + <section id="combining-multiple-versions-library-files-into-one-image"> + <title>Combining Multiple Versions of Library Files into One Image</title> + + <para> + The build system offers the ability to build libraries with different + target optimizations or architecture formats and combine these together + into one system image. + You can link different binaries in the image + against the different libraries as needed for specific use cases. + This feature is called "Multilib." + </para> + + <para> + An example would be where you have most of a system compiled in 32-bit + mode using 32-bit libraries, but you have something large, like a database + engine, that needs to be a 64-bit application and uses 64-bit libraries. + Multilib allows you to get the best of both 32-bit and 64-bit libraries. + </para> + + <para> + While the Multilib feature is most commonly used for 32 and 64-bit differences, + the approach the build system uses facilitates different target optimizations. + You could compile some binaries to use one set of libraries and other binaries + to use a different set of libraries. + The libraries could differ in architecture, compiler options, or other + optimizations. + </para> + + <para> + Several examples exist in the + <filename>meta-skeleton</filename> layer found in the + <link linkend='source-directory'>Source Directory</link>: + <itemizedlist> + <listitem><para><filename>conf/multilib-example.conf</filename> + configuration file</para></listitem> + <listitem><para><filename>conf/multilib-example2.conf</filename> + configuration file</para></listitem> + <listitem><para><filename>recipes-multilib/images/core-image-multilib-example.bb</filename> + recipe</para></listitem> + </itemizedlist> + </para> + + <section id='preparing-to-use-multilib'> + <title>Preparing to Use Multilib</title> + + <para> + User-specific requirements drive the Multilib feature. + Consequently, there is no one "out-of-the-box" configuration that likely + exists to meet your needs. + </para> + + <para> + In order to enable Multilib, you first need to ensure your recipe is + extended to support multiple libraries. + Many standard recipes are already extended and support multiple libraries. + You can check in the <filename>meta/conf/multilib.conf</filename> + configuration file in the + <link linkend='source-directory'>Source Directory</link> to see how this is + done using the + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></ulink> + variable. + Eventually, all recipes will be covered and this list will + not be needed. + </para> + + <para> + For the most part, the Multilib class extension works automatically to + extend the package name from <filename>${PN}</filename> to + <filename>${MLPREFIX}${PN}</filename>, where <filename>MLPREFIX</filename> + is the particular multilib (e.g. "lib32-" or "lib64-"). + Standard variables such as + <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-RPROVIDES'><filename>RPROVIDES</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>, and + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> + are automatically extended by the system. + If you are extending any manual code in the recipe, you can use the + <filename>${MLPREFIX}</filename> variable to ensure those names are extended + correctly. + This automatic extension code resides in <filename>multilib.bbclass</filename>. + </para> + </section> + + <section id='using-multilib'> + <title>Using Multilib</title> + + <para> + After you have set up the recipes, you need to define the actual + combination of multiple libraries you want to build. + You accomplish this through your <filename>local.conf</filename> + configuration file in the + <link linkend='build-directory'>Build Directory</link>. + An example configuration would be as follows: + <literallayout class='monospaced'> + MACHINE = "qemux86-64" + require conf/multilib.conf + MULTILIBS = "multilib:lib32" + DEFAULTTUNE_virtclass-multilib-lib32 = "x86" + IMAGE_INSTALL_append = " lib32-glib-2.0" + </literallayout> + This example enables an + additional library named <filename>lib32</filename> alongside the + normal target packages. + When combining these "lib32" alternatives, the example uses "x86" for tuning. + For information on this particular tuning, see + <filename>meta/conf/machine/include/ia32/arch-ia32.inc</filename>. + </para> + + <para> + The example then includes <filename>lib32-glib-2.0</filename> + in all the images, which illustrates one method of including a + multiple library dependency. + You can use a normal image build to include this dependency, + for example: + <literallayout class='monospaced'> + $ bitbake core-image-sato + </literallayout> + You can also build Multilib packages specifically with a command like this: + <literallayout class='monospaced'> + $ bitbake lib32-glib-2.0 + </literallayout> + </para> + </section> + + <section id='additional-implementation-details'> + <title>Additional Implementation Details</title> + + <para> + Generic implementation details as well as details that are + specific to package management systems exist. + Following are implementation details that exist regardless + of the package management system: + <itemizedlist> + <listitem><para>The typical convention used for the + class extension code as used by + Multilib assumes that all package names specified + in + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink> + that contain <filename>${PN}</filename> have + <filename>${PN}</filename> at the start of the name. + When that convention is not followed and + <filename>${PN}</filename> appears at + the middle or the end of a name, problems occur. + </para></listitem> + <listitem><para>The + <ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_VENDOR'><filename>TARGET_VENDOR</filename></ulink> + value under Multilib will be extended to + "-<replaceable>vendor</replaceable>ml<replaceable>multilib</replaceable>" + (e.g. "-pokymllib32" for a "lib32" Multilib with + Poky). + The reason for this slightly unwieldy contraction + is that any "-" characters in the vendor + string presently break Autoconf's + <filename>config.sub</filename>, and + other separators are problematic for different + reasons. + </para></listitem> + </itemizedlist> + </para> +' + <para> + For the RPM Package Management System, the following implementation details + exist: + <itemizedlist> + <listitem><para>A unique architecture is defined for the Multilib packages, + along with creating a unique deploy folder under + <filename>tmp/deploy/rpm</filename> in the + <link linkend='build-directory'>Build Directory</link>. + For example, consider <filename>lib32</filename> in a + <filename>qemux86-64</filename> image. + The possible architectures in the system are "all", "qemux86_64", + "lib32_qemux86_64", and "lib32_x86".</para></listitem> + <listitem><para>The <filename>${MLPREFIX}</filename> variable is stripped from + <filename>${PN}</filename> during RPM packaging. + The naming for a normal RPM package and a Multilib RPM package in a + <filename>qemux86-64</filename> system resolves to something similar to + <filename>bash-4.1-r2.x86_64.rpm</filename> and + <filename>bash-4.1.r2.lib32_x86.rpm</filename>, respectively. + </para></listitem> + <listitem><para>When installing a Multilib image, the RPM backend first + installs the base image and then installs the Multilib libraries. + </para></listitem> + <listitem><para>The build system relies on RPM to resolve the identical files in the + two (or more) Multilib packages.</para></listitem> + </itemizedlist> + </para> + + <para> + For the IPK Package Management System, the following implementation details exist: + <itemizedlist> + <listitem><para>The <filename>${MLPREFIX}</filename> is not stripped from + <filename>${PN}</filename> during IPK packaging. + The naming for a normal RPM package and a Multilib IPK package in a + <filename>qemux86-64</filename> system resolves to something like + <filename>bash_4.1-r2.x86_64.ipk</filename> and + <filename>lib32-bash_4.1-rw_x86.ipk</filename>, respectively. + </para></listitem> + <listitem><para>The IPK deploy folder is not modified with + <filename>${MLPREFIX}</filename> because packages with and without + the Multilib feature can exist in the same folder due to the + <filename>${PN}</filename> differences.</para></listitem> + <listitem><para>IPK defines a sanity check for Multilib installation + using certain rules for file comparison, overridden, etc. + </para></listitem> + </itemizedlist> + </para> + </section> + </section> + + <section id='installing-multiple-versions-of-the-same-library'> + <title>Installing Multiple Versions of the Same Library</title> + + <para> + Situations can exist where you need to install and use + multiple versions of the same library on the same system + at the same time. + These situations almost always exist when a library API + changes and you have multiple pieces of software that + depend on the separate versions of the library. + To accommodate these situations, you can install multiple + versions of the same library in parallel on the same system. + </para> + + <para> + The process is straightforward as long as the libraries use + proper versioning. + With properly versioned libraries, all you need to do to + individually specify the libraries is create separate, + appropriately named recipes where the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> part of the + name includes a portion that differentiates each library version + (e.g.the major part of the version number). + Thus, instead of having a single recipe that loads one version + of a library (e.g. <filename>clutter</filename>), you provide + multiple recipes that result in different versions + of the libraries you want. + As an example, the following two recipes would allow the + two separate versions of the <filename>clutter</filename> + library to co-exist on the same system: + <literallayout class='monospaced'> + clutter-1.6_1.6.20.bb + clutter-1.8_1.8.4.bb + </literallayout> + Additionally, if you have other recipes that depend on a given + library, you need to use the + <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> + variable to create the dependency. + Continuing with the same example, if you want to have a recipe + depend on the 1.8 version of the <filename>clutter</filename> + library, use the following in your recipe: + <literallayout class='monospaced'> + DEPENDS = "clutter-1.8" + </literallayout> + </para> + </section> + </section> + + <section id='dev-optionally-using-an-external-toolchain'> + <title>Optionally Using an External Toolchain</title> + + <para> + You might want to use an external toolchain as part of your + development. + If this is the case, the fundamental steps you need to accomplish + are as follows: + <itemizedlist> + <listitem><para> + Understand where the installed toolchain resides. + For cases where you need to build the external toolchain, + you would need to take separate steps to build and install + the toolchain. + </para></listitem> + <listitem><para> + Make sure you add the layer that contains the toolchain to + your <filename>bblayers.conf</filename> file through the + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink> + variable. + </para></listitem> + <listitem><para> + Set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNAL_TOOLCHAIN'><filename>EXTERNAL_TOOLCHAIN</filename></ulink> + variable in your <filename>local.conf</filename> file + to the location in which you installed the toolchain. + </para></listitem> + </itemizedlist> + A good example of an external toolchain used with the Yocto Project + is <trademark class='registered'>Mentor Graphics</trademark> + Sourcery G++ Toolchain. + You can see information on how to use that particular layer in the + <filename>README</filename> file at + <ulink url='http://github.com/MentorEmbedded/meta-sourcery/'></ulink>. + You can find further information by reading about the + <ulink url='&YOCTO_DOCS_REF_URL;#var-TCMODE'><filename>TCMODE</filename></ulink> + variable in the Yocto Project Reference Manual's variable glossary. + </para> + </section> + + <section id='creating-partitioned-images'> + <title>Creating Partitioned Images</title> + + <para> + Creating an image for a particular hardware target using the + OpenEmbedded build system does not necessarily mean you can boot + that image as is on your device. + Physical devices accept and boot images in various ways depending + on the specifics of the device. + Usually, information about the hardware can tell you what image + format the device requires. + Should your device require multiple partitions on an SD card, flash, + or an HDD, you can use the OpenEmbedded Image Creator, + <filename>wic</filename>, to create the properly partitioned image. + </para> + + <para> + The <filename>wic</filename> command generates partitioned images + from existing OpenEmbedded build artifacts. + Image generation is driven by partitioning commands contained + in an Openembedded kickstart file (<filename>.wks</filename>) + specified either directly on the command line or as one of a + selection of canned <filename>.wks</filename> files as shown + with the <filename>wic list images</filename> command in the + "<link linkend='using-a-provided-kickstart_file'>Using an Existing Kickstart File</link>" + section. + When applied to a given set of build artifacts, the result is an + image or set of images that can be directly written onto media and + used on a particular system. + </para> + + <para> + The <filename>wic</filename> command and the infrastructure + it is based on is by definition incomplete. + Its purpose is to allow the generation of customized images, + and as such was designed to be completely extensible through a + plugin interface. + See the + "<link linkend='openembedded-kickstart-plugins'>Plugins</link>" + section for information on these plugins. + </para> + + <para> + This section provides some background information on + <filename>wic</filename>, describes what you need to have in + place to run the tool, provides instruction on how to use + <filename>wic</filename>, and provides several examples. + </para> + + <section id='wic-background'> + <title>Background</title> + + <para> + This section provides some background on the + <filename>wic</filename> utility. + While none of this information is required to use + <filename>wic</filename>, you might find it interesting. + <itemizedlist> + <listitem><para> + The name "wic" is derived from OpenEmbedded + Image Creator (oeic). + The "oe" diphthong in "oeic" was promoted to the + letter "w", because "oeic" is both difficult to remember and + pronounce.</para></listitem> + <listitem><para> + <filename>wic</filename> is loosely based on the + Meego Image Creator (<filename>mic</filename>) + framework. + The <filename>wic</filename> implementation has been + heavily modified to make direct use of OpenEmbedded + build artifacts instead of package installation and + configuration, which are already incorporated within + the OpenEmbedded artifacts.</para></listitem> + <listitem><para> + <filename>wic</filename> is a completely independent + standalone utility that initially provides + easier-to-use and more flexible replacements for a + couple bits of existing functionality in OE Core's + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-image-live'><filename>image-live</filename></ulink> + class and <filename>mkefidisk.sh</filename> script. + The difference between + <filename>wic</filename> and those examples is + that with <filename>wic</filename> the + functionality of those scripts is implemented + by a general-purpose partitioning language, which is + based on Redhat kickstart syntax.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id='wic-requirements'> + <title>Requirements</title> + + <para> + In order to use the <filename>wic</filename> utility + with the OpenEmbedded Build system, your system needs + to meet the following requirements: + <itemizedlist> + <listitem><para>The Linux distribution on your + development host must support the Yocto Project. + See the + "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>" + section in the Yocto Project Reference Manual for this + list of distributions.</para></listitem> + <listitem><para> + The standard system utilities, such as + <filename>cp</filename>, must be installed on your + development host system. + </para></listitem> + <listitem><para> + You need to have the build artifacts already + available, which typically means that you must + have already created an image using the + Openembedded build system (e.g. + <filename>core-image-minimal</filename>). + While it might seem redundant to generate an image in + order to create an image using + <filename>wic</filename>, the current version of + <filename>wic</filename> requires the artifacts + in the form generated by the build system. + </para></listitem> + <listitem><para> + You must build several native tools, which are tools + built to run on the build system: + <literallayout class='monospaced'> + $ bitbake parted-native dosfstools-native mtools-native + </literallayout> + </para></listitem> + <listitem><para> + You must have sourced one of the build environment + setup scripts (i.e. + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>) + found in the + <link linkend='build-directory'>Build Directory</link>. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='wic-getting-help'> + <title>Getting Help</title> + + <para> + You can get general help for the <filename>wic</filename> + by entering the <filename>wic</filename> command by itself + or by entering the command with a help argument as follows: + <literallayout class='monospaced'> + $ wic -h + $ wic --help + </literallayout> + </para> + + <para> + Currently, <filename>wic</filename> supports two commands: + <filename>create</filename> and <filename>list</filename>. + You can get help for these commands as follows: + <literallayout class='monospaced'> + $ wic help <replaceable>command</replaceable> + </literallayout> + </para> + + <para> + You can also get detailed help on a number of topics + from the help system. + The output of <filename>wic --help</filename> + displays a list of available help + topics under a "Help topics" heading. + You can have the help system display the help text for + a given topic by prefacing the topic with + <filename>wic help</filename>: + <literallayout class='monospaced'> + $ wic help <replaceable>help_topic</replaceable> + </literallayout> + </para> + + <para> + You can find out more about the images + <filename>wic</filename> creates using the existing + kickstart files with the following form of the command: + <literallayout class='monospaced'> + $ wic list <replaceable>image</replaceable> help + </literallayout> + where <filename><replaceable>image</replaceable></filename> is either + <filename>directdisk</filename> or + <filename>mkefidisk</filename>. + </para> + </section> + + <section id='operational-modes'> + <title>Operational Modes</title> + + <para> + You can use <filename>wic</filename> in two different + modes, depending on how much control you need for + specifying the Openembedded build artifacts that are + used for creating the image: Raw and Cooked: + <itemizedlist> + <listitem><para><emphasis>Raw Mode:</emphasis> + You explicitly specify build artifacts through + command-line arguments.</para></listitem> + <listitem><para><emphasis>Cooked Mode:</emphasis> + The current + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + setting and image name are used to automatically locate + and provide the build artifacts.</para></listitem> + </itemizedlist> + </para> + + <para> + Regardless of the mode you use, you need to have the build + artifacts ready and available. + Additionally, the environment must be set up using the + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink> + script found in the + <link linkend='build-directory'>Build Directory</link>. + </para> + + <section id='raw-mode'> + <title>Raw Mode</title> + + <para> + The general form of the 'wic' command in raw mode is: + <literallayout class='monospaced'> + $ wic create <replaceable>image_name</replaceable>.wks [<replaceable>options</replaceable>] [...] + + Where: + + <replaceable>image_name</replaceable>.wks + An OpenEmbedded kickstart file. You can provide + your own custom file or use a file from a set of + existing files as described by further options. + + -o <replaceable>OUTDIR</replaceable>, --outdir=<replaceable>OUTDIR</replaceable> + The name of a directory in which to create image. + + -i <replaceable>PROPERTIES_FILE</replaceable>, --infile=<replaceable>PROPERTIES_FILE</replaceable> + The name of a file containing the values for image + properties as a JSON file. + + -e <replaceable>IMAGE_NAME</replaceable>, --image-name=<replaceable>IMAGE_NAME</replaceable> + The name of the image from which to use the artifacts + (e.g. <filename>core-image-sato</filename>). + + -r <replaceable>ROOTFS_DIR</replaceable>, --rootfs-dir=<replaceable>ROOTFS_DIR</replaceable> + The path to the <filename>/rootfs</filename> directory to use as the + <filename>.wks</filename> rootfs source. + + -b <replaceable>BOOTIMG_DIR</replaceable>, --bootimg-dir=<replaceable>BOOTIMG_DIR</replaceable> + The path to the directory containing the boot artifacts + (e.g. <filename>/EFI</filename> or <filename>/syslinux</filename>) to use as the <filename>.wks</filename> bootimg + source. + + -k <replaceable>KERNEL_DIR</replaceable>, --kernel-dir=<replaceable>KERNEL_DIR</replaceable> + The path to the directory containing the kernel to use + in the <filename>.wks</filename> boot image. + + -n <replaceable>NATIVE_SYSROOT</replaceable>, --native-sysroot=<replaceable>NATIVE_SYSROOT</replaceable> + The path to the native sysroot containing the tools to use + to build the image. + + -s, --skip-build-check + Skips the build check. + + -D, --debug + Output debug information. + </literallayout> + <note> + You do not need root privileges to run + <filename>wic</filename>. + In fact, you should not run as root when using the + utility. + </note> + </para> + </section> + + <section id='cooked-mode'> + <title>Cooked Mode</title> + + <para> + The general form of the <filename>wic</filename> command + using Cooked Mode is: + <literallayout class='monospaced'> + $ wic create <replaceable>kickstart_file</replaceable> -e <replaceable>image_name</replaceable> + + Where: + + <replaceable>kickstart_file</replaceable> + An OpenEmbedded kickstart file. You can provide your own + custom file or supplied file. + + <replaceable>image_name</replaceable> + Specifies the image built using the OpenEmbedded build + system. + </literallayout> + This form is the simplest and most user-friendly, as it + does not require specifying all individual parameters. + All you need to provide is your own + <filename>.wks</filename> file or one provided with the + release. + </para> + </section> + </section> + + <section id='using-a-provided-kickstart_file'> + <title>Using an Existing Kickstart File</title> + + <para> + If you do not want to create your own + <filename>.wks</filename> file, you can use an existing + file provided by the <filename>wic</filename> installation. + Use the following command to list the available files: + <literallayout class='monospaced'> + $ wic list images + directdisk Create a 'pcbios' direct disk image + mkefidisk Create an EFI disk image + </literallayout> + When you use an existing file, you do not have to use the + <filename>.wks</filename> extension. + Here is an example in Raw Mode that uses the + <filename>directdisk</filename> file: + <literallayout class='monospaced'> + $ wic create directdisk -r <replaceable>rootfs_dir</replaceable> -b <replaceable>bootimg_dir</replaceable> \ + -k <replaceable>kernel_dir</replaceable> -n <replaceable>native_sysroot</replaceable> + </literallayout> + </para> + + <para> + Here are the actual partition language commands + used in the <filename>mkefidisk.wks</filename> file to generate + an image: + <literallayout class='monospaced'> + # short-description: Create an EFI disk image + # long-description: Creates a partitioned EFI disk image that the user + # can directly dd to boot media. + + part /boot --source bootimg-efi --ondisk sda --label msdos --active --align 1024 + + part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024 + + part swap --ondisk sda --size 44 --label swap1 --fstype=swap + + bootloader --timeout=10 --append="rootwait rootfstype=ext3 console=ttyPCH0,115200 console=tty0 vmalloc=256MB snd-hda-intel.enable_msi=0" + </literallayout> + </para> + </section> + + <section id='wic-usage-examples'> + <title>Examples</title> + + <para> + This section provides several examples that show how to use + the <filename>wic</filename> utility. + All the examples assume the list of requirements in the + "<link linkend='wic-requirements'>Requirements</link>" section + have been met. + The examples assume the previously generated image is + <filename>core-image-minimal</filename>. + </para> + + <section id='generate-an-image-using-a-provided-kickstart-file'> + <title>Generate an Image using an Existing Kickstart File</title> + + <para> + This example runs in Cooked Mode and uses the + <filename>mkefidisk</filename> kickstart file: + <literallayout class='monospaced'> + $ wic create mkefidisk -e core-image-minimal + Checking basic build environment... + Done. + + Creating image(s)... + + Info: The new image(s) can be found here: + /var/tmp/wic/build/mkefidisk-201310230946-sda.direct + + The following build artifacts were used to create the image(s): + ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/rootfs + BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/hddimg + KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel + NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux + + + The image(s) were created using OE kickstart file: + /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/mkefidisk.wks + </literallayout> + This example shows the easiest way to create an image + by running in Cooked Mode and using the + <filename>-e</filename> option with an existing kickstart + file. + All that is necessary is to specify the image used to + generate the artifacts. + Your <filename>local.conf</filename> needs to have the + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + variable set to the machine you are using, which is + "minnow" in this example. + </para> + + <para> + The output specifies the exact image created as well as + where it was created. + The output also names the artifacts used and the exact + <filename>.wks</filename> script that was used to generate + the image. + <note> + You should always verify the details provided in the + output to make sure that the image was indeed created + exactly as expected. + </note> + </para> + + <para> + Continuing with the example, you can now directly + <filename>dd</filename> the image to a USB stick, or + whatever media for which you built your image, + and boot the resulting media: + <literallayout class='monospaced'> + $ sudo dd if=/var/tmp/wic/build/mkefidisk-201310230946-sda.direct of=/dev/sdb + [sudo] password for trz: + 182274+0 records in + 182274+0 records out + 93324288 bytes (93 MB) copied, 14.4777 s, 6.4 MB/s + [trz@empanada ~]$ sudo eject /dev/sdb + </literallayout> + </para> + </section> + + <section id='using-a-modified-kickstart-file'> + <title>Using a Modified Kickstart File</title> + + <para> + Because <filename>wic</filename> image creation is driven + by the kickstart file, it is easy to affect image creation + by changing the parameters in the file. + This next example demonstrates that through modification + of the <filename>directdisk</filename> kickstart file. + </para> + + <para> + As mentioned earlier, you can use the command + <filename>wic list images</filename> to show the list + of existing kickstart files. + The directory in which these files reside is + <filename>scripts/lib/image/canned-wks/</filename> + located in the + <link linkend='source-directory'>Source Directory</link>. + Because the available files reside in this directory, you + can create and add your own custom files to the directory. + Subsequent use of the <filename>wic list images</filename> + command would then include your kickstart files. + </para> + + <para> + In this example, the existing + <filename>directdisk</filename> file already does most + of what is needed. + However, for the hardware in this example, the image will + need to boot from <filename>sdb</filename> instead of + <filename>sda</filename>, which is what the + <filename>directdisk</filename> kickstart file uses. + </para> + + <para> + The example begins by making a copy of the + <filename>directdisk.wks</filename> file in the + <filename>scripts/lib/image/canned-wks</filename> + directory and then changing the lines that specify the + target disk from which to boot. + <literallayout class='monospaced'> + $ cp /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks \ + /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisksdb.wks + </literallayout> + Next, the example modifies the + <filename>directdisksdb.wks</filename> file and changes all + instances of "<filename>--ondisk sda</filename>" + to "<filename>--ondisk sdb</filename>". + The example changes the following two lines and leaves the + remaining lines untouched: + <literallayout class='monospaced'> + part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024 + part / --source rootfs --ondisk sdb --fstype=ext3 --label platform --align 1024 + </literallayout> + Once the lines are changed, the example generates the + <filename>directdisksdb</filename> image. + The command points the process at the + <filename>core-image-minimal</filename> artifacts for the + Next Unit of Computing (nuc) + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + the <filename>local.conf</filename>. + <literallayout class='monospaced'> + $ wic create directdisksdb -e core-image-minimal + Checking basic build environment... + Done. + + Creating image(s)... + + Info: The new image(s) can be found here: + /var/tmp/wic/build/directdisksdb-201310231131-sdb.direct + + The following build artifacts were used to create the image(s): + ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/nuc-poky-linux/core-image-minimal/1.0-r0/rootfs + BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/nuc/usr/share + KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/nuc/usr/src/kernel + NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux + + + The image(s) were created using OE kickstart file: + /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisksdb.wks + </literallayout> + Continuing with the example, you can now directly + <filename>dd</filename> the image to a USB stick, or + whatever media for which you built your image, + and boot the resulting media: + <literallayout class='monospaced'> + $ sudo dd if=/var/tmp/wic/build/directdisksdb-201310231131-sdb.direct of=/dev/sdb + 86018+0 records in + 86018+0 records out + 44041216 bytes (44 MB) copied, 13.0734 s, 3.4 MB/s + [trz@empanada tmp]$ sudo eject /dev/sdb + </literallayout> + </para> + </section> + + <section id='creating-an-image-based-on-core-image-minimal-and-crownbay-noemgd'> + <title>Creating an Image Based on <filename>core-image-minimal</filename> and <filename>crownbay-noemgd</filename></title> + + <para> + This example creates an image based on + <filename>core-image-minimal</filename> and a + <filename>crownbay-noemgd</filename> + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + that works right out of the box. + <literallayout class='monospaced'> + $ wic create directdisk -e core-image-minimal + + Checking basic build environment... + Done. + + Creating image(s)... + + Info: The new image(s) can be found here: + /var/tmp/wic/build/directdisk-201309252350-sda.direct + + The following build artifacts were used to create the image(s): + + ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs + BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share + KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel + NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel + + The image(s) were created using OE kickstart file: + /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks + </literallayout> + </para> + </section> + + <section id='using-a-modified-kickstart-file-and-running-in-raw-mode'> + <title>Using a Modified Kickstart File and Running in Raw Mode</title> + + <para> + This next example manually specifies each build artifact + (runs in Raw Mode) and uses a modified kickstart file. + The example also uses the <filename>-o</filename> option + to cause <filename>wic</filename> to create the output + somewhere other than the default + <filename>/var/tmp/wic</filename> directory: + <literallayout class='monospaced'> + $ wic create ~/test.wks -o /home/trz/testwic --rootfs-dir \ + /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs \ + --bootimg-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share \ + --kernel-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel \ + --native-sysroot /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux + + Creating image(s)... + + Info: The new image(s) can be found here: + /home/trz/testwic/build/test-201309260032-sda.direct + + The following build artifacts were used to create the image(s): + + ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs + BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share + KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel + NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel + + The image(s) were created using OE kickstart file: + /home/trz/test.wks + </literallayout> + For this example, + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + did not have to be specified in the + <filename>local.conf</filename> file since the artifact is + manually specified. + </para> + </section> + </section> + + <section id='openembedded-kickstart-plugins'> + <title>Plugins</title> + + <para> + Plugins allow <filename>wic</filename> functionality to + be extended and specialized by users. + This section documents the plugin interface, which is + currently restricted to source plugins. + </para> + + <para> + Source plugins provide a mechanism to customize + various aspects of the image generation process in + <filename>wic</filename>, mainly the contents of + partitions. + The plugins provide a mechanism for mapping values + specified in <filename>.wks</filename> files using the + <filename>--source</filename> keyword to a + particular plugin implementation that populates a + corresponding partition. + </para> + + <para> + A source plugin is created as a subclass of + <filename>SourcePlugin</filename>. + The plugin file containing it is added to + <filename>scripts/lib/wic/plugins/source/</filename> to + make the plugin implementation available to the + <filename>wic</filename> implementation. + For more information, see + <filename>scripts/lib/wic/pluginbase.py</filename>. + </para> + + <para> + Source plugins can also be implemented and added by + external layers. + As such, any plugins found in a + <filename>scripts/lib/wic/plugins/source/</filename> + directory in an external layer are also made + available. + </para> + + <para> + When the <filename>wic</filename> implementation needs + to invoke a partition-specific implementation, it looks + for the plugin that has the same name as the + <filename>--source</filename> parameter given to + that partition. + For example, if the partition is set up as follows: + <literallayout class='monospaced'> + part /boot --source bootimg-pcbios ... + </literallayout> + The methods defined as class members of the plugin + having the matching <filename>bootimg-pcbios.name</filename> + class member are used. + </para> + + <para> + To be more concrete, here is the plugin definition that + matches a + <filename>--source bootimg-pcbios</filename> usage, + along with an example + method called by the <filename>wic</filename> implementation + when it needs to invoke an implementation-specific + partition-preparation function: + <literallayout class='monospaced'> + class BootimgPcbiosPlugin(SourcePlugin): + name = 'bootimg-pcbios' + + @classmethod + def do_prepare_partition(self, part, ...) + </literallayout> + If the subclass itself does not implement a function, a + default version in a superclass is located and + used, which is why all plugins must be derived from + <filename>SourcePlugin</filename>. + </para> + + <para> + The <filename>SourcePlugin</filename> class defines the + following methods, which is the current set of methods + that can be implemented or overridden by + <filename>--source</filename> plugins. + Any methods not implemented by a + <filename>SourcePlugin</filename> subclass inherit the + implementations present in the + <filename>SourcePlugin</filename> class. + For more information, see the + <filename>SourcePlugin</filename> source for details: + </para> + + <para> + <itemizedlist> + <listitem><para><emphasis><filename>do_prepare_partition()</filename>:</emphasis> + Called to do the actual content population for a + partition. + In other words, the method prepares the final + partition image that is incorporated into the + disk image. + </para></listitem> + <listitem><para><emphasis><filename>do_configure_partition()</filename>:</emphasis> + Called before + <filename>do_prepare_partition()</filename>. + This method is typically used to create custom + configuration files for a partition (e.g. syslinux or + grub configuration files). + </para></listitem> + <listitem><para><emphasis><filename>do_install_disk()</filename>:</emphasis> + Called after all partitions have been prepared and + assembled into a disk image. + This method provides a hook to allow finalization of a + disk image, (e.g. writing an MBR). + </para></listitem> + <listitem><para><emphasis><filename>do_stage_partition()</filename>:</emphasis> + Special content-staging hook called before + <filename>do_prepare_partition()</filename>. + This method is normally empty.</para> + <para>Typically, a partition just uses the passed-in + parameters (e.g. the unmodified value of + <filename>bootimg_dir</filename>). + However, in some cases things might need to be + more tailored. + As an example, certain files might additionally + need to be taken from + <filename>bootimg_dir + /boot</filename>. + This hook allows those files to be staged in a + customized fashion. + <note> + <filename>get_bitbake_var()</filename> + allows you to access non-standard variables + that you might want to use for this. + </note> + </para></listitem> + </itemizedlist> + </para> + + <para> + This scheme is extensible. + Adding more hooks is a simple matter of adding more + plugin methods to <filename>SourcePlugin</filename> and + derived classes. + The code that then needs to call the plugin methods uses + <filename>plugin.get_source_plugin_methods()</filename> + to find the method or methods needed by the call. + Retrieval of those methods is accomplished + by filling up a dict with keys + containing the method names of interest. + On success, these will be filled in with the actual + methods. + Please see the <filename>wic</filename> + implementation for examples and details. + </para> + </section> + + <section id='openembedded-kickstart-wks-reference'> + <title>OpenEmbedded Kickstart (.wks) Reference</title> + + <para> + The current <filename>wic</filename> implementation supports + only the basic kickstart partitioning commands: + <filename>partition</filename> (or <filename>part</filename> + for short) and <filename>bootloader</filename>. + <note> + Future updates will implement more commands and options. + If you use anything that is not specifically + supported, results can be unpredictable. + </note> + </para> + + <para> + The following is a list of the commands, their syntax, + and meanings. + The commands are based on the Fedora + kickstart versions but with modifications to + reflect <filename>wic</filename> capabilities. + You can see the original documentation for those commands + at the following links: + <itemizedlist> + <listitem><para> + <ulink url='http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition'>http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition</ulink> + </para></listitem> + <listitem><para> + <ulink url='http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader'>http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader</ulink> + </para></listitem> + </itemizedlist> + </para> + + <section id='command-part-or-partition'> + <title>Command: part or partition</title> + + <para> + Either of these commands create a partition on the system + and uses the following syntax: + <literallayout class='monospaced'> + part [<replaceable>mntpoint</replaceable>] + partition [<replaceable>mntpoint</replaceable>] + </literallayout> + If you do not provide + <replaceable>mntpoint</replaceable>, wic creates a partition + but does not mount it. + </para> + + <para> + The <filename><replaceable>mntpoint</replaceable></filename> + is where the + partition will be mounted and must be of one of the + following forms: + <itemizedlist> + <listitem><para><filename>/<replaceable>path</replaceable></filename>: + For example, <filename>/</filename>, + <filename>/usr</filename>, or + <filename>/home</filename></para></listitem> + <listitem><para><filename>swap</filename>: + The created partition is used as swap space. + </para></listitem> + </itemizedlist> + </para> + + <para> + Specifying a <replaceable>mntpoint</replaceable> causes + the partition to automatically be mounted. + Wic achieves this by adding entries to the filesystem + table (fstab) during image generation. + In order for wic to generate a valid fstab, you must + also provide one of the <filename>--ondrive</filename>, + <filename>--ondisk</filename>, or + <filename>--use-uuid</filename> partition options as part + of the command. + Here is an example using "/" as the mountpoint. + The command uses "--ondisk" to force the partition onto + the <filename>sdb</filename> disk: + <literallayout class='monospaced'> + part / --source rootfs --ondisk sdb --fstype=ext3 --label platform --align 1024 + </literallayout> + </para> + + <para> + Here is a list that describes other supported options you + can use with the <filename>part</filename> and + <filename>partition</filename> commands: + <itemizedlist> + <listitem><para><emphasis><filename>--size</filename>:</emphasis> + The minimum partition size in MBytes. + Specify an integer value such as 500. + Do not append the number with "MB". + You do not need this option if you use + <filename>--source</filename>.</para></listitem> + <listitem><para><emphasis><filename>--source</filename>:</emphasis> + This option is a + <filename>wic</filename>-specific option that + names the source of the data that populates + the partition. + The most common value for this option is + "rootfs", but you can use any value that maps to + a valid source plugin. + For information on the source plugins, see the + "<link linkend='openembedded-kickstart-plugins'>Plugins</link>" + section.</para> + <para>If you use + <filename>--source rootfs</filename>, + <filename>wic</filename> creates a partition as + large as needed and to fill it with the contents of + the root filesystem pointed to by the + <filename>-r</filename> command-line option + or the equivalent rootfs derived from the + <filename>-e</filename> command-line + option. + The filesystem type used to create the + partition is driven by the value of the + <filename>--fstype</filename> option + specified for the partition. + See the entry on + <filename>--fstype</filename> that + follows for more information. + </para> + <para>If you use + <filename>--source <replaceable>plugin-name</replaceable></filename>, + <filename>wic</filename> creates a partition as + large as needed and fills it with the contents of + the partition that is generated by the + specified plugin name using the data pointed + to by the <filename>-r</filename> command-line + option or the equivalent rootfs derived from the + <filename>-e</filename> command-line + option. + Exactly what those contents and filesystem type end + up being are dependent on the given plugin + implementation. + </para> + <para>If you do not use the + <filename>--source</filename> option, the + <filename>wic</filename> command creates an empty + partition. + Consequently, you must use the + <filename>--size</filename> option to specify the + size of the empty partition. + </para></listitem> + <listitem><para><emphasis><filename>--ondisk</filename> or <filename>--ondrive</filename>:</emphasis> + Forces the partition to be created on a particular + disk.</para></listitem> + <listitem><para><emphasis><filename>--fstype</filename>:</emphasis> + Sets the file system type for the partition. + Valid values are: + <itemizedlist> + <listitem><para><filename>ext4</filename> + </para></listitem> + <listitem><para><filename>ext3</filename> + </para></listitem> + <listitem><para><filename>ext2</filename> + </para></listitem> + <listitem><para><filename>btrfs</filename> + </para></listitem> + <listitem><para><filename>squashfs</filename> + </para></listitem> + <listitem><para><filename>swap</filename> + </para></listitem> + </itemizedlist></para></listitem> + <listitem><para><emphasis><filename>--fsoptions</filename>:</emphasis> + Specifies a free-form string of options to be + used when mounting the filesystem. + This string will be copied into the + <filename>/etc/fstab</filename> file of the + installed system and should be enclosed in + quotes. + If not specified, the default string + is "defaults". + </para></listitem> + <listitem><para><emphasis><filename>--label label</filename>:</emphasis> + Specifies the label to give to the filesystem to + be made on the partition. + If the given label is already in use by another + filesystem, a new label is created for the + partition.</para></listitem> + <listitem><para><emphasis><filename>--active</filename>:</emphasis> + Marks the partition as active.</para></listitem> + <listitem><para><emphasis><filename>--align (in KBytes)</filename>:</emphasis> + This option is a <filename>wic</filename>-specific + option that says to start a partition on an + x KBytes boundary.</para></listitem> + <listitem><para><emphasis><filename>--no-table</filename>:</emphasis> + This option is a <filename>wic</filename>-specific + option. + Using the option reserves space for the partition + and causes it to become populated. + However, the partition is not added to the + partition table. + </para></listitem> + <listitem><para><emphasis><filename>--extra-space</filename>:</emphasis> + This option is a <filename>wic</filename>-specific + option that adds extra space after the space + filled by the content of the partition. + The final size can go beyond the size specified + by the <filename>--size</filename> option. + The default value is 10 Mbytes. + </para></listitem> + <listitem><para><emphasis><filename>--overhead-factor</filename>:</emphasis> + This option is a <filename>wic</filename>-specific + option that multiplies the size of the partition by + the option's value. + You must supply a value greater than or equal to + "1". + The default value is "1.3". + </para></listitem> + <listitem><para><emphasis><filename>--part-type</filename>:</emphasis> + This option is a <filename>wic</filename>-specific + option that specifies the partition type globally + unique identifier (GUID) for GPT partitions. + You can find the list of partition type GUIDs + at + <ulink url='http://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs'></ulink>. + </para></listitem> + <listitem><para><emphasis><filename>--use-uuid</filename>:</emphasis> + This option is a <filename>wic</filename>-specific + option that causes <filename>wic</filename> to + generate a random GUID for the partition. + The generated identifier is used in the bootloader + configuration to specify the root partition. + </para></listitem> + <listitem><para><emphasis><filename>--uuid</filename>:</emphasis> + This option is a <filename>wic</filename>-specific + option that specifies the partition UUID. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='command-bootloader'> + <title>Command: bootloader</title> + + <para> + This command specifies how the boot loader should be + configured and supports the following options: + <note> + Bootloader functionality and boot partitions are + implemented by the various + <filename>--source</filename> + plugins that implement bootloader functionality. + The bootloader command essentially provides a means of + modifying bootloader configuration. + </note> + <itemizedlist> + <listitem><para><emphasis><filename>--timeout</filename>:</emphasis> + Specifies the number of seconds before the + bootloader times out and boots the default option. + </para></listitem> + <listitem><para><emphasis><filename>--append</filename>:</emphasis> + Specifies kernel parameters. + These parameters will be added to the syslinux + <filename>APPEND</filename> or + <filename>grub</filename> kernel command line. + </para></listitem> + <listitem><para><emphasis><filename>--configfile</filename>:</emphasis> + Specifies a user-defined configuration file for + the bootloader. + You can provide a full pathname for the file or + a file that exists in the + <filename>canned-wks</filename> folder. + This option overrides all other bootloader options. + </para></listitem> + </itemizedlist> + </para> + </section> + </section> + </section> + + <section id='configuring-the-kernel'> + <title>Configuring the Kernel</title> + + <para> + Configuring the Yocto Project kernel consists of making sure the + <filename>.config</filename> file has all the right information + in it for the image you are building. + You can use the <filename>menuconfig</filename> tool and + configuration fragments to make sure your + <filename>.config</filename> file is just how you need it. + You can also save known configurations in a + <filename>defconfig</filename> file that the build system can use + for kernel configuration. + </para> + + <para> + This section describes how to use <filename>menuconfig</filename>, + create and use configuration fragments, and how to interactively + modify your <filename>.config</filename> file to create the + leanest kernel configuration file possible. + </para> + + <para> + For more information on kernel configuration, see the + "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>" + section in the Yocto Project Linux Kernel Development Manual. + </para> + + <section id='using-menuconfig'> + <title>Using <filename>menuconfig</filename></title> + + <para> + The easiest way to define kernel configurations is to set them through the + <filename>menuconfig</filename> tool. + This tool provides an interactive method with which + to set kernel configurations. + For general information on <filename>menuconfig</filename>, see + <ulink url='http://en.wikipedia.org/wiki/Menuconfig'></ulink>. + </para> + + <para> + To use the <filename>menuconfig</filename> tool in the Yocto Project development + environment, you must launch it using BitBake. + Thus, the environment must be set up using the + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink> + script found in the + <link linkend='build-directory'>Build Directory</link>. + You must also be sure of the state of your build in the + <link linkend='source-directory'>Source Directory</link>. + The following commands run <filename>menuconfig</filename> + assuming the Source Directory's top-level folder is + <filename>~/poky</filename>: + <literallayout class='monospaced'> + $ cd poky + $ source oe-init-build-env + $ bitbake linux-yocto -c kernel_configme -f + $ bitbake linux-yocto -c menuconfig + </literallayout> + Once <filename>menuconfig</filename> comes up, its standard + interface allows you to interactively examine and configure + all the kernel configuration parameters. + After making your changes, simply exit the tool and save your + changes to create an updated version of the + <filename>.config</filename> configuration file. + </para> + + <para> + Consider an example that configures the <filename>linux-yocto-3.14</filename> + kernel. + The OpenEmbedded build system recognizes this kernel as + <filename>linux-yocto</filename>. + Thus, the following commands from the shell in which you previously sourced the + environment initialization script cleans the shared state cache and the + <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink> + directory and then runs <filename>menuconfig</filename>: + <literallayout class='monospaced'> + $ bitbake linux-yocto -c menuconfig + </literallayout> + </para> + + <para> + Once <filename>menuconfig</filename> launches, use the interface + to navigate through the selections to find the configuration settings in + which you are interested. + For example, consider the <filename>CONFIG_SMP</filename> configuration setting. + You can find it at <filename>Processor Type and Features</filename> under + the configuration selection <filename>Symmetric Multi-processing Support</filename>. + After highlighting the selection, use the arrow keys to select or deselect + the setting. + When you are finished with all your selections, exit out and save them. + </para> + + <para> + Saving the selections updates the <filename>.config</filename> configuration file. + This is the file that the OpenEmbedded build system uses to configure the + kernel during the build. + You can find and examine this file in the Build Directory in + <filename>tmp/work/</filename>. + The actual <filename>.config</filename> is located in the area where the + specific kernel is built. + For example, if you were building a Linux Yocto kernel based on the + Linux 3.14 kernel and you were building a QEMU image targeted for + <filename>x86</filename> architecture, the + <filename>.config</filename> file would be located here: + <literallayout class='monospaced'> + poky/build/tmp/work/qemux86-poky-linux/linux-yocto-3.14.11+git1+84f... + ...656ed30-r1/linux-qemux86-standard-build + </literallayout> + <note> + The previous example directory is artificially split and many of the characters + in the actual filename are omitted in order to make it more readable. + Also, depending on the kernel you are using, the exact pathname + for <filename>linux-yocto-3.14...</filename> might differ. + </note> + </para> + + <para> + Within the <filename>.config</filename> file, you can see the kernel settings. + For example, the following entry shows that symmetric multi-processor support + is not set: + <literallayout class='monospaced'> + # CONFIG_SMP is not set + </literallayout> + </para> + + <para> + A good method to isolate changed configurations is to use a combination of the + <filename>menuconfig</filename> tool and simple shell commands. + Before changing configurations with <filename>menuconfig</filename>, copy the + existing <filename>.config</filename> and rename it to something else, + use <filename>menuconfig</filename> to make + as many changes as you want and save them, then compare the renamed configuration + file against the newly created file. + You can use the resulting differences as your base to create configuration fragments + to permanently save in your kernel layer. + <note> + Be sure to make a copy of the <filename>.config</filename> and don't just + rename it. + The build system needs an existing <filename>.config</filename> + from which to work. + </note> + </para> + </section> + + <section id='creating-a-defconfig-file'> + <title>Creating a <filename>defconfig</filename> File</title> + + <para> + A <filename>defconfig</filename> file is simply a + <filename>.config</filename> renamed to "defconfig". + You can use a <filename>defconfig</filename> file + to retain a known set of kernel configurations from which the + OpenEmbedded build system can draw to create the final + <filename>.config</filename> file. + <note> + Out-of-the-box, the Yocto Project never ships a + <filename>defconfig</filename> or + <filename>.config</filename> file. + The OpenEmbedded build system creates the final + <filename>.config</filename> file used to configure the + kernel. + </note> + </para> + + <para> + To create a <filename>defconfig</filename>, start with a + complete, working Linux kernel <filename>.config</filename> + file. + Copy that file to the appropriate + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename> + directory in your layer's + <filename>recipes-kernel/linux</filename> directory, and rename + the copied file to "defconfig". + Then, add the following lines to the linux-yocto + <filename>.bbappend</filename> file in your layer: + <literallayout class='monospaced'> + FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" + SRC_URI += "file://defconfig" + </literallayout> + The + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + tells the build system how to search for the file, while the + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink> + extends the + <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink> + variable (search directories) to include the + <filename>${PN}</filename> directory you created to hold the + configuration changes. + <note> + The build system applies the configurations from the + <filename>defconfig</filename> file before applying any + subsequent configuration fragments. + The final kernel configuration is a combination of the + configurations in the <filename>defconfig</filename> + file and any configuration fragments you provide. + You need to realize that if you have any configuration + fragments, the build system applies these on top of and + after applying the existing defconfig file configurations. + </note> + For more information on configuring the kernel, see the + "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>" + and + "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#generating-configuration-files'>Generating Configuration Files</ulink>" + sections, both in the Yocto Project Linux Kernel Development + Manual. + </para> + </section> + + <section id='creating-config-fragments'> + <title>Creating Configuration Fragments</title> + + <para> + Configuration fragments are simply kernel options that appear in a file + placed where the OpenEmbedded build system can find and apply them. + Syntactically, the configuration statement is identical to what would appear + in the <filename>.config</filename> file, which is in the + <link linkend='build-directory'>Build Directory</link>: + <literallayout class='monospaced'> + tmp/work/<replaceable>arch</replaceable>-poky-linux/linux-yocto-<replaceable>release_specific_string</replaceable>/linux-<replaceable>arch</replaceable>-<replaceable>build_type</replaceable> + </literallayout> + </para> + + <para> + It is simple to create a configuration fragment. + For example, issuing the following from the shell creates a configuration fragment + file named <filename>my_smp.cfg</filename> that enables multi-processor support + within the kernel: + <literallayout class='monospaced'> + $ echo "CONFIG_SMP=y" >> my_smp.cfg + </literallayout> + <note> + All configuration fragment files must use the + <filename>.cfg</filename> extension in order for the + OpenEmbedded build system to recognize them as a + configuration fragment. + </note> + </para> + + <para> + Where do you put your configuration fragment files? + You can place these files in the same area pointed to by + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>. + The OpenEmbedded build system picks up the configuration and + adds it to the kernel's configuration. + For example, suppose you had a set of configuration options + in a file called <filename>myconfig.cfg</filename>. + If you put that file inside a directory named + <filename>linux-yocto</filename> that resides in the same + directory as the kernel's append file and then add a + <filename>SRC_URI</filename> statement such as the following + to the kernel's append file, those configuration options + will be picked up and applied when the kernel is built. + <literallayout class='monospaced'> + SRC_URI += "file://myconfig.cfg" + </literallayout> + </para> + + <para> + As mentioned earlier, you can group related configurations into multiple files and + name them all in the <filename>SRC_URI</filename> statement as well. + For example, you could group separate configurations specifically for Ethernet and graphics + into their own files and add those by using a <filename>SRC_URI</filename> statement like the + following in your append file: + <literallayout class='monospaced'> + SRC_URI += "file://myconfig.cfg \ + file://eth.cfg \ + file://gfx.cfg" + </literallayout> + </para> + </section> + + <section id='fine-tuning-the-kernel-configuration-file'> + <title>Fine-Tuning the Kernel Configuration File</title> + + <para> + You can make sure the <filename>.config</filename> file is as lean or efficient as + possible by reading the output of the kernel configuration fragment audit, + noting any issues, making changes to correct the issues, and then repeating. + </para> + + <para> + As part of the kernel build process, the + <filename>do_kernel_configcheck</filename> task runs. + This task validates the kernel configuration by checking the final + <filename>.config</filename> file against the input files. + During the check, the task produces warning messages for the following + issues: + <itemizedlist> + <listitem><para>Requested options that did not make the final + <filename>.config</filename> file.</para></listitem> + <listitem><para>Configuration items that appear twice in the same + configuration fragment.</para></listitem> + <listitem><para>Configuration items tagged as "required" that were overridden. + </para></listitem> + <listitem><para>A board overrides a non-board specific option.</para></listitem> + <listitem><para>Listed options not valid for the kernel being processed. + In other words, the option does not appear anywhere.</para></listitem> + </itemizedlist> + <note> + The <filename>do_kernel_configcheck</filename> task can + also optionally report if an option is overridden during + processing. + </note> + </para> + + <para> + For each output warning, a message points to the file + that contains a list of the options and a pointer to the + configuration fragment that defines them. + Collectively, the files are the key to streamlining the + configuration. + </para> + + <para> + To streamline the configuration, do the following: + <orderedlist> + <listitem><para>Start with a full configuration that you + know works - it builds and boots successfully. + This configuration file will be your baseline. + </para></listitem> + <listitem><para>Separately run the + <filename>do_kernel_configme</filename> and + <filename>do_kernel_configcheck</filename> tasks. + </para></listitem> + <listitem><para>Take the resulting list of files from the + <filename>do_kernel_configcheck</filename> task + warnings and do the following: + <itemizedlist> + <listitem><para> + Drop values that are redefined in the fragment + but do not change the final + <filename>.config</filename> file. + </para></listitem> + <listitem><para> + Analyze and potentially drop values from the + <filename>.config</filename> file that override + required configurations. + </para></listitem> + <listitem><para> + Analyze and potentially remove non-board + specific options. + </para></listitem> + <listitem><para> + Remove repeated and invalid options. + </para></listitem> + </itemizedlist></para></listitem> + <listitem><para> + After you have worked through the output of the kernel + configuration audit, you can re-run the + <filename>do_kernel_configme</filename> and + <filename>do_kernel_configcheck</filename> tasks to + see the results of your changes. + If you have more issues, you can deal with them as + described in the previous step. + </para></listitem> + </orderedlist> + </para> + + <para> + Iteratively working through steps two through four eventually yields + a minimal, streamlined configuration file. + Once you have the best <filename>.config</filename>, you can build the Linux + Yocto kernel. + </para> + </section> + + <section id='determining-hardware-and-non-hardware-features-for-the-kernel-configuration-audit-phase'> + <title>Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase</title> + + <para> + This section describes part of the kernel configuration audit + phase that most developers can ignore. + During this part of the audit phase, the contents of the final + <filename>.config</filename> file are compared against the + fragments specified by the system. + These fragments can be system fragments, distro fragments, + or user specified configuration elements. + Regardless of their origin, the OpenEmbedded build system + warns the user if a specific option is not included in the + final kernel configuration. + </para> + + <para> + In order to not overwhelm the user with configuration warnings, + by default the system only reports on missing "hardware" + options because a missing hardware option could mean a boot + failure or that important hardware is not available. + </para> + + <para> + To determine whether or not a given option is "hardware" or + "non-hardware", the kernel Metadata contains files that + classify individual or groups of options as either hardware + or non-hardware. + To better show this, consider a situation where the + Yocto Project kernel cache contains the following files: + <literallayout class='monospaced'> + kernel-cache/features/drm-psb/hardware.cfg + kernel-cache/features/kgdb/hardware.cfg + kernel-cache/ktypes/base/hardware.cfg + kernel-cache/bsp/mti-malta32/hardware.cfg + kernel-cache/bsp/fsl-mpc8315e-rdb/hardware.cfg + kernel-cache/bsp/qemu-ppc32/hardware.cfg + kernel-cache/bsp/qemuarma9/hardware.cfg + kernel-cache/bsp/mti-malta64/hardware.cfg + kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg + kernel-cache/bsp/common-pc/hardware.cfg + kernel-cache/bsp/common-pc-64/hardware.cfg + kernel-cache/features/rfkill/non-hardware.cfg + kernel-cache/ktypes/base/non-hardware.cfg + kernel-cache/features/aufs/non-hardware.kcf + kernel-cache/features/ocf/non-hardware.kcf + kernel-cache/ktypes/base/non-hardware.kcf + kernel-cache/ktypes/base/hardware.kcf + kernel-cache/bsp/qemu-ppc32/hardware.kcf + </literallayout> + The following list provides explanations for the various + files: + <itemizedlist> + <listitem><para><filename>hardware.kcf</filename>: + Specifies a list of kernel Kconfig files that contain + hardware options only. + </para></listitem> + <listitem><para><filename>non-hardware.kcf</filename>: + Specifies a list of kernel Kconfig files that contain + non-hardware options only. + </para></listitem> + <listitem><para><filename>hardware.cfg</filename>: + Specifies a list of kernel + <filename>CONFIG_</filename> options that are hardware, + regardless of whether or not they are within a Kconfig + file specified by a hardware or non-hardware + Kconfig file (i.e. <filename>hardware.kcf</filename> or + <filename>non-hardware.kcf</filename>). + </para></listitem> + <listitem><para><filename>non-hardware.cfg</filename>: + Specifies a list of kernel + <filename>CONFIG_</filename> options that are + not hardware, regardless of whether or not they are + within a Kconfig file specified by a hardware or + non-hardware Kconfig file (i.e. + <filename>hardware.kcf</filename> or + <filename>non-hardware.kcf</filename>). + </para></listitem> + </itemizedlist> + Here is a specific example using the + <filename>kernel-cache/bsp/mti-malta32/hardware.cfg</filename>: + <literallayout class='monospaced'> + CONFIG_SERIAL_8250 + CONFIG_SERIAL_8250_CONSOLE + CONFIG_SERIAL_8250_NR_UARTS + CONFIG_SERIAL_8250_PCI + CONFIG_SERIAL_CORE + CONFIG_SERIAL_CORE_CONSOLE + CONFIG_VGA_ARB + </literallayout> + The kernel configuration audit automatically detects these + files (hence the names must be exactly the ones discussed here), + and uses them as inputs when generating warnings about the + final <filename>.config</filename> file. + </para> + + <para> + A user-specified kernel Metadata repository, or recipe space + feature, can use these same files to classify options that are + found within its <filename>.cfg</filename> files as hardware + or non-hardware, to prevent the OpenEmbedded build system from + producing an error or warning when an option is not in the + final <filename>.config</filename> file. + </para> + </section> + </section> + + <section id="patching-the-kernel"> + <title>Patching the Kernel</title> + + <para> + Patching the kernel involves changing or adding configurations to an existing kernel, + changing or adding recipes to the kernel that are needed to support specific hardware features, + or even altering the source code itself. + <note> + You can use the <filename>yocto-kernel</filename> script + found in the <link linkend='source-directory'>Source Directory</link> + under <filename>scripts</filename> to manage kernel patches and configuration. + See the "<ulink url='&YOCTO_DOCS_BSP_URL;#managing-kernel-patches-and-config-items-with-yocto-kernel'>Managing kernel Patches and Config Items with yocto-kernel</ulink>" + section in the Yocto Project Board Support Packages (BSP) Developer's Guide for + more information.</note> + </para> + + <para> + This example creates a simple patch by adding some QEMU emulator console + output at boot time through <filename>printk</filename> statements in the kernel's + <filename>calibrate.c</filename> source code file. + Applying the patch and booting the modified image causes the added + messages to appear on the emulator's console. + </para> + + <para> + The example assumes a clean build exists for the <filename>qemux86</filename> + machine in a + <link linkend='source-directory'>Source Directory</link> + named <filename>poky</filename>. + Furthermore, the <link linkend='build-directory'>Build Directory</link> is + <filename>build</filename> and is located in <filename>poky</filename> and + the kernel is based on the Linux 3.4 kernel. + </para> + + <para> + Also, for more information on patching the kernel, see the + "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#applying-patches'>Applying Patches</ulink>" + section in the Yocto Project Linux Kernel Development Manual. + </para> + + <section id='create-a-layer-for-your-changes'> + <title>Create a Layer for your Changes</title> + + <para> + The first step is to create a layer so you can isolate your + changes. + Rather than use the <filename>yocto-layer</filename> script + to create the layer, this example steps through the process + by hand. + If you want information on the script that creates a general + layer, see the + "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>" + section. + </para> + + <para> + These two commands create a directory you can use for your + layer: + <literallayout class='monospaced'> + $ cd ~/poky + $ mkdir meta-mylayer + </literallayout> + Creating a directory that follows the Yocto Project layer naming + conventions sets up the layer for your changes. + The layer is where you place your configuration files, append + files, and patch files. + To learn more about creating a layer and filling it with the + files you need, see the "<link linkend='understanding-and-creating-layers'>Understanding + and Creating Layers</link>" section. + </para> + </section> + + <section id='finding-the-kernel-source-code'> + <title>Finding the Kernel Source Code</title> + + <para> + Each time you build a kernel image, the kernel source code is fetched + and unpacked into the following directory: + <literallayout class='monospaced'> + ${S}/linux + </literallayout> + See the "<link linkend='finding-the-temporary-source-code'>Finding Temporary Source Code</link>" + section and the + <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> variable + for more information about where source is kept during a build. + </para> + + <para> + For this example, we are going to patch the + <filename>init/calibrate.c</filename> file + by adding some simple console <filename>printk</filename> statements that we can + see when we boot the image using QEMU. + </para> + </section> + + <section id='creating-the-patch'> + <title>Creating the Patch</title> + + <para> + Two methods exist by which you can create the patch: + <link linkend='using-devtool-in-your-workflow'><filename>devtool</filename></link> and + <link linkend='using-a-quilt-workflow'>Quilt</link>. + For kernel patches, the Git workflow is more appropriate. + This section assumes the Git workflow and shows the steps specific to + this example. + <orderedlist> + <listitem><para><emphasis>Change the working directory</emphasis>: + Change to where the kernel source code is before making + your edits to the <filename>calibrate.c</filename> file: + <literallayout class='monospaced'> + $ cd ~/poky/build/tmp/work/qemux86-poky-linux/linux-yocto-${PV}-${PR}/linux + </literallayout> + Because you are working in an established Git repository, + you must be in this directory in order to commit your changes + and create the patch file. + <note>The <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> and + <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> variables + represent the version and revision for the + <filename>linux-yocto</filename> recipe. + The <filename>PV</filename> variable includes the Git meta and machine + hashes, which make the directory name longer than you might + expect. + </note></para></listitem> + <listitem><para><emphasis>Edit the source file</emphasis>: + Edit the <filename>init/calibrate.c</filename> file to have the + following changes: + <literallayout class='monospaced'> + void calibrate_delay(void) + { + unsigned long lpj; + static bool printed; + int this_cpu = smp_processor_id(); + + printk("*************************************\n"); + printk("* *\n"); + printk("* HELLO YOCTO KERNEL *\n"); + printk("* *\n"); + printk("*************************************\n"); + + if (per_cpu(cpu_loops_per_jiffy, this_cpu)) { + . + . + . + </literallayout></para></listitem> + <listitem><para><emphasis>Stage and commit your changes</emphasis>: + These Git commands display the modified file, stage it, and then + commit the file: + <literallayout class='monospaced'> + $ git status + $ git add init/calibrate.c + $ git commit -m "calibrate: Add printk example" + </literallayout></para></listitem> + <listitem><para><emphasis>Generate the patch file</emphasis>: + This Git command creates the a patch file named + <filename>0001-calibrate-Add-printk-example.patch</filename> + in the current directory. + <literallayout class='monospaced'> + $ git format-patch -1 + </literallayout> + </para></listitem> + </orderedlist> + </para> + </section> + + <section id='set-up-your-layer-for-the-build'> + <title>Set Up Your Layer for the Build</title> + + <para>These steps get your layer set up for the build: + <orderedlist> + <listitem><para><emphasis>Create additional structure</emphasis>: + Create the additional layer structure: + <literallayout class='monospaced'> + $ cd ~/poky/meta-mylayer + $ mkdir conf + $ mkdir recipes-kernel + $ mkdir recipes-kernel/linux + $ mkdir recipes-kernel/linux/linux-yocto + </literallayout> + The <filename>conf</filename> directory holds your configuration files, while the + <filename>recipes-kernel</filename> directory holds your append file and + your patch file.</para></listitem> + <listitem><para><emphasis>Create the layer configuration file</emphasis>: + Move to the <filename>meta-mylayer/conf</filename> directory and create + the <filename>layer.conf</filename> file as follows: + <literallayout class='monospaced'> + # We have a conf and classes directory, add to BBPATH + BBPATH .= ":${LAYERDIR}" + + # We have recipes-* directories, add to BBFILES + BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ + ${LAYERDIR}/recipes-*/*/*.bbappend" + + BBFILE_COLLECTIONS += "mylayer" + BBFILE_PATTERN_mylayer = "^${LAYERDIR}/" + BBFILE_PRIORITY_mylayer = "5" + </literallayout> + Notice <filename>mylayer</filename> as part of the last three + statements.</para></listitem> + <listitem><para><emphasis>Create the kernel recipe append file</emphasis>: + Move to the <filename>meta-mylayer/recipes-kernel/linux</filename> directory and create + the <filename>linux-yocto_3.4.bbappend</filename> file as follows: + <literallayout class='monospaced'> + FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" + + SRC_URI += "file://0001-calibrate-Add-printk-example.patch" + </literallayout> + The <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink> + and <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + statements enable the OpenEmbedded build system to find the patch file. + For more information on using append files, see the + "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" + section. + </para></listitem> + <listitem><para><emphasis>Put the patch file in your layer</emphasis>: + Move the <filename>0001-calibrate-Add-printk-example.patch</filename> file to + the <filename>meta-mylayer/recipes-kernel/linux/linux-yocto</filename> + directory.</para></listitem> + </orderedlist> + </para> + </section> + + <section id='set-up-for-the-build'> + <title>Set Up for the Build</title> + + <para> + Do the following to make sure the build parameters are set up for the example. + Once you set up these build parameters, they do not have to change unless you + change the target architecture of the machine you are building: + <itemizedlist> + <listitem><para><emphasis>Build for the correct target architecture:</emphasis> Your + selected <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + definition within the <filename>local.conf</filename> file in the + <link linkend='build-directory'>Build Directory</link> + specifies the target architecture used when building the Linux kernel. + By default, <filename>MACHINE</filename> is set to + <filename>qemux86</filename>, which specifies a 32-bit + <trademark class='registered'>Intel</trademark> Architecture + target machine suitable for the QEMU emulator.</para></listitem> + <listitem><para><emphasis>Identify your <filename>meta-mylayer</filename> + layer:</emphasis> The + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink> + variable in the + <filename>bblayers.conf</filename> file found in the + <filename>poky/build/conf</filename> directory needs to have the path to your local + <filename>meta-mylayer</filename> layer. + By default, the <filename>BBLAYERS</filename> variable contains paths to + <filename>meta</filename>, <filename>meta-poky</filename>, and + <filename>meta-yocto-bsp</filename> in the + <filename>poky</filename> Git repository. + Add the path to your <filename>meta-mylayer</filename> location: + <literallayout class='monospaced'> + BBLAYERS ?= " \ + $HOME/poky/meta \ + $HOME/poky/meta-poky \ + $HOME/poky/meta-yocto-bsp \ + $HOME/poky/meta-mylayer \ + " + </literallayout></para></listitem> + </itemizedlist> + </para> + </section> + + <section id='build-the-modified-qemu-kernel-image'> + <title>Build the Modified QEMU Kernel Image</title> + + <para> + The following steps build your modified kernel image: + <orderedlist> + <listitem><para><emphasis>Be sure your build environment is initialized</emphasis>: + Your environment should be set up since you previously sourced + the + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + script. + If it is not, source the script again from <filename>poky</filename>. + <literallayout class='monospaced'> + $ cd ~/poky + $ source &OE_INIT_FILE; + </literallayout> + </para></listitem> + <listitem><para><emphasis>Clean up</emphasis>: + Be sure to clean the shared state out by using BitBake + to run from within the Build Directory the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleansstate'><filename>do_cleansstate</filename></ulink> + task as follows: + <literallayout class='monospaced'> + $ bitbake -c cleansstate linux-yocto + </literallayout></para> + <para> + <note> + Never remove any files by hand from the + <filename>tmp/deploy</filename> + directory inside the + <link linkend='build-directory'>Build Directory</link>. + Always use the various BitBake clean tasks to + clear out previous build artifacts. + For information on the clean tasks, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-clean'><filename>do_clean</filename></ulink>", + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleanall'><filename>do_cleanall</filename></ulink>", + and + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleansstate'><filename>do_cleansstate</filename></ulink>" + sections all in the Yocto Project Reference + Manual. + </note> + </para></listitem> + <listitem><para><emphasis>Build the image</emphasis>: + Next, build the kernel image using this command: + <literallayout class='monospaced'> + $ bitbake -k linux-yocto + </literallayout></para></listitem> + </orderedlist> + </para> + </section> + + <section id='boot-the-image-and-verify-your-changes'> + <title>Boot the Image and Verify Your Changes</title> + + <para> + These steps boot the image and allow you to see the changes + <orderedlist> + <listitem><para><emphasis>Boot the image</emphasis>: + Boot the modified image in the QEMU emulator + using this command: + <literallayout class='monospaced'> + $ runqemu qemux86 + </literallayout></para></listitem> + <listitem><para><emphasis>Verify the changes</emphasis>: + Log into the machine using <filename>root</filename> with no password and then + use the following shell command to scroll through the console's boot output. + <literallayout class='monospaced'> + # dmesg | less + </literallayout> + You should see the results of your <filename>printk</filename> statements + as part of the output.</para></listitem> + </orderedlist> + </para> + </section> + </section> + + <section id='making-images-more-secure'> + <title>Making Images More Secure</title> + + <para> + Security is of increasing concern for embedded devices. + Consider the issues and problems discussed in just this + sampling of work found across the Internet: + <itemizedlist> + <listitem><para><emphasis> + "<ulink url='https://www.schneier.com/blog/archives/2014/01/security_risks_9.html'>Security Risks of Embedded Systems</ulink>"</emphasis> + by Bruce Schneier + </para></listitem> + <listitem><para><emphasis> + "<ulink url='http://internetcensus2012.bitbucket.org/paper.html'>Internet Census 2012</ulink>"</emphasis> + by Carna Botnet</para></listitem> + <listitem><para><emphasis> + "<ulink url='http://elinux.org/images/6/6f/Security-issues.pdf'>Security Issues for Embedded Devices</ulink>"</emphasis> + by Jake Edge + </para></listitem> + </itemizedlist> + </para> + + <para> + When securing your image is of concern, there are steps, tools, + and variables that you can consider to help you reach the + security goals you need for your particular device. + Not all situations are identical when it comes to making an + image secure. + Consequently, this section provides some guidance and suggestions + for consideration when you want to make your image more secure. + <note> + Because the security requirements and risks are + different for every type of device, this section cannot + provide a complete reference on securing your custom OS. + It is strongly recommended that you also consult other sources + of information on embedded Linux system hardening and on + security. + </note> + </para> + + <section id='general-considerations'> + <title>General Considerations</title> + + <para> + General considerations exist that help you create more + secure images. + You should consider the following suggestions to help + make your device more secure: + <itemizedlist> + <listitem><para> + Scan additional code you are adding to the system + (e.g. application code) by using static analysis + tools. + Look for buffer overflows and other potential + security problems. + </para></listitem> + <listitem><para> + Pay particular attention to the security for + any web-based administration interface. + </para> + <para>Web interfaces typically need to perform + administrative functions and tend to need to run with + elevated privileges. + Thus, the consequences resulting from the interface's + security becoming compromised can be serious. + Look for common web vulnerabilities such as + cross-site-scripting (XSS), unvalidated inputs, + and so forth.</para> + <para>As with system passwords, the default credentials + for accessing a web-based interface should not be the + same across all devices. + This is particularly true if the interface is enabled + by default as it can be assumed that many end-users + will not change the credentials. + </para></listitem> + <listitem><para> + Ensure you can update the software on the device to + mitigate vulnerabilities discovered in the future. + This consideration especially applies when your + device is network-enabled. + </para></listitem> + <listitem><para> + Ensure you remove or disable debugging functionality + before producing the final image. + For information on how to do this, see the + "<link linkend='considerations-specific-to-the-openembedded-build-system'>Considerations Specific to the OpenEmbedded Build System</link>" + section. + </para></listitem> + <listitem><para> + Ensure you have no network services listening that + are not needed. + </para></listitem> + <listitem><para> + Remove any software from the image that is not needed. + </para></listitem> + <listitem><para> + Enable hardware support for secure boot functionality + when your device supports this functionality. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='security-flags'> + <title>Security Flags</title> + + <para> + The Yocto Project has security flags that you can enable that + help make your build output more secure. + The security flags are in the + <filename>meta/conf/distro/include/security_flags.inc</filename> + file in your + <link linkend='source-directory'>Source Directory</link> + (e.g. <filename>poky</filename>). + <note> + Depending on the recipe, certain security flags are enabled + and disabled by default. + </note> + </para> + + <para> +<!-- + The GCC/LD flags in <filename>security_flags.inc</filename> + enable more secure code generation. + By including the <filename>security_flags.inc</filename> + file, you enable flags to the compiler and linker that cause + them to generate more secure code. + <note> + The GCC/LD flags are enabled by default in the + <filename>poky-lsb</filename> distribution. + </note> +--> + Use the following line in your + <filename>local.conf</filename> file or in your custom + distribution configuration file to enable the security + compiler and linker flags for your build: + <literallayout class='monospaced'> + require conf/distro/include/security_flags.inc + </literallayout> + </para> + </section> + + <section id='considerations-specific-to-the-openembedded-build-system'> + <title>Considerations Specific to the OpenEmbedded Build System</title> + + <para> + You can take some steps that are specific to the + OpenEmbedded build system to make your images more secure: + <itemizedlist> + <listitem><para> + Ensure "debug-tweaks" is not one of your selected + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>. + When creating a new project, the default is to provide you + with an initial <filename>local.conf</filename> file that + enables this feature using the + <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> variable with the line: + <literallayout class='monospaced'> + EXTRA_IMAGE_FEATURES = "debug-tweaks" + </literallayout> + To disable that feature, simply comment out that line in your + <filename>local.conf</filename> file, or + make sure <filename>IMAGE_FEATURES</filename> does not contain + "debug-tweaks" before producing your final image. + Among other things, leaving this in place sets the + root password as blank, which makes logging in for + debugging or inspection easy during + development but also means anyone can easily log in + during production. + </para></listitem> + <listitem><para> + It is possible to set a root password for the image + and also to set passwords for any extra users you might + add (e.g. administrative or service type users). + When you set up passwords for multiple images or + users, you should not duplicate passwords. + </para> + <para> + To set up passwords, use the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers</filename></ulink> + class, which is the preferred method. + For an example on how to set up both root and user + passwords, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers.bbclass</filename></ulink>" + section. + <note> + When adding extra user accounts or setting a + root password, be cautious about setting the + same password on every device. + If you do this, and the password you have set + is exposed, then every device is now potentially + compromised. + If you need this access but want to ensure + security, consider setting a different, + random password for each device. + Typically, you do this as a separate step after + you deploy the image onto the device. + </note> + </para></listitem> + <listitem><para> + Consider enabling a Mandatory Access Control (MAC) + framework such as SMACK or SELinux and tuning it + appropriately for your device's usage. + You can find more information in the + <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/'><filename>meta-selinux</filename></ulink> + layer. + </para></listitem> + </itemizedlist> + </para> + + <para> + </para> + </section> + + <section id='tools-for-hardening-your-image'> + <title>Tools for Hardening Your Image</title> + + <para> + The Yocto Project provides tools for making your image + more secure. + You can find these tools in the + <filename>meta-security</filename> layer of the + <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Yocto Project Source Repositories</ulink>. + </para> + </section> + </section> + + <section id='creating-your-own-distribution'> + <title>Creating Your Own Distribution</title> + + <para> + When you build an image using the Yocto Project and + do not alter any distribution + <link linkend='metadata'>Metadata</link>, you are creating a + Poky distribution. + If you wish to gain more control over package alternative + selections, compile-time options, and other low-level + configurations, you can create your own distribution. + </para> + + <para> + To create your own distribution, the basic steps consist of + creating your own distribution layer, creating your own + distribution configuration file, and then adding any needed + code and Metadata to the layer. + The following steps provide some more detail: + <itemizedlist> + <listitem><para><emphasis>Create a layer for your new distro:</emphasis> + Create your distribution layer so that you can keep your + Metadata and code for the distribution separate. + It is strongly recommended that you create and use your own + layer for configuration and code. + Using your own layer as compared to just placing + configurations in a <filename>local.conf</filename> + configuration file makes it easier to reproduce the same + build configuration when using multiple build machines. + See the + "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>" + section for information on how to quickly set up a layer. + </para></listitem> + <listitem><para><emphasis>Create the distribution configuration file:</emphasis> + The distribution configuration file needs to be created in + the <filename>conf/distro</filename> directory of your + layer. + You need to name it using your distribution name + (e.g. <filename>mydistro.conf</filename>). + <note> + The + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> + variable in your + <filename>local.conf</filename> file determines the + name of your distribution. + </note></para> + <para>You can split out parts of your configuration file + into include files and then "require" them from within + your distribution configuration file. + Be sure to place the include files in the + <filename>conf/distro/include</filename> directory of + your layer. + A common example usage of include files would be to + separate out the selection of desired version and revisions + for individual recipes. +</para> + <para>Your configuration file needs to set the following + required variables: + <literallayout class='monospaced'> + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME'><filename>DISTRO_NAME</filename></ulink> + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION'><filename>DISTRO_VERSION</filename></ulink> + </literallayout> + These following variables are optional and you typically + set them from the distribution configuration file: + <literallayout class='monospaced'> + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS'><filename>DISTRO_EXTRA_RDEPENDS</filename></ulink> + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS'><filename>DISTRO_EXTRA_RRECOMMENDS</filename></ulink> + <ulink url='&YOCTO_DOCS_REF_URL;#var-TCLIBC'><filename>TCLIBC</filename></ulink> + </literallayout> + <tip> + If you want to base your distribution configuration file + on the very basic configuration from OE-Core, you + can use + <filename>conf/distro/defaultsetup.conf</filename> as + a reference and just include variables that differ + as compared to <filename>defaultsetup.conf</filename>. + Alternatively, you can create a distribution + configuration file from scratch using the + <filename>defaultsetup.conf</filename> file + or configuration files from other distributions + such as Poky or Angstrom as references. + </tip></para></listitem> + <listitem><para><emphasis>Provide miscellaneous variables:</emphasis> + Be sure to define any other variables for which you want to + create a default or enforce as part of the distribution + configuration. + You can include nearly any variable from the + <filename>local.conf</filename> file. + The variables you use are not limited to the list in the + previous bulleted item.</para></listitem> + <listitem><para><emphasis>Point to Your distribution configuration file:</emphasis> + In your <filename>local.conf</filename> file in the + <link linkend='build-directory'>Build Directory</link>, + set your + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> + variable to point to your distribution's configuration file. + For example, if your distribution's configuration file is + named <filename>mydistro.conf</filename>, then you point + to it as follows: + <literallayout class='monospaced'> + DISTRO = "mydistro" + </literallayout></para></listitem> + <listitem><para><emphasis>Add more to the layer if necessary:</emphasis> + Use your layer to hold other information needed for the + distribution: + <itemizedlist> + <listitem><para>Add recipes for installing + distro-specific configuration files that are not + already installed by another recipe. + If you have distro-specific configuration files + that are included by an existing recipe, you should + add an append file (<filename>.bbappend</filename>) + for those. + For general information and recommendations + on how to add recipes to your layer, see the + "<link linkend='creating-your-own-layer'>Creating Your Own Layer</link>" + and + "<link linkend='best-practices-to-follow-when-creating-layers'>Best Practices to Follow When Creating Layers</link>" + sections.</para></listitem> + <listitem><para>Add any image recipes that are specific + to your distribution.</para></listitem> + <listitem><para>Add a <filename>psplash</filename> + append file for a branded splash screen. + For information on append files, see the + "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" + section.</para></listitem> + <listitem><para>Add any other append files to make + custom changes that are specific to individual + recipes.</para></listitem> + </itemizedlist></para></listitem> + </itemizedlist> + </para> + </section> + + <section id='creating-a-custom-template-configuration-directory'> + <title>Creating a Custom Template Configuration Directory</title> + + <para> + If you are producing your own customized version + of the build system for use by other users, you might + want to customize the message shown by the setup script or + you might want to change the template configuration files (i.e. + <filename>local.conf</filename> and + <filename>bblayers.conf</filename>) that are created in + a new build directory. + </para> + + <para> + The OpenEmbedded build system uses the environment variable + <filename>TEMPLATECONF</filename> to locate the directory + from which it gathers configuration information that ultimately + ends up in the + <link linkend='build-directory'>Build Directory's</link> + <filename>conf</filename> directory. + By default, <filename>TEMPLATECONF</filename> is set as + follows in the <filename>poky</filename> repository: + <literallayout class='monospaced'> + TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf} + </literallayout> + This is the directory used by the build system to find templates + from which to build some key configuration files. + If you look at this directory, you will see the + <filename>bblayers.conf.sample</filename>, + <filename>local.conf.sample</filename>, and + <filename>conf-notes.txt</filename> files. + The build system uses these files to form the respective + <filename>bblayers.conf</filename> file, + <filename>local.conf</filename> file, and display the list of + BitBake targets when running the setup script. + </para> + + <para> + To override these default configuration files with + configurations you want used within every new + Build Directory, simply set the + <filename>TEMPLATECONF</filename> variable to your directory. + The <filename>TEMPLATECONF</filename> variable is set in the + <filename>.templateconf</filename> file, which is in the + top-level + <link linkend='source-directory'>Source Directory</link> + folder (e.g. <filename>poky</filename>). + Edit the <filename>.templateconf</filename> so that it can locate + your directory. + </para> + + <para> + Best practices dictate that you should keep your + template configuration directory in your custom distribution layer. + For example, suppose you have a layer named + <filename>meta-mylayer</filename> located in your home directory + and you want your template configuration directory named + <filename>myconf</filename>. + Changing the <filename>.templateconf</filename> as follows + causes the OpenEmbedded build system to look in your directory + and base its configuration files on the + <filename>*.sample</filename> configuration files it finds. + The final configuration files (i.e. + <filename>local.conf</filename> and + <filename>bblayers.conf</filename> ultimately still end up in + your Build Directory, but they are based on your + <filename>*.sample</filename> files. + <literallayout class='monospaced'> + TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf} + </literallayout> + </para> + + <para> + Aside from the <filename>*.sample</filename> configuration files, + the <filename>conf-notes.txt</filename> also resides in the + default <filename>meta-poky/conf</filename> directory. + The scripts that set up the build environment + (i.e. + <ulink url="&YOCTO_DOCS_REF_URL;#structure-core-script"><filename>&OE_INIT_FILE;</filename></ulink> + and + <ulink url="&YOCTO_DOCS_REF_URL;#structure-memres-core-script"><filename>oe-init-build-env-memres</filename></ulink>) + use this file to display BitBake targets as part of the script + output. + Customizing this <filename>conf-notes.txt</filename> file is a + good way to make sure your list of custom targets appears + as part of the script's output. + </para> + + <para> + Here is the default list of targets displayed as a result of + running either of the setup scripts: + <literallayout class='monospaced'> + You can now run 'bitbake <target>' + + Common targets are: + core-image-minimal + core-image-sato + meta-toolchain + meta-ide-support + </literallayout> + </para> + + <para> + Changing the listed common targets is as easy as editing your + version of <filename>conf-notes.txt</filename> in your + custom template configuration directory and making sure you + have <filename>TEMPLATECONF</filename> set to your directory. + </para> + </section> + + <section id='building-a-tiny-system'> + <title>Building a Tiny System</title> + + <para> + Very small distributions have some significant advantages such + as requiring less on-die or in-package memory (cheaper), better + performance through efficient cache usage, lower power requirements + due to less memory, faster boot times, and reduced development + overhead. + Some real-world examples where a very small distribution gives + you distinct advantages are digital cameras, medical devices, + and small headless systems. + </para> + + <para> + This section presents information that shows you how you can + trim your distribution to even smaller sizes than the + <filename>poky-tiny</filename> distribution, which is around + 5 Mbytes, that can be built out-of-the-box using the Yocto Project. + </para> + + <section id='tiny-system-overview'> + <title>Overview</title> + + <para> + The following list presents the overall steps you need to + consider and perform to create distributions with smaller + root filesystems, achieve faster boot times, maintain your critical + functionality, and avoid initial RAM disks: + <itemizedlist> + <listitem><para> + <link linkend='goals-and-guiding-principles'>Determine your goals and guiding principles.</link> + </para></listitem> + <listitem><para> + <link linkend='understand-what-gives-your-image-size'>Understand what contributes to your image size.</link> + </para></listitem> + <listitem><para> + <link linkend='trim-the-root-filesystem'>Reduce the size of the root filesystem.</link> + </para></listitem> + <listitem><para> + <link linkend='trim-the-kernel'>Reduce the size of the kernel.</link> + </para></listitem> + <listitem><para> + <link linkend='remove-package-management-requirements'>Eliminate packaging requirements.</link> + </para></listitem> + <listitem><para> + <link linkend='look-for-other-ways-to-minimize-size'>Look for other ways to minimize size.</link> + </para></listitem> + <listitem><para> + <link linkend='iterate-on-the-process'>Iterate on the process.</link> + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='goals-and-guiding-principles'> + <title>Goals and Guiding Principles</title> + + <para> + Before you can reach your destination, you need to know + where you are going. + Here is an example list that you can use as a guide when + creating very small distributions: + <itemizedlist> + <listitem><para>Determine how much space you need + (e.g. a kernel that is 1 Mbyte or less and + a root filesystem that is 3 Mbytes or less). + </para></listitem> + <listitem><para>Find the areas that are currently + taking 90% of the space and concentrate on reducing + those areas. + </para></listitem> + <listitem><para>Do not create any difficult "hacks" + to achieve your goals.</para></listitem> + <listitem><para>Leverage the device-specific + options.</para></listitem> + <listitem><para>Work in a separate layer so that you + keep changes isolated. + For information on how to create layers, see + the "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" section. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='understand-what-gives-your-image-size'> + <title>Understand What Contributes to Your Image Size</title> + + <para> + It is easiest to have something to start with when creating + your own distribution. + You can use the Yocto Project out-of-the-box to create the + <filename>poky-tiny</filename> distribution. + Ultimately, you will want to make changes in your own + distribution that are likely modeled after + <filename>poky-tiny</filename>. + <note> + To use <filename>poky-tiny</filename> in your build, + set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> + variable in your + <filename>local.conf</filename> file to "poky-tiny" + as described in the + "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>" + section. + </note> + </para> + + <para> + Understanding some memory concepts will help you reduce the + system size. + Memory consists of static, dynamic, and temporary memory. + Static memory is the TEXT (code), DATA (initialized data + in the code), and BSS (uninitialized data) sections. + Dynamic memory represents memory that is allocated at runtime: + stacks, hash tables, and so forth. + Temporary memory is recovered after the boot process. + This memory consists of memory used for decompressing + the kernel and for the <filename>__init__</filename> + functions. + </para> + + <para> + To help you see where you currently are with kernel and root + filesystem sizes, you can use two tools found in the + <link linkend='source-directory'>Source Directory</link> in + the <filename>scripts/tiny/</filename> directory: + <itemizedlist> + <listitem><para><filename>ksize.py</filename>: Reports + component sizes for the kernel build objects. + </para></listitem> + <listitem><para><filename>dirsize.py</filename>: Reports + component sizes for the root filesystem.</para></listitem> + </itemizedlist> + This next tool and command help you organize configuration + fragments and view file dependencies in a human-readable form: + <itemizedlist> + <listitem><para><filename>merge_config.sh</filename>: + Helps you manage configuration files and fragments + within the kernel. + With this tool, you can merge individual configuration + fragments together. + The tool allows you to make overrides and warns you + of any missing configuration options. + The tool is ideal for allowing you to iterate on + configurations, create minimal configurations, and + create configuration files for different machines + without having to duplicate your process.</para> + <para>The <filename>merge_config.sh</filename> script is + part of the Linux Yocto kernel Git repositories + (i.e. <filename>linux-yocto-3.14</filename>, + <filename>linux-yocto-3.10</filename>, + <filename>linux-yocto-3.8</filename>, and so forth) + in the + <filename>scripts/kconfig</filename> directory.</para> + <para>For more information on configuration fragments, + see the + "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#generating-configuration-files'>Generating Configuration Files</ulink>" + section of the Yocto Project Linux Kernel Development + Manual and the "<link linkend='creating-config-fragments'>Creating Configuration Fragments</link>" + section, which is in this manual.</para></listitem> + <listitem><para><filename>bitbake -u depexp -g <replaceable>bitbake_target</replaceable></filename>: + Using the BitBake command with these options brings up + a Dependency Explorer from which you can view file + dependencies. + Understanding these dependencies allows you to make + informed decisions when cutting out various pieces of the + kernel and root filesystem.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id='trim-the-root-filesystem'> + <title>Trim the Root Filesystem</title> + + <para> + The root filesystem is made up of packages for booting, + libraries, and applications. + To change things, you can configure how the packaging happens, + which changes the way you build them. + You can also modify the filesystem itself or select a different + filesystem. + </para> + + <para> + First, find out what is hogging your root filesystem by running the + <filename>dirsize.py</filename> script from your root directory: + <literallayout class='monospaced'> + $ cd <replaceable>root-directory-of-image</replaceable> + $ dirsize.py 100000 > dirsize-100k.log + $ cat dirsize-100k.log + </literallayout> + You can apply a filter to the script to ignore files under + a certain size. + The previous example filters out any files below 100 Kbytes. + The sizes reported by the tool are uncompressed, and thus + will be smaller by a relatively constant factor in a + compressed root filesystem. + When you examine your log file, you can focus on areas of the + root filesystem that take up large amounts of memory. + </para> + + <para> + You need to be sure that what you eliminate does not cripple + the functionality you need. + One way to see how packages relate to each other is by using + the Dependency Explorer UI with the BitBake command: + <literallayout class='monospaced'> + $ cd <replaceable>image-directory</replaceable> + $ bitbake -u depexp -g <replaceable>image</replaceable> + </literallayout> + Use the interface to select potential packages you wish to + eliminate and see their dependency relationships. + </para> + + <para> + When deciding how to reduce the size, get rid of packages that + result in minimal impact on the feature set. + For example, you might not need a VGA display. + Or, you might be able to get by with <filename>devtmpfs</filename> + and <filename>mdev</filename> instead of + <filename>udev</filename>. + </para> + + <para> + Use your <filename>local.conf</filename> file to make changes. + For example, to eliminate <filename>udev</filename> and + <filename>glib</filename>, set the following in the + local configuration file: + <literallayout class='monospaced'> + VIRTUAL-RUNTIME_dev_manager = "" + </literallayout> + </para> + + <para> + Finally, you should consider exactly the type of root + filesystem you need to meet your needs while also reducing + its size. + For example, consider <filename>cramfs</filename>, + <filename>squashfs</filename>, <filename>ubifs</filename>, + <filename>ext2</filename>, or an <filename>initramfs</filename> + using <filename>initramfs</filename>. + Be aware that <filename>ext3</filename> requires a 1 Mbyte + journal. + If you are okay with running read-only, you do not need this + journal. + </para> + + <note> + After each round of elimination, you need to rebuild your + system and then use the tools to see the effects of your + reductions. + </note> + + + </section> + + <section id='trim-the-kernel'> + <title>Trim the Kernel</title> + + <para> + The kernel is built by including policies for hardware-independent + aspects. + What subsystems do you enable? + For what architecture are you building? + Which drivers do you build by default? + <note>You can modify the kernel source if you want to help + with boot time. + </note> + </para> + + <para> + Run the <filename>ksize.py</filename> script from the top-level + Linux build directory to get an idea of what is making up + the kernel: + <literallayout class='monospaced'> + $ cd <replaceable>top-level-linux-build-directory</replaceable> + $ ksize.py > ksize.log + $ cat ksize.log + </literallayout> + When you examine the log, you will see how much space is + taken up with the built-in <filename>.o</filename> files for + drivers, networking, core kernel files, filesystem, sound, + and so forth. + The sizes reported by the tool are uncompressed, and thus + will be smaller by a relatively constant factor in a compressed + kernel image. + Look to reduce the areas that are large and taking up around + the "90% rule." + </para> + + <para> + To examine, or drill down, into any particular area, use the + <filename>-d</filename> option with the script: + <literallayout class='monospaced'> + $ ksize.py -d > ksize.log + </literallayout> + Using this option breaks out the individual file information + for each area of the kernel (e.g. drivers, networking, and + so forth). + </para> + + <para> + Use your log file to see what you can eliminate from the kernel + based on features you can let go. + For example, if you are not going to need sound, you do not + need any drivers that support sound. + </para> + + <para> + After figuring out what to eliminate, you need to reconfigure + the kernel to reflect those changes during the next build. + You could run <filename>menuconfig</filename> and make all your + changes at once. + However, that makes it difficult to see the effects of your + individual eliminations and also makes it difficult to replicate + the changes for perhaps another target device. + A better method is to start with no configurations using + <filename>allnoconfig</filename>, create configuration + fragments for individual changes, and then manage the + fragments into a single configuration file using + <filename>merge_config.sh</filename>. + The tool makes it easy for you to iterate using the + configuration change and build cycle. + </para> + + <para> + Each time you make configuration changes, you need to rebuild + the kernel and check to see what impact your changes had on + the overall size. + </para> + </section> + + <section id='remove-package-management-requirements'> + <title>Remove Package Management Requirements</title> + + <para> + Packaging requirements add size to the image. + One way to reduce the size of the image is to remove all the + packaging requirements from the image. + This reduction includes both removing the package manager + and its unique dependencies as well as removing the package + management data itself. + </para> + + <para> + To eliminate all the packaging requirements for an image, + be sure that "package-management" is not part of your + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> + statement for the image. + When you remove this feature, you are removing the package + manager as well as its dependencies from the root filesystem. + </para> + </section> + + <section id='look-for-other-ways-to-minimize-size'> + <title>Look for Other Ways to Minimize Size</title> + + <para> + Depending on your particular circumstances, other areas that you + can trim likely exist. + The key to finding these areas is through tools and methods + described here combined with experimentation and iteration. + Here are a couple of areas to experiment with: + <itemizedlist> + <listitem><para><filename>glibc</filename>: + In general, follow this process: + <orderedlist> + <listitem><para>Remove <filename>glibc</filename> + features from + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> + that you think you do not need.</para></listitem> + <listitem><para>Build your distribution. + </para></listitem> + <listitem><para>If the build fails due to missing + symbols in a package, determine if you can + reconfigure the package to not need those + features. + For example, change the configuration to not + support wide character support as is done for + <filename>ncurses</filename>. + Or, if support for those characters is needed, + determine what <filename>glibc</filename> + features provide the support and restore the + configuration. + </para></listitem> + <listitem><para>Rebuild and repeat the process. + </para></listitem> + </orderedlist></para></listitem> + <listitem><para><filename>busybox</filename>: + For BusyBox, use a process similar as described for + <filename>glibc</filename>. + A difference is you will need to boot the resulting + system to see if you are able to do everything you + expect from the running system. + You need to be sure to integrate configuration fragments + into Busybox because BusyBox handles its own core + features and then allows you to add configuration + fragments on top. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='iterate-on-the-process'> + <title>Iterate on the Process</title> + + <para> + If you have not reached your goals on system size, you need + to iterate on the process. + The process is the same. + Use the tools and see just what is taking up 90% of the root + filesystem and the kernel. + Decide what you can eliminate without limiting your device + beyond what you need. + </para> + + <para> + Depending on your system, a good place to look might be + Busybox, which provides a stripped down + version of Unix tools in a single, executable file. + You might be able to drop virtual terminal services or perhaps + ipv6. + </para> + </section> + </section> + + <section id='building-images-for-more-than-one-machine'> + <title>Building Images for More than One Machine</title> + + <para> + A common scenario developers face is creating images for several + different machines that use the same software environment. + In this situation, it is tempting to set the + tunings and optimization flags for each build specifically for + the targeted hardware (i.e. "maxing out" the tunings). + Doing so can considerably add to build times and package feed + maintenance collectively for the machines. + For example, selecting tunes that are extremely specific to a + CPU core used in a system might enable some micro optimizations + in GCC for that particular system but would otherwise not gain + you much of a performance difference across the other systems + as compared to using a more general tuning across all the builds + (e.g. setting + <ulink url='var-DEFAULTTUNE'><filename>DEFAULTTUNE</filename></ulink> + specifically for each machine's build). + Rather than "max out" each build's tunings, you can take steps that + cause the OpenEmbedded build system to reuse software across the + various machines where it makes sense. + </para> + <para> + If build speed and package feed maintenance are considerations, + you should consider the points in this section that can help you + optimize your tunings to best consider build times and package + feed maintenance. + <itemizedlist> + <listitem><para><emphasis>Share the Build Directory:</emphasis> + If at all possible, share the + <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> + across builds. + The Yocto Project supports switching between different + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> + values in the same <filename>TMPDIR</filename>. + This practice is well supported and regularly used by + developers when building for multiple machines. + When you use the same <filename>TMPDIR</filename> for + multiple machine builds, the OpenEmbedded build system can + reuse the existing native and often cross-recipes for + multiple machines. + Thus, build time decreases. + <note> + If + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> + settings change or fundamental configuration settings + such as the filesystem layout, you need to work with + a clean <filename>TMPDIR</filename>. + Sharing <filename>TMPDIR</filename> under these + circumstances might work but since it is not + guaranteed, you should use a clean + <filename>TMPDIR</filename>. + </note> + </para></listitem> + <listitem><para><emphasis>Enable the Appropriate Package Architecture:</emphasis> + By default, the OpenEmbedded build system enables three + levels of package architectures: "all", "tune" or "package", + and "machine". + Any given recipe usually selects one of these package + architectures (types) for its output. + Depending for what a given recipe creates packages, making + sure you enable the appropriate package architecture can + directly impact the build time.</para> + <para>A recipe that just generates scripts can enable + "all" architecture because there are no binaries to build. + To specifically enable "all" architecture, be sure your + recipe inherits the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink> + class. + This class is useful for "all" architectures because it + configures many variables so packages can be used across + multiple architectures.</para> + <para>If your recipe needs to generate packages that are + machine-specific or when one of the build or runtime + dependencies is already machine-architecture dependent, + which makes your recipe also machine-architecture dependent, + make sure your recipe enables the "machine" package + architecture through the + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink> + variable: + <literallayout class='monospaced'> + PACKAGE_ARCH = "${MACHINE_ARCH}" + </literallayout> + When you do not specifically enable a package + architecture through the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>, + The OpenEmbedded build system defaults to the + <ulink url='&YOCTO_DOCS_REF_URL;#var-TUNE_PKGARCH'><filename>TUNE_PKGARCH</filename></ulink> + setting: + <literallayout class='monospaced'> + PACKAGE_ARCH = "${TUNE_PKGARCH}" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Choose a Generic Tuning File if Possible:</emphasis> + Some tunes are more generic and can run on multiple targets + (e.g. an <filename>armv5</filename> set of packages could + run on <filename>armv6</filename> and + <filename>armv7</filename> processors in most cases). + Similarly, <filename>i486</filename> binaries could work + on <filename>i586</filename> and higher processors. + You should realize, however, that advances on newer + processor versions would not be used.</para> + <para>If you select the same tune for several different + machines, the OpenEmbedded build system reuses software + previously built, thus speeding up the overall build time. + Realize that even though a new sysroot for each machine is + generated, the software is not recompiled and only one + package feed exists. + </para></listitem> + <listitem><para><emphasis>Manage Granular Level Packaging:</emphasis> + Sometimes cases exist where injecting another level + of package architecture beyond the three higher levels + noted earlier can be useful. + For example, consider the <filename>emgd</filename> + graphics stack in the + <filename>meta-intel</filename> layer. + In this layer, a subset of software exists that is + compiled against something different from the rest of the + generic packages. + You can examine the key code in the + <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Source Repositories</ulink> + "daisy" branch in + <filename>classes/emgd-gl.bbclass</filename>. + For a specific set of packages, the code redefines + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>. + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXTRA_ARCHS'><filename>PACKAGE_EXTRA_ARCHS</filename></ulink> + is then appended with this extra tune name in + <filename>meta-intel-emgd.inc</filename>. + The result is that when searching for packages, the + build system uses a four-level search and the packages + in this new level are preferred as compared to the standard + tune. + The overall result is that the build system reuses most + software from the common tune except for specific cases + as needed. + </para></listitem> + <listitem><para><emphasis>Use Tools to Debug Issues:</emphasis> + Sometimes you can run into situations where software is + being rebuilt when you think it should not be. + For example, the OpenEmbedded build system might not be + using shared state between machines when you think it + should be. + These types of situations are usually due to references + to machine-specific variables such as + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLE'><filename>SERIAL_CONSOLE</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-XSERVER'><filename>XSERVER</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'><filename>MACHINE_FEATURES</filename></ulink>, + and so forth in code that is supposed to only be + tune-specific or when the recipe depends + (<ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-RSUGGESTS'><filename>RSUGGESTS</filename></ulink>, + and so forth) on some other recipe that already has + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink> + defined as "${MACHINE_ARCH}". + <note> + Patches to fix any issues identified are most welcome + as these issues occasionally do occur. + </note></para> + <para>For such cases, you can use some tools to help you + sort out the situation: + <itemizedlist> + <listitem><para><emphasis><filename>sstate-diff-machines.sh</filename>:</emphasis> + You can find this tool in the + <filename>scripts</filename> directory of the + Source Repositories. + See the comments in the script for information on + how to use the tool. + </para></listitem> + <listitem><para><emphasis>BitBake's "-S printdiff" Option:</emphasis> + Using this option causes BitBake to try to + establish the closest signature match it can + (e.g. in the shared state cache) and then run + <filename>bitbake-diffsigs</filename> over the + matches to determine the stamps and delta where + these two stamp trees diverge. + </para></listitem> + </itemizedlist> + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='working-with-packages'> + <title>Working with Packages</title> + + <para> + This section describes a few tasks that involve packages: + <itemizedlist> + <listitem><para> + <link linkend='excluding-packages-from-an-image'>Excluding packages from an image</link> + </para></listitem> + <listitem><para> + <link linkend='incrementing-a-package-revision-number'>Incrementing a package revision number</link> + </para></listitem> + <listitem><para> + <link linkend='handling-optional-module-packaging'>Handling optional module packaging</link> + </para></listitem> + <listitem><para> + <link linkend='using-runtime-package-management'>Using Runtime Package Management</link> + </para></listitem> + <listitem><para> + <link linkend='testing-packages-with-ptest'>Setting up and running package test (ptest)</link> + </para></listitem> + </itemizedlist> + </para> + + <section id='excluding-packages-from-an-image'> + <title>Excluding Packages from an Image</title> + + <para> + You might find it necessary to prevent specific packages + from being installed into an image. + If so, you can use several variables to direct the build + system to essentially ignore installing recommended packages + or to not install a package at all. + </para> + + <para> + The following list introduces variables you can use to + prevent packages from being installed into your image. + Each of these variables only works with IPK and RPM + package types. + Support for Debian packages does not exist. + Also, you can use these variables from your + <filename>local.conf</filename> file or attach them to a + specific image recipe by using a recipe name override. + For more detail on the variables, see the descriptions in the + Yocto Project Reference Manual's glossary chapter. + <itemizedlist> + <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></ulink>: + Use this variable to specify "recommended-only" + packages that you do not want installed. + </para></listitem> + <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS'><filename>NO_RECOMMENDATIONS</filename></ulink>: + Use this variable to prevent all "recommended-only" + packages from being installed. + </para></listitem> + <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>: + Use this variable to prevent specific packages from + being installed regardless of whether they are + "recommended-only" or not. + You need to realize that the build process could + fail with an error when you + prevent the installation of a package whose presence + is required by an installed package. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='incrementing-a-package-revision-number'> + <title>Incrementing a Package Revision Number</title> + + <para> + If a committed change results in changing the package output, + then the value of the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> + variable needs to be increased (or "bumped"). + Increasing <filename>PR</filename> occurs one of two ways: + <itemizedlist> + <listitem><para>Automatically using a Package Revision + Service (PR Service).</para></listitem> + <listitem><para>Manually incrementing the + <filename>PR</filename> variable.</para></listitem> + </itemizedlist> + </para> + + <para> + Given that one of the challenges any build system and its + users face is how to maintain a package feed that is compatible + with existing package manager applications such as + RPM, APT, and OPKG, using an automated system is much + preferred over a manual system. + In either system, the main requirement is that version + numbering increases in a linear fashion and that a number of + version components exist that support that linear progression. + </para> + + <para> + The following two sections provide information on the PR Service + and on manual <filename>PR</filename> bumping. + </para> + + <section id='working-with-a-pr-service'> + <title>Working With a PR Service</title> + + <para> + As mentioned, attempting to maintain revision numbers in the + <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> + is error prone, inaccurate, and causes problems for people + submitting recipes. + Conversely, the PR Service automatically generates + increasing numbers, particularly the revision field, + which removes the human element. + <note> + For additional information on using a PR Service, you + can see the + <ulink url='&YOCTO_WIKI_URL;/wiki/PR_Service'>PR Service</ulink> + wiki page. + </note> + </para> + + <para> + The Yocto Project uses variables in order of + decreasing priority to facilitate revision numbering (i.e. + <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and + <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> + for epoch, version, and revision, respectively). + The values are highly dependent on the policies and + procedures of a given distribution and package feed. + </para> + + <para> + Because the OpenEmbedded build system uses + "<ulink url='&YOCTO_DOCS_REF_URL;#checksums'>signatures</ulink>", + which are unique to a given build, the build system + knows when to rebuild packages. + All the inputs into a given task are represented by a + signature, which can trigger a rebuild when different. + Thus, the build system itself does not rely on the + <filename>PR</filename> numbers to trigger a rebuild. + The signatures, however, can be used to generate + <filename>PR</filename> values. + </para> + + <para> + The PR Service works with both + <filename>OEBasic</filename> and + <filename>OEBasicHash</filename> generators. + The value of <filename>PR</filename> bumps when the + checksum changes and the different generator mechanisms + change signatures under different circumstances. + </para> + + <para> + As implemented, the build system includes values from + the PR Service into the <filename>PR</filename> field as + an addition using the form "<filename>.x</filename>" so + <filename>r0</filename> becomes <filename>r0.1</filename>, + <filename>r0.2</filename> and so forth. + This scheme allows existing <filename>PR</filename> values + to be used for whatever reasons, which include manual + <filename>PR</filename> bumps, should it be necessary. + </para> + + <para> + By default, the PR Service is not enabled or running. + Thus, the packages generated are just "self consistent". + The build system adds and removes packages and + there are no guarantees about upgrade paths but images + will be consistent and correct with the latest changes. + </para> + + <para> + The simplest form for a PR Service is for it to exist + for a single host development system that builds the + package feed (building system). + For this scenario, you can enable a local PR Service by + setting + <ulink url='&YOCTO_DOCS_REF_URL;#var-PRSERV_HOST'><filename>PRSERV_HOST</filename></ulink> + in your <filename>local.conf</filename> file in the + <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>: + <literallayout class='monospaced'> + PRSERV_HOST = "localhost:0" + </literallayout> + Once the service is started, packages will automatically + get increasing <filename>PR</filename> values and + BitBake will take care of starting and stopping the server. + </para> + + <para> + If you have a more complex setup where multiple host + development systems work against a common, shared package + feed, you have a single PR Service running and it is + connected to each building system. + For this scenario, you need to start the PR Service using + the <filename>bitbake-prserv</filename> command: + <literallayout class='monospaced'> + bitbake-prserv --host <replaceable>ip</replaceable> --port <replaceable>port</replaceable> --start + </literallayout> + In addition to hand-starting the service, you need to + update the <filename>local.conf</filename> file of each + building system as described earlier so each system + points to the server and port. + </para> + + <para> + It is also recommended you use build history, which adds + some sanity checks to package versions, in conjunction with + the server that is running the PR Service. + To enable build history, add the following to each building + system's <filename>local.conf</filename> file: + <literallayout class='monospaced'> + # It is recommended to activate "buildhistory" for testing the PR service + INHERIT += "buildhistory" + BUILDHISTORY_COMMIT = "1" + </literallayout> + For information on build history, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#maintaining-build-output-quality'>Maintaining Build Output Quality</ulink>" + section in the Yocto Project Reference Manual. + </para> + + <note> + <para>The OpenEmbedded build system does not maintain + <filename>PR</filename> information as part of the + shared state (sstate) packages. + If you maintain an sstate feed, its expected that either + all your building systems that contribute to the sstate + feed use a shared PR Service, or you do not run a PR + Service on any of your building systems. + Having some systems use a PR Service while others do + not leads to obvious problems.</para> + <para>For more information on shared state, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink>" + section in the Yocto Project Reference Manual.</para> + </note> + </section> + + <section id='manually-bumping-pr'> + <title>Manually Bumping PR</title> + + <para> + The alternative to setting up a PR Service is to manually + bump the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> + variable. + </para> + + <para> + If a committed change results in changing the package output, + then the value of the PR variable needs to be increased + (or "bumped") as part of that commit. + For new recipes you should add the <filename>PR</filename> + variable and set its initial value equal to "r0", which is the default. + Even though the default value is "r0", the practice of adding it to a new recipe makes + it harder to forget to bump the variable when you make changes + to the recipe in future. + </para> + + <para> + If you are sharing a common <filename>.inc</filename> file with multiple recipes, + you can also use the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-INC_PR'>INC_PR</ulink></filename> + variable to ensure that + the recipes sharing the <filename>.inc</filename> file are rebuilt when the + <filename>.inc</filename> file itself is changed. + The <filename>.inc</filename> file must set <filename>INC_PR</filename> + (initially to "r0"), and all recipes referring to it should set <filename>PR</filename> + to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed. + If the <filename>.inc</filename> file is changed then its + <filename>INC_PR</filename> should be incremented. + </para> + + <para> + When upgrading the version of a package, assuming the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'>PV</ulink></filename> + changes, the <filename>PR</filename> variable should be + reset to "r0" (or "$(INC_PR).0" if you are using + <filename>INC_PR</filename>). + </para> + + <para> + Usually, version increases occur only to packages. + However, if for some reason <filename>PV</filename> changes but does not + increase, you can increase the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PE'>PE</ulink></filename> + variable (Package Epoch). + The <filename>PE</filename> variable defaults to "0". + </para> + + <para> + Version numbering strives to follow the + <ulink url='http://www.debian.org/doc/debian-policy/ch-controlfields.html'> + Debian Version Field Policy Guidelines</ulink>. + These guidelines define how versions are compared and what "increasing" a version means. + </para> + </section> + </section> + + <section id='handling-optional-module-packaging'> + <title>Handling Optional Module Packaging</title> + + <para> + Many pieces of software split functionality into optional + modules (or plug-ins) and the plug-ins that are built + might depend on configuration options. + To avoid having to duplicate the logic that determines what + modules are available in your recipe or to avoid having + to package each module by hand, the OpenEmbedded build system + provides functionality to handle module packaging dynamically. + </para> + + <para> + To handle optional module packaging, you need to do two things: + <itemizedlist> + <listitem><para>Ensure the module packaging is actually + done.</para></listitem> + <listitem><para>Ensure that any dependencies on optional + modules from other recipes are satisfied by your recipe. + </para></listitem> + </itemizedlist> + </para> + + <section id='making-sure-the-packaging-is-done'> + <title>Making Sure the Packaging is Done</title> + + <para> + To ensure the module packaging actually gets done, you use + the <filename>do_split_packages</filename> function within + the <filename>populate_packages</filename> Python function + in your recipe. + The <filename>do_split_packages</filename> function + searches for a pattern of files or directories under a + specified path and creates a package for each one it finds + by appending to the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink> + variable and setting the appropriate values for + <filename>FILES_packagename</filename>, + <filename>RDEPENDS_packagename</filename>, + <filename>DESCRIPTION_packagename</filename>, and so forth. + Here is an example from the <filename>lighttpd</filename> + recipe: + <literallayout class='monospaced'> + python populate_packages_prepend () { + lighttpd_libdir = d.expand('${libdir}') + do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$', + 'lighttpd-module-%s', 'Lighttpd module for %s', + extra_depends='') + } + </literallayout> + The previous example specifies a number of things in the + call to <filename>do_split_packages</filename>. + <itemizedlist> + <listitem><para>A directory within the files installed + by your recipe through <filename>do_install</filename> + in which to search.</para></listitem> + <listitem><para>A regular expression used to match module + files in that directory. + In the example, note the parentheses () that mark + the part of the expression from which the module + name should be derived.</para></listitem> + <listitem><para>A pattern to use for the package names. + </para></listitem> + <listitem><para>A description for each package. + </para></listitem> + <listitem><para>An empty string for + <filename>extra_depends</filename>, which disables + the default dependency on the main + <filename>lighttpd</filename> package. + Thus, if a file in <filename>${libdir}</filename> + called <filename>mod_alias.so</filename> is found, + a package called <filename>lighttpd-module-alias</filename> + is created for it and the + <ulink url='&YOCTO_DOCS_REF_URL;#var-DESCRIPTION'><filename>DESCRIPTION</filename></ulink> + is set to "Lighttpd module for alias".</para></listitem> + </itemizedlist> + </para> + + <para> + Often, packaging modules is as simple as the previous + example. + However, more advanced options exist that you can use + within <filename>do_split_packages</filename> to modify its + behavior. + And, if you need to, you can add more logic by specifying + a hook function that is called for each package. + It is also perfectly acceptable to call + <filename>do_split_packages</filename> multiple times if + you have more than one set of modules to package. + </para> + + <para> + For more examples that show how to use + <filename>do_split_packages</filename>, see the + <filename>connman.inc</filename> file in the + <filename>meta/recipes-connectivity/connman/</filename> + directory of the <filename>poky</filename> + <link linkend='yocto-project-repositories'>source repository</link>. + You can also find examples in + <filename>meta/classes/kernel.bbclass</filename>. + </para> + + <para> + Following is a reference that shows + <filename>do_split_packages</filename> mandatory and + optional arguments: + <literallayout class='monospaced'> + Mandatory arguments + + root + The path in which to search + file_regex + Regular expression to match searched files. + Use parentheses () to mark the part of this + expression that should be used to derive the + module name (to be substituted where %s is + used in other function arguments as noted below) + output_pattern + Pattern to use for the package names. Must + include %s. + description + Description to set for each package. Must + include %s. + + Optional arguments + + postinst + Postinstall script to use for all packages + (as a string) + recursive + True to perform a recursive search - default + False + hook + A hook function to be called for every match. + The function will be called with the following + arguments (in the order listed): + + f + Full path to the file/directory match + pkg + The package name + file_regex + As above + output_pattern + As above + modulename + The module name derived using file_regex + + extra_depends + Extra runtime dependencies (RDEPENDS) to be + set for all packages. The default value of None + causes a dependency on the main package + (${PN}) - if you do not want this, pass empty + string '' for this parameter. + aux_files_pattern + Extra item(s) to be added to FILES for each + package. Can be a single string item or a list + of strings for multiple items. Must include %s. + postrm + postrm script to use for all packages (as a + string) + allow_dirs + True to allow directories to be matched - + default False + prepend + If True, prepend created packages to PACKAGES + instead of the default False which appends them + match_path + match file_regex on the whole relative path to + the root rather than just the file name + aux_files_pattern_verbatim + Extra item(s) to be added to FILES for each + package, using the actual derived module name + rather than converting it to something legal + for a package name. Can be a single string item + or a list of strings for multiple items. Must + include %s. + allow_links + True to allow symlinks to be matched - default + False + summary + Summary to set for each package. Must include %s; + defaults to description if not set. + </literallayout> + </para> + </section> + + <section id='satisfying-dependencies'> + <title>Satisfying Dependencies</title> + + <para> + The second part for handling optional module packaging + is to ensure that any dependencies on optional modules + from other recipes are satisfied by your recipe. + You can be sure these dependencies are satisfied by + using the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> variable. + Here is an example that continues with the + <filename>lighttpd</filename> recipe shown earlier: + <literallayout class='monospaced'> + PACKAGES_DYNAMIC = "lighttpd-module-.*" + </literallayout> + The name specified in the regular expression can of + course be anything. + In this example, it is <filename>lighttpd-module-</filename> + and is specified as the prefix to ensure that any + <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink> + and <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink> + on a package name starting with the prefix are satisfied + during build time. + If you are using <filename>do_split_packages</filename> + as described in the previous section, the value you put in + <filename>PACKAGES_DYNAMIC</filename> should correspond to + the name pattern specified in the call to + <filename>do_split_packages</filename>. + </para> + </section> + </section> + + <section id='using-runtime-package-management'> + <title>Using Runtime Package Management</title> + + <para> + During a build, BitBake always transforms a recipe into one or + more packages. + For example, BitBake takes the <filename>bash</filename> recipe + and currently produces the <filename>bash-dbg</filename>, + <filename>bash-staticdev</filename>, + <filename>bash-dev</filename>, <filename>bash-doc</filename>, + <filename>bash-locale</filename>, and + <filename>bash</filename> packages. + Not all generated packages are included in an image. + </para> + + <para> + In several situations, you might need to update, add, remove, + or query the packages on a target device at runtime + (i.e. without having to generate a new image). + Examples of such situations include: + <itemizedlist> + <listitem><para> + You want to provide in-the-field updates to deployed + devices (e.g. security updates). + </para></listitem> + <listitem><para> + You want to have a fast turn-around development cycle + for one or more applications that run on your device. + </para></listitem> + <listitem><para> + You want to temporarily install the "debug" packages + of various applications on your device so that + debugging can be greatly improved by allowing + access to symbols and source debugging. + </para></listitem> + <listitem><para> + You want to deploy a more minimal package selection of + your device but allow in-the-field updates to add a + larger selection for customization. + </para></listitem> + </itemizedlist> + </para> + + <para> + In all these situations, you have something similar to a more + traditional Linux distribution in that in-field devices + are able to receive pre-compiled packages from a server for + installation or update. + Being able to install these packages on a running, + in-field device is what is termed "runtime package + management". + </para> + + <para> + In order to use runtime package management, you + need a host/server machine that serves up the pre-compiled + packages plus the required metadata. + You also need package manipulation tools on the target. + The build machine is a likely candidate to act as the server. + However, that machine does not necessarily have to be the + package server. + The build machine could push its artifacts to another machine + that acts as the server (e.g. Internet-facing). + </para> + + <para> + A simple build that targets just one device produces + more than one package database. + In other words, the packages produced by a build are separated + out into a couple of different package groupings based on + criteria such as the target's CPU architecture, the target + board, or the C library used on the target. + For example, a build targeting the <filename>qemuarm</filename> + device produces the following three package databases: + <filename>all</filename>, <filename>armv5te</filename>, and + <filename>qemuarm</filename>. + If you wanted your <filename>qemuarm</filename> device to be + aware of all the packages that were available to it, + you would need to point it to each of these databases + individually. + In a similar way, a traditional Linux distribution usually is + configured to be aware of a number of software repositories + from which it retrieves packages. + </para> + + <para> + Using runtime package management is completely optional and + not required for a successful build or deployment in any + way. + But if you want to make use of runtime package management, + you need to do a couple things above and beyond the basics. + The remainder of this section describes what you need to do. + </para> + + <section id='runtime-package-management-build'> + <title>Build Considerations</title> + + <para> + This section describes build considerations that you need + to be aware of in order to provide support for runtime + package management. + </para> + + <para> + When BitBake generates packages it needs to know + what format or formats to use. + In your configuration, you use the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink> + variable to specify the format. + <note> + You can choose to have more than one format but you must + provide at least one. + </note> + </para> + + <para> + If you would like your image to start off with a basic + package database of the packages in your current build + as well as have the relevant tools available on the + target for runtime package management, you can include + "package-management" in the + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> + variable. + Including "package-management" in this + configuration variable ensures that when the image + is assembled for your target, the image includes + the currently-known package databases as well as + the target-specific tools required for runtime + package management to be performed on the target. + However, this is not strictly necessary. + You could start your image off without any databases + but only include the required on-target package + tool(s). + As an example, you could include "opkg" in your + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink> + variable if you are using the IPK package format. + You can then initialize your target's package database(s) + later once your image is up and running. + </para> + + <para> + Whenever you perform any sort of build step that can + potentially generate a package or modify an existing + package, it is always a good idea to re-generate the + package index with: + <literallayout class='monospaced'> + $ bitbake package-index + </literallayout> + Realize that it is not sufficient to simply do the + following: + <literallayout class='monospaced'> + $ bitbake <replaceable>some-package</replaceable> package-index + </literallayout> + This is because BitBake does not properly schedule the + <filename>package-index</filename> target fully after any + other target has completed. + Thus, be sure to run the package update step separately. + </para> + + <para> + As described below in the + "<link linkend='runtime-package-management-target-ipk'>Using IPK</link>" + section, if you are using IPK as your package format, you + can make use of the + <filename>distro-feed-configs</filename> recipe provided + by <filename>meta-oe</filename> in order to configure your + target to use your IPK databases. + </para> + + <para> + When your build is complete, your packages reside in the + <filename>${TMPDIR}/deploy/<replaceable>package-format</replaceable></filename> + directory. + For example, if <filename>${TMPDIR}</filename> + is <filename>tmp</filename> and your selected package type + is IPK, then your IPK packages are available in + <filename>tmp/deploy/ipk</filename>. + </para> + </section> + + <section id='runtime-package-management-server'> + <title>Host or Server Machine Setup</title> + + <para> + Typically, packages are served from a server using + HTTP. + However, other protocols are possible. + If you want to use HTTP, then setup and configure a + web server, such as Apache 2 or lighttpd, on the machine + serving the packages. + </para> + + <para> + As previously mentioned, the build machine can act as the + package server. + In the following sections that describe server machine + setups, the build machine is assumed to also be the server. + </para> + + <section id='package-server-apache'> + <title>Serving Packages via Apache 2</title> + + <para> + This example assumes you are using the Apache 2 + server: + <orderedlist> + <listitem><para> + Add the directory to your Apache + configuration, which you can find at + <filename>/etc/httpd/conf/httpd.conf</filename>. + Use commands similar to these on the + development system. + These example commands assume a top-level + <link linkend='source-directory'>Source Directory</link> + named <filename>poky</filename> in your home + directory. + The example also assumes an RPM package type. + If you are using a different package type, such + as IPK, use "ipk" in the pathnames: + <literallayout class='monospaced'> + <VirtualHost *:80> + .... + Alias /rpm ~/poky/build/tmp/deploy/rpm + <Directory "~/poky/build/tmp/deploy/rpm"> + Options +Indexes + </Directory> + </VirtualHost> + </literallayout></para></listitem> + <listitem><para> + Reload the Apache configuration as described + in this step. + For all commands, be sure you have root + privileges. + </para> + + <para> + If your development system is using Fedora or + CentOS, use the following: + <literallayout class='monospaced'> + # service httpd reload + </literallayout> + For Ubuntu and Debian, use the following: + <literallayout class='monospaced'> + # /etc/init.d/apache2 reload + </literallayout> + For OpenSUSE, use the following: + <literallayout class='monospaced'> + # /etc/init.d/apache2 reload + </literallayout></para></listitem> + <listitem><para> + If you are using Security-Enhanced Linux + (SELinux), you need to label the files as + being accessible through Apache. + Use the following command from the development + host. + This example assumes RPM package types: + <literallayout class='monospaced'> + # chcon -R -h -t httpd_sys_content_t tmp/deploy/rpm + </literallayout></para></listitem> + </orderedlist> + </para> + </section> + + <section id='package-server-lighttpd'> + <title>Serving Packages via lighttpd</title> + + <para> + If you are using lighttpd, all you need + to do is to provide a link from your + <filename>${TMPDIR}/deploy/<replaceable>package-format</replaceable></filename> + directory to lighttpd's document-root. + You can determine the specifics of your lighttpd + installation by looking through its configuration file, + which is usually found at: + <filename>/etc/lighttpd/lighttpd.conf</filename>. + </para> + + <para> + For example, if you are using IPK, lighttpd's + document-root is set to + <filename>/var/www/lighttpd</filename>, and you had + packages for a target named "BOARD", + then you might create a link from your build location + to lighttpd's document-root as follows: + <literallayout class='monospaced'> + # ln -s $(PWD)/tmp/deploy/ipk /var/www/lighttpd/BOARD-dir + </literallayout> + </para> + + <para> + At this point, you need to start the lighttpd server. + The method used to start the server varies by + distribution. + However, one basic method that starts it by hand is: + <literallayout class='monospaced'> + # lighttpd -f /etc/lighttpd/lighttpd.conf + </literallayout> + </para> + </section> + </section> + + <section id='runtime-package-management-target'> + <title>Target Setup</title> + + <para> + Setting up the target differs depending on the + package management system. + This section provides information for RPM and IPK. + </para> + + <section id='runtime-package-management-target-rpm'> + <title>Using RPM</title> + + <para> + The application for performing runtime package + management of RPM packages on the target is called + <filename>smart</filename>. + </para> + + <para> + On the target machine, you need to inform + <filename>smart</filename> of every package database + you want to use. + As an example, suppose your target device can use the + following three package databases from a server named + <filename>server.name</filename>: + <filename>all</filename>, <filename>i586</filename>, + and <filename>qemux86</filename>. + Given this example, issue the following commands on the + target: + <literallayout class='monospaced'> + # smart channel --add all type=rpm-md baseurl=http://server.name/rpm/all + # smart channel --add i585 type=rpm-md baseurl=http://server.name/rpm/i586 + # smart channel --add qemux86 type=rpm-md baseurl=http://server.name/rpm/qemux86 + </literallayout> + Also from the target machine, fetch the repository + information using this command: + <literallayout class='monospaced'> + # smart update + </literallayout> + You can now use the <filename>smart query</filename> + and <filename>smart install</filename> commands to + find and install packages from the repositories. + </para> + </section> + + <section id='runtime-package-management-target-ipk'> + <title>Using IPK</title> + + <para> + The application for performing runtime package + management of IPK packages on the target is called + <filename>opkg</filename>. + </para> + + <para> + In order to inform <filename>opkg</filename> of the + package databases you want to use, simply create one + or more <filename>*.conf</filename> files in the + <filename>/etc/opkg</filename> directory on the target. + The <filename>opkg</filename> application uses them + to find its available package databases. + As an example, suppose you configured your HTTP server + on your machine named + <filename>www.mysite.com</filename> to serve files + from a <filename>BOARD-dir</filename> directory under + its document-root. + In this case, you might create a configuration + file on the target called + <filename>/etc/opkg/base-feeds.conf</filename> that + contains: + <literallayout class='monospaced'> + src/gz all http://www.mysite.com/BOARD-dir/all + src/gz armv7a http://www.mysite.com/BOARD-dir/armv7a + src/gz beaglebone http://www.mysite.com/BOARD-dir/beaglebone + </literallayout> + </para> + + <para> + As a way of making it easier to generate and make + these IPK configuration files available on your + target, simply define + <ulink url='&YOCTO_DOCS_REF_URL;#var-FEED_DEPLOYDIR_BASE_URI'><filename>FEED_DEPLOYDIR_BASE_URI</filename></ulink> + to point to your server and the location within the + document-root which contains the databases. + For example: if you are serving your packages over + HTTP, your server's IP address is 192.168.7.1, and + your databases are located in a directory called + <filename>BOARD-dir</filename> underneath your HTTP + server's document-root, you need to set + <filename>FEED_DEPLOYDIR_BASE_URI</filename> to + <filename>http://192.168.7.1/BOARD-dir</filename> and + a set of configuration files will be generated for you + in your target to work with this feed. + </para> + + <para> + On the target machine, fetch (or refresh) the + repository information using this command: + <literallayout class='monospaced'> + # opkg update + </literallayout> + You can now use the <filename>opkg list</filename> and + <filename>opkg install</filename> commands to find and + install packages from the repositories. + </para> + </section> + </section> + </section> + + <section id='testing-packages-with-ptest'> + <title>Testing Packages With ptest</title> + + <para> + A Package Test (ptest) runs tests against packages built + by the OpenEmbedded build system on the target machine. + A ptest contains at least two items: the actual test, and + a shell script (<filename>run-ptest</filename>) that starts + the test. + The shell script that starts the test must not contain + the actual test - the script only starts the test. + On the other hand, the test can be anything from a simple + shell script that runs a binary and checks the output to + an elaborate system of test binaries and data files. + </para> + + <para> + The test generates output in the format used by + Automake: + <literallayout class='monospaced'> + <replaceable>result</replaceable>: <replaceable>testname</replaceable> + </literallayout> + where the result can be <filename>PASS</filename>, + <filename>FAIL</filename>, or <filename>SKIP</filename>, + and the testname can be any identifying string. + </para> + + <para> + For a list of Yocto Project recipes that are already + enabled with ptest, see the + <ulink url='https://wiki.yoctoproject.org/wiki/Ptest'>Ptest</ulink> + wiki page. + <note> + A recipe is "ptest-enabled" if it inherits the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink> + class. + </note> + </para> + + <section id='adding-ptest-to-your-build'> + <title>Adding ptest to Your Build</title> + + <para> + To add package testing to your build, add the + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> + and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> + variables to your <filename>local.conf</filename> file, + which is found in the + <link linkend='build-directory'>Build Directory</link>: + <literallayout class='monospaced'> + DISTRO_FEATURES_append = " ptest" + EXTRA_IMAGE_FEATURES += "ptest-pkgs" + </literallayout> + Once your build is complete, the ptest files are installed + into the + <filename>/usr/lib/<replaceable>package</replaceable>/ptest</filename> + directory within the image, where + <filename><replaceable>package</replaceable></filename> + is the name of the package. + </para> + </section> + + <section id='running-ptest'> + <title>Running ptest</title> + + <para> + The <filename>ptest-runner</filename> package installs a + shell script that loops through all installed ptest test + suites and runs them in sequence. + Consequently, you might want to add this package to + your image. + </para> + </section> + + <section id='getting-your-package-ready'> + <title>Getting Your Package Ready</title> + + <para> + In order to enable a recipe to run installed ptests + on target hardware, + you need to prepare the recipes that build the packages + you want to test. + Here is what you have to do for each recipe: + <itemizedlist> + <listitem><para><emphasis>Be sure the recipe + inherits the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink> + class:</emphasis> + Include the following line in each recipe: + <literallayout class='monospaced'> + inherit ptest + </literallayout> + </para></listitem> + <listitem><para><emphasis>Create <filename>run-ptest</filename>:</emphasis> + This script starts your test. + Locate the script where you will refer to it + using + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>. + Here is an example that starts a test for + <filename>dbus</filename>: + <literallayout class='monospaced'> + #!/bin/sh + cd test + make -k runtest-TESTS + </literallayout> + </para></listitem> + <listitem><para><emphasis>Ensure dependencies are + met:</emphasis> + If the test adds build or runtime dependencies + that normally do not exist for the package + (such as requiring "make" to run the test suite), + use the + <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> + and + <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink> + variables in your recipe in order for the package + to meet the dependencies. + Here is an example where the package has a runtime + dependency on "make": + <literallayout class='monospaced'> + RDEPENDS_${PN}-ptest += "make" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Add a function to build the + test suite:</emphasis> + Not many packages support cross-compilation of + their test suites. + Consequently, you usually need to add a + cross-compilation function to the package. + </para> + + <para>Many packages based on Automake compile and + run the test suite by using a single command + such as <filename>make check</filename>. + However, the host <filename>make check</filename> + builds and runs on the same computer, while + cross-compiling requires that the package is built + on the host but executed for the target + architecture (though often, as in the case for + ptest, the execution occurs on the host). + The built version of Automake that ships with the + Yocto Project includes a patch that separates + building and execution. + Consequently, packages that use the unaltered, + patched version of <filename>make check</filename> + automatically cross-compiles.</para> + <para>Regardless, you still must add a + <filename>do_compile_ptest</filename> function to + build the test suite. + Add a function similar to the following to your + recipe: + <literallayout class='monospaced'> + do_compile_ptest() { + oe_runmake buildtest-TESTS + } + </literallayout> + </para></listitem> + <listitem><para><emphasis>Ensure special configurations + are set:</emphasis> + If the package requires special configurations + prior to compiling the test code, you must + insert a <filename>do_configure_ptest</filename> + function into the recipe. + </para></listitem> + <listitem><para><emphasis>Install the test + suite:</emphasis> + The <filename>ptest</filename> class + automatically copies the file + <filename>run-ptest</filename> to the target and + then runs make <filename>install-ptest</filename> + to run the tests. + If this is not enough, you need to create a + <filename>do_install_ptest</filename> function and + make sure it gets called after the + "make install-ptest" completes. + </para></listitem> + </itemizedlist> + </para> + </section> + </section> + </section> + + <section id='working-with-source-files'> + <title>Working with Source Files</title> + + <para> + The OpenEmbedded build system works with source files located + through the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + variable. + When you build something using BitBake, a big part of the operation + is locating and downloading all the source tarballs. + For images, downloading all the source for various packages can + take a significant amount of time. + </para> + + <para> + This section presents information for working with source + files that can lead to more efficient use of resources and + time. + </para> + + <section id='setting-up-effective-mirrors'> + <title>Setting up Effective Mirrors</title> + + <para> + As mentioned, a good deal that goes into a Yocto Project + build is simply downloading all of the source tarballs. + Maybe you have been working with another build system + (OpenEmbedded or Angstrom) for which you have built up a + sizable directory of source tarballs. + Or, perhaps someone else has such a directory for which you + have read access. + If so, you can save time by adding statements to your + configuration file so that the build process checks local + directories first for existing tarballs before checking the + Internet. + </para> + + <para> + Here is an efficient way to set it up in your + <filename>local.conf</filename> file: + <literallayout class='monospaced'> + SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/" + INHERIT += "own-mirrors" + BB_GENERATE_MIRROR_TARBALLS = "1" + # BB_NO_NETWORK = "1" + </literallayout> + </para> + + <para> + In the previous example, the + <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink> + variable causes the OpenEmbedded build system to generate + tarballs of the Git repositories and store them in the + <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> + directory. + Due to performance reasons, generating and storing these + tarballs is not the build system's default behavior. + </para> + + <para> + You can also use the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PREMIRRORS'><filename>PREMIRRORS</filename></ulink> + variable. + For an example, see the variable's glossary entry in the + Yocto Project Reference Manual. + </para> + </section> + + <section id='getting-source-files-and-suppressing-the-build'> + <title>Getting Source Files and Suppressing the Build</title> + + <para> + Another technique you can use to ready yourself for a + successive string of build operations, is to pre-fetch + all the source files without actually starting a build. + This technique lets you work through any download issues + and ultimately gathers all the source files into your + download directory + <ulink url='&YOCTO_DOCS_REF_URL;#structure-build-downloads'><filename>build/downloads</filename></ulink>, + which is located with + <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>. + </para> + + <para> + Use the following BitBake command form to fetch all the + necessary sources without starting the build: + <literallayout class='monospaced'> + $ bitbake -c fetchall <replaceable>target</replaceable> + </literallayout> + This variation of the BitBake command guarantees that you + have all the sources for that BitBake target should you + disconnect from the Internet and want to do the build + later offline. + </para> + </section> + </section> + + <section id="building-software-from-an-external-source"> + <title>Building Software from an External Source</title> + + <para> + By default, the OpenEmbedded build system uses the + <link linkend='build-directory'>Build Directory</link> when + building source code. + The build process involves fetching the source files, unpacking + them, and then patching them if necessary before the build takes + place. + </para> + + <para> + Situations exist where you might want to build software from source + files that are external to and thus outside of the + OpenEmbedded build system. + For example, suppose you have a project that includes a new BSP with + a heavily customized kernel. + And, you want to minimize exposing the build system to the + development team so that they can focus on their project and + maintain everyone's workflow as much as possible. + In this case, you want a kernel source directory on the development + machine where the development occurs. + You want the recipe's + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + variable to point to the external directory and use it as is, not + copy it. + </para> + + <para> + To build from software that comes from an external source, all you + need to do is inherit the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink> + class and then set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink> + variable to point to your external source code. + Here are the statements to put in your + <filename>local.conf</filename> file: + <literallayout class='monospaced'> + INHERIT += "externalsrc" + EXTERNALSRC_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>" + </literallayout> + </para> + + <para> + This next example shows how to accomplish the same thing by setting + <filename>EXTERNALSRC</filename> in the recipe itself or in the + recipe's append file: + <literallayout class='monospaced'> + EXTERNALSRC = "<replaceable>path</replaceable>" + EXTERNALSRC_BUILD = "<replaceable>path</replaceable>" + </literallayout> + <note> + In order for these settings to take effect, you must globally + or locally inherit the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink> + class. + </note> + </para> + + <para> + By default, <filename>externalsrc.bbclass</filename> builds + the source code in a directory separate from the external source + directory as specified by + <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>. + If you need to have the source built in the same directory in + which it resides, or some other nominated directory, you can set + <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD'><filename>EXTERNALSRC_BUILD</filename></ulink> + to point to that directory: + <literallayout class='monospaced'> + EXTERNALSRC_BUILD_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>" + </literallayout> + </para> + </section> + + <section id="selecting-an-initialization-manager"> + <title>Selecting an Initialization Manager</title> + + <para> + By default, the Yocto Project uses SysVinit as the initialization + manager. + However, support also exists for systemd, + which is a full replacement for init with + parallel starting of services, reduced shell overhead and other + features that are used by many distributions. + </para> + + <para> + If you want to use SysVinit, you do + not have to do anything. + But, if you want to use systemd, you must + take some steps as described in the following sections. + </para> + + <section id='using-systemd-exclusively'> + <title>Using systemd Exclusively</title> + + <para> + Set the these variables in your distribution configuration + file as follows: + <literallayout class='monospaced'> + DISTRO_FEATURES_append = " systemd" + VIRTUAL-RUNTIME_init_manager = "systemd" + </literallayout> + You can also prevent the SysVinit + distribution feature from + being automatically enabled as follows: + <literallayout class='monospaced'> + DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit" + </literallayout> + Doing so removes any redundant SysVinit scripts. + </para> + + <para> + To remove initscripts from your image altogether, + set this variable also: + <literallayout class='monospaced'> + VIRTUAL-RUNTIME_initscripts = "" + </literallayout> + </para> + + <para> + For information on the backfill variable, see + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>. + </para> + </section> + + <section id='using-systemd-for-the-main-image-and-using-sysvinit-for-the-rescue-image'> + <title>Using systemd for the Main Image and Using SysVinit for the Rescue Image</title> + + <para> + Set these variables in your distribution configuration + file as follows: + <literallayout class='monospaced'> + DISTRO_FEATURES_append = " systemd" + VIRTUAL-RUNTIME_init_manager = "systemd" + </literallayout> + Doing so causes your main image to use the + <filename>packagegroup-core-boot.bb</filename> recipe and + systemd. + The rescue/minimal image cannot use this package group. + However, it can install SysVinit + and the appropriate packages will have support for both + systemd and SysVinit. + </para> + </section> + </section> + + <section id="selecting-dev-manager"> + <title>Selecting a Device Manager</title> + + <para> + The Yocto Project provides multiple ways to manage the device + manager (<filename>/dev</filename>): + <itemizedlist> + <listitem><para><emphasis>Persistent and Pre-Populated<filename>/dev</filename>:</emphasis> + For this case, the <filename>/dev</filename> directory + is persistent and the required device nodes are created + during the build. + </para></listitem> + <listitem><para><emphasis>Use <filename>devtmpfs</filename> with a Device Manager:</emphasis> + For this case, the <filename>/dev</filename> directory + is provided by the kernel as an in-memory file system and + is automatically populated by the kernel at runtime. + Additional configuration of device nodes is done in user + space by a device manager like + <filename>udev</filename> or + <filename>busybox-mdev</filename>. + </para></listitem> + </itemizedlist> + </para> + + <section id="static-dev-management"> + <title>Using Persistent and Pre-Populated<filename>/dev</filename></title> + + <para> + To use the static method for device population, you need to + set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink> + variable to "0" as follows: + <literallayout class='monospaced'> + USE_DEVFS = "0" + </literallayout> + </para> + + <para> + The content of the resulting <filename>/dev</filename> + directory is defined in a Device Table file. + The + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_DEVICE_TABLES'><filename>IMAGE_DEVICE_TABLES</filename></ulink> + variable defines the Device Table to use and should be set + in the machine or distro configuration file. + Alternatively, you can set this variable in your + <filename>local.conf</filename> configuration file. + </para> + + <para> + If you do not define the + <filename>IMAGE_DEVICE_TABLES</filename> variable, the default + <filename>device_table-minimal.txt</filename> is used: + <literallayout class='monospaced'> + IMAGE_DEVICE_TABLES = "device_table-mymachine.txt" + </literallayout> + </para> + + <para> + The population is handled by the <filename>makedevs</filename> + utility during image creation: + </para> + </section> + + <section id="devtmpfs-dev-management"> + <title>Using <filename>devtmpfs</filename> and a Device Manager</title> + + <para> + To use the dynamic method for device population, you need to + use (or be sure to set) the + <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink> + variable to "1", which is the default: + <literallayout class='monospaced'> + USE_DEVFS = "1" + </literallayout> + With this setting, the resulting <filename>/dev</filename> + directory is populated by the kernel using + <filename>devtmpfs</filename>. + Make sure the corresponding kernel configuration variable + <filename>CONFIG_DEVTMPFS</filename> is set when building + you build a Linux kernel. + </para> + + <para> + All devices created by <filename>devtmpfs</filename> will be + owned by <filename>root</filename> and have permissions + <filename>0600</filename>. + </para> + + <para> + To have more control over the device nodes, you can use a + device manager like <filename>udev</filename> or + <filename>busybox-mdev</filename>. + You choose the device manager by defining the + <filename>VIRTUAL-RUNTIME_dev_manager</filename> variable + in your machine or distro configuration file. + Alternatively, you can set this variable in your + <filename>local.conf</filename> configuration file: + <literallayout class='monospaced'> + VIRTUAL-RUNTIME_dev_manager = "udev" + + # Some alternative values + # VIRTUAL-RUNTIME_dev_manager = "busybox-mdev" + # VIRTUAL-RUNTIME_dev_manager = "systemd" + </literallayout> + </para> + </section> + </section> + + <section id="platdev-appdev-srcrev"> + <title>Using an External SCM</title> + + <para> + If you're working on a recipe that pulls from an external Source + Code Manager (SCM), it is possible to have the OpenEmbedded build + system notice new recipe changes added to the SCM and then build + the resulting packages that depend on the new recipes by using + the latest versions. + This only works for SCMs from which it is possible to get a + sensible revision number for changes. + Currently, you can do this with Apache Subversion (SVN), Git, and + Bazaar (BZR) repositories. + </para> + + <para> + To enable this behavior, the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> + of the recipe needs to reference + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>. + Here is an example: + <literallayout class='monospaced'> + PV = "1.2.3+git${SRCPV}" + </literallayout> + Then, you can add the following to your + <filename>local.conf</filename>: + <literallayout class='monospaced'> + SRCREV_pn-<replaceable>PN</replaceable> = "${AUTOREV}" + </literallayout> + <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> + is the name of the recipe for which you want to enable automatic source + revision updating. + </para> + + <para> + If you do not want to update your local configuration file, you can + add the following directly to the recipe to finish enabling + the feature: + <literallayout class='monospaced'> + SRCREV = "${AUTOREV}" + </literallayout> + </para> + + <para> + The Yocto Project provides a distribution named + <filename>poky-bleeding</filename>, whose configuration + file contains the line: + <literallayout class='monospaced'> + require conf/distro/include/poky-floating-revisions.inc + </literallayout> + This line pulls in the listed include file that contains + numerous lines of exactly that form: + <literallayout class='monospaced'> + #SRCREV_pn-opkg-native ?= "${AUTOREV}" + #SRCREV_pn-opkg-sdk ?= "${AUTOREV}" + #SRCREV_pn-opkg ?= "${AUTOREV}" + #SRCREV_pn-opkg-utils-native ?= "${AUTOREV}" + #SRCREV_pn-opkg-utils ?= "${AUTOREV}" + SRCREV_pn-gconf-dbus ?= "${AUTOREV}" + SRCREV_pn-matchbox-common ?= "${AUTOREV}" + SRCREV_pn-matchbox-config-gtk ?= "${AUTOREV}" + SRCREV_pn-matchbox-desktop ?= "${AUTOREV}" + SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}" + SRCREV_pn-matchbox-panel-2 ?= "${AUTOREV}" + SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}" + SRCREV_pn-matchbox-terminal ?= "${AUTOREV}" + SRCREV_pn-matchbox-wm ?= "${AUTOREV}" + SRCREV_pn-settings-daemon ?= "${AUTOREV}" + SRCREV_pn-screenshot ?= "${AUTOREV}" + . + . + . + </literallayout> + These lines allow you to experiment with building a + distribution that tracks the latest development source + for numerous packages. + <note><title>Caution</title> + The <filename>poky-bleeding</filename> distribution + is not tested on a regular basis. + Keep this in mind if you use it. + </note> + </para> + </section> + + <section id='creating-a-read-only-root-filesystem'> + <title>Creating a Read-Only Root Filesystem</title> + + <para> + Suppose, for security reasons, you need to disable + your target device's root filesystem's write permissions + (i.e. you need a read-only root filesystem). + Or, perhaps you are running the device's operating system + from a read-only storage device. + For either case, you can customize your image for + that behavior. + </para> + + <note> + Supporting a read-only root filesystem requires that the system and + applications do not try to write to the root filesystem. + You must configure all parts of the target system to write + elsewhere, or to gracefully fail in the event of attempting to + write to the root filesystem. + </note> + + <section id='creating-the-root-filesystem'> + <title>Creating the Root Filesystem</title> + + <para> + To create the read-only root filesystem, simply add the + "read-only-rootfs" feature to your image. + Using either of the following statements in your + image recipe or from within the + <filename>local.conf</filename> file found in the + <link linkend='build-directory'>Build Directory</link> + causes the build system to create a read-only root filesystem: + <literallayout class='monospaced'> + IMAGE_FEATURES = "read-only-rootfs" + </literallayout> + or + <literallayout class='monospaced'> + EXTRA_IMAGE_FEATURES += "read-only-rootfs" + </literallayout> + </para> + + <para> + For more information on how to use these variables, see the + "<link linkend='usingpoky-extend-customimage-imagefeatures'>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and <filename>EXTRA_IMAGE_FEATURES</filename></link>" + section. + For information on the variables, see + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> + and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>. + </para> + </section> + + <section id='post-installation-scripts'> + <title>Post-Installation Scripts</title> + + <para> + It is very important that you make sure all + post-Installation (<filename>pkg_postinst</filename>) scripts + for packages that are installed into the image can be run + at the time when the root filesystem is created during the + build on the host system. + These scripts cannot attempt to run during first-boot on the + target device. + With the "read-only-rootfs" feature enabled, + the build system checks during root filesystem creation to make + sure all post-installation scripts succeed. + If any of these scripts still need to be run after the root + filesystem is created, the build immediately fails. + These build-time checks ensure that the build fails + rather than the target device fails later during its + initial boot operation. + </para> + + <para> + Most of the common post-installation scripts generated by the + build system for the out-of-the-box Yocto Project are engineered + so that they can run during root filesystem creation + (e.g. post-installation scripts for caching fonts). + However, if you create and add custom scripts, you need + to be sure they can be run during this file system creation. + </para> + + <para> + Here are some common problems that prevent + post-installation scripts from running during root filesystem + creation: + <itemizedlist> + <listitem><para> + <emphasis>Not using $D in front of absolute + paths:</emphasis> + The build system defines + <filename>$</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink> + when the root filesystem is created. + Furthermore, <filename>$D</filename> is blank when the + script is run on the target device. + This implies two purposes for <filename>$D</filename>: + ensuring paths are valid in both the host and target + environments, and checking to determine which + environment is being used as a method for taking + appropriate actions. + </para></listitem> + <listitem><para> + <emphasis>Attempting to run processes that are + specific to or dependent on the target + architecture:</emphasis> + You can work around these attempts by using native + tools, which run on the host system, + to accomplish the same tasks, or + by alternatively running the processes under QEMU, + which has the <filename>qemu_run_binary</filename> + function. + For more information, see the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-qemu'><filename>qemu</filename></ulink> + class.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id='areas-with-write-access'> + <title>Areas With Write Access</title> + + <para> + With the "read-only-rootfs" feature enabled, + any attempt by the target to write to the root filesystem at + runtime fails. + Consequently, you must make sure that you configure processes + and applications that attempt these types of writes do so + to directories with write access (e.g. + <filename>/tmp</filename> or <filename>/var/run</filename>). + </para> + </section> + </section> + + <section id="performing-automated-runtime-testing"> + <title>Performing Automated Runtime Testing</title> + + <para> + The OpenEmbedded build system makes available a series of automated + tests for images to verify runtime functionality. + You can run these tests on either QEMU or actual target hardware. + Tests are written in Python making use of the + <filename>unittest</filename> module, and the majority of them + run commands on the target system over SSH. + This section describes how you set up the environment to use these + tests, run available tests, and write and add your own tests. + </para> + + <section id='enabling-tests'> + <title>Enabling Tests</title> + + <para> + Depending on whether you are planning to run tests using + QEMU or on the hardware, you have to take + different steps to enable the tests. + See the following subsections for information on how to + enable both types of tests. + </para> + + <section id='qemu-image-enabling-tests'> + <title>Enabling Runtime Tests on QEMU</title> + + <para> + In order to run tests, you need to do the following: + <itemizedlist> + <listitem><para><emphasis>Set up to avoid interaction + with <filename>sudo</filename> for networking:</emphasis> + To accomplish this, you must do one of the + following: + <itemizedlist> + <listitem><para>Add + <filename>NOPASSWD</filename> for your user + in <filename>/etc/sudoers</filename> either for + all commands or just for + <filename>runqemu-ifup</filename>. + You must provide the full path as that can + change if you are using multiple clones of the + source repository. + <note> + On some distributions, you also need to + comment out "Defaults requiretty" in + <filename>/etc/sudoers</filename>. + </note></para></listitem> + <listitem><para>Manually configure a tap interface + for your system.</para></listitem> + <listitem><para>Run as root the script in + <filename>scripts/runqemu-gen-tapdevs</filename>, + which should generate a list of tap devices. + This is the option typically chosen for + Autobuilder-type environments. + </para></listitem> + </itemizedlist></para></listitem> + <listitem><para><emphasis>Set the + <filename>DISPLAY</filename> variable:</emphasis> + You need to set this variable so that you have an X + server available (e.g. start + <filename>vncserver</filename> for a headless machine). + </para></listitem> + <listitem><para><emphasis>Be sure your host's firewall + accepts incoming connections from + 192.168.7.0/24:</emphasis> + Some of the tests (in particular smart tests) start an + HTTP server on a random high number port, which is + used to serve files to the target. + The smart module serves + <filename>${DEPLOY_DIR}/rpm</filename> so it can run + smart channel commands. That means your host's firewall + must accept incoming connections from 192.168.7.0/24, + which is the default IP range used for tap devices + by <filename>runqemu</filename>.</para></listitem> + <listitem><para><emphasis>Be sure your host has the + correct packages installed:</emphasis> + Depending your host's distribution, you need + to have the following packages installed: + <itemizedlist> + <listitem><para>Ubuntu and Debian: + <filename>sysstat</filename> and + <filename>iproute2</filename> + </para></listitem> + <listitem><para>OpenSUSE: + <filename>sysstat</filename> and + <filename>iproute2</filename> + </para></listitem> + <listitem><para>Fedora: + <filename>sysstat</filename> and + <filename>iproute</filename> + </para></listitem> + <listitem><para>CentOS: + <filename>sysstat</filename> and + <filename>iproute</filename> + </para></listitem> + </itemizedlist> + </para></listitem> + </itemizedlist> + </para> + + <para> + Once you start running the tests, the following happens: + <orderedlist> + <listitem><para>A copy of the root filesystem is written + to <filename>${WORKDIR}/testimage</filename>. + </para></listitem> + <listitem><para>The image is booted under QEMU using the + standard <filename>runqemu</filename> script. + </para></listitem> + <listitem><para>A default timeout of 500 seconds occurs + to allow for the boot process to reach the login prompt. + You can change the timeout period by setting + <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink> + in the <filename>local.conf</filename> file. + </para></listitem> + <listitem><para>Once the boot process is reached and the + login prompt appears, the tests run. + The full boot log is written to + <filename>${WORKDIR}/testimage/qemu_boot_log</filename>. + </para></listitem> + <listitem><para>Each test module loads in the order found + in <filename>TEST_SUITES</filename>. + You can find the full output of the commands run over + SSH in + <filename>${WORKDIR}/testimgage/ssh_target_log</filename>. + </para></listitem> + <listitem><para>If no failures occur, the task running the + tests ends successfully. + You can find the output from the + <filename>unittest</filename> in the task log at + <filename>${WORKDIR}/temp/log.do_testimage</filename>. + </para></listitem> + </orderedlist> + </para> + </section> + + <section id='hardware-image-enabling-tests'> + <title>Enabling Runtime Tests on Hardware</title> + + <para> + The OpenEmbedded build system can run tests on real + hardware, and for certain devices it can also deploy + the image to be tested onto the device beforehand. + </para> + + <para> + For automated deployment, a "master image" is installed + onto the hardware once as part of setup. + Then, each time tests are to be run, the following + occurs: + <orderedlist> + <listitem><para>The master image is booted into and + used to write the image to be tested to + a second partition. + </para></listitem> + <listitem><para>The device is then rebooted using an + external script that you need to provide. + </para></listitem> + <listitem><para>The device boots into the image to be + tested. + </para></listitem> + </orderedlist> + </para> + + <para> + When running tests (independent of whether the image + has been deployed automatically or not), the device is + expected to be connected to a network on a + pre-determined IP address. + You can either use static IP addresses written into + the image, or set the image to use DHCP and have your + DHCP server on the test network assign a known IP address + based on the MAC address of the device. + </para> + + <para> + In order to run tests on hardware, you need to set + <filename>TEST_TARGET</filename> to an appropriate value. + For QEMU, you do not have to change anything, the default + value is "QemuTarget". + For running tests on hardware, the following options exist: + <itemizedlist> + <listitem><para><emphasis>"SimpleRemoteTarget":</emphasis> + Choose "SimpleRemoteTarget" if you are going to + run tests on a target system that is already + running the image to be tested and is available + on the network. + You can use "SimpleRemoteTarget" in conjunction + with either real hardware or an image running + within a separately started QEMU or any + other virtual machine manager. + </para></listitem> + <listitem><para><emphasis>"GummibootTarget":</emphasis> + Choose "GummibootTarget" if your hardware is + an EFI-based machine with + <filename>gummiboot</filename> as bootloader and + <filename>core-image-testmaster</filename> + (or something similar) is installed. + Also, your hardware under test must be in a + DHCP-enabled network that gives it the same IP + address for each reboot.</para> + <para>If you choose "GummibootTarget", there are + additional requirements and considerations. + See the + "<link linkend='selecting-gummiboottarget'>Selecting GummibootTarget</link>" + section, which follows, for more information. + </para></listitem> + <listitem><para><emphasis>"BeagleBoneTarget":</emphasis> + Choose "BeagleBoneTarget" if you are deploying + images and running tests on the BeagleBone + "Black" or original "White" hardware. + For information on how to use these tests, see the + comments at the top of the BeagleBoneTarget + <filename>meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py</filename> + file. + </para></listitem> + <listitem><para><emphasis>"EdgeRouterTarget":</emphasis> + Choose "EdgeRouterTarget" is you are deploying + images and running tests on the Ubiquiti Networks + EdgeRouter Lite. + For information on how to use these tests, see the + comments at the top of the EdgeRouterTarget + <filename>meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py</filename> + file. + </para></listitem> + <listitem><para><emphasis>"GrubTarget":</emphasis> + Choose the "supports deploying images and running + tests on any generic PC that boots using GRUB. + For information on how to use these tests, see the + comments at the top of the GrubTarget + <filename>meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py</filename> + file. + </para></listitem> + <listitem><para><emphasis>"<replaceable>your-target</replaceable>":</emphasis> + Create your own custom target if you want to run + tests when you are deploying images and running + tests on a custom machine within your BSP layer. + To do this, you need to add a Python unit that + defines the target class under + <filename>lib/oeqa/controllers/</filename> within + your layer. + You must also provide an empty + <filename>__init__.py</filename>. + For examples, see files in + <filename>meta-yocto-bsp/lib/oeqa/controllers/</filename>. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='selecting-gummiboottarget'> + <title>Selecting GummibootTarget</title> + + <para> + If you did not set <filename>TEST_TARGET</filename> to + "GummibootTarget", then you do not need any information + in this section. + You can skip down to the + "<link linkend='qemu-image-running-tests'>Running Tests</link>" + section. + </para> + + <para> + If you did set <filename>TEST_TARGET</filename> to + "GummibootTarget", you also need to perform a one-time + setup of your master image by doing the following: + <orderedlist> + <listitem><para><emphasis>Set <filename>EFI_PROVIDER</filename>:</emphasis> + Be sure that <filename>EFI_PROVIDER</filename> + is as follows: + <literallayout class='monospaced'> + EFI_PROVIDER = "gummiboot" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Build the master image:</emphasis> + Build the <filename>core-image-testmaster</filename> + image. + The <filename>core-image-testmaster</filename> + recipe is provided as an example for a + "master" image and you can customize the image + recipe as you would any other recipe. + </para> + <para>Here are the image recipe requirements: + <itemizedlist> + <listitem><para>Inherits + <filename>core-image</filename> + so that kernel modules are installed. + </para></listitem> + <listitem><para>Installs normal linux utilities + not busybox ones (e.g. + <filename>bash</filename>, + <filename>coreutils</filename>, + <filename>tar</filename>, + <filename>gzip</filename>, and + <filename>kmod</filename>). + </para></listitem> + <listitem><para>Uses a custom + Initial RAM Disk (initramfs) image with a + custom installer. + A normal image that you can install usually + creates a single rootfs partition. + This image uses another installer that + creates a specific partition layout. + Not all Board Support Packages (BSPs) + can use an installer. + For such cases, you need to manually create + the following partition layout on the + target: + <itemizedlist> + <listitem><para>First partition mounted + under <filename>/boot</filename>, + labeled "boot". + </para></listitem> + <listitem><para>The main rootfs + partition where this image gets + installed, which is mounted under + <filename>/</filename>. + </para></listitem> + <listitem><para>Another partition + labeled "testrootfs" where test + images get deployed. + </para></listitem> + </itemizedlist> + </para></listitem> + </itemizedlist> + </para></listitem> + <listitem><para><emphasis>Install image:</emphasis> + Install the image that you just built on the target + system. + </para></listitem> + </orderedlist> + </para> + + <para> + The final thing you need to do when setting + <filename>TEST_TARGET</filename> to "GummibootTarget" is + to set up the test image: + <orderedlist> + <listitem><para><emphasis>Set up your <filename>local.conf</filename> file:</emphasis> + Make sure you have the following statements in + your <filename>local.conf</filename> file: + <literallayout class='monospaced'> + IMAGE_FSTYPES += "tar.gz" + INHERIT += "testimage" + TEST_TARGET = "GummibootTarget" + TEST_TARGET_IP = "192.168.2.3" + </literallayout> + </para></listitem> + <listitem><para><emphasis>Build your test image:</emphasis> + Use BitBake to build the image: + <literallayout class='monospaced'> + $ bitbake core-image-sato + </literallayout> + </para></listitem> + </orderedlist> + </para> + </section> + + <section id='power-control'> + <title>Power Control</title> + + <para> + For most hardware targets other than SimpleRemoteTarget, + you can control power: + <itemizedlist> + <listitem><para> + You can use + <filename>TEST_POWERCONTROL_CMD</filename> + together with + <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename> + as a command that runs on the host and does power + cycling. + The test code passes one argument to that command: + off, on or cycle (off then on). + Here is an example that could appear in your + <filename>local.conf</filename> file: + <literallayout class='monospaced'> + TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1" + </literallayout> + In this example, the expect script does the + following: + <literallayout class='monospaced'> + ssh test@10.11.12.1 "pyctl nuc1 <replaceable>arg</replaceable>" + </literallayout> + It then runs a Python script that controls power + for a label called <filename>nuc1</filename>. + <note> + You need to customize + <filename>TEST_POWERCONTROL_CMD</filename> + and + <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename> + for your own setup. + The one requirement is that it accepts + "on", "off", and "cycle" as the last argument. + </note> + </para></listitem> + <listitem><para> + When no command is defined, it connects to the + device over SSH and uses the classic reboot command + to reboot the device. + Classic reboot is fine as long as the machine + actually reboots (i.e. the SSH test has not + failed). + It is useful for scenarios where you have a simple + setup, typically with a single board, and where + some manual interaction is okay from time to time. + </para></listitem> + </itemizedlist> + If you have no hardware to automatically perform power + control but still wish to experiment with automated + hardware testing, you can use the dialog-power-control + script that shows a dialog prompting you to perform the + required power action. + This script requires either KDialog or Zenity to be + installed. + To use this script, set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_POWERCONTROL_CMD'><filename>TEST_POWERCONTROL_CMD</filename></ulink> + variable as follows: + <literallayout class='monospaced'> + TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control" + </literallayout> + </para> + </section> + + <section id='serial-console-connection'> + <title>Serial Console Connection</title> + + <para> + For test target classes requiring a serial console + to interact with the bootloader (e.g. BeagleBoneTarget, + EdgeRouterTarget, and GrubTarget), you need to + specify a command to use to connect to the serial console + of the target machine by using the + <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_CMD'><filename>TEST_SERIALCONTROL_CMD</filename></ulink> + variable and optionally the + <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_EXTRA_ARGS'><filename>TEST_SERIALCONTROL_EXTRA_ARGS</filename></ulink> + variable. + </para> + + <para> + These cases could be a serial terminal program if the + machine is connected to a local serial port, or a + <filename>telnet</filename> or + <filename>ssh</filename> command connecting to a remote + console server. + Regardless of the case, the command simply needs to + connect to the serial console and forward that connection + to standard input and output as any normal terminal + program does. + For example, to use the picocom terminal program on + serial device <filename>/dev/ttyUSB0</filename> + at 115200bps, you would set the variable as follows: + <literallayout class='monospaced'> + TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" + </literallayout> + For local devices where the serial port device disappears + when the device reboots, an additional "serdevtry" wrapper + script is provided. + To use this wrapper, simply prefix the terminal command + with + <filename>${COREBASE}/scripts/contrib/serdevtry</filename>: + <literallayout class='monospaced'> + TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b +115200 /dev/ttyUSB0" + </literallayout> + </para> + </section> + </section> + + <section id="qemu-image-running-tests"> + <title>Running Tests</title> + + <para> + You can start the tests automatically or manually: + <itemizedlist> + <listitem><para><emphasis>Automatically running tests:</emphasis> + To run the tests automatically after the + OpenEmbedded build system successfully creates an image, + first set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink> + variable to "1" in your <filename>local.conf</filename> + file in the + <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>: + <literallayout class='monospaced'> + TEST_IMAGE = "1" + </literallayout> + Next, build your image. + If the image successfully builds, the tests will be + run: + <literallayout class='monospaced'> + bitbake core-image-sato + </literallayout></para></listitem> + <listitem><para><emphasis>Manually running tests:</emphasis> + To manually run the tests, first globally inherit the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink> + class by editing your <filename>local.conf</filename> + file: + <literallayout class='monospaced'> + INHERIT += "testimage" + </literallayout> + Next, use BitBake to run the tests: + <literallayout class='monospaced'> + bitbake -c testimage <replaceable>image</replaceable> + </literallayout></para></listitem> + </itemizedlist> + </para> + + <para> + All test files reside in + <filename>meta/lib/oeqa/runtime</filename> in the + <link linkend='source-directory'>Source Directory</link>. + A test name maps directly to a Python module. + Each test module may contain a number of individual tests. + Tests are usually grouped together by the area + tested (e.g tests for systemd reside in + <filename>meta/lib/oeqa/runtime/systemd.py</filename>). + </para> + + <para> + You can add tests to any layer provided you place them in the + proper area and you extend + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink> + in the <filename>local.conf</filename> file as normal. + Be sure that tests reside in + <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename>. + <note> + Be sure that module names do not collide with module names + used in the default set of test modules in + <filename>meta/lib/oeqa/runtime</filename>. + </note> + </para> + + <para> + You can change the set of tests run by appending or overriding + <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink> + variable in <filename>local.conf</filename>. + Each name in <filename>TEST_SUITES</filename> represents a + required test for the image. + Test modules named within <filename>TEST_SUITES</filename> + cannot be skipped even if a test is not suitable for an image + (e.g. running the RPM tests on an image without + <filename>rpm</filename>). + Appending "auto" to <filename>TEST_SUITES</filename> causes the + build system to try to run all tests that are suitable for the + image (i.e. each test module may elect to skip itself). + </para> + + <para> + The order you list tests in <filename>TEST_SUITES</filename> + is important and influences test dependencies. + Consequently, tests that depend on other tests should be added + after the test on which they depend. + For example, since the <filename>ssh</filename> test + depends on the + <filename>ping</filename> test, "ssh" needs to come after + "ping" in the list. + The test class provides no re-ordering or dependency handling. + <note> + Each module can have multiple classes with multiple test + methods. + And, Python <filename>unittest</filename> rules apply. + </note> + </para> + + <para> + Here are some things to keep in mind when running tests: + <itemizedlist> + <listitem><para>The default tests for the image are defined + as: + <literallayout class='monospaced'> + DEFAULT_TEST_SUITES_pn-<replaceable>image</replaceable> = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg" + </literallayout></para></listitem> + <listitem><para>Add your own test to the list of the + by using the following: + <literallayout class='monospaced'> + TEST_SUITES_append = " mytest" + </literallayout></para></listitem> + <listitem><para>Run a specific list of tests as follows: + <literallayout class='monospaced'> + TEST_SUITES = "test1 test2 test3" + </literallayout> + Remember, order is important. + Be sure to place a test that is dependent on another test + later in the order.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id="exporting-tests"> + <title>Exporting Tests</title> + + <para> + You can export tests so that they can run independently of + the build system. + Exporting tests is required if you want to be able to hand + the test execution off to a scheduler. + You can only export tests that are defined in + <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>. + </para> + + <para> + If your image is already built, make sure the following are set + in your <filename>local.conf</filename> file. + Be sure to provide the IP address you need: + <literallayout class='monospaced'> + TEST_EXPORT_ONLY = "1" + TEST_TARGET = "simpleremote" + TEST_TARGET_IP = "192.168.7.2" + TEST_SERVER_IP = "192.168.7.1" + </literallayout> + You can then export the tests with the following: + <literallayout class='monospaced'> + $ bitbake core-image-sato -c testimage + </literallayout> + Exporting the tests places them in the + <link linkend='build-directory'>Build Directory</link> in + <filename>tmp/testimage/core-image-sato</filename>, which + is controlled by the + <filename>TEST_EXPORT_DIR</filename> variable. + </para> + + <para> + You can now run the tests outside of the build environment: + <literallayout class='monospaced'> + $ cd tmp/testimage/core-image-sato + $ ./runexported.py testdata.json + </literallayout> + <note> + This "export" feature does not deploy or boot the target + image. + Your target (be it a Qemu or hardware one) + has to already be up and running when you call + <filename>runexported.py</filename> + </note> + </para> + + <para> + The exported data (i.e. <filename>testdata.json</filename>) + contains paths to the Build Directory. + Thus, the contents of the directory can be moved + to another machine as long as you update some paths in the + JSON. + Usually, you only care about the + <filename>${DEPLOY_DIR}/rpm</filename> directory + (assuming the RPM and Smart tests are enabled). + Consequently, running the tests on other machine + means that you have to move the contents and call + <filename>runexported.py</filename> with + "--deploy-dir <replaceable>path</replaceable>" as + follows: + <literallayout class='monospaced'> + ./runexported.py --deploy-dir /new/path/on/this/machine testdata.json + </literallayout> + <filename>runexported.py</filename> accepts other arguments + as well as described using <filename>--help</filename>. + </para> + </section> + + <section id="qemu-image-writing-new-tests"> + <title>Writing New Tests</title> + + <para> + As mentioned previously, all new test files need to be in the + proper place for the build system to find them. + New tests for additional functionality outside of the core + should be added to the layer that adds the functionality, in + <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename> + (as long as + <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink> + is extended in the layer's + <filename>layer.conf</filename> file as normal). + Just remember the following: + <itemizedlist> + <listitem><para>Filenames need to map directly to test + (module) names. + </para></listitem> + <listitem><para>Do not use module names that + collide with existing core tests. + </para></listitem> + <listitem><para>Minimally, an empty + <filename>__init__.py</filename> file must exist + in the runtime directory. + </para></listitem> + </itemizedlist> + </para> + + <para> + To create a new test, start by copying an existing module + (e.g. <filename>syslog.py</filename> or + <filename>gcc.py</filename> are good ones to use). + Test modules can use code from + <filename>meta/lib/oeqa/utils</filename>, which are helper + classes. + </para> + + <note> + Structure shell commands such that you rely on them and they + return a single code for success. + Be aware that sometimes you will need to parse the output. + See the <filename>df.py</filename> and + <filename>date.py</filename> modules for examples. + </note> + + <para> + You will notice that all test classes inherit + <filename>oeRuntimeTest</filename>, which is found in + <filename>meta/lib/oetest.py</filename>. + This base class offers some helper attributes, which are + described in the following sections: + </para> + + <section id='qemu-image-writing-tests-class-methods'> + <title>Class Methods</title> + + <para> + Class methods are as follows: + <itemizedlist> + <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis> + Returns "True" if <filename>pkg</filename> is in the + installed package list of the image, which is based + on the manifest file that is generated during the + <filename>do_rootfs</filename> task. + </para></listitem> + <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis> + Returns "True" if the feature is in + <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='qemu-image-writing-tests-class-attributes'> + <title>Class Attributes</title> + + <para> + Class attributes are as follows: + <itemizedlist> + <listitem><para><emphasis><filename>pscmd</filename>:</emphasis> + Equals "ps -ef" if <filename>procps</filename> is + installed in the image. + Otherwise, <filename>pscmd</filename> equals + "ps" (busybox). + </para></listitem> + <listitem><para><emphasis><filename>tc</filename>:</emphasis> + The called test context, which gives access to the + following attributes: + <itemizedlist> + <listitem><para><emphasis><filename>d</filename>:</emphasis> + The BitBake datastore, which allows you to + use stuff such as + <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>. + </para></listitem> + <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis> + Used internally. + The tests do not need these. + </para></listitem> + <listitem><para><emphasis><filename>filesdir</filename>:</emphasis> + The absolute path to + <filename>meta/lib/oeqa/runtime/files</filename>, + which contains helper files for tests meant + for copying on the target such as small + files written in C for compilation. + </para></listitem> + <listitem><para><emphasis><filename>target</filename>:</emphasis> + The target controller object used to deploy + and start an image on a particular target + (e.g. QemuTarget, SimpleRemote, and + GummibootTarget). + Tests usually use the following: + <itemizedlist> + <listitem><para><emphasis><filename>ip</filename>:</emphasis> + The target's IP address. + </para></listitem> + <listitem><para><emphasis><filename>server_ip</filename>:</emphasis> + The host's IP address, which is + usually used by the "smart" test + suite. + </para></listitem> + <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis> + The single, most used method. + This command is a wrapper for: + <filename>ssh root@host "cmd"</filename>. + The command returns a tuple: + (status, output), which are what + their names imply - the return code + of "cmd" and whatever output + it produces. + The optional timeout argument + represents the number of seconds the + test should wait for "cmd" to + return. + If the argument is "None", the + test uses the default instance's + timeout period, which is 300 + seconds. + If the argument is "0", the test + runs until the command returns. + </para></listitem> + <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis> + <filename>scp localpath root@ip:remotepath</filename>. + </para></listitem> + <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis> + <filename>scp root@host:remotepath localpath</filename>. + </para></listitem> + </itemizedlist></para></listitem> + </itemizedlist></para></listitem> + </itemizedlist> + </para> + </section> + + <section id='qemu-image-writing-tests-instance-attributes'> + <title>Instance Attributes</title> + + <para> + A single instance attribute exists, which is + <filename>target</filename>. + The <filename>target</filename> instance attribute is + identical to the class attribute of the same name, which + is described in the previous section. + This attribute exists as both an instance and class + attribute so tests can use + <filename>self.target.run(cmd)</filename> in instance + methods instead of + <filename>oeRuntimeTest.tc.target.run(cmd)</filename>. + </para> + </section> + </section> + </section> + + <section id="platdev-gdb-remotedebug"> + <title>Debugging With the GNU Project Debugger (GDB) Remotely</title> + + <para> + GDB allows you to examine running programs, which in turn helps you to understand and fix problems. + It also allows you to perform post-mortem style analysis of program crashes. + GDB is available as a package within the Yocto Project and is + installed in SDK images by default. + See the "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" chapter + in the Yocto Project Reference Manual for a description of these images. + You can find information on GDB at <ulink url="http://sourceware.org/gdb/"/>. + </para> + + <tip> + For best results, install debug (<filename>-dbg</filename>) packages + for the applications you are going to debug. + Doing so makes extra debug symbols available that give you more + meaningful output. + </tip> + + <para> + Sometimes, due to memory or disk space constraints, it is not possible + to use GDB directly on the remote target to debug applications. + These constraints arise because GDB needs to load the debugging information and the + binaries of the process being debugged. + Additionally, GDB needs to perform many computations to locate information such as function + names, variable names and values, stack traces and so forth - even before starting the + debugging process. + These extra computations place more load on the target system and can alter the + characteristics of the program being debugged. + </para> + + <para> + To help get past the previously mentioned constraints, you can use Gdbserver. + Gdbserver runs on the remote target and does not load any debugging information + from the debugged process. + Instead, a GDB instance processes the debugging information that is run on a + remote computer - the host GDB. + The host GDB then sends control commands to Gdbserver to make it stop or start the debugged + program, as well as read or write memory regions of that debugged program. + All the debugging information loaded and processed as well + as all the heavy debugging is done by the host GDB. + Offloading these processes gives the Gdbserver running on the target a chance to remain + small and fast. + <note> + By default, source files are part of the + <filename>*-dbg</filename> packages in order to enable GDB + to show source lines in its output. + You can save further space on the target by setting the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_DEBUG_SPLIT_STYLE'><filename>PACKAGE_DEBUG_SPLIT_STYLE</filename></ulink> + variable to "debug-without-src" so that these packages do not + include the source files. + </note> + </para> + + <para> + Because the host GDB is responsible for loading the debugging information and + for doing the necessary processing to make actual debugging happen, + you have to make sure the host can access the unstripped binaries complete + with their debugging information and also be sure the target is compiled with no optimizations. + The host GDB must also have local access to all the libraries used by the + debugged program. + Because Gdbserver does not need any local debugging information, the binaries on + the remote target can remain stripped. + However, the binaries must also be compiled without optimization + so they match the host's binaries. + </para> + + <para> + To remain consistent with GDB documentation and terminology, the binary being debugged + on the remote target machine is referred to as the "inferior" binary. + For documentation on GDB see the + <ulink url="http://sourceware.org/gdb/documentation/">GDB site</ulink>. + </para> + + <para> + The remainder of this section describes the steps you need to take + to debug using the GNU project debugger. + </para> + + <section id='platdev-gdb-remotedebug-setup'> + <title>Set Up the Cross-Development Debugging Environment</title> + + <para> + Before you can initiate a remote debugging session, you need + to be sure you have set up the cross-development environment, + toolchain, and sysroot. + The <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-intro'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink> + describes this process. + </para> + </section> + + <section id="platdev-gdb-remotedebug-launch-gdbserver"> + <title>Launch Gdbserver on the Target</title> + + <para> + Make sure Gdbserver is installed on the target. + If it is not, install the package + <filename>gdbserver</filename>, which needs the + <filename>libthread-db1</filename> package. + </para> + + <para> + Here is an example, that when entered from the host, + connects to the target and launches Gdbserver in order to + "debug" a binary named <filename>helloworld</filename>: + <literallayout class='monospaced'> + $ gdbserver localhost:2345 /usr/bin/helloworld + </literallayout> + Gdbserver should now be listening on port 2345 for debugging + commands coming from a remote GDB process that is running on + the host computer. + Communication between Gdbserver and the host GDB are done + using TCP. + To use other communication protocols, please refer to the + <ulink url='http://www.gnu.org/software/gdb/'>Gdbserver documentation</ulink>. + </para> + </section> + + <section id="platdev-gdb-remotedebug-launch-gdb"> + <title>Launch GDB on the Host Computer</title> + + <para> + Running GDB on the host computer takes a number of stages, which + this section describes. + </para> + + <section id="platdev-gdb-remotedebug-launch-gdb-buildcross"> + <title>Build the Cross-GDB Package</title> + <para> + A suitable GDB cross-binary is required that runs on your + host computer but also knows about the the ABI of the + remote target. + You can get this binary from the + <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>. + Here is an example where the toolchain has been installed + in the default directory + <filename>/opt/poky/&DISTRO;</filename>: + <literallayout class='monospaced'> + /opt/poky/&DISTRO;/sysroots/i686-pokysdk-linux/usr/bin/armv7a-vfp-neon-poky-linux-gnueabi/arm-poky-linux-gnueabi-gdb + </literallayout> + where <filename>arm</filename> is the target architecture + and <filename>linux-gnueabi</filename> is the target ABI. + </para> + + <para> + Alternatively, you can use BitBake to build the + <filename>gdb-cross</filename> binary. + Here is an example: + <literallayout class='monospaced'> + $ bitbake gdb-cross + </literallayout> + Once the binary is built, you can find it here: + <literallayout class='monospaced'> + tmp/sysroots/<replaceable>host-arch</replaceable>/usr/bin/<replaceable>target-platform</replaceable>/<replaceable>target-abi</replaceable>-gdb + </literallayout> + </para> + </section> + + <section id='create-the-gdb-initialization-file'> + <title>Create the GDB Initialization File and Point to Your Root Filesystem</title> + + <para> + Aside from the GDB cross-binary, you also need a GDB + initialization file in the same top directory in which + your binary resides. + When you start GDB on your host development system, GDB + finds this initialization file and executes all the + commands within. + For information on the <filename>.gdbinit</filename>, see + "<ulink url='http://sourceware.org/gdb/onlinedocs/gdb/'>Debugging with GDB</ulink>", + which is maintained by + <ulink url='http://www.sourceware.org'>sourceware.org</ulink>. + </para> + + <para> + You need to add a statement in the + <filename>~/.gdbinit</filename> file that points to your + root filesystem. + Here is an example that points to the root filesystem for + an ARM-based target device: + <literallayout class='monospaced'> + set sysroot ~/sysroot_arm + </literallayout> + </para> + </section> + + <section id="platdev-gdb-remotedebug-launch-gdb-launchhost"> + <title>Launch the Host GDB</title> + + <para> + Before launching the host GDB, you need to be sure + you have sourced the cross-debugging environment script, + which if you installed the root filesystem in the default + location is at <filename>/opt/poky/&DISTRO;</filename> + and begins with the string "environment-setup". + For more information, see the + <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Software Development Kit (SDK) Developer's + Guide</ulink>. + </para> + + <para> + Finally, switch to the directory where the binary resides + and run the <filename>cross-gdb</filename> binary. + Provide the binary file you are going to debug. + For example, the following command continues with the + example used in the previous section by loading + the <filename>helloworld</filename> binary as well as the + debugging information: + <literallayout class='monospaced'> + $ arm-poky-linux-gnuabi-gdb helloworld + </literallayout> + The commands in your <filename>.gdbinit</filename> execute + and the GDB prompt appears. + </para> + </section> + </section> + + <section id='platdev-gdb-connect-to-the-remote-gdb-server'> + <title>Connect to the Remote GDB Server</title> + + <para> + From the target, you need to connect to the remote GDB + server that is running on the host. + You need to specify the remote host and port. + Here is the command continuing with the example: + <literallayout class='monospaced'> + target remote 192.168.7.2:2345 + </literallayout> + </para> + </section> + + <section id="platdev-gdb-remotedebug-launch-gdb-using"> + <title>Use the Debugger</title> + + <para> + You can now proceed with debugging as normal - as if you were debugging + on the local machine. + For example, to instruct GDB to break in the "main" function and then + continue with execution of the inferior binary use the following commands + from within GDB: + <literallayout class='monospaced'> + (gdb) break main + (gdb) continue + </literallayout> + </para> + + <para> + For more information about using GDB, see the project's online documentation at + <ulink url="http://sourceware.org/gdb/download/onlinedocs/"/>. + </para> + </section> + </section> + + <section id='debugging-parallel-make-races'> + <title>Debugging Parallel Make Races</title> + + <para> + A parallel <filename>make</filename> race occurs when the build + consists of several parts that are run simultaneously and + a situation occurs when the output or result of one + part is not ready for use with a different part of the build that + depends on that output. + Parallel make races are annoying and can sometimes be difficult + to reproduce and fix. + However, some simple tips and tricks exist that can help + you debug and fix them. + This section presents a real-world example of an error encountered + on the Yocto Project autobuilder and the process used to fix it. + <note> + If you cannot properly fix a <filename>make</filename> race + condition, you can work around it by clearing either the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink> + variables. + </note> + </para> + + <section id='the-failure'> + <title>The Failure</title> + + <para> + For this example, assume that you are building an image that + depends on the "neard" package. + And, during the build, BitBake runs into problems and + creates the following output. + <note> + This example log file has longer lines artificially + broken to make the listing easier to read. + </note> + If you examine the output or the log file, you see the + failure during <filename>make</filename>: + <literallayout class='monospaced'> + | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common'] + | DEBUG: Executing shell function do_compile + | NOTE: make -j 16 + | make --no-print-directory all-am + | /bin/mkdir -p include/near + | /bin/mkdir -p include/near + | /bin/mkdir -p include/near + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/types.h include/near/types.h + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/log.h include/near/log.h + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h + | /bin/mkdir -p include/near + | /bin/mkdir -p include/near + | /bin/mkdir -p include/near + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/tag.h include/near/tag.h + | /bin/mkdir -p include/near + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h + | /bin/mkdir -p include/near + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h + | /bin/mkdir -p include/near + | /bin/mkdir -p include/near + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/setting.h include/near/setting.h + | /bin/mkdir -p include/near + | /bin/mkdir -p include/near + | /bin/mkdir -p include/near + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/device.h include/near/device.h + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/snep.h include/near/snep.h + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/version.h include/near/version.h + | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ + 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h + | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h + | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/ + build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/ + yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0 + -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/ + lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/ + tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/yocto-slave/ + nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/ + yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3 + -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\" + -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c + -o tools/snep-send.o tools/snep-send.c + | In file included from tools/snep-send.c:16:0: + | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory + | #include <near/dbus.h> + | ^ + | compilation terminated. + | make[1]: *** [tools/snep-send.o] Error 1 + | make[1]: *** Waiting for unfinished jobs.... + | make: *** [all] Error 2 + | ERROR: oe_runmake failed + </literallayout> + </para> + </section> + + <section id='reproducing-the-error'> + <title>Reproducing the Error</title> + + <para> + Because race conditions are intermittent, they do not + manifest themselves every time you do the build. + In fact, most times the build will complete without problems + even though the potential race condition exists. + Thus, once the error surfaces, you need a way to reproduce it. + </para> + + <para> + In this example, compiling the "neard" package is causing the + problem. + So the first thing to do is build "neard" locally. + Before you start the build, set the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink> + variable in your <filename>local.conf</filename> file to + a high number (e.g. "-j 20"). + Using a high value for <filename>PARALLEL_MAKE</filename> + increases the chances of the race condition showing up: + <literallayout class='monospaced'> + $ bitbake neard + </literallayout> + </para> + + <para> + Once the local build for "neard" completes, start a + <filename>devshell</filename> build: + <literallayout class='monospaced'> + $ bitbake neard -c devshell + </literallayout> + For information on how to use a + <filename>devshell</filename>, see the + "<link linkend='platdev-appdev-devshell'>Using a Development Shell</link>" + section. + </para> + + <para> + In the <filename>devshell</filename>, do the following: + <literallayout class='monospaced'> + $ make clean + $ make tools/snep-send.o + </literallayout> + The <filename>devshell</filename> commands cause the failure + to clearly be visible. + In this case, a missing dependency exists for the "neard" + Makefile target. + Here is some abbreviated, sample output with the + missing dependency clearly visible at the end: + <literallayout class='monospaced'> + i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/...... + . + . + . + tools/snep-send.c + In file included from tools/snep-send.c:16:0: + tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory + #include <near/dbus.h> + ^ + compilation terminated. + make: *** [tools/snep-send.o] Error 1 + $ + </literallayout> + </para> + </section> + + <section id='creating-a-patch-for-the-fix'> + <title>Creating a Patch for the Fix</title> + + <para> + Because there is a missing dependency for the Makefile + target, you need to patch the + <filename>Makefile.am</filename> file, which is generated + from <filename>Makefile.in</filename>. + You can use Quilt to create the patch: + <literallayout class='monospaced'> + $ quilt new parallelmake.patch + Patch patches/parallelmake.patch is now on top + $ quilt add Makefile.am + File Makefile.am added to patch patches/parallelmake.patch + </literallayout> + For more information on using Quilt, see the + "<link linkend='using-a-quilt-workflow'>Using Quilt in Your Workflow</link>" + section. + </para> + + <para> + At this point you need to make the edits to + <filename>Makefile.am</filename> to add the missing + dependency. + For our example, you have to add the following line + to the file: + <literallayout class='monospaced'> + tools/snep-send.$(OBJEXT): include/near/dbus.h + </literallayout> + </para> + + <para> + Once you have edited the file, use the + <filename>refresh</filename> command to create the patch: + <literallayout class='monospaced'> + $ quilt refresh + Refreshed patch patches/parallelmake.patch + </literallayout> + Once the patch file exists, you need to add it back to the + originating recipe folder. + Here is an example assuming a top-level + <link linkend='source-directory'>Source Directory</link> + named <filename>poky</filename>: + <literallayout class='monospaced'> + $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard + </literallayout> + The final thing you need to do to implement the fix in the + build is to update the "neard" recipe (i.e. + <filename>neard-0.14.bb</filename>) so that the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + statement includes the patch file. + The recipe file is in the folder above the patch. + Here is what the edited <filename>SRC_URI</filename> + statement would look like: + <literallayout class='monospaced'> + SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \ + file://neard.in \ + file://neard.service.in \ + file://parallelmake.patch \ + " + </literallayout> + </para> + + <para> + With the patch complete and moved to the correct folder and + the <filename>SRC_URI</filename> statement updated, you can + exit the <filename>devshell</filename>: + <literallayout class='monospaced'> + $ exit + </literallayout> + </para> + </section> + + <section id='testing-the-build'> + <title>Testing the Build</title> + + <para> + With everything in place, you can get back to trying the + build again locally: + <literallayout class='monospaced'> + $ bitbake neard + </literallayout> + This build should succeed. + </para> + + <para> + Now you can open up a <filename>devshell</filename> again + and repeat the clean and make operations as follows: + <literallayout class='monospaced'> + $ bitbake neard -c devshell + $ make clean + $ make tools/snep-send.o + </literallayout> + The build should work without issue. + </para> + + <para> + As with all solved problems, if they originated upstream, you + need to submit the fix for the recipe in OE-Core and upstream + so that the problem is taken care of at its source. + See the + "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + section for more information. + </para> + </section> + </section> + +<!-- + <section id="platdev-oprofile"> + <title>Profiling with OProfile</title> + + <para> + <ulink url="http://oprofile.sourceforge.net/">OProfile</ulink> is a + statistical profiler well suited for finding performance + bottlenecks in both user-space software and in the kernel. + This profiler provides answers to questions like "Which functions does my application spend + the most time in when doing X?" + Because the OpenEmbedded build system is well integrated with OProfile, it makes profiling + applications on target hardware straight forward. + <note> + For more information on how to set up and run OProfile, see the + "<ulink url='&YOCTO_DOCS_PROF_URL;#profile-manual-oprofile'>oprofile</ulink>" + section in the Yocto Project Profiling and Tracing Manual. + </note> + </para> + + <para> + To use OProfile, you need an image that has OProfile installed. + The easiest way to do this is with "tools-profile" in the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'>IMAGE_FEATURES</ulink></filename> variable. + You also need debugging symbols to be available on the system where the analysis + takes place. + You can gain access to the symbols by using "dbg-pkgs" in the + <filename>IMAGE_FEATURES</filename> variable or by + installing the appropriate debug (<filename>-dbg</filename>) + packages. + </para> + + <para> + For successful call graph analysis, the binaries must preserve the frame + pointer register and should also be compiled with the + <filename>-fno-omit-framepointer</filename> flag. + You can achieve this by setting the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SELECTED_OPTIMIZATION'>SELECTED_OPTIMIZATION</ulink></filename> + variable with the following options: + <literallayout class='monospaced'> + -fexpensive-optimizations + -fno-omit-framepointer + -frename-registers + -O2 + </literallayout> + You can also achieve it by setting the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-DEBUG_BUILD'>DEBUG_BUILD</ulink></filename> + variable to "1" in the <filename>local.conf</filename> configuration file. + If you use the <filename>DEBUG_BUILD</filename> variable, + you also add extra debugging information that can make the debug + packages large. + </para> + + <section id="platdev-oprofile-target"> + <title>Profiling on the Target</title> + + <para> + Using OProfile, you can perform all the profiling work on the target device. + A simple OProfile session might look like the following: + </para> + + <para> + <literallayout class='monospaced'> + # opcontrol ‐‐reset + # opcontrol ‐‐start ‐‐separate=lib ‐‐no-vmlinux -c 5 + . + . + [do whatever is being profiled] + . + . + # opcontrol ‐‐stop + $ opreport -cl + </literallayout> + </para> + + <para> + In this example, the <filename>reset</filename> command clears any previously profiled data. + The next command starts OProfile. + The options used when starting the profiler separate dynamic library data + within applications, disable kernel profiling, and enable callgraphing up to + five levels deep. + <note> + To profile the kernel, you would specify the + <filename>‐‐vmlinux=/path/to/vmlinux</filename> option. + The <filename>vmlinux</filename> file is usually in the source directory in the + <filename>/boot/</filename> directory and must match the running kernel. + </note> + </para> + + <para> + After you perform your profiling tasks, the next command stops the profiler. + After that, you can view results with the <filename>opreport</filename> command with options + to see the separate library symbols and callgraph information. + </para> + + <para> + Callgraphing logs information about time spent in functions and about a function's + calling function (parent) and called functions (children). + The higher the callgraphing depth, the more accurate the results. + However, higher depths also increase the logging overhead. + Consequently, you should take care when setting the callgraphing depth. + <note> + On ARM, binaries need to have the frame pointer enabled for callgraphing to work. + To accomplish this use the <filename>-fno-omit-framepointer</filename> option + with <filename>gcc</filename>. + </note> + </para> + + <para> + For more information on using OProfile, see the OProfile + online documentation at + <ulink url="http://oprofile.sourceforge.net/docs/"/>. + </para> + </section> + + <section id="platdev-oprofile-oprofileui"> + <title>Using OProfileUI</title> + + <para> + A graphical user interface for OProfile is also available. + You can download and build this interface from the Yocto Project at + <ulink url="&YOCTO_GIT_URL;/cgit.cgi/oprofileui/"></ulink>. + If the "tools-profile" image feature is selected, all necessary binaries + are installed onto the target device for OProfileUI interaction. + For a list of image features that ship with the Yocto Project, + see the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-features-image'>Image Features</ulink>" + section in the Yocto Project Reference Manual. + </para> + + <para> + Even though the source directory usually includes all needed patches on the target device, you + might find you need other OProfile patches for recent OProfileUI features. + If so, see the <ulink url='&YOCTO_GIT_URL;/cgit.cgi/oprofileui/tree/README'> + OProfileUI README</ulink> for the most recent information. + </para> + + <section id="platdev-oprofile-oprofileui-online"> + <title>Online Mode</title> + + <para> + Using OProfile in online mode assumes a working network connection with the target + hardware. + With this connection, you just need to run "oprofile-server" on the device. + By default, OProfile listens on port 4224. + <note> + You can change the port using the <filename>‐‐port</filename> command-line + option. + </note> + </para> + + <para> + The client program is called <filename>oprofile-viewer</filename> and its UI is relatively + straight forward. + You access key functionality through the buttons on the toolbar, which + are duplicated in the menus. + Here are the buttons: + <itemizedlist> + <listitem><para><emphasis>Connect:</emphasis> Connects to the remote host. + You can also supply the IP address or hostname.</para></listitem> + <listitem><para><emphasis>Disconnect:</emphasis> Disconnects from the target. + </para></listitem> + <listitem><para><emphasis>Start:</emphasis> Starts profiling on the device. + </para></listitem> + <listitem><para><emphasis>Stop:</emphasis> Stops profiling on the device and + downloads the data to the local host. + Stopping the profiler generates the profile and displays it in the viewer. + </para></listitem> + <listitem><para><emphasis>Download:</emphasis> Downloads the data from the + target and generates the profile, which appears in the viewer.</para></listitem> + <listitem><para><emphasis>Reset:</emphasis> Resets the sample data on the device. + Resetting the data removes sample information collected from previous + sampling runs. + Be sure you reset the data if you do not want to include old sample information. + </para></listitem> + <listitem><para><emphasis>Save:</emphasis> Saves the data downloaded from the + target to another directory for later examination.</para></listitem> + <listitem><para><emphasis>Open:</emphasis> Loads previously saved data. + </para></listitem> + </itemizedlist> + </para> + + <para> + The client downloads the complete profile archive from + the target to the host for processing. + This archive is a directory that contains the sample data, the object files, + and the debug information for the object files. + The archive is then converted using the <filename>oparchconv</filename> script, which is + included in this distribution. + The script uses <filename>opimport</filename> to convert the archive from + the target to something that can be processed on the host. + </para> + + <para> + Downloaded archives reside in the + <link linkend='build-directory'>Build Directory</link> in + <filename>tmp</filename> and are cleared up when they are no longer in use. + </para> + + <para> + If you wish to perform kernel profiling, you need to be sure + a <filename>vmlinux</filename> file that matches the running kernel is available. + In the source directory, that file is usually located in + <filename>/boot/vmlinux-<replaceable>kernelversion</replaceable></filename>, where + <filename><replaceable>kernelversion</replaceable></filename> is the version of the kernel. + The OpenEmbedded build system generates separate <filename>vmlinux</filename> + packages for each kernel it builds. + Thus, it should just be a question of making sure a matching package is + installed (e.g. <filename>opkg install kernel-vmlinux</filename>). + The files are automatically installed into development and profiling images + alongside OProfile. + A configuration option exists within the OProfileUI settings page that you can use to + enter the location of the <filename>vmlinux</filename> file. + </para> + + <para> + Waiting for debug symbols to transfer from the device can be slow, and it + is not always necessary to actually have them on the device for OProfile use. + All that is needed is a copy of the filesystem with the debug symbols present + on the viewer system. + The "<link linkend='platdev-gdb-remotedebug-launch-gdb'>Launch GDB on the Host Computer</link>" + section covers how to create such a directory within + the source directory and how to use the OProfileUI Settings + Dialog to specify the location. + If you specify the directory, it will be used when the file checksums + match those on the system you are profiling. + </para> + </section> + + <section id="platdev-oprofile-oprofileui-offline"> + <title>Offline Mode</title> + + <para> + If network access to the target is unavailable, you can generate + an archive for processing in <filename>oprofile-viewer</filename> as follows: + <literallayout class='monospaced'> + # opcontrol ‐‐reset + # opcontrol ‐‐start ‐‐separate=lib ‐‐no-vmlinux -c 5 + . + . + [do whatever is being profiled] + . + . + # opcontrol ‐‐stop + # oparchive -o my_archive + </literallayout> + </para> + + <para> + In the above example, <filename>my_archive</filename> is the name of the + archive directory where you would like the profile archive to be kept. + After the directory is created, you can copy it to another host and load it + using <filename>oprofile-viewer</filename> open functionality. + If necessary, the archive is converted. + </para> + </section> + </section> + </section> +--> + + <section id='maintaining-open-source-license-compliance-during-your-products-lifecycle'> + <title>Maintaining Open Source License Compliance During Your Product's Lifecycle</title> + + <para> + One of the concerns for a development organization using open source + software is how to maintain compliance with various open source + licensing during the lifecycle of the product. + While this section does not provide legal advice or + comprehensively cover all scenarios, it does + present methods that you can use to + assist you in meeting the compliance requirements during a software + release. + </para> + + <para> + With hundreds of different open source licenses that the Yocto + Project tracks, it is difficult to know the requirements of each + and every license. + However, the requirements of the major FLOSS licenses can begin + to be covered by + assuming that three main areas of concern exist: + <itemizedlist> + <listitem><para>Source code must be provided.</para></listitem> + <listitem><para>License text for the software must be + provided.</para></listitem> + <listitem><para>Compilation scripts and modifications to the + source code must be provided. + </para></listitem> + </itemizedlist> + There are other requirements beyond the scope of these + three and the methods described in this section + (e.g. the mechanism through which source code is distributed). + </para> + + <para> + As different organizations have different methods of complying with + open source licensing, this section is not meant to imply that + there is only one single way to meet your compliance obligations, + but rather to describe one method of achieving compliance. + The remainder of this section describes methods supported to meet the + previously mentioned three requirements. + Once you take steps to meet these requirements, + and prior to releasing images, sources, and the build system, + you should audit all artifacts to ensure completeness. + <note> + The Yocto Project generates a license manifest during + image creation that is located + in <filename>${DEPLOY_DIR}/licenses/<replaceable>image_name-datestamp</replaceable></filename> + to assist with any audits. + </note> + </para> + + <section id='providing-the-source-code'> + <title>Providing the Source Code</title> + + <para> + Compliance activities should begin before you generate the + final image. + The first thing you should look at is the requirement that + tops the list for most compliance groups - providing + the source. + The Yocto Project has a few ways of meeting this + requirement. + </para> + + <para> + One of the easiest ways to meet this requirement is + to provide the entire + <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> + used by the build. + This method, however, has a few issues. + The most obvious is the size of the directory since it includes + all sources used in the build and not just the source used in + the released image. + It will include toolchain source, and other artifacts, which + you would not generally release. + However, the more serious issue for most companies is accidental + release of proprietary software. + The Yocto Project provides an + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-archiver'><filename>archiver</filename></ulink> + class to help avoid some of these concerns. + </para> + + <para> + Before you employ <filename>DL_DIR</filename> or the + archiver class, you need to decide how you choose to + provide source. + The source archiver class can generate tarballs and SRPMs + and can create them with various levels of compliance in mind. + </para> + + <para> + One way of doing this (but certainly not the only way) is to + release just the source as a tarball. + You can do this by adding the following to the + <filename>local.conf</filename> file found in the + <link linkend='build-directory'>Build Directory</link>: + <literallayout class='monospaced'> + INHERIT += "archiver" + ARCHIVER_MODE[src] = "original" + </literallayout> + During the creation of your image, the source from all + recipes that deploy packages to the image is placed within + subdirectories of + <filename>DEPLOY_DIR/sources</filename> based on the + <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink> + for each recipe. + Releasing the entire directory enables you to comply with + requirements concerning providing the unmodified source. + It is important to note that the size of the directory can + get large. + </para> + + <para> + A way to help mitigate the size issue is to only release + tarballs for licenses that require the release of + source. + Let us assume you are only concerned with GPL code as + identified with the following: + <literallayout class='monospaced'> + $ cd poky/build/tmp/deploy/sources + $ mkdir ~/gpl_source_release + $ for dir in */*GPL*; do cp -r $dir ~/gpl_source_release; done + </literallayout> + At this point, you could create a tarball from the + <filename>gpl_source_release</filename> directory and + provide that to the end user. + This method would be a step toward achieving compliance + with section 3a of GPLv2 and with section 6 of GPLv3. + </para> + </section> + + <section id='providing-license-text'> + <title>Providing License Text</title> + + <para> + One requirement that is often overlooked is inclusion + of license text. + This requirement also needs to be dealt with prior to + generating the final image. + Some licenses require the license text to accompany + the binary. + You can achieve this by adding the following to your + <filename>local.conf</filename> file: + <literallayout class='monospaced'> + COPY_LIC_MANIFEST = "1" + COPY_LIC_DIRS = "1" + LICENSE_CREATE_PACKAGE = "1" + </literallayout> + Adding these statements to the configuration file ensures + that the licenses collected during package generation + are included on your image. + <note> + <para>Setting all three variables to "1" results in the + image having two copies of the same license file. + One copy resides in + <filename>/usr/share/common-licenses</filename> and + the other resides in + <filename>/usr/share/license</filename>.</para> + + <para>The reason for this behavior is because + <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_DIRS'><filename>COPY_LIC_DIRS</filename></ulink> + and + <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_MANIFEST'><filename>COPY_LIC_MANIFEST</filename></ulink> + add a copy of the license when the image is built but do not + offer a path for adding licenses for newly installed packages + to an image. + <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_CREATE_PACKAGE'><filename>LICENSE_CREATE_PACKAGE</filename></ulink> + adds a separate package and an upgrade path for adding + licenses to an image.</para> + </note> + </para> + + <para> + As the source archiver has already archived the original + unmodified source that contains the license files, + you would have already met the requirements for inclusion + of the license information with source as defined by the GPL + and other open source licenses. + </para> + </section> + + <section id='providing-compilation-scripts-and-source-code-modifications'> + <title>Providing Compilation Scripts and Source Code Modifications</title> + + <para> + At this point, we have addressed all we need to address + prior to generating the image. + The next two requirements are addressed during the final + packaging of the release. + </para> + + <para> + By releasing the version of the OpenEmbedded build system + and the layers used during the build, you will be providing both + compilation scripts and the source code modifications in one + step. + </para> + + <para> + If the deployment team has a + <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP layer</ulink> + and a distro layer, and those those layers are used to patch, + compile, package, or modify (in any way) any open source + software included in your released images, you + might be required to to release those layers under section 3 of + GPLv2 or section 1 of GPLv3. + One way of doing that is with a clean + checkout of the version of the Yocto Project and layers used + during your build. + Here is an example: + <literallayout class='monospaced'> + # We built using the &DISTRO_NAME_NO_CAP; branch of the poky repo + $ git clone -b &DISTRO_NAME_NO_CAP; git://git.yoctoproject.org/poky + $ cd poky + # We built using the release_branch for our layers + $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer + $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer + # clean up the .git repos + $ find . -name ".git" -type d -exec rm -rf {} \; + </literallayout> + One thing a development organization might want to consider + for end-user convenience is to modify + <filename>meta-poky/conf/bblayers.conf.sample</filename> to + ensure that when the end user utilizes the released build + system to build an image, the development organization's + layers are included in the <filename>bblayers.conf</filename> + file automatically: + <literallayout class='monospaced'> + # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf + # changes incompatibly + LCONF_VERSION = "6" + + BBPATH = "${TOPDIR}" + BBFILES ?= "" + + BBLAYERS ?= " \ + ##OEROOT##/meta \ + ##OEROOT##/meta-poky \ + ##OEROOT##/meta-yocto-bsp \ + ##OEROOT##/meta-mylayer \ + " + </literallayout> + Creating and providing an archive of the + <link linkend='metadata'>Metadata</link> layers + (recipes, configuration files, and so forth) + enables you to meet your + requirements to include the scripts to control compilation + as well as any modifications to the original source. + </para> + </section> + </section> + + <section id='using-the-error-reporting-tool'> + <title>Using the Error Reporting Tool</title> + + <para> + The error reporting tool allows you to + submit errors encountered during builds to a central database. + Outside of the build environment, you can use a web interface to + browse errors, view statistics, and query for errors. + The tool works using a client-server system where the client + portion is integrated with the installed Yocto Project + <link linkend='source-directory'>Source Directory</link> + (e.g. <filename>poky</filename>). + The server receives the information collected and saves it in a + database. + </para> + + <para> + A live instance of the error reporting server exists at + <ulink url='http://errors.yoctoproject.org'></ulink>. + This server exists so that when you want to get help with + build failures, you can submit all of the information on the + failure easily and then point to the URL in your bug report + or send an email to the mailing list. + <note> + If you send error reports to this server, the reports become + publicly visible. + </note> + </para> + + <section id='enabling-and-using-the-tool'> + <title>Enabling and Using the Tool</title> + + <para> + By default, the error reporting tool is disabled. + You can enable it by inheriting the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-report-error'><filename>report-error</filename></ulink> + class by adding the following statement to the end of + your <filename>local.conf</filename> file in your + <link linkend='build-directory'>Build Directory</link>. + <literallayout class='monospaced'> + INHERIT += "report-error" + </literallayout> + </para> + + <para> + By default, the error reporting feature stores information in + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LOG_DIR'><filename>LOG_DIR</filename></ulink><filename>}/error-report</filename>. + However, you can specify a directory to use by adding the following + to your <filename>local.conf</filename> file: + <literallayout class='monospaced'> + ERR_REPORT_DIR = "path" + </literallayout> + Enabling error reporting causes the build process to collect + the errors and store them in a file as previously described. + When the build system encounters an error, it includes a + command as part of the console output. + You can run the command to send the error file to the server. + For example, the following command sends the errors to an + upstream server: + <literallayout class='monospaced'> + $ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt + </literallayout> + In the previous example, the errors are sent to a public + database available at + <ulink url='http://errors.yoctoproject.org'></ulink>, which is + used by the entire community. + If you specify a particular server, you can send the errors + to a different database. + Use the following command for more information on available + options: + <literallayout class='monospaced'> + $ send-error-report --help + </literallayout> + </para> + + <para> + When sending the error file, you are prompted to review the + data being sent as well as to provide a name and optional + email address. + Once you satisfy these prompts, the command returns a link + from the server that corresponds to your entry in the database. + For example, here is a typical link: + <literallayout class='monospaced'> + http://errors.yoctoproject.org/Errors/Details/9522/ + </literallayout> + Following the link takes you to a web interface where you can + browse, query the errors, and view statistics. + </para> + </section> + + <section id='disabling-the-tool'> + <title>Disabling the Tool</title> + + <para> + To disable the error reporting feature, simply remove or comment + out the following statement from the end of your + <filename>local.conf</filename> file in your + <link linkend='build-directory'>Build Directory</link>. + <literallayout class='monospaced'> + INHERIT += "report-error" + </literallayout> + </para> + </section> + + <section id='setting-up-your-own-error-reporting-server'> + <title>Setting Up Your Own Error Reporting Server</title> + + <para> + If you want to set up your own error reporting server, you + can obtain the code from the Git repository at + <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/'></ulink>. + Instructions on how to set it up are in the README document. + </para> + </section> + </section> +</chapter> + +<!-- +vim: expandtab tw=80 ts=4 +--> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-customization.xsl b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-customization.xsl new file mode 100644 index 000000000..523ea3c5e --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-customization.xsl @@ -0,0 +1,27 @@ +<?xml version='1.0'?> +<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> + + <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> + +<!-- + + <xsl:import href="../template/1.76.1/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> + + <xsl:import href="http://docbook.sourceforge.net/release/xsl/1.76.1/xhtml/docbook.xsl" /> + +--> + + <xsl:include href="../template/permalinks.xsl"/> + <xsl:include href="../template/section.title.xsl"/> + <xsl:include href="../template/component.title.xsl"/> + <xsl:include href="../template/division.title.xsl"/> + <xsl:include href="../template/formal.object.heading.xsl"/> + + <xsl:param name="html.stylesheet" select="'dev-style.css'" /> + <xsl:param name="chapter.autolabel" select="1" /> + <xsl:param name="appendix.autolabel" select="A" /> + <xsl:param name="section.autolabel" select="1" /> + <xsl:param name="section.label.includes.component.label" select="1" /> + <xsl:param name="generate.id.attributes" select="1" /> + +</xsl:stylesheet> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-eclipse-customization.xsl b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-eclipse-customization.xsl new file mode 100644 index 000000000..6d7b5fbb6 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-eclipse-customization.xsl @@ -0,0 +1,35 @@ +<?xml version='1.0'?> +<xsl:stylesheet + xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns="http://www.w3.org/1999/xhtml" + xmlns:fo="http://www.w3.org/1999/XSL/Format" + version="1.0"> + + <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/eclipse/eclipse3.xsl" /> + +<!-- + + <xsl:import href="../template/1.76.1/docbook-xsl-1.76.1/eclipse/eclipse3.xsl" /> + + <xsl:import + href="http://docbook.sourceforge.net/release/xsl/1.76.1/eclipse/eclipse3.xsl" /> + +--> + + <xsl:param name="chunker.output.indent" select="'yes'"/> + <xsl:param name="chunk.quietly" select="1"/> + <xsl:param name="chunk.first.sections" select="1"/> + <xsl:param name="chunk.section.depth" select="10"/> + <xsl:param name="use.id.as.filename" select="1"/> + <xsl:param name="ulink.target" select="'_self'" /> + <xsl:param name="base.dir" select="'html/dev-manual/'"/> + <xsl:param name="html.stylesheet" select="'../book.css'"/> + <xsl:param name="eclipse.manifest" select="0"/> + <xsl:param name="create.plugin.xml" select="0"/> + <xsl:param name="suppress.navigation" select="1"/> + <xsl:param name="generate.index" select="0"/> + <xsl:param name="chapter.autolabel" select="1" /> + <xsl:param name="appendix.autolabel" select="1" /> + <xsl:param name="section.autolabel" select="1" /> + <xsl:param name="section.label.includes.component.label" select="1" /> +</xsl:stylesheet> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-intro.xml b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-intro.xml new file mode 100644 index 000000000..caa066e82 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-intro.xml @@ -0,0 +1,247 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<chapter id='dev-manual-intro'> + +<title>The Yocto Project Development Manual</title> + <section id='dev-intro'> + <title>Introduction</title> + + <para> + Welcome to the Yocto Project Development Manual! + This manual provides information on how to use the Yocto Project to + develop embedded Linux images and user-space applications that + run on targeted devices. + The manual provides an overview of image, kernel, and + user-space application development using the Yocto Project. + Because much of the information in this manual is general, it + contains many references to other sources where you can find more + detail. + For example, you can find detailed information on Git, repositories, + and open source in general in many places on the Internet. + Another example specific to the Yocto Project is how to quickly + set up your host development system and build an image, which you + find in the + <ulink url='&YOCTO_DOCS_QS_URL;'>Yocto Project Quick Start</ulink>. + </para> + + <para> + The Yocto Project Development Manual does, however, provide + guidance and examples on how to change the kernel source code, + reconfigure the kernel, and develop an application using + <filename>devtool</filename>. + </para> + + <note> + By default, using the Yocto Project creates a Poky distribution. + However, you can create your own distribution by providing key + <link linkend='metadata'>Metadata</link>. + A good example is Angstrom, which has had a distribution + based on the Yocto Project since its inception. + Other examples include commercial distributions like + <ulink url='https://www.yoctoproject.org/organization/wind-river-systems'>Wind River Linux</ulink>, + <ulink url='https://www.yoctoproject.org/organization/mentor-graphics'>Mentor Embedded Linux</ulink>, + <ulink url='https://www.yoctoproject.org/organization/enea-ab'>ENEA Linux</ulink> + and <ulink url='https://www.yoctoproject.org/ecosystem/member-organizations'>others</ulink>. + See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>" + section for more information. + </note> + </section> + + <section id='what-this-manual-provides'> + <title>What This Manual Provides</title> + + <para> + The following list describes what you can get from this manual: + <itemizedlist> + <listitem><para>Information that lets you get set + up to develop using the Yocto Project.</para></listitem> + <listitem><para>Information to help developers who are new to + the open source environment and to the distributed revision + control system Git, which the Yocto Project uses. + </para></listitem> + <listitem><para>An understanding of common end-to-end + development models and tasks.</para></listitem> + <listitem><para>Information about common development tasks + generally used during image development for + embedded devices. + </para></listitem> + <listitem><para>Information on using the Yocto Project + integration of the QuickEMUlator (QEMU), which lets you + simulate running on hardware an image you have built using + the OpenEmbedded build system. + </para></listitem> + <listitem><para>Many references to other sources of related + information.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id='what-this-manual-does-not-provide'> + <title>What this Manual Does Not Provide</title> + + <para> + This manual will not give you the following: + <itemizedlist> + <listitem><para><emphasis>Step-by-step instructions when those instructions exist in other Yocto + Project documentation:</emphasis> + For example, the + <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink> + manual contains detailed instructions on how to install an + SDK, which is used to develop applications for target + hardware. + </para></listitem> + <listitem><para><emphasis>Reference material:</emphasis> + This type of material resides in an appropriate reference manual. + For example, system variables are documented in the + <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>. + </para></listitem> + <listitem><para><emphasis>Detailed public information that is not specific to the Yocto Project:</emphasis> + For example, exhaustive information on how to use Git is covered better through the + Internet than in this manual. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='other-information'> + <title>Other Information</title> + + <para> + Because this manual presents overview information for many different + topics, supplemental information is recommended for full + comprehension. + The following list presents other sources of information you might find helpful: + <itemizedlist> + <listitem><para><emphasis><ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>: + </emphasis> The home page for the Yocto Project provides lots of information on the project + as well as links to software and documentation. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_QS_URL;'>Yocto Project Quick Start</ulink>:</emphasis> + This short document lets you get started + with the Yocto Project and quickly begin building an image. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>:</emphasis> + This manual is a reference + guide to the OpenEmbedded build system, which is based on BitBake. + The build system is sometimes referred to as "Poky". + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>:</emphasis> + This guide provides information that lets you get going + with the standard or extensible SDK. + An SDK, with its cross-development toolchains, allows you + to develop projects inside or outside of the Yocto Project + environment. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>:</emphasis> + This guide defines the structure for BSP components. + Having a commonly understood structure encourages standardization. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink>:</emphasis> + This manual describes how to work with Linux Yocto kernels as well as provides a bit + of conceptual information on the construction of the Yocto Linux kernel tree. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_PROF_URL;'>Yocto Project Profiling and Tracing Manual</ulink>:</emphasis> + This manual presents a set of common and generally useful tracing and + profiling schemes along with their applications (as appropriate) to each tool. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_TOAST_URL;'>Toaster User Manual</ulink>:</emphasis> + This manual introduces and describes how to set up and use + Toaster, which is a web interface to the Yocto Project's + <link linkend='build-system-term'>OpenEmbedded Build System</link>. + </para></listitem> + <listitem><para><emphasis> + <ulink url='http://www.youtube.com/watch?v=3ZlOu-gLsh0'> + Eclipse IDE Yocto Plug-in</ulink>:</emphasis> + A step-by-step instructional video that + demonstrates how an application developer uses Yocto Plug-in features within + the Eclipse IDE. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_WIKI_URL;/wiki/FAQ'>FAQ</ulink>:</emphasis> + A list of commonly asked questions and their answers. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_RELEASE_NOTES;'>Release Notes</ulink>:</emphasis> + Features, updates and known issues for the current + release of the Yocto Project. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_HOME_URL;/tools-resources/projects/toaster'>Toaster</ulink>:</emphasis> + An Application Programming Interface (API) and web-based + interface to the OpenEmbedded build system, which uses + BitBake, that reports build information. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_HOME_URL;/tools-resources/projects/build-appliance'>Build Appliance</ulink>:</emphasis> + A virtual machine that + enables you to build and boot a custom embedded Linux image + with the Yocto Project using a non-Linux development system. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_BUGZILLA_URL;'>Bugzilla</ulink>:</emphasis> + The bug tracking application the Yocto Project uses. + If you find problems with the Yocto Project, you should report them using this + application. + </para></listitem> + <listitem><para><emphasis>Yocto Project Mailing Lists:</emphasis> + To subscribe to the Yocto Project mailing + lists, click on the following URLs and follow the instructions: + <itemizedlist> + <listitem><para><ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'></ulink> + for a Yocto Project Discussions mailing list. + </para></listitem> + <listitem><para><ulink url='&YOCTO_LISTS_URL;/listinfo/poky'></ulink> + for a Yocto Project Discussions mailing list about the + OpenEmbedded build system (Poky). + </para></listitem> + <listitem><para><ulink url='&YOCTO_LISTS_URL;/listinfo/yocto-announce'></ulink> + for a mailing list to receive official Yocto Project announcements + as well as Yocto Project milestones. + </para></listitem> + <listitem><para><ulink url='&YOCTO_LISTS_URL;/listinfo'></ulink> + for a listing of all public mailing lists on + <filename>lists.yoctoproject.org</filename>. + </para></listitem> + </itemizedlist></para></listitem> + <listitem><para><emphasis>Internet Relay Chat (IRC):</emphasis> + Two IRC channels on freenode are available + for Yocto Project and Poky discussions: <filename>#yocto</filename> and + <filename>#poky</filename>, respectively. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&OE_HOME_URL;'>OpenEmbedded</ulink>:</emphasis> + The build system used by the Yocto Project. + This project is the upstream, generic, embedded distribution + from which the Yocto Project derives its build system (Poky) + and to which it contributes. + </para></listitem> + <listitem><para><emphasis> + <ulink url='http://www.openembedded.org/wiki/BitBake'>BitBake</ulink>:</emphasis> + The tool used by the OpenEmbedded build system + to process project metadata. + </para></listitem> + <listitem><para><emphasis> + <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual:</ulink></emphasis> + A comprehensive guide to the BitBake tool. + If you want information on BitBake, see this manual. + </para></listitem> + <listitem><para><emphasis> + <ulink url='http://wiki.qemu.org/Index.html'>Quick EMUlator (QEMU)</ulink>:</emphasis> + An open-source machine emulator and virtualizer. + </para></listitem> + </itemizedlist> + </para> + </section> +</chapter> +<!-- +vim: expandtab tw=80 ts=4 +--> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-model.xml b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-model.xml new file mode 100644 index 000000000..ff44a3f68 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-model.xml @@ -0,0 +1,2195 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<chapter id='dev-manual-model'> + +<title>Common Development Models</title> + +<para> + Many development models exist for which you can use the Yocto Project. + This chapter overviews simple methods that use tools provided by the + Yocto Project: + <itemizedlist> + <listitem><para><emphasis>System Development:</emphasis> + System Development covers Board Support Package (BSP) development + and kernel modification or configuration. + For an example on how to create a BSP, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>" + section in the Yocto Project Board Support Package (BSP) + Developer's Guide. + For more complete information on how to work with the kernel, + see the + <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink>. + </para></listitem> + <listitem><para><emphasis>User Application Development:</emphasis> + User Application Development covers development of applications + that you intend to run on target hardware. + For information on how to set up your host development system for + user-space application development, see the + <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. + For a simple example of user-space application development using + the <trademark class='trade'>Eclipse</trademark> IDE, see the + "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-developing-applications-using-eclipse'>Developing Applications Using <trademark class='trade'>Eclipse</trademark></ulink>" section. + </para></listitem> + <listitem><para><emphasis>Temporary Source Code Modification:</emphasis> + Direct modification of temporary source code is a convenient + development model to quickly iterate and develop towards a + solution. + Once you implement the solution, you should of course take + steps to get the changes upstream and applied in the affected + recipes. + </para></listitem> + <listitem><para><emphasis>Image Development using Toaster:</emphasis> + You can use <ulink url='&YOCTO_HOME_URL;/Tools-resources/projects/toaster'>Toaster</ulink> + to build custom operating system images within the build + environment. + Toaster provides an efficient interface to the OpenEmbedded build + that allows you to start builds and examine build statistics. + </para></listitem> + <listitem><para><emphasis>Using a Development Shell:</emphasis> + You can use a + <link linkend='platdev-appdev-devshell'><filename>devshell</filename></link> + to efficiently debug + commands or simply edit packages. + Working inside a development shell is a quick way to set up the + OpenEmbedded build environment to work on parts of a project. + </para></listitem> + </itemizedlist> +</para> + +<section id='system-development-model'> + <title>System Development Workflow</title> + + <para> + System development involves modification or creation of an image that you want to run on + a specific hardware target. + Usually, when you want to create an image that runs on embedded hardware, the image does + not require the same number of features that a full-fledged Linux distribution provides. + Thus, you can create a much smaller image that is designed to use only the + features for your particular hardware. + </para> + + <para> + To help you understand how system development works in the Yocto Project, this section + covers two types of image development: BSP creation and kernel modification or + configuration. + </para> + + <section id='developing-a-board-support-package-bsp'> + <title>Developing a Board Support Package (BSP)</title> + + <para> + A BSP is a collection of recipes that, when applied during a build, results in + an image that you can run on a particular board. + Thus, the package when compiled into the new image, supports the operation of the board. + </para> + + <note> + For a brief list of terms used when describing the development process in the Yocto Project, + see the "<link linkend='yocto-project-terms'>Yocto Project Terms</link>" section. + </note> + + <para> + The remainder of this section presents the basic + steps used to create a BSP using the Yocto Project's + <ulink url='&YOCTO_DOCS_BSP_URL;#using-the-yocto-projects-bsp-tools'>BSP Tools</ulink>. + Although not required for BSP creation, the + <filename>meta-intel</filename> repository, which contains + many BSPs supported by the Yocto Project, is part of the example. + </para> + + <para> + For an example that shows how to create a new layer using the tools, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>" + section in the Yocto Project Board Support Package (BSP) Developer's Guide. + </para> + + <para> + The following illustration and list summarize the BSP creation general workflow. + </para> + + <para> + <imagedata fileref="figures/bsp-dev-flow.png" width="6in" depth="7in" align="center" scalefit="1" /> + </para> + + <para> + <orderedlist> + <listitem><para><emphasis>Set up your host development system to support + development using the Yocto Project</emphasis>: See the + "<ulink url='&YOCTO_DOCS_QS_URL;#the-linux-distro'>The Linux Distribution</ulink>" + and the + "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Build Host Packages</ulink>" sections both + in the Yocto Project Quick Start for requirements.</para></listitem> + <listitem><para><emphasis>Establish a local copy of the project files on your + system</emphasis>: You need this <link linkend='source-directory'>Source + Directory</link> available on your host system. + Having these files on your system gives you access to the build + process and to the tools you need. + For information on how to set up the Source Directory, + see the + "<link linkend='getting-setup'>Getting Set Up</link>" section.</para></listitem> + <listitem><para><emphasis>Establish the <filename>meta-intel</filename> + repository on your system</emphasis>: Having local copies + of these supported BSP layers on your system gives you + access to layers you might be able to build on or modify + to create your BSP. + For information on how to get these files, see the + "<link linkend='getting-setup'>Getting Set Up</link>" section.</para></listitem> + <listitem><para><emphasis>Create your own BSP layer using the + <ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'><filename>yocto-bsp</filename></ulink> script</emphasis>: + Layers are ideal for + isolating and storing work for a given piece of hardware. + A layer is really just a location or area in which you place + the recipes and configurations for your BSP. + In fact, a BSP is, in itself, a special type of layer. + The simplest way to create a new BSP layer that is compliant with the + Yocto Project is to use the <filename>yocto-bsp</filename> script. + For information about that script, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>" + section in the Yocto Project Board Support (BSP) Developer's Guide. + </para> + + <para> + Another example that illustrates a layer + is an application. + Suppose you are creating an application that has + library or other dependencies in order for it to + compile and run. + The layer, in this case, would be where all the + recipes that define those dependencies are kept. + The key point for a layer is that it is an isolated + area that contains all the relevant information for + the project that the OpenEmbedded build system knows + about. + For more information on layers, see the + "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" + section. + For more information on BSP layers, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" + section in the Yocto Project Board Support Package (BSP) + Developer's Guide. + <note> + <para> + Five BSPs exist that are part of the Yocto Project release: + <filename>beaglebone</filename> (ARM), + <filename>mpc8315e</filename> (PowerPC), + and <filename>edgerouter</filename> (MIPS). + The recipes and configurations for these five BSPs + are located and dispersed within the + <link linkend='source-directory'>Source Directory</link>. + </para> + + <para> + Three core Intel BSPs exist as part of the Yocto + Project release in the + <filename>meta-intel</filename> layer: + <itemizedlist> + <listitem><para><filename>intel-core2-32</filename>, + which is a BSP optimized for the Core2 family of CPUs + as well as all CPUs prior to the Silvermont core. + </para></listitem> + <listitem><para><filename>intel-corei7-64</filename>, + which is a BSP optimized for Nehalem and later + Core and Xeon CPUs as well as Silvermont and later + Atom CPUs, such as the Baytrail SoCs. + </para></listitem> + <listitem><para><filename>intel-quark</filename>, + which is a BSP optimized for the Intel Galileo + gen1 & gen2 development boards. + </para></listitem> + </itemizedlist> + </para> + </note> + </para> + + <para>When you set up a layer for a new BSP, you should follow a standard layout. + This layout is described in the + "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout'>Example Filesystem Layout</ulink>" + section of the Board Support Package (BSP) Development Guide. + In the standard layout, you will notice a suggested structure for recipes and + configuration information. + You can see the standard layout for a BSP by examining + any supported BSP found in the <filename>meta-intel</filename> layer inside + the Source Directory.</para></listitem> + <listitem><para><emphasis>Make configuration changes to your new BSP + layer</emphasis>: The standard BSP layer structure organizes the files you need + to edit in <filename>conf</filename> and several <filename>recipes-*</filename> + directories within the BSP layer. + Configuration changes identify where your new layer is on the local system + and identify which kernel you are going to use. + When you run the <filename>yocto-bsp</filename> script, you are able to interactively + configure many things for the BSP (e.g. keyboard, touchscreen, and so forth). + </para></listitem> + <listitem><para><emphasis>Make recipe changes to your new BSP layer</emphasis>: Recipe + changes include altering recipes (<filename>.bb</filename> files), removing + recipes you do not use, and adding new recipes or append files + (<filename>.bbappend</filename>) that you need to support your hardware. + </para></listitem> + <listitem><para><emphasis>Prepare for the build</emphasis>: Once you have made all the + changes to your BSP layer, there remains a few things + you need to do for the OpenEmbedded build system in order for it to create your image. + You need to get the build environment ready by sourcing an environment setup script + (i.e. <filename>oe-init-build-env</filename> or + <filename>oe-init-build-env-memres</filename>) + and you need to be sure two key configuration files are configured appropriately: + the <filename>conf/local.conf</filename> and the + <filename>conf/bblayers.conf</filename> file. + You must make the OpenEmbedded build system aware of your new layer. + See the + "<link linkend='enabling-your-layer'>Enabling Your Layer</link>" section + for information on how to let the build system know about your new layer.</para> + <para>The entire process for building an image is overviewed in the section + "<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>" section + of the Yocto Project Quick Start. + You might want to reference this information.</para></listitem> + <listitem><para><emphasis>Build the image</emphasis>: The OpenEmbedded build system + uses the BitBake tool to build images based on the type of image you want to create. + You can find more information about BitBake in the + <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. + </para> + <para>The build process supports several types of images to satisfy different needs. + See the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" chapter + in the Yocto Project Reference Manual for information on + supported images.</para></listitem> + </orderedlist> + </para> + + <para> + You can view a video presentation on "Building Custom Embedded Images with Yocto" + at <ulink url='http://free-electrons.com/blog/elc-2011-videos'>Free Electrons</ulink>. + After going to the page, just search for "Embedded". + You can also find supplemental information in the + <ulink url='&YOCTO_DOCS_BSP_URL;'> + Yocto Project Board Support Package (BSP) Developer's Guide</ulink>. + Finally, there is helpful material and links on this + <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_creating_one_generic_Atom_BSP_from_another'>wiki page</ulink>. + Although a bit dated, you might find the information on the wiki + helpful. + </para> + </section> + + <section id='modifying-the-kernel'> + <title><anchor id='kernel-spot' />Modifying the Kernel</title> + + <para> + Kernel modification involves changing the Yocto Project kernel, which could involve changing + configuration options as well as adding new kernel recipes. + Configuration changes can be added in the form of configuration fragments, while recipe + modification comes through the kernel's <filename>recipes-kernel</filename> area + in a kernel layer you create. + </para> + + <para> + The remainder of this section presents a high-level overview of the Yocto Project + kernel architecture and the steps to modify the kernel. + You can reference the + "<link linkend='patching-the-kernel'>Patching the Kernel</link>" section + for an example that changes the source code of the kernel. + For information on how to configure the kernel, see the + "<link linkend='configuring-the-kernel'>Configuring the Kernel</link>" section. + For more information on the kernel and on modifying the kernel, see the + <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink>. + </para> + + <section id='kernel-overview'> + <title>Kernel Overview</title> + + <para> + Traditionally, when one thinks of a patched kernel, they think of a base kernel + source tree and a fixed structure that contains kernel patches. + The Yocto Project, however, employs mechanisms that, in a sense, result in a kernel source + generator. + By the end of this section, this analogy will become clearer. + </para> + + <para> + You can find a web interface to the Yocto Project kernel source repositories at + <ulink url='&YOCTO_GIT_URL;'></ulink>. + If you look at the interface, you will see to the left a grouping of + Git repositories titled "Yocto Linux Kernel." + Within this group, you will find several kernels supported by + the Yocto Project: + <itemizedlist> + <listitem><para><emphasis> + <filename>linux-yocto-3.14</filename></emphasis> - The + stable Yocto Project kernel to use with the Yocto + Project Releases 1.6 and 1.7. + This kernel is based on the Linux 3.14 released kernel. + </para></listitem> + <listitem><para><emphasis> + <filename>linux-yocto-3.17</filename></emphasis> - An + additional, unsupported Yocto Project kernel used with + the Yocto Project Release 1.7. + This kernel is based on the Linux 3.17 released kernel. + </para></listitem> + <listitem><para><emphasis> + <filename>linux-yocto-3.19</filename></emphasis> - The + stable Yocto Project kernel to use with the Yocto + Project Release 1.8. + This kernel is based on the Linux 3.19 released kernel. + </para></listitem> + <listitem><para><emphasis> + <filename>linux-yocto-4.1</filename></emphasis> - The + stable Yocto Project kernel to use with the Yocto + Project Release 2.0. + This kernel is based on the Linux 4.1 released kernel. + </para></listitem> + <listitem><para><emphasis> + <filename>linux-yocto-4.4</filename></emphasis> - The + stable Yocto Project kernel to use with the Yocto + Project Release 2.1. + This kernel is based on the Linux 4.4 released kernel. + </para></listitem> + <listitem><para><emphasis> + <filename>linux-yocto-dev</filename></emphasis> - A + development kernel based on the latest upstream release + candidate available. + </para></listitem> + </itemizedlist> + <note> + Long Term Support Initiative (LTSI) for Yocto Project kernels + is as follows: + <itemizedlist> + <listitem><para>For Yocto Project releases 1.7, 1.8, and 2.0, + the LTSI kernel is <filename>linux-yocto-3.14</filename>. + </para></listitem> + <listitem><para>For Yocto Project release 2.1, the + LTSI kernel is <filename>linux-yocto-4.1</filename>. + </para></listitem> + </itemizedlist> + </note> + </para> + + <para> + The kernels are maintained using the Git revision control system + that structures them using the familiar "tree", "branch", and "leaf" scheme. + Branches represent diversions from general code to more specific code, while leaves + represent the end-points for a complete and unique kernel whose source files, + when gathered from the root of the tree to the leaf, accumulate to create the files + necessary for a specific piece of hardware and its features. + The following figure displays this concept: + <para> + <imagedata fileref="figures/kernel-overview-1.png" + width="6in" depth="6in" align="center" scale="100" /> + </para> + + <para> + Within the figure, the "Kernel.org Branch Point" represents the point in the tree + where a supported base kernel is modified from the Linux kernel. + For example, this could be the branch point for the <filename>linux-yocto-3.19</filename> + kernel. + Thus, everything further to the right in the structure is based on the + <filename>linux-yocto-3.19</filename> kernel. + Branch points to the right in the figure represent where the + <filename>linux-yocto-3.19</filename> kernel is modified for specific hardware + or types of kernels, such as real-time kernels. + Each leaf thus represents the end-point for a kernel designed to run on a specific + targeted device. + </para> + + <para> + The overall result is a Git-maintained repository from which all the supported + kernel types can be derived for all the supported devices. + A big advantage to this scheme is the sharing of common features by keeping them in + "larger" branches within the tree. + This practice eliminates redundant storage of similar features shared among kernels. + </para> + + <note> + Keep in mind the figure does not take into account all the supported Yocto + Project kernel types, but rather shows a single generic kernel just for conceptual purposes. + Also keep in mind that this structure represents the Yocto Project source repositories + that are either pulled from during the build or established on the host development system + prior to the build by either cloning a particular kernel's Git repository or by + downloading and unpacking a tarball. + </note> + + <para> + Upstream storage of all the available kernel source code is one thing, while + representing and using the code on your host development system is another. + Conceptually, you can think of the kernel source repositories as all the + source files necessary for all the supported kernels. + As a developer, you are just interested in the source files for the kernel on + which you are working. + And, furthermore, you need them available on your host system. + </para> + + <para> + Kernel source code is available on your host system a couple of different + ways. + If you are working in the kernel all the time, you probably would want + to set up your own local Git repository of the kernel tree. + If you just need to make some patches to the kernel, you can access + temporary kernel source files that were extracted and used + during a build. + We will just talk about working with the temporary source code. + For more information on how to get kernel source code onto your + host system, see the + "<link linkend='local-kernel-files'>Yocto Project Kernel</link>" + bulleted item earlier in the manual. + </para> + + <para> + What happens during the build? + When you build the kernel on your development system, all files needed for the build + are taken from the source repositories pointed to by the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> variable + and gathered in a temporary work area + where they are subsequently used to create the unique kernel. + Thus, in a sense, the process constructs a local source tree specific to your + kernel to generate the new kernel image - a source generator if you will. + </para> + The following figure shows the temporary file structure + created on your host system when the build occurs. + This + <link linkend='build-directory'>Build Directory</link> contains all the + source files used during the build. + </para> + + <para> + <imagedata fileref="figures/kernel-overview-2-generic.png" + width="6in" depth="5in" align="center" scale="100" /> + </para> + + <para> + Again, for additional information on the Yocto Project kernel's + architecture and its branching strategy, see the + <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink>. + You can also reference the + "<link linkend='patching-the-kernel'>Patching the Kernel</link>" + section for a detailed example that modifies the kernel. + </para> + </section> + + <section id='kernel-modification-workflow'> + <title>Kernel Modification Workflow</title> + + <para> + This illustration and the following list summarizes the kernel modification general workflow. + </para> + + <para> + <imagedata fileref="figures/kernel-dev-flow.png" + width="6in" depth="5in" align="center" scalefit="1" /> + </para> + + <para> + <orderedlist> + <listitem><para><emphasis>Set up your host development system to support + development using the Yocto Project</emphasis>: See + "<ulink url='&YOCTO_DOCS_QS_URL;#the-linux-distro'>The Linux Distribution</ulink>" and + "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Build Host Packages</ulink>" sections both + in the Yocto Project Quick Start for requirements.</para></listitem> + <listitem><para><emphasis>Establish a local copy of project files on your + system</emphasis>: Having the <link linkend='source-directory'>Source + Directory</link> on your system gives you access to the build process and tools + you need. + For information on how to get these files, see the bulleted item + "<link linkend='local-yp-release'>Yocto Project Release</link>" earlier in this manual. + </para></listitem> + <listitem><para><emphasis>Establish the temporary kernel source files</emphasis>: + Temporary kernel source files are kept in the + <link linkend='build-directory'>Build Directory</link> + created by the + OpenEmbedded build system when you run BitBake. + If you have never built the kernel in which you are + interested, you need to run an initial build to + establish local kernel source files.</para> + <para>If you are building an image for the first time, you need to get the build + environment ready by sourcing an environment setup script + (i.e. <filename>oe-init-build-env</filename> or + <filename>oe-init-build-env-memres</filename>). + You also need to be sure two key configuration files + (<filename>local.conf</filename> and <filename>bblayers.conf</filename>) + are configured appropriately.</para> + <para>The entire process for building an image is overviewed in the + "<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>" + section of the Yocto Project Quick Start. + You might want to reference this information. + You can find more information on BitBake in the + <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. + </para> + <para>The build process supports several types of images to satisfy different needs. + See the "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" chapter in + the Yocto Project Reference Manual for information on supported images. + </para></listitem> + <listitem><para><emphasis>Make changes to the kernel source code if + applicable</emphasis>: Modifying the kernel does not always mean directly + changing source files. + However, if you have to do this, you make the changes to the files in the + Build Directory.</para></listitem> + <listitem><para><emphasis>Make kernel configuration changes if applicable</emphasis>: + If your situation calls for changing the kernel's + configuration, you can use + <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#generating-configuration-files'><filename>menuconfig</filename></ulink>, + which allows you to interactively develop and test the + configuration changes you are making to the kernel. + Saving changes you make with + <filename>menuconfig</filename> updates + the kernel's <filename>.config</filename> file. + <note><title>Warning</title> + Try to resist the temptation to directly edit an + existing <filename>.config</filename> file, which is + found in the Build Directory at + <filename>tmp/sysroots/<replaceable>machine-name</replaceable>/kernel</filename>. + Doing so, can produce unexpected results when the + OpenEmbedded build system regenerates the configuration + file. + </note> + Once you are satisfied with the configuration + changes made using <filename>menuconfig</filename> + and you have saved them, you can directly compare the + resulting <filename>.config</filename> file against an + existing original and gather those changes into a + <link linkend='creating-config-fragments'>configuration fragment file</link> + to be referenced from within the kernel's + <filename>.bbappend</filename> file.</para> + + <para>Additionally, if you are working in a BSP layer + and need to modify the BSP's kernel's configuration, + you can use the + <ulink url='&YOCTO_DOCS_BSP_URL;#managing-kernel-patches-and-config-items-with-yocto-kernel'><filename>yocto-kernel</filename></ulink> + script as well as <filename>menuconfig</filename>. + The <filename>yocto-kernel</filename> script lets + you interactively set up kernel configurations. + </para></listitem> + <listitem><para><emphasis>Rebuild the kernel image with your changes</emphasis>: + Rebuilding the kernel image applies your changes. + </para></listitem> + </orderedlist> + </para> + </section> + </section> +</section> + +<section id='application-development-workflow-using-an-sdk'> + <title>Application Development Workflow Using an SDK</title> + + <para> + Standard and extensible Software Development Kits (SDK) make it easy + to develop applications inside or outside of the Yocto Project + development environment. + Tools exist to help the application developer during any phase + of development. + For information on how to install and use an SDK, see the + <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-intro'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. + </para> +</section> + +<section id="dev-modifying-source-code"> + <title>Modifying Source Code</title> + + <para> + A common development workflow consists of modifying project source + files that are external to the Yocto Project and then integrating + that project's build output into an image built using the + OpenEmbedded build system. + Given this scenario, development engineers typically want to stick + to their familiar project development tools and methods, which allows + them to just focus on the project. + </para> + + <para> + Several workflows exist that allow you to develop, build, and test + code that is going to be integrated into an image built using the + OpenEmbedded build system. + This section describes two: + <itemizedlist> + <listitem><para><emphasis><filename>devtool</filename>:</emphasis> + A set of tools to aid in working on the source code built by + the OpenEmbedded build system. + Section + "<link linkend='using-devtool-in-your-workflow'>Using <filename>devtool</filename> in Your Workflow</link>" + describes this workflow. + If you want more information that showcases the workflow, click + <ulink url='https://drive.google.com/a/linaro.org/file/d/0B3KGzY5fW7laTDVxUXo3UDRvd2s/view'>here</ulink> + for a presentation by Trevor Woerner that, while somewhat dated, + provides detailed background information and a complete + working tutorial. + </para></listitem> + <listitem><para><emphasis><ulink url='http://savannah.nongnu.org/projects/quilt'>Quilt</ulink>:</emphasis> + A powerful tool that allows you to capture source + code changes without having a clean source tree. + While Quilt is not the preferred workflow of the two, this + section includes it for users that are committed to using + the tool. + See the + "<link linkend='using-a-quilt-workflow'>Using Quilt in Your Workflow</link>" + section for more information. + </para></listitem> + </itemizedlist> + </para> + + <section id='using-devtool-in-your-workflow'> + <title>Using <filename>devtool</filename> in Your Workflow</title> + + <para> + As mentioned earlier, <filename>devtool</filename> helps + you easily develop projects whose build output must be part of + an image built using the OpenEmbedded build system. + </para> + + <para> + Three entry points exist that allow you to develop using + <filename>devtool</filename>: + <itemizedlist> + <listitem><para><emphasis><filename>devtool add</filename></emphasis> + </para></listitem> + <listitem><para><emphasis><filename>devtool modify</filename></emphasis> + </para></listitem> + <listitem><para><emphasis><filename>devtool upgrade</filename></emphasis> + </para></listitem> + </itemizedlist> + </para> + + <para> + The remainder of this section presents these workflows. + </para> + + <section id='use-devtool-to-integrate-new-code'> + <title>Use <filename>devtool add</filename> to Integrate New Code</title> + + <para> + The <filename>devtool add</filename> command generates + a new recipe based on existing source code. + This command takes advantage of the + <link linkend='devtool-the-workspace-layer-structure'>workspace</link> + layer that many <filename>devtool</filename> commands + use. + The command is flexible enough to allow you to extract source + code into both the workspace or a separate local Git repository + and to use existing code that does not need to be extracted. + </para> + + <para> + Depending on your particular scenario, the arguments and options + you use with <filename>devtool add</filename> form different + combinations. + The following diagram shows common development flows + you would use with the <filename>devtool add</filename> + command: + </para> + + <para> + <imagedata fileref="figures/devtool-add-flow.png" align="center" /> + </para> + + <para> + <orderedlist> + <listitem><para><emphasis>Generating the New Recipe</emphasis>: + The top part of the flow shows three scenarios by which + you could use <filename>devtool add</filename> to + generate a recipe based on existing source code.</para> + + <para>In a shared development environment, it is + typical where other developers are responsible for + various areas of source code. + As a developer, you are probably interested in using + that source code as part of your development using + the Yocto Project. + All you need is access to the code, a recipe, and a + controlled area in which to do your work.</para> + + <para>Within the diagram, three possible scenarios + feed into the <filename>devtool add</filename> workflow: + <itemizedlist> + <listitem><para><emphasis>Left</emphasis>: + The left scenario represents a common situation + where the source code does not exist locally + and needs to be extracted. + In this situation, you just let it get + extracted to the default workspace - you do not + want it in some specific location outside of the + workspace. + Thus, everything you need will be located in the + workspace: + <literallayout class='monospaced'> + $ devtool add <replaceable>recipe fetchuri</replaceable> + </literallayout> + With this command, <filename>devtool</filename> + creates a recipe and an append file in the + workspace as well as extracts the upstream + source files into a local Git repository also + within the <filename>sources</filename> folder. + </para></listitem> + <listitem><para><emphasis>Middle</emphasis>: + The middle scenario also represents a situation where + the source code does not exist locally. + In this case, the code is again upstream + and needs to be extracted to some + local area - this time outside of the default + workspace. + As always, if required <filename>devtool</filename> creates + a Git repository locally during the extraction. + Furthermore, the first positional argument + <replaceable>srctree</replaceable> in this case + identifies where the + <filename>devtool add</filename> command + will locate the extracted code outside of the + workspace: + <literallayout class='monospaced'> + $ devtool add <replaceable>recipe srctree fetchuri</replaceable> + </literallayout> + In summary, the source code is pulled from + <replaceable>fetchuri</replaceable> and extracted + into the location defined by + <replaceable>srctree</replaceable> as a local + Git repository.</para> + + <para>Within workspace, <filename>devtool</filename> + creates both the recipe and an append file + for the recipe. + </para></listitem> + <listitem><para><emphasis>Right</emphasis>: + The right scenario represents a situation + where the source tree (srctree) has been + previously prepared outside of the + <filename>devtool</filename> workspace. + </para> + + <para>The following command names the recipe + and identifies where the existing source tree + is located: + <literallayout class='monospaced'> + $ devtool add <replaceable>recipe srctree</replaceable> + </literallayout> + The command examines the source code and creates + a recipe for it placing the recipe into the + workspace.</para> + + <para>Because the extracted source code already exists, + <filename>devtool</filename> does not try to + relocate it into the workspace - just the new + the recipe is placed in the workspace.</para> + + <para>Aside from a recipe folder, the command + also creates an append folder and places an initial + <filename>*.bbappend</filename> within. + </para></listitem> + </itemizedlist> + </para></listitem> + <listitem><para><emphasis>Edit the Recipe</emphasis>: + At this point, you can use <filename>devtool edit-recipe</filename> + to open up the editor as defined by the + <filename>$EDITOR</filename> environment variable + and modify the file: + <literallayout class='monospaced'> + $ devtool edit-recipe <replaceable>recipe</replaceable> + </literallayout> + From within the editor, you can make modifications to the + recipe that take affect when you build it later. + </para></listitem> + <listitem><para><emphasis>Build the Recipe or Rebuild the Image</emphasis>: + At this point in the flow, the next step you + take depends on what you are going to do with + the new code.</para> + <para>If you need to take the build output and eventually + move it to the target hardware, you would use + <filename>devtool build</filename>: + <note> + You could use <filename>bitbake</filename> to build + the recipe as well. + </note> + <literallayout class='monospaced'> + $ devtool build <replaceable>recipe</replaceable> + </literallayout></para> + <para>On the other hand, if you want an image to + contain the recipe's packages for immediate deployment + onto a device (e.g. for testing purposes), you can use + the <filename>devtool build-image</filename> command: + <literallayout class='monospaced'> + $ devtool build-image <replaceable>image</replaceable> + </literallayout> + </para></listitem> + <listitem><para><emphasis>Deploy the Build Output</emphasis>: + When you use the <filename>devtool build</filename> + command to build out your recipe, you probably want to + see if the resulting build output works as expected on target + hardware. + <note> + This step assumes you have a previously built + image that is already either running in QEMU or + running on actual hardware. + Also, it is assumed that for deployment of the image + to the target, SSH is installed in the image and if + the image is running on real hardware that you have + network access to and from your development machine. + </note> + You can deploy your build output to that target hardware by + using the <filename>devtool deploy-target</filename> command: + <literallayout class='monospaced'> + $ devtool deploy-target <replaceable>recipe target</replaceable> + </literallayout> + The <replaceable>target</replaceable> is a live target machine + running as an SSH server.</para> + + <para>You can, of course, also deploy the image you build + using the <filename>devtool build-image</filename> command + to actual hardware. + However, <filename>devtool</filename> does not provide a + specific command that allows you to do this. + </para></listitem> + <listitem><para><emphasis>Optionally Update the Recipe With Patch Files</emphasis>: + Once you are satisfied with the recipe, if you have made + any changes to the source tree that you want to have + applied by the recipe, you need to generate patches + from those changes. + You do this before moving the recipe + to its final layer and cleaning up the workspace area + <filename>devtool</filename> uses. + This optional step is especially relevant if you are + using or adding third-party software.</para> + <para>To convert commits created using Git to patch files, + use the <filename>devtool update-recipe</filename> command. + <note> + Any changes you want to turn into patches must be + committed to the Git repository in the source tree. + </note> + <literallayout class='monospaced'> + $ devtool update-recipe <replaceable>recipe</replaceable> + </literallayout> + </para></listitem> + <listitem><para><emphasis>Move the Recipe to its Permanent Layer</emphasis>: + Before cleaning up the workspace, you need to move the + final recipe to its permanent layer. + You must do this before using the + <filename>devtool reset</filename> command if you want to + retain the recipe. + </para></listitem> + <listitem><para><emphasis>Reset the Recipe</emphasis>: + As a final step, you can restore the state such that + standard layers and the upstream source is used to build + the recipe rather than data in the workspace. + To reset the recipe, use the <filename>devtool reset</filename> + command: + <literallayout class='monospaced'> + $ devtool reset <replaceable>recipe</replaceable> + </literallayout> + </para></listitem> + </orderedlist> + </para> + </section> + + <section id='devtool-use-devtool-modify-to-enable-work-on-code-associated-with-an-existing-recipe'> + <title>Use <filename>devtool modify</filename> to Enable Work on Code Associated with an Existing Recipe</title> + + <para> + The <filename>devtool modify</filename> command prepares the + way to work on existing code that already has a recipe in + place. + The command is flexible enough to allow you to extract code, + specify the existing recipe, and keep track of and gather any + patch files from other developers that are + associated with the code. + </para> + + <para> + Depending on your particular scenario, the arguments and options + you use with <filename>devtool modify</filename> form different + combinations. + The following diagram shows common development flows + you would use with the <filename>devtool modify</filename> + command: + </para> + + <para> + <imagedata fileref="figures/devtool-modify-flow.png" align="center" /> + </para> + + <para> + <orderedlist> + <listitem><para><emphasis>Preparing to Modify the Code</emphasis>: + The top part of the flow shows three scenarios by which + you could use <filename>devtool modify</filename> to + prepare to work on source files. + Each scenario assumes the following: + <itemizedlist> + <listitem><para>The recipe exists in some layer external + to the <filename>devtool</filename> workspace. + </para></listitem> + <listitem><para>The source files exist upstream in an + un-extracted state or locally in a previously + extracted state. + </para></listitem> + </itemizedlist> + The typical situation is where another developer has + created some layer for use with the Yocto Project and + their recipe already resides in that layer. + Furthermore, their source code is readily available + either upstream or locally. + <itemizedlist> + <listitem><para><emphasis>Left</emphasis>: + The left scenario represents a common situation + where the source code does not exist locally + and needs to be extracted. + In this situation, the source is extracted + into the default workspace location. + The recipe, in this scenario, is in its own + layer outside the workspace + (i.e. + <filename>meta-</filename><replaceable>layername</replaceable>). + </para> + + <para>The following command identifies the recipe + and by default extracts the source files: + <literallayout class='monospaced'> + $ devtool modify <replaceable>recipe</replaceable> + </literallayout> + Once <filename>devtool</filename>locates the recipe, + it uses the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + variable to locate the source code and + any local patch files from other developers are + located. + <note> + You cannot provide an URL for + <replaceable>srctree</replaceable> when using the + <filename>devtool modify</filename> command. + </note> + With this scenario, however, since no + <replaceable>srctree</replaceable> argument exists, the + <filename>devtool modify</filename> command by default + extracts the source files to a Git structure. + Furthermore, the location for the extracted source is the + default area within the workspace. + The result is that the command sets up both the source + code and an append file within the workspace with the + recipe remaining in its original location. + </para></listitem> + <listitem><para><emphasis>Middle</emphasis>: + The middle scenario represents a situation where + the source code also does not exist locally. + In this case, the code is again upstream + and needs to be extracted to some + local area as a Git repository. + The recipe, in this scenario, is again in its own + layer outside the workspace.</para> + + <para>The following command tells + <filename>devtool</filename> what recipe with + which to work and, in this case, identifies a local + area for the extracted source files that is outside + of the default workspace: + <literallayout class='monospaced'> + $ devtool modify <replaceable>recipe srctree</replaceable> + </literallayout> + As with all extractions, the command uses + the recipe's <filename>SRC_URI</filename> to locate the + source files. + Once the files are located, the command by default + extracts them. + Providing the <replaceable>srctree</replaceable> + argument instructs <filename>devtool</filename> where + place the extracted source.</para> + + <para>Within workspace, <filename>devtool</filename> + creates an append file for the recipe. + The recipe remains in its original location but + the source files are extracted to the location you + provided with <replaceable>srctree</replaceable>. + </para></listitem> + <listitem><para><emphasis>Right</emphasis>: + The right scenario represents a situation + where the source tree + (<replaceable>srctree</replaceable>) exists as a + previously extracted Git structure outside of + the <filename>devtool</filename> workspace. + In this example, the recipe also exists + elsewhere in its own layer. + </para> + + <para>The following command tells + <filename>devtool</filename> the recipe + with which to work, uses the "-n" option to indicate + source does not need to be extracted, and uses + <replaceable>srctree</replaceable> to point to the + previously extracted source files: + <literallayout class='monospaced'> + $ devtool modify -n <replaceable>recipe srctree</replaceable> + </literallayout> + </para> + + <para>Once the command finishes, it creates only + an append file for the recipe in the workspace. + The recipe and the source code remain in their + original locations. + </para></listitem> + </itemizedlist> + </para></listitem> + <listitem><para><emphasis>Edit the Source</emphasis>: + Once you have used the <filename>devtool modify</filename> + command, you are free to make changes to the source + files. + You can use any editor you like to make and save + your source code modifications. + </para></listitem> + <listitem><para><emphasis>Build the Recipe</emphasis>: + Once you have updated the source files, you can build + the recipe. + You can either use <filename>devtool build</filename> or + <filename>bitbake</filename>. + Either method produces build output that is stored + in + <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>. + </para></listitem> + <listitem><para><emphasis>Deploy the Build Output</emphasis>: + When you use the <filename>devtool build</filename> + command or <filename>bitbake</filename> to build out your + recipe, you probably want to see if the resulting build + output works as expected on target hardware. + <note> + This step assumes you have a previously built + image that is already either running in QEMU or + running on actual hardware. + Also, it is assumed that for deployment of the image + to the target, SSH is installed in the image and if + the image is running on real hardware that you have + network access to and from your development machine. + </note> + You can deploy your build output to that target hardware by + using the <filename>devtool deploy-target</filename> command: + <literallayout class='monospaced'> + $ devtool deploy-target <replaceable>recipe target</replaceable> + </literallayout> + The <replaceable>target</replaceable> is a live target machine + running as an SSH server.</para> + + <para>You can, of course, also deploy the image you build + using the <filename>devtool build-image</filename> command + to actual hardware. + However, <filename>devtool</filename> does not provide a + specific command that allows you to do this. + </para></listitem> + <listitem><para><emphasis>Optionally Create Patch Files for Your Changes</emphasis>: + After you have debugged your changes, you can + use <filename>devtool update-recipe</filename> to + generate patch files for all the commits you have + made. + <note> + Patch files are generated only for changes + you have committed. + </note> + <literallayout class='monospaced'> + $ devtool update-recipe <replaceable>recipe</replaceable> + </literallayout> + By default, the + <filename>devtool update-recipe</filename> command + creates the patch files in a folder named the same + as the recipe beneath the folder in which the recipe + resides, and updates the recipe's + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + statement to point to the generated patch files. + <note> + You can use the + "--append <replaceable>LAYERDIR</replaceable>" + option to cause the command to create append files + in a specific layer rather than the default + recipe layer. + </note> + </para></listitem> + <listitem><para><emphasis>Restore the Workspace</emphasis>: + The <filename>devtool reset</filename> restores the + state so that standard layers and upstream sources are + used to build the recipe rather than what is in the + workspace. + <literallayout class='monospaced'> + $ devtool reset <replaceable>recipe</replaceable> + </literallayout> + </para></listitem> + </orderedlist> + </para> + </section> + + <section id='devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software'> + <title>Use <filename>devtool upgrade</filename> to Create a Version of the Recipe that Supports a Newer Version of the Software</title> + + <para> + The <filename>devtool upgrade</filename> command updates + an existing recipe so that you can build it for an updated + set of source files. + The command is flexible enough to allow you to specify + source code revision and versioning schemes, extract code into + or out of the <filename>devtool</filename> workspace, and + work with any source file forms that the fetchers support. + </para> + + <para> + Depending on your particular scenario, the arguments and options + you use with <filename>devtool upgrade</filename> form different + combinations. + The following diagram shows a common development flow + you would use with the <filename>devtool modify</filename> + command: + </para> + + <para> + <imagedata fileref="figures/devtool-upgrade-flow.png" align="center" /> + </para> + + <para> + <orderedlist> + <listitem><para><emphasis>Initiate the Upgrade</emphasis>: + The top part of the flow shows a typical scenario by which + you could use <filename>devtool upgrade</filename>. + The following conditions exist: + <itemizedlist> + <listitem><para>The recipe exists in some layer external + to the <filename>devtool</filename> workspace. + </para></listitem> + <listitem><para>The source files for the new release + exist adjacent to the same location pointed to by + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + in the recipe (e.g. a tarball with the new version + number in the name, or as a different revision in + the upstream Git repository). + </para></listitem> + </itemizedlist> + A common situation is where third-party software has + undergone a revision so that it has been upgraded. + The recipe you have access to is likely in your own layer. + Thus, you need to upgrade the recipe to use the + newer version of the software: + <literallayout class='monospaced'> + $ devtool upgrade -V <replaceable>version recipe</replaceable> + </literallayout> + By default, the <filename>devtool upgrade</filename> command + extracts source code into the <filename>sources</filename> + directory in the workspace. + If you want the code extracted to any other location, you + need to provide the <replaceable>srctree</replaceable> + positional argument with the command as follows: + <literallayout class='monospaced'> + $ devtool upgrade -V <replaceable>version recipe srctree</replaceable> + </literallayout> + Also, in this example, the "-V" option is used to specify + the new version. + If the source files pointed to by the + <filename>SRC_URI</filename> statement in the recipe are + in a Git repository, you must provide the "-S" option and + specify a revision for the software.</para> + + <para>Once <filename>devtool</filename> locates the recipe, + it uses the <filename>SRC_URI</filename> variable to locate + the source code and any local patch files from other + developers are located. + The result is that the command sets up the source + code, the new version of the recipe, and an append file + all within the workspace. + </para></listitem> + <listitem><para><emphasis>Resolve any Conflicts created by the Upgrade</emphasis>: + At this point, there could be some conflicts due to the + software being upgraded to a new version. + This would occur if your recipe specifies some patch files in + <filename>SRC_URI</filename> that conflict with changes + made in the new version of the software. + If this is the case, you need to resolve the conflicts + by editing the source and following the normal + <filename>git rebase</filename> conflict resolution + process.</para> + + <para>Before moving onto the next step, be sure to resolve any + such conflicts created through use of a newer or different + version of the software. + </para></listitem> + <listitem><para><emphasis>Build the Recipe</emphasis>: + Once you have your recipe in order, you can build it. + You can either use <filename>devtool build</filename> or + <filename>bitbake</filename>. + Either method produces build output that is stored + in + <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>. + </para></listitem> + <listitem><para><emphasis>Deploy the Build Output</emphasis>: + When you use the <filename>devtool build</filename> + command or <filename>bitbake</filename> to build out your + recipe, you probably want to see if the resulting build + output works as expected on target hardware. + <note> + This step assumes you have a previously built + image that is already either running in QEMU or + running on actual hardware. + Also, it is assumed that for deployment of the image + to the target, SSH is installed in the image and if + the image is running on real hardware that you have + network access to and from your development machine. + </note> + You can deploy your build output to that target hardware by + using the <filename>devtool deploy-target</filename> command: + <literallayout class='monospaced'> + $ devtool deploy-target <replaceable>recipe target</replaceable> + </literallayout> + The <replaceable>target</replaceable> is a live target machine + running as an SSH server.</para> + + <para>You can, of course, also deploy the image you build + using the <filename>devtool build-image</filename> command + to actual hardware. + However, <filename>devtool</filename> does not provide a + specific command that allows you to do this. + </para></listitem> + <listitem><para><emphasis>Optionally Create Patch Files for Your Changes</emphasis>: + After you have debugged your changes, you can + use <filename>devtool update-recipe</filename> to + generate patch files for all the commits you have + made. + <note> + Patch files are generated only for changes + you have committed. + </note> + <literallayout class='monospaced'> + $ devtool update-recipe <replaceable>recipe</replaceable> + </literallayout> + By default, the + <filename>devtool update-recipe</filename> command + creates the patch files in a folder named the same + as the recipe beneath the folder in which the recipe + resides, and updates the recipe's + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + statement to point to the generated patch files. + </para></listitem> + <listitem><para><emphasis>Move the Recipe to its Permanent Layer</emphasis>: + Before cleaning up the workspace, you need to move the + final recipe to its permanent layer. + You can either overwrite the original recipe or you can + overlay the upgraded recipe into a separate layer. + You must do this before using the + <filename>devtool reset</filename> command if you want to + retain the upgraded recipe. + </para></listitem> + <listitem><para><emphasis>Restore the Workspace</emphasis>: + The <filename>devtool reset</filename> restores the + state so that standard layers and upstream sources are + used to build the recipe rather than what is in the + workspace. + <literallayout class='monospaced'> + $ devtool reset <replaceable>recipe</replaceable> + </literallayout> + </para></listitem> + </orderedlist> + </para> + </section> + </section> + + <section id='devtool-quick-reference'> + <title><filename>devtool</filename> Quick Reference</title> + + <para> + <filename>devtool</filename> has more functionality than simply + adding a new recipe and the supporting Metadata to a temporary + workspace layer. + This section provides a short reference on + <filename>devtool</filename> and its commands. + </para> + + <section id='devtool-getting-help'> + <title>Getting Help</title> + + <para> + The easiest way to get help with the + <filename>devtool</filename> command is using the + <filename>--help</filename> option: + <literallayout class='monospaced'> + usage: devtool [--basepath BASEPATH] [--bbpath BBPATH] [-d] [-q] + [--color COLOR] [-h] + <subcommand> ... + + OpenEmbedded development tool + + optional arguments: + --basepath BASEPATH Base directory of SDK / build directory + --bbpath BBPATH Explicitly specify the BBPATH, rather than getting it + from the metadata + -d, --debug Enable debug output + -q, --quiet Print only errors + --color COLOR Colorize output (where COLOR is auto, always, never) + -h, --help show this help message and exit + + subcommands: + Beginning work on a recipe: + add Add a new recipe + modify Modify the source for an existing recipe + upgrade Upgrade an existing recipe + Getting information: + status Show workspace status + search Search available recipes + Working on a recipe in the workspace: + build Build a recipe + edit-recipe Edit a recipe file in your workspace + configure-help Get help on configure script options + update-recipe Apply changes from external source tree to recipe + reset Remove a recipe from your workspace + Testing changes on target: + deploy-target Deploy recipe output files to live target machine + undeploy-target Undeploy recipe output files in live target machine + build-image Build image including workspace recipe packages + Advanced: + create-workspace Set up workspace in an alternative location + extract Extract the source for an existing recipe + sync Synchronize the source tree for an existing recipe + Use devtool <subcommand> --help to get help on a specific command + </literallayout> + </para> + + <para> + As directed in the general help output, you can get more + syntax on a specific command by providing the command + name and using <filename>--help</filename>: + <literallayout class='monospaced'> + $ devtool add --help + usage: devtool add [-h] [--same-dir | --no-same-dir] [--fetch URI] + [--version VERSION] [--no-git] [--binary] [--also-native] + [--src-subdir SUBDIR] + [recipename] [srctree] [fetchuri] + + Adds a new recipe to the workspace to build a specified source tree. Can + optionally fetch a remote URI and unpack it to create the source tree. + + positional arguments: + recipename Name for new recipe to add (just name - no version, + path or extension). If not specified, will attempt to + auto-detect it. + srctree Path to external source tree. If not specified, a + subdirectory of + /home/scottrif/poky/build/workspace/sources will be + used. + fetchuri Fetch the specified URI and extract it to create the + source tree + + optional arguments: + -h, --help show this help message and exit + --same-dir, -s Build in same directory as source + --no-same-dir Force build in a separate build directory + --fetch URI, -f URI Fetch the specified URI and extract it to create the + source tree (deprecated - pass as positional argument + instead) + --version VERSION, -V VERSION + Version to use within recipe (PV) + --no-git, -g If fetching source, do not set up source tree as a git + repository + --binary, -b Treat the source tree as something that should be + installed verbatim (no compilation, same directory + structure). Useful with binary packages e.g. RPMs. + --also-native Also add native variant (i.e. support building recipe + for the build host as well as the target machine) + --src-subdir SUBDIR Specify subdirectory within source tree to use + </literallayout> + </para> + </section> + + <section id='devtool-the-workspace-layer-structure'> + <title>The Workspace Layer Structure</title> + + <para> + <filename>devtool</filename> uses a "Workspace" layer + in which to accomplish builds. + This layer is not specific to any single + <filename>devtool</filename> command but is rather a common + working area used across the tool. + </para> + + <para> + The following figure shows the workspace structure: + </para> + + <para> + <imagedata fileref="figures/build-workspace-directory.png" + width="6in" depth="5in" align="left" scale="70" /> + </para> + + <para> + <literallayout class='monospaced'> + attic - A directory created if devtool believes it preserve + anything when you run "devtool reset". For example, if you + run "devtool add", make changes to the recipe, and then + run "devtool reset", devtool takes notice that the file has + been changed and moves it into the attic should you still + want the recipe. + + README - Provides information on what is in workspace layer and how to + manage it. + + .devtool_md5 - A checksum file used by devtool. + + appends - A directory that contains *.bbappend files, which point to + external source. + + conf - A configuration directory that contains the layer.conf file. + + recipes - A directory containing recipes. This directory contains a + folder for each directory added whose name matches that of the + added recipe. devtool places the <replaceable>recipe</replaceable>.bb file + within that sub-directory. + + sources - A directory containing a working copy of the source files used + when building the recipe. This is the default directory used + as the location of the source tree when you do not provide a + source tree path. This directory contains a folder for each + set of source files matched to a corresponding recipe. + </literallayout> + </para> + </section> + + <section id='devtool-adding-a-new-recipe-to-the-workspace'> + <title>Adding a New Recipe to the Workspace Layer</title> + + <para> + Use the <filename>devtool add</filename> command to add a new recipe + to the workspace layer. + The recipe you add should not exist - + <filename>devtool</filename> creates it for you. + The source files the recipe uses should exist in an external + area. + </para> + + <para> + The following example creates and adds a new recipe named + <filename>jackson</filename> to a workspace layer the tool creates. + The source code built by the recipes resides in + <filename>/home/scottrif/sources/jackson</filename>: + <literallayout class='monospaced'> + $ devtool add jackson /home/scottrif/sources/jackson + </literallayout> + </para> + + <para> + If you add a recipe and the workspace layer does not exist, + the command creates the layer and populates it as + described in + "<link linkend='devtool-the-workspace-layer-structure'>The Workspace Layer Structure</link>" + section. + </para> + + <para> + Running <filename>devtool add</filename> when the + workspace layer exists causes the tool to add the recipe, + append files, and source files into the existing workspace layer. + The <filename>.bbappend</filename> file is created to point + to the external source tree. + </para> + </section> + + <section id='devtool-extracting-the-source-for-an-existing-recipe'> + <title>Extracting the Source for an Existing Recipe</title> + + <para> + Use the <filename>devtool extract</filename> command to + extract the source for an existing recipe. + When you use this command, you must supply the root name + of the recipe (i.e. no version, paths, or extensions), and + you must supply the directory to which you want the source + extracted. + </para> + + <para> + Additional command options let you control the name of a + development branch into which you can checkout the source + and whether or not to keep a temporary directory, which is + useful for debugging. + </para> + </section> + + <section id='devtool-synchronizing-a-recipes-extracted-source-tree'> + <title>Synchronizing a Recipe's Extracted Source Tree</title> + + <para> + Use the <filename>devtool sync</filename> command to + synchronize a previously extracted source tree for an + existing recipe. + When you use this command, you must supply the root name + of the recipe (i.e. no version, paths, or extensions), and + you must supply the directory to which you want the source + extracted. + </para> + + <para> + Additional command options let you control the name of a + development branch into which you can checkout the source + and whether or not to keep a temporary directory, which is + useful for debugging. + </para> + </section> + + <section id='devtool-modifying-a-recipe'> + <title>Modifying an Existing Recipe</title> + + <para> + Use the <filename>devtool modify</filename> command to begin + modifying the source of an existing recipe. + This command is very similar to the + <link linkend='devtool-adding-a-new-recipe-to-the-workspace'><filename>add</filename></link> + command except that it does not physically create the + recipe in the workspace layer because the recipe already + exists in an another layer. + </para> + + <para> + The <filename>devtool modify</filename> command extracts the + source for a recipe, sets it up as a Git repository if the + source had not already been fetched from Git, checks out a + branch for development, and applies any patches from the recipe + as commits on top. + You can use the following command to checkout the source + files: + <literallayout class='monospaced'> + $ devtool modify <replaceable>recipe</replaceable> + </literallayout> + Using the above command form, <filename>devtool</filename> uses + the existing recipe's + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + statement to locate the upstream source, extracts the source + into the default sources location in the workspace. + The default development branch used is "devtool". + </para> + </section> + + <section id='devtool-edit-an-existing-recipe'> + <title>Edit an Existing Recipe</title> + + <para> + Use the <filename>devtool edit-recipe</filename> command + to run the default editor, which is identified using the + <filename>EDITOR</filename> variable, on the specified recipe. + </para> + + <para> + When you use the <filename>devtool edit-recipe</filename> + command, you must supply the root name of the recipe + (i.e. no version, paths, or extensions). + Also, the recipe file itself must reside in the workspace + as a result of the <filename>devtool add</filename> or + <filename>devtool upgrade</filename> commands. + However, you can override that requirement by using the + "-a" or "--any-recipe" option. + Using either of these options allows you to edit any recipe + regardless of its location. + </para> + </section> + + <section id='devtool-updating-a-recipe'> + <title>Updating a Recipe</title> + + <para> + Use the <filename>devtool update-recipe</filename> command to + update your recipe with patches that reflect changes you make + to the source files. + For example, if you know you are going to work on some + code, you could first use the + <link linkend='devtool-modifying-a-recipe'><filename>devtool modify</filename></link> + command to extract the code and set up the workspace. + After which, you could modify, compile, and test the code. + </para> + + <para> + When you are satisfied with the results and you have committed + your changes to the Git repository, you can then + run the <filename>devtool update-recipe</filename> to create the + patches and update the recipe: + <literallayout class='monospaced'> + $ devtool update-recipe <replaceable>recipe</replaceable> + </literallayout> + If you run the <filename>devtool update-recipe</filename> + without committing your changes, the command ignores the + changes. + </para> + + <para> + Often, you might want to apply customizations made to your + software in your own layer rather than apply them to the + original recipe. + If so, you can use the + <filename>-a</filename> or <filename>--append</filename> + option with the <filename>devtool update-recipe</filename> + command. + These options allow you to specify the layer into which to + write an append file: + <literallayout class='monospaced'> + $ devtool update-recipe <replaceable>recipe</replaceable> -a <replaceable>base-layer-directory</replaceable> + </literallayout> + The <filename>*.bbappend</filename> file is created at the + appropriate path within the specified layer directory, which + may or may not be in your <filename>bblayers.conf</filename> + file. + If an append file already exists, the command updates it + appropriately. + </para> + </section> + + <section id='devtool-upgrading-a-recipe'> + <title>Upgrading a Recipe</title> + + <para> + Use the <filename>devtool upgrade</filename> command + to upgrade an existing recipe to a new upstream version. + The command puts the upgraded recipe file into the + workspace along with any associated files, and extracts + the source tree to a specified location should patches + need rebased or added to as a result of the upgrade. + </para> + + <para> + When you use the <filename>devtool upgrade</filename> command, + you must supply the root name of the recipe (i.e. no version, + paths, or extensions), and you must supply the directory + to which you want the source extracted. + Additional command options let you control things such as + the version number to which you want to upgrade (i.e. the + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>), + the source revision to which you want to upgrade (i.e. the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>, + whether or not to apply patches, and so forth. + </para> + </section> + + <section id='devtool-resetting-a-recipe'> + <title>Resetting a Recipe</title> + + <para> + Use the <filename>devtool reset</filename> command to remove a + recipe and its configuration (e.g. the corresponding + <filename>.bbappend</filename> file) from the workspace layer. + Realize that this command deletes the recipe and the + append file. + The command does not physically move them for you. + Consequently, you must be sure to physically relocate your + updated recipe and the append file outside of the workspace + layer before running the <filename>devtool reset</filename> + command. + </para> + + <para> + If the <filename>devtool reset</filename> command detects that + the recipe or the append files have been modified, the + command preserves the modified files in a separate "attic" + subdirectory under the workspace layer. + </para> + + <para> + Here is an example that resets the workspace directory that + contains the <filename>mtr</filename> recipe: + <literallayout class='monospaced'> + $ devtool reset mtr + NOTE: Cleaning sysroot for recipe mtr... + NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/mtr as-is; if you no + longer need it then please delete it manually + $ + </literallayout> + </para> + </section> + + <section id='devtool-building-your-recipe'> + <title>Building Your Recipe</title> + + <para> + Use the <filename>devtool build</filename> command to cause the + OpenEmbedded build system to build your recipe. + The <filename>devtool build</filename> command is equivalent to + <filename>bitbake -c populate_sysroot</filename>. + </para> + + <para> + When you use the <filename>devtool build</filename> command, + you must supply the root name of the recipe (i.e. no version, + paths, or extensions). + You can use either the "-s" or the "--disable-parallel-make" + option to disable parallel makes during the build. + Here is an example: + <literallayout class='monospaced'> + $ devtool build <replaceable>recipe</replaceable> + </literallayout> + </para> + </section> + + <section id='devtool-building-your-image'> + <title>Building Your Image</title> + + <para> + Use the <filename>devtool build-image</filename> command + to build an image, extending it to include packages from + recipes in the workspace. + Using this command is useful when you want an image that + ready for immediate deployment onto a device for testing. + For proper integration into a final image, you need to + edit your custom image recipe appropriately. + </para> + + <para> + When you use the <filename>devtool build-image</filename> + command, you must supply the name of the image. + This command has no command line options: + <literallayout class='monospaced'> + $ devtool build-image <replaceable>image</replaceable> + </literallayout> + </para> + </section> + + <section id='devtool-deploying-your-software-on-the-target-machine'> + <title>Deploying Your Software on the Target Machine</title> + + <para> + Use the <filename>devtool deploy-target</filename> command to + deploy the recipe's build output to the live target machine: + <literallayout class='monospaced'> + $ devtool deploy-target <replaceable>recipe</replaceable> <replaceable>target</replaceable> + </literallayout> + The <replaceable>target</replaceable> is the address of the + target machine, which must be running an SSH server (i.e. + <filename>user@hostname[:destdir]</filename>). + </para> + + <para> + This command deploys all files installed during the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> + task. + Furthermore, you do not need to have package management enabled + within the target machine. + If you do, the package manager is bypassed. + <note><title>Notes</title> + <para> + The <filename>deploy-target</filename> + functionality is for development only. + You should never use it to update an image that will be + used in production. + </para> + </note> + </para> + </section> + + <section id='devtool-removing-your-software-from-the-target-machine'> + <title>Removing Your Software from the Target Machine</title> + + <para> + Use the <filename>devtool undeploy-target</filename> command to + remove deployed build output from the target machine. + For the <filename>devtool undeploy-target</filename> command to + work, you must have previously used the + <link linkend='devtool-deploying-your-software-on-the-target-machine'><filename>devtool deploy-target</filename></link> + command. + <literallayout class='monospaced'> + $ devtool undeploy-target <replaceable>recipe</replaceable> <replaceable>target</replaceable> + </literallayout> + The <replaceable>target</replaceable> is the address of the + target machine, which must be running an SSH server (i.e. + <filename>user@hostname</filename>). + </para> + </section> + + <section id='devtool-creating-the-workspace'> + <title>Creating the Workspace Layer in an Alternative Location</title> + + <para> + Use the <filename>devtool create-workspace</filename> command to + create a new workspace layer in your + <link linkend='build-directory'>Build Directory</link>. + When you create a new workspace layer, it is populated with the + <filename>README</filename> file and the + <filename>conf</filename> directory only. + </para> + + <para> + The following example creates a new workspace layer in your + current working and by default names the workspace layer + "workspace": + <literallayout class='monospaced'> + $ devtool create-workspace + </literallayout> + </para> + + <para> + You can create a workspace layer anywhere by supplying + a pathname with the command. + The following command creates a new workspace layer named + "new-workspace": + <literallayout class='monospaced'> + $ devtool create-workspace /home/scottrif/new-workspace + </literallayout> + </para> + </section> + + <section id='devtool-get-the-status-of-the-recipes-in-your-workspace'> + <title>Get the Status of the Recipes in Your Workspace</title> + + <para> + Use the <filename>devtool status</filename> command to + list the recipes currently in your workspace. + Information includes the paths to their respective + external source trees. + </para> + + <para> + The <filename>devtool status</filename> command has no + command-line options: + <literallayout class='monospaced'> + devtool status + </literallayout> + Following is sample output after using + <link linkend='devtool-adding-a-new-recipe-to-the-workspace'><filename>devtool add</filename></link> + to create and add the <filename>mtr_0.86.bb</filename> recipe + to the <filename>workspace</filename> directory: + <literallayout class='monospaced'> + $ devtool status + mtr: /home/scottrif/poky/build/workspace/sources/mtr (/home/scottrif/poky/build/workspace/recipes/mtr/mtr_0.86.bb) + $ + </literallayout> + </para> + </section> + + <section id='devtool-search-for-available-target-recipes'> + <title>Search for Available Target Recipes</title> + + <para> + Use the <filename>devtool search</filename> command to + search for available target recipes. + The command matches the recipe name, package name, + description, and installed files. + The command displays the recipe name as a result of a + match. + </para> + + <para> + When you use the <filename>devtool search</filename> command, + you must supply a <replaceable>keyword</replaceable>. + The command uses the <replaceable>keyword</replaceable> when + searching for a match. + </para> + </section> + </section> + + <section id="using-a-quilt-workflow"> + <title>Using Quilt in Your Workflow</title> + + <para> + <ulink url='http://savannah.nongnu.org/projects/quilt'>Quilt</ulink> + is a powerful tool that allows you to capture source code changes + without having a clean source tree. + This section outlines the typical workflow you can use to modify + source code, test changes, and then preserve the changes in the + form of a patch all using Quilt. + <note><title>Tip</title> + With regard to preserving changes to source files if you + clean a recipe or have <filename>rm_work</filename> enabled, + the workflow described in the + "<link linkend='using-devtool-in-your-workflow'>Using <filename>devtool</filename> in Your Workflow</link>" + section is a safer development flow than than the flow that + uses Quilt. + </note> + </para> + + <para> + Follow these general steps: + <orderedlist> + <listitem><para><emphasis>Find the Source Code:</emphasis> + Temporary source code used by the OpenEmbedded build system + is kept in the + <link linkend='build-directory'>Build Directory</link>. + See the + "<link linkend='finding-the-temporary-source-code'>Finding Temporary Source Code</link>" + section to learn how to locate the directory that has the + temporary source code for a particular package. + </para></listitem> + <listitem><para><emphasis>Change Your Working Directory:</emphasis> + You need to be in the directory that has the temporary source code. + That directory is defined by the + <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> + variable.</para></listitem> + <listitem><para><emphasis>Create a New Patch:</emphasis> + Before modifying source code, you need to create a new patch. + To create a new patch file, use <filename>quilt new</filename> as below: + <literallayout class='monospaced'> + $ quilt new my_changes.patch + </literallayout></para></listitem> + <listitem><para><emphasis>Notify Quilt and Add Files:</emphasis> + After creating the patch, you need to notify Quilt about the files + you plan to edit. + You notify Quilt by adding the files to the patch you just created: + <literallayout class='monospaced'> + $ quilt add file1.c file2.c file3.c + </literallayout> + </para></listitem> + <listitem><para><emphasis>Edit the Files:</emphasis> + Make your changes in the source code to the files you added + to the patch. + </para></listitem> + <listitem><para><emphasis>Test Your Changes:</emphasis> + Once you have modified the source code, the easiest way to + your changes is by calling the + <filename>do_compile</filename> task as shown in the + following example: + <literallayout class='monospaced'> + $ bitbake -c compile -f <replaceable>package</replaceable> + </literallayout> + The <filename>-f</filename> or <filename>--force</filename> + option forces the specified task to execute. + If you find problems with your code, you can just keep editing and + re-testing iteratively until things work as expected. + <note>All the modifications you make to the temporary source code + disappear once you run the + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-clean'><filename>do_clean</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleanall'><filename>do_cleanall</filename></ulink> + tasks using BitBake (i.e. + <filename>bitbake -c clean <replaceable>package</replaceable></filename> + and + <filename>bitbake -c cleanall <replaceable>package</replaceable></filename>). + Modifications will also disappear if you use the <filename>rm_work</filename> + feature as described in the + "<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>" + section of the Yocto Project Quick Start. + </note></para></listitem> + <listitem><para><emphasis>Generate the Patch:</emphasis> + Once your changes work as expected, you need to use Quilt to generate the final patch that + contains all your modifications. + <literallayout class='monospaced'> + $ quilt refresh + </literallayout> + At this point, the <filename>my_changes.patch</filename> file has all your edits made + to the <filename>file1.c</filename>, <filename>file2.c</filename>, and + <filename>file3.c</filename> files.</para> + <para>You can find the resulting patch file in the <filename>patches/</filename> + subdirectory of the source (<filename>S</filename>) directory.</para></listitem> + <listitem><para><emphasis>Copy the Patch File:</emphasis> + For simplicity, copy the patch file into a directory named <filename>files</filename>, + which you can create in the same directory that holds the recipe + (<filename>.bb</filename>) file or the + append (<filename>.bbappend</filename>) file. + Placing the patch here guarantees that the OpenEmbedded build system will find + the patch. + Next, add the patch into the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> + of the recipe. + Here is an example: + <literallayout class='monospaced'> + SRC_URI += "file://my_changes.patch" + </literallayout></para></listitem> + </orderedlist> + </para> + </section> + + <section id='finding-the-temporary-source-code'> + <title>Finding Temporary Source Code</title> + + <para> + You might find it helpful during development to modify the + temporary source code used by recipes to build packages. + For example, suppose you are developing a patch and you need to + experiment a bit to figure out your solution. + After you have initially built the package, you can iteratively + tweak the source code, which is located in the + <link linkend='build-directory'>Build Directory</link>, and then + you can force a re-compile and quickly test your altered code. + Once you settle on a solution, you can then preserve your changes + in the form of patches. + If you are using Quilt for development, see the + "<link linkend='using-a-quilt-workflow'>Using Quilt in Your Workflow</link>" + section for more information. + </para> + + <para> + During a build, the unpacked temporary source code used by recipes + to build packages is available in the Build Directory as + defined by the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename> variable. + Below is the default value for the <filename>S</filename> variable as defined in the + <filename>meta/conf/bitbake.conf</filename> configuration file in the + <link linkend='source-directory'>Source Directory</link>: + <literallayout class='monospaced'> + S = "${WORKDIR}/${BP}" + </literallayout> + You should be aware that many recipes override the <filename>S</filename> variable. + For example, recipes that fetch their source from Git usually set + <filename>S</filename> to <filename>${WORKDIR}/git</filename>. + <note> + The + <ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink> + represents the base recipe name, which consists of the name and version: + <literallayout class='monospaced'> + BP = "${BPN}-${PV}" + </literallayout> + </note> + </para> + + <para> + The path to the work directory for the recipe + (<ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>) + is defined as follows: + <literallayout class='monospaced'> + ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} + </literallayout> + The actual directory depends on several things: + <itemizedlist> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>: + The top-level build output directory</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-MULTIMACH_TARGET_SYS'><filename>MULTIMACH_TARGET_SYS</filename></ulink>: + The target system identifier</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>: + The recipe name</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-EXTENDPE'><filename>EXTENDPE</filename></ulink>: + The epoch - (if + <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink> + is not specified, which is usually the case for most + recipes, then <filename>EXTENDPE</filename> is blank)</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>: + The recipe version</listitem> + <listitem><ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>: + The recipe revision</listitem> + </itemizedlist> + </para> + + <para> + As an example, assume a Source Directory top-level folder + named <filename>poky</filename>, a default Build Directory at + <filename>poky/build</filename>, and a + <filename>qemux86-poky-linux</filename> machine target + system. + Furthermore, suppose your recipe is named + <filename>foo_1.3.0.bb</filename>. + In this case, the work directory the build system uses to + build the package would be as follows: + <literallayout class='monospaced'> + poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 + </literallayout> + </para> + + <para> + Now that you know where to locate the directory that has the + temporary source code, you can use a Quilt as described in section + "<link linkend='using-a-quilt-workflow'>Using Quilt in Your Workflow</link>" + to make your edits, test the changes, and preserve the changes in + the form of patches. + </para> + </section> +</section> + +<section id='image-development-using-toaster'> + <title>Image Development Using Toaster</title> + + <para> + Toaster is a web interface to the Yocto Project's OpenEmbedded build + system. + You can initiate builds using Toaster as well as examine the results + and statistics of builds. + See the + <ulink url='&YOCTO_DOCS_TOAST_URL;#toaster-manual-intro'>Toaster User Manual</ulink> + for information on how to set up and use Toaster to build images. + </para> +</section> + +<section id="platdev-appdev-devshell"> + <title>Using a Development Shell</title> + + <para> + When debugging certain commands or even when just editing packages, + <filename>devshell</filename> can be a useful tool. + When you invoke <filename>devshell</filename>, all tasks up to and + including + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> + are run for the specified target. + Then, a new terminal is opened and you are placed in + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>, + the source directory. + In the new terminal, all the OpenEmbedded build-related environment variables are + still defined so you can use commands such as <filename>configure</filename> and + <filename>make</filename>. + The commands execute just as if the OpenEmbedded build system were executing them. + Consequently, working this way can be helpful when debugging a build or preparing + software to be used with the OpenEmbedded build system. + </para> + + <para> + Following is an example that uses <filename>devshell</filename> on a target named + <filename>matchbox-desktop</filename>: + <literallayout class='monospaced'> + $ bitbake matchbox-desktop -c devshell + </literallayout> + </para> + + <para> + This command spawns a terminal with a shell prompt within the OpenEmbedded build environment. + The <ulink url='&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL'><filename>OE_TERMINAL</filename></ulink> + variable controls what type of shell is opened. + </para> + + <para> + For spawned terminals, the following occurs: + <itemizedlist> + <listitem><para>The <filename>PATH</filename> variable includes the + cross-toolchain.</para></listitem> + <listitem><para>The <filename>pkgconfig</filename> variables find the correct + <filename>.pc</filename> files.</para></listitem> + <listitem><para>The <filename>configure</filename> command finds the + Yocto Project site files as well as any other necessary files.</para></listitem> + </itemizedlist> + </para> + + <para> + Within this environment, you can run configure or compile + commands as if they were being run by + the OpenEmbedded build system itself. + As noted earlier, the working directory also automatically changes to the + Source Directory (<ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>). + </para> + + <para> + To manually run a specific task using <filename>devshell</filename>, + run the corresponding <filename>run.*</filename> script in + the + <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/temp</filename> + directory (e.g., + <filename>run.do_configure.</filename><replaceable>pid</replaceable>). + If a task's script does not exist, which would be the case if the task was + skipped by way of the sstate cache, you can create the task by first running + it outside of the <filename>devshell</filename>: + <literallayout class='monospaced'> + $ bitbake -c <replaceable>task</replaceable> + </literallayout> + <note><title>Notes</title> + <itemizedlist> + <listitem><para>Execution of a task's <filename>run.*</filename> + script and BitBake's execution of a task are identical. + In other words, running the script re-runs the task + just as it would be run using the + <filename>bitbake -c</filename> command. + </para></listitem> + <listitem><para>Any <filename>run.*</filename> file that does not + have a <filename>.pid</filename> extension is a + symbolic link (symlink) to the most recent version of that + file. + </para></listitem> + </itemizedlist> + </note> + </para> + + <para> + Remember, that the <filename>devshell</filename> is a mechanism that allows + you to get into the BitBake task execution environment. + And as such, all commands must be called just as BitBake would call them. + That means you need to provide the appropriate options for + cross-compilation and so forth as applicable. + </para> + + <para> + When you are finished using <filename>devshell</filename>, exit the shell + or close the terminal window. + </para> + + <note><title>Notes</title> + <itemizedlist> + <listitem><para> + It is worth remembering that when using <filename>devshell</filename> + you need to use the full compiler name such as <filename>arm-poky-linux-gnueabi-gcc</filename> + instead of just using <filename>gcc</filename>. + The same applies to other applications such as <filename>binutils</filename>, + <filename>libtool</filename> and so forth. + BitBake sets up environment variables such as <filename>CC</filename> + to assist applications, such as <filename>make</filename> to find the correct tools. + </para></listitem> + <listitem><para> + It is also worth noting that <filename>devshell</filename> still works over + X11 forwarding and similar situations. + </para></listitem> + </itemizedlist> + </note> +</section> + +</chapter> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-newbie.xml b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-newbie.xml new file mode 100644 index 000000000..75c992f16 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-newbie.xml @@ -0,0 +1,1710 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<chapter id='dev-manual-newbie'> + +<title>The Yocto Project Open Source Development Environment</title> + +<para> + This chapter helps you understand the Yocto Project as an open source development project. + In general, working in an open source environment is very different from working in a + closed, proprietary environment. + Additionally, the Yocto Project uses specific tools and constructs as part of its development + environment. + This chapter specifically addresses open source philosophy, using the + Yocto Project in a team environment, source repositories, Yocto Project + terms, licensing, the open source distributed version control system Git, + workflows, bug tracking, and how to submit changes. +</para> + +<section id='open-source-philosophy'> + <title>Open Source Philosophy</title> + + <para> + Open source philosophy is characterized by software development directed by peer production + and collaboration through an active community of developers. + Contrast this to the more standard centralized development models used by commercial software + companies where a finite set of developers produces a product for sale using a defined set + of procedures that ultimately result in an end product whose architecture and source material + are closed to the public. + </para> + + <para> + Open source projects conceptually have differing concurrent agendas, approaches, and production. + These facets of the development process can come from anyone in the public (community) that has a + stake in the software project. + The open source environment contains new copyright, licensing, domain, and consumer issues + that differ from the more traditional development environment. + In an open source environment, the end product, source material, and documentation are + all available to the public at no cost. + </para> + + <para> + A benchmark example of an open source project is the Linux kernel, which was initially conceived + and created by Finnish computer science student Linus Torvalds in 1991. + Conversely, a good example of a non-open source project is the + <trademark class='registered'>Windows</trademark> family of operating + systems developed by <trademark class='registered'>Microsoft</trademark> Corporation. + </para> + + <para> + Wikipedia has a good historical description of the Open Source Philosophy + <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>. + You can also find helpful information on how to participate in the Linux Community + <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>. + </para> +</section> + +<section id="usingpoky-changes-collaborate"> + <title>Using the Yocto Project in a Team Environment</title> + + <para> + It might not be immediately clear how you can use the Yocto + Project in a team environment, or scale it for a large team of + developers. + One of the strengths of the Yocto Project is that it is extremely + flexible. + Thus, you can adapt it to many different use cases and scenarios. + However, these characteristics can cause a struggle if you are trying + to create a working setup that scales across a large team. + </para> + + <para> + To help with these types of situations, this section presents + some of the project's most successful experiences, + practices, solutions, and available technologies that work well. + Keep in mind, the information here is a starting point. + You can build off it and customize it to fit any + particular working environment and set of practices. + </para> + + <section id='best-practices-system-configurations'> + <title>System Configurations</title> + + <para> + Systems across a large team should meet the needs of + two types of developers: those working on the contents of the + operating system image itself and those developing applications. + Regardless of the type of developer, their workstations must + be both reasonably powerful and run Linux. + </para> + + <section id='best-practices-application-development'> + <title>Application Development</title> + + <para> + For developers who mainly do application level work + on top of an existing software stack, + the following list shows practices that work best. + For information on using a Software Development Kit (SDK), see + the + <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-intro'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>: + <itemizedlist> + <listitem><para>Use a pre-built toolchain that + contains the software stack itself. + Then, develop the application code on top of the + stack. + This method works well for small numbers of relatively + isolated applications.</para></listitem> + <listitem><para>When possible, use the Yocto Project + plug-in for the <trademark class='trade'>Eclipse</trademark> IDE + and SDK development practices. + For more information, see the + "<ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>". + </para></listitem> + <listitem><para>Keep your cross-development toolchains + updated. + You can do this through provisioning either as new + toolchain downloads or as updates through a package + update mechanism using <filename>opkg</filename> + to provide updates to an existing toolchain. + The exact mechanics of how and when to do this are a + question for local policy.</para></listitem> + <listitem><para>Use multiple toolchains installed locally + into different locations to allow development across + versions.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id='best-practices-core-system-development'> + <title>Core System Development</title> + + <para> + For core system development, it is often best to have the + build system itself available on the developer workstations + so developers can run their own builds and directly + rebuild the software stack. + You should keep the core system unchanged as much as + possible and do your work in layers on top of the core system. + Doing so gives you a greater level of portability when + upgrading to new versions of the core system or Board + Support Packages (BSPs). + You can share layers amongst the developers of a particular + project and contain the policy configuration that defines + the project. + </para> + + <para> + Aside from the previous best practices, there exists a number + of tips and tricks that can help speed up core development + projects: + <itemizedlist> + <listitem><para>Use a + <ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink> + (sstate) among groups of developers who are on a + fast network. + The best way to share sstate is through a + Network File System (NFS) share. + The first user to build a given component for the + first time contributes that object to the sstate, + while subsequent builds from other developers then + reuse the object rather than rebuild it themselves. + </para> + <para>Although it is possible to use other protocols for the + sstate such as HTTP and FTP, you should avoid these. + Using HTTP limits the sstate to read-only and + FTP provides poor performance. + </para></listitem> + <listitem><para>Have autobuilders contribute to the sstate + pool similarly to how the developer workstations + contribute. + For information, see the + "<link linkend='best-practices-autobuilders'>Autobuilders</link>" + section.</para></listitem> + <listitem><para>Build stand-alone tarballs that contain + "missing" system requirements if for some reason + developer workstations do not meet minimum system + requirements such as latest Python versions, + <filename>chrpath</filename>, or other tools. + You can install and relocate the tarball exactly as you + would the usual cross-development toolchain so that + all developers can meet minimum version requirements + on most distributions.</para></listitem> + <listitem><para>Use a small number of shared, + high performance systems for testing purposes + (e.g. dual, six-core Xeons with 24 Gbytes of RAM + and plenty of disk space). + Developers can use these systems for wider, more + extensive testing while they continue to develop + locally using their primary development system. + </para></listitem> + <listitem><para>Enable the PR Service when package feeds + need to be incremental with continually increasing + <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'>PR</ulink> + values. + Typically, this situation occurs when you use or + publish package feeds and use a shared state. + You should enable the PR Service for all users who + use the shared state pool. + For more information on the PR Service, see the + "<link linkend='working-with-a-pr-service'>Working With a PR Service</link>". + </para></listitem> + </itemizedlist> + </para> + </section> + </section> + + <section id='best-practices-source-control-management'> + <title>Source Control Management (SCM)</title> + + <para> + Keeping your + <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> + and any software you are developing under the + control of an SCM system that is compatible + with the OpenEmbedded build system is advisable. + Of the SCMs BitBake supports, the + Yocto Project team strongly recommends using + <link linkend='git'>Git</link>. + Git is a distributed system that is easy to backup, + allows you to work remotely, and then connects back to the + infrastructure. + <note> + For information about BitBake, see the + <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. + </note> + </para> + + <para> + It is relatively easy to set up Git services and create + infrastructure like + <ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>, + which is based on server software called + <filename>gitolite</filename> with <filename>cgit</filename> + being used to generate the web interface that lets you view the + repositories. + The <filename>gitolite</filename> software identifies users + using SSH keys and allows branch-based + access controls to repositories that you can control as little + or as much as necessary. + </para> + + <note> + The setup of these services is beyond the scope of this manual. + However, sites such as these exist that describe how to perform + setup: + <itemizedlist> + <listitem><para><ulink url='http://git-scm.com/book/ch4-8.html'>Git documentation</ulink>: + Describes how to install <filename>gitolite</filename> + on the server.</para></listitem> + <listitem><para><ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>: + All topics for <filename>gitolite</filename>. + </para></listitem> + <listitem><para><ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>: + Documentation on how to create interfaces and frontends + for Git.</para></listitem> + </itemizedlist> + </note> + </section> + + <section id='best-practices-autobuilders'> + <title>Autobuilders</title> + + <para> + Autobuilders are often the core of a development project. + It is here that changes from individual developers are brought + together and centrally tested and subsequent decisions about + releases can be made. + Autobuilders also allow for "continuous integration" style + testing of software components and regression identification + and tracking. + </para> + + <para> + See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>" + for more information and links to buildbot. + The Yocto Project team has found this implementation + works well in this role. + A public example of this is the Yocto Project + Autobuilders, which we use to test the overall health of the + project. + </para> + + <para> + The features of this system are: + <itemizedlist> + <listitem><para>Highlights when commits break the build. + </para></listitem> + <listitem><para>Populates an sstate cache from which + developers can pull rather than requiring local + builds.</para></listitem> + <listitem><para>Allows commit hook triggers, + which trigger builds when commits are made. + </para></listitem> + <listitem><para>Allows triggering of automated image booting + and testing under the QuickEMUlator (QEMU). + </para></listitem> + <listitem><para>Supports incremental build testing and + from-scratch builds.</para></listitem> + <listitem><para>Shares output that allows developer + testing and historical regression investigation. + </para></listitem> + <listitem><para>Creates output that can be used for releases. + </para></listitem> + <listitem><para>Allows scheduling of builds so that resources + can be used efficiently.</para></listitem> + </itemizedlist> + </para> + </section> + + <section id='best-practices-policies-and-change-flow'> + <title>Policies and Change Flow</title> + + <para> + The Yocto Project itself uses a hierarchical structure and a + pull model. + Scripts exist to create and send pull requests + (i.e. <filename>create-pull-request</filename> and + <filename>send-pull-request</filename>). + This model is in line with other open source projects where + maintainers are responsible for specific areas of the project + and a single maintainer handles the final "top-of-tree" merges. + </para> + + <note> + You can also use a more collective push model. + The <filename>gitolite</filename> software supports both the + push and pull models quite easily. + </note> + + <para> + As with any development environment, it is important + to document the policy used as well as any main project + guidelines so they are understood by everyone. + It is also a good idea to have well structured + commit messages, which are usually a part of a project's + guidelines. + Good commit messages are essential when looking back in time and + trying to understand why changes were made. + </para> + + <para> + If you discover that changes are needed to the core layer of the + project, it is worth sharing those with the community as soon + as possible. + Chances are if you have discovered the need for changes, someone + else in the community needs them also. + </para> + </section> + + <section id='best-practices-summary'> + <title>Summary</title> + + <para> + This section summarizes the key recommendations described in the + previous sections: + <itemizedlist> + <listitem><para>Use <link linkend='git'>Git</link> + as the source control system.</para></listitem> + <listitem><para>Maintain your Metadata in layers that make sense + for your situation. + See the "<link linkend='understanding-and-creating-layers'>Understanding + and Creating Layers</link>" section for more information on + layers.</para></listitem> + <listitem><para> + Separate the project's Metadata and code by using + separate Git repositories. + See the + "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>" + section for information on these repositories. + See the + "<link linkend='getting-setup'>Getting Set Up</link>" + section for information on how to set up local Git + repositories for related upstream Yocto Project + Git repositories. + </para></listitem> + <listitem><para>Set up the directory for the shared state cache + (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>) + where it makes sense. + For example, set up the sstate cache on a system used + by developers in the same organization and share the + same source directories on their machines. + </para></listitem> + <listitem><para>Set up an Autobuilder and have it populate the + sstate cache and source directories.</para></listitem> + <listitem><para>The Yocto Project community encourages you + to send patches to the project to fix bugs or add features. + If you do submit patches, follow the project commit + guidelines for writing good commit messages. + See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + section.</para></listitem> + <listitem><para>Send changes to the core sooner than later + as others are likely to run into the same issues. + For some guidance on mailing lists to use, see the list in the + "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + section. + For a description of the available mailing lists, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>" + section in the Yocto Project Reference Manual. + </para></listitem> + </itemizedlist> + </para> + </section> +</section> + +<section id='yocto-project-repositories'> + <title>Yocto Project Source Repositories</title> + + <para> + The Yocto Project team maintains complete source repositories for all + Yocto Project files at + <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>. + This web-based source code browser is organized into categories by + function such as IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and + so forth. + From the interface, you can click on any particular item in the "Name" + column and see the URL at the bottom of the page that you need to clone + a Git repository for that particular item. + Having a local Git repository of the + <link linkend='source-directory'>Source Directory</link>, which is + usually named "poky", allows + you to make changes, contribute to the history, and ultimately enhance + the Yocto Project's tools, Board Support Packages, and so forth. + </para> + + <para> + For any supported release of Yocto Project, you can also go to the + <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and + select the "Downloads" tab and get a released tarball of the + <filename>poky</filename> repository or any supported BSP tarballs. + Unpacking these tarballs gives you a snapshot of the released + files. + <note><title>Notes</title> + <itemizedlist> + <listitem><para> + The recommended method for setting up the Yocto Project + <link linkend='source-directory'>Source Directory</link> + and the files for supported BSPs + (e.g., <filename>meta-intel</filename>) is to use + <link linkend='git'>Git</link> to create a local copy of + the upstream repositories. + </para></listitem> + <listitem><para> + Be sure to always work in matching branches for both + the selected BSP repository and the + <link linkend='source-directory'>Source Directory</link> + (i.e. <filename>poky</filename>) repository. + For example, if you have checked out the "master" branch + of <filename>poky</filename> and you are going to use + <filename>meta-intel</filename>, be sure to checkout the + "master" branch of <filename>meta-intel</filename>. + </para></listitem> + </itemizedlist> + </note> + </para> + + <para> + In summary, here is where you can get the project files needed for development: + <itemizedlist> + <listitem><para id='source-repositories'><emphasis><ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink></emphasis> + This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto + Metadata Layers. + You can create local copies of Git repositories for each of these areas.</para> + <para> + <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" /> + </para></listitem> + <listitem><para><anchor id='index-downloads' /><emphasis><ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink></emphasis> + This is an index of releases such as + the <trademark class='trade'>Eclipse</trademark> + Yocto Plug-in, miscellaneous support, Poky, Pseudo, installers for cross-development toolchains, + and all released versions of Yocto Project in the form of images or tarballs. + Downloading and extracting these files does not produce a local copy of the + Git repository but rather a snapshot of a particular release or image.</para> + <para> + <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" /> + </para></listitem> + <listitem><para><emphasis>"Downloads" page for the + <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>:</emphasis> + Access this page by going to the website and then selecting + the "Downloads" tab. + This page allows you to download any Yocto Project + release or Board Support Package (BSP) in tarball form. + The tarballs are similar to those found in the + <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para> + <para> + <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" /> + </para></listitem> + </itemizedlist> + </para> +</section> + +<section id='yocto-project-terms'> + <title>Yocto Project Terms</title> + + <para> + Following is a list of terms and definitions users new to the Yocto Project development + environment might find helpful. + While some of these terms are universal, the list includes them just in case: + <itemizedlist> + <listitem><para><emphasis>Append Files:</emphasis> Files that append build information to + a recipe file. + Append files are known as BitBake append files and <filename>.bbappend</filename> files. + The OpenEmbedded build system expects every append file to have a corresponding + recipe (<filename>.bb</filename>) file. + Furthermore, the append file and corresponding recipe file + must use the same root filename. + The filenames can differ only in the file type suffix used (e.g. + <filename>formfactor_0.0.bb</filename> and <filename>formfactor_0.0.bbappend</filename>). + </para> + <para>Information in append files extends or overrides the + information in the similarly-named recipe file. + For an example of an append file in use, see the + "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" section. + <note> + Append files can also use wildcard patterns in their version numbers + so they can be applied to more than one version of the underlying recipe file. + </note> + </para></listitem> + <listitem><para id='bitbake-term'><emphasis>BitBake:</emphasis> + The task executor and scheduler used by the OpenEmbedded build + system to build images. + For more information on BitBake, see the + <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. + </para></listitem> + <listitem> + <para id='build-directory'><emphasis>Build Directory:</emphasis> + This term refers to the area used by the OpenEmbedded build + system for builds. + The area is created when you <filename>source</filename> the + setup environment script that is found in the Source Directory + (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>). + The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink> + variable points to the Build Directory.</para> + + <para> + You have a lot of flexibility when creating the Build + Directory. + Following are some examples that show how to create the + directory. + The examples assume your + <link linkend='source-directory'>Source Directory</link> is + named <filename>poky</filename>: + <itemizedlist> + <listitem><para>Create the Build Directory inside your + Source Directory and let the name of the Build + Directory default to <filename>build</filename>: + <literallayout class='monospaced'> + $ cd $HOME/poky + $ source &OE_INIT_FILE; + </literallayout></para></listitem> + <listitem><para>Create the Build Directory inside your + home directory and specifically name it + <filename>test-builds</filename>: + <literallayout class='monospaced'> + $ cd $HOME + $ source poky/&OE_INIT_FILE; test-builds + </literallayout></para></listitem> + <listitem><para> + Provide a directory path and + specifically name the Build Directory. + Any intermediate folders in the pathname must + exist. + This next example creates a Build Directory named + <filename>YP-&POKYVERSION;</filename> + in your home directory within the existing + directory <filename>mybuilds</filename>: + <literallayout class='monospaced'> + $cd $HOME + $ source $HOME/poky/&OE_INIT_FILE; $HOME/mybuilds/YP-&POKYVERSION; + </literallayout></para></listitem> + </itemizedlist> + <note> + By default, the Build Directory contains + <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>, + which is a temporary directory the build system uses for + its work. + <filename>TMPDIR</filename> cannot be under NFS. + Thus, by default, the Build Directory cannot be under NFS. + However, if you need the Build Directory to be under NFS, + you can set this up by setting <filename>TMPDIR</filename> + in your <filename>local.conf</filename> file + to use a local drive. + Doing so effectively separates <filename>TMPDIR</filename> + from <filename>TOPDIR</filename>, which is the Build + Directory. + </note> + </para></listitem> + <listitem><para><emphasis>Classes:</emphasis> Files that provide for logic encapsulation + and inheritance so that commonly used patterns can be defined once and then easily used + in multiple recipes. + For reference information on the Yocto Project classes, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes'>Classes</ulink>" chapter of the + Yocto Project Reference Manual. + Class files end with the <filename>.bbclass</filename> filename extension. + </para></listitem> + <listitem><para><emphasis>Configuration File:</emphasis> + Configuration information in various <filename>.conf</filename> + files provides global definitions of variables. + The <filename>conf/local.conf</filename> configuration file in + the + <link linkend='build-directory'>Build Directory</link> + contains user-defined variables that affect every build. + The <filename>meta-poky/conf/distro/poky.conf</filename> + configuration file defines Yocto "distro" configuration + variables used only when building with this policy. + Machine configuration files, which + are located throughout the + <link linkend='source-directory'>Source Directory</link>, define + variables for specific hardware and are only used when building + for that target (e.g. the + <filename>machine/beaglebone.conf</filename> configuration + file defines variables for the Texas Instruments ARM Cortex-A8 + development board). + Configuration files end with a <filename>.conf</filename> + filename extension. + </para></listitem> + <listitem><para id='cross-development-toolchain'> + <emphasis>Cross-Development Toolchain:</emphasis> + In general, a cross-development toolchain is a collection of + software development tools and utilities that run on one + architecture and allow you to develop software for a + different, or targeted, architecture. + These toolchains contain cross-compilers, linkers, and + debuggers that are specific to the target architecture. + </para> + + <para>The Yocto Project supports two different cross-development + toolchains: + <itemizedlist> + <listitem><para>A toolchain only used by and within + BitBake when building an image for a target + architecture.</para></listitem> + <listitem><para>A relocatable toolchain used outside of + BitBake by developers when developing applications + that will run on a targeted device. + </para></listitem> + </itemizedlist> + </para> + + <para> + Creation of these toolchains is simple and automated. + For information on toolchain concepts as they apply to the + Yocto Project, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain-generation'>Cross-Development Toolchain Generation</ulink>" + section in the Yocto Project Reference Manual. + You can also find more information on using the + relocatable toolchain in the + <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. + </para></listitem> + <listitem><para><emphasis>Image:</emphasis> + An image is an artifact of the BitBake build process given + a collection of recipes and related Metadata. + Images are the binary output that run on specific hardware or + QEMU and are used for specific use-cases. + For a list of the supported image types that the Yocto Project provides, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" + chapter in the Yocto Project Reference Manual.</para></listitem> + <listitem><para id='layer'><emphasis>Layer:</emphasis> A collection of recipes representing the core, + a BSP, or an application stack. + For a discussion specifically on BSP Layers, see the + "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" + section in the Yocto Project Board Support Packages (BSP) + Developer's Guide.</para></listitem> + <listitem><para id='metadata'><emphasis>Metadata:</emphasis> + The files that BitBake parses when building an image. + In general, Metadata includes recipes, classes, and + configuration files. + In the context of the kernel ("kernel Metadata"), + it refers to Metadata in the <filename>meta</filename> + branches of the kernel source Git repositories. + </para></listitem> + <listitem><para id='oe-core'><emphasis>OE-Core:</emphasis> A core set of Metadata originating + with OpenEmbedded (OE) that is shared between OE and the Yocto Project. + This Metadata is found in the <filename>meta</filename> directory of the + <link linkend='source-directory'>Source Directory</link>.</para></listitem> + <listitem><para id='build-system-term'><emphasis>OpenEmbedded Build System:</emphasis> + The build system specific to the Yocto Project. + The OpenEmbedded build system is based on another project known + as "Poky", which uses + <link linkend='bitbake-term'>BitBake</link> as the task + executor. + Throughout the Yocto Project documentation set, the + OpenEmbedded build system is sometimes referred to simply + as "the build system". + If other build systems, such as a host or target build system + are referenced, the documentation clearly states the + difference. + <note> + For some historical information about Poky, see the + <link linkend='poky'>Poky</link> term. + </note> + </para></listitem> + <listitem><para><emphasis>Package:</emphasis> + In the context of the Yocto Project, this term refers to a + recipe's packaged output produced by BitBake (i.e. a + "baked recipe"). + A package is generally the compiled binaries produced from the + recipe's sources. + You "bake" something by running it through BitBake.</para> + <para>It is worth noting that the term "package" can, in general, have subtle + meanings. For example, the packages referred to in the + "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Build Host Packages</ulink>" section are + compiled binaries that, when installed, add functionality to your Linux + distribution.</para> + <para>Another point worth noting is that historically within the Yocto Project, + recipes were referred to as packages - thus, the existence of several BitBake + variables that are seemingly mis-named, + (e.g. <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and + <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>). + </para></listitem> + <listitem><para><emphasis>Package Groups:</emphasis> + Arbitrary groups of software Recipes. + You use package groups to hold recipes that, when built, + usually accomplish a single task. + For example, a package group could contain the recipes for a + company’s proprietary or value-add software. + Or, the package group could contain the recipes that enable + graphics. + A package group is really just another recipe. + Because package group files are recipes, they end with the + <filename>.bb</filename> filename extension.</para></listitem> + <listitem><para id='poky'><emphasis>Poky:</emphasis> + The term "poky" can mean several things. + In its most general sense, it is an open-source + project that was initially developed by OpenedHand. + With OpenedHand, poky was developed off of the existing + OpenEmbedded build system becoming a commercially + supportable build system for embedded Linux. + After Intel Corporation acquired OpenedHand, the + project poky became the basis for the Yocto Project's + build system.</para> + <para>Within the Yocto Project source repositories, + <filename>poky</filename> exists as a separate Git + repository you can clone to yield a local copy on your + host system. + Thus, "poky" can refer to the local copy of the Source + Directory used for development within the Yocto + Project.</para> + <para>Finally, "poky" can refer to the default + <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> + (i.e. distribution) created when you use the Yocto + Project in conjunction with the + <filename>poky</filename> repository to build an image. + </para></listitem> + <listitem><para><emphasis>Recipe:</emphasis> + A set of instructions for building packages. + A recipe describes where you get source code, which patches + to apply, how to configure the source, how to compile it and so on. + Recipes also describe dependencies for libraries or for other + recipes. + Recipes represent the logical unit of execution, the software + to build, the images to build, and use the + <filename>.bb</filename> file extension. + </para></listitem> + <listitem> + <para id='source-directory'><emphasis>Source Directory:</emphasis> + This term refers to the directory structure created as a result + of creating a local copy of the <filename>poky</filename> Git + repository <filename>git://git.yoctoproject.org/poky</filename> + or expanding a released <filename>poky</filename> tarball. + <note> + Creating a local copy of the <filename>poky</filename> + Git repository is the recommended method for setting up + your Source Directory. + </note> + Sometimes you might hear the term "poky directory" used to refer + to this directory structure. + <note> + The OpenEmbedded build system does not support file or + directory names that contain spaces. + Be sure that the Source Directory you use does not contain + these types of names. + </note></para> + + <para>The Source Directory contains BitBake, Documentation, + Metadata and other files that all support the Yocto Project. + Consequently, you must have the Source Directory in place on + your development system in order to do any development using + the Yocto Project.</para> + + <para>When you create a local copy of the Git repository, you + can name the repository anything you like. + Throughout much of the documentation, "poky" + is used as the name of the top-level folder of the local copy of + the poky Git repository. + So, for example, cloning the <filename>poky</filename> Git + repository results in a local Git repository whose top-level + folder is also named "poky".</para> + + <para>While it is not recommended that you use tarball expansion + to set up the Source Directory, if you do, the top-level + directory name of the Source Directory is derived from the + Yocto Project release tarball. + For example, downloading and unpacking + <filename>&YOCTO_POKY_TARBALL;</filename> results in a + Source Directory whose root folder is named + <filename>&YOCTO_POKY;</filename>.</para> + + <para>It is important to understand the differences between the + Source Directory created by unpacking a released tarball as + compared to cloning + <filename>git://git.yoctoproject.org/poky</filename>. + When you unpack a tarball, you have an exact copy of the files + based on the time of release - a fixed release point. + Any changes you make to your local files in the Source Directory + are on top of the release and will remain local only. + On the other hand, when you clone the <filename>poky</filename> + Git repository, you have an active development repository with + access to the upstream repository's branches and tags. + In this case, any local changes you make to the local + Source Directory can be later applied to active development + branches of the upstream <filename>poky</filename> Git + repository.</para> + + <para>For more information on concepts related to Git + repositories, branches, and tags, see the + "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>" + section.</para></listitem> + <listitem><para><emphasis>Task:</emphasis> + A unit of execution for BitBake (e.g. + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>, + <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>, + and so forth). + </para></listitem> + <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories + that are not local to the development system but located in a master area that is controlled + by the maintainer of the source code. + For example, in order for a developer to work on a particular piece of code, they need to + first get a copy of it from an "upstream" source.</para></listitem> + </itemizedlist> + </para> +</section> + +<section id='licensing'> + <title>Licensing</title> + + <para> + Because open source projects are open to the public, they have different licensing structures in place. + License evolution for both Open Source and Free Software has an interesting history. + If you are interested in this history, you can find basic information here: + <itemizedlist> + <listitem><para><ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink> + </para></listitem> + <listitem><para><ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license + history</ulink></para></listitem> + </itemizedlist> + </para> + + <para> + In general, the Yocto Project is broadly licensed under the Massachusetts Institute of Technology + (MIT) License. + MIT licensing permits the reuse of software within proprietary software as long as the + license is distributed with that software. + MIT is also compatible with the GNU General Public License (GPL). + Patches to the Yocto Project follow the upstream licensing scheme. + You can find information on the MIT license + <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>. + You can find information on the GNU GPL <ulink url='http://www.opensource.org/licenses/LGPL-3.0'> + here</ulink>. + </para> + + <para> + When you build an image using the Yocto Project, the build process uses a + known list of licenses to ensure compliance. + You can find this list in the + <link linkend='source-directory'>Source Directory</link> at + <filename>meta/files/common-licenses</filename>. + Once the build completes, the list of all licenses found and used during that build are + kept in the + <link linkend='build-directory'>Build Directory</link> at + <filename>tmp/deploy/licenses</filename>. + </para> + + <para> + If a module requires a license that is not in the base list, the build process + generates a warning during the build. + These tools make it easier for a developer to be certain of the licenses with which + their shipped products must comply. + However, even with these tools it is still up to the developer to resolve potential licensing issues. + </para> + + <para> + The base list of licenses used by the build process is a combination of the Software Package + Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects. + <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of the Linux Foundation + that maintains a specification + for a standard format for communicating the components, licenses, and copyrights + associated with a software package. + <ulink url='http://opensource.org'>OSI</ulink> is a corporation dedicated to the Open Source + Definition and the effort for reviewing and approving licenses that + conform to the Open Source Definition (OSD). + </para> + + <para> + You can find a list of the combined SPDX and OSI licenses that the + Yocto Project uses in the + <filename>meta/files/common-licenses</filename> directory in your + <link linkend='source-directory'>Source Directory</link>. + </para> + + <para> + For information that can help you maintain compliance with various + open source licensing during the lifecycle of a product created using + the Yocto Project, see the + "<link linkend='maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</link>" + section. + </para> +</section> + +<section id='git'> + <title>Git</title> + + <para> + The Yocto Project makes extensive use of Git, + which is a free, open source distributed version control system. + Git supports distributed development, non-linear development, and can handle large projects. + It is best that you have some fundamental understanding of how Git tracks projects and + how to work with Git if you are going to use the Yocto Project for development. + This section provides a quick overview of how Git works and provides you with a summary + of some essential Git commands. + </para> + + <para> + For more information on Git, see + <ulink url='http://git-scm.com/documentation'></ulink>. + If you need to download Git, go to <ulink url='http://git-scm.com/download'></ulink>. + </para> + + <section id='repositories-tags-and-branches'> + <title>Repositories, Tags, and Branches</title> + + <para> + As mentioned earlier in the section + "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>", + the Yocto Project maintains source repositories at + <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>. + If you look at this web-interface of the repositories, each item is a separate + Git repository. + </para> + + <para> + Git repositories use branching techniques that track content change (not files) + within a project (e.g. a new feature or updated documentation). + Creating a tree-like structure based on project divergence allows for excellent historical + information over the life of a project. + This methodology also allows for an environment from which you can do lots of + local experimentation on projects as you develop changes or new features. + </para> + + <para> + A Git repository represents all development efforts for a given project. + For example, the Git repository <filename>poky</filename> contains all changes + and developments for Poky over the course of its entire life. + That means that all changes that make up all releases are captured. + The repository maintains a complete history of changes. + </para> + + <para> + You can create a local copy of any repository by "cloning" it with the Git + <filename>clone</filename> command. + When you clone a Git repository, you end up with an identical copy of the + repository on your development system. + Once you have a local copy of a repository, you can take steps to develop locally. + For examples on how to clone Git repositories, see the + "<link linkend='getting-setup'>Getting Set Up</link>" section. + </para> + + <para> + It is important to understand that Git tracks content change and + not files. + Git uses "branches" to organize different development efforts. + For example, the <filename>poky</filename> repository has + several branches that include the current + <filename>&DISTRO_NAME_NO_CAP;</filename> branch, the + <filename>master</filename> branch, and many branches for past + Yocto Project releases. + You can see all the branches by going to + <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and + clicking on the + <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename> + link beneath the "Branch" heading. + </para> + + <para> + Each of these branches represents a specific area of development. + The <filename>master</filename> branch represents the current or most recent + development. + All other branches represent offshoots of the <filename>master</filename> + branch. + </para> + + <para> + When you create a local copy of a Git repository, the copy has the same set + of branches as the original. + This means you can use Git to create a local working area (also called a branch) + that tracks a specific development branch from the source Git repository. + in other words, you can define your local Git environment to work on any development + branch in the repository. + To help illustrate, here is a set of commands that creates a local copy of the + <filename>poky</filename> Git repository and then creates and checks out a local + Git branch that tracks the Yocto Project &DISTRO; Release (&DISTRO_NAME;) development: + <literallayout class='monospaced'> + $ cd ~ + $ git clone git://git.yoctoproject.org/poky + $ cd poky + $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP; + </literallayout> + In this example, the name of the top-level directory of your local + <link linkend='source-directory'>Source Directory</link> + is "poky" and the name of that local working area (local branch) + you just created and checked out is "&DISTRO_NAME_NO_CAP;". + The files in your local repository now reflect the same files that + are in the "&DISTRO_NAME_NO_CAP;" development branch of the + Yocto Project's "poky" upstream repository. + It is important to understand that when you create and checkout a + local working branch based on a branch name, + your local environment matches the "tip" of that development branch + at the time you created your local branch, which could be + different from the files at the time of a similarly named release. + In other words, creating and checking out a local branch based on + the "&DISTRO_NAME_NO_CAP;" branch name is not the same as + cloning and checking out the "master" branch. + Keep reading to see how you create a local snapshot of a Yocto + Project Release. + </para> + + <para> + Git uses "tags" to mark specific changes in a repository. + Typically, a tag is used to mark a special point such as the final + change before a project is released. + You can see the tags used with the <filename>poky</filename> Git + repository by going to + <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and + clicking on the + <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename> + link beneath the "Tag" heading. + </para> + + <para> + Some key tags are + <filename>dizzy-12.0.0</filename>, + <filename>fido-13.0.0</filename>, + <filename>jethro-14.0.0</filename>, and + <filename>&DISTRO_NAME_NO_CAP;-&POKYVERSION;</filename>. + These tags represent Yocto Project releases. + </para> + + <para> + When you create a local copy of the Git repository, you also have access to all the + tags. + Similar to branches, you can create and checkout a local working Git branch based + on a tag name. + When you do this, you get a snapshot of the Git repository that reflects + the state of the files when the change was made associated with that tag. + The most common use is to checkout a working branch that matches a specific + Yocto Project release. + Here is an example: + <literallayout class='monospaced'> + $ cd ~ + $ git clone git://git.yoctoproject.org/poky + $ cd poky + $ git checkout -b my-&DISTRO_NAME_NO_CAP;-&POKYVERSION; &DISTRO_NAME_NO_CAP;-&POKYVERSION; + </literallayout> + In this example, the name of the top-level directory of your local Yocto Project + Files Git repository is <filename>poky</filename>. + And, the name of the local branch you have created and checked out is + <filename>my-&DISTRO_NAME_NO_CAP;-&POKYVERSION;</filename>. + The files in your repository now exactly match the Yocto Project &DISTRO; + Release tag (<filename>&DISTRO_NAME_NO_CAP;-&POKYVERSION;</filename>). + It is important to understand that when you create and checkout a local + working branch based on a tag, your environment matches a specific point + in time and not the entire development branch. + </para> + </section> + + <section id='basic-commands'> + <title>Basic Commands</title> + + <para> + Git has an extensive set of commands that lets you manage changes and perform + collaboration over the life of a project. + Conveniently though, you can manage with a small set of basic operations and workflows + once you understand the basic philosophy behind Git. + You do not have to be an expert in Git to be functional. + A good place to look for instruction on a minimal set of Git commands is + <ulink url='http://git-scm.com/documentation'>here</ulink>. + If you need to download Git, you can do so + <ulink url='http://git-scm.com/download'>here</ulink>, although + any reasonably current Linux distribution should already have an + installable package for Git. + </para> + + <para> + If you do not know much about Git, you should educate + yourself by visiting the links previously mentioned. + </para> + + <para> + The following list briefly describes some basic Git operations as a way to get started. + As with any set of commands, this list (in most cases) simply shows the base command and + omits the many arguments they support. + See the Git documentation for complete descriptions and strategies on how to use these commands: + <itemizedlist> + <listitem><para><emphasis><filename>git init</filename>:</emphasis> Initializes an empty Git repository. + You cannot use Git commands unless you have a <filename>.git</filename> repository.</para></listitem> + <listitem><para><emphasis><filename>git clone</filename>:</emphasis> + Creates a local clone of a Git repository. + During collaboration, this command allows you to create a + local Git repository that is on equal footing with a fellow + developer’s Git repository. + </para></listitem> + <listitem><para><emphasis><filename>git add</filename>:</emphasis> Stages updated file contents + to the index that + Git uses to track changes. + You must stage all files that have changed before you can commit them.</para></listitem> + <listitem><para><emphasis><filename>git commit</filename>:</emphasis> Creates a "commit" that documents + the changes you made. + Commits are used for historical purposes, for determining if a maintainer of a project + will allow the change, and for ultimately pushing the change from your local Git repository + into the project’s upstream (or master) repository.</para></listitem> + <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that + possibly need to be staged and committed.</para></listitem> + <listitem><para><emphasis><filename>git checkout</filename> <replaceable>branch-name</replaceable>:</emphasis> Changes + your working branch. + This command is analogous to "cd".</para></listitem> + <listitem><para><emphasis><filename>git checkout –b</filename> <replaceable>working-branch</replaceable>:</emphasis> Creates + a working branch on your local machine where you can isolate work. + It is a good idea to use local branches when adding specific features or changes. + This way if you do not like what you have done you can easily get rid of the work.</para></listitem> + <listitem><para><emphasis><filename>git branch</filename>:</emphasis> Reports + existing local branches and + tells you the branch in which you are currently working.</para></listitem> + <listitem><para><emphasis><filename>git branch -D</filename> <replaceable>branch-name</replaceable>:</emphasis> + Deletes an existing local branch. + You need to be in a local branch other than the one you are deleting + in order to delete <replaceable>branch-name</replaceable>.</para></listitem> + <listitem><para><emphasis><filename>git pull</filename>:</emphasis> Retrieves information + from an upstream Git + repository and places it in your local Git repository. + You use this command to make sure you are synchronized with the repository + from which you are basing changes (.e.g. the master branch).</para></listitem> + <listitem><para><emphasis><filename>git push</filename>:</emphasis> + Sends all your committed local changes to an upstream Git + repository (e.g. a contribution repository). + The maintainer of the project draws from these repositories + when adding changes to the project’s master repository or + other development branch. + </para></listitem> + <listitem><para><emphasis><filename>git merge</filename>:</emphasis> Combines or adds changes from one + local branch of your repository with another branch. + When you create a local Git repository, the default branch is named "master". + A typical workflow is to create a temporary branch for isolated work, make and commit your + changes, switch to your local master branch, merge the changes from the temporary branch into the + local master branch, and then delete the temporary branch.</para></listitem> + <listitem><para><emphasis><filename>git cherry-pick</filename>:</emphasis> Choose and apply specific + commits from one branch into another branch. + There are times when you might not be able to merge all the changes in one branch with + another but need to pick out certain ones.</para></listitem> + <listitem><para><emphasis><filename>gitk</filename>:</emphasis> Provides a GUI view of the branches + and changes in your local Git repository. + This command is a good way to graphically see where things have diverged in your + local repository.</para></listitem> + <listitem><para><emphasis><filename>git log</filename>:</emphasis> Reports a history of your changes to the + repository.</para></listitem> + <listitem><para><emphasis><filename>git diff</filename>:</emphasis> Displays line-by-line differences + between your local working files and the same files in the upstream Git repository that your + branch currently tracks.</para></listitem> + </itemizedlist> + </para> + </section> +</section> + +<section id='workflows'> + <title>Workflows</title> + + <para> + This section provides some overview on workflows using Git. + In particular, the information covers basic practices that describe roles and actions in a + collaborative development environment. + Again, if you are familiar with this type of development environment, you might want to just + skip this section. + </para> + + <para> + The Yocto Project files are maintained using Git in a "master" branch whose Git history + tracks every change and whose structure provides branches for all diverging functionality. + Although there is no need to use Git, many open source projects do so. + For the Yocto Project, a key individual called the "maintainer" is responsible for the "master" + branch of a given Git repository. + The "master" branch is the “upstream” repository where the final builds of the project occur. + The maintainer is responsible for accepting changes from other developers and for + organizing the underlying branch structure to reflect release strategies and so forth. + <note>For information on finding out who is responsible for (maintains) + a particular area of code, see the + "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + section. + </note> + </para> + + <para> + The project also has an upstream contribution Git repository named + <filename>poky-contrib</filename>. + You can see all the branches in this repository using the web interface + of the + <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink> organized + within the "Poky Support" area. + These branches temporarily hold changes to the project that have been + submitted or committed by the Yocto Project development team and by + community members who contribute to the project. + The maintainer determines if the changes are qualified to be moved + from the "contrib" branches into the "master" branch of the Git + repository. + </para> + + <para> + Developers (including contributing community members) create and maintain cloned repositories + of the upstream "master" branch. + These repositories are local to their development platforms and are used to develop changes. + When a developer is satisfied with a particular feature or change, they "push" the changes + to the appropriate "contrib" repository. + </para> + + <para> + Developers are responsible for keeping their local repository up-to-date with "master". + They are also responsible for straightening out any conflicts that might arise within files + that are being worked on simultaneously by more than one person. + All this work is done locally on the developer’s machines before anything is pushed to a + "contrib" area and examined at the maintainer’s level. + </para> + + <para> + A somewhat formal method exists by which developers commit changes and push them into the + "contrib" area and subsequently request that the maintainer include them into "master" + This process is called “submitting a patch” or "submitting a change." + For information on submitting patches and changes, see the + "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" section. + </para> + + <para> + To summarize the environment: a single point of entry exists for + changes into the project’s "master" branch of the Git repository, + which is controlled by the project’s maintainer. + And, a set of developers exist who independently develop, test, and + submit changes to "contrib" areas for the maintainer to examine. + The maintainer then chooses which changes are going to become a + permanent part of the project. + </para> + + <para> + <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" /> + </para> + + <para> + While each development environment is unique, there are some best practices or methods + that help development run smoothly. + The following list describes some of these practices. + For more information about Git workflows, see the workflow topics in the + <ulink url='http://book.git-scm.com'>Git Community Book</ulink>. + <itemizedlist> + <listitem><para><emphasis>Make Small Changes:</emphasis> It is best to keep the changes you commit + small as compared to bundling many disparate changes into a single commit. + This practice not only keeps things manageable but also allows the maintainer + to more easily include or refuse changes.</para> + <para>It is also good practice to leave the repository in a state that allows you to + still successfully build your project. In other words, do not commit half of a feature, + then add the other half as a separate, later commit. + Each commit should take you from one buildable project state to another + buildable state.</para></listitem> + <listitem><para><emphasis>Use Branches Liberally:</emphasis> It is very easy to create, use, and + delete local branches in your working Git repository. + You can name these branches anything you like. + It is helpful to give them names associated with the particular feature or change + on which you are working. + Once you are done with a feature or change and have merged it + into your local master branch, simply discard the temporary + branch.</para></listitem> + <listitem><para><emphasis>Merge Changes:</emphasis> The <filename>git merge</filename> + command allows you to take the + changes from one branch and fold them into another branch. + This process is especially helpful when more than a single developer might be working + on different parts of the same feature. + Merging changes also automatically identifies any collisions or "conflicts" + that might happen as a result of the same lines of code being altered by two different + developers.</para></listitem> + <listitem><para><emphasis>Manage Branches:</emphasis> Because branches are easy to use, you should + use a system where branches indicate varying levels of code readiness. + For example, you can have a "work" branch to develop in, a "test" branch where the code or + change is tested, a "stage" branch where changes are ready to be committed, and so forth. + As your project develops, you can merge code across the branches to reflect ever-increasing + stable states of the development.</para></listitem> + <listitem><para><emphasis>Use Push and Pull:</emphasis> The push-pull workflow is based on the + concept of developers "pushing" local commits to a remote repository, which is + usually a contribution repository. + This workflow is also based on developers "pulling" known states of the project down into their + local development repositories. + The workflow easily allows you to pull changes submitted by other developers from the + upstream repository into your work area ensuring that you have the most recent software + on which to develop. + The Yocto Project has two scripts named <filename>create-pull-request</filename> and + <filename>send-pull-request</filename> that ship with the release to facilitate this + workflow. + You can find these scripts in the <filename>scripts</filename> + folder of the + <link linkend='source-directory'>Source Directory</link>. + For information on how to use these scripts, see the + "<link linkend='pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</link>" section. + </para></listitem> + <listitem><para><emphasis>Patch Workflow:</emphasis> This workflow allows you to notify the + maintainer through an email that you have a change (or patch) you would like considered + for the "master" branch of the Git repository. + To send this type of change, you format the patch and then send the email using the Git commands + <filename>git format-patch</filename> and <filename>git send-email</filename>. + For information on how to use these scripts, see the + "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + section. + </para></listitem> + </itemizedlist> + </para> +</section> + +<section id='tracking-bugs'> + <title>Tracking Bugs</title> + + <para> + The Yocto Project uses its own implementation of + <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink> to track bugs. + Implementations of Bugzilla work well for group development because they track bugs and code + changes, can be used to communicate changes and problems with developers, can be used to + submit and review patches, and can be used to manage quality assurance. + The home page for the Yocto Project implementation of Bugzilla is + <ulink url='&YOCTO_BUGZILLA_URL;'>&YOCTO_BUGZILLA_URL;</ulink>. + </para> + + <para> + Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself + such as when discovering an issue with some component of the build system that acts contrary + to the documentation or your expectations. + Following is the general procedure for submitting a new bug using the Yocto Project + Bugzilla. + You can find more information on defect management, bug tracking, and feature request + processes all accomplished through the Yocto Project Bugzilla on the + <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>wiki page</ulink>. + <orderedlist> + <listitem><para>Always use the Yocto Project implementation of Bugzilla to submit + a bug.</para></listitem> + <listitem><para>When submitting a new bug, be sure to choose the appropriate + Classification, Product, and Component for which the issue was found. + Defects for the Yocto Project fall into one of seven classifications: + Yocto Project Components, Infrastructure, Build System & Metadata, + Documentation, QA/Testing, Runtime and Hardware. + Each of these Classifications break down into multiple Products and, in some + cases, multiple Components.</para></listitem> + <listitem><para>Use the bug form to choose the correct Hardware and Architecture + for which the bug applies.</para></listitem> + <listitem><para>Indicate the Yocto Project version you were using when the issue + occurred.</para></listitem> + <listitem><para>Be sure to indicate the Severity of the bug. + Severity communicates how the bug impacted your work.</para></listitem> + <listitem><para>Select the appropriate "Documentation change" item + for the bug. + Fixing a bug may or may not affect the Yocto Project + documentation.</para></listitem> + <listitem><para>Provide a brief summary of the issue. + Try to limit your summary to just a line or two and be sure to capture the + essence of the issue.</para></listitem> + <listitem><para>Provide a detailed description of the issue. + You should provide as much detail as you can about the context, behavior, output, + and so forth that surrounds the issue. + You can even attach supporting files for output from logs by + using the "Add an attachment" button.</para></listitem> + <listitem><para>Be sure to copy the appropriate people in the + "CC List" for the bug. + See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + section for information about finding out who is responsible + for code.</para></listitem> + <listitem><para>Submit the bug by clicking the "Submit Bug" button.</para></listitem> + </orderedlist> + </para> +</section> + +<section id='how-to-submit-a-change'> + <title>How to Submit a Change</title> + + <para> + Contributions to the Yocto Project and OpenEmbedded are very welcome. + Because the system is extremely configurable and flexible, we recognize that developers + will want to extend, configure or optimize it for their specific uses. + You should send patches to the appropriate mailing list so that they + can be reviewed and merged by the appropriate maintainer. + </para> + + <para> + Before submitting any change, be sure to find out who you should be + notifying. + Several methods exist through which you find out who you should be copying + or notifying: + <itemizedlist> + <listitem><para><emphasis>Maintenance File:</emphasis> + Examine the <filename>maintainers.inc</filename> file, which is + located in the + <link linkend='source-directory'>Source Directory</link> + at <filename>meta-poky/conf/distro/include</filename>, to + see who is responsible for code. + </para></listitem> + <listitem><para><emphasis>Board Support Package (BSP) README Files:</emphasis> + For BSP maintainers of supported BSPs, you can examine + individual BSP <filename>README</filename> files. + In addition, some layers (such as the <filename>meta-intel</filename> layer), + include a <filename>MAINTAINERS</filename> file which contains + a list of all supported BSP maintainers for that layer. + </para></listitem> + <listitem><para><emphasis>Search by File:</emphasis> + Using <link linkend='git'>Git</link>, you can enter the + following command to bring up a short list of all commits + against a specific file: + <literallayout class='monospaced'> + git shortlog -- <replaceable>filename</replaceable> + </literallayout> + Just provide the name of the file for which you are interested. + The information returned is not ordered by history but does + include a list of all committers grouped by name. + From the list, you can see who is responsible for the bulk of + the changes against the file. + </para></listitem> + </itemizedlist> + </para> + + <para> + For a list of the Yocto Project and related mailing lists, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>" section in + the Yocto Project Reference Manual. + </para> + + <para> + Here is some guidance on which mailing list to use for what type of change: + <itemizedlist> + <listitem><para>For changes to the core + <link linkend='metadata'>Metadata</link>, send your patch to the + <ulink url='&OE_LISTS_URL;/listinfo/openembedded-core'>openembedded-core</ulink> mailing list. + For example, a change to anything under the <filename>meta</filename> or + <filename>scripts</filename> directories + should be sent to this mailing list.</para></listitem> + <listitem><para>For changes to BitBake (anything under the <filename>bitbake</filename> + directory), send your patch to the + <ulink url='&OE_LISTS_URL;/listinfo/bitbake-devel'>bitbake-devel</ulink> mailing list.</para></listitem> + <listitem><para>For changes to <filename>meta-poky</filename>, send your patch to the + <ulink url='&YOCTO_LISTS_URL;/listinfo/poky'>poky</ulink> mailing list.</para></listitem> + <listitem><para>For changes to other layers hosted on + <filename>yoctoproject.org</filename> (unless the + layer's documentation specifies otherwise), tools, and Yocto Project + documentation, use the + <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> mailing list.</para></listitem> + <listitem><para>For additional recipes that do not fit into the core Metadata, + you should determine which layer the recipe should go into and submit the + change in the manner recommended by the documentation (e.g. README) supplied + with the layer. If in doubt, please ask on the + <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> or + <ulink url='&OE_LISTS_URL;/listinfo/openembedded-devel'>openembedded-devel</ulink> + mailing lists.</para></listitem> + </itemizedlist> + </para> + + <para> + When you send a patch, be sure to include a "Signed-off-by:" + line in the same style as required by the Linux kernel. + Adding this line signifies that you, the submitter, have agreed to the Developer's Certificate of Origin 1.1 + as follows: + <literallayout class='monospaced'> + Developer's Certificate of Origin 1.1 + + By making a contribution to this project, I certify that: + + (a) The contribution was created in whole or in part by me and I + have the right to submit it under the open source license + indicated in the file; or + + (b) The contribution is based upon previous work that, to the best + of my knowledge, is covered under an appropriate open source + license and I have the right under that license to submit that + work with modifications, whether created in whole or in part + by me, under the same open source license (unless I am + permitted to submit under a different license), as indicated + in the file; or + + (c) The contribution was provided directly to me by some other + person who certified (a), (b) or (c) and I have not modified + it. + + (d) I understand and agree that this project and the contribution + are public and that a record of the contribution (including all + personal information I submit with it, including my sign-off) is + maintained indefinitely and may be redistributed consistent with + this project or the open source license(s) involved. + </literallayout> + </para> + + <para> + In a collaborative environment, it is necessary to have some sort of standard + or method through which you submit changes. + Otherwise, things could get quite chaotic. + One general practice to follow is to make small, controlled changes. + Keeping changes small and isolated aids review, makes merging/rebasing easier + and keeps the change history clean when anyone needs to refer to it in future. + </para> + + <para> + When you make a commit, you must follow certain standards established by the + OpenEmbedded and Yocto Project development teams. + For each commit, you must provide a single-line summary of the change and you + should almost always provide a more detailed description of what you did (i.e. + the body of the commit message). + The only exceptions for not providing a detailed description would be if your + change is a simple, self-explanatory change that needs no further description + beyond the summary. + Here are the guidelines for composing a commit message: + <itemizedlist> + <listitem><para>Provide a single-line, short summary of the change. + This summary is typically viewable in the "shortlist" of changes. + Thus, providing something short and descriptive that gives the reader + a summary of the change is useful when viewing a list of many commits. + This short description should be prefixed by the recipe name (if changing a recipe), or + else the short form path to the file being changed. + </para></listitem> + <listitem><para>For the body of the commit message, provide detailed information + that describes what you changed, why you made the change, and the approach + you used. It may also be helpful if you mention how you tested the change. + Provide as much detail as you can in the body of the commit message. + </para></listitem> + <listitem><para> + If the change addresses a specific bug or issue that is + associated with a bug-tracking ID, include a reference to that + ID in your detailed description. + For example, the Yocto Project uses a specific convention for + bug references - any commit that addresses a specific bug should + use the following form for the detailed description: + <literallayout class='monospaced'> + Fixes [YOCTO #<replaceable>bug-id</replaceable>] + + <replaceable>detailed description of change</replaceable> + </literallayout></para></listitem> + Where <replaceable>bug-id</replaceable> is replaced with the + specific bug ID from the Yocto Project Bugzilla instance. + </itemizedlist> + </para> + + <para> + You can find more guidance on creating well-formed commit messages at this OpenEmbedded + wiki page: + <ulink url='&OE_HOME_URL;/wiki/Commit_Patch_Message_Guidelines'></ulink>. + </para> + + <para> + The next two sections describe general instructions for both pushing + changes upstream and for submitting changes as patches. + </para> + + <section id='pushing-a-change-upstream'> + <title>Using Scripts to Push a Change Upstream and Request a Pull</title> + + <para> + The basic flow for pushing a change to an upstream "contrib" Git repository is as follows: + <itemizedlist> + <listitem><para>Make your changes in your local Git repository.</para></listitem> + <listitem><para>Stage your changes by using the <filename>git add</filename> + command on each file you changed.</para></listitem> + <listitem><para> + Commit the change by using the + <filename>git commit</filename> command. + Be sure to provide a commit message that follows the + project’s commit message standards as described earlier. + </para></listitem> + <listitem><para> + Push the change to the upstream "contrib" repository by + using the <filename>git push</filename> command. + </para></listitem> + <listitem><para>Notify the maintainer that you have pushed a change by making a pull + request. + The Yocto Project provides two scripts that conveniently let you generate and send + pull requests to the Yocto Project. + These scripts are <filename>create-pull-request</filename> and + <filename>send-pull-request</filename>. + You can find these scripts in the <filename>scripts</filename> directory + within the <link linkend='source-directory'>Source Directory</link>.</para> + <para>Using these scripts correctly formats the requests without introducing any + whitespace or HTML formatting. + The maintainer that receives your patches needs to be able to save and apply them + directly from your emails. + Using these scripts is the preferred method for sending patches.</para> + <para>For help on using these scripts, simply provide the + <filename>-h</filename> argument as follows: + <literallayout class='monospaced'> + $ poky/scripts/create-pull-request -h + $ poky/scripts/send-pull-request -h + </literallayout></para></listitem> + </itemizedlist> + </para> + + <para> + You can find general Git information on how to push a change upstream in the + <ulink url='http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows'>Git Community Book</ulink>. + </para> + </section> + + <section id='submitting-a-patch'> + <title>Using Email to Submit a Patch</title> + + <para> + You can submit patches without using the <filename>create-pull-request</filename> and + <filename>send-pull-request</filename> scripts described in the previous section. + However, keep in mind, the preferred method is to use the scripts. + </para> + + <para> + Depending on the components changed, you need to submit the email to a specific + mailing list. + For some guidance on which mailing list to use, see the list in the + "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + section. + For a description of the available mailing lists, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>" + section in the Yocto Project Reference Manual. + </para> + + <para> + Here is the general procedure on how to submit a patch through email without using the + scripts: + <itemizedlist> + <listitem><para>Make your changes in your local Git repository.</para></listitem> + <listitem><para>Stage your changes by using the <filename>git add</filename> + command on each file you changed.</para></listitem> + <listitem><para>Commit the change by using the + <filename>git commit --signoff</filename> command. + Using the <filename>--signoff</filename> option identifies you as the person + making the change and also satisfies the Developer's Certificate of + Origin (DCO) shown earlier.</para> + <para>When you form a commit, you must follow certain standards established by the + Yocto Project development team. + See the earlier section + "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" + for Yocto Project commit message standards.</para></listitem> + <listitem><para>Format the commit into an email message. + To format commits, use the <filename>git format-patch</filename> command. + When you provide the command, you must include a revision list or a number of patches + as part of the command. + For example, either of these two commands takes your most + recent single commit and formats it as an email message in + the current directory: + <literallayout class='monospaced'> + $ git format-patch -1 + </literallayout> + or + <literallayout class='monospaced'> + $ git format-patch HEAD~ + </literallayout></para> + <para>After the command is run, the current directory contains a + numbered <filename>.patch</filename> file for the commit.</para> + <para>If you provide several commits as part of the command, + the <filename>git format-patch</filename> command produces a + series of numbered files in the current directory – one for each commit. + If you have more than one patch, you should also use the + <filename>--cover</filename> option with the command, which generates a + cover letter as the first "patch" in the series. + You can then edit the cover letter to provide a description for + the series of patches. + For information on the <filename>git format-patch</filename> command, + see <filename>GIT_FORMAT_PATCH(1)</filename> displayed using the + <filename>man git-format-patch</filename> command.</para> + <note>If you are or will be a frequent contributor to the Yocto Project + or to OpenEmbedded, you might consider requesting a contrib area and the + necessary associated rights.</note></listitem> + <listitem><para>Import the files into your mail client by using the + <filename>git send-email</filename> command. + <note>In order to use <filename>git send-email</filename>, you must have the + the proper Git packages installed. + For Ubuntu, Debian, and Fedora the package is <filename>git-email</filename>.</note></para> + <para>The <filename>git send-email</filename> command sends email by using a local + or remote Mail Transport Agent (MTA) such as + <filename>msmtp</filename>, <filename>sendmail</filename>, or through a direct + <filename>smtp</filename> configuration in your Git <filename>config</filename> + file. + If you are submitting patches through email only, it is very important + that you submit them without any whitespace or HTML formatting that + either you or your mailer introduces. + The maintainer that receives your patches needs to be able to save and + apply them directly from your emails. + A good way to verify that what you are sending will be applicable by the + maintainer is to do a dry run and send them to yourself and then + save and apply them as the maintainer would.</para> + <para>The <filename>git send-email</filename> command is the preferred method + for sending your patches since there is no risk of compromising whitespace + in the body of the message, which can occur when you use your own mail client. + The command also has several options that let you + specify recipients and perform further editing of the email message. + For information on how to use the <filename>git send-email</filename> command, + see <filename>GIT-SEND-EMAIL(1)</filename> displayed using + the <filename>man git-send-email</filename> command. + </para></listitem> + </itemizedlist> + </para> + </section> +</section> +</chapter> +<!-- +vim: expandtab tw=80 ts=4 +--> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-qemu.xml b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-qemu.xml new file mode 100644 index 000000000..41c18298a --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-qemu.xml @@ -0,0 +1,433 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<chapter id='dev-manual-qemu'> + +<title>Using the Quick EMUlator (QEMU)</title> + +<para> + Quick EMUlator (QEMU) is an Open Source project the Yocto Project uses + as part of its development "tool set". + As such, the information in this chapter is limited to the + Yocto Project integration of QEMU and not QEMU in general. + For official information and documentation on QEMU, see the + following references: + <itemizedlist> + <listitem><para><emphasis><ulink url='http://wiki.qemu.org/Main_Page'>QEMU Website</ulink>:</emphasis> + The official website for the QEMU Open Source project. + </para></listitem> + <listitem><para><emphasis><ulink url='http://wiki.qemu.org/Manual'>Documentation</ulink>:</emphasis> + The QEMU user manual. + </para></listitem> + </itemizedlist> +</para> + +<para> + This chapter provides an overview of the Yocto Project's integration of + QEMU, a description of how you use QEMU and its various options, running + under a Network File System (NFS) server, and a few tips and tricks you + might find helpful when using QEMU. +</para> + +<section id='qemu-overview'> + <title>Overview</title> + + <para> + Within the context of the Yocto Project, QEMU is an + emulator and virtualization machine that allows you to run a complete + image you have built using the Yocto Project as just another task + on your build system. + QEMU is useful for running and testing images and applications on + supported Yocto Project architectures without having actual hardware. + Among other things, the Yocto Project uses QEMU to run automated + Quality Assurance (QA) tests on final images shipped with each + release. + </para> + + <para> + QEMU is made available with the Yocto Project a number of ways. + One method is to install a Software Development Kit (SDK). + For more information on how to make sure you have + QEMU available, see the + <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-intro'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. + </para> +</section> + +<section id='qemu-running-qemu'> + <title>Running QEMU</title> + + <para> + Running QEMU involves having your build environment set up, having the + right artifacts available, and understanding how to use the many + options that are available to you when you start QEMU using the + <filename>runqemu</filename> command. + </para> + + <section id='qemu-setting-up-the-environment'> + <title>Setting Up the Environment</title> + + <para> + You run QEMU in the same environment from which you run BitBake. + This means you need to source a build environment script (i.e. + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>). + </para> + </section> + + <section id='qemu-using-the-runqemu-command'> + <title>Using the <filename>runqemu</filename> Command</title> + + <para> + The basic <filename>runqemu</filename> command syntax is as + follows: + <literallayout class='monospaced'> + $ runqemu [<replaceable>option</replaceable> ] [...] + </literallayout> + Based on what you provide on the command line, + <filename>runqemu</filename> does a good job of figuring out what + you are trying to do. + For example, by default, QEMU looks for the most recently built + image according to the timestamp when it needs to look for an + image. + Minimally, through the use of options, you must provide either + a machine name, a virtual machine image + (<filename>*.vmdk</filename>), or a kernel image + (<filename>*.bin</filename>). + </para> + + <para> + Following is a description of <filename>runqemu</filename> + options you can provide on the command line: + <note><title>Tip</title> + If you do provide some "illegal" option combination or perhaps + you do not provide enough in the way of options, + <filename>runqemu</filename> provides appropriate error + messaging to help you correct the problem. + </note> + <itemizedlist> + <listitem><para><replaceable>QEMUARCH</replaceable>: + The QEMU machine architecture, which must be "qemux86", + "qemux86_64", "qemuarm", "qemumips", "qemumipsel", + “qemumips64", "qemush4", "qemuppc", "qemumicroblaze", + or "qemuzynq". + </para></listitem> + <listitem><para><filename><replaceable>VM</replaceable></filename>: + The virtual machine image, which must be a + <filename>.vmdk</filename> file. + Use this option when you want to boot a + <filename>.vmdk</filename> image. + The image filename you provide must contain one of the + following strings: "qemux86-64", "qemux86", "qemuarm", + "qemumips64", "qemumips", "qemuppc", or "qemush4". + </para></listitem> + <listitem><para><replaceable>ROOTFS</replaceable>: + A root filesystem that has one of the following + filetype extensions: "ext2", "ext3", "ext4", "jffs2", + "nfs", or "btrfs". + If the filename you provide for this option uses “nfs”, it + must provide an explicit root filesystem path. + </para></listitem> + <listitem><para><replaceable>KERNEL</replaceable>: + A kernel image, which is a <filename>.bin</filename> file. + When you provide a <filename>.bin</filename> file, + <filename>runqemu</filename> detects it and assumes the + file is a kernel image. + </para></listitem> + <listitem><para><replaceable>MACHINE</replaceable>: + The architecture of the QEMU machine, which must be one + of the following: "qemux86", + "qemux86-64", "qemuarm", "qemumips", "qemumipsel", + “qemumips64", "qemush4", "qemuppc", "qemumicroblaze", + or "qemuzynq". + The <replaceable>MACHINE</replaceable> and + <replaceable>QEMUARCH</replaceable> options are basically + identical. + If you do not provide a <replaceable>MACHINE</replaceable> + option, <filename>runqemu</filename> tries to determine + it based on other options. + </para></listitem> + <listitem><para><filename>ramfs</filename>: + Indicates you are booting an initial RAM disk (initramfs) + image, which means the <filename>FSTYPE</filename> is + <filename>cpio.gz</filename>. + </para></listitem> + <listitem><para><filename>iso</filename>: + Indicates you are booting an ISO image, which means the + <filename>FSTYPE</filename> is + <filename>.iso</filename>. + </para></listitem> + <listitem><para><filename>nographic</filename>: + Disables the video console, which sets the console to + "ttys0". + </para></listitem> + <listitem><para><filename>serial</filename>: + Enables a serial console on + <filename>/dev/ttyS0</filename>. + </para></listitem> + <listitem><para><filename>biosdir</filename>: + Establishes a custom directory for BIOS, VGA BIOS and + keymaps. + </para></listitem> + <listitem><para><filename>biosfilename</filename>: + Establishes a custom BIOS name. + </para></listitem> + <listitem><para><filename>qemuparams=\"<replaceable>xyz</replaceable>\"</filename>: + Specifies custom QEMU parameters. + Use this option to pass options other than the simple + "kvm" and "serial" options. + </para></listitem> + <listitem><para><filename>bootparams=\"<replaceable>xyz</replaceable>\"</filename>: + Specifies custom boot parameters for the kernel. + </para></listitem> + <listitem><para><filename>audio</filename>: + Enables audio in QEMU. + The <replaceable>MACHINE</replaceable> option must be + either "qemux86" or "qemux86-64" in order for audio to be + enabled. + Additionally, the <filename>snd_intel8x0</filename> + or <filename>snd_ens1370</filename> driver must be + installed in linux guest. + </para></listitem> + <listitem><para><filename>slirp</filename>: + Enables "slirp" networking, which is a different way + of networking that does not need root access + but also is not as easy to use or comprehensive + as the default. + </para></listitem> + <listitem><para id='kvm-cond'><filename>kvm</filename>: + Enables KVM when running "qemux86" or "qemux86-64" + QEMU architectures. + For KVM to work, all the following conditions must be met: + <itemizedlist> + <listitem><para> + Your <replaceable>MACHINE</replaceable> must be either +qemux86" or "qemux86-64". + </para></listitem> + <listitem><para> + Your build host has to have the KVM modules + installed, which are + <filename>/dev/kvm</filename>. + </para></listitem> + <listitem><para> + The build host <filename>/dev/kvm</filename> + directory has to be both writable and readable. + </para></listitem> + </itemizedlist> + </para></listitem> + <listitem><para><filename>kvm-vhost</filename>: + Enables KVM with VHOST support when running "qemux86" or "qemux86-64" + QEMU architectures. + For KVM with VHOST to work, the following conditions must + be met: + <itemizedlist> + <listitem><para> + <link linkend='kvm-cond'>kvm</link> option + conditions must be met. + </para></listitem> + <listitem><para> + Your build host has to have virtio net device, which + are <filename>/dev/vhost-net</filename>. + </para></listitem> + <listitem><para> + The build host <filename>/dev/vhost-net</filename> + directory has to be either readable or writable + and “slirp-enabled”. + </para></listitem> + </itemizedlist> + </para></listitem> + <listitem><para><filename>publicvnc</filename>: + Enables a VNC server open to all hosts. + </para></listitem> + </itemizedlist> + </para> + + <para> + For further understanding regarding option use with + <filename>runqemu</filename>, consider some examples. + </para> + + <para> + This example starts QEMU with + <replaceable>MACHINE</replaceable> set to "qemux86". + Assuming a standard + <link linkend='build-directory'>Build Directory</link>, + <filename>runqemu</filename> automatically finds the + <filename>bzImage-qemux86.bin</filename> image file and + the + <filename>core-image-minimal-qemux86-20140707074611.rootfs.ext3</filename> + (assuming the current build created a + <filename>core-image-minimal</filename> image). + <note> + When more than one image with the same name exists, QEMU finds + and uses the most recently built image according to the + timestamp. + </note> + <literallayout class='monospaced'> + $ runqemu qemux86 + </literallayout> + This example produces the exact same results as the + previous example. + This command, however, specifically provides the image + and root filesystem type. + <literallayout class='monospaced'> + $ runqemu qemux86 core-image-minimal ext3 + </literallayout> + This example specifies to boot an initial RAM disk image + and to enable audio in QEMU. + For this case, <filename>runqemu</filename> set the + internal variable <filename>FSTYPE</filename> to + "cpio.gz". + Also, for audio to be enabled, an appropriate driver must + be installed (see the previous description for the + <filename>audio</filename> option for more information). + <literallayout class='monospaced'> + $ runqemu qemux86 ramfs audio + </literallayout> + This example does not provide enough information for + QEMU to launch. + While the command does provide a root filesystem type, it + must also minimally provide a + <replaceable>MACHINE</replaceable>, + <replaceable>KERNEL</replaceable>, or + <replaceable>VM</replaceable> option. + <literallayout class='monospaced'> + $ runqemu ext3 + </literallayout> + This example specifies to boot a virtual machine image + (<filename>.vmdk</filename> file). + From the <filename>.vmdk</filename>, + <filename>runqemu</filename> determines the QEMU + architecture (<replaceable>MACHINE</replaceable>) to be + "qemux86" and the root filesystem type to be "vmdk". + <literallayout class='monospaced'> + $ runqemu /home/scott-lenovo/vm/core-image-minimal-qemux86.vmdk + </literallayout> + </para> + </section> +</section> + +<section id='qemu-running-under-a-network-file-system-nfs-server'> + <title>Running Under a Network File System (NFS) Server</title> + + <para> + One method for running QEMU is to run it on an NFS server. + This is useful when you need to access the same file system from both + the build and the emulated system at the same time. + It is also worth noting that the system does not need root privileges + to run. + It uses a user space NFS server to avoid that. + This section describes how to set up for running QEMU using an NFS + server and then how you can start and stop the server. + </para> + + <section id='qemu-setting-up-to-use-nfs'> + <title>Setting Up to Use NFS</title> + + <para> + Once you are able to run QEMU in your environment, you can use the + <filename>runqemu-extract-sdk</filename> script, which is located + in the <filename>scripts</filename> directory along with + <filename>runqemu</filename> script. + The <filename>runqemu-extract-sdk</filename> takes a root + file system tarball and extracts it into a location that you + specify. + Then, when you run <filename>runqemu</filename>, you can specify + the location that has the file system to pass it to QEMU. + Here is an example that takes a file system and extracts it to + a directory named <filename>test-nfs</filename>: + <literallayout class='monospaced'> + runqemu-extract-sdk ./tmp/deploy/images/qemux86/core-image-sato-qemux86.tar.bz2 test-nfs + </literallayout> + Once you have extracted the file system, you can run + <filename>runqemu</filename> normally with the additional + location of the file system. + You can then also make changes to the files within + <filename>./test-nfs</filename> and see those changes appear in the + image in real time. + Here is an example using the <filename>qemux86</filename> image: + <literallayout class='monospaced'> + runqemu qemux86 ./test-nfs + </literallayout> + </para> + </section> + + <section id='qemu-starting-and-stopping-nfs'> + <title>Starting and Stopping NFS</title> + + <para> + You can manually start and stop the NFS share using these + commands: + <itemizedlist> + <listitem><para><emphasis><filename>start</filename>:</emphasis> + Starts the NFS share: + <literallayout class='monospaced'> + runqemu-export-rootfs start <replaceable>file-system-location</replaceable> + </literallayout> + </para></listitem> + <listitem><para><emphasis><filename>stop</filename>:</emphasis> + Stops the NFS share: + <literallayout class='monospaced'> + runqemu-export-rootfs stop <replaceable>file-system-location</replaceable> + </literallayout> + </para></listitem> + <listitem><para><emphasis><filename>restart</filename>:</emphasis> + Restarts the NFS share: + <literallayout class='monospaced'> + runqemu-export-rootfs restart <replaceable>file-system-location</replaceable> + </literallayout> + </para></listitem> + </itemizedlist> + </para> + </section> +</section> + +<section id='qemu-tips-and-tricks'> + <title>Tips and Tricks</title> + + <para> + The following list describes things you can do to make running QEMU + in the context of the Yocto Project a better experience: + <itemizedlist> + <listitem><para><emphasis>Switching Between Consoles:</emphasis> + When booting or running QEMU, you can switch between + supported consoles by using + Ctrl+Alt+<replaceable>number</replaceable>. + For example, Ctrl+Alt+3 switches you to the serial console as + long as that console is enabled. + Being able to switch consoles is helpful, for example, if the + main QEMU console breaks for some reason. + <note> + Usually, "2" gets you to the main console and "3" gets you + to the serial console. + </note> + </para></listitem> + <listitem><para><emphasis>Removing the Splash Screen:</emphasis> + You can remove the splash screen when QEMU is booting by + using Alt+left. + Removing the splash screen allows you to see what is happening + in the background. + </para></listitem> + <listitem><para><emphasis>Disabling the Cursor Grab:</emphasis> + The default QEMU integration captures the cursor within the + main window. + It does this since standard mouse devices only provide relative + input and not absolute coordinates. + You then have to break out of the grab using the "Ctrl+Alt" key + combination. + However, the Yocto Project's integration of QEMU enables the + wacom USB touch pad driver by default to allow input of absolute + coordinates. + This default means that the mouse can enter and leave the + main window without the grab taking effect leading to a better + user experience. + </para></listitem> + </itemizedlist> + </para> +</section> + +</chapter> +<!-- +vim: expandtab tw=80 ts=4 +--> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual-start.xml b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-start.xml new file mode 100644 index 000000000..23bf8eb0e --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual-start.xml @@ -0,0 +1,435 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<chapter id='dev-manual-start'> + +<title>Getting Started with the Yocto Project</title> + +<para> + This chapter introduces the Yocto Project and gives you an idea of what you need to get started. + You can find enough information to set up your development host and build or use images for + hardware supported by the Yocto Project by reading the + <ulink url='&YOCTO_DOCS_QS_URL;'>Yocto Project Quick Start</ulink>. +</para> + +<para> + The remainder of this chapter summarizes what is in the Yocto Project Quick Start and provides + some higher-level concepts you might want to consider. +</para> + +<section id='introducing-the-yocto-project'> + <title>Introducing the Yocto Project</title> + + <para> + The Yocto Project is an open-source collaboration project focused on embedded Linux development. + The project currently provides a build system that is + referred to as the + <link linkend='build-system-term'>OpenEmbedded build system</link> + in the Yocto Project documentation. + The Yocto Project provides various ancillary tools for the embedded developer + and also features the Sato reference User Interface, which is optimized for + stylus-driven, low-resolution screens. + </para> + + <para> + You can use the OpenEmbedded build system, which uses + <link linkend='bitbake-term'>BitBake</link>, to develop complete Linux + images and associated user-space applications for architectures based + on ARM, MIPS, PowerPC, x86 and x86-64. + <note> + By default, using the Yocto Project creates a Poky distribution. + However, you can create your own distribution by providing key + <link linkend='metadata'>Metadata</link>. + See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>" + section for more information. + </note> + While the Yocto Project does not provide a strict testing framework, + it does provide or generate for you artifacts that let you perform target-level and + emulated testing and debugging. + Additionally, if you are an <trademark class='trade'>Eclipse</trademark> + IDE user, you can install an Eclipse Yocto Plug-in to allow you to + develop within that familiar environment. + </para> +</section> + +<section id='getting-setup'> + <title>Getting Set Up</title> + + <para> + Here is what you need to use the Yocto Project: + <itemizedlist> + <listitem><para><emphasis>Host System:</emphasis> You should have a reasonably current + Linux-based host system. + You will have the best results with a recent release of Fedora, + openSUSE, Debian, Ubuntu, or CentOS as these releases are frequently tested against the Yocto Project + and officially supported. + For a list of the distributions under validation and their status, see the + "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>" section + in the Yocto Project Reference Manual and the wiki page at + <ulink url='&YOCTO_WIKI_URL;/wiki/Distribution_Support'>Distribution Support</ulink>.</para> + <para> + You should also have about 50 Gbytes of free disk space for building images. + </para></listitem> + <listitem><para><emphasis>Packages:</emphasis> The OpenEmbedded build system + requires that certain packages exist on your development system (e.g. Python 2.7). + See "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Build Host Packages</ulink>" + section in the Yocto Project Quick Start and the + "<ulink url='&YOCTO_DOCS_REF_URL;#required-packages-for-the-host-development-system'>Required Packages for the Host Development System</ulink>" + section in the Yocto Project Reference Manual for the exact + package requirements and the installation commands to install + them for the supported distributions. + </para></listitem> + <listitem id='local-yp-release'><para><emphasis>Yocto Project Release:</emphasis> + You need a release of the Yocto Project locally installed on + your development system. + The documentation refers to this set of locally installed files + as the <link linkend='source-directory'>Source Directory</link>. + You create your Source Directory by using + <link linkend='git'>Git</link> to clone a local copy + of the upstream <filename>poky</filename> repository, + or by downloading and unpacking a tarball of an official + Yocto Project release. + The preferred method is to create a clone of the repository. + </para> + <para>Working from a copy of the upstream repository allows you + to contribute back into the Yocto Project or simply work with + the latest software on a development branch. + Because Git maintains and creates an upstream repository with + a complete history of changes and you are working with a local + clone of that repository, you have access to all the Yocto + Project development branches and tag names used in the upstream + repository.</para> + <note>You can view the Yocto Project Source Repositories at + <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink> + </note> + <para>The following transcript shows how to clone the + <filename>poky</filename> Git repository into the current + working directory. + The command creates the local repository in a directory + named <filename>poky</filename>. + For information on Git used within the Yocto Project, see + the "<link linkend='git'>Git</link>" section. + <literallayout class='monospaced'> + $ git clone git://git.yoctoproject.org/poky + Cloning into 'poky'... + remote: Counting objects: 226790, done. + remote: Compressing objects: 100% (57465/57465), done. + remote: Total 226790 (delta 165212), reused 225887 (delta 164327) + Receiving objects: 100% (226790/226790), 100.98 MiB | 263 KiB/s, done. + Resolving deltas: 100% (165212/165212), done. + </literallayout></para> + <para>For another example of how to set up your own local Git + repositories, see this + <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'> + wiki page</ulink>, which describes how to create local + Git repositories for both + <filename>poky</filename> and <filename>meta-intel</filename>. + </para> + <para> + You can also get the Yocto Project Files by downloading + Yocto Project releases from the + <ulink url="&YOCTO_HOME_URL;">Yocto Project website</ulink>. + From the website, you just click "Downloads" in the navigation + pane to the left to display all Yocto Project downloads. + Current and archived releases are available for download. + Nightly and developmental builds are also maintained at + <ulink url="&YOCTO_AB_NIGHTLY_URL;"></ulink>. + One final site you can visit for information on Yocto Project + releases is the + <ulink url='&YOCTO_WIKI_URL;/wiki/Releases'>Releases</ulink> + wiki. + </para></listitem> + <listitem id='local-kernel-files'><para><emphasis>Yocto Project Kernel:</emphasis> + If you are going to be making modifications to a supported Yocto Project kernel, you + need to establish local copies of the source. + You can find Git repositories of supported Yocto Project kernels organized under + "Yocto Linux Kernel" in the Yocto Project Source Repositories at + <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.</para> + <para>This setup can involve creating a bare clone of the Yocto Project kernel and then + copying that cloned repository. + You can create the bare clone and the copy of the bare clone anywhere you like. + For simplicity, it is recommended that you create these structures outside of the + Source Directory, which is usually named <filename>poky</filename>.</para> + <para>As an example, the following transcript shows how to create the bare clone + of the <filename>linux-yocto-3.19</filename> kernel and then create a copy of + that clone. + <note>When you have a local Yocto Project kernel Git repository, you can + reference that repository rather than the upstream Git repository as + part of the <filename>clone</filename> command. + Doing so can speed up the process.</note></para> + <para>In the following example, the bare clone is named + <filename>linux-yocto-3.19.git</filename>, while the + copy is named <filename>my-linux-yocto-3.19-work</filename>: + <literallayout class='monospaced'> + $ git clone --bare git://git.yoctoproject.org/linux-yocto-3.19 linux-yocto-3.19.git + Cloning into bare repository 'linux-yocto-3.19.git'... + remote: Counting objects: 3983256, done. + remote: Compressing objects: 100% (605006/605006), done. + remote: Total 3983256 (delta 3352832), reused 3974503 (delta 3344079) + Receiving objects: 100% (3983256/3983256), 843.66 MiB | 1.07 MiB/s, done. + Resolving deltas: 100% (3352832/3352832), done. + Checking connectivity... done. + </literallayout></para> + <para>Now create a clone of the bare clone just created: + <literallayout class='monospaced'> + $ git clone linux-yocto-3.19.git my-linux-yocto-3.19-work + Cloning into 'my-linux-yocto-3.19-work'... + done. + Checking out files: 100% (48440/48440), done. + </literallayout></para></listitem> + <listitem id='meta-yocto-kernel-extras-repo'><para><emphasis> + The <filename>meta-yocto-kernel-extras</filename> Git Repository</emphasis>: + The <filename>meta-yocto-kernel-extras</filename> Git repository contains Metadata needed + only if you are modifying and building the kernel image. + In particular, it contains the kernel BitBake append (<filename>.bbappend</filename>) + files that you + edit to point to your locally modified kernel source files and to build the kernel + image. + Pointing to these local files is much more efficient than requiring a download of the + kernel's source files from upstream each time you make changes to the kernel.</para> + <para>You can find the <filename>meta-yocto-kernel-extras</filename> Git Repository in the + "Yocto Metadata Layers" area of the Yocto Project Source Repositories at + <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>. + It is good practice to create this Git repository inside the Source Directory.</para> + <para>Following is an example that creates the <filename>meta-yocto-kernel-extras</filename> Git + repository inside the Source Directory, which is named <filename>poky</filename> + in this case: + <literallayout class='monospaced'> + $ cd ~/poky + $ git clone git://git.yoctoproject.org/meta-yocto-kernel-extras meta-yocto-kernel-extras + Cloning into 'meta-yocto-kernel-extras'... + remote: Counting objects: 727, done. + remote: Compressing objects: 100% (452/452), done. + remote: Total 727 (delta 260), reused 719 (delta 252) + Receiving objects: 100% (727/727), 536.36 KiB | 240 KiB/s, done. + Resolving deltas: 100% (260/260), done. + </literallayout></para></listitem> + <listitem><para id='supported-board-support-packages-(bsps)'><emphasis>Supported Board Support Packages (BSPs):</emphasis> + The Yocto Project supports many BSPs, which are maintained in + their own layers or in layers designed to contain several + BSPs. + To get an idea of machine support through BSP layers, you can + look at the + <ulink url='&YOCTO_RELEASE_DL_URL;/machines'>index of machines</ulink> + for the release.</para> + + <para>The Yocto Project uses the following BSP layer naming + scheme: + <literallayout class='monospaced'> + meta-<replaceable>bsp_name</replaceable> + </literallayout> + where <replaceable>bsp_name</replaceable> is the recognized + BSP name. + Here is an example: + <literallayout class='monospaced'> + meta-raspberrypi + </literallayout> + See the + "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" + section in the Yocto Project Board Support Package (BSP) + Developer's Guide for more information on BSP Layers.</para> + + <para>A useful Git repository released with the Yocto + Project is <filename>meta-intel</filename>, which is a + parent layer that contains many supported + <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>. + You can locate the <filename>meta-intel</filename> Git + repository in the "Yocto Metadata Layers" area of the Yocto + Project Source Repositories at + <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.</para> + + <para>Using + <link linkend='git'>Git</link> to create a local clone of the + upstream repository can be helpful if you are working with + BSPs. + Typically, you set up the <filename>meta-intel</filename> + Git repository inside the Source Directory. + For example, the following transcript shows the steps to clone + <filename>meta-intel</filename>. + <note> + Be sure to work in the <filename>meta-intel</filename> + branch that matches your + <link linkend='source-directory'>Source Directory</link> + (i.e. <filename>poky</filename>) branch. + For example, if you have checked out the "master" branch + of <filename>poky</filename> and you are going to use + <filename>meta-intel</filename>, be sure to checkout the + "master" branch of <filename>meta-intel</filename>. + </note> + <literallayout class='monospaced'> + $ cd ~/poky + $ git clone git://git.yoctoproject.org/meta-intel.git + Cloning into 'meta-intel'... + remote: Counting objects: 11917, done. + remote: Compressing objects: 100% (3842/3842), done. + remote: Total 11917 (delta 6840), reused 11699 (delta 6622) + Receiving objects: 100% (11917/11917), 2.92 MiB | 2.88 MiB/s, done. + Resolving deltas: 100% (6840/6840), done. + Checking connectivity... done. + </literallayout></para> + + <para>The same + <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>wiki page</ulink> + referenced earlier covers how to set up the + <filename>meta-intel</filename> Git repository. + </para></listitem> + <listitem><para><emphasis>Eclipse Yocto Plug-in:</emphasis> If you are developing + applications using the Eclipse Integrated Development Environment (IDE), + you will need this plug-in. + See the + "<ulink url='&YOCTO_DOCS_SDK_URL;#setting-up-the-eclipse-ide'>Setting up the Eclipse IDE</ulink>" + section in the Yocto Project Software Development Kit (SDK) + Developer's Guide for more information.</para></listitem> + </itemizedlist> + </para> +</section> + +<section id='building-images'> + <title>Building Images</title> + + <para> + The build process creates an entire Linux distribution, including the toolchain, from source. + For more information on this topic, see the + "<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>" + section in the Yocto Project Quick Start. + </para> + + <para> + The build process is as follows: + <orderedlist> + <listitem><para>Make sure you have set up the Source Directory described in the + previous section.</para></listitem> + <listitem><para>Initialize the build environment by sourcing a build + environment script (i.e. + <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> + or + <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>). + </para></listitem> + <listitem><para>Optionally ensure the <filename>conf/local.conf</filename> configuration file, + which is found in the + <link linkend='build-directory'>Build Directory</link>, + is set up how you want it. + This file defines many aspects of the build environment including + the target machine architecture through the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'>MACHINE</ulink></filename> variable, + the packaging format used during the build + (<ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>), + and a centralized tarball download directory through the + <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'>DL_DIR</ulink></filename> variable.</para></listitem> + <listitem><para> + Build the image using the <filename>bitbake</filename> command. + If you want information on BitBake, see the + <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. + </para></listitem> + <listitem><para>Run the image either on the actual hardware or using the QEMU + emulator.</para></listitem> + </orderedlist> + </para> +</section> + +<section id='using-pre-built-binaries-and-qemu'> + <title>Using Pre-Built Binaries and QEMU</title> + + <para> + Another option you have to get started is to use pre-built binaries. + The Yocto Project provides many types of binaries with each release. + See the "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" + chapter in the Yocto Project Reference Manual + for descriptions of the types of binaries that ship with a Yocto Project + release. + </para> + + <para> + Using a pre-built binary is ideal for developing software + applications to run on your target hardware. + To do this, you need to be able to access the appropriate + cross-toolchain tarball for the architecture on which you are + developing. + If you are using an SDK type image, the image ships with the complete + toolchain native to the architecture (i.e. a toolchain designed to + run on the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>). + If you are not using an SDK type image, you need to separately download + and install the stand-alone Yocto Project cross-toolchain tarball. + </para> + + <para> + Regardless of the type of image you are using, you need to download the pre-built kernel + that you will boot in the QEMU emulator and then download and extract the target root + filesystem for your target machine’s architecture. + You can get architecture-specific binaries and file systems from + <ulink url='&YOCTO_MACHINES_DL_URL;'>machines</ulink>. + You can get installation scripts for stand-alone toolchains from + <ulink url='&YOCTO_TOOLCHAIN_DL_URL;'>toolchains</ulink>. + Once you have all your files, you set up the environment to emulate the hardware + by sourcing an environment setup script. + Finally, you start the QEMU emulator. + You can find details on all these steps in the + <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. + You can learn more about using QEMU with the Yocto Project in the + "<link linkend='dev-manual-qemu'>Using the Quick EMUlator (QEMU)</link>" + section. + </para> + + <para> + Using QEMU to emulate your hardware can result in speed issues + depending on the target and host architecture mix. + For example, using the <filename>qemux86</filename> image in the emulator + on an Intel-based 32-bit (x86) host machine is fast because the target and + host architectures match. + On the other hand, using the <filename>qemuarm</filename> image on the same Intel-based + host can be slower. + But, you still achieve faithful emulation of ARM-specific issues. + </para> + + <para> + To speed things up, the QEMU images support using <filename>distcc</filename> + to call a cross-compiler outside the emulated system. + If you used <filename>runqemu</filename> to start QEMU, and the + <filename>distccd</filename> application is present on the host system, any + BitBake cross-compiling toolchain available from the build system is automatically + used from within QEMU simply by calling <filename>distcc</filename>. + You can accomplish this by defining the cross-compiler variable + (e.g. <filename>export CC="distcc"</filename>). + Alternatively, if you are using a suitable SDK image or the appropriate + stand-alone toolchain is present, + the toolchain is also automatically used. + </para> + + <note> + Several mechanisms exist that let you connect to the system running on the + QEMU emulator: + <itemizedlist> + <listitem><para>QEMU provides a framebuffer interface that makes standard + consoles available.</para></listitem> + <listitem><para>Generally, headless embedded devices have a serial port. + If so, you can configure the operating system of the running image + to use that port to run a console. + The connection uses standard IP networking.</para></listitem> + <listitem><para> + SSH servers exist in some QEMU images. + The <filename>core-image-sato</filename> QEMU image has a + Dropbear secure shell (SSH) server that runs with the root + password disabled. + The <filename>core-image-full-cmdline</filename> and + <filename>core-image-lsb</filename> QEMU images + have OpenSSH instead of Dropbear. + Including these SSH servers allow you to use standard + <filename>ssh</filename> and <filename>scp</filename> commands. + The <filename>core-image-minimal</filename> QEMU image, + however, contains no SSH server. + </para></listitem> + <listitem><para>You can use a provided, user-space NFS server to boot the QEMU session + using a local copy of the root filesystem on the host. + In order to make this connection, you must extract a root filesystem tarball by using the + <filename>runqemu-extract-sdk</filename> command. + After running the command, you must then point the <filename>runqemu</filename> + script to the extracted directory instead of a root filesystem image file.</para></listitem> + </itemizedlist> + </note> +</section> +</chapter> +<!-- +vim: expandtab tw=80 ts=4 +--> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-manual.xml b/import-layers/yocto-poky/documentation/dev-manual/dev-manual.xml new file mode 100644 index 000000000..791a8cb6a --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-manual.xml @@ -0,0 +1,129 @@ +<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<book id='dev-manual' lang='en' + xmlns:xi="http://www.w3.org/2003/XInclude" + xmlns="http://docbook.org/ns/docbook" + > + <bookinfo> + + <mediaobject> + <imageobject> + <imagedata fileref='figures/dev-title.png' + format='SVG' + align='left' scalefit='1' width='100%'/> + </imageobject> + </mediaobject> + + <title> + Yocto Project Development Manual + </title> + + <authorgroup> + <author> + <firstname>Scott</firstname> <surname>Rifenbark</surname> + <affiliation> + <orgname>Intel Corporation</orgname> + </affiliation> + <email>srifenbark@gmail.com</email> + </author> + </authorgroup> + + <revhistory> + <revision> + <revnumber>1.1</revnumber> + <date>6 October 2011</date> + <revremark>The initial document released with the Yocto Project 1.1 Release.</revremark> + </revision> + <revision> + <revnumber>1.2</revnumber> + <date>April 2012</date> + <revremark>Released with the Yocto Project 1.2 Release.</revremark> + </revision> + <revision> + <revnumber>1.3</revnumber> + <date>October 2012</date> + <revremark>Released with the Yocto Project 1.3 Release.</revremark> + </revision> + <revision> + <revnumber>1.4</revnumber> + <date>April 2013</date> + <revremark>Released with the Yocto Project 1.4 Release.</revremark> + </revision> + <revision> + <revnumber>1.5</revnumber> + <date>October 2013</date> + <revremark>Released with the Yocto Project 1.5 Release.</revremark> + </revision> + <revision> + <revnumber>1.5.1</revnumber> + <date>January 2014</date> + <revremark>Released with the Yocto Project 1.5.1 Release.</revremark> + </revision> + <revision> + <revnumber>1.6</revnumber> + <date>April 2014</date> + <revremark>Released with the Yocto Project 1.6 Release.</revremark> + </revision> + <revision> + <revnumber>1.7</revnumber> + <date>October 2014</date> + <revremark>Released with the Yocto Project 1.7 Release.</revremark> + </revision> + <revision> + <revnumber>1.8</revnumber> + <date>April 2015</date> + <revremark>Released with the Yocto Project 1.8 Release.</revremark> + </revision> + <revision> + <revnumber>2.0</revnumber> + <date>October 2015</date> + <revremark>Released with the Yocto Project 2.0 Release.</revremark> + </revision> + <revision> + <revnumber>2.1</revnumber> + <date>April 2016</date> + <revremark>Released with the Yocto Project 2.1 Release.</revremark> + </revision> + </revhistory> + + <copyright> + <year>©RIGHT_YEAR;</year> + <holder>Linux Foundation</holder> + </copyright> + + <legalnotice> + <para> + Permission is granted to copy, distribute and/or modify this document under + the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-sa/2.0/uk/"> + Creative Commons Attribution-Share Alike 2.0 UK: England & Wales</ulink> as published by + Creative Commons. + </para> + + <note> + For the latest version of this manual associated with this + Yocto Project release, see the + <ulink url='&YOCTO_DOCS_DEV_URL;'>Yocto Project Development Manual</ulink> + from the Yocto Project website. + </note> + </legalnotice> + + </bookinfo> + + <xi:include href="dev-manual-intro.xml"/> + + <xi:include href="dev-manual-start.xml"/> + + <xi:include href="dev-manual-newbie.xml"/> + + <xi:include href="dev-manual-model.xml"/> + + <xi:include href="dev-manual-common-tasks.xml"/> + + <xi:include href="dev-manual-qemu.xml"/> + +</book> +<!-- +vim: expandtab tw=80 ts=4 +--> diff --git a/import-layers/yocto-poky/documentation/dev-manual/dev-style.css b/import-layers/yocto-poky/documentation/dev-manual/dev-style.css new file mode 100644 index 000000000..6d0aa8e9f --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/dev-style.css @@ -0,0 +1,988 @@ +/* + Generic XHTML / DocBook XHTML CSS Stylesheet. + + Browser wrangling and typographic design by + Oyvind Kolas / pippin@gimp.org + + Customised for Poky by + Matthew Allum / mallum@o-hand.com + + Thanks to: + Liam R. E. Quin + William Skaggs + Jakub Steiner + + Structure + --------- + + The stylesheet is divided into the following sections: + + Positioning + Margins, paddings, width, font-size, clearing. + Decorations + Borders, style + Colors + Colors + Graphics + Graphical backgrounds + Nasty IE tweaks + Workarounds needed to make it work in internet explorer, + currently makes the stylesheet non validating, but up until + this point it is validating. + Mozilla extensions + Transparency for footer + Rounded corners on boxes + +*/ + + + /*************** / + / Positioning / +/ ***************/ + +body { + font-family: Verdana, Sans, sans-serif; + + min-width: 640px; + width: 80%; + margin: 0em auto; + padding: 2em 5em 5em 5em; + color: #333; +} + +h1,h2,h3,h4,h5,h6,h7 { + font-family: Arial, Sans; + color: #00557D; + clear: both; +} + +h1 { + font-size: 2em; + text-align: left; + padding: 0em 0em 0em 0em; + margin: 2em 0em 0em 0em; +} + +h2.subtitle { + margin: 0.10em 0em 3.0em 0em; + padding: 0em 0em 0em 0em; + font-size: 1.8em; + padding-left: 20%; + font-weight: normal; + font-style: italic; +} + +h2 { + margin: 2em 0em 0.66em 0em; + padding: 0.5em 0em 0em 0em; + font-size: 1.5em; + font-weight: bold; +} + +h3.subtitle { + margin: 0em 0em 1em 0em; + padding: 0em 0em 0em 0em; + font-size: 142.14%; + text-align: right; +} + +h3 { + margin: 1em 0em 0.5em 0em; + padding: 1em 0em 0em 0em; + font-size: 140%; + font-weight: bold; +} + +h4 { + margin: 1em 0em 0.5em 0em; + padding: 1em 0em 0em 0em; + font-size: 120%; + font-weight: bold; +} + +h5 { + margin: 1em 0em 0.5em 0em; + padding: 1em 0em 0em 0em; + font-size: 110%; + font-weight: bold; +} + +h6 { + margin: 1em 0em 0em 0em; + padding: 1em 0em 0em 0em; + font-size: 110%; + font-weight: bold; +} + +.authorgroup { + background-color: transparent; + background-repeat: no-repeat; + padding-top: 256px; + background-image: url("figures/dev-title.png"); + background-position: left top; + margin-top: -256px; + padding-right: 50px; + margin-left: 0px; + text-align: right; + width: 740px; +} + +h3.author { + margin: 0em 0me 0em 0em; + padding: 0em 0em 0em 0em; + font-weight: normal; + font-size: 100%; + color: #333; + clear: both; +} + +.author tt.email { + font-size: 66%; +} + +.titlepage hr { + width: 0em; + clear: both; +} + +.revhistory { + padding-top: 2em; + clear: both; +} + +.toc, +.list-of-tables, +.list-of-examples, +.list-of-figures { + padding: 1.33em 0em 2.5em 0em; + color: #00557D; +} + +.toc p, +.list-of-tables p, +.list-of-figures p, +.list-of-examples p { + padding: 0em 0em 0em 0em; + padding: 0em 0em 0.3em; + margin: 1.5em 0em 0em 0em; +} + +.toc p b, +.list-of-tables p b, +.list-of-figures p b, +.list-of-examples p b{ + font-size: 100.0%; + font-weight: bold; +} + +.toc dl, +.list-of-tables dl, +.list-of-figures dl, +.list-of-examples dl { + margin: 0em 0em 0.5em 0em; + padding: 0em 0em 0em 0em; +} + +.toc dt { + margin: 0em 0em 0em 0em; + padding: 0em 0em 0em 0em; +} + +.toc dd { + margin: 0em 0em 0em 2.6em; + padding: 0em 0em 0em 0em; +} + +div.glossary dl, +div.variablelist dl { +} + +.glossary dl dt, +.variablelist dl dt, +.variablelist dl dt span.term { + font-weight: normal; + width: 20em; + text-align: right; +} + +.variablelist dl dt { + margin-top: 0.5em; +} + +.glossary dl dd, +.variablelist dl dd { + margin-top: -1em; + margin-left: 25.5em; +} + +.glossary dd p, +.variablelist dd p { + margin-top: 0em; + margin-bottom: 1em; +} + + +div.calloutlist table td { + padding: 0em 0em 0em 0em; + margin: 0em 0em 0em 0em; +} + +div.calloutlist table td p { + margin-top: 0em; + margin-bottom: 1em; +} + +div p.copyright { + text-align: left; +} + +div.legalnotice p.legalnotice-title { + margin-bottom: 0em; +} + +p { + line-height: 1.5em; + margin-top: 0em; + +} + +dl { + padding-top: 0em; +} + +hr { + border: solid 1px; +} + + +.mediaobject, +.mediaobjectco { + text-align: center; +} + +img { + border: none; +} + +ul { + padding: 0em 0em 0em 1.5em; +} + +ul li { + padding: 0em 0em 0em 0em; +} + +ul li p { + text-align: left; +} + +table { + width :100%; +} + +th { + padding: 0.25em; + text-align: left; + font-weight: normal; + vertical-align: top; +} + +td { + padding: 0.25em; + vertical-align: top; +} + +p a[id] { + margin: 0px; + padding: 0px; + display: inline; + background-image: none; +} + +a { + text-decoration: underline; + color: #444; +} + +pre { + overflow: auto; +} + +a:hover { + text-decoration: underline; + /*font-weight: bold;*/ +} + +/* This style defines how the permalink character + appears by itself and when hovered over with + the mouse. */ + +[alt='Permalink'] { color: #eee; } +[alt='Permalink']:hover { color: black; } + + +div.informalfigure, +div.informalexample, +div.informaltable, +div.figure, +div.table, +div.example { + margin: 1em 0em; + padding: 1em; + page-break-inside: avoid; +} + + +div.informalfigure p.title b, +div.informalexample p.title b, +div.informaltable p.title b, +div.figure p.title b, +div.example p.title b, +div.table p.title b{ + padding-top: 0em; + margin-top: 0em; + font-size: 100%; + font-weight: normal; +} + +.mediaobject .caption, +.mediaobject .caption p { + text-align: center; + font-size: 80%; + padding-top: 0.5em; + padding-bottom: 0.5em; +} + +.epigraph { + padding-left: 55%; + margin-bottom: 1em; +} + +.epigraph p { + text-align: left; +} + +.epigraph .quote { + font-style: italic; +} +.epigraph .attribution { + font-style: normal; + text-align: right; +} + +span.application { + font-style: italic; +} + +.programlisting { + font-family: monospace; + font-size: 80%; + white-space: pre; + margin: 1.33em 0em; + padding: 1.33em; +} + +.tip, +.warning, +.caution, +.note { + margin-top: 1em; + margin-bottom: 1em; + +} + +/* force full width of table within div */ +.tip table, +.warning table, +.caution table, +.note table { + border: none; + width: 100%; +} + + +.tip table th, +.warning table th, +.caution table th, +.note table th { + padding: 0.8em 0.0em 0.0em 0.0em; + margin : 0em 0em 0em 0em; +} + +.tip p, +.warning p, +.caution p, +.note p { + margin-top: 0.5em; + margin-bottom: 0.5em; + padding-right: 1em; + text-align: left; +} + +.acronym { + text-transform: uppercase; +} + +b.keycap, +.keycap { + padding: 0.09em 0.3em; + margin: 0em; +} + +.itemizedlist li { + clear: none; +} + +.filename { + font-size: medium; + font-family: Courier, monospace; +} + + +div.navheader, div.heading{ + position: absolute; + left: 0em; + top: 0em; + width: 100%; + background-color: #cdf; + width: 100%; +} + +div.navfooter, div.footing{ + position: fixed; + left: 0em; + bottom: 0em; + background-color: #eee; + width: 100%; +} + + +div.navheader td, +div.navfooter td { + font-size: 66%; +} + +div.navheader table th { + /*font-family: Georgia, Times, serif;*/ + /*font-size: x-large;*/ + font-size: 80%; +} + +div.navheader table { + border-left: 0em; + border-right: 0em; + border-top: 0em; + width: 100%; +} + +div.navfooter table { + border-left: 0em; + border-right: 0em; + border-bottom: 0em; + width: 100%; +} + +div.navheader table td a, +div.navfooter table td a { + color: #777; + text-decoration: none; +} + +/* normal text in the footer */ +div.navfooter table td { + color: black; +} + +div.navheader table td a:visited, +div.navfooter table td a:visited { + color: #444; +} + + +/* links in header and footer */ +div.navheader table td a:hover, +div.navfooter table td a:hover { + text-decoration: underline; + background-color: transparent; + color: #33a; +} + +div.navheader hr, +div.navfooter hr { + display: none; +} + + +.qandaset tr.question td p { + margin: 0em 0em 1em 0em; + padding: 0em 0em 0em 0em; +} + +.qandaset tr.answer td p { + margin: 0em 0em 1em 0em; + padding: 0em 0em 0em 0em; +} +.answer td { + padding-bottom: 1.5em; +} + +.emphasis { + font-weight: bold; +} + + + /************* / + / decorations / +/ *************/ + +.titlepage { +} + +.part .title { +} + +.subtitle { + border: none; +} + +/* +h1 { + border: none; +} + +h2 { + border-top: solid 0.2em; + border-bottom: solid 0.06em; +} + +h3 { + border-top: 0em; + border-bottom: solid 0.06em; +} + +h4 { + border: 0em; + border-bottom: solid 0.06em; +} + +h5 { + border: 0em; +} +*/ + +.programlisting { + border: solid 1px; +} + +div.figure, +div.table, +div.informalfigure, +div.informaltable, +div.informalexample, +div.example { + border: 1px solid; +} + + + +.tip, +.warning, +.caution, +.note { + border: 1px solid; +} + +.tip table th, +.warning table th, +.caution table th, +.note table th { + border-bottom: 1px solid; +} + +.question td { + border-top: 1px solid black; +} + +.answer { +} + + +b.keycap, +.keycap { + border: 1px solid; +} + + +div.navheader, div.heading{ + border-bottom: 1px solid; +} + + +div.navfooter, div.footing{ + border-top: 1px solid; +} + + /********* / + / colors / +/ *********/ + +body { + color: #333; + background: white; +} + +a { + background: transparent; +} + +a:hover { + background-color: #dedede; +} + + +h1, +h2, +h3, +h4, +h5, +h6, +h7, +h8 { + background-color: transparent; +} + +hr { + border-color: #aaa; +} + + +.tip, .warning, .caution, .note { + border-color: #fff; +} + + +.tip table th, +.warning table th, +.caution table th, +.note table th { + border-bottom-color: #fff; +} + + +.warning { + background-color: #f0f0f2; +} + +.caution { + background-color: #f0f0f2; +} + +.tip { + background-color: #f0f0f2; +} + +.note { + background-color: #f0f0f2; +} + +.glossary dl dt, +.variablelist dl dt, +.variablelist dl dt span.term { + color: #044; +} + +div.figure, +div.table, +div.example, +div.informalfigure, +div.informaltable, +div.informalexample { + border-color: #aaa; +} + +pre.programlisting { + color: black; + background-color: #fff; + border-color: #aaa; + border-width: 2px; +} + +.guimenu, +.guilabel, +.guimenuitem { + background-color: #eee; +} + + +b.keycap, +.keycap { + background-color: #eee; + border-color: #999; +} + + +div.navheader { + border-color: black; +} + + +div.navfooter { + border-color: black; +} + +.writernotes { + color: red; +} + + + /*********** / + / graphics / +/ ***********/ + +/* +body { + background-image: url("images/body_bg.jpg"); + background-attachment: fixed; +} + +.navheader, +.note, +.tip { + background-image: url("images/note_bg.jpg"); + background-attachment: fixed; +} + +.warning, +.caution { + background-image: url("images/warning_bg.jpg"); + background-attachment: fixed; +} + +.figure, +.informalfigure, +.example, +.informalexample, +.table, +.informaltable { + background-image: url("images/figure_bg.jpg"); + background-attachment: fixed; +} + +*/ +h1, +h2, +h3, +h4, +h5, +h6, +h7{ +} + +/* +Example of how to stick an image as part of the title. + +div.article .titlepage .title +{ + background-image: url("figures/white-on-black.png"); + background-position: center; + background-repeat: repeat-x; +} +*/ + +div.preface .titlepage .title, +div.colophon .title, +div.chapter .titlepage .title, +div.article .titlepage .title +{ +} + +div.section div.section .titlepage .title, +div.sect2 .titlepage .title { + background: none; +} + + +h1.title { + background-color: transparent; + background-repeat: no-repeat; + height: 256px; + text-indent: -9000px; + overflow:hidden; +} + +h2.subtitle { + background-color: transparent; + text-indent: -9000px; + overflow:hidden; + width: 0px; + display: none; +} + + /*************************************** / + / pippin.gimp.org specific alterations / +/ ***************************************/ + +/* +div.heading, div.navheader { + color: #777; + font-size: 80%; + padding: 0; + margin: 0; + text-align: left; + position: absolute; + top: 0px; + left: 0px; + width: 100%; + height: 50px; + background: url('/gfx/heading_bg.png') transparent; + background-repeat: repeat-x; + background-attachment: fixed; + border: none; +} + +div.heading a { + color: #444; +} + +div.footing, div.navfooter { + border: none; + color: #ddd; + font-size: 80%; + text-align:right; + + width: 100%; + padding-top: 10px; + position: absolute; + bottom: 0px; + left: 0px; + + background: url('/gfx/footing_bg.png') transparent; +} +*/ + + + + /****************** / + / nasty ie tweaks / +/ ******************/ + +/* +div.heading, div.navheader { + width:expression(document.body.clientWidth + "px"); +} + +div.footing, div.navfooter { + width:expression(document.body.clientWidth + "px"); + margin-left:expression("-5em"); +} +body { + padding:expression("4em 5em 0em 5em"); +} +*/ + + /**************************************** / + / mozilla vendor specific css extensions / +/ ****************************************/ +/* +div.navfooter, div.footing{ + -moz-opacity: 0.8em; +} + +div.figure, +div.table, +div.informalfigure, +div.informaltable, +div.informalexample, +div.example, +.tip, +.warning, +.caution, +.note { + -moz-border-radius: 0.5em; +} + +b.keycap, +.keycap { + -moz-border-radius: 0.3em; +} +*/ + +table tr td table tr td { + display: none; +} + + +hr { + display: none; +} + +table { + border: 0em; +} + + .photo { + float: right; + margin-left: 1.5em; + margin-bottom: 1.5em; + margin-top: 0em; + max-width: 17em; + border: 1px solid gray; + padding: 3px; + background: white; +} + .seperator { + padding-top: 2em; + clear: both; + } + + #validators { + margin-top: 5em; + text-align: right; + color: #777; + } + @media print { + body { + font-size: 8pt; + } + .noprint { + display: none; + } + } + + +.tip, +.note { + background: #f0f0f2; + color: #333; + padding: 20px; + margin: 20px; +} + +.tip h3, +.note h3 { + padding: 0em; + margin: 0em; + font-size: 2em; + font-weight: bold; + color: #333; +} + +.tip a, +.note a { + color: #333; + text-decoration: underline; +} + +.footnote { + font-size: small; + color: #333; +} + +/* Changes the announcement text */ +.tip h3, +.warning h3, +.caution h3, +.note h3 { + font-size:large; + color: #00557D; +} diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/bsp-dev-flow.png b/import-layers/yocto-poky/documentation/dev-manual/figures/bsp-dev-flow.png Binary files differnew file mode 100644 index 000000000..540b0abb9 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/bsp-dev-flow.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/build-workspace-directory.png b/import-layers/yocto-poky/documentation/dev-manual/figures/build-workspace-directory.png Binary files differnew file mode 100644 index 000000000..5387d33f0 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/build-workspace-directory.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/dev-title.png b/import-layers/yocto-poky/documentation/dev-manual/figures/dev-title.png Binary files differnew file mode 100644 index 000000000..d3cac4a7e --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/dev-title.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-add-flow.png b/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-add-flow.png Binary files differnew file mode 100644 index 000000000..c09e60e35 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-add-flow.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-modify-flow.png b/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-modify-flow.png Binary files differnew file mode 100644 index 000000000..cd7f4d05b --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-modify-flow.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-upgrade-flow.png b/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-upgrade-flow.png Binary files differnew file mode 100644 index 000000000..d25168c84 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/devtool-upgrade-flow.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/git-workflow.png b/import-layers/yocto-poky/documentation/dev-manual/figures/git-workflow.png Binary files differnew file mode 100644 index 000000000..e401330a1 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/git-workflow.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/index-downloads.png b/import-layers/yocto-poky/documentation/dev-manual/figures/index-downloads.png Binary files differnew file mode 100644 index 000000000..41251d5d1 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/index-downloads.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-dev-flow.png b/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-dev-flow.png Binary files differnew file mode 100644 index 000000000..009105d12 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-dev-flow.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-overview-1.png b/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-overview-1.png Binary files differnew file mode 100644 index 000000000..116c0b9bd --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-overview-1.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-overview-2-generic.png b/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-overview-2-generic.png Binary files differnew file mode 100644 index 000000000..cb970eae7 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/kernel-overview-2-generic.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/recipe-workflow.png b/import-layers/yocto-poky/documentation/dev-manual/figures/recipe-workflow.png Binary files differnew file mode 100644 index 000000000..c0e960b13 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/recipe-workflow.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/source-repos.png b/import-layers/yocto-poky/documentation/dev-manual/figures/source-repos.png Binary files differnew file mode 100644 index 000000000..65c5f29a4 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/source-repos.png diff --git a/import-layers/yocto-poky/documentation/dev-manual/figures/yp-download.png b/import-layers/yocto-poky/documentation/dev-manual/figures/yp-download.png Binary files differnew file mode 100644 index 000000000..5770be688 --- /dev/null +++ b/import-layers/yocto-poky/documentation/dev-manual/figures/yp-download.png |
