Getting Started: Building Android From Source
Difficulty Level: Intermediate-Advanced
Introduction
From browsing around XDA-Developers you have probably seen a large number of ROMs which are built from source. This includes many of the popular multi-device custom ROM projects such as CyanogenMod and AOKP.
This chapter aims to teach you the basics of setting up the required environment to get started on building Android from source.
This guide is adapted from the Android guidelines on building. Also, please bear in mind that this guide presumes you have access to a computer or virtual machine running a 64-bit Debian-based Linux operating system such as Ubuntu (the procedures of setting this up are well outwith the scope of this book). You do not need to use Debian, but these instructions have been designed for it, so procedures may vary.
From browsing around XDA-Developers you have probably seen a large number of ROMs which are built from source. This includes many of the popular multi-device custom ROM projects such as CyanogenMod and AOKP.
This chapter aims to teach you the basics of setting up the required environment to get started on building Android from source.
This guide is adapted from the Android guidelines on building. Also, please bear in mind that this guide presumes you have access to a computer or virtual machine running a 64-bit Debian-based Linux operating system such as Ubuntu (the procedures of setting this up are well outwith the scope of this book). You do not need to use Debian, but these instructions have been designed for it, so procedures may vary.
Initial Setup
Before beginning this process, please ensure you have sufficient hard drive space. The Android source code alone consumes around 9 GB of hard drive space, and to carry out a single device build at least 30 GB will be required. If building for more devices, this can easily exceed 100 GB.
In order to build Android from source successfully, you will require Python 2.7, GNU Make 3.81 or 3.82, Git 1.7 or later, and the Oracle (Sun) Java Development Kit version 6.
The following packages should also be installed via “sudo apt-get install”, followed by the list of packages. The web copy of this book can be used, so that you can copy-paste this list.
Next, in order to address an incorrect library with Ubuntu 11.10 and above, run the following command:
The next step is optional, but can offer significant performance gains while compiling, by using ccache (a compiler cache tool), which caches the output of the compiler, to save time when compiling a file which has not been changed since the last compile.
To enable ccache, edit your ~/.bashrc or ~/.profile file (or equivalent), and add the following line
After you log out and back in again, this change will take effect.
By default, the Android build process creates a subfolder named “out” within the root of your source tree. While for most users this is best, in some situations it can be advantageous to move the output directory to another file system. If, for example, you have a striped RAID array, it is beneficial to store the output directory on this array. All of the files within the out directory can be re-generated in the event of filesystem problems (if the sources were held on another filesystem).
To change the output directory to another filesystem, use the command
Within the defined output directory, a new folder will be created, named after your current source tree directory. For instance, if you have source trees as /source/master1 and /source/master2 and OUT_DIR_COMMON_BASE is set to /output, the output directories will be/output/master1 and /output/master2.
It is therefore important to ensure you do not have multiple sources stored in directories with the same name, as they would end up sharing an output directory, with unpredictable results.
This is only supported when building Jelly Bean (4.1) and newer.
Before beginning this process, please ensure you have sufficient hard drive space. The Android source code alone consumes around 9 GB of hard drive space, and to carry out a single device build at least 30 GB will be required. If building for more devices, this can easily exceed 100 GB.
In order to build Android from source successfully, you will require Python 2.7, GNU Make 3.81 or 3.82, Git 1.7 or later, and the Oracle (Sun) Java Development Kit version 6.
The following packages should also be installed via “sudo apt-get install”, followed by the list of packages. The web copy of this book can be used, so that you can copy-paste this list.
Next, in order to address an incorrect library with Ubuntu 11.10 and above, run the following command:
The next step is optional, but can offer significant performance gains while compiling, by using ccache (a compiler cache tool), which caches the output of the compiler, to save time when compiling a file which has not been changed since the last compile.
To enable ccache, edit your ~/.bashrc or ~/.profile file (or equivalent), and add the following line
After you log out and back in again, this change will take effect.
By default, the Android build process creates a subfolder named “out” within the root of your source tree. While for most users this is best, in some situations it can be advantageous to move the output directory to another file system. If, for example, you have a striped RAID array, it is beneficial to store the output directory on this array. All of the files within the out directory can be re-generated in the event of filesystem problems (if the sources were held on another filesystem).
To change the output directory to another filesystem, use the command
Within the defined output directory, a new folder will be created, named after your current source tree directory. For instance, if you have source trees as /source/master1 and /source/master2 and OUT_DIR_COMMON_BASE is set to /output, the output directories will be/output/master1 and /output/master2.
It is therefore important to ensure you do not have multiple sources stored in directories with the same name, as they would end up sharing an output directory, with unpredictable results.
This is only supported when building Jelly Bean (4.1) and newer.
Preparing Repo
At this point, the basic setup of the build environment has now been configured, and it’s time to obtain the Android sources.
Firstly, the repo tool must be installed. This is a tool that allows the Android source code to be downloaded automatically from the hundreds of individual repositories in use by Android. Ensure you have a “bin” folder within your home directory, and that it is within your path by running the commands
The latter command can be added to your ~/.bashrc script, to ensure the bin folder is always added to your path variable. The repo script can now be downloaded (run this command from within the ~/bin directory), and the correct permissions set.
At this point, the basic setup of the build environment has now been configured, and it’s time to obtain the Android sources.
Firstly, the repo tool must be installed. This is a tool that allows the Android source code to be downloaded automatically from the hundreds of individual repositories in use by Android. Ensure you have a “bin” folder within your home directory, and that it is within your path by running the commands
The latter command can be added to your ~/.bashrc script, to ensure the bin folder is always added to your path variable. The repo script can now be downloaded (run this command from within the ~/bin directory), and the correct permissions set.
Initialising a Repo and Syncing
Initialisation of the repo creates the bare skeleton directory structure required for storing the Android source code. Create a directory within your home directory where you wish to copy the source code to (as mentioned previously, this drive requires a large amount of free space).
Within this directory, run the command
Here, the URL of the repository you wish to clone should be entered, and the branch desired. This information differs for the ROM source you wish to obtain. By referring to the project’s gitrepositories, and looking for a README, the suitable command should be given. The branch defines the Android version you wish to clone, such as “ics” or “jellybean” (using the CyanogenMod naming scheme)
After the repo initialisation completes, a new hidden directory called “.repo” should have been created within this folder, containing the manifest files that detail what should be synced from the remote source repository.
It is now time to download the actual sources – this will take a significant amount of time, so it is best to leave the computer to complete this and return later. Run the command “repo sync”, and it will begin syncing. In the event of the process being interrupted, re-run repo sync, and the process will be resumed, although a small amount of data may be re-downloaded.
In future, when it is desired to update the source code to obtain the latest updates to the ROM you are building, the command “repo sync” can be used on its own to update the source tree. Be aware that doing this will remove and eliminate any changes you have made to your local sources.
Initialisation of the repo creates the bare skeleton directory structure required for storing the Android source code. Create a directory within your home directory where you wish to copy the source code to (as mentioned previously, this drive requires a large amount of free space).
Within this directory, run the command
Here, the URL of the repository you wish to clone should be entered, and the branch desired. This information differs for the ROM source you wish to obtain. By referring to the project’s gitrepositories, and looking for a README, the suitable command should be given. The branch defines the Android version you wish to clone, such as “ics” or “jellybean” (using the CyanogenMod naming scheme)
After the repo initialisation completes, a new hidden directory called “.repo” should have been created within this folder, containing the manifest files that detail what should be synced from the remote source repository.
It is now time to download the actual sources – this will take a significant amount of time, so it is best to leave the computer to complete this and return later. Run the command “repo sync”, and it will begin syncing. In the event of the process being interrupted, re-run repo sync, and the process will be resumed, although a small amount of data may be re-downloaded.
In future, when it is desired to update the source code to obtain the latest updates to the ROM you are building, the command “repo sync” can be used on its own to update the source tree. Be aware that doing this will remove and eliminate any changes you have made to your local sources.
Carrying out an Actual Build
To carry out an actual build for a device, it’s necessary to have some device-specific components and source code. Pure Android (Google AOSP) only supports a small subset of available devices (mainly Nexus devices, with a few exceptions that have been added). In order to make a build for your device, you will need a suitably device configuration. Often these are included within the main source download, or are obtained through a method documented by the developers of theROM.
TIP: The device tree from an AOSP ROM is not necessarily compatible with a CyanogenMod (or other custom ROM), and vice versa. In fact, without changes, it will not allow the build to complete. For this reason, the correct type of device tree is required.
Once you have obtained the suitable device sources (you can add a git repository to the .repo/local_manifest.xml file if desired, in order to add it to the repositories updated via a “reposync”), it is time to prepare the build environment.
First, the envsetup.sh script must be executed, which sets up and imports all available device configs. It also adds a few “macro” commands to your environment, such as “chroot”, which will change directory to the root of the Android source tree.This step must be carried out every time you start a new shell session, or reboot. Run the command
Now, the target device should be selected and configured using the “lunch” command. Type “lunch”, and a list of the available devices will be offered. Enter the number of your device to select it for building. Be aware that the device names are often “codenames” rather than final shipping names, but you can research this in your device forum.
Once you know your device’s name, you can directly invoke the command “lunch -userdebug”. For example, the command
would be invoked to select the Galaxy S2 (i9100) in a CyanogenMod build (hence the “cm_” prefix). Userdebug refers to the type of build being carried out – a userdebug build allows rootaccess, and has some basic debug features available for use in diagnosing issues.
At this point, Android is finally ready to be built! To carry out a build, the “make” command is used. There are some arguments that can be used to speed up this process though on multi-core systems. By using the “-j X” option, where X is twice the number of CPU cores in a system, Android will be built with parallel threads running to make use of the multiple CPU cores.
In some ROMs, it is possible to use the command “mka” to automatically carry this out. Before actually building though, it is important to decide what type of build is required. Running “make” will produce a basic, image based build, which is not easily flashed to a device. Chances are, you want to build a flash-able update.zip style file which can easily be installed to a device. To do this, use the command “make otapackage” (for CyanogenMod ROMs, “make bacon” is used instead).
To carry out a new build by clearing out the prepared/packaged output files, run the command
This will remove the compiled files from previous builds. At this point, make can be invoked to run a build as before.
To carry out a clean build from scratch (or clear out the large “out” directory if you no longer need the files produced), run the command
This will remove the compiled files from previous builds. At this point, make can be invoked to run a build from scratch as before.
Once the compilation process is completed, the update.zip can be found in the path out/target/product/. There may be multiple zips here, so ensure you are selecting the correct one, as named by the build process in the final lines when the process ended. This zip file can be flashed to the device using recovery, as detailed earlier in this book.
Umair Saifullah
To carry out an actual build for a device, it’s necessary to have some device-specific components and source code. Pure Android (Google AOSP) only supports a small subset of available devices (mainly Nexus devices, with a few exceptions that have been added). In order to make a build for your device, you will need a suitably device configuration. Often these are included within the main source download, or are obtained through a method documented by the developers of theROM.
TIP: The device tree from an AOSP ROM is not necessarily compatible with a CyanogenMod (or other custom ROM), and vice versa. In fact, without changes, it will not allow the build to complete. For this reason, the correct type of device tree is required.
Once you have obtained the suitable device sources (you can add a git repository to the .repo/local_manifest.xml file if desired, in order to add it to the repositories updated via a “reposync”), it is time to prepare the build environment.
First, the envsetup.sh script must be executed, which sets up and imports all available device configs. It also adds a few “macro” commands to your environment, such as “chroot”, which will change directory to the root of the Android source tree.This step must be carried out every time you start a new shell session, or reboot. Run the command
Now, the target device should be selected and configured using the “lunch” command. Type “lunch”, and a list of the available devices will be offered. Enter the number of your device to select it for building. Be aware that the device names are often “codenames” rather than final shipping names, but you can research this in your device forum.
Once you know your device’s name, you can directly invoke the command “lunch -userdebug”. For example, the command
would be invoked to select the Galaxy S2 (i9100) in a CyanogenMod build (hence the “cm_” prefix). Userdebug refers to the type of build being carried out – a userdebug build allows rootaccess, and has some basic debug features available for use in diagnosing issues.
At this point, Android is finally ready to be built! To carry out a build, the “make” command is used. There are some arguments that can be used to speed up this process though on multi-core systems. By using the “-j X” option, where X is twice the number of CPU cores in a system, Android will be built with parallel threads running to make use of the multiple CPU cores.
In some ROMs, it is possible to use the command “mka” to automatically carry this out. Before actually building though, it is important to decide what type of build is required. Running “make” will produce a basic, image based build, which is not easily flashed to a device. Chances are, you want to build a flash-able update.zip style file which can easily be installed to a device. To do this, use the command “make otapackage” (for CyanogenMod ROMs, “make bacon” is used instead).
To carry out a new build by clearing out the prepared/packaged output files, run the command
This will remove the compiled files from previous builds. At this point, make can be invoked to run a build as before.
To carry out a clean build from scratch (or clear out the large “out” directory if you no longer need the files produced), run the command
This will remove the compiled files from previous builds. At this point, make can be invoked to run a build from scratch as before.
Once the compilation process is completed, the update.zip can be found in the path out/target/product/. There may be multiple zips here, so ensure you are selecting the correct one, as named by the build process in the final lines when the process ended. This zip file can be flashed to the device using recovery, as detailed earlier in this book.
Umair Saifullah
