From Code to Dex — A Compilation Story

All Android code lines dream about one thing — to one day be part of a dex file, packaged in an APK and run (error) free on a device. In this blog post I want to tell you about the journey that your app’s code goes through, on its way to becoming part of dex files.

The App Module And His Friends

The main character in our story is our application module (the one in <project_dir>/app). But usually the application module hangs out with some friends — the library .jar dependencies, his bigger brothers — the .aar dependencies, and possibly even some library modules.

The application module and the library module each contain their own .java source code, resource file and proguard-rules.pro. The .aar dependency incorporates the .class bytecode files, together with the resource files and the proguard.txt file. The .jar dependency contains the .class files. All of these files need to be compiled to one (or more) dex file(s), and then afterwards packaged in an APK and run on an Android device.

Target: The Dex File

Dex files are Dalvik executable files for both ART and Dalvik runtimes. They combine the power of our four protagonists creating the bytecode that runs on Android devices.

The dex file is most known for the infamous 65k method count limit but it contains more than just a method table. More precisely, it aggregates content from the app module and all of its dependencies. The dex file format contains the following elements:

1. File Header
2. String Table
3. Class List
4. Field Table
5. Method Table
6. Class Definition Table
7. Field List
8. Method List
9. Code Header
10. Local Variable List

But the road to the dex file is long and winding. First, the Java source code is compiled using javac (part of the JDK) to create the .class files. Afterwards, using the dx tool (part of the Android SDK build tools), the Java bytecode .class files are translated to the .dex files.

Jack And Jill To The Rescue!

Two heroes appear in the lives of our characters starting with Android M, to make the road to dex less long and winding: Java Android Compiler Kit (aka Jack) and his friend the Jack Intermediate Library Linker (aka Jill). Jack and Jill tools are at the core of a new Android toolchain. They improve build times and simplify development by reducing dependencies on other tools.

Enabling Jack And Jill

To use the power of these toolchain heroes, all you need to do is enable them in your build.gradle file.

android {
    ...
    buildToolsRevision '21.1.1'
    defaultConfig {
      // Enable the Jack build tools.
      useJack = true
    }
    ...
}

Jack And Jill Super Compiler Powers

Enabling Jack brings forward the first of his powers: allowing you to start using Java 8 features in your app after adding the sourceCompatibility and targetCompatibility compile options in your build.gradle file.

Jack works directly only with one of our characters: the application module. It will compile the .java files from <project_dir>/app directly to .dex files. Jack uses .jack library files, that contain the pre-compiled dex code, a .jayce file, the resources needed and meta information.

The pre-dex from each library are used when compiling, speeding up the process.

Jill works with the other characters: library modules, .aar and .jar dependencies. From those, Jill will use just the .class files to create a .jayce file — an intermediate bytecode file. The .jayce file is then bundled together with the resources from the library dependencies to create a .jack file. From there on this library file is handled by Jack.

If you set minifyEnabled true in your build.gradle file, Jack will use the proguard-rules.po and proguard.txt files to handle shrinking and obfuscation of your code.

Jack’s powers don’t stop here. Incremental compilation is supported. With Jack, only components modified since the last compilation, together with their dependencies, are recompiled. So, when only some components were modified, the compilation time can decrease considerably compared to a full compilation.

Pre-dexing and incremental compilation do not work when shrinking/obfuscation/repackaging is enabled.

Jack brings our story to an end by putting together all the files, classes, methods and so on, from your app and other dependencies, to create the dex file. If the number of all the methods is bigger than 65K and support for multidex is enabled, then Jack handles the splitting into multiple dex files.

TL;DR

Here’s how each of the toolchains work:

Legacy javac toolchain:

javac (.java → .class) → dx (.class → .dex)

New Jack toolchain:

Jack (.java → .jack → .dex)

Jack speeds up the compilation time and handles shrinking, obfuscation, repackaging and multidex. Before you start using it, keep in mind that it’s still considered experimental.

If you want to find out more details about our heroes’ compiling journey, check out the following resources: compiling with Jack, Eric Lafortune’s talk on The Jack and Jill build system, and Jesse Wilson’s talk on Dex.