Annotation processing has been around since Java 1.5 was released in 2006. Runtime reflection always suited my needs but I’ve recently been much into low level performance, so I started tinkering with it.

Though there exist great processors like Butterknife or Dagger/2, I always try to deeply understand implementation details and so I created a very simple annotation processor example. It does less than 1% of what butterknife is able to do, but it is a good starting point for understanding how automatic annotation-based java source code generation works.

One thing to make clear from the very beginning is that a processor, does not change an existing class source code. It only can write new source code files. This is important to better think about what kind of code will be generated, and keep in mind for example, scope access from automatically generated to existing code. Other interesting fact is that a processor recognizes annotations in compiled code as well as in plain source code.

In this example, I will try to recreate @BindView from Butterknife. The example is able to automatically bind all attributes in a Class object annotated with @BindView . The example generates code of the form v= getViewById(id) for each annotated field, and binds them just by calling Binder.Bind(this). E.g:

public class MainActivity extends AppCompatActivity {

    // public/package access so that binder object can set
    // value directly.
    // otherwise it should set the value reflectively.
    @BindView(id = R.id.text_view)
    TextView text;
    @BindView(id = R.id.edit_text_view)
    EditText text22;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.layout);

        // solve and bind annotations to views.
        Binder.Bind(this);

        text.setText("APT worked!!");
    }
}

To make this work, I created a gradle project with three different modules:

1. Annotations module, which contains the annotations that the processor will handle. Here also resides the example Binder object. This module is referenced by the application, and the processor modules.
2. Processor module. Here is where the annotations will be processed and new source code will be generated. This module needs a reference to the Annotations module for obvious reasons. Important to note is that the annotation processing is a whole java application, and runs in an independent JVM. Here you can include desired dependencies because ultimately, you are running a whole Java app. An application can have many different Annotation processors, each of which will run on its own JVM.
   One final thought about the example processor is that it mostly is aimed at annotated java source code, and not compiled classes.
3. Application module. This module references the Annotations module for runtime, but uses the Processor module for compilation and code generation purposes. The processor is used as an apt dependency.
   The application contains a couple classes with attributes annotated with @BindView for testing purposes.

Annotations module

This is a very simple module which defines just one annotation to be processed: @BindView . It can be applied only to class fields, and it will be retained on the class file (RetentionPolicy.CLASS), not needing to be available at runtime (RententionPolicy.RUNTIME). Here’s its definition:

@Retention(RetentionPolicy.CLASS)
@Target(ElementType.FIELD)
public @interface BindView {
    public int id();
}

The other object in this module is the Binder object. Which is invoked like Binder.Bind(this) to generate source code to bind all annotated view types with its view resource id. How it works is easy. For each object passed to Bind method, the Binder expects the annotation processor to have created a class of the form: object.getClass().getCanonicalName() + "$$Binder" . The Binder is something like:

public static final void Bind( Object obj ) {
    String str = obj.getClass().getCanonicalName();
    String clazz = str + "$$Binder";

    try {
        Class _c = Class.forName( clazz );
        IBinder binder = (IBinder)_c.newInstance();
        // call generated code
        binder.bind( obj );  
    } catch( Exception x ) {

        // no binder class exists.
    }
}

Generated $$Binder classes implement a bind method. This method is generated by the processor, and performs the binding for all @BindView annotated objects.

Annotation Processor

This is where things get actually really interesting. An annotation processor needs to implement javax.annotation.processing.Processor. Usually by extending javax.annotation.processing.AbstractProcessor.

The first thing to note in our processor’s code is the following annotation to the processor class itself:

@AutoService(Processor.class)
public class Processor extends AbstractProcessor {
   ...
}

This, is in fact another annotation processor directive. If needed, it creates a file: META-INF/services/javax.annotation.processing.Processor with our annotation processor full qualified class name inside. In our case: com.hyper.processor.Processor. This annotation must be added with a module dependency to com.google.auto.service:auto-service:1.0-rc2'.
Yes, it is an annotation processor invoked from our annotation processor. Isn’t it simply nice ?

To make things quick, we just need to declare what annotations the processor will act upon. Describe annotations’ fully qualified class names as strings:

public Set<String> getSupportedAnnotationTypes() {
    // we'll accept just one annotation.
    HashSet<String> ret = new HashSet<String>();
    ret.add("com.hyper.annotation.BindView");
    return ret;
}

tell what source version the processor accepts:

public SourceVersion getSupportedSourceVersion() {
    // accept latest version possible   
    return SourceVersion.latestSupported();
}

and override the process method, which is where the actual magic happens. The naive implementation for this example is:

This processor just expects fields annotated with @BindView. Keeps track of all the annotated fields per class, and then generates a file for each Class with annotated fields. For each Class, a file of the form Class.getCanonicalName() + “$$Binder” is created. This is a convention, so that at runtime, this class can be reflectively instantiated and all annotation-bound views solved.

The collection of identified annotated elements, with the specified annotations set defined in getSupportedAnnotationTypes are of type Element. Not Class. We are in the land of meta programming, building code blocks on the fly, and we get information directly either from the java compiler, or the java bytecode. There is an Element for each type of code annotated element. An element can be a class, attribute, interface, etc. In our example, we annotate fields, so VariableElement is the type of each annotated element passed in to the processor’s process method.

I created a helper class to deal with VariableElements, and to be able to recognize its class type, variable name, value, etc. The annotation is received as a javax.lang.model.element.AnnotationMirror object, so obtaining annotation values is easy, see the constructor of AnnotationInfoView:

Note that annotated element’s name, type, etc. is obtained from the VariableElement object as string types. From here, you could find the class, and reflect if needed.

The processor final piece, is to write the generated java files. I extended AbstractProcessor which upon a call to its init method, it saves a reference to a ProcessingEnvironment object. This object has environment specific object, like Messager which allows to print messages to the console while the processor is working, or Filer, which allows to create a JavaFileObject to which write the generated java code. Writing the generated java code is just a matter of creating a Writer like:

String fname =class_element + "$$Binder";
JavaFileObject jfo = processingEnv
    .getFiler()
    .createSourceFile(fname);
Writer writer = jfo.openWriter();

and write to it with a simple java Writer. There are wonderful tools to aid you in code generation.

Example project structure

The example is an Android Studio gradle project. For this to work properly, the Application gradle file must include the following on the buildScript/dependencies which will bring annotation processing tool plugin to Android Studio:

buildscript {
    repositories {
       ...
    }
    dependencies {
        classpath 'com.neenbedankt.gradle.plugins:android-apt:1.4'

The Application module (where the processor is going to execute), must include these in its gradle file:

apply plugin: 'com.android.application'
apply plugin: 'com.neenbedankt.android-apt'

android {
   ...
}

dependencies {
    compile fileTree(include: ['*.jar'], dir: 'libs')
    compile 'com.android.support:appcompat-v7:25.1.0'
    compile project(':annotation')  // include annotations module
    apt project(':processor')       // run apt
}

The processor module, must include the following in its dependencies:

dependencies {
    compile fileTree(include: ['*.jar'], dir: 'libs')
    compile 'com.google.auto.service:auto-service:1.0-rc2'
    compile project(':annotation')  // include annotations module
}

Example source code

There’s lots of pending stuff about annotation processing. This is a bare minimum example which does one simple thing, basic enough so to to expose processor core ideas, and have a minimum already setup project to tinker with: https://github.com/hyperandroid/apt-test