YAAT – Yet another AST tutorial

Although there are several articles and tutorials on and about on how to use Eclipse AST (Abstract Syntax Tree), none was able to tell me what I needed to know. Therefore I decided to write this tutorial with accompanying source code to fill in the gaps.

First of some references to the other articles in case you need to know what they are on about:

• Abstract Syntax Tree by Thomas Kuhn, Eye Media GmbH and Olivier Thomann, IBM Ottawa Lab (20 november 2006) at Eclipse article corner: This article describes the process of refactoring existing code using AST. Small portions of code are moved, created or deleted.
• Then there is a portion of how to manipulate Java code in the Eclipse (3.3) help documentation
• Of course there is always the API documentation that can help you out.
• At the EclipseCon 2008 Martin Aeschliman from IBM Research Switzerland presented the JDT fundamentals that give some additional hints on the AST.

Now what is this all about? As in the AST article our code is in the form of a plugin. But this is not the main point though we make the plugin a „Hello-World“-Plugin that contributes a button to the toolbar. On clicking this button a Java source file is generated with the main helloWorld method. This is done with specifying the necessary code in a string and generate a CompilationUnit from it.
(The compilation unit can also be gained by parsing an existing class, e.g. some class rump generated with JET, but that is not the focus of this article)
Having generated the AST of a simple class we want add further methods to this class. There are two interessting ways to do that:

1. Generate new AST nodes and add it the tree
2. Copy the methods from another AST tree

So let’s plunge in. This first listing shows how the compilation unit is produced.

String helloWorld ="\n"+
"public class HelloWorld {\n"+
"\n"+
" private String name=\"\"\n\n"+
" /**\n"+
" * \n"+
" */\n"+
" public void sayHello() {\n"+
" System.out.println(\"Hello \"+name+\"!\");\n"+
" }\n"+
"\n"+
"}";
ASTParser parser = ASTParser.newParser(AST.JLS3); // Java 5.0 and up
parser.setKind(ASTParser.K_COMPILATION_UNIT);
parser.setSource(helloWorld.toCharArray());
parser.setResolveBindings(true);
parser.setBindingsRecovery(true);
unit = (CompilationUnit) parser.createAST(null /* IProgrssMonitor*/);
document=new Document(helloWorld);

Listing 1: from class ch.sahits.tut.ast.ClassGenerator

Now that we have an Abstract Syntax Tree representation of the HelloWorld class we can start manipulate it. Therefore we add a method to set name. To do that we have to generate new node for the MethodDeclaration. To this node we add all the needed bits.
As the name suggests this is quite abstract and my produce considerable brain pain. In such a case I find it extremly helpful to visualise the DOM tree I want to create. Here is the API documentation(look in the package org.eclipse.jdt.core.dom) a great help since if you know with which element to start you can find your way through the needed nodes which have all meaningful names.

The next step is to add a method by coping an AST subtree. Normally you will be working on some project where you can access some class to get the Abstract Syntax subtree you want to copy. Since this article is somewhat simplified we will generate our AST we copy from like the AST in Listing 1. At the end of the article I will show how a class in a project can be accessed.
So far this article tells you nothing new since the code is basically identical with the second Link. Let us now take a look at coping a subtree. AST nodes cannot be re-parented, once connected to an AST, they cannot be attached to a different place of the tree. Though it is easy to create a copy from a subtree: ASTNode ASTNode.copySubtree(AST target, ASTNode nodeToCopy). So the tricky part is to find the nodeToCopy in the source:

CompilationUnit unit2 = (CompilationUnit) parser2.createAST(null /* IProgrssMonitor*/);
AST ast2=unit2.getAST();
ASTRewrite astRewrite= ASTRewrite.create(ast);
// find the node to be copied
List delcs =((TypeDeclaration)unit2.types().get(0)).bodyDeclarations();
// Add the node
MethodDeclaration method = null;
for (Iterator iterator = delcs.iterator(); iterator.hasNext();) {
BodyDeclaration decl = (BodyDeclaration) iterator.next();
if (decl instanceof MethodDeclaration){
if (((MethodDeclaration)decl).getName().getIdentifier().equals("getName")){
method = (MethodDeclaration) decl;
break;
}
}
}

Listing 2: from class ch.sahits.tut.ast.ClassGenerator
And now the saving back:

MethodDeclaration copy = (MethodDeclaration) ASTNode.copySubtree(ast, method);
List bodyDecls = ((TypeDeclaration)unit.types().get(0)).bodyDeclarations();
bodyDecls.add(copy);
ListRewrite lrw = astRewrite.getListRewrite(((TypeDeclaration)unit.types().get(0)), TypeDeclaration.BODY_DECLARATIONS_PROPERTY);
lrw.insertLast(copy, null);

Listing 3: from class ch.sahits.tut.ast.ClassGenerator
In our second case (the node created from scratch) the saving is identical from the second line. You substitute the copy with method since method was created on the same AST.
Creating a file from a compilation unit generated in such a way (not parsed from an existing source file) is easy:

String newSource = document.get(); // created in the last line of Listing 1
tempFile=File.createTempFile("HelloWorld", "java");
FileOutputStream fos = new FileOutputStream(tempFile);
fos.write(newSource.getBytes());
fos.flush();

If we had parsed from a class we would have had to undertake the following steps:

How to get at a source file existing in a project? Well first you have to retrieve the Java projects from the workspace, then select a project, go to the source folder and pass it down until you find your class:

public File findSourceFile(Class className) throws JavaModelException{
IJavaProject[] projects= JavaCore.create(ResourcesPlugin.getWorkspace().getRoot()).getJavaProjects();
IPackageFragmentRoot[] srcs =projects[0].getAllPackageFragmentRoots();
String path = className.getName();
for (int i = 0; i < srcs.length; i++) { if (srcs[i].getPackageFragment(path).exists()){ // found IResource res= srcs[i].getPackageFragment(path).getResource(); return new File(res.getLocationURI()); } } return null; }
Listing 4: from class ch.sahits.tut.ast.FindSource
You can just as easy retrieve an instance of an ICompilationUnit that is needed to parse the AST. Just replace the return statement with the following expression:srcs[i].getPackageFragment(path).getCompilationUnit(path.substring(path.lastIndexOf(".")+1)+".java");

The source code to this article can be found at SourceForge.