{"id":228,"date":"2008-05-23T19:19:13","date_gmt":"2008-05-23T18:19:13","guid":{"rendered":"http:\/\/blog.sahits.ch\/?p=228"},"modified":"2017-12-24T21:42:28","modified_gmt":"2017-12-24T20:42:28","slug":"yaat-yet-another-ast-tutorial","status":"publish","type":"post","link":"http:\/\/sahits.ch\/blog\/blog\/2008\/05\/23\/yaat-yet-another-ast-tutorial\/","title":{"rendered":"YAAT &#8211; Yet another AST tutorial"},"content":{"rendered":"<p>Although there are several articles and tutorials on and about on how to use Eclipse AST (Abstract Syntax Tree), none was able to tell me what I needed to know. Therefore I decided to write this tutorial with accompanying source code to fill in the gaps.<br \/>\n<!--more--><br \/>\nFirst of some references to the other articles in case you need to know what they are on about:<\/p>\n<ul>\n<li><a name=\"link1><\/a><a href=\"http:\/\/www.eclipse.org\/articles\/article.php?file=Article-JavaCodeManipulation_AST\/index.html\">Abstract Syntax Tree<\/a> by Thomas Kuhn, Eye Media GmbH and Olivier Thomann, IBM Ottawa Lab (20 november 2006) at Eclipse article corner: This article describes the process of refactoring existing code using AST. Small portions of code are moved, created or deleted.<\/li>\n<li><a name=\"link2\"><\/a>Then there is a portion of how to <a href=\"http:\/\/help.eclipse.org\/help33\/index.jsp?topic=\/org.eclipse.jdt.doc.isv\/guide\/jdt_api_manip.htm\">manipulate Java code<\/a> in the Eclipse (3.3) help documentation<\/li>\n<li><a name=\"link3\"><\/a>Of course there is always the <a href=\"http:\/\/help.eclipse.org\/help33\/index.jsp?topic=\/org.eclipse.jdt.doc.isv\/reference\/api\/index.html\">API documentation<\/a> that can help you out.<\/li>\n<li>At the EclipseCon 2008 Martin Aeschliman from IBM Research Switzerland presented the <a href=\"http:\/\/www.eclipsecon.org\/2008\/sub\/attachments\/JDT_fundamentals.ppt\">JDT fundamentals<\/a> that give some additional hints on the AST.<\/li>\n<\/ul>\n<p>Now what is this all about? As in the <a href=\"#link1\">AST article<\/a> our code is in the form of a plugin. But this is not the main point though  we make the plugin a &#8222;Hello-World&#8220;-Plugin that contributes a button to the toolbar. On clicking this button a Java source file is generated with the main helloWorld method. This is done with specifying the necessary code in a string and generate a CompilationUnit from it.<br \/>\n(The compilation unit can also be gained by parsing an existing class, e.g. some class rump generated with JET, but that is not the focus of this article)<br \/>\nHaving generated the AST of a simple class we want add further methods to this class. There are two interessting ways to do that:<\/p>\n<ol>\n<li>Generate new AST nodes and add it the tree<\/li>\n<li>Copy the methods from another AST tree<\/li>\n<\/ol>\n<hr \/>\n<p>So let&#8217;s plunge in. This first listing shows how the compilation unit is produced.<br \/>\n<code><br \/>\n\t\tString helloWorld =\"\\n\"+<br \/>\n\t\t\t\"public class HelloWorld {\\n\"+<br \/>\n\t\t\t\"\\n\"+<br \/>\n\t\t\t\"\tprivate String name=\\\"\\\"\\n\\n\"+<br \/>\n\t\t\t\"\t\/**\\n\"+<br \/>\n\t\t\t\"\t * \\n\"+<br \/>\n\t\t\t\"\t *\/\\n\"+<br \/>\n\t\t\t\"\t public void sayHello() {\\n\"+<br \/>\n\t\t\t\"\t\tSystem.out.println(\\\"Hello \\\"+name+\\\"!\\\");\\n\"+<br \/>\n\t\t\t\"\t }\\n\"+<br \/>\n\t\t\t\"\\n\"+<br \/>\n\t\t\t\"}\";<br \/>\n\t\tASTParser parser = ASTParser.newParser(AST.JLS3); \/\/ Java 5.0 and up<br \/>\n\t\tparser.setKind(ASTParser.K_COMPILATION_UNIT);<br \/>\n\t\tparser.setSource(helloWorld.toCharArray());<br \/>\n\t\tparser.setResolveBindings(true);<br \/>\n\t\tparser.setBindingsRecovery(true);<br \/>\n\t\tunit = (CompilationUnit) parser.createAST(null \/* IProgrssMonitor*\/);<br \/>\n\t\tdocument=new Document(helloWorld);<br \/>\n<\/code><br \/>\nListing 1: from class ch.sahits.tut.ast.ClassGenerator<\/p>\n<p>Now that we have an Abstract Syntax Tree representation of the HelloWorld class we can start manipulate it. Therefore we add a method to set name. To do that we have to generate new node for the MethodDeclaration. To this node we add all the needed bits.<br \/>\nAs the name suggests this is quite abstract and my produce considerable brain pain. In such a case I find it extremly helpful to visualise the DOM tree I want to create. Here is the <a href=\"#link3\">API documentation<\/a>(look in the package org.eclipse.jdt.core.dom) a great help since if you know with which element to start you can find your way through the needed nodes which have all meaningful names.<br \/>\n<img decoding=\"async\" src=\"http:\/\/www.sahits.ch\/blog\/wp-content\/uploads\/2008\/05\/ast_setname.gif\" alt=\"AST for setName\"\/><br \/>\nThe next step is to add a method by coping an AST subtree. Normally you will be working on some project where you can access some class to get the Abstract Syntax subtree you want to copy. Since this article is somewhat simplified we will generate our AST we copy from like the AST in Listing 1. At the end of the article I will show how a class in a project can be accessed.<br \/>\nSo far this article tells you nothing new since the code is basically identical with the <a href=\"link2\">second Link<\/a>. Let us now take a look at coping a subtree. AST nodes cannot be re-parented, once connected to an AST, they cannot be attached to a different place of the tree. Though it is easy to create a copy from a subtree: <code>ASTNode ASTNode.copySubtree(AST target, ASTNode nodeToCopy)<\/code>. So the tricky part is to find the nodeToCopy in the source:<br \/>\n<code><br \/>\n\t\tCompilationUnit unit2 = (CompilationUnit) parser2.createAST(null \/* IProgrssMonitor*\/);<br \/>\n\t\tAST ast2=unit2.getAST();<br \/>\n\t\tASTRewrite astRewrite= ASTRewrite.create(ast);<br \/>\n\t\t\/\/ find the node to be copied<br \/>\n\t\tList<bodydeclaration> delcs =((TypeDeclaration)unit2.types().get(0)).bodyDeclarations();<br \/>\n\t\t\/\/ Add the node<br \/>\n\t\tMethodDeclaration method = null;<br \/>\n\t\tfor (Iterator iterator = delcs.iterator(); iterator.hasNext();) {<br \/>\n\t\t\tBodyDeclaration decl = (BodyDeclaration) iterator.next();<br \/>\n\t\t\tif (decl instanceof MethodDeclaration){<br \/>\n\t\t\t\tif (((MethodDeclaration)decl).getName().getIdentifier().equals(\"getName\")){<br \/>\n\t\t\t\t\tmethod = (MethodDeclaration) decl;<br \/>\n\t\t\t\t\tbreak;<br \/>\n\t\t\t\t}<br \/>\n\t\t\t}<br \/>\n\t\t}<br \/>\n<\/bodydeclaration><\/code><br \/>\nListing 2: from class ch.sahits.tut.ast.ClassGenerator<br \/>\nAnd now the saving back:<br \/>\n<code><br \/>\n\t\tMethodDeclaration copy = (MethodDeclaration) ASTNode.copySubtree(ast, method);<br \/>\n\t\tList<bodydeclaration> bodyDecls = ((TypeDeclaration)unit.types().get(0)).bodyDeclarations();<br \/>\n\t\tbodyDecls.add(copy);<br \/>\n\t\tListRewrite lrw = astRewrite.getListRewrite(((TypeDeclaration)unit.types().get(0)), TypeDeclaration.BODY_DECLARATIONS_PROPERTY);<br \/>\n\t\tlrw.insertLast(copy, null);<br \/>\n<code><br \/>\nListing 3: from class ch.sahits.tut.ast.ClassGenerator<br \/>\nIn our second case (the node created from scratch) the saving is identical from the second line. You substitute the <\/code><code>copy<\/code> with <code>method<\/code> since method was created on the same AST.<br \/>\nCreating a file from a compilation unit generated in such a way (not parsed from an existing source file) is easy:<br \/>\n<code><br \/>\n\t\tString newSource = document.get(); \/\/ created in the last line of Listing 1<br \/>\n\t\ttempFile=File.createTempFile(\"HelloWorld\", \"java\");<br \/>\n\t\tFileOutputStream fos = new FileOutputStream(tempFile);<br \/>\n\t\tfos.write(newSource.getBytes());<br \/>\n\t\tfos.flush();<br \/>\n<\/code><br \/>\nIf we had parsed from a class we would have had to undertake the following steps:<\/p>\n<ol>\n<li>We need an ASTRewriter: <code>ASTRewrite astRewrite= ASTRewrite.create(ast);<\/code><\/li>\n<li>Before the fist modification on the instance is done start the recording on the compilation unit: <code>unit.recordModifications();<\/code><\/li>\n<li>After the change write the change back:<code><br \/>\n\/\/ computation of the text edits<br \/>\n   TextEdit edits = astRewrite.rewriteAST(document,null \/* default Options *\/);<br \/>\n   \/\/ computation of the new source code<br \/>\n   edits.apply(document);<br \/>\n   String newSource = document.get();<br \/>\n   \/\/ update of the compilation unit<br \/>\n  ICompilationUnit cu = (ICompilationUnit)unit.getJavaElement();<br \/>\n   cu.getBuffer().setContents(newSource);<\/code>\n<\/li>\n<\/ol>\n<p>How to get at a source file existing in a project? Well first you have to retrieve the Java projects from the workspace, then select a project, go to the source folder and pass it down until you find your class:<br \/>\n<code><br \/>\n\tpublic File findSourceFile(Class className) throws JavaModelException{<br \/>\n\t\tIJavaProject[] projects= JavaCore.create(ResourcesPlugin.getWorkspace().getRoot()).getJavaProjects();<br \/>\n\t\tIPackageFragmentRoot[] srcs =projects[0].getAllPackageFragmentRoots();<br \/>\n\t\tString path = className.getName();<br \/>\n\t\tfor (int i = 0; i < srcs.length; i++) {\n\t\t\tif (srcs[i].getPackageFragment(path).exists()){\n\t\t\t\t\/\/ found\n\t\t\t\tIResource res= srcs[i].getPackageFragment(path).getResource();\n\t\t\t\treturn new File(res.getLocationURI());\n\t\t\t}\n\t\t}\n\t\treturn null;\n\t}\n<\/code><br \/>\nListing 4: from class ch.sahits.tut.ast.FindSource<br \/>\nYou can just as easy retrieve an instance of an ICompilationUnit that is needed to parse the AST. Just replace the return statement with the following expression:<\/code><code>srcs[i].getPackageFragment(path).getCompilationUnit(path.substring(path.lastIndexOf(\".\")+1)+\".java\");<\/code><\/p>\n<p>The source code to this article can be found at <a href=\"http:\/\/sourceforge.net\/projects\/ast-tutorial\">SourceForge<\/a>.<\/bodydeclaration><\/code><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Although there are several articles and tutorials on and about on how to use Eclipse AST (Abstract Syntax Tree), none was able to tell me what I needed to know. Therefore I decided to write this tutorial with accompanying source code to fill in the gaps.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,7,6],"tags":[91,92,304,301],"class_list":["post-228","post","type-post","status-publish","format-standard","hentry","category-eclipse","category-java","category-programmieren","tag-ast","tag-code-generation","tag-eclipse","tag-java"],"_links":{"self":[{"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/posts\/228","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/comments?post=228"}],"version-history":[{"count":1,"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/posts\/228\/revisions"}],"predecessor-version":[{"id":2573,"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/posts\/228\/revisions\/2573"}],"wp:attachment":[{"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/media?parent=228"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/categories?post=228"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/sahits.ch\/blog\/wp-json\/wp\/v2\/tags?post=228"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}