AI Parser Project

Needs --enable-native-access=ALL-UNNAMED

The AI Parser Project consists of several central components that work closely together to support the analysis and editing of Java code. The project comprises two main components: generating Java visitor classes using Python and implementing the generated classes in Java. The primary components of the Java implementation include the JavaLanguageServer, TreeSitter, and the GraphConstructor.

Key Components

Starting the Language Server

The process begins with starting the JavaLanguageServer, which is the main component of the system.

Setting Up the GraphConstructor

The GraphConstructor is set up to create and manage data structures that represent the Java code.

Initializing TreeSitter

TreeSitter is initialized to parse Java source code files and convert them into Abstract Syntax Trees (ASTs).

Parsing Files with TreeSitter

TreeSitter parses the Java files and creates ASTs, which can then be further analyzed and processed.

Outputting the Graph

Generate the IR and the graph-structure

Configuration File (node-types.json)

The node-types.json file defines the structure and properties of the AST nodes for the Java source files. It contains information about each node type, including its fields, children, and, if applicable, subtypes.

Python Script (generate_visitors.py)

The Python script generate_visitors.py reads the configuration file and automatically generates Java visitor classes for each defined AST node type. Here is a summary of the main functions of the script:

Reading the Configuration File

The JSON configuration file is read to understand the AST node types and their properties.

Generating Visitor Classes and Interfaces

For each AST node type, the following artifacts are generated:

• Visitor Interface: Defines an interface implemented by all visitors. It contains the visit method that describes how the node type should be processed.
• Visitor Class: Implements the visitor interface. It contains constructors for creating the visitor from an AST node and the visit method, which throws an UnsupportedOperationException.
• Object Class: A simple Java class that serves as an object for the AST node type.
• Interface for Object: An interface implemented by the object class.

Helper Class (VisitorHelperJava.java)

A generated class containing the createVisitorFromNode method. This method uses a switch statement based on the AST node type to instantiate the correct visitor.

The generated Java visitor classes and helper classes are stored in the directory src/com/cgi/itc/daas/parser/java/visitor/.

Java Implementation

Here are the main components:

JavaLanguageServer

Orchestrates the functionality of the language server and provides an interface for interacting with the editor. Key functions include:

• Initialization and Configuration: The server is initialized with the project parameters to start processing Java source files.
• Communication via LSP: The server communicates with the editor via the Language Server Protocol to provide editor functions such as code completion, error diagnosis, and navigation.
• Integration with TreeSitter: The JavaLanguageServer uses TreeSitter for the syntactic analysis of Java code. It uses JavaTreeSitter to parse Java files into an AST and pass the data to the GraphConstructor.

JavaTreeSitter.java

The main class that coordinates the processing of Java source files using TreeSitter and the generated visitor classes. TreeSitter is the central syntactic analysis library used in the project. The main functions include:

• Parsing Java Source Files: TreeSitterJava provides the Java-specific grammar and converts Java code into an AST.
• AST Navigation and Analysis: The AST is used to navigate through the Java code, identify specific code elements, and extract relevant information.
• Collaboration with Visitor Classes: TreeSitter works with generated visitor classes to perform specific actions on the AST nodes. These visitor classes are used to collect metrics, create diagrams, or conduct specific code analyses.

Visitor Classes and Interfaces (Examples)

• JavaNodeVisitor.java, JavaNodeVisitorIfc.java: Generated visitor class and interface for a specific AST node type.
• JavaNodeObj.java, JavaNodeObjIfc.java: Generated object class and interface for a specific AST node type.

GraphConstructor

Creates a graph based on the information provided by TreeSitter. It uses nodes and edges to represent relationships and structures in the source code. The toJsonWrapper() method in the GraphConstructor Class is designed to convert the internal data structures of the graph, specifically nodesById and edges, into a JSON format. It then writes this JSON structure into a file named out.json.

This method essentially serves as a bridge between the internal representation of the graph and a JSON file, allowing the graph data to be stored, and to generate the IR (3.1.1.1 Intermediate Representation (IR))