Why am I getting a NullPointerException when I import lexers in Antlr?

I am using antlr 3 and Antlrworks. Here is my setup:

lexer Base //contains basic tokens - like WS, number etc.

lexer Specific //contains my language specific tokens - AND derives from Base lexer

parse开发者_如何学Pythonr specific //parser for my language

combined grammer -> imports specific lexer and specific parser

When I generate, I always get a NPE (in Java). The reason is that the reference to the Base lexer in the generated specific lexer is not initialized.

Am I missing something?

Impossible to tell without seeing how you're importing the grammars.

Bear in mind that:

• lexer grammars can only import other lexer grammars;
• parser grammars can only import other parser grammars;
• tree grammars can only import other tree grammars;
• combined grammars can import lexer- and parser grammars (but not other combined grammars!).

In your case, that would look like:

BaseLexer.g

lexer grammar BaseLexer;

Num   : '0'..'9'+;
Space : ' ' | '\t';

SpecificLexer.g

lexer grammar SpecificLexer;

import BaseLexer;

SpecificTokenA : 'specificA';
SpecificTokenB : 'specificB';

SpecificParser.g

parser grammar SpecificParser;

specific :  SpecificTokenA |  SpecificTokenB;

Combined.g

grammar Combined;

import SpecificLexer, SpecificParser;

parse 
  :  Num Space specific EOF 
     {
       System.out.println("Parsed:\n  Num      = " + 
           $Num.text + "\n  specific = " + $specific.text);
     } 
  ;

and to test it all, use the class:

import org.antlr.runtime.*;

public class Main {
  public static void main(String[] args) throws Exception {
    SpecificLexer lexer = new SpecificLexer(new ANTLRStringStream("42 specificB"));
    CombinedParser parser = new CombinedParser(new CommonTokenStream(lexer));
    parser.parse();
  }
}

Now generate the lexers and parsers and run the Main class:

java -cp antlr-3.3.jar org.antlr.Tool BaseLexer.g
java -cp antlr-3.3.jar org.antlr.Tool SpecificLexer.g
java -cp antlr-3.3.jar org.antlr.Tool SpecificParser.g
java -cp antlr-3.3.jar org.antlr.Tool Combined.g
javac -cp antlr-3.3.jar *.java
java -cp .:antlr-3.3.jar Main

which will print the following to your console:

Parsed:
  Num      = 42
  specific = specificB

Also tested with ANTLRWorks 1.4.3.

I have encountered the same problem using antlr 3.2. Upgrading is not straightforward, because I'm faced with OutOfMemory issues.

I tried different solution, e.g. direct import of both lexer grammars in the parser grammar, without success. In the end I had to copy the Base lexer grammar in every Specific lexer grammar.