ANTLR 2.7.5 review

by on

ANTLR, ANother Tool for Language Recognition, (formerly PCCTS) is a language tool that provides a framework for constructing recogniz

License: Public Domain
File size: 885K
Developer: Terence Parr
0 stars award from

ANTLR, ANother Tool for Language Recognition, (formerly PCCTS) is a language tool that provides a framework for constructing recognizers, compilers, and translators from grammatical descriptions containing Java, C#, C++, or Python actions.

ANTLR provides excellent support for tree construction, tree walking, and translation. There are currently about 5,000 ANTLR source downloads a month.

What's New in This Release:

moved consume(); consumeUntil(...) to recover() method you can override in parsers only. New method signature: public void recover(BitSet tokenSet) throws TokenStreamException;
CommonASTWithHiddenTokens fixed tree builders copy hidden tokens
ASTFactory; added overloaded method public void setASTNodeClass(Class c);
default charVocabulary is 0..127 for ASCII range. ~'n', for example, gave really wacky results since charVocabulary had no defined range.
BaseAST.equals() didn't work when node text was null.
LLkParser.consume now throws TokenStreamException as Parser.consume does.

C++ changes:

Added TokenStreamRewriteEngine C++ port + example.
Fixes for sign extension problems that sneaked in 2.7.4.
Fixes for some quoting bugs in C++ codegeneration.
Fixes for some wrong extensions in generated docbook and HTML files (for -html/-docbook commandline options).
Made codegenerator more unicode friendly.
Tweaks for preliminary UTF8 support. Includes an example that is capable of parsing UTF8 input (more proof of concept than what is to be). (Using a custom InputBuffer and CharScanner). *Feedback on this is appreciated.*
Fixed codegeneration bug ('a'..'z'!) would trigger generation of invalid C++ code.
Added missing warning for redefinition of AST types for nodes. (Thanks to Sergey Bogdanov)
Bugfix for AST factory initialization code.
Fixed default argument mangling for generated rule methods.
Fixed sign extension problem in error messages in support lib (triggered by MSVC)
Fixed type generated for saveIndex and _begin variables. Thanks to Brad Schick for the patch.
getFilename/Line/Column methods added to TokenStreamRecognitionException (thanks to Heinz Beckers)
Added lexer states to the flex lexer example. It now seems to parse the same set of java as the other antlr java parsers.
Small tweaks to include files.
Doc tweaks.
Revised configure/makefiles by Wolfgang Haefelinger.

C# Changes

Changes for C# support (by Anthony Oguntimehin, Kunle Odutola & Micheal Jordan):

Factored out IToken and IHiddenStreamToken interfaces.
ASTPair now supports a protocol for reusing instances. Generated C# code creates far fewer ASTPair instances.
The values of LA(1) and LA(2) are now cached in lexers
ASTArray creation/usage replaced with C# variable length parameter lists. ASTArray remains for backwards compatibility only.
Several bugfixes in antlr.debug to better support ParseView-style tools [Suggested by Eric Taucher.]
Updated the C# code generator to generate code that uses use cached LA(x) values, reuses ASTPair instances, and uses the new IToken and IHiddenStreamToken interfaces
Re-organised the ANTLR runtime library for the .NET platform. Library is now packages as two assemblies:
antlr.runtime.dll - required, core library types and functionality
antlr.astframe.dll - optional, types and functionality for displaying astframes
The upshot of all this is that projects that don't need the ASTFrame functionality aren't forced to load the System.Drawing and System.Windows.Forms assemblies that it depends on.
C# VS.NET solution and project files & names have been updated to 2.7.5. Also new project file for the astframe assembly.
Updated the NAnt build file(s) to support building the new two-assembly runtime.
Incorporated latest fixes for the java grammar into the C# 'java' example project
Updated runtime documentation to accomodate the new two-assembly C# runtime.

ANTLR 2.7.5 keywords