ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.

Last update: Jan 05, 2023

Overview

ANTLR v4

Build status

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It's widely used to build languages, tools, and frameworks. From a grammar, ANTLR generates a parser that can build parse trees and also generates a listener interface (or visitor) that makes it easy to respond to the recognition of phrases of interest.

Authors and major contributors

Terence Parr, [email protected] ANTLR project lead and supreme dictator for life University of San Francisco
Sam Harwell (Tool co-author, Java and original C# target)
Eric Vergnaud (Javascript, Python2, Python3 targets and maintenance of C# target)
Peter Boyer (Go target)
Mike Lischke (C++ completed target)
Dan McLaughlin (C++ initial target)
David Sisson (C++ initial target and test)
Janyou (Swift target)
Ewan Mellor, Hanzhou Shi (Swift target merging)
Ben Hamilton (Full Unicode support in serialized ATN and all languages' runtimes for code points > U+FFFF)
Marcos Passos (PHP target)
Lingyu Li (Dart target)

Useful information

Release notes
Getting started with v4
Official site
Documentation
FAQ
ANTLR code generation targets
(Currently: Java, C#, Python2|3, JavaScript, Go, C++, Swift, Dart, PHP)
Java API
ANTLR v3
v3 to v4 Migration, differences

You might also find the following pages useful, particularly if you want to mess around with the various target languages.

The Definitive ANTLR 4 Reference

Programmers run into parsing problems all the time. Whether it’s a data format like JSON, a network protocol like SMTP, a server configuration file for Apache, a PostScript/PDF file, or a simple spreadsheet macro language—ANTLR v4 and this book will demystify the process. ANTLR v4 has been rewritten from scratch to make it easier than ever to build parsers and the language applications built on top. This completely rewritten new edition of the bestselling Definitive ANTLR Reference shows you how to take advantage of these new features.

You can buy the book The Definitive ANTLR 4 Reference at amazon or an electronic version at the publisher's site.

You will find the Book source code useful.

Additional grammars

This repository is a collection of grammars without actions where the root directory name is the all-lowercase name of the language parsed by the grammar. For example, java, cpp, csharp, c, etc...

Comments

$New extended Unicode escape \u{10ABCD} to support Unicode literals > U+FFFF$

New extended Unicode escape \u{10ABCD} to support Unicode literals > U+FFFF

Fixes #276 .

This used to be a WIP PR, but it's now ready for review.

This PR introduces a new extended Unicode escape \u{10ABCD} in ANTLR4 grammars to support Unicode literal values > U+FFFF.

The serialized ATN represents any atom or range with a Unicode value > U+FFFF as a set. Any such set is serialized in the ATN with 32-bit arguments.

I bumped the UUID, since this changes the serialized ATN format.

I included lots of tests and made sure everything is passing on Linux, Mac, and Windows.
type:feature unicode

opened by bhamiltoncx 115
Terrible Golang performance
Stackoverflow: https://stackoverflow.com/questions/72266899/golang-performance-issues

Google group: https://groups.google.com/g/antlr-discussion/c/OdhAIsy2GfI

Example code: https://github.com/movelazar/perf-repro

A simple rule such as:

1 EQ 2 OR 1 EQ 2 OR 1 EQ 2 OR 1 EQ 2 OR 1 EQ 2

takes exponentially longer to parse the more 1 EQ 2 OR clauses there are. This does not happen in python (by my testing) or CSharp, Dart, Java (by stackoverflow comment).

On my machine, # of lines vs parse time:

11: 0.5s 12: 1.2s 13: 3.2s 14: 8.1s 15: 21.9s 16: 57.5s

Given that Python doesn't face this issue I can't imagine I'm doing something terrible in my grammar.

Issue goes away if I put parens on things but that's not a real solution.

On 4.10.1, first noticed with 4.9.1.

Any help is greatly appreciated. Surprised I can't find others with this issue.
type:bug target:go comp:performance
opened by movelazar 101
splitting version numbers for targets
Hiya: @pboyer, @mike-lischke, @janyou, @ewanmellor, @hanjoes, @ericvergnaud, @lingyv-li, @marcospassos

Eric has raised the point that it would be nice to be able to make quick patches to the various runtimes; e.g., there is a stopping bug now in the JavaScript target. He proposes something along these lines:

any change in the tool or the runtime algorithm bumps the middle version #: 4.9 -> 4.10 -> 4.11

any bug fix in a runtime we bump the last digit of that runtime only: 4.9 -> 4.9.1 -> 4.9.2

if bumping the java runtime for bug fix we also bump the tool since it contains the runtime

This is in optimal as people have criticized me in the past for bumping, say, 4.6 to 4.7 for some minor changes. It also has the problem that 4.9.x will not mean the same thing in two different targets possibly, as each target will now have their own version number.

Rather than break up all of the targets into separate repositories or similar, can you guys think of a better solution? Any suggestions? The goal here is to allow more rapid target releases, and independent of me having to do a major release of the tool.
type:question
opened by parrt 94
Improve memory usage and perf of CodePointCharStream: Use 8-bit, 16-bit, or 32-bit buffer

This greatly improves the memory usage and performance of CodePointCharStream by ensuring the internal storage uses either an 8-bit buffer (for Unicode code points <= U+00FF), 16-bit buffer (for Unicode code points <= U+FFFF), or a 32-bit buffer (Unicode code points > U+FFFF).

I split out the internal storage into a class CodePointBuffer which has a CodePointBuffer.Builder class which has the logic to upgrade from 8-bit to 16-bit to 32-bit storage.

I found the perf hotspot in CodePointCharStream on master was the virtual method calls from CharStream.LA(offset) into IntBuffer.

Refactoring it into CodePointBuffer didn't help (in fact, it added another virtual method call).

To fix the perf, I made CodePointCharStream an abstract class and made three concrete subclasses: CodePoint8BitCharStream, CodePoint16BitCharStream, and CodePoint32BitCharStream which directly access the array of underlying code points in the CodePointBuffer without virtual method calls.
lexers target:java comp:performance

opened by bhamiltoncx 85
initial discussion to start integration of new targets
As promised, I am now ready to integrate the new ANTLR target languages you folks have been working on. This issue is meant to get everybody in sync, check status, and discuss the proper order of integration and resolve issues etc.

There are two administrative details to get out of the way first:

Please let me know if there is another github user that should be added to one of the categories. Or, of course, if you would like your user ID removed from this discussion.

Nothing can be merged into antlr/antlr4 unless every single committer has added themselves to the contributors.txt file. It's onerous, particularly for simple commits, but it is requirement for anything merged into the master. Eclipse foundation lawyers tell me that we have one of the cleanest licenses out there and it contributes to ANTLR's widespread use because companies are not afraid to use the software. See the genesis of such heinous requirements in SCO v IBM. This means lead target authors have to go back through their committers list quickly and ask them to sign the contributors file with a new commit. Or, they can remove that commit and enter their own version of the functionality, being careful not to violate copyright on the previous.

As we proceed, please keep in mind that I have a difficult role, balancing the needs of multiple targets and keeping discussions in the civil and practical zone. Decisions I make come from the perspective of over 25 years managing and leading this project. I look forward to incorporating your hard work into the main antlr repo.

C++ current location

@mike-lischke

@DanMcLaughlin

@nburles

@davesisson

Go current location, previous discussion

@pboyer

Swift current location: unclear, previous discussion

@jeffreyguenther

@hanjoes

@janyou

@ewanmellor

Likely interested/supporting humans (scraped from github issues):

@RYDB3RG

@wjkohnen

@willfaught

@parrt

@sharwell

@ericvergnaud

type:improvement target:swift target:cpp target:go
opened by parrt 84
Add a new CharStream that converts the symbols to upper or lower case.
This is useful for many of the case insensitive grammars found at https://github.com/antlr/grammars-v4/ which assume the input would be all upper or lower case. Related discussion can be found at https://github.com/antlr/antlr4test-maven-plugin/issues/1

It would be used like so:

input, _ := antlr.NewFileStream("filename") in = antlr.NewCaseChangingStream(is, true) // true forces upper case symbols, false forces lower case. lexer := parser.NewAbnfLexer(in)

While writing this, I found other people have written their own similar implementations (go, java). It makes sense to place this in the core, so everyone can use it.

I would love for the grammar to have a option that says the lexer should upper/lower case all input, and then this code could be moved into the generated Lexer, and no user would need to explicitly use a CaseChangingStream (similar to what's discussed in #1002).
lexers comp:runtime target:java target:javascript target:go
opened by bramp 69
Swift Target

I did a quick search and I didn't see anything written about this yet. What's the likelihood of a Swift target for ANTLR?

There are C#, Javascript, and Python targets at the moment.

What does it take to implement a target? Given that Swift is more Java-like, it seems like it should be possible. Maybe start with a code translator if there is one for (Java to Swift), and iterate towards a more idiomatic implementation.
type:question

opened by jeffreyguenther 69
Clean up ATN serialization: rm UUID and shifting by value of 2
I think we don't need the UUID in the serialization, since it has not changed in a decade. We can bump the version number and remove the UU ID

I did some tests and there seems to be no reason to shift the values in the serialized ATN by 2 for the purposes of improving the UTF-8 encoding for the Java target.

If you guys agree, we can make this small change for cleanup purposes. I'm happy to do it if you guys don't want to. The second fix will require changes to each target but it's trivial to fix.
atn-analysis type:cleanup
opened by parrt 68
Preparing for 4.9.3 release

It's that time of year again! @pboyer, @mike-lischke, @janyou, @ewanmellor, @hanjoes, @ericvergnaud, @lingyv-li, @marcospassos Shall we do a 4.9.3 release?

I went through and marked all of the merged PRs and related issues with 4.9.3 and try to tag them according to their target. Would you guys like to go through the PRs to see if there's something that should be merged quickly?

opened by parrt 68
[CSharp] #2021 fixes nuget packaging options to avoid missing dll exceptions

@ericvergnaud Hi, I modified csproj options a bit, now I can get a working nuget package locally without the issue we described in #2021. I added .net 3.5 as a target to "main" csproj along with netstandard, since it's easier to keep track of requirements for both sets of api's when editing code and, ideally, both targets can be packed into a nuget package with a single command. Right now it's possible only on Windows via msbuild /t:pack or Visual Studio; unfortunately, due to https://github.com/Microsoft/msbuild/issues/1333, right now dotnet build pack does not work for .net 3.5 target the way it should, so I adjusted the existing script to create packages from .nuspec and different solutions for different targets.
comp:build target:csharp

opened by listerenko 68
A few updates to the Unicode documentation.

It should be made clear that the recommended use of CharStreams.fromPath() is a Java-only solution. The other targets just have their ANTLRInputStream class extended to support full Unicode.
comp:doc

opened by mike-lischke 61
Assorted problems in calling the wrong wrapper for reachesIntoOuterContext
This is a serious bug in the CSharp runtime, ParserATNSimulator.cs.

reachesIntoOuterContext is an integer that tracks the depth of how far we dip into the outer context. This field must be interpreted with the bit map mask SUPPRESS_PRECEDENCE_FILTER, and careful attention placed on whether to access the field raw or with the bit map mask.

In Java, this code accesses the field raw.

In CSharp, the field is accessed through a method that applies the bit map mask. This is a serious problem in that it causes ATN parser trace divergence.

I did a cursory check on the other targets, and I am not confident the field access is done correctly across targets.

The fix is the change ParserATNInterpreter.cs:

diff --git a/runtime/CSharp/src/Atn/ParserATNSimulator.cs b/runtime/CSharp/src/Atn/ParserATNSimulator.cs index 4a7a6a3d5..8463bfcc3 100644 --- a/runtime/CSharp/src/Atn/ParserATNSimulator.cs +++ b/runtime/CSharp/src/Atn/ParserATNSimulator.cs @@ -1579,7 +1579,7 @@ namespace Antlr4.Runtime.Atn // This assignment also propagates the // isPrecedenceFilterSuppressed() value to the new // configuration. - c.reachesIntoOuterContext = config.OuterContextDepth; + c.reachesIntoOuterContext = config.reachesIntoOuterContext; ClosureCheckingStopState(c, configSet, closureBusy, collectPredicates, fullCtx, depth - 1, treatEofAsEpsilon); }

(Note, the source code should really not be using tabs. Tab expansion is editor specific.)

atn-analysis type:bug target:csharp
opened by kaby76 3

Python runtime test failure with 4.10 and later

After upgrading the antlr4 python runtime past 4.9.1 (tested 4.10.1 and 4.11.1) in nixpkgs, we've been seeing its test suite fail with

Traceback (most recent call last):
  File "/build/source/runtime/Python3/tests/ctest.py", line 10, in <module>
    from parser.cparser import CParser
  File "/build/source/runtime/Python3/tests/parser/cparser.py", line 631, in <module>
    class CParser ( Parser ):
  File "/build/source/runtime/Python3/tests/parser/cparser.py", line 635, in CParser
    atn = ATNDeserializer().deserialize(serializedATN())
  File "/nix/store/ch8i929c63av55h9nxkinifh61mazf1h-python3.10-antlr4-python3-runtime-4.11.1/lib/python3.10/site-packages/antlr4/atn/ATNDeserializer.py", line 28, in deserialize
    self.checkVersion()
  File "/nix/store/ch8i929c63av55h9nxkinifh61mazf1h-python3.10-antlr4-python3-runtime-4.11.1/lib/python3.10/site-packages/antlr4/atn/ATNDeserializer.py", line 50, in checkVersion
    raise Exception("Could not deserialize ATN with version " + str(version) + " (expected " + str(SERIALIZED_VERSION) + ").")
Exception: Could not deserialize ATN with version  (expected 4).

The version returned seems to be \x03, while the test suite expects it to be 4.

While similar to #3997 and #3895 we are running the test suite, not our own code.

We're running python ctest.py from the tests directory.

opened by mweinelt 4

Suggestion for the Test Rig GUI tool

This is not a defect or reporting a problem. It is a suggestion that the GUI tool have a new button added between "OK" and "Export as PNG". That can be named "Reload".

The idea being that I can edit/save a test source file and just click "reload" rather than doing what I do which is close the tool and go back to my command window and rerun the batch file that starts the GUI.

Thoughts?

opened by Korporal 0

invalid syntax `self.from = None` in python3 generated parsers

Please include the following information

python3 antlr4 version is 4.9.3

smallest possible grammar and code that reproduces the behavior

import sys
import antlr4
from PrestoSqlLexer import PrestoSqlLexer
from PrestoSqlParser import PrestoSqlParser


def main():
    input_stream = antlr4.CharStream('select 1')
    lexer = PrestoSqlLexer(input_stream)
    tokens = antlr4.CommonTokenStream(lexer)
    tokens.fill()
    print([token.text for token in tokens.tokens][:-1])


if __name__ == '__main__':
    main()

description of the expected behavior and actual behavior Pointers to suspicious code regions are also very welcome.

error:

Traceback (most recent call last):
  File "/Users/clients/autocomplete/parseQuery.py", line 4, in <module>
    from PrestoSqlParser import PrestoSqlParser
  File "/Users/clients/autocomplete/PrestoSqlParser.py", line 1837
    self.from = None # QualifiedNameContext
         ^^^^
SyntaxError: invalid syntax

opened by chengchengpei 2

Releases(4.11.1)

4.11.1(Sep 4, 2022)

Just fixes 4.11.0 release issue. I forgot to change runtime tool version so it didn't say SNAPSHOT.
Source code(tar.gz)
Source code(zip)
4.11.0(Sep 4, 2022)
4.11.0 consist primarily:

a major rebuild of the Go target by Jim Idle. Please see Go target doc for more information, as it now requires the go module stuff.

huge internal refractoring of testing and various target speed improvements by Ivan Kochurkin

Getting all targets to run on all operating systems using github actions by HS.

The details by type and target follow.

Issues fixed

Disable failing CI tests in master (comp:build, comp:testing)

Create accessor for Go-based IntervalSet.intervals (target:go)

Grammar Name Conflict Golang with SPARQL issue (target:go, type:cleanup)

Dependency declaration error in ANTLR 4.10.1 (comp:build)

Drop old version of Visual Studio C++ (2013, 2015, 2017) (comp:build, target:cpp)

Circular grammar inclusion causes stack overflow in the tool. (comp:tool, type:bug)

Cpp, Go, JavaScript, Python2/3: Template rendering error. (code-gen, comp:runtime, target:java, target:javascript, target:python2, target:python3, target:go)

Improvements, features

Augment error message during testing to include full cause of problem. (type:improvement, comp:testing)

Include swift & tool verification in CI workflow (type:improvement, comp:build, cross-platform-issue, target:swift)

Issue #3783: CI Check Builds (type:improvement, comp:build, cross-platform-issue, comp:testing)

Parallel lock free testing, remove potential deadlocks, cache static data, go to descriptor via test (comp:runtime, type:improvement, comp:testing)

update getting-started doc (type:improvement, comp:doc)

Getting Started has error (type:improvement, comp:doc)

new nuget directory for building ANTLR4 C++ runtime as 3 Nuget packages (type:improvement, comp:build, target:cpp)

Add interp tool like TestRig (comp:tool, type:feature)

Issue 3720: Java 2 Security issue (type:improvement, target:java)

Cpp: Disable warnings for external project (type:bug, type:improvement, target:cpp)

Fix Docker README for arm OS user (type:improvement, comp:doc)

Pull requests grouped by target

csharp target

Runtime tests fixes (C++, Python, PHP, C#) (target:python2, target:python3, target:csharp, target:cpp, comp:testing, target:php)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

cpp target

Fix 2016 round 2 (type:bug, target:java, target:javascript, target:python2, target:python3, target:swift, target:cpp, target:go)

Cpp: Link to threads library (comp:build, target:cpp)

[C++] Remove libuuid dependency (comp:build, target:cpp, type:cleanup)

Fix CMake syntax for variable expansion (comp:build, cross-platform-issue, target:cpp)

Don't make class SementicContext::Empty final. (cross-platform-issue, target:cpp)

new nuget directory for building ANTLR4 C++ runtime as 3 Nuget packages (type:improvement, comp:build, target:cpp)

Runtime tests fixes (C++, Python, PHP, C#) (target:python2, target:python3, target:csharp, target:cpp, comp:testing, target:php)

Cpp: Disable warnings for external project (type:bug, type:improvement, target:cpp)

FindANTLR: Fix version RegEx (type:bug, comp:build, target:cpp)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

Improve ExternalProject cmake file (actions, comp:build, target:cpp)

Windows specific fixes (comp:build, cross-platform-issue, target:cpp)

go target

feat: Add a deprecation message to the existing v1 module (target:go, type:cleanup)

Feature/gomod4110 (target:go, type:cleanup)

Optimize go tests runner and renable CircleCI go tests (comp:build, target:go, comp:testing)

fix: #2016 Fix Go template list reference, go runtime and got test te… (target:go, type:cleanup)

Fix 2016 round 2 (type:bug, target:java, target:javascript, target:python2, target:python3, target:swift, target:cpp, target:go)

doc: Start improving docs for #1374 (comp:doc, target:go, type:cleanup)

fix: #2016 - Generate correct iGo code for lists in a grammar, such a… (grammars, target:go)

fix: #2826 Go template is incorrect for dynamic scopes (type:bug, target:go)

fix: #3758 Allow for string being a keyword and fix go template to us… (target:go, type:cleanup)

feat: Reduce initial memory allocations for collections (parsers, target:go)

Feature/newhash (target:go, comp:performance)

fix: Fixes for https://github.com/antlr/antlr4/issues/3718 (type:bug, target:go, type:cleanup)

Add Go accessor for IntervalSet.intervals (target:go)

Ensure that only the tokens needed are fetched (target:java, target:go, comp:performance)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

java target

Fix 2016 round 2 (type:bug, target:java, target:javascript, target:python2, target:python3, target:swift, target:cpp, target:go)

Add @SuppressWarnings("CheckReturnValue") to prevent error_prone lib errors. (target:java)

Fixing bug identified by https://github.com/antlr/antlr4/discussions/… (target:java, type:cleanup)

Ensure that only the tokens needed are fetched (target:java, target:go, comp:performance)

Issue 3720: Java 2 Security issue (type:improvement, target:java)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

Fix OSGi imports (comp:build, target:java)

javascript target

Fix 2016 round 2 (type:bug, target:java, target:javascript, target:python2, target:python3, target:swift, target:cpp, target:go)

add license prefix (target:javascript)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

python2 target

Fix 2016 round 2 (type:bug, target:java, target:javascript, target:python2, target:python3, target:swift, target:cpp, target:go)

[py2] Fix SyntaxError due to F string (type:bug, target:python2)

[Python 3] use build and twine to publish source and wheel (comp:build, target:python2, target:python3)

Fix #3508: Document the $parser attribute and its use in target-agnostic grammars. (target:python2, target:python3, comp:doc)

Runtime tests fixes (C++, Python, PHP, C#) (target:python2, target:python3, target:csharp, target:cpp, comp:testing, target:php)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

Restore support for py3.5 by removing f-string in setup.py (target:python2)

python3 target

Fix 2016 round 2 (type:bug, target:java, target:javascript, target:python2, target:python3, target:swift, target:cpp, target:go)

[Python 3] use build and twine to publish source and wheel (comp:build, target:python2, target:python3)

Fix #3508: Document the $parser attribute and its use in target-agnostic grammars. (target:python2, target:python3, comp:doc)

Runtime tests fixes (C++, Python, PHP, C#) (target:python2, target:python3, target:csharp, target:cpp, comp:testing, target:php)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

swift target

Fix 2016 round 2 (type:bug, target:java, target:javascript, target:python2, target:python3, target:swift, target:cpp, target:go)

Include swift & tool verification in CI workflow (type:improvement, comp:build, cross-platform-issue, target:swift)

Swift runtime tests run on Windows! (comp:build, target:swift, comp:testing)

[Swift] let the Swift Package Manager choose between static or dynamic linking (comp:build, target:swift)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

Update Swift Package to support static library (comp:build, target:swift)

php target

Runtime tests fixes (C++, Python, PHP, C#) (target:python2, target:python3, target:csharp, target:cpp, comp:testing, target:php)

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

dart target

Constant folding for Python and other runtimes (code-gen, target:java, target:javascript, target:python2, target:python3, target:csharp, target:swift, target:cpp, target:go, target:dart, comp:performance, target:php)

Contributors

11 items: Jim Idle (jimidle)

10 items: Terence Parr (parrt)

10 items: Ivan Kochurkin (KvanTTT)

6 items: HS (hs-apotell)

4 items: Robert Adam (Krzmbrzl)

2 items: James Taylor (JamesRTaylor)

2 items: Nicksxs (Nicksxs)

1 items: Tim McCormack (timmc)

1 items: Travis Thieman (thieman)

1 items: Qijia Liu (eagleoflqj)

1 items: Ross Patterson (RossPatterson)

1 items: ericvergnaud

1 items: Bryan Tan (Technius)

1 items: Henner Zeller (hzeller)

1 items: GCer-Hidenori

1 items: Eduardo Hernández Valdez (saucesaft)

1 items: 100mango

1 items: Mark Symons (msymons)

1 items: Francis ANDRE (zosrothko)

1 items: Guillaume (Candide7)

1 items: Tristan Swadell (TristonianJones)

1 items: Will Dazey (dazey3)

1 items: uji (uji)

1 items: Robert Varga (rovarga)

1 items: Hell_Ghost (HellGhost)

1 items: Alex Mykyta (amykyta3)

1 items: Nikolay Ognyanov (nikolayo)

1 items: Andrew Pashkin (AndreiPashkin)

Source code(tar.gz)
Source code(zip)
4.10.1(Apr 15, 2022)
Tiny update to fix build issue where java requirement for runtime was 11 not 1.8.

What's Changed

[C++] Remove reference to antlrcpp:s2ws (#3656) by @jcking in https://github.com/antlr/antlr4/pull/3657

Update publishing instruction for Dart by @lingyv-li in https://github.com/antlr/antlr4/pull/3655

Prep 4.10.1 by @parrt in https://github.com/antlr/antlr4/pull/3660

Full Changelog: https://github.com/antlr/antlr4/compare/v4.10.0...4.10.1
Source code(tar.gz)
Source code(zip)
4.10(Apr 11, 2022)
ANTLR version 4.10

This is a very major release with a number of important changes. There have been many valuable contributions, but I welcome @KvanTTT and @jcking as recent "official" major ANTLR contributors. :)

WARNING: Generated 4.10 lexers and parsers are incompatible with code generated by previous versions of ANTLR. You must regenerate all of your code from grammars to use the new runtime. This is true of all targets (except probably javascript).

Repo branching structure

We have changed the branching structure of the repository. The default branch for this repo remains master and it is the latest stable release with tags for the various releases; e.g., see release tag 4.9.3. We now do development work in branch dev between releases and all pull requests should be derived from that branch. The dev branch is merged back into master to cut a release and the release state is tagged (e.g., with 4.10-rc.1 or 4.10.) Visually our process looks roughly like this:

Targets such as Go that pull directly from the repository can use the default master branch but can also pull from the active dev branch:

$ go get github.com/antlr/antlr4/runtime/Go/[email protected]

Developers certificate of origin

In order to bring ANTLR more in line with current standard standards for contribution processes, as of 4.10, ANTLR uses the Linux Foundation's Developer Certificate of Origin, DCO, version 1.1. See file https://github.com/antlr/antlr4/raw/master/developer-cert-of-origin.txt. It is simpler than the original contributors license agreement, which required programmers to sign the contributors.txt file, which has now moved to file historical-contributors-agreement.txt.

Each commit in pull requests must have a "signature", which is simple as using -s (not -S) on the git commit command:

$ git commit -s -m 'This is my commit message'

Github's pull request process enforces the sig and gives instructions on how to fix any commits that lack the sig. See https://github.com/apps/dco for more info.

4.10-generated parsers incompatible with previous runtimes

ANTLR not only generates recursive-descent parsers; it generates a state machine called an augmented transition network (ATN) in serialized form as a bunch of integers stored in the generated parser and lexer files. This serialization format was changed for 4.10 to remove a size limit on the supported ATNs. See https://github.com/antlr/antlr4/pull/3591.

The key point here is that we changed the version number stored inside the serialization format and so, in order to use this new version of ANTLR, you must regenerate all of your lexers and parsers using the 4.10 tool and then use the new runtime. Parsers generated with 4.10 or not compatible with previous versions of the runtime.

Increasing minimum java version

Going forward, we are using Java 11 for the source code and the compiled .class files for the ANTLR tool. The Java runtime target, however, and the associated runtime tests use Java 8 (bumping up from Java 7).

JavaScript target

The JS target has been substantially reworked.

Miscellaneous improvements

Lexer rules can use the caseInsensitive option now.

Improved error messages

We rebuilt the runtime testing rig and made continuous integration improvements; part of this was done to support more recent versions of Java

We cleaned up the maven pom and brought it up to date (still has errors if somebody can make a suggestion.)

Changes report

The following report is generated by scripts/github_release_notes.py.

Issues fixed

C++ runtime: Version identifier macro ? (target:cpp, type:cleanup)

Generating XPath lexer/parser (actions, type:bug)

do we need this C++ ATN serialization? (target:cpp, type:cleanup)

Incorrect type of token with number 0xFFFF because of incorrect ATN serialization (atn-analysis, type:bug)

Clean up ATN serialization: rm UUID and shifting by value of 2 (atn-analysis, type:cleanup)

The parseFile method of the InterpreterDataReader class is missing code: "line = br.readLine();" (type:bug, target:java)

antlr.runtime.standard 4.9.3 invalid strong name. (type:bug, comp:build, target:csharp)

Serialized ATN data element 810567 element 11 out of range 0..65535 (atn-analysis, type:cleanup)

Go target, unable to check when custom error strategy is in recovery mode (target:go)

Escape issue for characeters (grammars, type:bug)

antlr4 java.lang.NullPointerException Antlr 4 4.8 (grammars, comp:tool, type:bug)

UnsupportedOperationException while generating code for large grammars. (atn-analysis, type:cleanup)

Add a more understandable message than "Serialized ATN data element .... element ... out of range 0..65535" (atn-analysis, type:cleanup)

avoid java.lang.StackOverflowError (lexers, error-handling)

Getting this error: Exception in thread "main" java.lang.UnsupportedOperationException: Serialized ATN data element out of range (atn-analysis, type:cleanup)

Improvements, features

Updated getting started with Cpp documentation. (type:improvement, comp:doc)

Escape bad words during grammar generation (code-gen, type:improvement)

Implement caseInsensitive option (lexers, options, type:improvement)

Some tool bugfixes (error-handling, comp:tool, type:improvement, type:cleanup)

Pull requests grouped by target

csharp target

use ints not chars for C# (code-gen, atn-analysis, target:csharp)

C# and Go runtime tests fixes on non-Windows OS (comp:runtime, target:csharp, target:go, comp:testing)

C#, JavaScript, Go runtime tests performance improvements (target:javascript, target:csharp, target:go, comp:performance)

cpp target

[C++] Remove more dynamic_cast usage (target:cpp)

[C++] Introduce version macros (target:cpp, type:cleanup)

[C++] Free ATNConfig lookup set in readonly ATNConfigSet (target:cpp, type:cleanup)

[C++] Implement configurable PredictionContextMergeCache (atn-analysis, target:cpp, type:cleanup)

Allow to switch off building C++ tests, needed if used as sub-project (comp:build, target:cpp)

[C++] Minor improvements to PredictionContext (target:cpp, type:cleanup)

[C++] Cleanup DFA, DFAState, LexerAction, and yet more performance improvements (target:cpp, comp:performance, type:cleanup)

[C++] Avoid copying statically generated serialized ATNs (target:cpp, comp:performance)

[C++] Add T::is for type hierarchy checks and remove some dynamic_cast (target:cpp, type:cleanup)

[C++] Devirtualize hand rolled RTTI for performance (target:cpp, comp:performance)

[C++] Refactor PredictionContext and yet more performance improvements (target:cpp, comp:performance)

[C++] Fix bugs in SemanticContext (target:cpp, comp:performance)

[C++] Refactor and optimize SemanticContext (atn-analysis, target:cpp, type:cleanup)

[C++] Add ANTLR4CPP_PUBLIC attributes to various symbols (target:cpp, type:cleanup)

Update editorconfig for c++ (target:cpp)

[C++] Avoid using dynamic_cast where possible by using hand rolled RTTI (actions, target:cpp, type:cleanup)

[C++] Fix generated Lexer static data constructor (target:cpp, type:cleanup)

[C++] Force generated static data type name to titlecase (target:cpp, type:cleanup)

[C++] Remove unused LexerATNSimulator::match_calls (target:cpp, type:cleanup)

[C++] Switch to camel case for generated static data variable names (target:cpp, type:cleanup)

[C++] Remove duplicate includes and remove unused includes (target:cpp, type:cleanup)

[C++] Lazily deserialize ATN in generated code (target:cpp, type:cleanup)

[C++] Cleanup ATNDeserializer and remove related deprecated methods from ATNSimulator (target:cpp, type:cleanup)

[C++] Improve multithreaded performance, fix TSAN error, and fix profiling ATN simulator setup (target:cpp)

[C++] Remove now unused antlrcpp::Guid (target:cpp)

[C++] Fix const correctness in ATN and DFA (target:cpp)

[C++] Fix missing virtual destructors (target:cpp, type:cleanup)

[C++] Optimize LL1Analyzer (target:cpp)

[C++] Update cmake README.md to C++17 (comp:build, target:cpp)

[C++] Fix Windows min/max macro collision (target:cpp)

[C++] Optimizations and cleanups and const correctness, oh my (target:cpp, type:cleanup)

cpp/cmake: Fix library install directories (target:cpp)

[C++] Improve MurmurHash performance for Clang (target:cpp, comp:performance)

[C++] Fix bugs in UnbufferedCharStream (target:cpp)

[C++] Improve whitespace escaping (target:cpp, type:cleanup)

[C++] Clean up antlr4::misc::Interval and antlr4::dfa (target:cpp)

Fixed C++ runtime demo (target:cpp)

[C++] Optimize ATN deserialization (target:cpp, comp:performance)

[C++] Cleanup EMPTY_VOCABULARY uses and remove SingleWriteMultipleRea… (target:cpp)

[C++] Implement standalone Unicode encoding and decoding handling (target:cpp)

[C++] Switch to std::any and deprecate antlrcpp::Any (target:cpp)

[C++] Fix ambiguous ANTLRInputStream constructor overload (target:cpp)

[C++] Partially undo commit a8d828016e54adfc403326d8e56d93e7c75c27fc (target:cpp)

[C++] Modernize Vocabulary using std::string_view (target:cpp)

[C++] Switch from NOEXCEPT to noexcept (target:cpp)

Fix reported memory leak (target:cpp)

Add .clang-format file to ensure consistent style of C++ (target:cpp)

[C++] Bump minimum required version to C++17 (target:cpp)

[C++] Upstream various performance, build, and best practice fixes (target:cpp)

[C++] Cleanup ATNDeserializer interface (target:cpp)

Fix TSAN warnings related to dfa::DFA::s0 (target:cpp)

Corrected/clarified the Linux C++ runtime compilation instructions (comp:build, comp:doc, target:cpp)

go target

[Go] Fix symbol collision in generated lexers and parsers (type:bug, target:go)

[Go] Do not export Array2DHashSet which is an implementation detail (target:go, type:cleanup)

[Go] Cleanup and fix ATN deserialization verification (target:go, type:cleanup)

Don't decode utf16 in ATN deserialization for Go; it's just a bunch of uint16 (atn-analysis, target:go)

[Go] Port locking algorithm from C++ to Go (target:go, threading)

perf: Add default implementation for Visit in ParseTreeVisitor. (trees-contexts, target:go)

[Go] Fix for issue 3557 -- get "go test" working again (target:go, type:cleanup)

make InErrorRecoveryMode public for go (target:go)

Fix bail error strategy nil interface panic (trees-contexts, type:bug, target:go)

[Go] Fix (*BitSet).equals (target:go)

[Go] Fix BitSet OR operation (target:go)

[Go] Make BaseGoTest::initializeRuntime static (target:go, comp:testing)

[Go] Improve BitSet implementation (target:go)

C# and Go runtime tests fixes on non-Windows OS (comp:runtime, target:csharp, target:go, comp:testing)

C#, JavaScript, Go runtime tests performance improvements (target:javascript, target:csharp, target:go, comp:performance)

Get rid of reflection in Go.stg (target:go)

Use atomic instead of a shared mutex for DFA.s0 (target:go)

java target

[Java] Preserve serialized ATN version 3 compatibility (atn-analysis, target:java)

Fix .interp file parsing test for the Java runtime. (target:java)

Remove unused variable from generated code (target:java)

Update javadoc to 8 and 11 (comp:build, target:java, comp:doc)

use src 11 for tool, but 8 for plugin/runtime (comp:build, target:java)

[Java] Fix remaining complaints from ErrorProne static analysis (target:java, type:cleanup)

javascript target

add license prefix (target:javascript, type:cleanup)

Refactor js runtime (target:javascript)

JavaScript: Change string to int array for serialized ATN (code-gen, atn-analysis, target:javascript, type:cleanup)

[JavaScript] Migrate from jest to jasmine (target:javascript)

Add browser fs fallback (target:javascript)

C#, JavaScript, Go runtime tests performance improvements (target:javascript, target:csharp, target:go, comp:performance)

python2 target

Convert Python2/3 to use int arrays not strings for ATN encodings (atn-analysis, target:python2, target:python3)

python3 target

Convert Python2/3 to use int arrays not strings for ATN encodings (atn-analysis, target:python2, target:python3)

[Python3] rename pygrun, add entry_point (comp:build, target:python3)

Python local import fixes (comp:build, target:python3)

swift target

[Swift] An interval ought to be a value (target:swift, type:cleanup)

[Swift] reduce Optionals in APIs (target:swift, type:cleanup)

[Swift] more improved optional handling (target:swift, type:cleanup)

[Swift] use stdlib (single pass) (target:swift)

[Docs] Update Swift Docs (comp:doc, target:swift)

Use release build config instead of debug for Swift runtime tests (improve performance more than 50%) (target:swift, comp:testing, comp:performance)

Update Swift template to silence warnings about unmodified class reference (target:swift)

Update Swift runtime doc (comp:doc, target:swift)

Use AnyObject instead of class. (target:swift)

[Swift] Use proper constants (target:swift)

[Swift] Use for in loops (target:swift, type:cleanup)

[Swift] improvements to equality functions (target:swift, type:cleanup)

Add Support For Swift Package Manager (comp:build, target:swift)

php target

Fix code style and typing to support PHP 8 (target:php, type:cleanup)

Fix PHP codegen to support int ATN serialization (atn-analysis, target:php, type:cleanup)

dart target

Dart int atn (atn-analysis, target:dart, type:cleanup)

Fix Dart linter errors under strict mode (target:dart, type:cleanup)

Remove late modifier for Dart generated context classes (target:dart)

Contributors

62 items: Justin King (jcking)

31 items: Terence Parr (parrt)

16 items: Ivan Kochurkin (KvanTTT)

7 items: Dante Broggi (Dante-Broggi)

4 items: ericvergnaud

3 items: Henner Zeller (hzeller)

2 items: Marcos Passos (marcospassos)

2 items: 100mango

2 items: Tinker (Tinker1024)

1 items: chris-miner

1 items: chen quan (chenquan)

1 items: Ken Domino (kaby76)

1 items: Matt Willer (mattwiller)

1 items: Biswapriyo Nath (Biswa96)

1 items: Alexandr (Alex-Andrv)

1 items: Oleksandr Martyshchenko (Ketler13)

1 items: Mike Lischke (mike-lischke)

1 items: Eneko Alonso (eneko)

1 items: Nicholas Bollweg (bollwyvl)

1 items: Gerhard Barteling (Barteling)

1 items: hlstwizard

1 items: Yoshimasa Niwa (niw)

1 items: frog酋长 (ccrfrog)

1 items: Isaac (idraper)

1 items: Roman Gafurov (seth2810)

1 items: Benjamin Spiegel (SkittlePox)

1 items: Abe149

1 items: Karl Heinz Marbaise (khmarbaise)

1 items: Roger Far (rogerfar)

1 items: yonatanLehman

1 items: Miguel Alejandro Muñoz Gil (mimuogi)

1 items: Harry Chan (re-thc)

1 items: Jérôme BAROTIN (jbarotin)

1 items: Wolfgang Fahl (WolfgangFahl)

1 items: Andrea Giovacchini (agiovacchini)

Full Changelog: https://github.com/antlr/antlr4/compare/4.9.3...4.10
Source code(tar.gz)
Source code(zip)
4.9.3(Nov 6, 2021)
Issues fixed

Swift Target Crashes with Multi-Threading ()

JavaScript Runtime bug ()

Go target, cannot use superClass for the lexer grammar! (type:bug, target:go)

Python runtime is inconsistent with Java (target:python2)

FunctionDef source extract using getText() (target:javascript)

Provide .NET Framework target in the csharp nuget package (target:csharp)

Go target for Antlr tool, type ",int8" => "int8" (target:go)

Flutter/Dart web support (target:dart)

Allow Antlr Javascript runtime to be loaded into Kindle Touch (target:javascript)

Fix Go test suite (comp:build, target:go)

Weird error (error-handling, comp:tool)

Improvements, features

[C++] Use faster alternative to dynamic_cast when not testing inherit… (type:improvement, target:cpp)

Stackoverflow after upgrading from 4.6 to 4.7 (error-handling, type:improvement)

Pull requests grouped by target

csharp target

cpp target

[C++] Use faster alternative to dynamic_cast when not testing inherit… (type:improvement, target:cpp)

[C++] Refactor Guid to use std::array instead of std::vector (target:cpp)

Fix some thread sanitizer warnings in C++ runtime (target:cpp)

Change utfcpp git url to use https (target:cpp)

Add explanation about lifetime of parse tree to cpp-target.md#memory-management (target:cpp)

Patch utfcpp build (comp:build, target:cpp)

Revert "Cpp target: No building tests/samples for external utfcpp" (type:bug, target:cpp)

Fix typo in ExternalAntlr4Cpp.cmake (target:cpp)

[C++] Performance improvement in finally (target:cpp)

go target

Fix for Issue 3319 (type:bug, target:go)

[Go] Fix null pointer dereference (target:go)

fix: use 32bit murmur3 same with java, implement array2dhashset which… (target:go)

Fix tests for go runtime (target:go, comp:testing)

Runtime(Go): Add LICENSE to module (target:go)

Add go.mod to go runtime module (comp:build, target:go)

Making compatibility with actions and predicates in Go better (actions, target:go)

java target

Upgrade com.ibm.icu:icu4j to latest (69.1) to avoid potential vulnerabilities it brings (target:java, unicode)

python2 target

make Python2 runtime keep consistent with Java (target:python2)

swift target

fix compiler error (target:swift)

[Swift] Changes to Mutex-es to guard the staticly shared [DFA] (target:swift)

Swift InterpreterDataReader and deserialize fixes (target:swift)

Silence Swift compiler warnings in Swift runtime (target:swift)

Fix Swift runtime ANTLRInputStream that can’t read Unicode scalars (target:swift)

php target

dart target

[Dart] Workaround for windows build issue (target:dart)

[Dart] Support Dart multiple platforms. (target:dart)

[Dart] Add case changing char stream example (target:dart)

[Dart] Fix crash when parsing high index options (target:dart)

Configure dart debug flags through environment declaration (target:dart)

Export recognizer class for dart target (target:dart)

Enable dart non nullable by default (NNBD) (comp:runtime, target:dart)

Contributors

9 items: ericvergnaud

4 items: Justin King (jcking)

4 items: Larry Li (lingyv-li)

3 items: Ken Domino (kaby76)

3 items: Vladimir-Chan

3 items: Joaquín León (Joakker)

2 items: Terence Parr (parrt)

2 items: Ivan Kochurkin (KvanTTT)

2 items: Johan Appelgren (appel1)

2 items: skef

2 items: Yoshimasa Niwa (niw)

1 items: Steven Kalt (SKalt)

1 items: Martin van Wingerden (martinvw)

1 items: Rishuv Mehta (rmehta33)

1 items: Viveksharma48b (Gravity-I-Pull-You-Down)

1 items: XenoAmess (XenoAmess)

1 items: Yi-Hong Lin (ansiemens)

1 items: Jeffsky (jjeffcaii)

1 items: Rachid Lamouri (rachidlamouri)

1 items: ksyx

1 items: renzhentaxi

1 items: Yuxin Liu (Claire) (Yuxin45)

1 items: Marc Auberer (marcauberer)

1 items: Julian Bissekkou (JulianBissekkou)

1 items: Milan Aleksić (milanaleksic)

1 items: Nathanael Demacon (quantumsheep)

1 items: Mitar (mitar)

1 items: Sergii Kliuchnyk (redexp)

1 items: Michal Harakal (michalharakal)

1 items: Alexander Bigerl (bigerl)

1 items: Josua Jäger (JaggerJo)

1 items: Mike Lischke (mike-lischke)

1 items: b1f6c1c4

1 items: Damian Andrei (xTachyon)

1 items: Minjoo Sur (minjoosur)

1 items: Ricardo Canastro (canastro)

1 items: Renan (renancaraujo)

1 items: A. Hooper (ahooper)

1 items: IdkGoodName (IdkGoodName)

1 items: Niels Basjes (nielsbasjes)

Source code(tar.gz)
Source code(zip)
4.9.2(Mar 11, 2021)
Issues fixed

CSharp and Java produce different results for identical input, identical tokens (target:csharp, type:bug)

Improvements, features

Moved away from travis-ci.com.

Pull requests grouped by target

csharp target

fix a typo in the csharp-target doc (target:csharp)

Issue 2693 (comp:build, target:csharp)

Fix for https://github.com/antlr/antlr4/issues/3049 (comp:build, target:csharp)

add script (comp:build, target:csharp)

Update ll1 analyzer (target:csharp, target:javascript, target:python2, target:python3, type:bug)

cpp target

Fixed ANTLRInputStream and ANTLRFileStream (target:cpp)

Cpp target: allow building runtime with newer C++ standards (comp:build, target:cpp)

javascript target

fix javascript import in example code (target:javascript)

Fix javascript intervals (target:javascript)

Update ll1 analyzer (target:csharp, target:javascript, target:python2, target:python3, type:bug)

python2 target

Update ll1 analyzer (target:csharp, target:javascript, target:python2, target:python3, type:bug)

python3 target

Fix TextIO compatibility with Python 3.6+ in LexerFile (target:python3)

Fix 3052 (target:python3, type:bug)

Update ll1 analyzer (target:csharp, target:javascript, target:python2, target:python3, type:bug)

swift target

work around Swift crash where reading hashValue on SemanticContext.AN… (actions, target:swift)

Fix swift warnings (target:swift)

Contributors

12 items: ericvergnaud

3 items: Ken Domino (kaby76)

1 items: Mike Lischke (mike-lischke)

1 items: 不是油条 (hackeris)

1 items: Karl Heinz Marbaise (khmarbaise)

1 items: Xingyu Xie (namasikanam)

1 items: Felix N (felixn)

1 items: Taras Sotnikov (tsotnikov)

1 items: Ivan Kochurkin (KvanTTT)

1 items: Kko (l215884529)

Source code(tar.gz)
Source code(zip)
4.9.1(Jan 5, 2021)
Issues fixed

Invalid C# code being generated (code-gen)

netstandard 2.0 build (comp:build, target:csharp)

Pull requests grouped by target

csharp target

Fix typo (target:csharp)

ensure 1 statement per line to help debug (target:csharp)

C# runtime upgrade (comp:build, target:csharp)

cpp target

travis: upgrade c++ build to focal & llvm 10 (comp:build, target:cpp)

go target

java target

javascript target

upgrade ini (target:javascript)

Potential fix to #2980 (target:javascript)

rollback partially incorrect fix for #2902 (target:javascript)

Fixed exception clauses in JS template. (target:javascript)

:ambulance: fix bad import (target:javascript)

python2 target

Fix incorrect python token string templates (target:python2, target:python3)

add optional encoding for string input in python 2 (target:python2)

python3 target

Improve python3 performance by adding slots (target:python3)

Fix incorrect python token string templates (target:python2, target:python3)

Py3 Runtime: distribute the tests (target:python3)

Contributors

8 items: ericvergnaud

2 items: Ivan Kochurkin (KvanTTT)

1 items: Alex Mykyta (amykyta3)

1 items: Ozzah

1 items: Tabari Alexander (tehbone)

1 items: David Phillips (electrum)

1 items: maxence-lefebvre

1 items: Jim Baltikauskas (JBaltika)

1 items: Felix N (felixn)

1 items: Michael R. Crusoe (mr-c)

Source code(tar.gz)
Source code(zip)
4.9(Nov 24, 2020)
New DART target

Thanks to @lingyv-li, we have a DART target!

Issues fixed

[Python3] Large import time cost paid for inline unittests ()

Dart tests fail (target:dart)

C++20 can not construct a std::string with a UTF-8 encoded string literal ()

Warning in generated javascript code when using node 14 ()

Python3 parser generation incorrect by ANTLR 4.7.1 and problem in Parser.py (target:python3)

PHP target generates invalid output when $ is used as part of the literal in lexer rule (target:php)

[C++][CMake] find_package() fails when the C++ runtime is make-install'ed ()

[Go] Generated lexer and parser file names are always in lower case with underscore ()

Syntax error in antlr4/runtime/Swift/boot.py (target:swift)

Improvements, features

Avoid stepping through ANTLR generated code when debugging (C#) (target:csharp, type:improvement)

replace gramamr with grammar (type:improvement)

Dart Target (target:dart, type:improvement)

Pull requests grouped by target

csharp target

Made C# Trace Listener usable for most cases (target:csharp)

Avoid stepping through ANTLR generated code when debugging (C#) (target:csharp, type:improvement)

refactored AntlrInputStream.cs by removing a redundand if-statement (target:csharp)

Clarify and expand documentation for runtime ParseTreeWalker (target:csharp)

NullReferenceException in CSharp target #2304 (target:csharp)

cpp target

cpp-target.md link broken fix (comp:doc, target:cpp)

Fix linker issue when compiling Cpp runtime with CMake and MSVC on Windows (comp:build, target:cpp)

Fix regressions in travis Cpp tests (target:cpp)

Fix clang compile error with --stdlib=libc++ (target:cpp)

[C++ runtime] Remove codecvt header (deprecated in C++17) in cpp runtime. (target:cpp)

Fix C++ target README example (comp:doc, target:cpp)

[C++ codegen] Add default cases to AltBlock and OptionalBlock (target:cpp)

[C++ codegen] Remove use of the u8"" string literal prefix (target:cpp)

[C++ codegen] Add explicit keyword to constructor (target:cpp)

[C++ runtime] Fix static const ODR violations (target:cpp)

[C++ runtime] Add AntlrInputStream(std::string_view) constructor (target:cpp)

[C++ runtime] Improvements of Vocabulary::fromTokenNames (target:cpp)

[C++ runtime]: Improve coding style of auto usages (target:cpp)

C++ target: don't generate extra/duplicate semicolons (target:cpp)

C++ target: escape ? character to prevent accidental trigraphs (target:cpp)

Fixed link to CMake example in cpp-target.md (comp:doc, target:cpp)

Update cpp-target.md doc (tree::ParseTreeWalker::DEFAULT is now a reference) (actions, target:cpp)

Fix CMake packaging (target:cpp)

Fix #2727: improve setTokenFactory in Cpp target (target:cpp)

go target

[Go] Thread-safe ANTLR codegen (target:go)

[Golang] enable syntax highlighting (comp:doc, target:go)

java target

javascript target

Corrected Webpack configuration in JavaScript documentation (comp:doc, target:javascript)

Javascript es6 migration (target:javascript)

Fix template issue (target:javascript)

Fix #2902 (target:javascript)

Fix a NPE (target:javascript, type:bug)

fix for issues in #1759 (target:javascript)

Javascript: migrate prototypical root modules to ES6 (target:javascript)

Javascript: fix some cyclic dependencies (target:javascript)

Javascript: migrate prototypical ATN objects to ES6 (target:javascript)

Javascript: browser support increased with babel transformation (target:javascript)

Drop browser tests (target:javascript)

Javascript: migrate prototypical Tree objects to es6 classes (target:javascript)

Javascript: migrate prototypical DFA objects to es6 classes (target:javascript)

python2 target

Fix code generation for catch statements in python (target:python2, target:python3)

python3 target

Changed Python runtime Lexer.py, Parser.py to support version >= 3.5 (target:python3)

Fix undefined value (target:python3)

changed python3 runtime Lexer.py, Parser.py to support version > 5. (target:python3)

separate py3 runtime tests from src tree + actually run said tests in CI (target:python3)

Fix code generation for catch statements in python (target:python2, target:python3)

swift target

Added missing swift version of class ParseTreeProperty (target:swift)

[Swift] Update codegen template (target:swift)

added throws declaration missing from generated swift code (target:swift)

php target

[PHP] Escape dollar sign in target string literal (target:php)

fix composer command in documentation (comp:doc, target:php)

dart target

Fix travis ci for Dart pin version (target:dart)

Pin dart version to 2.8.4 to get unit tests passing (comp:build, target:dart)

Dart Target (target:dart, type:improvement)

Contributors

7 items: Camilo Roca (carocad)

6 items: ericvergnaud

5 items: Nathan Burles (nburles)

3 items: cliid (cliid)

3 items: Felix N (felixn)

3 items: Larry Li (lingyv-li)

3 items: sullis

2 items: Adam Wójs (adamwojs)

2 items: Rob McGregor (rmcgregor1990)

1 items: Michel Hartmann (MichelHartmann)

1 items: Adarsh Bhat (adarshbhat)

1 items: Martin Mirchev (Marti2203)

1 items: Rex Jaeschke (RexJaeschke)

1 items: Muhammed Zakir (MuhammedZakir)

1 items: trenki2

1 items: Arunav Sanyal (Khalian)

1 items: Tian Siyuan (tiansiyuan)

1 items: Arthur Sonzogni (ArthurSonzogni)

1 items: demons (Clcanny)

1 items: ericliu (eric-buaa-cn)

1 items: Steven Johnstone (stevenjohnstone)

1 items: Terence Parr (parrt)

1 items: Shan M (sha-N)

1 items: Akos Kiss (akosthekiss)

1 items: Gareth Andrew Lloyd (Ignition)

1 items: Crasse maximum (IohannRabeson)

1 items: Pablo Blanco Celdrán (sigmasoldi3r)

1 items: Kilian Kilger (cohomology)

1 items: Franco Victorio (fvictorio)

1 items: David MARKEY (d-markey)

1 items: [OXID-PS] Keywan Ghadami (keywan-ghadami-oxid)

1 items: gomerser

1 items: Moses Paul R (iammosespaulr)

1 items: Tristan Swadell (TristonianJones)

1 items: Martin van Wingerden (martinvw)

1 items: Adam Grabski (Agrabski)

1 items: Leon Wehmeier (lwehmeier)

1 items: Denis Kyashif (deniskyashif)

1 items: Song Yang (XsongyangX)

1 items: Mark Lilback (mlilback)

1 items: StochasticTinkr

1 items: Nathanael Demacon (quantumsheep)

1 items: gerrie-myburgh

1 items: Julián Bermúdez Ortega (julibert)

1 items: Steve Vinoski (vinoski)

1 items: praveen4463

1 items: Ben List (listba)

1 items: Markus Franke (markus-franke)

1 items: Eason Lin (easonlin404)

1 items: thron7

1 items: delins

1 items: Abhijith Neil Abraham (abhijithneilabraham)

1 items: Massimo Santini (mapio)

1 items: t-gergely

1 items: wkhemir

1 items: Ivan Kochurkin (KvanTTT)

1 items: xsIceman

1 items: Alexander Selling (sellingsolutions)

Source code(tar.gz)
Source code(zip)
4.8(Jan 20, 2020)
Thanks to @marcospassos, ANTLR 4.8 introduces a PHP target! See PHP PRs

Issues fixed

C# Antlr4.Runtime.Standard v4.7.2.0 assembly signature problem (comp:tool, status:duplicate, target:csharp)

Improvements, features

cpp target

C++ VS project updates for next ANTLR4 release (comp:build, comp:doc, target:cpp)

don't unnecessary fetch all tokens when grabbing an interval of text … (target:cpp)

fix cpp cmake for emscripten (target:cpp)

go target

Remove unnecessary Travis files. (target:go)

javascript target

Fix placement of @members in Javascript target lexers (target:javascript)

Fix main entry point for JavaScript runtime npm package (target:javascript)

swift target

[Swift] Migrate Swift runtime to Swift 5. (target:swift)

[Swift] Check process exit status during tests. (target:swift)

Contributors

5 items: Ewan Mellor (ewanmellor)

1 items: Mike Lischke (mike-lischke)

1 items: Marcos Passos (marcospassos)

1 items: jimmy-the-eek

1 items: Tabari Alexander (tehbone)

1 items: Zhiqiang Lu (zqlu)

1 items: WalterCouto

1 items: Markus Zancolo (kuegi)

Source code(tar.gz)
Source code(zip)
4.7.2(Dec 18, 2018)
Issues fixed

Antlrv4 failed installation (actions)

[cpp] Setting contextSuperClass='antlr4::RuleContextWithAltNum' doesn't work (target:cpp)

Please publish JDK 9 Automatic-Module-Name entries (comp:build)

Improvements, features

Assign token names to string literals in parse rule contexts (code-gen, type:improvement)

File path handling fixes (comp:tool, type:improvement)

Fixed warning from MingGW and error from Clang Windows (target:cpp, type:improvement)

Fix null pointers + remove unused lines (type:improvement)

Pull requests grouped by target

csharp target

fix NPE (comp:build, target:csharp)

Fix #1955 python (target:csharp, target:python2, target:python3)

Updated csharp-target doc (comp:doc, target:csharp)

Make C# NuGet package name more visible in runtime docs (comp:doc, target:csharp)

cpp target

virtual dtor for ParseTreeProperty (target:cpp)

Do not throw std::bad_cast from antlrcpp::Any::is (target:cpp)

Add const accessors to antlrcpp::Any C++ class (target:cpp)

Fix for antlr#2164. (target:cpp)

Applied C++ changes for PR #1955. (target:cpp)

Fix #2350 (target:cpp)

Add support c++ target on android (comp:build, target:cpp)

Changes to cpp runtime to make antlr work with Chromium build (target:cpp)

Fixed warning from MingGW and error from Clang Windows (target:cpp, type:improvement)

Fix #1855, #2211 (comp:runtime, target:cpp)

Support for CMake Packages in the Cpp Runtime (target:cpp)

Force usage of 'previousContext'. (target:cpp)

Don't create exception objects if you don't throw them. (target:cpp)

Patch for multi-threaded parsing with the C++ runtime (target:cpp)

Add missing -std=c++11 flag when using non-appleclang compiler on macosx (comp:build, target:cpp)

Add Visual Studio 2017 project to Cpp runtime. Also change define for static library setting to be ANTLR4CPP_STATIC, in all projects. (comp:build, target:cpp)

C++ runtime: Make Parser::exitRule() non-virtual (comp:performance, target:cpp)

go target

java target

Avoid StringIndexOutOfBoundException (target:java, type:bug, unicode)

Java.stg: use static functions instead of inline arrays (code-gen, target:java)

javascript target

Fix #1955 javascript (target:javascript)

fix indentation in javascript target examples (comp:doc, target:javascript)

JavaScript parse tree visitor: check if ctx.children is non-null in visitChildren (target:javascript, type:bug)

Remove unused _loadString parameter (target:javascript)

python2 target

Fix #1955 python (target:csharp, target:python2, target:python3)

fixes #2188 (target:python2, target:python3)

python3 target

Fix #1955 python (target:csharp, target:python2, target:python3)

Fix install dependency for 'typing' library in python3.4 (target:python3)

Update to python 3.6 (comp:build, target:python3)

Allow importing StdinStream from antlr4 package (target:python3)

Add StdninStream for python3 runtime. Closes: #2271 (target:python3)

Fixes #2301 (target:python3)

Enable ParserATNSimulator debug mode in Python3 target (target:python3, type:bug)

fixes #2188 (target:python2, target:python3)

[Python3] return result instead of None in visitor (target:python3)

swift target

[Swift-runtime] Make sure mutexes are being freed accordingly (target:swift)

[Swift] Syntax error in antlr4/runtime/Swift/boot.py (target:swift)

[Swift] Make ParserATNSimulator.precedenceTransition return a nullable value. (target:swift)

Move the Swift runtime to Swift 4.2 (target:swift)

Testability and build improvements (comp:build, comp:testing, target:swift)

Enable some performance tests on Swift. (comp:testing, target:swift)

[Swift] Report InputMismatchException with original context information (target:swift)

[Swift] Minor tidyups in the stream code. (target:swift)

[Swift] Make ATNDeserializationOptions a simple struct. (target:swift)

[Swift] Remove HashMap and fix Parser caches. (target:swift)

Contributors

11 items: Ewan Mellor (ewanmellor)

7 items: ericvergnaud

4 items: Mike Lischke (mike-lischke)

2 items: Prateek Gulati (prateekgulati)

2 items: Adriaan Duz (nxtstep)

2 items: Steve Vinoski (vinoski)

2 items: Vladimir Goncharov (AmatanHead)

2 items: Maxim Novikov (m-novikov)

2 items: Mark Raynsford (io7m)

1 items: hannemann-tamas

1 items: Prashast Srivastava (prashast)

1 items: Alex Mykyta (amykyta3)

1 items: Sergey Chupov (scadgek)

1 items: Terence Parr (parrt)

1 items: Ivan Kochurkin (KvanTTT)

1 items: Lucas Henrique (LucasHMS)

1 items: 吴晟 Wu Sheng (wu-sheng)

1 items: Ale Contenti (alecont)

1 items: Dan Halperin (dhalperi)

1 items: Stephen Vance (srvance)

1 items: sinopsysHK

1 items: johnvanderholt

1 items: Joe R. Smith (solussd)

1 items: uvguy

1 items: Daniel Clifford (dannoc)

1 items: Chase (ChaseOxide)

1 items: Jędrzej Biedrzycki (TAndronicus)

1 items: Raz Friman (razfriman)

1 items: Zongyuan Zuo (EternalPhane)

1 items: Sönke Schau (xgcssch)

1 items: Kaspar Emanuel (kasbah)

1 items: Robert (robertbrignull)

1 items: Corey Kosak (kosak)

1 items: jkmar

1 items: Kyle Sletten (kbsletten)

1 items: Ajay Panyala (ajaypanyala)

1 items: Nicolas Croad (niccroad)

1 items: janlinde

1 items: Enes SOYLU (enessoylu)

Source code(tar.gz)
Source code(zip)
4.7.1(Dec 9, 2017)
ANTLR version 4.7.1 is a minor release but with lots of little improvements and bug fixes. You can find the pull requests grouped by target language below. Also, please find below the contributor list (auto-generated from the issues and pull request).

Summary of new features and improvements

You can now import lexers with modes into other lexer grammars [...]

The -o and -lib commandline options didn't always do the obvious thing and in fact presented some problems. With clean that up but requires a new -Xexact-output-dir command line option to enable it to avoid breaking tools built on previous versions of ANTLR [...]

For languages that support case insensitive keywords such as SQL, ANTLR does not have a built-in mechanism. Programmers either specify case insensitive lexers rules directly or they can build a special input stream that presents characters as uppercase to the lexer. I have provided documentation that describes both approaches including sample streams in various targets. [...]

The Swift runtime was migrated to Swift 4. [...]

You can generate SVG now from parse trees in Java target. [...]

The NuGet package now includes the .NET Core DLL and supports both .NET 3.5 and up, as well as .NET Core. [...]

Issues fixed

Python3 runtime says 4.6 not 4.7 (status:invalid, target:python3)

Can not generate java files from Rust grammar file (actions, code-gen, type:bug)

Unexpected code generation location behavior (cross-platform-issue, type:bug)

Compiling seperate lexer and parser in subdirectory fails (comp:build, status:duplicate, type:bug)

Improvements, features

Changed rotation to support 32 and 64-bit architectures as noted in issue #2060. (target:go, type:improvement)

Case insensitivity doc (comp:doc, comp:runtime, lexers, type:improvement)

Import lexers with modes into other lexer grammars (comp:tool, lexers, type:improvement)

Add an accessLevel parser option. (actions, type:improvement)

Support building C++ Runtime with Intel compilers (target:cpp, type:improvement)

Improve the "gnu" message format (comp:tool, error-handling, type:improvement)

Security related issue with System.getenv (comp:runtime, type:improvement)

New unreachable token value warning (comp:tool, error-handling, type:improvement)

Simplify output path generation; new -Xexact-output-dir option (actions, comp:build, comp:tool, type:improvement)

Remove ANTLRError.nullPointer from the Swift runtime. (type:improvement)

Migrate the Swift runtime to Swift 4. (target:swift, type:improvement)

Report InputMismatchException with original context information (error-handling, target:java, type:improvement)

Many fixes to antlr4-maven-plugin dependency analysis (comp:build, type:improvement)

Implemented TokenStreamRewriter (comp:runtime, target:go, type:improvement)

added SVG export of parse tree (target:java, trees-contexts, type:improvement)

NuGet packaging for .NET Core (comp:build, target:csharp, type:improvement)

CPP target runtime: fixed a memleak (target:cpp, type:improvement)

Support importing multi-mode lexer grammars (comp:runtime, lexers, type:improvement)

Pull requests grouped by target

csharp target

Add extra diagnostics and a retry to the C# runtime tests and upgrade the macOS runtime (comp:build, comp:testing, target:csharp)

#2021 fixes nuget packaging options to avoid missing dll exceptions (comp:build, target:csharp)

fix to the ATN deserialiser issue on big endian architecture (target:csharp)

for linux target, bring up to jdk8 since jdk7 no longer available for Trusty; for linux and osx targets, bring up dotnet from 1.0.3 to 1.0.4; Trusty image now has mvn 3.3.9 builtin, removed install code (comp:build, target:csharp)

Set version to 4.7.1 because 4.7.0 already exists on Nuget (target:csharp)

NuGet packaging for .NET Core (comp:build, target:csharp, type:improvement)

speed up tests; rename runtime dll ready for nuget (target:csharp)

update to .netcore v1.0.1; add osx build (comp:build, target:csharp)

cpp target

The correction for cmake depends on the demo (comp:build, target:cpp)

Support building C++ Runtime with Intel compilers (target:cpp, type:improvement)

CMake build fixes for Visual Studio (comp:build, target:cpp)

CMake build fixes for Visual Studio (comp:build, target:cpp)

Add MSVC support to c++ cmake build (comp:build, target:cpp)

Remove the useless visibility attribute from this "enum class" declaration (target:cpp)

C++ runtime: Fix two issues found by coverity (target:cpp)

Build fixes after previous C++ patch (comp:build, target:cpp)

Rework C++ Runtime Interval, IntervalSet, ATN and ATNState (atn-analysis, target:cpp)

Build fix/correction for UTF32 conversion. (target:cpp)

Small optimizations (target:cpp)

C++ runtime changes for high warning levels (target:cpp)

Fix listener example in C++ documentation. (comp:doc, target:cpp)

Make ParseTreeWalker::DEFAULT provide an IterativeParseTreeWalker (target:cpp, trees-contexts)

Fix the utf8 convertor for VS2017 (target:cpp, unicode)

Update C++ target doc for latest API changes - fixes #1824. (comp:doc, target:cpp)

Channel parameter in c-tor is now a size_t - fixes #1827. (target:cpp)

Small fix in cmake file. Fixes #1809. (comp:build, target:cpp)

CPP target runtime: fixed a memleak (target:cpp, type:improvement)

go target

Export GetHiddenTokens routines (target:go)

Fix issue with tree finding when the underlying slice needs to be resized. (target:go)

Changed rotation to support 32 and 64-bit architectures as noted in issue #2060. (target:go, type:improvement)

Add RuleContext interface methods to reserved words (target:go)

Revert "[Go] Interval" (target:go)

Placed the CommonSetStuff code inside a scope (code-gen, target:go)

Export Go lexer methods (target:go)

Allowing Go runtime compilation to succeed under ARM (target:go)

Add CommonTokenStream test. (target:go)

Interval (target:go)

Implemented TokenStreamRewriter (comp:runtime, target:go, type:improvement)

Add getter method to Interval. (target:go)

Go file header complies with standardised Go 'generated code' header (code-gen, target:go)

Fix #1925 (target:go, target:javascript, target:python2, target:python3, unicode)

Add proper error handling semantics to FileStream. (target:go)

java target

Fixes #2069 to catch env var security exception (target:java, type:bug)

Fix codegen error (Java) when a lexer has only fragment tokens (code-gen, target:java)

Fixed empty CodePointCharStream throwing exception on getText. (target:java, unicode)

Report InputMismatchException with original context information (error-handling, target:java, type:improvement)

Avoid adding to closureBusy before all ATNConfig properties are set (atn-analysis, comp:performance, target:java, type:bug)

added SVG export of parse tree (target:java, trees-contexts, type:improvement)

javascript target

enable auto-suggest engines by exposing the lexer ATN as a property, similar to the parser ATN (target:javascript)

Fix Interval usage in ErrorStrategy.js (target:javascript)

Require superClass for Lexer in split mode for JavaScript target (target:javascript)

fix inconsistent naming in target stg (target:javascript)

Fix #1925 (target:go, target:javascript, target:python2, target:python3, unicode)

Add new keyword to return proper InputStreams (target:javascript, type:bug)

python2,3 targets

fix #2031 (target:python2)

Python: add public getter to parsers to access syntax error count. (target:python2, target:python3)

Fix #1925 (target:go, target:javascript, target:python2, target:python3, unicode)

Fix input mismatch errors not reported the same way as Java in Python (comp:testing, error-handling, target:python2, target:python3)

Use spaces consistently to indent the generated Python code. (target:python2, target:python3)

Import RuleContext into the antlr4 module namespace. See Issue #2012. (target:python3)

Python 3 codegen probably didn't mean to limit ATN segment size to 2 XOR 31 (target:python3, type:bug)

swift target

Fix parse-cancellation in BailErrorStrategy. (target:swift)

Remove Recognizer.tokenTypeMapCache and .ruleIndexMapCache. (target:swift)

Fix initialization of {Lexer,Parser}Interpreter.decisionToDFA. (target:swift)

Remove Triple.swift. (target:swift)

Make ATNSimulator.sharedContextCache non-optional. (target:swift)

Remove unused ATN.modeNameToStartState. (target:swift)

Some minor tidyups. (target:swift)

Make DFA.precedenceDfa be a "let" rather than a "var". (target:swift)

Replace IntervalSet.setReadonly with makeReadonly. (target:swift)

Make all the mutexes "let" rather than "var". (target:swift)

Tidy-ups inside the Swift runtime around ParseTree, ParserRuleContext (target:swift)

Remove use of deprecated String functions (target:swift)

Open the error strategies to subclasses. (target:swift)

Tidy ATNDeserializer. (target:swift)

Report errors in ANTLRFileStream. (target:swift)

Implement UUID.init(mostSigBits, leastSigBits) in the Swift runtime. (target:swift)

Fix ParseTreePatternMatcher.split in the Swift runtime. (target:swift)

Make CharStream.init(CharStream) required (in the Swift runtime). (target:swift)

Fix UnbufferedCharStream in the Swift runtime. (target:swift)

Fix retain cycles in the Swift runtime (target:swift)

Implement UnbufferedCharStream in the Swift runtime. (target:swift)

Tidy up the exception behavior around IntervalSet and ATNConfigSet. (target:swift)

Tidy up the exception handling around RecognitionException. (target:swift)

Tidyups inside the Swift runtime (target:swift)

Update boot.py and swift-target.md to reflect Swift 4 migration changes. (target:swift)

Improve MurmurHash (target:swift)

Migrate the Swift runtime to Swift 4. (target:swift, type:improvement)

Updated swift-target.md. (comp:doc, target:swift)

Swift Documentations (comp:doc, target:swift)

Swift Target Linux Support (target:swift)

Supporting Script for Swift Runtime (target:swift)

Fixing Problems after Swift3.1 Release (target:swift)

Fixing Xcode project setting. (comp:build, target:swift)

Obsolete String.contains(_:) of Swift runtime (target:swift)

Contributors

42 items: Ewan Mellor (ewanmellor)

8 items: Terence Parr (parrt)

8 items: Mike Lischke (mike-lischke)

7 items: hanjoes (hanjoes)

5 items: ericvergnaud

4 items: David Sisson (davesisson)

4 items: Maxim (gendalph)

4 items: Sam Harwell (sharwell)

3 items: Corey Kosak (kosak)

3 items: Dong Xie (xied75)

2 items: JavaDDD

2 items: Ivan Kochurkin (KvanTTT)

2 items: Andrew Brampton (bramp)

2 items: Sachin Jain (sachinjain024)

2 items: shirou (shirou)

2 items: Arshinskiy Mike (neatnerd)

2 items: Matt Hauck (matthauck)

2 items: Renáta Hodován (renatahodovan)

2 items: jm-mikkelsen

2 items: lys0716

1 items: Oran Epelbaum (oranoran)

1 items: Nicolas Croad (niccroad)

1 items: zqlu (zqlu)

1 items: Ajay Panyala (ajaypanyala)

1 items: Jason Mooberry (jasonmoo)

1 items: simkimsia

1 items: Rostislav Listerenko (kaedvann)

1 items: fallnirvana (moon-fall)

1 items: Robert Jacobson (rljacobson)

1 items: Bráulio Bezerra (brauliobz)

1 items: Masashi Fujita (objectx)

1 items: Sandy Corn (onekosha)

1 items: Eddy Reyes (ereyes01)

1 items: Milan Aleksić (milanaleksic)

1 items: Tiago Mazzutti (tiagomazzutti)

1 items: Niels Basjes (nielsbasjes)

1 items: Venkat Peri (venkatperi)

1 items: Vaibhav Vaingankar (vaibhavaingankar09)

1 items: Maurice van der Pot (Griffon26)

1 items: Dan Halperin (dhalperi)

1 items: Sergey Parshukov (jBugman)

1 items: alim (alimg)

1 items: rlfnb

1 items: Daniel Katz (danielkatz)

1 items: Erik A. Brandstadmoen (erikbra)

1 items: Will Fuqua (waf)

1 items: Jim Allman (jimallman)

1 items: Shravan Narayan (shravanrn)

1 items: Brad Hards (bradh)

1 items: Floor (FloorGoddijn)

1 items: Ben Hamilton (Ben Gertzfield) (bhamiltoncx)

1 items: Jonas Auer (cmd-johnson)

1 items: Gerald Rosenberg (grosenberg)

1 items: Arthur Fabre (arthurfabre)

Source code(tar.gz)
Source code(zip)
4.7(Mar 30, 2017)
ANTLR version 4.7 is a major release with many improvements and bug fixes.

Summary of new features, improvements, fixes

The primary improvement is that ANTLR and all code generation targets can now handle full 21-bit unicode thanks to the superhuman effort of Ben Hamilton, @bhamiltoncx. After much thought concerning the "create stream" interface, I have decided on the following end result: C++, Python, Go, and Swift APIs didn't need any changes to support full Unicode code points, so I decided to leave those alone. Java, C#, and JavaScript runtimes required changes. Rather than gutting and changing the interface of the previous ANTLRFileStream etc..., I have deprecated those and introduced a CharStreams.fromXXX factory style interface for creating streams. For example, CharStreams.fromFileName("foo.txt")). See the new unicode documentation and a complete list of all pull requests related to Unicode.

UnbufferedCharStream for Java and C# targets was upgraded to use UTF-8 rather than the locale default encoding; they further support U+10FFFF Unicode code points. C++ is the only other target with the unbuffered stream and it worked as-is.

Change UnbufferedCharStream to use code points

Use UTF-8 by default in UnbufferedCharStream and allow specifying charset

C#: Change UnbufferedCharStream to use 32-bit Unicode code points and 32-bit buffer

In addition to creating CharStreams capable of supporting 21-bit Unicode, we added notation for Unicode code points beyond 16 bits (4 hexadecimal digits). The usual notation \uABCD still works for the basic code points but you can now use \u{1FFFF}..\u{10FFFF} to access the full range:

New extended Unicode escape \u{10ABCD} to support Unicode literals > U+FFFF

Support UTF-32 escape sequences

You can also include all characters matching Unicode properties such as [\p{Emoji}]:

Add new \p{Extended_Pictographic} and related Unicode property escapes

Also support Unicode enumerated properties via \p{Foo=Bar} syntax

Update docs for new Unicode literal and property escapes

New class EscapeSequenceParsing with \p{UnicodeProperty} support

New \p{Letter} Unicode property escape

A few last escapes: \p{EmojiPresentation=EmojiDefault} and \p{EmojiPresentation=TextDefault}

UnicodeData: Also support Unicode blocks

The C# target now supports .NET Core Support, thanks to David Neumann @lecode-official and Dong Xie @xied75!

We did quite a bit of cleaning up in the lexer escape char and char set areas (big thanks to Ivan Kochurkin @KvanTTT):

Escapes sequence recognition failure in character sets

Throw "invalid escape sequence" for double quote in string literals: '\"'

Fixed incorrect "used multiple times" warning.

error-> warnings. Fixes #1537

Reverted unterminated ranges: [+-], [-+], [-].

More accurate error messages

Added invalid charset error (for ranges without start or end) (comp:tool, error-handling)

The Go runtime was significantly speeded up.

Go Runtime: Murmur Hash and various optimizations

Initialize decisionToDFA in init in Go.stg

The XPath tree matching library used an ANTLR grammar itself, which caused a cyclic dependency in the build whereby ANTLR version v was required to build version v. I implemented a handbuilt lexer to get rid of this dependency on a grammar: XPathLexer not updated in release process, Fixes #1620. Make handbuilt lexer to avoid cyclic dependence of tool and plugin.

The Java, Swift, and C++ targets have added a hook to the construction of parse tree leaf nodes: factor out the creation of error nodes and terminal nodes in the parser. Code new TerminalNodeImpl(..) and new ErrorNodeImpl(...) instead now calls a few factory methods in the Parser that you can override in your application.

Issues fixed

C++ target:

SIGABRT when using TokenStreamRewrite::Delete method

Cpp runtime: compile needed header

compilation warning

Segmentation fault on TokenStreamRewriter destructor call

Runtime build failure on gcc 6.3.1

Fixed regression introduced by wrong optimization. Fixes #1708. (target:cpp)

Fixed crash with multi threaded parsers warming up at the same time. (target:cpp)

Improve error message in exceptions (target:cpp)

Using new Unicode syntax for C++ demo project. (target:cpp, unicode)

Fix syntaxError prototype issue (target:cpp)

Fixed a number of data type + signedness issues in the C++ target (target:cpp)

Some compilers need the functional include. (target:cpp)

Fix wrong include path of antlr4cpp runtime in cmake/ExternalAntlr4Cpp.cmake (target:cpp)

Fix parse tree property (target:cpp)

No static libs anymore for Windows C++ runtime. (comp:build, target:cpp)

Lr loop fix (target:cpp, type:improvement)

adapt code to compile with msys2 mingw compiler (comp:build, target:cpp, type:improvement)

Fix for VS 2013 runtime builds (C++) (comp:doc, target:cpp)

Implemented enhanced CommonToken::toString method

Using new Unicode syntax for C++ demo project. (target:cpp, unicode)

fix an issue where loading antlr from an IE web worker would fail (target:javascript)

Issue 1483 (target:cpp, type:improvement)

JavaScript target:

JavaScript target: super class of TraceListener is not correctly set

BailErrorStrategy in JavaScript

JavaScript visitors not visiting children

Javascript Target documentation contains no reference to existing NPM package?

Doc antlr4 / doc / javascript-target.md outdated, states there is no NPM package for antlr4 runtime

tweak wildcard

Fixed null pointer exception for JS target

Fix missing variable declarations

fix an issue where loading antlr from an IE web worker would fail

fix typo in javascript visitor

Update npm related docs

missing js export

Python2/3:

python2: Bug in IntervalSet.py:removeOne causes exception when taking complement

Python 2 - missing IllegalStateException import in PredictionContext.py

fix 'CommonToken' object has no attribute 'stopIndex' in Python{2|3}

Fix a Python 2 typo

Python 2 - missing - ErrorNode and TerminalNode - imports in Parser.py

Python 2 - missing IllegalStateException import in PredictionContext.py

C#:

Fixed portability problems in C# target. Also cleaned up some XML do…

Fix C# Pair.cs, Right arrow escaping in XML Comments

Fix #1298 for CSharp

Remove unused C# runtime method Utils.ToCharArray

Look in /usr/local/bin before /usr/bin for mono

Go:

go target: copyright notices are interpreted as package documentation

Added "action" to badWords set for Go runtime.

Remove lower case formatting on Go types & super import

Use single-line comments for copyright

Format Go runtime files

Java:

Add visitor tests for Java runtime API

augment TerminalNode with setParent()

Swift:

Remove generated files from repository and add testRig for Swift target (target:swift)

Tweak Repo to use SwiftPackageManager. (target:swift)

Tool or all-target-runtime related:

Wildcards were not handled properly, they were treated as sets! fix some typos Sam noticed.

crash upon bad grammar

Antlr 4.6 does not like same named rule element labels for different alternative labels

More accurate error messages

Added invalid charset error (for ranges without start or end)

New tool utility class Unicode

The final fix (hopefully) for alternative labels check in left recursive rules

Implement support for optional getters

Disable label checks for left recursive rules

New doc 'Contributing to ANTLR'

Channel names & constants in lexer, improved modes record format.

Source code(tar.gz)
Source code(zip)
4.6(Dec 15, 2016)
ANTLR version 4.6 is a major release with many features and bug fixes.

Summary of new features, improvements, fixes

NEW TARGETS: C++, Swift, Go:

Peter Boyer (Go target)

Mike Lischke (C++ completed target)

Dan McLaughlin (C++ initial target)

David Sisson (C++ initial target and test)

Janyou (Swift target)

Ewan Mellor, Hanzhou Shi (Swift target merging)

Major performance improvement discovered by @sharwell. We were seeing a number of reports of nonterminating parses for very large expressions with grammars that used left recursive rule specifications (like e : e '*' e | INT ;). Sam noticed that the parsing engine detected an ambiguity between two choices and resolved it properly but that both would lead to a valid parse. It turns out that the first choice is what led to really slow parsing but the second choice is much faster and still yields a valid parse. Most targets have added that improvement. Here is the discussion of the issue and solution and the primary patch for java.

Error handling improved in cases related to LL(1) decisions for optional and closure subrules. Matching sets of tokens were also sometimes considered resynchronization (error) tokens. Single token deletions that occurred at the start of expressions were sometimes not included in the parse tree as error nodes.

Add sync() calls back in for LL(1) decisions. (code-gen, error-handling)

Matched tokens being marked as error (comp:runtime, error-handling, trees-contexts, type:bug)

Single-token deletion error node not in parse tree (error-handling, trees-contexts, type:bug)

The tool itself generates better error messages:

Crash with NullPointerException in SymbolChecks.checkForModeConflicts with empty lexer (comp:tool, grammars, type:bug)

Have ANTLR4 prevent conflict with user rule names by behind-the-scenes renaming of its own variables (status:cannot-reproduce)

Add compile-time errors/warnings for use of incompatible lexer commands (comp:tool, error-handling)

character set issues (comp:tool, error-handling, type:improvement)

Duplicated and incompatible commands (comp:tool, error-handling, type:improvement)

We fixed a number of issues with the build system

Tidy poms (comp:build, type:improvement)

divide and conquer Travis CI (comp:build, type:improvement)

Version information omitted from MANIFEST.MF in antlr-4.5.1-complete.jar (comp:build, type:bug)

antrl4--sources.jar does not contain sources for runtime (comp:build, type:bug)

OSGi module name does not reflect Maven module name (comp:build, type:improvement)

Running a limited subset of runtime-testsuite (comp:testing, type:improvement)

Improvements to the ANTLR4 mvn plug-in:

Grammar dependency management for Maven plugin (comp:build, comp:maven-plugin, type:improvement)

antlr4-maven-plugin: code generated in wrong encoding (comp:maven-plugin, type:improvement)

The ANTLR unit test mechanism was completely rewritten but it should not affect anyone other than the development team

Refactor runtime test generation mechanism (comp:testing, type:improvement)

We assume java 1.7 not 1.6 now.

Drop support for 1.6, move pom.xml etc. to JDK 1.7 (cross-platform-issue, type:improvement)

Miscellaneous

make sure BSD 3-clause license prefix is consistent and on all appropriate files (type:improvement)

Allow paths as tokenVocab option (comp:build, comp:tool, type:improvement)

add TokenStreamRewriter to Python2 runtime (target:python2, type:improvement)

Add an iterative version of the ParseTreeWalker (comp:runtime, target:java, trees-contexts, type:improvement)

indicate token types as well as numbers in -tokens dump (target:java, type:improvement)

Binary parsing (comp:doc, type:improvement, type:question)

Issues fixed

Python 2 Target's ErrorStrategy Creates Problematic Tokens (target:python2, type:bug)

Prefix rules that also recurse are not detected (grammars, type:bug)

mvn compile failing. Could not resolve dependencies (comp:build, comp:doc, type:bug)

Repeated epsilon transitions in ATN from left-recursive grammar (atn-analysis, type:bug)

Add sync() calls back in for LL(1) decisions. (code-gen, error-handling)

Array2DHashSet.clear does not reset threshold (target:java, type:bug)

ANTLR 4.5.3 & java.lang.StackOverflowError (comp:tool, type:bug)

Matched tokens being marked as error (comp:runtime, error-handling, trees-contexts, type:bug)

Single-token deletion error node not in parse tree (error-handling, trees-contexts, type:bug)

$e.v yields incorrect value 0 in "e returns [int v] : '1' {$v = 1;} | '(' e ')' {$v = $e.v;} ;" (actions, grammars, type:bug)

Strange code in JavaScript - PredictionContext.js line 191 (target:javascript, type:bug)

A bug of the antlr4 python3 runtime 4.5.3 (target:python3, type:bug)

Empty alternative in rule causes syntax error in *Parser.py for Python2/3 targets (target:python2, target:python3, type:bug)

getText method with arguments not always works correctly (target:java, type:bug)

maven plugin dependency tracking ignores imported files (comp:build, comp:maven-plugin, comp:tool)

NullPointerException on invalid attribute reference (comp:tool, type:bug)

getLookaheadParseTrees() should reset or create new error handler each alt but doesn't (comp:runtime, trees-contexts, type:bug)

getAllPossibleParseTrees() doesn't work when ambig input reaches eof. (comp:runtime, trees-contexts, type:bug)

Changing '+' to '*' crashes antlr4 (atn-analysis, parsers, type:bug)

Crash with NullPointerException in SymbolChecks.checkForModeConflicts with empty lexer (comp:tool, grammars, type:bug)

LB() can try to scan to i < 0 (comp:runtime, type:bug)

Incorrect childs generation with array of element label (grammars, status:invalid)

TokenStreamRewriter for Python (target:python2)

getTokenTypeMap should test all token type values (comp:runtime, target:java, type:bug)

Version information omitted from MANIFEST.MF in antlr-4.5.1-complete.jar (comp:build, type:bug)

antrl4--sources.jar does not contain sources for runtime (comp:build, type:bug)

template error: attribute parser isn't defined (target:javascript, type:bug)

Add compile-time errors/warnings for use of incompatible lexer commands (comp:tool, error-handling)

Source code(tar.gz)
Source code(zip)
4.5.3(Mar 31, 2016)
Features / improvements

New grammar option: contextSuperClass. All parse tree internal nodes will derive from this. Default is ParserRuleContext. Should derive from ultimately RuleContext at minimum. Java target can use contextSuperClass=org.antlr.v4.runtime.RuleContextWithAltNum for convenience. It adds a backing field for altNumber, the alt matched for the associated rule node.

Added getMaxTokenType() to Vocabulary interface

Complete list of pull requests for 4.5.3 but most of those are fixing bugs.

Issues

Complete list of issues closed/solved for 4.5.2.
Source code(tar.gz)
Source code(zip)
4.5.2(Mar 28, 2016)
Features / improvements

removed final (hopefully!) dependency on Java AWT lib in runtime.

moved doc, faq from wiki to the repo.

added more tool error checks.

major Python speed improvement with better hash function (minutes to seconds)

Complete list of pull requests for 4.5.2 but most of those are fixing bugs.

Issues

Summary: some clean up in JavaScript and Python targets. Minor issues in Java/jar.

Complete list of issues closed/solved for 4.5.2.
Source code(tar.gz)
Source code(zip)
4.5.1(Jul 15, 2015)
Bug fix and repo reorganization release

We fixed number of important bugs but also combined the various target repositories, such as antlr/antlr4-python2, into the main antlr/antlr4 repository.

For the Java target only, there is also a new feature: a parser interpreter that tracks which alternative or label was match for a particular parse tree node, which is often useful during debugging. It is used in the 1.7 release of the ANTLR Intellij Plugin.

Summary of Major Changes

Maven build was added back in, thanks to new contributor Jason van Zyl; you can just open the topmost pom.xml in your favorite IDE

We completely overhauled runtime test generation

We merged all existing target repositories such as antlr4-python2 into the main antlr4 repo so that everything is now included in a single spot.

We removed runtime jar dependency on Swing. Class org.antlr.v4.runtime.misc.TestRig has moved to org.antlr.v4.gui.TestRig but we left a proxy in so that org.antlr.v4.runtime.misc.TestRig still works. The org.antlr.v4.runtime.tree.gui package moved to org.antlr.v4.gui in the tool area from the runtime. A few classes from org.antlr.v4.runtime.misc had to move. Convenience methods for saving/viewing parse trees were moved from RuleContext (parse tree) and org.antlr.v4.runtime.tree.Trees to org.antlr.v4.gui.Trees.

You can view all Issues fixed in 4.5.1, all pull requests merged and all commits for this release.

Features

Add goto to reserved words for java

Create functionality to get all possible parse trees for ambiguous input.

Makes TestRig's dialog to save and restore its state

Parser.setProfile() should keep settings like prediction mode

Make Grammar-object-aware ParserInterpreter (This interpreter will have parse tree nodes that know which alternative or label was matched for a particular parse tree subtree root.)

Check for nonconforming left-recursive rules.

Bug Fixes

Null pointer in named alternative with attributes

NullPointerException in bad recursion

Parser.setProfile() should keep settings like prediction mode

Allow parsing decision override for (decision, input index) in parsers

Random ArrayIndexOutOfBounds exceptions coming from ParserATNSimulator

Empty rule matched first gives invalid source interval

EOF leaf not included in parent tree source interval

Rule parameter in left recursive rules disallowed

wrong location on import error message reference to undefined rule

NotNull import is not deleted in Java.stg

Download Binaries

Download the ANTLR tool and all target runtimes at the antlr.org site.

The Java jars are OSGi compatible so you should be able to use them within Eclipse.

Runtime Libraries for Language Targets

Java is embedded with the tool, but is also available from maven-central

C# (zipped .dll) is a download from antlr.org. Might be in NuGet too in the future.

JavaScript is a download from antlr.org. Might also be registered as a npm package for node.js

Python2 and Python 3 are in PyPi

Sam Harwell's alternative C# target is available through NuGet. Note that this alternative target is distributed with a standalone copy of the ANTLR Tool, and only works with code generated by that standalone copy.

Documentation

Java runtime API (4.5.1)

Java Tool API (4.5.1)

ANTLR 4 Maven plugin (4.5.1)

Language Targets Info

As of 4.5, the standard distribution of ANTLR can generate code in the following languages:

Java

Python2

Python3

JavaScript

C#

In addition, the following languages are supported by standalone release(s) of the tool.

C# (Sam Harwell's)

Source code(tar.gz)
Source code(zip)
4.5.1-beta-1(Jun 30, 2015)
Bug fix and repo reorganization release

We fixed number of important bugs but also combined the various target repositories, such as antlr/antlr4-python2, into the main antlr/antlr4 repository.

For the Java target only, there is also a new feature: a parser interpreter that tracks which alternative or label was match for a particular parse tree node, which is often useful during debugging. It is used in the 1.7 release of the ANTLR Intellij Plugin.

Summary of Major Changes

Maven build was added back in, thanks to new contributor Jason van Zyl; you can just open the topmost pom.xml in your favorite IDE

We completely overhauled runtime test generation

We merged all existing target repositories such as antlr4-python2 into the main antlr4 repo so that everything is now included in a single spot.

You can view all Issues fixed in 4.5.1, all pull requests merged and all commits for this release.

Features

Add goto to reserved words for java

Create functionality to get all possible parse trees for ambiguous input.

Makes TestRig's dialog to save and restore its state

Parser.setProfile() should keep settings like prediction mode

Make Grammar-object-aware ParserInterpreter (This interpreter will have parse tree nodes that know which alternative or label was matched for a particular parse tree subtree root.)

Check for nonconforming left-recursive rules.

Bug Fixes

Null pointer in named alternative with attributes

NullPointerException in bad recursion

Parser.setProfile() should keep settings like prediction mode

Allow parsing decision override for (decision, input index) in parsers

Random ArrayIndexOutOfBounds exceptions coming from ParserATNSimulator

Empty rule matched first gives invalid source interval

EOF leaf not included in parent tree source interval

Rule parameter in left recursive rules disallowed

wrong location on import error message reference to undefined rule

NotNull import is not deleted in Java.stg

Download Binaries

Download the ANTLR tool and all target runtimes at the antlr.org site.

The Java jars are OSGi compatible so you should be able to use them within Eclipse.

Runtime Libraries for Language Targets

Java is embedded with the tool, but is also available from maven-central

C# (zipped .dll) is a download from antlr.org. Might be in NuGet too in the future.

JavaScript is a download from antlr.org. Might also be registered as a npm package for node.js

Python2 and Python 3 are in PyPi

Sam Harwell's alternative C# target is available through NuGet. Note that this alternative target is distributed with a standalone copy of the ANTLR Tool, and only works with code generated by that standalone copy.

Documentation

Java runtime API (4.5.1)

Java Tool API (4.5.1)

ANTLR 4 Maven plugin (4.5.1)

Language Targets Info

As of 4.5, the standard distribution of ANTLR can generate code in the following languages:

Java

Python2

Python3

JavaScript

C#

In addition, the following languages are supported by standalone release(s) of the tool.

C# (Sam Harwell's)

Source code(tar.gz)
Source code(zip)
antlr-4.5.1-beta-1-complete.jar(1.41 MB)
antlr-runtime-4.5.1-beta-1.jar(367.97 KB)
4.5(Jan 23, 2015)
Major Milestone Release

This release is primarily about the new JavaScript code generation target and the integration of a C# target derived from Sam Harwell's ANTLR project. Sam's is still available of course but we wanted a single tool that would generate Java, C#, Python2, Python3, and JavaScript. There are also a number of bug fixes in 4.5 as you can see below.

You need to regenerate your lexers and parsers using the latest version of the 4.5 Tool to use the 4.5 runtime.

Download Binaries

Download the ANTLR tool and all target runtimes at the antlr.org site.

The Java jars are OSGi compatible so you should be able to use them within Eclipse.

Runtime Libraries for Language Targets

Java is embedded with the tool, but is also available from maven-central (will be but you can get 4.5-SNAPSHOT right now until I figure out deploying.)

C# (zipped .dll) is a download from antlr.org. Might be in NuGet too in the future.

JavaScript is a download from antlr.org. Might also be registered as a npm package for node.js

Python2 and Python 3 are in PyPi

Sam Harwell's alternative C# target is available through NuGet. Note that this alternative target is distributed with a standalone copy of the ANTLR Tool, and only works with code generated by that standalone copy.

Documentation

Java runtime API (4.5)

Java Tool API (4.5)

ANTLR 4 Maven plugin (4.5)

Summary of Changes

You can view all issues closed for this release, all pull requests merged and all commits for this release.

Be aware that we no longer deploy a maven antlr4-annotations artifact and in fact we no longer use Java annotations in the Java runtime and so that code is disappeared.

Bug Fixes

Excess token consumption during recovery

exception when importing grammar

Generated tokenNames field in lexers is useless

Maven build has test failures due to order of token types

Suppress warning 109 for matching options during import

Parser.getTrace()

exception when importing grammar

ANTLRInputStream.getSourceName() has @NotNull annotation, but returns null

deprecate ErrorType.ALL_OPS_NEED_SAME_ASSOC

Multiline comments along with token declaration in lexer

NullUsageProcessor not compatible with java>6

Cannot start a new lexer mode right after some comments comment

Docstring for TokenStreamRewriter is incorrect

Language Targets Info

As of 4.5, the standard distribution of ANTLR can generate code in the following languages:

Java

Python2

Python3

JavaScript

C#

In addition, the following languages are supported by standalone release(s) of the tool.

C# (Sam Harwell's)

Also, Sam Harwell has an ANTLR 4 "Optimized" fork, which is currently consistent with ANTLR 4.4.
Source code(tar.gz)
Source code(zip)
4.5-rc-2(Jan 14, 2015)

This version has an important bug fix, #670, and a big update to the build system while we try to accommodate maven and its deployed mechanism.

Also be aware that we no longer deploy a maven antlr4-annotations artifact, which is only used at compile time for runtime and the ANTLR tool itself. If you use the runtime jars, you should not needed at all. If you need to compile/build, grab stuff from github.
Source code(tar.gz)
Source code(zip)
4.5-rc-1(Jan 10, 2015)
Major Milestone Release (release candidate)

This release is primarily about the new JavaScript code generation target and the integration of a C# target derived from Sam Harwell's ANTLR project. Sam's is still available of course but we wanted a single tool that would generate Java, C#, Python2, Python3, and JavaScript. There are also a number of bug fixes in 4.5 as you can see below.

We highly recommend that you regenerate your lexers and parsers using the latest version of the Tool.

Download Binaries

Download ANTLR jars at the antlr.org site.

Runtime Libraries for Language Targets

Java is embedded with the tool, but is also available from maven-central (will be)

C# (zipped .dll) is a download from antlr.org. Might be in NuGet too in the future.

JavaScript is a download from antlr.org. Might also be registered as a npm package for node.js

Python2 and Python 3 are in PyPi

C# (Sam Harwell's version) is available through NuGet. Note that this release is distributed with a standalone copy of the ANTLR Tool, and does not work with code generated by this combined distribution.

Documentation

Java runtime API (4.5)

Java Tool API (4.5)

ANTLR 4 Maven plugin (4.3) we'll update mvn stuff once we are sure that this release candidate is okay.

Summary of Changes

You can view all issues closed for this release, all pull requests merged and all commits for this release.

Bug Fixes

Generated tokenNames field in lexers is useless

Maven build has test failures due to order of token types

Suppress warning 109 for matching options during import

Parser.getTrace()

exception when importing grammar

ANTLRInputStream.getSourceName() has @NotNull annotation, but returns null

deprecate ErrorType.ALL_OPS_NEED_SAME_ASSOC

Multiline comments along with token declaration in lexer

NullUsageProcessor not compatible with java>6

Cannot start a new lexer mode right after some comments comment

Docstring for TokenStreamRewriter is incorrect

IDE Support

We have not updated the IDE plugins for this prerelease.

Language Targets

As of 4.5, the standard distribution of ANTLR can generate code in the following languages:

Java

Python2

Python3

JavaScript

C#

In addition, the following languages are supported by standalone release(s) of the tool.

C# (Sam Harwell's)

Also, Sam Harwell has an ANTLR 4 "Optimized" fork, which is currently consistent with ANTLR 4.4.
Source code(tar.gz)
Source code(zip)
4.4(Jul 16, 2014)
Major Milestone Release

ANTLR 4.4 is a major feature and minor bugfix update to ANTLR 4. The primary enhancement is the introduction of both Python2 and Python3 language targets, which you can access with -Dlanguage=Python2 from the ANTLR command line. For C#, please use Visual Studio plugin/tool that Sam Harwell has built.

Please note that tool error messages have been altered to remove unnecessary single quotes and we have added some more specific error messages.

This release includes a new build script called bild.py and was not build using maven. bild.py is a Python script I made that looks like a makefile and self bootstraps by pulling in the required bilder.py library from the net. See http://bildtool.org. On a unix box, just type "./bild.py all" to build antlr from source (if you are so inclined).

We highly recommend that you regenerate your lexers and parsers using the latest version of the Tool.

Download Binaries

Download ANTLR jars at the antlr.org site.

Documentation

Java runtime API (4.4)

Java Tool API (4.4)

ANTLR 4 Maven plugin (4.3)

Summary of Changes

You can view all issues closed for this release here, and all commits for this release here.

Bug Fixes

NullPtr and bad tree parse upon parser rule ref in lexer rule in combined grammar (#661)

Specifying an unknown target crashed the tool. (https://github.com/antlr/antlr4/commit/163ec98afc995f3935cb0971e512de3781dd8919)

IDE Support

| Project | Release which Supports ANTLR 4.4 | | --- | --- | | ANTLRWorks 2 | | | IntelliJ Plugin | 1.5 | | Eclipse Plugin | | | Visual Studio Extension | 1.2.1 |

Alternate Language Targets

As of 4.4, ANTLR can generate code in the following languages:

Java

Python2

Python3

C#

Also, Sam Harwell has an ANTLR 4 "Optimized" fork, which is currently consistent with ANTLR 4.3.
Source code(tar.gz)
Source code(zip)
4.3(Jun 19, 2014)
Major Milestone Release

ANTLR 4.3 is a major feature and bugfix update to ANTLR 4. For most users upgrading from ANTLR 4.2, this is a source- and binary-compatible update. However, to maximize the benefits provided by the update, we highly recommend that you regenerate your lexers and parsers using the latest version of the Tool.

Download Binaries

Download ANTLR jars at the antlr.org site.

Documentation

Java runtime API (4.3)

Java Tool API (4.3)

ANTLR 4 Maven plugin (4.3)

Summary of Changes

You can view all issues closed for this release here, and all commits for this release here.

Features

Parser profiler (#583)

Tool/Runtime version mismatch detection (#572, #601, #634, #635)

Improvements

Improved support for additional targets (#554, #566, #569, #609, #610)

Add clearDFA() method to easily clear the DFA cache (#584)

Improved documentation

Bug Fixes

Fix issue with tokens named ATN (#561)

Fix generated JavaDoc for listener and visitor methods for context objects created by labeled alternatives in a rule (#543)

Fix errors in support for parsing off-channel tokens (#544, #622, #626)

Fix references that alias the enclosing rule name (#571)

Fix issues with use of <assoc=right> (#542)

Fix issues with unsupported and/or ignored semantic predicates (#588, #590)

Allow multiple actions and/or predicates to appear at the end of alternatives in a left-recursive rule (#625)

Fix bugs in infrequently used IntervalSet code (#623)

Fix issues with file encoding handling (#563, #618)

Breaking Changes

Within an embedded action or semantic predicate, references to the enclosing rule can no longer be written as $ruleName. Instead, use the special symbol $ctx to refer to the enclosing rule. Parsers already generated from grammars prior to this change are not affected, but the grammar will need to be updated prior to generating code with ANTLR 4.3. This change was required as part of correcting #571.

IntervalSet.getMinElement now returns the first element, even if that element is negative. Code which relied on this method to return a non-negative value even if the set contained negative values will need to be updated.

Related Projects

The following tables list several related projects, along with the version number corresponding to this release. The tables will be updated as each of the projects release updates to support this release of ANTLR 4.

IDE Support

| Project | Release which Supports ANTLR 4.3 | | --- | --- | | ANTLRWorks 2 | | | IntelliJ Plugin | | | Eclipse Plugin | | | Visual Studio Extension | 1.2.1 |

Alternate Language Targets

| Project | Release which Supports ANTLR 4.3 | | --- | --- | | ANTLR 4 "Optimized" fork | 4.3-opt | | C# Target | 4.3 |
Source code(tar.gz)
Source code(zip)
4.2.2(Apr 7, 2014)
Minor Bug Fix Release

ANTLR 4.2.2 is a minor bugfix update to ANTLR 4. This update is source- and binary-compatible with ANTLR 4.2 for virtually all users (see the Breaking Changes section at the end). Users updating from earlier versions are encouraged to read the release notes for ANTLR 4.2.

Documentation

Java runtime API (4.2.2)

Java Tool API (4.2.2)

ANTLR 4 Maven plugin (4.2.2)

Summary of Changes

You can view all issues closed for this release here, and all commits for this release here.

Bug Fixes

Improve the handling of grammars containing multiple explicit references to EOF (#531, #534, #537, #539)

Improve the ability of generated parsers to report localized errors in the presence of semantic predicates (#529, #530, #533, parrt/antlr4#134, #538)

Other minor fixes and cleanup (#528, #532, #538, #540)

Breaking Changes

The signature of the following methods changed to support the proper handling of grammars with multiple explicit references to EOF. These methods are rarely, if ever, overridden in client applications.

LexerATNSimulator.closure

LexerATNSimulator.getEpsilonTarget

ParserATNSimulator.closure

ParserATNSimulator.closureCheckingStopState

ParserATNSimulator.closure_

ParserATNSimulator.getEpsilonTarget

Related Projects

The following tables list several related projects, along with the version number corresponding to this release. The tables will be updated as each of the projects release updates to support this release of ANTLR 4.

IDE Support

| Project | Release which Supports ANTLR 4.2.2 | | --- | --- | | ANTLRWorks 2 | 2.2.2 | | IntelliJ Plugin | 1.1 | | Eclipse Plugin | 0.3.0 | | Visual Studio Extension | 1.2.1 |

Alternate Language Targets

| Project | Release which Supports ANTLR 4.2.2 | | --- | --- | | ANTLR 4 "Optimized" fork | 4.2.2-opt | | C# Target | 4.2.2-alpha001 |
Source code(tar.gz)
Source code(zip)
4.2.1(Mar 25, 2014)
Minor Bug Fix Release

ANTLR 4.2.1 is a minor bugfix update to ANTLR 4. This update is source- and binary-compatible with ANTLR 4.2. Users updating from earlier versions are encouraged to read the release notes for ANTLR 4.2.

Documentation

Please see the latest release for links to the current documentation.

Summary of Changes

You can view all issues closed for this release here, and all commits for this release here.

Bug Fixes

Fix the ParserRuleContext which is returned by the parser interpreter when parsing starts with a left-recursive rule (#461)

Fix the accessor methods generated for rules where multiple alternatives use the same label (#433)

Add new warning and error messages to help users (#472, #489, #500)

Fix lexer action execution in the middle of rules (#469)

Fix bug in adaptivePredict for certain left recursive rules (#509)

Other smaller fixes (#474, #495)

Source code(tar.gz)
Source code(zip)
4.2(Feb 4, 2014)
Major Milestone Release

ANTLR 4.2 is a major feature and bugfix update to ANTLR 4. For most users upgrading from ANTLR 4.1, this is a source- and binary-compatible update. However, to maximize the benefits provided by the update, we highly recommend that you regenerate your lexers and parsers using the latest version of the Tool.

Documentation

Please see the latest release for links to the current documentation.

Summary of Changes

You can view all issues closed for this release here, and all commits for this release here.

Features

XPath-like syntax for selecting parse tree nodes

Parse tree pattern matching using concrete syntax

LexerInterpreter and ParserInterpreter are much more powerful and now part of the runtime library (details)

Improvements

Improved left-recursion elimination algorithm in the Tool (#268, #303, #313, #314, #355)

Substantial performance improvements for grammars containing left-recursive rules (#97, #192, #300, #400, #401)

antlr4-maven-plugin now prints proper error messages, and is now compatible with Eclipse m2e out-of-the-box (#319, #342)

A new annotation processor verifies correct use of @NotNull and @Nullable at compile time, and is available through Maven separately from the rest of ANTLR 4 in the new antlr4-annotations artifact (#434, #435) Note: This feature is only available for Maven users; not as part of the standalone ANTLR downloads.

Initial support for Java 8 (#337, #430)

The Tool supports grammar files in UTF-8 encoding with a byte order mark, a common default for text editors on Windows systems (#175)

Bug Fixes

Many other bug fixes

Source code(tar.gz)
Source code(zip)
4.2-SNAPSHOT(Jan 23, 2014)
Major Milestone Release (Preview Release)

ANTLR 4.2 is a major feature and bugfix update to ANTLR 4. For most users upgrading from ANTLR 4.1, this is a source- and binary-compatible update. However, to maximize the benefits provided by the update, we highly recommend that you regenerate your lexers and parsers using the latest version of the Tool.

Prerelease Library

Download the pre-release jar for non-maven users.

The current prerelease build of ANTLR 4.2 is available through Sonatype's Maven snapshot repository. To enable your Maven builds to use the snapshot releases, include the following in your projects pom.xml or in your system ~/.m2/settings.xml.

<profiles> <profile> <id>allow-snapshots</id> <activation><activeByDefault>true</activeByDefault></activation> <repositories> <repository> <id>snapshots-repo</id> <url>https://oss.sonatype.org/content/repositories/snapshots</url> <releases><enabled>false</enabled></releases> <snapshots><enabled>true</enabled></snapshots> </repository> </repositories> </profile> </profiles>

With the snapshot repository enabled, you can reference the ANTLR 4 runtime with the following:

<dependency> <groupId>org.antlr</groupId> <artifactId>antlr4-runtime</artifactId> <version>4.2-SNAPSHOT</version> <scope>compile</scope> </dependency>

Documentation

Please see the latest release for links to the current documentation.

Summary of Changes

You can view all issues closed for this release here, and all commits for this release here.

Features

XPath-like syntax for selecting parse tree nodes

Parse tree pattern matching using concrete syntax

LexerInterpreter and ParserInterpreter are much more powerful and now part of the runtime library (details)

Improvements

Improved left-recursion elimination algorithm in the Tool (#268, #303, #313, #314, #355)

Substantial performance improvements for grammars containing left-recursive rules (#97, #192, #300, #400, #401)

antlr4-maven-plugin now prints proper error messages, and is now compatible with Eclipse m2e out-of-the-box (#319, #342)

A new annotation processor verifies correct use of @NotNull and @Nullable at compile time, and is available through Maven separately from the rest of ANTLR 4 in the new antlr4-annotations artifact (#434, #435)

Initial support for Java 8 (#337, #430)

The Tool supports grammar files in UTF-8 encoding with a byte order mark, a common default for text editors on Windows systems (#175)

Bug Fixes

Many other bug fixes

Source code(tar.gz)
Source code(zip)
4.1(Jul 23, 2013)
Feature and Bug Fix Release (minor breaking changes)

You can view all issues closed for this release here, and all commits for this release here.

New Features

This list may be incomplete, and does not include ordinary bug fixes.

Additional static analysis to report warnings and errors in grammars.

Improved support for other targets (currently Java and C# targets are available).

Added parse tree JTree pane and Export to PNG feature to parse tree viewer.

Improved runtime API documentation.

Improved parsing performance.

Support \uFFFF in lexers.

Updated Java grammars to support Java 7 syntax, including full support for Unicode escape sequences.

Breaking Changes

This list may be incomplete

The following changes are likely to have a significant impact on user code.

The serialized ATN representation has changed and is not backwards compatible. All parsers must be regenerated before they can be used with the ANTLR 4.1 runtime.

The following changes are not likely to break user code (infrequently used features or code).

Changed the formatting of the default error messages printed to console (could affect tools which parse this raw output).

Remove the ParserRuleContext.altNum field.

String literals and lexer character sets cannot span multiple lines in the grammar. Use the escape characters \n and/or \r to include newline characters in a literal or set.

The Pair and Triple classes in the runtime are now immutable, and Triple does not extend Pair.

If you extended the ANTLRErrorStrategy interface (or one of its derived classes), be aware of significant changes to those files.

If you extended LexerATNSimulator or ParserATNSimulator, be aware of significant changes in those classes.

Removed ParserInterpreter.predictATN.

If you use code to clear the cached DFA, be aware that the invariants have changed and the previous code will result in an exception with ANTLR 4.1.

Old method (ANTLR 4.0)

Arrays.fill(parser.getInterpreter().decisionToDFA, null);

New method (ANTLR 4.1)

ATN atn = parser.getATN(); for (int i = 0; i < parser.getInterpreter().decisionToDFA.length; i++) { parser.getInterpreter().decisionToDFA[i] = new DFA(atn.getDecisionState(i), i); }

Source code(tar.gz)
Source code(zip)

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.

Related tags

Overview

ANTLR v4

Authors and major contributors

Useful information

The Definitive ANTLR 4 Reference

Additional grammars

Comments

Releases(4.11.1)

4.11.1(Sep 4, 2022)

4.11.0(Sep 4, 2022)

Issues fixed

Improvements, features

Pull requests grouped by target

csharp target

cpp target

go target

java target

javascript target

python2 target

python3 target

swift target

php target

dart target

Contributors

4.10.1(Apr 15, 2022)

What's Changed

4.10(Apr 11, 2022)

ANTLR version 4.10

Repo branching structure

Developers certificate of origin

4.10-generated parsers incompatible with previous runtimes

Increasing minimum java version

JavaScript target

Miscellaneous improvements

Changes report

Issues fixed

Improvements, features

Pull requests grouped by target

csharp target

cpp target

go target

java target

javascript target

python2 target

python3 target

swift target

php target

dart target

Contributors

4.9.3(Nov 6, 2021)

Issues fixed

Improvements, features

Pull requests grouped by target

csharp target

cpp target

go target

java target

python2 target

swift target

php target

dart target

Contributors

4.9.2(Mar 11, 2021)

Issues fixed

Improvements, features

Pull requests grouped by target

csharp target

cpp target

javascript target

python2 target

python3 target

swift target

Contributors

4.9.1(Jan 5, 2021)

Issues fixed

Pull requests grouped by target

csharp target

cpp target