WisiToken Overview

WisiToken is an LALR, LR1, and Packrat parser generator and run-time. The grammar can be specified in a bison-like input file, or as Ada statements. The LR parser supports generalized parsing (parallel parsers to handle conflicts) and syntax error recovery (inserting and deleting tokens to allow parsing to continue).

The primary customer for WisiToken is the Emacs Ada mode indentation, fontification, and navigation engine, which uses an LR1 parser generated by WisiToken.

The Ada grammar specified in the Ada Language Reference manual is not LALR(1); it has shift/reduce and reduce/reduce conflicts. Modifying the grammar to be LALR(1) is hard, and complicates adding future changes to the language.. Using a generalized LALR parser to handle the conflicts allows keeping the grammar (mostly) in its original form.

Recovering from syntax errors allows indentation, fontification, and navigation in files that are in the process of being edited. The algorithm is described here.

WisiToken provides two lexers:

WisiToken is distributed under GPL version 3, with the GNAT modification that allows use in non-GPL projects.

WisiToken is a significant rewrite of OpenToken, deleting support for recursive-descent, adding syntax error correction in the parser, and adding support for simple Packrat parsers.

current version: 2.1

User guide

WisiToken may be obtained in two ways:

It relies on external libraries:

WisiToken has been tested with AdaCore GNAT Community Edition 2019. It uses Ada 2012 containers, so it requires an Ada 2012 compiler. It also uses some gnatcoll packages, so it may only work with GNAT.

Some packages have SPARK annotations, and can be proven by SPARK.

WisiToken is maintained by Stephen Leake. Submit bugs directly to Stephen. Discussion about WisiToken is on the newsgroup comp.lang.ada, or the Emacs Ada mode mailing list.

History

Version ?

Version 2.1

Version 2.0

Version 1.3.1

Version 1.3.0

Version 1.2.0

Version 1.1.0

Version 1.0.1

Version 1.0


my home page

powered by Ada Last modified: Fri Jul 24 09:43:26 Pacific Daylight Time 2020