Comparison of parser generators
Page Module:Message box/ambox.css has no content.
This article needs additional citations for verification. (July 2023) |
This is a list of notable lexer generators and parser generators for various language classes.
Regular languages
Regular languages are a category of languages (sometimes termed Chomsky Type 3) which can be matched by a state machine (more specifically, by a deterministic finite automaton or a nondeterministic finite automaton) constructed from a regular expression. In particular, a regular language can match constructs like "A follows B", "Either A or B", "A, followed by zero or more instances of B", but cannot match constructs which require consistency between non-adjacent elements, such as "some instances of A followed by the same number of instances of B", and also cannot express the concept of recursive "nesting" ("every A is eventually followed by a matching B"). A classic example of a problem which a regular grammar cannot handle is the question of whether a given string contains correctly nested parentheses. (This is typically handled by a Chomsky Type 2 grammar, also termed a context-free grammar.)
Deterministic context-free languages
Context-free languages are a category of languages (sometimes termed Chomsky Type 2) which can be matched by a sequence of replacement rules, each of which essentially maps each non-terminal element to a sequence of terminal elements and/or other nonterminal elements. Grammars of this type can match anything that can be matched by a regular grammar, and furthermore, can handle the concept of recursive "nesting" ("every A is eventually followed by a matching B"), such as the question of whether a given string contains correctly nested parentheses. The rules of Context-free grammars are purely local, however, and therefore cannot handle questions that require non-local analysis such as "Does a declaration exist for every variable that is used in a function?". To do so technically would require a more sophisticated grammar, like a Chomsky Type 1 grammar, also termed a context-sensitive grammar. However, parser generators for context-free grammars often support the ability for user-written code to introduce limited amounts of context-sensitivity. (For example, upon encountering a variable declaration, user-written code could save the name and type of the variable into an external data structure, so that these could be checked against later variable references detected by the parser.)
The deterministic context-free languages are a proper subset of the context-free languages which can be efficiently parsed by deterministic pushdown automata.
Parsing expression grammars, deterministic Boolean grammars
This table compares parser generators with parsing expression grammars, deterministic Boolean grammars.
| Name | Parsing algorithm | Output languages | Grammar, code | Development platform | License |
|---|---|---|---|---|---|
| AustenX | Packrat (modified) | Java | Template:D-P | All | Free, BSD |
| Aurochs | Packrat | C, OCaml, Java | Template:D-A | All | Free, GNU GPL |
| BNFlite | Recursive descent | C++ | Template:D-A | All | Free, MIT |
| Canopy | Packrat | Java, JavaScript, Python, Ruby | Template:D-P | All | Free, GNU GPL |
| CL-peg | Packrat | Common Lisp | Template:D-A | All | Free, MIT |
| Drat! | Packrat | D | Template:D-A | All | Free, GNU GPL |
| Frisby | Packrat | Haskell | Template:D-A | All | Free, BSD |
| grammar::peg | Packrat | Tcl | Template:D-A | All | Free, BSD |
| Grako | Packrat + Cut + Left Recursion | Python, C++ (beta) | Template:D-P | All | Free, BSD |
| IronMeta | Packrat | C# | Template:D-A | Windows | Free, BSD |
| Laja | 2-phase scannerless top-down backtracking + runtime support | Java | Template:D-P | All | Free, GNU GPL |
| lars::Parser | Packrat (supporting left-recursion and grammar ambiguity) | C++ | Identical | All | Free, BSD |
| LPeg | Parsing machine | Lua | Template:D-A | All | Free, MIT |
| lug | Parsing machine | C++17 | Template:D-A | All | Free, MIT |
| Mouse | Recursive descent (modified, limited memoization and left-recursion) | Java | Template:D-P | Java virtual machine | Free, Apache 2.0 |
| Narwhal | Packrat | C | Template:D-A | POSIX, Windows | Free, BSD |
| Nearley | Earley | JavaScript | Template:D-A | All | Free, MIT |
| Nemerle.Peg | Recursive descent + Pratt | Nemerle | Template:D-P | All | Free, BSD |
| neotoma | Packrat | Erlang | Template:D-P | All | Free, MIT |
| nez[36] | Parsing machine | Java, C | Template:D-P | Java virtual machine | Free, BSD |
| NPEG | Recursive descent | C# | Template:D-A | All | Free, MIT |
| OMeta | Packrat (modified, partial memoization) | JavaScript, Squeak, Python | Template:D-A | All | Free, MIT |
| PackCC | Packrat (modified, left-recursion support) | C | Template:D-A | All | Free, MIT |
| Packrat | Packrat | Scheme | Template:D-A | All | Free, MIT |
| Pappy | Packrat | Haskell | Template:D-A | All | Free, BSD |
| parboiled | Recursive descent | Java, Scala | Template:D-A | Java virtual machine | Free, Apache 2.0 |
| Lambda PEG | Recursive descent | Java | Template:D-A | Java virtual machine | Free, Apache 2.0 |
| parsepp | Recursive descent | C++ | Template:D-A | All | Free, public domain |
| Parsnip | Packrat | C++ | Template:D-A | Windows | Free, GNU GPL |
| Patterns | Parsing machine | Swift | Identical | All | Free, MIT |
| peg | Recursive descent | C | Template:D-A | All | Free, MIT |
| PEG.js | Packrat (partial memoization) | JavaScript | Template:D-A | All | Free, MIT |
| Peggy[37] | Packrat (partial memoization) | JavaScript | Template:D-A | All | Free, MIT |
| Pegasus | Recursive descent, Packrat (selectively) | C# | Template:D-A | Windows | Free, MIT |
| pegc | Recursive descent | C | Template:D-A | All | Free, public domain |
| pest | Recursive descent | Rust | Template:D-P | All | Free, MIT, Apache 2.0 |
| PetitParser | Packrat | Smalltalk, Java, Dart | Template:D-A | All | Free, MIT |
| PEGTL[38] | Recursive descent | C++11, C++17 | Template:D-A | All | Free, Boost |
| Parser Grammar Engine (PGE) | Hybrid recursive descent / operator precedence[39] | Parrot bytecode | Template:D-A | Parrot virtual machine | Free, Artistic 2.0 |
| PyPy rlib | Packrat | Python | Template:D-A | All | Free, MIT |
| Rats! | Packrat | Java | Template:D-A | Java virtual machine | Free, GNU LGPL |
| Spirit2 | Recursive descent | C++ | Template:D-A | All | Free, Boost |
| Treetop | Recursive descent | Ruby | Template:D-A | All | Free, MIT |
| Yard | Recursive descent | C++ | Template:D-A | All | Free, MIT or public domain |
| Waxeye | Parsing machine | C, Java, JavaScript, Python, Racket, Ruby | Template:D-P | All | Free, MIT |
| PHP PEG | PEG Parser? | PHP | Template:D-A | All | Free, BSD |
General context-free, conjunctive, or Boolean languages
This table compares parser generator languages with a general context-free grammar, a conjunctive grammar, or a Boolean grammar.
| Name | Parsing algorithm | Input grammar notation | Output languages | Grammar, code | Lexer | Development platform | IDE | License |
|---|---|---|---|---|---|---|---|---|
| ACCENT | Earley | Yacc variant | C | Template:D-A | external | All | No | Free, GNU GPL |
| APaGeD | GLR, LALR(1), LL(k) | ? | D | Template:D-A | generated | All | No | Free, Artistic |
| Bison | LALR(1), LR(1), IELR(1), GLR | Yacc | C, C++, D,[40] Java, XML | Template:D-A, except XML | external | All | No | Free, GNU GPL |
| DMS Software Reengineering Toolkit | GLR | ? | Parlanse | Template:D-A | generated | Windows | No | Proprietary |
| DParser | Scannerless GLR | ? | C | Template:D-A | scannerless | POSIX | No | Free, BSD |
| Dypgen | Runtime-extensible GLR | ? | OCaml | Template:D-A | generated | All | No | Free, CeCILL-B |
| E3 | Earley | ? | OCaml | Template:D-A | external, or scannerless | All | No | ? |
| Elkhound | GLR | ? | C++, OCaml | Template:D-A | external | All | No | Free, BSD |
| GDK | LALR(1), GLR | ? | C, Lex, Haskell, HTML, Java, Object Pascal, Yacc | Template:D-A | generated | POSIX | No | Free, MIT |
| Happy | LALR, GLR | ? | Haskell | Template:D-A | external | All | No | Free, BSD |
| Hime Parser Generator | GLR | ? | C#, Java, Rust | Template:D-P | generated | .NET framework, Java virtual machine | No | Free, GNU LGPL |
| IronText Library | LALR(1), GLR | C# | C# | Template:D-A | generated or external | .NET framework | No | Free, Apache 2.0 |
| Jison | LALR(1), LR(0), SLR(1) | Yacc | JavaScript, C#, PHP | Template:D-A | generated | All | No | Free, MIT |
| Syntax | LALR(1), LR(0), SLR(1) CLR(1) LL(1) | JSON/Yacc | JavaScript, Python, PHP, Ruby, C++, C#, Rust, Java | Template:D-A | generated | All | No | Free, MIT |
| Laja | Scannerless, two phase | Laja | Java | Template:D-P | scannerless | All | No | Free, GNU GPL |
| ModelCC | Earley | Annotated class model | Java | Generated | generated | All | No | Free, BSD |
| P3 | Earley–combinators | BNF-like | OCaml | Template:D-A | external, or scannerless | All | No | ? |
| P4 | Earley–combinators, infinitary CFGs | BNF-like | OCaml | Template:D-A | external, or scannerless | All | No | ? |
| Scannerless Boolean Parser | Scannerless GLR (Boolean grammars) | ? | Haskell, Java | Template:D-P | scannerless | Java virtual machine | No | Free, BSD |
| SDF/SGLR | Scannerless GLR | SDF | C, Java | Template:D-P | scannerless | All | Yes | Free, BSD |
| SmaCC | GLR(1), LALR(1), LR(1) | ? | Smalltalk | Template:D-A | internal | All | Yes | Free, MIT |
| SPARK | Earley | ? | Python | Template:D-A | external | All | No | Free, MIT |
| Tom | GLR | ? | C | Generated | none | All | No | Free, "No licensing or copyright restrictions" |
| UltraGram | LALR, LR, GLR | ? | C++, C#, Java, Visual Basic .NET | Template:D-P | generated | Windows | Yes | Proprietary |
| Wormhole | Pruning, LR, GLR, Scannerless GLR | ? | C, Python | Template:D-A | scannerless | Windows | No | Free, MIT |
| Whale Calf | General tabular, SLL(k), Linear normal form (conjunctive grammars), LR, Binary normal form (Boolean grammars) | ? | C++ | Template:D-P | external | All | No | Proprietary |
| yaep | Earley | Yacc-like | C | Template:D-A | external | All | No | Free, GNU LGPL |
Context-sensitive grammars
This table compares parser generators with context-sensitive grammars.
| Name | Parsing algorithm | Input grammar notation | Boolean grammar abilities | Development platform | License |
|---|---|---|---|---|---|
| bnf2xml | Recursive descent (is a text filter output is xml) | simple BNF[<span title="Script error: No such module "decodeEncode".">clarification needed] grammar (input matching), output is xml | ? | Beta, and not a full EBNF parser | Free, GNU GPL |
See also
References
Page Template:Reflist/styles.css has no content.
- ^ Page Module:Citation/CS1/styles.css has no content."Ragel State Machine Compiler".
- ^ http://www.colm.net/open-source/ragel/ [verification needed]
- ^ Page Module:Citation/CS1/styles.css has no content."Adaptive LL(*) Parsing: The Power of Dynamic Analysis" (PDF). Terence Parr. Retrieved 2016-04-03.
- ^ Page Module:Citation/CS1/styles.css has no content.Boyland, John; Spiewak, Daniel (2010-09-17). "Tool Paper: ScalaBison Recursive Ascent-Descent Parser Generator". Electronic Notes in Theoretical Computer Science. Proceedings of the Ninth Workshop on Language Descriptions Tools and Applications (LDTA 2009). 253 (7): 65–74. doi:10.1016/j.entcs.2010.08.032. ISSN 1571-0661.
- ^ Page Module:Citation/CS1/styles.css has no content."Beaver - a LALR Parser Generator". beaver.sourceforge.net. Retrieved 2023-09-16.
- ^ Page Module:Citation/CS1/styles.css has no content.Newton, Jim E.; Demaille, Akim; Verna, Didier (2016-05-09). "Type-Checking of Heterogeneous Sequences in Common Lisp" (PDF). Proceedings of the 9th European Lisp Symposium on European Lisp Symposium. ELS2016. Kraków, Poland: European Lisp Scientific Activities Association: 13–20. ISBN 978-2-9557474-0-7.
- ^ Page Module:Citation/CS1/styles.css has no content."CL-Yacc — a LALR(1) parser generator for Common Lisp". www.irif.fr. Retrieved 2023-09-16.
- ^ Page Module:Citation/CS1/styles.css has no content.Hosseinpour, Sahereh; Alavi Milani, Mir Mohammad Reza; Pehlivan, Hüseyin (July 2018). "A Step-by-Step Solution Methodology for Mathematical Expressions". Symmetry. 10 (7): 285. Bibcode:2018Symm...10..285H. doi:10.3390/sym10070285. ISSN 2073-8994.
- ^ Page Module:Citation/CS1/styles.css has no content."CppCC's Home Page". cppcc.sourceforge.net. Retrieved 2023-09-16.
- ^ Page Module:Citation/CS1/styles.css has no content."Java Cup". pages.cs.wisc.edu. Retrieved 2023-09-16.
- ^ Page Module:Citation/CS1/styles.css has no content."CUP". www2.cs.tum.edu. Retrieved 2023-09-16.
- ^ Page Module:Citation/CS1/styles.css has no content.Thiemann, Peter; Neubauer, Matthias (2004-12-31). "Parameterized LR Parsing". Electronic Notes in Theoretical Computer Science. Proceedings of the Fourth Workshop on Language Descriptions, Tools, and Applications (LDTA 2004). 110: 115–132. doi:10.1016/j.entcs.2004.06.007. ISSN 1571-0661.
- ^ Page Module:Citation/CS1/styles.css has no content.Gray, Robert W.; Levi, Steven P.; Heuring, Vincent P.; Sloane, Anthony M.; Waite, William M. (1992). "Eli: a complete, flexible compiler construction system". Communications of the ACM. 35 (2): 121–130. doi:10.1145/129630.129637. ISSN 0001-0782. S2CID 5121773.
- ^ Page Module:Citation/CS1/styles.css has no content.Owens, Scott; Flatt, M.; Shivers, O.; McMullan, Benjamin (2004-10-01). "Lexer and Parser Generators in Scheme" (PDF). Scheme 2004: Proceedings of the Fifth Workshop on Scheme and Functional Programming.
- ^ a b Page Module:Citation/CS1/styles.css has no content.Areias, Hugo; Simões, Alberto; Henriques, P.; Cruz, Daniela Carneiro da (2010-09-01). Parser generation in Perl: an overview and available tools (PDF). Compilers, Programming Languages, Related Technologies and Applications 2010.
- ^ Page Module:Citation/CS1/styles.css has no content.Volkman, Victor (2007-07-19). "Let Your Parser Go for the GOLD". Developer.com. Retrieved 2023-11-04.
- ^ Page Module:Citation/CS1/styles.css has no content."Parsing in C#: All the Tools and Libraries You Can Use (Part 2) - DZone". dzone.com. Retrieved 2023-11-04.
- ^ Page Module:Citation/CS1/styles.css has no content.Ortin, Francisco; Quiroga, Jose; Rodriguez-Prieto, Oscar; Garcia, Miguel (2022-03-03). "An empirical evaluation of Lex/Yacc and ANTLR parser generation tools". PLOS ONE. 17 (3) e0264326. Bibcode:2022PLoSO..1764326O. doi:10.1371/journal.pone.0264326. ISSN 1932-6203. PMC 8893623. PMID 35239695.
- ^ Page Module:Citation/CS1/styles.css has no content.Enseling, Oliver (2000-12-29). "Build your own languages with JavaCC". InfoWorld. Retrieved 2023-11-04.
- ^ Page Module:Citation/CS1/styles.css has no content."JavaCC". JavaCC. Retrieved 2023-11-04.
- ^ Page Module:Citation/CS1/styles.css has no content."Building parsers for the web with JavaCC & GWT (Part one)". Chris Ainsley. 14 April 2014. Retrieved 2014-05-04.
- ^ Page Module:Citation/CS1/styles.css has no content."The Lemon Parser Generator". sqlite.org. Retrieved 2023-11-30.
- ^ Page Module:Citation/CS1/styles.css has no content."The Lezer Parser System".
- ^ Page Module:Citation/CS1/styles.css has no content."Building a ShopifyQL Code Editor". Shopify. Retrieved 2023-12-06.
- ^ Page Module:Citation/CS1/styles.css has no content."Sponsoring the Lezer parser system | Tines". www.tines.com. 2022-03-11. Retrieved 2023-12-06.
- ^ Page Module:Citation/CS1/styles.css has no content."An LR(*) parser generator for C++".
- ^ Page Module:Citation/CS1/styles.css has no content."Racc". i.loveruby.net. Retrieved 2021-11-26.
- ^ Page Module:Citation/CS1/styles.css has no content."Racc Grammar File Reference". i.loveruby.net. Retrieved 2021-11-26.
- ^ Page Module:Citation/CS1/styles.css has no content."The REX Parser Generator supports C, C++, Java, JavaScript, C#, Go, Haxe, Python, Scala, Typescript, XQuery, and XSLT".
- ^ Page Module:Citation/CS1/styles.css has no content."The SLK Parser Generator supports C, C++, Java, JavaScript, and C#, optional backtracking, free".
- ^ Page Module:Citation/CS1/styles.css has no content."License agreement". Archived from the original (TXT) on 2016-02-17.
- ^ Page Module:Citation/CS1/styles.css has no content."SLY (Sly Lex Yacc)".
- ^ Page Module:Citation/CS1/styles.css has no content."Tree-Sitter - An incremental parsing system for programming tools".
- ^ Page Module:Citation/CS1/styles.css has no content.Adam Ślosarski (2007). "Visual BNF – software for defining formal (LALR) grammars and parser generation for .net framework". Intralogic.
- ^ Page Module:Citation/CS1/styles.css has no content."Parse - Compile time (LR) type safe parser generator for C++". GitHub. 30 December 2021.
- ^ Page Module:Citation/CS1/styles.css has no content.Kuramitsu, Kimio (2015-11-26), Nez: practical open grammar language, arXiv:1511.08307
- ^ Maintained fork of PEG.js
- ^ Page Module:Citation/CS1/styles.css has no content.taocpp/PEGTL, The Art of C++, 2024-03-14, retrieved 2024-03-16
- ^ Page Module:Citation/CS1/styles.css has no content."Parrot: Grammar Engine". The Parrot Foundation. 2011.
PGE rules provide the full power of recursive descent parsing and operator precedence parsing.
[dead link] - ^ Page Module:Citation/CS1/styles.css has no content."Decl Summary (Bison 3.8.1)". www.gnu.org.
External links
- The Catalog of Compiler Construction Tools Script error: No such module "webarchive".
- Open Source Parser Generators in Java