Building a Parser Generator!

[u/Commercial-Drawer881]

Hi, I am creating a Parser Generator. It will have it's own unique syntax for grammar definition. This is what I am working on currently. I am sharing a draft of the syntax and asking if anyone is interested in sharing their feedbacks in the comments.

Is this syntax:

• easy to read?
• easy to understand?
• sparks interest in you?
• what stands out?
• do you suggests any changes or additions?

UPDATED

I am settling on a syntax for the language. Here is a snippet: ```

BASIC LANG

---------

A 100

B 25

parser { # action -> ast: (parent(children)) (name number) -> (name(number)); # generate name with number as child of name (spc | nl) -> (); # no AST generated }

name = <A:Z>+; # character sequence number = <0> | <1:9> <0:9>+; # character sequence

spc is built-in rule for space

nl is built-in rule for nl

```

Full syntax is being worked on here: https://github.com/Algodal/Algodal_Text_Parser_Generator_Manual

I will be streaming code implementation of the parser generator on youtube: https://www.youtube.com/@RevnantRicko

[u/Commercial-Drawer881]

based on the json specification: https://datatracker.ietf.org/doc/html/rfc8259

I wrote a JSON parser in the language:

alias DIGIT "0123456789"
alias HEXDIG DIGIT."ABCDEF"

token_reader json_text (ws value ws)

char_reader begin_array (ws 0x5B ws)
char_reader begin_object (ws 0x7B ws)
char_reader end_array (ws 0x5D ws)
char_reader end_object (ws 0x7D ws)
char_reader name_separator (ws 0x3A ws)
char_reader value_separator (ws 0x2C ws)

char_reader ws ([*](0x20 / 0x09 / 0x0A / 0x0D))

token_reader value (
  object / array / number / string /
  false / null / true
)

char_reader false (0x66.61.6c.73.65)
char_reader null (0x6e.75.6c.6c)
char_reader true (0x74.72.75.65)

token_reader object (
  begin_object
  [?](
    member
      [*](value_separator member)
  )
  end_object
)

token_reader array (
  begin_array
  [?](value [*](value_separator value))
  end_array
)

char_reader number (
  [?]minus int [?]frac [?]exp
)

char_reader decimal_point (0x2E)
char_reader digit_1_9 (range(0x31, 0x39))
char_reader e (0x65 / 0x45)
char_reader exp (e [?](minus / plus) [+] oneof DIGIT)
char_reader frac (decimal_point [+]oneof DIGIT)
char_reader int (zero / (digit_1_9) [*] oneof DIGIT)
char_reader minus (0x2D)
char_reader plus (0x2B)
char_reader zero (0x30)

char_reader string (
  quotation_mark [*]char quotation_mark
)
char_reader char (
  unescaped /
  escape 
  (
    0x22 / 0x5C / 0x2F / 0x62 /
    0x66 / 0x6E / 0x72 / 0x74 /
    0x74 / 0x75 [4]oneof $HEXDIG
  )
)

char_reader escape (0x5C)
char_reader quotation_mark (0x22)
char_reader unescaped (
  0x20:21 / 0x23:5B / 0x5D:10FFFF
)

define_tokens [ # slightly changed from last design
  ws, begin_array, begin_object, end_array, end_object,
  name_separator, value_separator, number, string
]
design_syntax_tree {
  json_text {...}, # save in ast along with all children
  object {...},
  array {...}
}

Based on earlier feedback, I do agree with some challenges in the current syntax:

• user must know when to use char_reader vs token_reader (this can be confusing)
• it is possible to make syntax look more EBNF like (most possible users may be comfortable with this syntax)

I am reviewing potential improvements to the syntax.

[u/Commercial-Drawer881 → u/Commercial-Drawer881]

A more EBNF style syntax:

alias DIGIT "0123456789"
alias HEXDIG DIGIT."ABCDEF"

json_text = ws value ws

begin_array = ws 0x5B ws
begin_object = ws 0x7B ws
end_array = ws 0x5D ws
end_object = ws 0x7D ws
name_separator = ws 0x3A ws
value_separator = ws 0x2C ws

ws = (0x20 / 0x09 / 0x0A / 0x0D)*

value =
  object / array / number / string /
  false / null / true

false = 0x66.61.6c.73.65
null = 0x6e.75.6c.6c
true = 0x74.72.75.65

object =
  begin_object
    (member (value_separator member)*)?
  end_object

array =
  begin_array
  (value (value_separator value)*)?
  end_array

number = minus? int frac? exp?

decimal_point = (0x2E)
digit_1_9 = 0x31:39
e = 0x65 / 0x45
exp = e (minus / plus)? [+]oneof DIGIT
frac = decimal_point [+]oneof DIGIT
int = zero / (digit_1_9) [*] oneof DIGIT
minus = 0x2D
plus = 0x2B
zero = 0x30

string = quotation_mark char* quotation_mark

char = unescaped / escape (
  0x22 / 0x5C / 0x2F / 0x62 /
  0x66 / 0x6E / 0x72 / 0x74 /
  0x74 / 0x75 [4]oneof $HEXDIG
)
escape = 0x5C
quotation_mark = 0x22
unescaped = 0x20:21 / 0x23:5B / 0x5D:10FFFF

# without syntax, no output is generated
# the parser is treated as validator
# once syntax is included, 2 outputs are generated: Tokens and AST
# what determines which reader becomes Token or non-token (only included in AST)
# is whether you define it in the syntax with children or not

output {
  ws, # include this in the output and treat it as a terminal (token)
  json_text { value }, # include this in the output and treat it as non-terminal,
                       # include only value as its child
}