discover/grub2: improve handling of word & delimiter tokens

Currently, the delimiter token handling is a little fragile: we try to
ignore non-inter-word delimiters in the lexer with a selective set of
regexes on the possible delimiter characters.

This means we don't need to handle potential delimiters in every grammar
rule, but there are other situations (not regex-able) where we may see
delimters, and this will cause a parse error.

Instead of relying on the regex behaviour, we have an 'inter_word' flag,
which is set when we see the first word token, and cleared when we see
an end-of-line token. We only emit TOKEN_DELIM when this flag is set.

This means that we only get the delim tokens when they're required -
when we're looking for word separators (becuase WORD DELIM WORD is
distinct from WORD WORD - eg "linux /vmlinux" and "x$var").

We add a few new tests for the "menuentry" and "if" syntax, with
different delimiter configurations.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

1 files changed, 15 insertions, 0 deletions

diff --git a/test/parser/test-grub2-noeol.c b/test/parser/test-grub2-noeol.c
new file mode 100644
index 0000000..e2db85e
--- /dev/null
+++ b/test/parser/test-grub2-noeol.c
@@ -0,0 +1,15 @@
+
+#include "parser-test.h"
+
+void run_test(struct parser_test *test)
+{
+	const char data[] = "true";
+
+	test_add_file_data(test, test->ctx->device, "/boot/grub/grubenv",
+			data, sizeof(data));
+
+	__test_read_conf_data(test, test->ctx->device,
+			"/boot/grub/grub.cfg", data, sizeof(data));
+
+	test_run_parser(test, "grub2");
+}