Today I effectively scrapped the entire AnalyseSyntax()-function in the compiler and wrote it all over again from the scratch. Then I realised that the rewrite wasn’t much better then the original so I scrapped that too. And I did it again. I ended up with version 4 of that procedure. The reason? Gathering the identifiers can be done in many ways. There are good ways and not so good ways. The essential difference of these algorithms derives from only one fact: which ever ends up being the most dynamical one, tends to make future updates easier.

So what is the key that all new identifiers have in common in BASIC languages? It’s the keyword “As”. My original system scanned the modifiers and flag values within statements with no strict relations between them. This resulted in lots of extra conditions and made my code a bit tangly. The next version tried to build statements keyword by keyword eventually forming the statement with all attributes included. Since keywords have dependencies on each other and strictly pre-defined legal order in which they can appear, it was made impossible to make them “conditional” especially when they appear before the actual keyword that identifies the statement.

Then there was an option to use regexp templates on structural representation of the statement. However, this can lead to problems with non-constant syntaxes like function or operator declarations, or array definitions; typically everything that can have comma-separated parameters.

Finally, the best solution (so far) was to scan the entire statement token by token, just like I did in the first place, but this time by defining rules how keywords can be linked. Basically, each modifier and flag value has list of those keywords that can lead to them – including line start. This will ensure that no keyword can be out of place nor out of context. In addition, it should be easy to add new keywords in the future.

So what is now done? It’s still the constants. But I hope I’ve created a base on which I can start building new things soon. Oh, and I did something else, too: You can now define new instances just like in Visual Basic, “Dim i As New k” is equivalent to “Dim i As k = New k“

The compiler is currently at a stage where syntax of program basic structures need to be validated. This pass involves parsing and data gathering, from which the first refers to syntax cheking of classes/functions/constants etc, and the latter is basically forming the Token Table for the program. There are 9 data structures to do: constants, enumerations, program flow, jumps, structures, variable definition, functions, operators and classes.

There is no “properties” in this list. While this is something that would be exciting to implement, I’ve decided not to – just yet. Properties are part of classes that involve usually two functions: getters and setters for certain attributes. Traditionally programmers used to have these two separate functions, but it’s more elegant to have normal assignments and values in spite of they actually perform more code under the hood. You could use something like this:

myClassVariable.SetMyPropertyValue(myValue)
variable = myClassVariable.GetMyPropertyValue()

However, the following way is cleaner:

myClassVariable.myProperty = value
variable = myClassVariable.myProperty

Both of these would still perform a function within the class. I’ll keep this item on my TODO-list, but let’s try to establish the core first 🙂

Back to the analysing process. So those 9 keyword statements need to be parsed in order for CoolBasic to be able to compile a list of all token names. Naturally, each token must have additional data assigned to them such as access modifiers, inheritance modifiers, overriding modifiers and certain flag values. I don’t want to attach all these fields to every token data because not all of them is never needed at the same time and this would only munch unneeded memory for large programs. So it’s quite challenging task to create any “optimal” way to store this kind of data that could bend to all needs at once.

The programming tool I’m using to create CoolBasic V3 compiler doesn’t really support classes and inheritance properly (woot! why can’t you just use Visual Studio?! – It’s not fast enough ^^). I’m using allocated memory blocks to store only the most relevant data. This will make my code somewhat uglier than I had hoped, but at least it’s memory efficient. Note to self: Just remember to maintain the clean-up functions accordingly, mmkay…

And finally, some good news. The consants are already done 🙂