CoolBasic Story
First code lines for CoolBasic were written 15 years ago. Read the whole story on our website.
(click the image below)
The story of BlitzBasic
3 months ago BlitzPlus went open-source. A few days ago Blitz3D followed. When I first read about BlitzPlus being made available on GitHub I kind of silently congratulated them for making that decision. To me it wasn’t surprising at all because of what has happened to the indie game industry and hobbyist game making in general for the past few years. The trend is strongly leaning towards mobile platforms – at least for 2D games. On PC you’d better be going with 3D nowadays – unless your 2D game is either one of those more complex genres, or when the touch interface isn’t enough. In other words, BlitzPlus had become somewhat obsolete (being 2D only and desktop only); it simply doesn’t answer to the modern game development demand. So in my opinion, it made perfect sense to go open-source since not them (The Blitz Research Lab), or anyone else would be making any real money out of it – especially now that they have two competing products of their own (i.e. BlitzMax and Monkey).
Originally, BlitzPlus was released after Blitz3D. I never understood why they wanted to do that – why develop a new product when they could’ve just brought the new features (native UI commands) into the existing product, Blitz3D. It would’ve strengthened the brand and not confused new customers. Not just that, but then they introduced BlitzMax whose main selling points were cross-plaform support (for the desktop) and language syntax enhancements. These changes actually do warrant a new product, though, because all of the breaking changes. From developer’s point of view, however, cross-platform support is a nightmare to establish, improve, and maintain; you’ll have to provide the base functionality for all supported platforms now, which will essentially multiply development time by an order of magnitude. This also explains why BlitzMax launched without 3D. They simply didn’t have time to do it, especially when the community expectations were “at least Blitz3D level of functionality”.
Thankfully, BlitzMax is rather modular. To this day it has been extended with countless libraries developed by the community. Much more than a one-man company can produce on its own. There was even a community driven 3D module in development. Sadly, it was shut down before release because Mark Sibly announced the “official 3D module” with some teaser screens and info. I believe there could’ve even been a small demo. Well… that official 3D module never came out. I don’t know what happened, but I guess developing a full featured 3D engine that works on all supported platforms the same way took so much time that Mark lost interest before it was finished. Just kidding.
A more reasonable theory is that back then the mobile revolution was happening, thanks to iPhone, and Mark saw the huge market potential. So maybe the point of interest just shifted. As BlitzMax already had the functionality that met the mobile game development needs (which is to create 2D games as easily as possible), why not utilize that. So BlitzMax now needed to support new platforms. But if you want to run games on a phone it will place some limits to the engine. A system that was designed to execute on a desktop was too heavy for a phone. This is when Mark first hinted about “bmax2” on his blog. The new system would have to be “lighter” which in itself would already require lots of code rewrite. So why not bring yet more syntax enhancements and make it a new product. Monkey was born.
Monkey targets 12 platforms, including Ouya and PSM. That’s pretty impressive. The only way to make this happen in a reasonable way is to translate a program written in the Monkey language (basically a BlitzBasic derivative) into some well-known other language and compile that for the target platform. Based on what Mark has written on his blog from 2011 onwards there has been quite some technical challenges to get everything to work on such a broad set of platforms. Fighting environment related bugs rather than having fun implementing new features into your product can get really tiresome after a while. And I think that explains why the development of Monkey has been slow. It’s still plagued by a large amount of little bugs. These are nasty because they seem so minor in the users’ eyes and if not fixed in a timely manner they will begin to lose interest and faith into the product’s quality. In addition, new potential customers first scan the forum for some general information, then find about the endless list of these small unfixed things, get a negative first impression, and move on in search of alternatives.
All these products. All these years. Yet there is no 3D functionality apart from Blitz3D. And B3D is still based on the ancient DirectX7. Even mobile games nowadays heavily utilize 3D because the sharp improvement in hardware performance in modern phones and tablets. What’s more, is that the explosion of mobile market has caught the interest of “more-than-just-indie” level game development companies. These shops have professional 3D artists, animators, programmers, composers and designers and release some really nice looking 3D titles for mobile devices. Mobile games are no longer all about 2D. And if Blitz Research Labs want to be offering a “serious” development tools that target these platforms, they’ll need a 3D engine and fast! Unity is already taking over as the “de facto” environment to develop 3D games for the mobile. You’re already late!
There is a roadmap for Monkey that hints about “mojo3d”.
Mojo3d is already in development – although it may not be called that once it’s finished. The basic idea here is to provide a simple, immediate mode, low level 3D API that people can use to write higher level stuff with, for example, a backwards compatible Mojo driver capable of enhanced effects/custom shaders etc. Currently, I am aiming for compatibility with all targets capable of gl2/gles2/d3d11. More on this is it develops…
It’s still not out, and reading that topic gave me the impression that it’s not going to be anytime soon. As I said earlier, the expectations are “minimum B3D level of functionality”, but this time for all target platforms. Mark, haven’t you learned anything? You really want to try and make it available on all 12 target platforms, including HTML5?! Even Unity doesn’t do anything like that (there’s a browser plug-in for rendering instead). This is a MASSIVE job to do; you can’t handle it alone. Not within a reasonable time anyway. At the same time your competitors are rocking away with their 30+ member development teams. Technology soars up faster than ever before. If you do this the engine will be outdated right from the get-go.
3 months ago, Mark dropped a bomb. He said in the roadmap topic:
Alas, there is no ETA on ‘next up’ Monkey features.
The sad truth is, Monkey sales are not good and it’s likely I will have to find some kind of ‘supplementary’ income soon – not easy for a guy who’s never held down a real job! Well, since I was 18 anyway. This is not likely to improve productivity, but I do plan on at least continuing to provide updates/fixes and improvements to current Monkey, pretty much as I have been doing recently. But ‘Monkey 2’ (as it was evolving into) is right now on hold.
It’s possible that Monkey 2 was supposed to be a successor for Blitz3D and/or BlitzMax. But since that is now out of the picture and Monkey is going to have only maintenance fixes from now on, I wouldn’t expect new products anytime soon. Which means that Blitz is losing the last momentum it had and will undoubtedly be overshadowed by the competing products.
Soon after that statement, Mark wrote a follow-up in his blog. The article was titled “A slightly depressing update…” where he elaborates about the low Monkey sales. BlitzPlus was announced open-source 2 days later.
He agrees that marketing could be better, but in my opinion fails to see some important points that all, in my opinion, play part in the problem. Anyway… this post was depressing also to read, and it has caused a lot of stir and concerns all over Monkey and BlitzBasic forums. I have always thought Mark’s writing style is a bit too aggressive, but this is not how you do PR. Even though you had to get it off your chest, you shoul’ve picked your words better.
And now Blitz3D went open-source too. For BlitzPlus it made sense, but I really don’t get the reasoning behind this move. Yes, it’s old (15 years, huh?), but it’s still *the 3D dev tool* in their whole product line. All you had to do was to update its engine pipeline to use DirectX 11, and perhaps add more commands that affect the entities’ appearance. Maybe physics too. Make it compatible with the more modern Windows versions. Or maybe MacOS and Linux. I know that you couldn’t apply the syntax enhancements that BlitzMax and Monkey have, but quite frankly, you didn’t have to. People would still use it. You don’t have to (and shouldn’t) compete with your own products. Monkey and Blitz3D serve different needs; one is cross-platform and the other is specialized in desktop (it doesn’t matter if it had a simpler syntax).
The problem is that the Blitz Research Lab has too many products. Instead of adding to existing products they develop a new one. And each time the new product is expected to also contain everything than the previous product did. In addition, they also keep introducing these breaking changes to either libraries or language syntax. Even though it may seem like a good idea to release something completely new every now and then (and get paid for of the purchase) the fact is that you have a flaw in the business model.
Blitz products have always been buy-once-get-lifetime-updates-for-free. This can only take you so far. You basically have to develop completely new products regularly to get (partly the same) users to pay more. And there comes a point where the amount of work is simply too much (especially for one man) to sustain this pattern. Too little return for too much work. Now, look at how some other companies do it… companies like Microsoft, Adobe, Autodesk etc. They too publish a new version of their most popular software every 1-3 years. But they always build on top of the previous version. If you take a moment and look how Visual Studio, for example, has evolved since 2003… it improves each time, but is still definitely based on the previous version. Moreover C# and VB.NET have both evolved as languages too, adding new features and syntax. But the developers still view C# as a single language and Visual Studio as a single product (or brand). As a result, the products are more mature, too.
Even though I think Monkey is a great product, it simply fails at marketing. If you’re a programmer, I completely understand that there might not be any interest in marketing. The Blitz Research Lab is a really small company. I’ve got the impression that Mark Sibly is completely self-employed, and I don’t think there’s anyone else to help with the “boring” things such as writing documentation, managing websites, or doing accounting. it would help a tremendous amount if you had a sales guy, or even another developer (who could handle the editor and website, for example). But the problem appears to be that they can’t afford to hire anyone. It’s a vicious circle that feeds itself: you can’t do everything needed → products suffer → sales suffer → starts over. It’ll generate frustration, create stress, lead to burnout, lose of interest etc.
As someone in the Monkey forums put it: “Monkey’s major issue is that it simply cannot keep up with the perceived competition in the form of Unity, Gamemaker and other well funded and well teamed tools, and the gap is going to grow over time.”
I think that BlitzPlus and BlitzMax were mistakes. Mark shoul’ve just developed Blitz3D to have their features (while still targeting multiple desktop platforms). But the mobile revolution blinded him. He took a big risk with Monkey and it didn’t turn out as well as expected. While the new language features such as classes with constructors, and even polymorphism do sound good on paper the fact is that at that point why would I not choose C++ instead (especially when you lose your sweetest carrot i.e. 3D). One might argue that BlitzBasic as a language would then be outdated. But that’s fine because the game runtime and easiness to use it were the things that mattered most. Blitz3D had its target audience. But not anymore, and only because its tech is too old.
While Monkey supporting mobile platforms is indeed a really nice thing, the problem is that those platforms already have official SDKs that everyone uses; If you’re a newbie game developer who wants to create his first iPhone game the first googling will not lead to Monkey website. It will lead to the SDK download and developer community sites. Also, you won’t find a way to solve your coding problems in Monkey but with the “more-adopted” way instead.
I don’t know how Mark will overcome this, but I think making Blitz3D open-source without having a replacing product already available, was made in a hurry and without thinking. I don’t think Monkey has any chance at competing with mobile game development tools (2D or 3D) anymore, but I don’t know if introducing yet another product will be the desperately needed medicine either 🙁 All I can say is: Mark, get a job to stabilize the income, take a time to think about what you want to do with the BlitzBasic brand and products. Put Monkey on Steam if you don’t want to handle marketing on your own.
BlitzPlus open-source topic:
http://www.blitzbasic.com/Community/posts.php?topic=102473
Blitz3D open-source topic:
http://www.blitzbasic.com/Community/posts.php?topic=102907
Source codes on GitHub:
https://github.com/blitz-research
Mark’s blog:
http://marksibly.blogspot.fi/
Monkey roadmap:
http://www.monkey-x.com/Community/posts.php?topic=5548
Tags: Misc
Thoughts about MonoGame
I’ve been toying with this idea of using MonoGame for the new CoolBasic for quite some time now. In theory it would offer many benefits such as the ability to build cross-platform game engines with minimal code changes. And since it’s based on XNA’s design there’s a lot of material already on the web so it has the potential to boost development productivity a great deal. Sounds good, right?
I’m already rather familiar with XNA so I was tempted to just get started with the default Windows project template. But I wanted to see how you’d create a cross-platform solution with OS specific setup code. For the most part, the only thing that’s different for each OS is the way you construct the game window; after that you just instantiate the game class and the rest is in common across all platforms.
So how to get started with this cross-platform project? It turns out that the MonoGame dev team have some plans for providing a set of official samples that, according to their words, should compile against all of the supported platforms (Android, Linux, OSX, Ouya, PSM, Windows, Windows Store, and Windows Phone). This should be the perfect reference material how I should build my Visual Studio solution. It even appears to be using the MonoGame Nuget package so surely I could just download it, open it, and be able to compile it with no additional setup? Well, not exactly…
First of all, the Nuget package configuration is broken due to some versioning issues. I couldn’t solve this with the graphical Nuget package manager in Visual Studio, and had to manually install the package from the PM command line instead.
Secondly, the Visual Studio solution doesn’t include all the needed projects! As it turned out, the entire Content project (where you have your assets such as textures, sounds and fonts) is missing. It’s present in the downloadable though, but when I attempted to add it to the solution it’d just fail at loading because of an “unknown project type”. After some googling I learn that apparently MonoGame doesn’t have the Content pipeline fully implemented yet. MonoGame is at version 3.2, how on earth a feature this essential is still missing? Maybe you can load compiled .xnb assets and use them in game, but you can’t create them without the real XNA1. This gave me an important clue – could the unrecognized Content project type be the real XNA content project…
XNA cannot be installed into Visual Studio 2013 directly, but luckily there’s a workaround. Once installed, indeed I was able to add the Content project to the solution (you’ll have to manually create a solution folder named “Content” and import it there). I now have 1) the game project, 2) the game’s content project, and 3) the XNA content builder project. Then you can compile and run the example!
The sample game is a simple 2D platformer where you can move left/right and jump. Except the game crashes if you jump. The reason was that MonoGame couldn’t find OpenAl.dll that is used for playing the jumping sound. This DLL is nowhere to be found within the downloaded package. I guess someone forgot to include the dependencies… moreover why is it even using OpenAL on Windows where DirectSound is available?
So I downloaded OpenAL and put the DLL in the output directory, then ran the game again. And it still crashes at playing the jump sound, this time due to some shady DivideByZeroException that originates from deep within OpenAL, completely out of my reach to debug or fix. The problem could be literally anywhere; maybe the sound is invalid? Did the content builder generate a valid xnb file? Did MonoGame load it correctly? Does OpenTK (yes, this is also included) do something with it before it ends up to OpenAL? Do I have the right version of OpenAL (since I had to separately download it)? I ended up commenting out all code that attempted playing sound. The game worked flawlessly after that.
This content building thing concerns me. If the primary way to loading game assets is via the content pipeline then there should be a fully working *official* way to compile the files into the xnb format. There are 3rd party tools, but if I was to use this for production, I really need the officially supported one. After some digging, it appears to be that one such tool is currently in development as a standalone application. The documentation states that it should come with the SDK, but it’s not there. So I downloaded the latest MonoGame source from GitHub in the hopes of being able to execute it from Visual Studio. Except that it doesn’t contain the full source code for it (for example, there’s no project file).
Finally I found the pipeline tool by installing the latest daily build from their TeamCity site (login as quest to view it). For whatever reason it’s installed into MSBuild under Program Files.
First impressions? Not very positive. I had to manually reconstruct the Visual Studio solution, adding and configuring the missing projects. Tools are missing or incomplete. Samples don’t build due to incomplete configuration and missing dependencies. Completely broken source code. Everything just seems so unfinished and unstable at the moment. And I merely touched the surface – who knows how many other issues like these there will be – especially when Linux or OSX enters the picture.
I’m not so sure anymore about the idea of enhanced productivity when working with MonoGame if I’m encountering bugs and unimplemented things like these right at the start. It may very well be that I end up wasting more time hunting weird MonoGame specific bugs than just use the real XNA (that just works). All in all, I was surprised how immature MonoGame still is, and I’m not really confident about its reliability and fitness for serious production use. For Windows I can always build the engine (more optimized even) with SharpDX. Maybe for other OS’ I’d fall back to MonoGame, when it’s hopefully in a more polished state.
Compiler analysis techniques
The CoolBasic Classic compiler has progressed again, and this time I’d like to share some interesting new features about its internal architecture. Using C# enables me to do certain things much more easily than the old procedural approach and the Object Oriented design really starts to kick in. The new compiler is highly structural, and it now records much more data about declared symbols, scopes, and execution paths. This also serves as a good foundation for some code analysis techniques. For example, the compiler will emit warnings for variables, types, and functions that are declared but never used, as well as if a variable is used before it has been initialized. Warnings like these will encourage the users to write better and cleaner code.
The original CoolBasic has almost no optimizations regarding the generated byte code. This is now different: Constant value analysis can ignore code branches (such as If/Else) if the condition can be evaluated to True or False at compile time. Thus, redundant code will not even make it to the final compiled program. Also, since we have constant value pre-evaluation, some expressions will be simplified before conversion to byte code. This ought to improve the runtime performance. In addition, thanks to internal scope-specific dictionaries, resolving branch targets does not produce linear search to all recorded labels (user-defined or generated). This should improve compile time performance greatly for large programs.
We’ll see how far we end up going with code analysis in the future… I’d love to collect data such as Cyclomatic Complexity or Number of Executable Statements in order to derive a general Maintainability Index out of the given source code.
Local scoping
All scopes now have their own list of local variables. This means that the user can declare variables in a code block such as If, For, or Case. These variables are allocated when the execution begins in that block and they will cease to exist after the execution exits the scope. Therefore, it is possible to declare several variables that have the same name as long as they aren’t conflicting in an enclosing block. Consider the following example:
Dim a As Integer
If a = 1 Then
Dim b As Integer = 2
// Variable 'b' is only visible within this If block.
// You cannot declare variable 'a' here because it is already declared
// outside.
EndIf
If a = 1 Then
Dim b As Integer = 3
// Variable 'b' is visible within this If block and any child blocks.
// Variable 'b' is different than the one declared in the previous
// If block.
For i As Integer = 1 To 10
// Variable 'i' is only visible within this For block.
// Varible 'b' is also visible here since it was declared
// in an enclosing block.
b = b + 1
Next
EndIf
It is also possible to declare scope specific constants in the same way. Upon entering a scope the Runtime will ensure that all of its local variables are initialized with zero. Similarly, local arrays and strings can now be freed upon leaving the scope.
Short-circuit And/Or
We’ll also change the way how the Boolean And and Or operators work. They have become ‘short-circuit’, meaning that if the end result of the Boolean operation can be determined by just evaluating the first (left) operand, then the program will not evaluate the right half at all. Again this will improve runtime performance, but users need to pay close attention to their code if the right side has any function calls that need to be executed regardless of the end result of the Boolean operation. This is easily fixed, though, by storing the right side expression into a temporary variable first.
Tags: Compiler, CoolBasic Classic
Framework Definition Files
The purpose
Ok, so I think it’s finally time to introduce something I’ve been working on for the past several weeks. This is actually one of the two “bigger” things that I referred to in my previous blog post. As I’ve been hinting a couple of times already, one of our strategies is to design our future coming game making tools in such a way that they’re as modular as possible. That is, we want the languages (such as CoolBasic Classic) to be separate from game engines (such as Cool VES) so that a language is merely a tool to write programs and the user can choose the target framework that best suits his/her needs. This also dictates that language compilers should not be directly dependent from the target execution system (read Virtual Machine). CoolBasic Classic, as a language, thus doesn’t really know about the functions and constants or any other type of symbols that are available in, say, Cool VES game engine. For example, unless explicitly specified, you couldn’t call LoadObject or PlaySound from CoolBasic Classic source, unless you wrote those functions yourself.
So you have to tell somehow to CoolBasic Classic compiler that these functions exist and how to invoke them. What happens under the hood, is that instead of executing CoolBasic source, a game engine (Cool VES) procedure shall be invoked. This is where the Framework Definition Files come in handy. As a part of Cool VES installation, this file is provided. The definition file is passed to the compiler during build operation. It contains a list of function and constant symbols that will get imported to the compiler’s symbol table to begin with. In this way, the compiler is able to recognize those services provided by the target framework, in our case Cool VES game engine.
There are two types of definition files: one is XML and the other is binary. They both host the exact same information about symbols. The XML format is intended for easy reading, and it can be consumed by 3rd party applications. Also, Cool Developer editor reads symbol listings from the XML file so it can highlight the code properly. However, parsing XML can be a bit slowish, and doing so every time we want to build and run our game will be an overkill, simply put. Luckily, the binary format is designed for fast reading and parsing, containing offsets for fast file seeking. The data is stored there as a structured tree so that full scope info is retained. We are planning to publish the binary format specifications (yes, a public document) at some point too, but this information is, of course, subject to change.
The editor application
Needless to say, we needed an editor to create and maintain these definition files. So I created a neat application for that. It’s an internal tool that the DevTeam uses, thus it’s not available to public. This hasn’t prevented it ending up being a quality piece of software, though, and I’ve actually put a bit more time on it than I originally planned. But that just makes a point, eh? We drive for quality! Quality takes time, but in the end, it’ll be worth it. This is how the editor looks:
It’s been written in C# (.NET framework 4.0), and that cool dark appearance (something like that will be in Cool Developer’s theme collection, too) is the Expression Dark theme, from Microsoft (and it’s also available for free at codeplex.com).
The video
As a bonus, I decided to try something new, and managed to conjure up something I think can be considered a video. That’s right, I made my first video ever, and put it on YouTube! I have absolutely zero skill and experience in video recording, editing, encoding, and presentation in general. It’s narrated in english (which is not my primary language), and I found out the hard way just exactly how difficult making a video like this can be. Despite careful planning and preparation, I found it surprisingly hard to not begin to stutter midway or how to keep your speech fluent without any moments of disturbing silence. Before the actual presentation, I actually wrote a 2,300 word transcript, but as soon as I got started with the presentation, I noticed I actually had no time to read anything (despite I’d learned half of it by heart). So I ended up improvising it all to the end.
Making the video took me all evening, and I learned how to zoom the screen only after uploading it on YouTube. All in all, I’m pleased to the outcome, but now I know how to do it better next time. Enjoy this 15 minute long demonstration (I recommend watching it in 720p, and yes I know it’s still blurry):
There’s also another video coming, stay tuned!
Tags: Cool VES