With the NewEngine project having been in development for over a year and a half, certain parts of the codebase have aged rather poorly.
For the past month or so, I've been working towards bringing the entire codebase to the same level of quality.
Until now, the extraction process would perform possibly tens of thousands of disk writes.
Normally this isn't a problem - even my Windows 2000 Compaq machine can extract a 10.000 line archive in ~8 seconds.
However, more recent versions of Windows come with Windows Defender.
When a Batch script attempts to write to a file on disk, Windows Defender's "realtime protection" will interrupt the write to ensure it's not malicious.
And again, normally this is fine, however in this case where extraction performs such a large amount of writes, extraction speed ends up being ~12x slower.
This issue can be worked around by disabling realtime protection entirely, although this is obviously not ideal.
The fix for this is batching writes together in chunks of ~1024 characters.
After batching was implemented, extraction speed dropped dramatically, now being only ~10% slower than with realtime protection disabled.
Previously, object loading was a complete mess.
One of the first scripts I created when this project was first started was object creation.
After a year and a half of conditions and systems changing, this script was full of nasty bugs and context specific hacks.
In addition to this, the script was both ugly and slow.
Object creation is now entirely rewritten, with the line count being brought down from 297 to just 87 (-70% in size!)
Due to some internal changes, I've decided to slightly alter the syntax of the createObject API.
Previously, createObject someObject ; someProperty=aValue ; anotherProperty=anotherValue would've been sufficient.
The correct way to write this now would be createObject name=someObject ; someProperty=aValue ; anotherProperty=anotherValue.
A big limitation of the old object rendering system, is how the renderInto property was exclusively reserved for rendering sprites relative to viewports.
This was due to the renderer being incredibly hardcoded and unflexible from an internal point of view.
Adding to the list of issues this system had, was a complete lack of edge culling.
Even if a sprite was outside the viewport, it would still be rendered.
You can see this issue happening in the blog post for State of Development under "Entry 4: Object sprite rendering relative to viewports".
With the rewritten object renderer, any object may be rendered relative to any other object with culling fully supported.
I'm not sure of the extent of what can be done using this system, although to give you a rough idea, all the following are possible by just changing the focusObject and renderInto properties:
- Viewports inside viewports inside viewports (infinitely nestable)
- Buttons inside viewports being rendered relative to a viewport focusing a player
- Viewports with sprite or text labels
It may not be apparent at first what the purpose of such a system, so I'll provide some examples:
- A room with of buttons clickable via mouse
- Screen transition effects by manipulating viewport properties
- Jumbotrons and televisions
- Cutscenes
Certain actions such as loading sprites or generating text labels were previously hardcoded and repeated in many different places.
This could potentially lead to bugs where data is only considered properly formatted and usable to certain APIs.
Having the same resource fail to load only in certain cases is confusing and leads to a bad experience for developers making games using NewEngine.
Anything which needs to load tile data such as loadLevel or setTile will now call loadTile.bat.
Having loading of a certain kind of resource be performed by a single-purpose script is beneficial for performance, codebase size, and ensures consistency.
Due to how the Batch interpreter works internally, performance degrades as memory usage increases.
For starters, certain scripts will set or load temporary data.
Any instance of a script creating temporary variables will now undefine said temporary variables before returning execution.
The above is only true for scripts found outside main.bat.
Spending the time to unload variables just to redefine them shortly after would be slower than not unloading.
The solution to this would be using generic variable names which may be overwritten later, such as num1, num2, string1, etc.
Microsoft introduced yet another problem into the Batch interpreter, requiring a workaround.
In Windows 11, a script started with start /b will refuse to properly return control to the parent when told to exit.
However, this behavior only occurs if the SystemRoot variable is undefined.
When NewEngine starts up, all unnecessary variables are undefined to save memory.
The exceptions to being unloaded are the following:
args ComSpec engineRuntimeVersion initProgress isModernRelease osArch systemRoot
In addition to this, I've manually tested NewEngine on almost every release of Windows from 2000 and up.
The full list of testing is as follows:
Windows 2000 (32bit)
Windows XP (32bit)
Windows Vista (64bit)
Windows 7 (64bit)
Windows 8 (32bit)
Windows 10 (64bit, 1507)
Windows 10 (64bit, 1709)
Windows 10 (64bit, 1803)
Windows 10 (64bit, latest)
Windows 11 (64bit, latest)
Windows 2000 has been tested on bare metal.
In fact, this entire blog post was written exclusively on the Compaq nc6400 running Windows 2000 on bare metal.
