Thread overview
Ported js13k game underrun to D targeting webassembly
Nov 17, 2018
Sebastiaan Koppe
Nov 18, 2018
blahness
Nov 18, 2018
Sebastiaan Koppe
Nov 18, 2018
Sebastiaan Koppe
Nov 19, 2018
Dukc
Nov 19, 2018
Sebastiaan Koppe
Nov 19, 2018
Dukc
November 17, 2018
Underrun is small game build by Dominic Szablewski for the 2018 js13kGames competition.

I decided to port it to D and to target webassembly. You can play the game here https://skoppe.github.io/spasm/examples/underrun/

It is part of my endeavour into the wonderful world of webassembly. I specifically choose this project so I could get a feel for how to call web api's other than the dom, i.e. WebGL and AudioContext.

Porting went pretty smooth, except for the fact that I wanted it to be done in betterC, which turned out to cripple lots of things I am used to rely on.

Things like:

- having to write closures by hand
- not being able to use stdx.allocator
- almost no use of phobos (even at CTFE)

D could definitely use some love in that area. I was specifically stumped that stdx.allocator doesn't work in betterC; if somewhere is a good place to use stdx.allocator you would expect it to be betterC, but alas.

Had lots of fun though. Hope you like it.
November 18, 2018
On Saturday, 17 November 2018 at 21:35:59 UTC, Sebastiaan Koppe wrote:
> Underrun is small game build by Dominic Szablewski for the 2018 js13kGames competition.
>
> I decided to port it to D and to target webassembly. You can play the game here https://skoppe.github.io/spasm/examples/underrun/

Good stuff. Any reason why the music & some of the sound effects are missing? The text when you bring a system back online is corrupted. This makes it hard to know what the goal is if you haven't played the original.



November 18, 2018
On Sunday, 18 November 2018 at 10:33:14 UTC, blahness wrote:
> Good stuff. Any reason why the music & some of the sound effects are missing?
Yep, I have only ported the most essential sound effects. The music I skipped entirely. It wouldn't be much work since the audio api / buffers are already there. I just didn't want to spend more time on an example.

> The text when you bring a system back online is corrupted.
I know, I have looked at it for quite a while and I just can't figure out why it is broken.

November 18, 2018
On 11/17/18 4:35 PM, Sebastiaan Koppe wrote:
> Underrun is small game build by Dominic Szablewski for the 2018 js13kGames competition.
> 
> I decided to port it to D and to target webassembly. You can play the game here https://skoppe.github.io/spasm/examples/underrun/
> 
> It is part of my endeavour into the wonderful world of webassembly. I specifically choose this project so I could get a feel for how to call web api's other than the dom, i.e. WebGL and AudioContext.
> 
> Porting went pretty smooth, except for the fact that I wanted it to be done in betterC, which turned out to cripple lots of things I am used to rely on.
> 
> Things like:
> 
> - having to write closures by hand
> - not being able to use stdx.allocator
> - almost no use of phobos (even at CTFE)
> 
> D could definitely use some love in that area. I was specifically stumped that stdx.allocator doesn't work in betterC; if somewhere is a good place to use stdx.allocator you would expect it to be betterC, but alas.
> 
> Had lots of fun though. Hope you like it.

Really cool. I didn't play the original, but I found this reasonably straightforward.

I love this this webassembly stuff, I think it's a fantastic demonstration, and test for the power of D.

A system to do closures by hand should be doable with templates I would think. Do you have a quick example?

-Steve
November 18, 2018
On Sunday, 18 November 2018 at 19:51:06 UTC, Steven Schveighoffer wrote:
> Really cool. I didn't play the original, but I found this reasonably straightforward.
>
> I love this this webassembly stuff, I think it's a fantastic demonstration, and test for the power of D.

Thanks, I think so too. Although there are definitely some hurdles to overcome.

> A system to do closures by hand should be doable with templates I would think. Do you have a quick example?

For example https://github.com/skoppe/spasm/blob/master/example/underrun/source/game/terminal.d#L259
The whole Handler struct is just to call `terminal_run_story` after `terminal_write_text` completes.

One thing that bugs me is that anything that allocates on the GC - like the closures here - can't be hooked into to call a custom allocator. That includes things like std.array.Appender, AA's, etc.
November 19, 2018
On Saturday, 17 November 2018 at 21:35:59 UTC, Sebastiaan Koppe wrote:
> [snip]

I had a look at your code, and just now I realized that spasm can really call JavaScript without any glue from JavaScript side.

This is huge! Now I can start to think about expanding my usage of D at the web page I'm developing, potentially replacing parts of my application made with Bridge.NET.

I know you did announce about Spasm earlier and I should have thanked you already. My fault: I realized that you made something that enables one to generate WebAsm, but didn't realize it enabled calling JS classes directly. Should have looked closer.

> Things like:
> - almost no use of phobos (even at CTFE)
>
> D could definitely use some love in that area. I was specifically stumped that stdx.allocator doesn't work in betterC; if somewhere is a good place to use stdx.allocator you would expect it to be betterC, but alas.

I agree. I, neither, have managed to link allocators in. But I quess you can use Phobos more than you think. Seeing how great stuff you have done, It may be that what I'm stating here may already be plain obvious to you, but I say it just in case anyway. You need a way to link in non-templated code. Two way's I'm aware of:

1. Manually compile your code and parts of Druntime/Phobos targetting 32-bit X86, link them manually with llvm-link using --only-needed and only then compile to JavaScript (Haven't tried with WebAsm, but theoretically should compile the same way). This way you don't need to manually stub what you don't use -well, when the linkers have same ideas about what you're calling, which's not always the case.

2: Use Vladimir's DScripten. I haven't tried, as I was limited by not being aware of anything like spasm to call Web API, but probably easier than my method above and brings larger portions of Phobos into user's reach. Also IIRC Vladimir managed to get allocators working.

Even with my relatively meager method, pipeline-style programming with std.algorithm and std.range over (often static) arrays is doable and practical in my experience.

November 19, 2018
On Monday, 19 November 2018 at 11:13:52 UTC, Dukc wrote:
> I had a look at your code, and just now I realized that spasm can really call JavaScript without any glue from JavaScript side.
I am not sure where you got that impression from, but I am afraid you'll still need js glue code.

> This is huge! Now I can start to think about expanding my usage of D at the web page I'm developing, potentially replacing parts of my application made with Bridge.NET.
The only thing I did with spasm was to use a subset of the D language to define a declarative language for creating html components, and I used lots of introspection to optimise the rendering of them.

Everything else comes from the wasm backend from llvm and it's support in ldc.

I really appreciate you getting excited about it but look, Spasm is still in an early stage. The code is rough, the documentation poor. And it has plenty missing that needs to be fixed before you should consider writing anything serious with it. The two most important being bindings to all web apis and memory deallocation.

Having said that, of course I want you to use it :)

> I know you did announce about Spasm earlier and I should have thanked you already. My fault: I realized that you made something that enables one to generate WebAsm, but didn't realize it enabled calling JS classes directly. Should have looked closer.
The way calling js classes from D (or any language out there) works, is to insert the js object in a big associative array at the js side with an int as the key. The int (handle) is passed to D. Anytime D wants to call a function or retrieve a property of that class it calls some glue code with the handle as first parameter. In the javascript glue code the handle is used to lookup the object and the corresponding function is called.

Since javascript allows you to call by string (e.g. `object["foobar"]()`), you can minimize glue code at the expense of decoding utf-8 strings.

> 1. Manually compile your code and parts of Druntime/Phobos targetting 32-bit X86, link them manually with llvm-link using --only-needed and only then compile to JavaScript (Haven't tried with WebAsm, but theoretically should compile the same way). This way you don't need to manually stub what you don't use -well, when the linkers have same ideas about what you're calling, which's not always the case.
That might work, but for the end-user it really needs to be as simple as `dub build`.

> 2: Use Vladimir's DScripten. I haven't tried, as I was limited by not being aware of anything like spasm to call Web API, but probably easier than my method above and brings larger portions of Phobos into user's reach. Also IIRC Vladimir managed to get allocators working.
Yep. The prototype version of spasm ran on top of dscripten-tools. I decided to move to native llvm wasm compilation instead, since I don't particularly like the emscripten toolchain.

> Even with my relatively meager method, pipeline-style programming with std.algorithm and std.range over (often static) arrays is doable and practical in my experience.
As long as std.algorithms/ranges don't allocate, they work.
November 19, 2018
On Monday, 19 November 2018 at 13:52:28 UTC, Sebastiaan Koppe wrote:

> I am not sure where you got that impression from, but I am afraid you'll still need js glue code.

I met the type JsHandle, found it's definiton and saw that it's only an alias for an int. I did remember seeing something similar in val.h (the emscripten header that's for using cpp to interface with JS classes) so I thought that you're calling the javascript interface directly with some knowledge how the function ABI looks like internally


> I really appreciate you getting excited about it but look, Spasm is still in an early stage. The code is rough, the documentation poor.

So is my code that translates to JavaScript. Pretty much anything is an improvement over having to write a wrapper functions in JS that can only use ints. So it's not poor for me!

>
> Having said that, of course I want you to use it :)
>

:). With good likelihood I will. I just need to find a good time and target for first translation attempt.


>> I know you did announce about Spasm earlier and I should have thanked you already. My fault: I realized that you made something that enables one to generate WebAsm, but didn't realize it enabled calling JS classes directly. Should have looked closer.
> The way calling js classes from D (or any language out there) works, is to insert the js object in a big associative array at the js side with an int as the key. The int (handle) is passed to D. Anytime D wants to call a function or retrieve a property of that class it calls some glue code with the handle as first parameter. In the javascript glue code the handle is used to lookup the object and the corresponding function is called.
>
> Since javascript allows you to call by string (e.g. `object["foobar"]()`), you can minimize glue code at the expense of decoding utf-8 strings.

Not quite as impressive I thought but still a clear improvement over what I do. It still sounds like one can do some light interfacing reasonably with your method. After all, it's always possible to hide those decoding things behind some interface that prematurely decodes the used function names.

>
>> 1. Manually compile your code and parts of Druntime/Phobos targetting 32-bit X86, link them manually with llvm-link using --only-needed and only then compile to JavaScript (Haven't tried with WebAsm, but theoretically should compile the same way). This way you don't need to manually stub what you don't use -well, when the linkers have same ideas about what you're calling, which's not always the case.
> That might work, but for the end-user it really needs to be as simple as `dub build`.

Definitely a big downside if making a library, for sure.

> I decided to move to native llvm wasm compilation instead, since I don't particularly like the emscripten toolchain.

It feels buggy, true.