Hey dlang community.

I've already been thinking in advance for DConf next year. If things keep going well for me, I may not have anything I can publicly talk about. So I've been thinking about what kind of capacity I can contribute to the conference.

The easy way would be to raise my hand for volunteering to be MC. I get good feedback about my humour and energy levels during my talks, and translating that to an entire event means I could probably relax for a change and not worry about making mistakes in my talk/actually getting everything ready in time.

Another thing that's becoming apparent though is that there needs to be more widely-available, simple to understand tutorials for the kind of work I do. There's Andrei's book, which is the first stop shop. And then there's Adam's D Cookbook. But I can't think of much else.

So. Being the kind of programming communist I am, I write something cool and think "I need to talk about this." Hence the posts I've made in here lately. But it also got me thinking. Maybe these kind of posts would make a great DConf talk. Or a website. Or whatever, I'm only coming in to this with the mentality of DConf next year.

This is something I've emailed to friends/posted to Facebook (but slightly sanitised so that the troll from Melbourne hiding behind a Tor connection won't derail the thread with "sex talk" complaints). It discusses stuff at a high level, with only a handful of specific examples. And I can go in to details. But keeping things understandable at a high level and using a bit of humour at the same time goes a very long way in my experience.

So if stuff like the following, but presented for a proper audience, seems like it'll be valuable, I'll add all these examples I'm writing to a presentation as I go throughout the year and submit that for my talk when the time comes.

===

I've been writing a client/server architecture. In D.

And I harp on about D, I know. But wow is it making things easy.

The kind of programming paradigms I'm using to get it down to "simple as pie usability" aren't widely known outside of the D community's best and brightest. But knowing what the language is capable of, I can just go ahead and do it and it works.

So you're a client and a server. Which means you're going to want to send and receive messages to each other. How would you do this in C++? Maybe an enum for message type so you can switch on it? Then some way of casting the data from the bytestream? And what about variable length messages? So rather than cast you need a proper deserialisation. Just send everything as JSON? Yeah, goodbye network bandwidth, but at least you don't have to mess around with binary.

The only method there that is low-maintenance for futureproofing is the JSON method I mentioned. And many people will stop there and be happy. But I'm not stopping there. I'm writing a gaming middleware, which implicitly means there's going to be large quantities of data I'll need to deal with.

So, how do you make maintenance a breeze when you write such a system from scratch?

Step one: All your messages are structs. "Value types" for you modern programmers.

Step two: Apply a user-defined attribute to your struct. I have ClientMsg for messages sent by the client, and ServerMsg for messages sent by the server. What if both client and server send the message?

@ClientMsg @ServerMsg struct Ping { }

Done.

Step three: Collect all messages from the message modules and stick them in a tuple. No need to register your message. It's done for you.

Step four: Filter that tuple out in to separate client and server message tuples.

Step five: Write a template that generates a size_t that hashes together the names of all members of any given message. This has the built-in benefit of loosely versioning your structures. And thanks to CTFE and collecting all message types earlier, we can check at compile time for hash collisions.

Step six: Serialisation of your structures starts out by sticking that size_t in a buffer, and then parsing each member of your struct and either copying in to the buffer directly or specifying a length before copying N elements from the array you've encountered. Deserialisation works in the exact opposite manner.

Step seven: In your receive function that takes a byte stream, put in a switch statement that looks a little bit like the following:

switch( msg.GetID )
{
    static foreach( Message; ServerMessages )
    {
    case ObjectIDOf!Message:
        Message deserialised = msg.FromBinary!Message;
        this.receive( deserialised );
    }
    default:
        break;
}

Yes, we basically generate at compile time an entire switch statement from structures. This currently will generate a fairly inefficient jump table, but you know, we can add an indexing declarator to that foreach statement and let the compiler optimise down to a proper jump table with a little bit more effort.

Step eight: Anyone that wants to add a message to your system will now immediately get a compile error if they don't implement void receive( ref MessageType msg ) in their server or client.

Step nine: Oh, remember Binderoo? Yeah. It can generate a C# representation from all this. So I can keep my server highly performant and write a frontend that won't make people want to send death threats to the company's Twitter account.

I'm writing this code for my start-up tech company, so being a rock solid base that is maintainable will be a key feature for reducing time and maintenance costs in the future. And honestly, I'm an expert level C++ programmer these days and know a thing or two about a few other languages, and none of the languages I'm familiar with can make things as simple as "write a struct". Getting anything like this out of C++, for example, requires you jump language hoops and then give up and write an external pre-processor.

On 04/06/2018 4:25 AM, Ethan wrote:
> Step seven: In your receive function that takes a byte stream, put in a switch statement that looks a little bit like the following:
> 
> switch( msg.GetID )
> {
>      static foreach( Message; ServerMessages )
>      {
>      case ObjectIDOf!Message:
>          Message deserialised = msg.FromBinary!Message;
>          this.receive( deserialised );
>      }
>      default:
>          break;
> }

Make that a final switch, that should generate better assembly.

On Sunday, 3 June 2018 at 16:25:29 UTC, Ethan wrote:
> Step seven: In your receive function that takes a byte stream, put in a switch statement that looks a little bit like the following:
>
> switch( msg.GetID )
> {
>     static foreach( Message; ServerMessages )
>     {
>     case ObjectIDOf!Message:
>         Message deserialised = msg.FromBinary!Message;
>         this.receive( deserialised );
>     }
>     default:
>         break;
> }

Why message detection is in receiver instead of infrastructure? And why not gather message types from receiver's interface with DbI (like WCF does)?

On Tuesday, 5 June 2018 at 13:33:18 UTC, Kagamin wrote:
> Why message detection is in receiver instead of infrastructure?

Because recursion. One of the messages I've written is a wrapper message for a multi-packet split message, and calls receive with the reconstructed byte buffer. Fairly elegant way to not special-case the thing that much.

> And why not gather message types from receiver's interface with DbI (like WCF does)?

There already is design by introspection. But I don't parse a type, I parse an entire module. The switch statement is being built through the results of an introspective pass.

This is quite deliberate. I'm writing a large-scale maintainable codebase. Having everything in one file is a sure way to reduce maintainability and thus productivity of the programmers maintaining it. Getting D to favour lots of smaller files means getting creative.

I *could* put all the messages in an interface and inherit from it... but that's a fairly old-school way of thinking. I don't need a giant virtual function table to enforce implementing message types when I can use outside-the-box introspective tricks and compile down to nothing.

There's also a design advantage to going message-first here. I'm forcing the maintainers to think of the data before they get to implementation. The existence of a struct already explicitly creates one piece of data - the message ID. Anything else you put in there is up to you. And being structs, means that you don't constantly have to maintain function signatures each time you want to add a value to a message for example.