Thread overview
Wrote a Protobuf-like D (de)serialisation library - asking for comments / suggestions
Jun 12, 2021
Sinisa Susnjar
Jun 13, 2021
Ali Çehreli
Jun 14, 2021
Sinisa Susnjar
Jun 14, 2021
Imperatorn
Jun 14, 2021
Sinisa Susnjar
June 12, 2021

Hi, D newbie here,

The library is on par with Google Protobuf performance-wise and does not need a pre-compiler like Protobuf does, but instead uses the meta programming facilities of D to (de)serialise D data types from/to binary messages. Some samples and unit tests are included.

Here's the code if anyone wants to have a look: https://github.com/sinisa-susnjar/msgbuf

(I am sure there is a lot of room for improvement)

June 12, 2021
On 6/12/21 2:59 PM, Sinisa Susnjar wrote:

> instead uses the meta programming facilities of D to (de)serialise D data types from/to binary messages.

Yeah! :)

I did the same at work.

> (I am sure there is a lot of room for improvement)

Without reading your code carefully, I can think of two optimizations that may prove to be useful:

- If a struct contains all bitwise copyable members, instead of (de)serializing each member individually, the whole struct can by memcpy'ed. This may be a performance gain especially for arrays of structs: You can memcpy the whole array at once.

- Instead of allocating memory for each value (e.g. for arrays with 'new'), you can maintain a function-static buffer and reuse it, allocating only as the current buffer is not large enough for new data. (No worries with multi-threading because each thread will have its own function-static buffer.)

Ali

June 14, 2021

On Saturday, 12 June 2021 at 21:59:13 UTC, Sinisa Susnjar wrote:

>

Hi, D newbie here,

The library is on par with Google Protobuf performance-wise and does not need a pre-compiler like Protobuf does, but instead uses the meta programming facilities of D to (de)serialise D data types from/to binary messages. Some samples and unit tests are included.

Here's the code if anyone wants to have a look: https://github.com/sinisa-susnjar/msgbuf

(I am sure there is a lot of room for improvement)

Is it on dub?

June 14, 2021
On Sunday, 13 June 2021 at 00:18:56 UTC, Ali Çehreli wrote:

> - If a struct contains all bitwise copyable members, instead of (de)serializing each member individually, the whole struct can by memcpy'ed. This may be a performance gain especially for arrays of structs: You can memcpy the whole array at once.
>
> - Instead of allocating memory for each value (e.g. for arrays with 'new'), you can maintain a function-static buffer and reuse it, allocating only as the current buffer is not large enough for new data. (No worries with multi-threading because each thread will have its own function-static buffer.)
>
> Ali

Thanks for the suggestion!
June 14, 2021

On Monday, 14 June 2021 at 07:14:27 UTC, Imperatorn wrote:

>

Is it on dub?

published it just now :)