Hello, all-

I'd like to present a library that I've been working on on-and-off for the last couple of months.

Protocol Buffers are a format produced by Google that acts as an Interface Description Language for serializable data. Basically, given a file containing lines like:

message Point {
	optional int32 x = 1 [default=166];
	required int32 y = 2;
	optional string label = 3;
	message Coord {
		required int32 a = 1;
		required int32 b = 2;
	}
}

You can get a structure that behaves as:

struct Point {
	int x=166;
	int y;
	string label;
	struct Coord {
		int a,b;
	}
}

What's the benefit? This structure also has some useful methods for serialization and deserialization to a well-documented format - in dproto's case, the resulting data can be stored in any ubyte[].

What makes this library special? Unlike the C++/Java/Python compilers for .proto files, this library does the conversion at compile time using mixins and string manipulation. All that's needed is to add the file directories to the string imports, and then:

	mixin ProtocolBuffer!"point.proto";

Where can you get it?
It's available as a Dub package ("dproto"), or on Github at
http://github.com/msoucy/dproto

dproto is licensed under the BSD 3-clause license, and I'm definitely open to suggestions for improvement - currently, a lot of the code was translated from Java and so there are some things that are quite unidiomatic.

-Matt Soucy

This is more of an FYI. I've been using/updating
https://github.com/opticron/ProtocolBuffer

Boost License
And while it doesn't have any helper functions, it can generate source at compile time.
Generates D1 code if requested

Been using it to walk OSM data for no particular reason
https://gist.github.com/JesseKPhillips/6051600

On Saturday, 5 October 2013 at 20:56:21 UTC, Matt Soucy wrote:
> message Point {
> 	optional int32 x = 1 [default=166];
> 	required int32 y = 2;
> 	optional string label = 3;
> 	message Coord {
> 		required int32 a = 1;
> 		required int32 b = 2;
> 	}
> }
>
> You can get a structure that behaves as:
>
> struct Point {
> 	int x=166;
> 	int y;
> 	string label;
> 	struct Coord {
> 		int a,b;
> 	}
> }

Should it be really like that? If you just declare Coord struct, it doesn't place a Coord instance in Point.

On 10/08/2013 04:11 AM, Kagamin wrote:
> On Saturday, 5 October 2013 at 20:56:21 UTC, Matt Soucy wrote:
>> message Point {
>>     optional int32 x = 1 [default=166];
>>     required int32 y = 2;
>>     optional string label = 3;
>>     message Coord {
>>         required int32 a = 1;
>>         required int32 b = 2;
>>     }
>> }
>>
>> You can get a structure that behaves as:
>>
>> struct Point {
>>     int x=166;
>>     int y;
>>     string label;
>>     struct Coord {
>>         int a,b;
>>     }
>> }
> 
> Should it be really like that? If you just declare Coord struct, it doesn't place a Coord instance in Point.
No, that's intentional. The idea was to show the possibility of nested structs. You can create a Coord with:

auto c = Point.Coord();

I admit the example is kind of poor, though.
-Matt Soucy

On 10/05/2013 06:20 PM, Jesse Phillips wrote:
> This is more of an FYI. I've been using/updating https://github.com/opticron/ProtocolBuffer
> 
> Boost License
> And while it doesn't have any helper functions, it can generate source
> at compile time.
> Generates D1 code if requested
> 
> Been using it to walk OSM data for no particular reason https://gist.github.com/JesseKPhillips/6051600
Thanks for the tip - I actually did find this one when I started using this, however I found it on another page that hadn't been updated (that I ironically can't find now), so I wrongfully assumed that it was abandoned. I did find the github page, but I wasn't thrilled by it.

Pros for opticron's:
	Much more idiomatic code
	Supports D1
	Much better unit tests
Pros for mine (highly based on opinion):
	Exposes slightly more convenient helpers
	(In my opinion) slightly easier to read
	Generated code looks like a POD struct, though with some "hidden"
attributes (optional variables support .clean(), .exists(), instead of
looking like myStruct.foo_exists())

Basically, mine is much more simplistic, which is both good and bad. I'd like to improve mine a bit more, and the one you pointed me at is pretty good. It gave me some things to improve on.

On Tuesday, 8 October 2013 at 08:11:34 UTC, Kagamin wrote:
> On Saturday, 5 October 2013 at 20:56:21 UTC, Matt Soucy wrote:
>> message Point {
>> 	optional int32 x = 1 [default=166];
>> 	required int32 y = 2;
>> 	optional string label = 3;
>> 	message Coord {
>> 		required int32 a = 1;
>> 		required int32 b = 2;
>> 	}
>> }
>>
>> You can get a structure that behaves as:
>>
>> struct Point {
>> 	int x=166;
>> 	int y;
>> 	string label;
>> 	struct Coord {
>> 		int a,b;
>> 	}
>> }
>
> Should it be really like that? If you just declare Coord struct, it doesn't place a Coord instance in Point.

It is correct. A message defined in another is just namespacing as it is in D. If you want to include the type then the message needs a field of that type. Groups are a way to combine this, but they are deprecated so who'd want to support that.

On Tuesday, 8 October 2013 at 20:50:22 UTC, Matt Soucy wrote:
> Thanks for the tip - I actually did find this one when I started using
> this, however I found it on another page that hadn't been updated (that
> I ironically can't find now), so I wrongfully assumed that it was
> abandoned. I did find the github page, but I wasn't thrilled by it.

You're thinking: https://256.makerslocal.org/wiki/index.php/ProtocolBuffer
I've been updating it from that

>
> Pros for opticron's:
> 	Much more idiomatic code
> 	Supports D1
> 	Much better unit tests
> Pros for mine (highly based on opinion):
> 	Exposes slightly more convenient helpers
> 	(In my opinion) slightly easier to read
> 	Generated code looks like a POD struct, though with some "hidden"
> attributes (optional variables support .clean(), .exists(), instead of
> looking like myStruct.foo_exists())

For the D2 output I've got it really close to POD. It declares fields as Nullable!T meaning myStruct.foo.isNull would check availability. I was intending to distinguish required/optional with this, but that would remove error checking (which I don't think is there yet).

> Basically, mine is much more simplistic, which is both good and bad. I'd
> like to improve mine a bit more, and the one you pointed me at is pretty
> good. It gave me some things to improve on.

Go for it, have fun.

On 10/09/2013 01:55 AM, Jesse Phillips wrote:
> Groups are a way to combine this, but they are deprecated so who'd want to support that.

Exactly, I didn't even bother planning out support for them.

-- 
Matt Soucy
http://msoucy.me/

On 10/09/2013 10:38 AM, Jesse Phillips wrote:
> On Tuesday, 8 October 2013 at 20:50:22 UTC, Matt Soucy wrote:
>> Thanks for the tip - I actually did find this one when I started using this, however I found it on another page that hadn't been updated (that I ironically can't find now), so I wrongfully assumed that it was abandoned. I did find the github page, but I wasn't thrilled by it.
> 
> You're thinking: https://256.makerslocal.org/wiki/index.php/ProtocolBuffer I've been updating it from that
> 
Exactly, that's the page! All I remembered about the location was that it was a wiki with a number in the URL...
>>
>> Pros for opticron's:
>>     Much more idiomatic code
>>     Supports D1
>>     Much better unit tests
>> Pros for mine (highly based on opinion):
>>     Exposes slightly more convenient helpers
>>     (In my opinion) slightly easier to read
>>     Generated code looks like a POD struct, though with some "hidden"
>> attributes (optional variables support .clean(), .exists(), instead of
>> looking like myStruct.foo_exists())
> 
> For the D2 output I've got it really close to POD. It declares fields as Nullable!T meaning myStruct.foo.isNull would check availability. I was intending to distinguish required/optional with this, but that would remove error checking (which I don't think is there yet).
> 
>> Basically, mine is much more simplistic, which is both good and bad. I'd like to improve mine a bit more, and the one you pointed me at is pretty good. It gave me some things to improve on.
> 
> Go for it, have fun.

-- 
Matt Soucy
http://msoucy.me/