> Do we even have protobuf package?

https://github.com/msoucy/dproto
it could receive some attention, there are pending issues

for RPC I've been using msgpackrpc since no gRPC was available. But would be nice to have gRPC.

NOTE: capnproto is a very interesting newer alternative to protobuf;
https://github.com/capnproto/capnproto-dlang shows:
Missing RPC part of Cap'n Proto.
helping out capnproto project (esp around RPC) could be another idea.

we definitely need a good way to do RPC in D, otherwise hard to integrate D with other services.

> I would consider them awful in a sense that there is no foundation to build them on. At best it will be a self-serving artifact poorly fitting with anything else.

But it would enable using D in places that were not previously possible (integrating with services); we could imaging providing a (semi) stable interface for grpc in D code and change implementation to use better foundations later




On Sun, Jan 21, 2018 at 9:54 PM, Dmitry Olshansky via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Monday, 22 January 2018 at 04:40:53 UTC, Andrew Benton wrote:
>>
>> On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote:
>>>
>>> I know a project where D could benefit from gRPC in D, which is not among the supported languages:
>>>
>>>   https://grpc.io/docs/
>>>
>>> Do you think gRPC support is worth adding to GSOC 2018 ideas?
>>>
>>>   https://wiki.dlang.org/GSOC_2018_Ideas
>>>
>>> Ali
>>
>>
>> An http/2 and gRPC solutions is probably necessary with tools like linkerd, envoy, and istio if D wants to be competitive in service mesh and distributed applications.
>>
>> http/2 and/or gRPC are both excellent ideas for GSoC 2018.
>
>
> I would consider them awful in a sense that there is no foundation to build them on. At best it will be a self-serving artifact poorly fitting with anything else.
>
> There is not even a standard way on handling IO as of yet.
> Basically do we want fiber-aware IO or blocking IO or explicit async with
> future/promise?
>
> Do we even have protobuf package?
>
>

On Monday, 22 January 2018 at 05:54:31 UTC, Dmitry Olshansky wrote:
> On Monday, 22 January 2018 at 04:40:53 UTC, Andrew Benton wrote:
>> On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote:
>>> I know a project where D could benefit from gRPC in D, which is not among the supported languages:
>>>
>>>   https://grpc.io/docs/
>>>
>>> Do you think gRPC support is worth adding to GSOC 2018 ideas?
>>>
>>>   https://wiki.dlang.org/GSOC_2018_Ideas
>>>
>>> Ali
>>
>> An http/2 and gRPC solutions is probably necessary with tools like linkerd, envoy, and istio if D wants to be competitive in service mesh and distributed applications.
>>
>> http/2 and/or gRPC are both excellent ideas for GSoC 2018.
>
> I would consider them awful in a sense that there is no foundation to build them on. At best it will be a self-serving artifact poorly fitting with anything else.

I think it would likely be built on top of vibe or collie instead of being standalone, but I see your point.

> There is not even a standard way on handling IO as of yet.
> Basically do we want fiber-aware IO or blocking IO or explicit async with future/promise?

You're right.  It's definitely a problem.  Given ease of use and the learning curve, I'd be in favor of fiber-aware IO or async/await types.

> Do we even have protobuf package?

We have at least two.  dproto is a package that supports proto2.  I've used it before and it's really nice.  There's also the eponymous protobuf package, which integrates with protoc and can also do proto3.  I haven't used it yet.

On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote:
> I know a project where D could benefit from gRPC in D, which is not among the supported languages:
>
>   https://grpc.io/docs/
>
> Do you think gRPC support is worth adding to GSOC 2018 ideas?
>
>   https://wiki.dlang.org/GSOC_2018_Ideas
>
> Ali

I can share a fresh experience from mentoring a student in a similar (also RPC) project last summer. We built native D-Bus bindings in D based on libasync. The student had had no previous experience with D or RPC, and within ~2.5 months of focused work she implemented the following:

1. (de)serialization of all D-Bus data types, including the use of compile-time reflection to recursively marshall structs, arrays, and variants. Except Variant, for which we decided to make our own D-Bus-specific tagged union type, all other D-Bus types are mapped to built-in D types.
2. A class to connect to the bus via libasync sockets, read the incoming messages and dispatch them to the registered handlers, and send messages to the bus.
3. Proxy (client) and server class templates that generate all the code necessary to make the remote calls look almost like local ones (the return value/out arguments are passed to a delegate that handles the output instead of being returned synchronously).

So, more or less an equivalent of vibe.d's REST interface generator, only with fewer customization points.

There were still some opportunities for refactorings and optimizations, so I wouldn't consider it production ready. Also, some planned features weren't implemented, such as a more convenient way for handling signals or allowing transports other than unix sockets on libasync. On the other hand, what we have is almost 100% covered with unit tests. This not only made adding successive layers quite pleasant, as little (if any) debugging of previously written stuff was ever necessary, but also helps to keep the stuff working as we modify it.

Based on my experience, I'd say that implementing gRPC may be of a right size for a GSoC project, as long as you split it into smaller components/layers, prioritize them, and focus on having at least the basic stuff usable and tested, instead of expecting it to cover all usage cases and be heavily optimized.

for grpc, we should add to dproto (which is pretty good protobuf library for D but lacks grpc) instead of starting from scratch, see https://github.com/msoucy/dproto/issues/113 [your advice/opinions on integrating with grpc?]


On Mon, Jan 22, 2018 at 12:24 PM, Adrian Matoga via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote:
>>
>> I know a project where D could benefit from gRPC in D, which is not among the supported languages:
>>
>>   https://grpc.io/docs/
>>
>> Do you think gRPC support is worth adding to GSOC 2018 ideas?
>>
>>   https://wiki.dlang.org/GSOC_2018_Ideas
>>
>> Ali
>
>
> I can share a fresh experience from mentoring a student in a similar (also RPC) project last summer. We built native D-Bus bindings in D based on libasync. The student had had no previous experience with D or RPC, and within ~2.5 months of focused work she implemented the following:
>
> 1. (de)serialization of all D-Bus data types, including the use of
> compile-time reflection to recursively marshall structs, arrays, and
> variants. Except Variant, for which we decided to make our own
> D-Bus-specific tagged union type, all other D-Bus types are mapped to
> built-in D types.
> 2. A class to connect to the bus via libasync sockets, read the incoming
> messages and dispatch them to the registered handlers, and send messages to
> the bus.
> 3. Proxy (client) and server class templates that generate all the code
> necessary to make the remote calls look almost like local ones (the return
> value/out arguments are passed to a delegate that handles the output instead
> of being returned synchronously).
>
> So, more or less an equivalent of vibe.d's REST interface generator, only with fewer customization points.
>
> There were still some opportunities for refactorings and optimizations, so I wouldn't consider it production ready. Also, some planned features weren't implemented, such as a more convenient way for handling signals or allowing transports other than unix sockets on libasync. On the other hand, what we have is almost 100% covered with unit tests. This not only made adding successive layers quite pleasant, as little (if any) debugging of previously written stuff was ever necessary, but also helps to keep the stuff working as we modify it.
>
> Based on my experience, I'd say that implementing gRPC may be of a right size for a GSoC project, as long as you split it into smaller components/layers, prioritize them, and focus on having at least the basic stuff usable and tested, instead of expecting it to cover all usage cases and be heavily optimized.
>