My use case is sending data to a socket.

One part of my program generates blocks of bytes, and the socket part tries to send them to the socket and then removes from the queue the number that got sent.

I am currently using a byte[] and using concatenation and slicing to maintain the queue but this seems like it will do many unnecessary copies, allocations etc. than are needed. I would do much better to maintain a fixed size buffer and maintain read and write positions etc.

I'm happy to write my own fixed sized buffer queue for this, but just wanted to ask if there was anything in the standard library that could be used to do this so I'm not reinventing it all? I don't need thread safety for this case. (I know i could probably use vibe-d to implement my whole socket sender but don't want to in this case for .... reasons)

On Monday, 8 October 2018 at 09:39:55 UTC, John Burton wrote:
> My use case is sending data to a socket.
>
> One part of my program generates blocks of bytes, and the socket part tries to send them to the socket and then removes from the queue the number that got sent.
>
> [...]

Try searching for "circular buffer". I'm sure http://code.dlang.org/packages/iopipe has them in some form but I can't find them with a cursory search.

On 10/8/18 6:31 AM, Nicholas Wilson wrote:
> On Monday, 8 October 2018 at 09:39:55 UTC, John Burton wrote:
>> My use case is sending data to a socket.
>>
>> One part of my program generates blocks of bytes, and the socket part tries to send them to the socket and then removes from the queue the number that got sent.
>>
>> [...]
> 
> Try searching for "circular buffer". I'm sure http://code.dlang.org/packages/iopipe has them in some form but I can't find them with a cursory search.

I called it a ring buffer:

http://schveiguy.github.io/iopipe/iopipe/buffer/RingBuffer.html

A couple notes here. The RingBuffer currently only works on Posix systems (I haven't had the motivation to dig into doing it on Windows, even though the docs mention VirtualAlloc). Also, the typical way one would use the RingBuffer is using rbufd in http://schveiguy.github.io/iopipe/iopipe/bufpipe/rbufd.html. But In the OP's case, if you aren't using iopipes to build a pipeline, that may prove more confusing than is worth (iopipe uses pull mechanisms exclusively, with somewhat novel mechanisms to enable buffered output). The RingBuffer type itself could be an easy-to-use mechanism for his use case.

Let me know if you decide to use it and need help. I'm always looking for more use cases for iopipe!

-Steve

On Monday, 8 October 2018 at 10:31:33 UTC, Nicholas Wilson wrote:
>
> Try searching for "circular buffer". I'm sure http://code.dlang.org/packages/iopipe has them in some form but I can't find them with a cursory search.

https://github.com/dlang-community/containers/blob/master/src/containers/cyclicbuffer.d

On Monday, 8 October 2018 at 09:39:55 UTC, John Burton wrote:
> My use case is sending data to a socket.
>

We have ported some containers from JAVA.

ByteBuffer is a basic container interface and widely used in JAVA.

See also:
https://github.com/huntlabs/hunt/blob/master/source/hunt/container/ByteBuffer.d
https://github.com/huntlabs/hunt/tree/master/examples/ContainerDemo/source

On Monday, 8 October 2018 at 09:39:55 UTC, John Burton wrote:
> I would do much better to maintain a fixed size buffer and maintain read and write positions etc.
>

Perhaps https://github.com/AuburnSounds/Dplug/blob/master/core/dplug/core/ringbuf.d#L16

On Sunday, 14 October 2018 at 13:07:30 UTC, Heromyth wrote:
> On Monday, 8 October 2018 at 09:39:55 UTC, John Burton wrote:
>> My use case is sending data to a socket.
>>
>
> We have ported some containers from JAVA.
>
> ByteBuffer is a basic container interface and widely used in JAVA.
>
> See also:
> https://github.com/huntlabs/hunt/blob/master/source/hunt/container/ByteBuffer.d
> https://github.com/huntlabs/hunt/tree/master/examples/ContainerDemo/source

Thanks for this, and everyone elses comments. This looks to be close to what I need.