May 18, 2023

On Thursday, 18 May 2023 at 01:31:21 UTC, Jacob Shtokolov wrote:

>
  1. Have you looked at the IOPipe library?

Yeah, I talked with schveiguy on the discord server for a bit; it honestly looks like a better concept than what I'm doing, lol. But I didn't notice it until yesterday. So maybe I should focus my efforts there? I don't know yet.

>
  1. What are the main benefits over the existing Ranges concept? Say, given your example:
import streams;

void readFileTo(S)(string filename, S stream) if (isOutputStream!(S, ubyte)) {
    import std.stdio;
    auto fIn = FileInputStream(filename);
    transferTo(fIn, stream);
}

With ranges, this would look something like:

import std;

void readFileTo(S)(string filename, ref S stream) if (isOutputRange!(S, ubyte[])) {
    File(filename).byChunk().copy(stream);
}

Which is more or less the same.

I would say that there isn't really a big benefit over using ranges in terms of how they're expressed, but more that I think (and I may be wrong) that streams are a simpler, easier concept for programmers to grasp, especially those that are migrating to D from some other language that uses a similar stream concept.

Another benefit is that, as pointed out by Guillame Pilot in another thread, phobos ranges (and most of phobos for that matter) has no real convention for naming schemes, and they generally don't try to be betterC-compatible, so it makes it difficult to use them in any low-level code.

Finally, my streams allow the code to handle errors more gracefully without needing exceptions, which isn't always convenient to do with ranges.

But you're right; to the average D programmer, streams are just a different flavor of accomplishing the same thing.

>
  1. In the README you write:
Phobos' concept of an Input Range relies on implicit buffering of results... This doesn't map as easily to many low-level resources

AFAIK, the read/write buffers are anywhere, except, probably, sendfile() and some combination of mmap and write. But I'm struggling to get how this streams concept maps onto sendfile as well.

Yeah, I suppose what I was trying to say, is that this library puts the programmer in more control of if and when buffers are allocated and used with IO. But of course for sendfile there's no need for streams.

May 22, 2023

On Thursday, 18 May 2023 at 16:10:44 UTC, Andrew wrote:

>

So maybe I should focus my efforts there? I don't know yet.

There is definitely a need for a good set of functions that are interchangeable between different file types in Phobos: for instance, Socket seems to have no such primitive as File.byChunk(), etc.

So if we can have this kind of functionality somehow compatible with the built-in std.algorithm and specifically for IO, that would be really cool!

What's your nickname in Discord, BTW? I feel like there are multiple on-going efforts from different people targeting the same core concept: the easy-to-use IO operations. Would be nice to exchange some ideas!

May 22, 2023

On 5/22/23 6:49 AM, Jacob Shtokolov wrote:

>

On Thursday, 18 May 2023 at 16:10:44 UTC, Andrew wrote:

>

So maybe I should focus my efforts there? I don't know yet.

More effort on iopipe is always welcome! The library needs a lot of polish.

>

There is definitely a need for a good set of functions that are interchangeable between different file types in Phobos: for instance, Socket seems to have no such primitive as File.byChunk(), etc.

Yeah, this is the intention of iopipe. Once you get to a buffer, you can use whatever you want on it. The idea is that I don't have to care whether it's a socket, file, or memory buffer, I can run my e.g. parser on it.

byChunk would be trivial to put on top of this (though there's little reason to use it in this context). I already have delimitedText and the more specific byLine pipes, which extend to the next delimiter code point (https://schveiguy.github.io/iopipe/iopipe/textpipe/delimitedText.html)

What iopipe lacks quite a bit is polish and probably a bunch of shortcuts (setting up a buffered stream is a lot more verbose than I would like). I also need to really focus on a formatting API.

>

So if we can have this kind of functionality somehow compatible with the built-in std.algorithm and specifically for IO, that would be really cool!

My next focus is async i/o. That is a precursor to what I really want to create -- a web server/framework. It's unfortunately slow going though, as this is spare time project for me.

>

What's your nickname in Discord, BTW? I feel like there are multiple on-going efforts from different people targeting the same core concept: the easy-to-use IO operations. Would be nice to exchange some ideas!

I believe Andrew's Discord is pretty straightforward. Also feel free to ping me if you want to discuss (schveiguy).

-Steve

1 2
Next ›   Last »