I need to write to a range created with outputRangeObject, then read from it. Is there a way to convert it to an input range?

On Friday, March 16, 2018 07:57:04 John Chapman via Digitalmars-d-learn wrote:
> I need to write to a range created with outputRangeObject, then read from it. Is there a way to convert it to an input range?

The output range API only supports the put function. That's it. The output range supports no way whatsoever to read data, and it has nothing to do with the input range API. As such, an output range cannot be generically converted into an input range, but some specific output ranges could be. For instance, std.array.Appender is an output range, and you get a dynamic array out of it, which would be an input range. So, if you have control over what output range you're dealing with, the simplest would be to just use Appender. If you don't have control over what output range you're dealing with, then your options depend on what the type you're dealing with is, but it won't work with fully generic code.

- Jonathan M Davis

On Friday, 16 March 2018 at 08:07:09 UTC, Jonathan M Davis wrote:
> For instance, std.array.Appender is an output range, and you get a dynamic array out of it, which would be an input range. So, if you have control over what output range you're dealing with, the simplest would be to just use Appender.

I think it's worth warning that appending on an array may invalidate all copies of it. Meaning that you should not append on an array while you're iterating over it, unless iterating by the original copy of the array. Note that foreach loop copies the range by default. Pass refRange(*array) to foreach instead.

On Saturday, March 17, 2018 06:58:04 Dukc via Digitalmars-d-learn wrote:
> On Friday, 16 March 2018 at 08:07:09 UTC, Jonathan M Davis wrote:
> > For instance, std.array.Appender is an output range, and you get a dynamic array out of it, which would be an input range. So, if you have control over what output range you're dealing with, the simplest would be to just use Appender.
>
> I think it's worth warning that appending on an array may invalidate all copies of it. Meaning that you should not append on an array while you're iterating over it, unless iterating by the original copy of the array. Note that foreach loop copies the range by default. Pass refRange(*array) to foreach instead.

Appending to a dynamic array doesn't invalidate anything. It just results in a new buffer being allocated and the dynamic array being changed to point to it instead of the dynamic array does not have enough capacity to append in place. Any other dynamic arrays which were slices of the previous buffer still refer to it and are perfectly valid.

Yes, if you want to treat a dynamic array as a reference type, it's not going to work to simply pass around a dynamic array, because dynamic arrays are pseudo-reference types and not reference types, and if you want a reference type, you're going to have to wrap the dynamic array in another type, but the fact that they're pseudo-reference types does not mean that anything is invalidated when appending.

Now, doing something like iterating over a dynamic array with a foreach loop and then appending to it in that same loop means that you're really operating on two dynamic arrays, because the loop is slices the dynamic array and is thus iterating over another dynamic array which (at least initially) refers to the same memory, and the appending that you're doing won't affect it. So, you do have to be careful about appending while iterating if you really want to be appending to the same dynamic array that you're iterating over, but everything involved will be valid regardless. It just may not have the behavior you want, because the dynamic array has been sliced.

In any case, based on the OP's question, it was my assumption that they intended to fully write to the output range and then operate on the results as on input range, which then has none of the problems associated with trying to put while you pop. If they want to put while they're popping, then things get far more complicated, and a dynamic array is probably not the best solution whether Appender is used or not.

- Jonathan M Davis

On Friday, 16 March 2018 at 07:57:04 UTC, John Chapman wrote:
> I need to write to a range created with outputRangeObject, then read from it. Is there a way to convert it to an input range?

Could you illustrate your problem a bit further?

In the literal sense, converting from an output to an input range would require using coroutines (where .empty() blocks until the output range has supplied the next element).

However, I suspect what you want to do is to write results from the output range to a buffer (e.g. an Appender) first, and after that – or possibly every so often in chunks – process the results further?

 — David

On Saturday, 17 March 2018 at 17:16:40 UTC, David Nadlinger wrote:
> On Friday, 16 March 2018 at 07:57:04 UTC, John Chapman wrote:
>> I need to write to a range created with outputRangeObject, then read from it. Is there a way to convert it to an input range?
>
> Could you illustrate your problem a bit further?

I'm trying to replace the old std.streams in my app with ranges. I'm interfacing with a networking library to which I supply a callback that when invoked provides the requested data. I write that data to an output range, but later on I need to read that data from the range too - which of course you can't do.

So what I'm looking for is the range-based equivalent of a MemoryStream.

On Saturday, 17 March 2018 at 17:51:50 UTC, John Chapman wrote:

> I'm trying to replace the old std.streams in my app with ranges. I'm interfacing with a networking library to which I supply a callback that when invoked provides the requested data. I write that data to an output range, but later on I need to read that data from the range too - which of course you can't do.
>
> So what I'm looking for is the range-based equivalent of a MemoryStream.

I suggest you to give a fast read to this [1], reactive streams.
The D implementation [2] uses as a base an output range.

They are pretty good in handling time based series event, it this is your usecase...

[1] http://reactivex.io
[2] https://github.com/lempiji/rx

/Paolo