On Sunday, 6 September 2015 at 04:48:56 UTC, bitwise wrote:
> On Sunday, 6 September 2015 at 03:42:20 UTC, BBasile wrote:
>> [...]
>> auto rng = StreamRange!(MemoryStream,long)(instance);
>> auto rng = StreamRange!(FileStream,float)(instance);
>> ---
>>
>> Actually I have nothing against your helper functions (except that they don't make Streams compatible with std.algorithm functions).
>
> Same problem... StreamRange(...) assumes that all data in the stream is of the same type.
>
>   -Bit

So Ranges are...

On Sunday, 6 September 2015 at 10:12:39 UTC, BBasile wrote:
> On Sunday, 6 September 2015 at 04:48:56 UTC, bitwise wrote:
>> On Sunday, 6 September 2015 at 03:42:20 UTC, BBasile wrote:
>>> [...]
>>> auto rng = StreamRange!(MemoryStream,long)(instance);
>>> auto rng = StreamRange!(FileStream,float)(instance);
>>> ---
>>>
>>> Actually I have nothing against your helper functions (except that they don't make Streams compatible with std.algorithm functions).
>>
>> Same problem... StreamRange(...) assumes that all data in the stream is of the same type.
>>
>>   -Bit
>
> So Ranges are...

So tell me how you expect to read a stream containing mixed data types...

On Sunday, 6 September 2015 at 18:52:34 UTC, bitwise wrote:
> On Sunday, 6 September 2015 at 10:12:39 UTC, BBasile wrote:
>> On Sunday, 6 September 2015 at 04:48:56 UTC, bitwise wrote:
>>> On Sunday, 6 September 2015 at 03:42:20 UTC, BBasile wrote:
>>>> [...]
>>>> auto rng = StreamRange!(MemoryStream,long)(instance);
>>>> auto rng = StreamRange!(FileStream,float)(instance);
>>>> ---
>>>>
>>>> Actually I have nothing against your helper functions (except that they don't make Streams compatible with std.algorithm functions).
>>>
>>> Same problem... StreamRange(...) assumes that all data in the stream is of the same type.
>>>
>>>   -Bit
>>
>> So Ranges are...
>
> So tell me how you expect to read a stream containing mixed data types...

You should have got that's even not what i was talking about.
Basic summary of the conversation:

- you: hello what do you thing about my helpers functions?
- others: nothing, we won't add this because streams are deprecated. Maybe they'll come back one day if someone comes with a kind of "rangified" stream solution.
- you: how can I "rangify" a stream ?
- me: hello, you can add the ranges primitives.
- you: yes but it doesn't work for my particular case.
- me: (sigh) I know, I was talking about how streams can be rangified in a general way.

I think that the misunderstanding happens after 'others' intervention: 'other' did not suggested that your helper functions can be made in the "Range" fashion.

On Sunday, 6 September 2015 at 20:05:20 UTC, BBasile wrote:
> On Sunday, 6 September 2015 at 18:52:34 UTC, bitwise wrote:
>> On Sunday, 6 September 2015 at 10:12:39 UTC, BBasile wrote:
>>> On Sunday, 6 September 2015 at 04:48:56 UTC, bitwise wrote:
>>>> On Sunday, 6 September 2015 at 03:42:20 UTC, BBasile wrote:
>>>>> [...]
>>>>
>>>> Same problem... StreamRange(...) assumes that all data in the stream is of the same type.
>>>>
>>>>   -Bit
>>>
>>> So Ranges are...
>>
>> So tell me how you expect to read a stream containing mixed data types...
>
> You should have got that's even not what i was talking about.
> Basic summary of the conversation:
>
> - you: hello what do you thing about my helpers functions?
> - others: nothing, we won't add this because streams are deprecated. Maybe they'll come back one day if someone comes with a kind of "rangified" stream solution.
> - you: how can I "rangify" a stream ?
> - me: hello, you can add the ranges primitives.
> - you: yes but it doesn't work for my particular case.
> - me: (sigh) I know, I was talking about how streams can be rangified in a general way.
>
> I think that the misunderstanding happens after 'others' intervention: 'other' did not suggested that your helper functions can be made in the "Range" fashion.

The problem is, that you are concerned with who said what instead of addressing the problem.

The point is, streams can contain mixed data types. Any adopted solution should address this, and you haven't presented one.

I believe something like this could work:

http://dpaste.dzfl.pl/5f0ee693631a

On Sunday, 6 September 2015 at 20:17:32 UTC, bitwise wrote:
> On Sunday, 6 September 2015 at 20:05:20 UTC, BBasile wrote:
>> On Sunday, 6 September 2015 at 18:52:34 UTC, bitwise wrote:
>>> [...]
>>
>> You should have got that's even not what i was talking about.
>> Basic summary of the conversation:
>>
>> - you: hello what do you thing about my helpers functions?
>> - others: nothing, we won't add this because streams are deprecated. Maybe they'll come back one day if someone comes with a kind of "rangified" stream solution.
>> - you: how can I "rangify" a stream ?
>> - me: hello, you can add the ranges primitives.
>> - you: yes but it doesn't work for my particular case.
>> - me: (sigh) I know, I was talking about how streams can be rangified in a general way.
>>
>> I think that the misunderstanding happens after 'others' intervention: 'other' did not suggested that your helper functions can be made in the "Range" fashion.
>
> The problem is, that you are concerned with who said what instead of addressing the problem.
>
> The point is, streams can contain mixed data types. Any adopted solution should address this, and you haven't presented one.
>
> I believe something like this could work:
>
> http://dpaste.dzfl.pl/5f0ee693631a

I don't care about your problem, adding some type-specific readers & writers is quite straight forward and the solution you proposed initially is Ok.

But...at a particular point you asked how Streams can be rangified. That was the only reason why i posted here. Roger this ?

On Sunday, 6 September 2015 at 21:33:36 UTC, BBasile wrote:
> I don't care about your problem

Then I suggest another thread.

On Saturday, 5 September 2015 at 19:59:03 UTC, bitwise wrote:
> Thanks for the explanation, but could you give an example of how Stream would be rangified?
>
> In the containers I'm writing, I'm returning a range from the findAll() function instead of an array:
> https://github.com/bitwise-github/d-containers/blob/master/collections/list.d#L901
>
> I suppose any method returning an array/string could work like this...Is this what you mean?
>
> It doesn't seem like the entire Stream class should have to be removed to make this change though.

I would have to study the problem to give a good answer. It's been too long since I looked at the issue for me to remember the finer details at this point. IIRC, a big part of the problem had something to do with the fact that you essentially want a range of bytes, but the underlying C APIs really give something more like a range of a range of bytes, because you read the data into a buffer rather than reading it a byte at a time (which gets even further complicated by the fact that you generally want to reuse that buffer and not allocate extra memory if you can avoid it). On some level at least, that's the problem that byLine and byChunk in std.stdio run into. I really don't remember the details of what needed to be done differently for solid stream support though.

But all of the stuff relating to getting objects out of a byte stream can almost certainly be done by building it on top of a range of bytes rather than having any of that talk to the disk or generally care where the data comes from, and I very much doubt that std.stream does anything like that. So, at minimum, what std.stream does would have to be shifted so that all of the transformations are done on top of ranges of ubyte rather than talking to the disk or network stack or whatever. But I'd have to go digging through the code to even know what std.stream does right now, and I don't have the time right now to figure out how it should work. It's been discussed before, and it was decided that what's there was not what we wanted, though how far off it is, I really don't know other than the fact that it's not range-based. Any stream solution should be consuming and producing ranges and should be agnostic to wherever the data comes from unless it has a _really_ good reason not to be.

Steven Schveighoffer has been working off and on on a replacement for std.stdio which supports streams and cleans up some of the uglier things with std.stdio.File, so we may get something out of that at some point, but he has yet to get far enough with it to actually present it for review and possible inclusion in Phobos. However, I haven't heard of any other work being done on streams, and it seems like everyone's simply making do without, for better or worse.

- Jonathan M Davis

On Sunday, 6 September 2015 at 21:40:08 UTC, bitwise wrote:
> On Sunday, 6 September 2015 at 21:33:36 UTC, BBasile wrote:
>> I don't care about your problem
>
> Then I suggest another thread.

Another thread is useless.
It was a dead end since the beginning. std.streams are dead:

https://github.com/D-Programming-Language/phobos/commit/7a6e685ec924d198dd3b3a6856a635d1452b2c93

On Tuesday, 8 September 2015 at 18:59:12 UTC, BBasile wrote:
> On Sunday, 6 September 2015 at 21:40:08 UTC, bitwise wrote:
>> On Sunday, 6 September 2015 at 21:33:36 UTC, BBasile wrote:
>>> I don't care about your problem
>>
>> Then I suggest another thread.
>
> Another thread is useless.
> It was a dead end since the beginning. std.streams are dead:
>
> https://github.com/D-Programming-Language/phobos/commit/7a6e685ec924d198dd3b3a6856a635d1452b2c93

Well, you're half right.
Streams do indeed appear to be dead ;)

On Tuesday, 8 September 2015 at 17:16:45 UTC, Jonathan M Davis wrote:
> On Saturday, 5 September 2015 at 19:59:03 UTC, bitwise wrote:
>> [...]
> I would have to study the problem to give a good answer. It's been too long since I looked at the issue for me to remember the finer details at this point. IIRC, a big part of the problem had something to do with the fact that you essentially want a range of bytes, but the underlying C APIs really give something more like a range of a range of bytes, because you read the data into a buffer rather than reading it a byte at a time (which gets even further complicated by the fact that you generally want to reuse that buffer and not allocate extra memory if you can avoid it). On some level at least, that's the problem that byLine and byChunk in std.stdio run into. I really don't remember the details of what needed to be done differently for solid stream support though.
>
> But all of the stuff relating to getting objects out of a byte stream can almost certainly be done by building it on top of a range of bytes rather than having any of that talk to the disk or generally care where the data comes from, and I very much doubt that std.stream does anything like that. So, at minimum, what std.stream does would have to be shifted so that all of the transformations are done on top of ranges of ubyte rather than talking to the disk or network stack or whatever. But I'd have to go digging through the code to even know what std.stream does right now, and I don't have the time right now to figure out how it should work. It's been discussed before, and it was decided that what's there was not what we wanted, though how far off it is, I really don't know other than the fact that it's not range-based. Any stream solution should be consuming and producing ranges and should be agnostic to wherever the data comes from unless it has a _really_ good reason not to be.
>
> Steven Schveighoffer has been working off and on on a replacement for std.stdio which supports streams and cleans up some of the uglier things with std.stdio.File, so we may get something out of that at some point, but he has yet to get far enough with it to actually present it for review and possible inclusion in Phobos. However, I haven't heard of any other work being done on streams, and it seems like everyone's simply making do without, for better or worse.
>
> - Jonathan M Davis

Ok, I guess I'll try rummaging through the archives and see if I can find the rest of the story... Or maybe just ask Steven where he's at if this becomes important.

2.070 seems quite a ways away though. I'm not sure if DMD is on a steady release schedule, but I imagine there is quite a lot of work to do in terms of optimization and refactoring before even 2.069 comes out.

Anyways, Thanks for the info,

    Bit