Thoughts?

private T readAndReturn(T)(InputStream stream)
{
    T ret;
    stream.read(ret);
    return ret;
}

alias readByte    = readAndReturn!byte;
alias readUByte   = readAndReturn!ubyte;
alias readShort   = readAndReturn!short;
alias readUShort  = readAndReturn!ushort;
alias readInt     = readAndReturn!int;
alias readUInt    = readAndReturn!uint;
alias readLong    = readAndReturn!long;
alias readULong   = readAndReturn!ulong;
alias readFloat   = readAndReturn!float;
alias readDouble  = readAndReturn!double;
alias readReal    = readAndReturn!real;
alias readIfloat  = readAndReturn!ifloat;
alias readIdouble = readAndReturn!idouble;
alias readIreal   = readAndReturn!ireal;
alias readCfloat  = readAndReturn!cfloat;
alias readCdouble = readAndReturn!cdouble;
alias readCreal   = readAndReturn!creal;
alias readChar    = readAndReturn!char;
alias readWchar   = readAndReturn!wchar;
alias readDchar   = readAndReturn!dchar;

On Friday, 4 September 2015 at 00:05:06 UTC, bitwise wrote:
> Thoughts?
> [...]

void main(string[] args)
{
    MemoryStream s = new MemoryStream();
    s.write(1);
    s.write(2);
    s.write(3);
    s.seek(0, SeekPos.Set);
    writeln(s.readInt());
    writeln(s.readInt());
    writeln(s.readInt());
}

On Friday, 4 September 2015 at 00:05:06 UTC, bitwise wrote:
> Thoughts?

std.stream is on the chopping block and has been for years, and it was decided at dconf that we'd actually get rid of it now rather than just have it say in the documentation that it's going away. As I understand it, the only reason that it wasn't deprecated in 2.068 was because of some sort of issue with the PR that forced it to be delayed. But there's no point in making any improvements to it, because that code is not going to be around for much longer.

- Jonathan M Davis

On Friday, 4 September 2015 at 14:40:43 UTC, Jonathan M Davis wrote:
> On Friday, 4 September 2015 at 00:05:06 UTC, bitwise wrote:
>> Thoughts?
>
> std.stream is on the chopping block and has been for years, and it was decided at dconf that we'd actually get rid of it now rather than just have it say in the documentation that it's going away. As I understand it, the only reason that it wasn't deprecated in 2.068 was because of some sort of issue with the PR that forced it to be delayed. But there's no point in making any improvements to it, because that code is not going to be around for much longer.
>
> - Jonathan M Davis

Wow...this is surprising. At first glance, the streams seemed to be fairly well done. What's the reason for them being removed? and what is meant to be the replacement?

   Bit

On Friday, 4 September 2015 at 22:36:01 UTC, bitwise wrote:
> On Friday, 4 September 2015 at 14:40:43 UTC, Jonathan M Davis wrote:
>> On Friday, 4 September 2015 at 00:05:06 UTC, bitwise wrote:
>>> Thoughts?
>>
>> std.stream is on the chopping block and has been for years, and it was decided at dconf that we'd actually get rid of it now rather than just have it say in the documentation that it's going away. As I understand it, the only reason that it wasn't deprecated in 2.068 was because of some sort of issue with the PR that forced it to be delayed. But there's no point in making any improvements to it, because that code is not going to be around for much longer.
>>
>> - Jonathan M Davis
>
> Wow...this is surprising. At first glance, the streams seemed to be fairly well done. What's the reason for them being removed? and what is meant to be the replacement?

It's old (from D1) and does not follow the current best practices or idioms. If we're going to have a stream solution, it needs to be range-based. That being said, ranges in and of themselves get pretty close to streams, and there simply hasn't been a push to come up with a range-based stream solution where the pieces that aren't already part of ranges are taken care of. std.stream has been marked as scheduled for deprecation for years, and no one has bothered to replace it. It comes up periodically, but no one seems to care enough to do anything about it. So, rather than leave it in that state, we decided that it would be better off to just deprecate it and then get rid of it rather than leave it in limbo with the documentation saying that it's not up to our standards and that we're going to get rid of it at some point. At some point, someone may come up with a range-based stream solution and get it into Phobos, but until then, we'll just use straight up ranges, which come close enough for most cases. std.stdio and std.mmfile make it quite possible to operate on files with ranges in a manner similar to streams, and std.stdio allows you to operate on stdin and stdout in a similar manner (since they're std.stdio.Files).

That being said, std.stream is boost-licensed, and anyone is free to copy it or fork it so long as the copyright notice is left on it. So, anyone who wants to use std.stream in their own code even after it's no longer in Phobos is free to do so. It's just that the std.stream code is then going to have to be part of their project rather than Phobos.

- Jonathan M Davis

On Saturday, 5 September 2015 at 06:15:17 UTC, Jonathan M Davis wrote:
> On Friday, 4 September 2015 at 22:36:01 UTC, bitwise wrote:
>> On Friday, 4 September 2015 at 14:40:43 UTC, Jonathan M Davis wrote:
>>> [...]
>>
>> Wow...this is surprising. At first glance, the streams seemed to be fairly well done. What's the reason for them being removed? and what is meant to be the replacement?
>
> It's old (from D1) and does not follow the current best practices or idioms. If we're going to have a stream solution, it needs to be range-based. That being said, ranges in and of themselves get pretty close to streams, and there simply hasn't been a push to come up with a range-based stream solution where the pieces that aren't already part of ranges are taken care of. std.stream has been marked as scheduled for deprecation for years, and no one has bothered to replace it. It comes up periodically, but no one seems to care enough to do anything about it. So, rather than leave it in that state, we decided that it would be better off to just deprecate it and then get rid of it rather than leave it in limbo with the documentation saying that it's not up to our standards and that we're going to get rid of it at some point. At some point, someone may come up with a range-based stream solution and get it into Phobos, but until then, we'll just use straight up ranges, which come close enough for most cases. std.stdio and std.mmfile make it quite possible to operate on files with ranges in a manner similar to streams, and std.stdio allows you to operate on stdin and stdout in a similar manner (since they're std.stdio.Files).
>
> That being said, std.stream is boost-licensed, and anyone is free to copy it or fork it so long as the copyright notice is left on it. So, anyone who wants to use std.stream in their own code even after it's no longer in Phobos is free to do so. It's just that the std.stream code is then going to have to be part of their project rather than Phobos.
>
> - Jonathan M Davis

Thanks for the explanation, but could you give an example of how Stream would be rangified?

In the containers I'm writing, I'm returning a range from the findAll() function instead of an array:
https://github.com/bitwise-github/d-containers/blob/master/collections/list.d#L901

I suppose any method returning an array/string could work like this...Is this what you mean?

It doesn't seem like the entire Stream class should have to be removed to make this change though.

On Saturday, 5 September 2015 at 19:59:03 UTC, bitwise wrote:
> On Saturday, 5 September 2015 at 06:15:17 UTC, Jonathan M Davis wrote:
>> On Friday, 4 September 2015 at 22:36:01 UTC, bitwise wrote:
>>> On Friday, 4 September 2015 at 14:40:43 UTC, Jonathan M Davis wrote:
>>>> [...]
>>>
>>> Wow...this is surprising. At first glance, the streams seemed to be fairly well done. What's the reason for them being removed? and what is meant to be the replacement?
>>
>> It's old (from D1) and does not follow the current best practices or idioms. If we're going to have a stream solution, it needs to be range-based. That being said, ranges in and of themselves get pretty close to streams, and there simply hasn't been a push to come up with a range-based stream solution where the pieces that aren't already part of ranges are taken care of. std.stream has been marked as scheduled for deprecation for years, and no one has bothered to replace it. It comes up periodically, but no one seems to care enough to do anything about it. So, rather than leave it in that state, we decided that it would be better off to just deprecate it and then get rid of it rather than leave it in limbo with the documentation saying that it's not up to our standards and that we're going to get rid of it at some point. At some point, someone may come up with a range-based stream solution and get it into Phobos, but until then, we'll just use straight up ranges, which come close enough for most cases. std.stdio and std.mmfile make it quite possible to operate on files with ranges in a manner similar to streams, and std.stdio allows you to operate on stdin and stdout in a similar manner (since they're std.stdio.Files).
>>
>> That being said, std.stream is boost-licensed, and anyone is free to copy it or fork it so long as the copyright notice is left on it. So, anyone who wants to use std.stream in their own code even after it's no longer in Phobos is free to do so. It's just that the std.stream code is then going to have to be part of their project rather than Phobos.
>>
>> - Jonathan M Davis
>
> Thanks for the explanation, but could you give an example of how Stream would be rangified?

this was not addressed to me but here is how it should be done:

---
module runnable;

import std.stdio;
import std.stream;
import std.algorithm;

class RangifiedStream: MemoryStream
{
    auto range()
    {
        return Range(this);
    }

    private struct Range
    {
        MemoryStream _str;
        ulong _curr;

        this(MemoryStream str)
        {
            _str = str;
        }

        void popFront()
        {
            _curr += 1;
        }

        @property ubyte front()
        {
            ubyte result;
            _str.position = _curr;
            _str.read(result);
            _str.position = _str.position - 1;
            return result;
        }

        @property bool empty()
        {
            return _curr == _str.size;
        }
    }
}

void main(string[] args)
{

    import std.range;
    assert( isInputRange!(RangifiedStream.Range));

    RangifiedStream str = new RangifiedStream;
    ubyte[] src = [0,1,2,3,4,5,6,7,8,9];
    str.write(src);

    str.range.each!(a => a.writeln);
}
---

The range has a reference to a stream. The range uses its own position and this is important since several ranges may co-exist at the same time. Here you just have a byte InputRange but it works...

For FileStream the performances will be terrible (bad), because the position is hold by a a structure specific to the OS...

I think I'll add this to my iz streams classes.

On Saturday, 5 September 2015 at 23:00:43 UTC, BBasile wrote:
> On Saturday, 5 September 2015 at 19:59:03 UTC, bitwise wrote:
>> On Saturday, 5 September 2015 at 06:15:17 UTC, Jonathan M Davis wrote:
>>> [...]
>>
>> Thanks for the explanation, but could you give an example of how Stream would be rangified?
>
> this was not addressed to me but here is how it should be done:
>
> ---
> module runnable;
>
> import std.stdio;
> import std.stream;
> import std.algorithm;
>
> class RangifiedStream: MemoryStream
> {
>     auto range()
>     {
>         return Range(this);
>     }
>
>     private struct Range
>     {
>         MemoryStream _str;
>         ulong _curr;
>
>         this(MemoryStream str)
>         {
>             _str = str;
>         }
>
>         void popFront()
>         {
>             _curr += 1;
>         }
>
>         @property ubyte front()
>         {
>             ubyte result;
>             _str.position = _curr;
>             _str.read(result);
>             _str.position = _str.position - 1;
>             return result;
>         }
>
>         @property bool empty()
>         {
>             return _curr == _str.size;
>         }
>     }
> }
>
> void main(string[] args)
> {
>
>     import std.range;
>     assert( isInputRange!(RangifiedStream.Range));
>
>     RangifiedStream str = new RangifiedStream;
>     ubyte[] src = [0,1,2,3,4,5,6,7,8,9];
>     str.write(src);
>
>     str.range.each!(a => a.writeln);
> }
> ---
>
> The range has a reference to a stream. The range uses its own position and this is important since several ranges may co-exist at the same time. Here you just have a byte InputRange but it works...
>
> For FileStream the performances will be terrible (bad), because the position is hold by a a structure specific to the OS...
>
> I think I'll add this to my iz streams classes.

If all we had was a flat array of bytes(or any uniform type), then we wouldn't need a stream at all.

My typical usage is something like this:

int a = 1;
float b = 2;
string s = "3";

MemoryStream ms = new MemoryStream();
ms.write(a);
ms.write(b);
ms.write(s);

// stash ms in a file, or send/receive over socket..

int a = ms.readInt();
float b = ms.readFloat();
string s = ms.readString();

doSomething(a, b, s);

I'm not sure how you would go about rangifying something like this.

On Sunday, 6 September 2015 at 02:19:41 UTC, bitwise wrote:
> On Saturday, 5 September 2015 at 23:00:43 UTC, BBasile wrote:
>> On Saturday, 5 September 2015 at 19:59:03 UTC, bitwise wrote:
>>> On Saturday, 5 September 2015 at 06:15:17 UTC, Jonathan M Davis wrote:
>>>> [...]
>>>
>>> Thanks for the explanation, but could you give an example of how Stream would be rangified?
>>
>> this was not addressed to me but here is how it should be done:
>>
>> ---
>> module runnable;
>>
>> import std.stdio;
>> import std.stream;
>> import std.algorithm;
>>
>> class RangifiedStream: MemoryStream
>> {
>>     auto range()
>>     {
>>         return Range(this);
>>     }
>>
>>     private struct Range
>>     {
>>         MemoryStream _str;
>>         ulong _curr;
>>
>>         this(MemoryStream str)
>>         {
>>             _str = str;
>>         }
>>
>>         void popFront()
>>         {
>>             _curr += 1;
>>         }
>>
>>         @property ubyte front()
>>         {
>>             ubyte result;
>>             _str.position = _curr;
>>             _str.read(result);
>>             _str.position = _str.position - 1;
>>             return result;
>>         }
>>
>>         @property bool empty()
>>         {
>>             return _curr == _str.size;
>>         }
>>     }
>> }
>>
>> void main(string[] args)
>> {
>>
>>     import std.range;
>>     assert( isInputRange!(RangifiedStream.Range));
>>
>>     RangifiedStream str = new RangifiedStream;
>>     ubyte[] src = [0,1,2,3,4,5,6,7,8,9];
>>     str.write(src);
>>
>>     str.range.each!(a => a.writeln);
>> }
>> ---
>>
>> The range has a reference to a stream. The range uses its own position and this is important since several ranges may co-exist at the same time. Here you just have a byte InputRange but it works...
>>
>> For FileStream the performances will be terrible (bad), because the position is hold by a a structure specific to the OS...
>>
>> I think I'll add this to my iz streams classes.
>
> If all we had was a flat array of bytes(or any uniform type), then we wouldn't need a stream at all.
>
> My typical usage is something like this:
>
> int a = 1;
> float b = 2;
> string s = "3";
>
> MemoryStream ms = new MemoryStream();
> ms.write(a);
> ms.write(b);
> ms.write(s);
>
> // stash ms in a file, or send/receive over socket..
>
> int a = ms.readInt();
> float b = ms.readFloat();
> string s = ms.readString();
>
> doSomething(a, b, s);
>
> I'm not sure how you would go about rangifying something like this.

Sorry but I just wanted to reply to your Q 'how Stream would be rangified?'. Put the primitives...period.

Obviously my previous example is not good because it's only for MemoryStream,
and OutputRange primitives miss.
Think more that the the Range will be defined for the Stream class:

---
struct StreamRange(ST, T)
if (is(ST : Stream))
{
    private:

        ulong _fpos, _bpos;
        ST _str;

    public:

        this(ST str)
        {
            _str = str;
            _bpos = str.size - T.sizeof;
        }

        @property T front()
        {
            T result;
            _str.position = _fpos;
            _str.read(&result, T.sizeof);
            _str.position = _fpos;
            return result;
        }

        @property T back()
        {
            T result;
            _str.position = _bpos;
            _str.read(&result, T.sizeof);
            _str.position = _bpos;
            return result;
        }

        @safe void popFront()
        {
            _fpos += T.sizeof;
        }

        @safe void popBack()
        {
            _bpos -= T.sizeof;
        }

        @property bool empty()
        {
            return (_fpos == _str.size) || (_fpos + T.sizeof > _str.size)
                    || (_bpos == 0) || (_bpos - T.sizeof < 0);
        }

        typeof(this) save()
        {
            typeof(this) result = typeof(this)(_str);
            result._fpos = _fpos;
            return result;
        }
}

// ....

auto rng = StreamRange!(MemoryStream,long)(instance);
auto rng = StreamRange!(FileStream,float)(instance);
---

Actually I have nothing against your helper functions (except that they don't make Streams compatible with std.algorithm functions).

On Sunday, 6 September 2015 at 03:42:20 UTC, BBasile wrote:
> [...]
> auto rng = StreamRange!(MemoryStream,long)(instance);
> auto rng = StreamRange!(FileStream,float)(instance);
> ---
>
> Actually I have nothing against your helper functions (except that they don't make Streams compatible with std.algorithm functions).

Same problem... StreamRange(...) assumes that all data in the stream is of the same type.

  -Bit