We currently have a problem with containers : it is very difficult to implement them in a way that is compliant type qualifiers. To restate the problem shortly, let's imagine we want to implement array's as a library type.

struct Array(T) {
    size_t length;
    T* ptr;
    // Methods
}

Now we have several problems with type qualifiers. For instance, Array!T won't implicitly cast to Array!const(T), and const(Array!T) now become a completely useless type, as ptr will become const by transitivity which will create internal inconsistencies.

This problem is known and makes it hard to provide a nice container library for D. This also imply a lot of circuitry in the compiler that can (and should) be provided as library.

So I propose to introduce the inout template parameter type. We declare as follow :

Array(inout T) {
    size_t length;
    T* ptr
}

The inout template parameter is a type parameter, and so don't overload on them (if both Array(T) and Array(inout T) the instantiation is ambiguous).

Within the template, T is always seen as inout, and only one instantiation occur for all top type qualifiers. Array!T and Array!const(T) refers to the same instance of Array. As a result, this makes it impossible to specialize Array on T's type qualifier.

The real type qualifier is determined from the outside. A user of Array!T consider inout as meaning mutable, when a user of Array!const(T) consider it as meaning const, for anything related to Array.

Implicit cast is allowed for instances of Array with different inout parameter's type qualifier, as long as implicit conversion between such qualifier is allowed. Array!T implicitly convert to Array!const(T) but not the other way around.

Finally, Array's type qualifier turtle down to inout parameters's type qualifier. alias A = Array!T; static assert(is(const(A)) == const(Array!const(T))); alias B = Array!immutable(T); static assert(is(const(A)) == const(Array!immutable(T)));

The idea popped in my mind yesterday, so it is not really super fleshed out, and I'm not sure if some horrible dark corner case makes it completely worthless. But it seems super promising to me, so I want to share. The current situation isn't satisfying and we desperately need a solution.

On Tue, Jun 11, 2013 at 11:08 PM, deadalnix <deadalnix@gmail.com> wrote:

> We currently have a problem with containers : it is very difficult to implement them in a way that is compliant type qualifiers. To restate the problem shortly, let's imagine we want to implement array's as a library type.
>
> struct Array(T) {
>     size_t length;
>     T* ptr;
>     // Methods
> }
>
> Now we have several problems with type qualifiers. For instance, Array!T won't implicitly cast to Array!const(T), and const(Array!T) now become a completely useless type, as ptr will become const by transitivity which will create internal inconsistencies.
>
> This problem is known and makes it hard to provide a nice container library for D. This also imply a lot of circuitry in the compiler that can (and should) be provided as library.
>
> So I propose to introduce the inout template parameter type. We declare as follow :
>
> Array(inout T) {
>     size_t length;
>     T* ptr
> }
>
> The inout template parameter is a type parameter, and so don't overload on
> them (if both Array(T) and Array(inout T) the instantiation is ambiguous).
>
> Within the template, T is always seen as inout, and only one instantiation occur for all top type qualifiers. Array!T and Array!const(T) refers to the same instance of Array. As a result, this makes it impossible to specialize Array on T's type qualifier.
>
> The real type qualifier is determined from the outside. A user of Array!T consider inout as meaning mutable, when a user of Array!const(T) consider it as meaning const, for anything related to Array.
>
> Implicit cast is allowed for instances of Array with different inout parameter's type qualifier, as long as implicit conversion between such qualifier is allowed. Array!T implicitly convert to Array!const(T) but not the other way around.
>
> Finally, Array's type qualifier turtle down to inout parameters's type
> qualifier. alias A = Array!T; static assert(is(const(A)) ==
> const(Array!const(T))); alias B = Array!immutable(T); static
> assert(is(const(A)) == const(Array!immutable(T)));
>
> The idea popped in my mind yesterday, so it is not really super fleshed out, and I'm not sure if some horrible dark corner case makes it completely worthless. But it seems super promising to me, so I want to share. The current situation isn't satisfying and we desperately need a solution.
>


Why not use a castConstSafe function that'll transform A!T into A!(const
T), generically:

auto ref castSafe(T,S)(auto ref S a){...}
S a; T b=castSafe!T(a); //same as T b=cast(T)(a) except that it only
compiles if S=>T only involves nonconst=>const conversions (works
recursively):
example: A!(double) => A!(const double) is allowed but not other direction.

More specifically:
auto ref castConstSafe(S)(auto ref S a){...} //transforms A!T into A!(const
T), generically

then:
void foo(B a) if (is(ElementType!B == const)){...}
A!(const double) a1;
A!(double) a2;
foo(a1.castConstSafe); //works
foo(a2.castConstSafe); //works

All it requires is to pass a.castConstSafe instead of a.

On Wednesday, 12 June 2013 at 06:08:58 UTC, deadalnix wrote:
> Within the template, T is always seen as inout, and only one instantiation occur for all top type qualifiers.

So are you adding inout as a type qualifier? Currently it is only a storage class usable in functions.

On Wednesday, 12 June 2013 at 10:38:20 UTC, Peter Alexander wrote:
> On Wednesday, 12 June 2013 at 06:08:58 UTC, deadalnix wrote:
>> Within the template, T is always seen as inout, and only one instantiation occur for all top type qualifiers.
>
> So are you adding inout as a type qualifier? Currently it is only a storage class usable in functions.

I choose inout as the idea is really the same as it is for function : a wildcard type qualifier for the callee, a known one for the caller. Here it is a type qualifier know by the instancier, not the instanciee.

If you consider template argument as compile time argument, as opposed to runtime arguments, this make sense.

On Wednesday, 12 June 2013 at 09:16:30 UTC, Timothee Cour wrote:
> Why not use a castConstSafe function that'll transform A!T into A!(const
> T), generically:
>
> auto ref castSafe(T,S)(auto ref S a){...}
> S a; T b=castSafe!T(a); //same as T b=cast(T)(a) except that it only
> compiles if S=>T only involves nonconst=>const conversions (works
> recursively):
> example: A!(double) => A!(const double) is allowed but not other direction.
>
> More specifically:
> auto ref castConstSafe(S)(auto ref S a){...} //transforms A!T into A!(const
> T), generically
>
> then:
> void foo(B a) if (is(ElementType!B == const)){...}
> A!(const double) a1;
> A!(double) a2;
> foo(a1.castConstSafe); //works
> foo(a2.castConstSafe); //works
>
> All it requires is to pass a.castConstSafe instead of a.

Because Foo!T and Foo!(const T) are completely unrelated types. Casting from one to the other gives 0 guarantees it actually works. For example:

//----
struct Foo(T)
{
    static if (is(T == const int))
    {
        void do_it()const{writeln("this");}
    }
    else
    {
        void do_it()const{writeln("that");}
    }
}

void main()
{
    Foo!int a;
    const(Foo!int)*  pa = &a;
    Foo!(const int)* pb = cast(Foo!(const int)*) &a;
    pa.do_it(); //prints "that"
    pb.do_it(); //prints "this"
}
//----

Of course, I could have also changed the members, making memory mapping incompatible, amongst others...

On Wednesday, 12 June 2013 at 06:08:58 UTC, deadalnix wrote:
> ...

I like it!

> The idea popped in my mind yesterday, so it is not really super fleshed out, and I'm not sure if some horrible dark corner case makes it completely worthless. But it seems super promising to me, so I want to share. The current situation isn't satisfying and we desperately need a solution.

Maybe...

> Within the template, T is always seen as inout, and only one instantiation occur for all top type qualifiers. Array!T and Array!const(T) refers to the same instance of Array. As a result, this makes it impossible to specialize Array on T's type qualifier.

OK, but how do you handle methods that rely on T being (potentially) mutable? For example:

//----
struct Foo(inout T)
{
    T a;
    static if (isAssignable!T) //So here, "T" is actually "inout T", correct?
    {
        void opAssign(T other)
        {a = other.a;}
    }
}
//----

Or is the idea that when instantiated with a "const T", non const methods are not compiled in for that instantiation...?

What about:

//----
struct Foo(inout T)
{
    T[10] buffer;
    size_t i = 0;
    ref T get() const {return buffer[i];}
    void setIndex(size_t i){this.i = i;}
}
//----

This time, the mutable method works, even when T is not mutable :/

I haven't thought through the implications, but it looks like there is a little something missing to make it work.

I DO like your proposition a lot. Being able to have templates that are all instanciated based on the Unqualed type is definitly a plus.

On Wednesday, 12 June 2013 at 12:37:13 UTC, monarch_dodra wrote:
> OK, but how do you handle methods that rely on T being (potentially) mutable? For example:
>
> //----
> struct Foo(inout T)
> {
>     T a;
>     static if (isAssignable!T) //So here, "T" is actually "inout T", correct?
>     {
>         void opAssign(T other)
>         {a = other.a;}
>     }
> }
> //----
>

T is not assignable. If it were, you couldn't cast implicitly to Foo!const(T) . You can still return a T by reference, the caller know if it is mutable or not.

> Or is the idea that when instantiated with a "const T", non const methods are not compiled in for that instantiation...?
>
> What about:
>
> //----
> struct Foo(inout T)
> {
>     T[10] buffer;
>     size_t i = 0;
>     ref T get() const {return buffer[i];}
>     void setIndex(size_t i){this.i = i;}
> }
> //----
>

Foo's type qualifier turtle down to inout parameter type qualifier. You are returning a ref to a const(T) not a ref T. This is a compile time error.

If you don't put const, then T's type is known by the caller, and the code is correct.

> I haven't thought through the implications, but it looks like there is a little something missing to make it work.
>

Yes, details may need to be sorted out.

> I DO like your proposition a lot. Being able to have templates that are all instanciated based on the Unqualed type is definitly a plus.

On Wednesday, 12 June 2013 at 06:08:58 UTC, deadalnix wrote:
> We currently have a problem with containers : it is very difficult to implement them in a way that is compliant type qualifiers. To restate the problem shortly, let's imagine we want to implement array's as a library type.
>
> struct Array(T) {
>     size_t length;
>     T* ptr;
>     // Methods
> }
>
> Now we have several problems with type qualifiers. For instance, Array!T won't implicitly cast to Array!const(T), and const(Array!T) now become a completely useless type, as ptr will become const by transitivity which will create internal inconsistencies.
>
> This problem is known and makes it hard to provide a nice container library for D. This also imply a lot of circuitry in the compiler that can (and should) be provided as library.
>
> So I propose to introduce the inout template parameter type. We declare as follow :
>
> Array(inout T) {
>     size_t length;
>     T* ptr
> }
>
> The inout template parameter is a type parameter, and so don't overload on them (if both Array(T) and Array(inout T) the instantiation is ambiguous).
>
> Within the template, T is always seen as inout, and only one instantiation occur for all top type qualifiers. Array!T and Array!const(T) refers to the same instance of Array. As a result, this makes it impossible to specialize Array on T's type qualifier.
>
> The real type qualifier is determined from the outside. A user of Array!T consider inout as meaning mutable, when a user of Array!const(T) consider it as meaning const, for anything related to Array.
>
> Implicit cast is allowed for instances of Array with different inout parameter's type qualifier, as long as implicit conversion between such qualifier is allowed. Array!T implicitly convert to Array!const(T) but not the other way around.
>
> Finally, Array's type qualifier turtle down to inout parameters's type qualifier. alias A = Array!T; static assert(is(const(A)) == const(Array!const(T))); alias B = Array!immutable(T); static assert(is(const(A)) == const(Array!immutable(T)));
>
> The idea popped in my mind yesterday, so it is not really super fleshed out, and I'm not sure if some horrible dark corner case makes it completely worthless. But it seems super promising to me, so I want to share. The current situation isn't satisfying and we desperately need a solution.

I think it needs to be fleshed out better...  Try to add the methods to your array class and think through the required validation. For example, an insert of const(T) would only work if implemented as a copy on write. It would also be good to think through how to handle class hierarchies.

I believe the correct handling requires awareness of covariant and contravariant types. I first heard about them in scala, but I think C# generics use a better syntax. If I remember correctly, you'll need two type arguments: array(in I, out O), where I is implicitly castable to O. If C derives from B which derives from A, then array!(B,B) is implicitly castable to array!(C,B), array!(B,A), and array!(C,A)

On Wed, 12 Jun 2013 02:08:48 -0400, deadalnix <deadalnix@gmail.com> wrote:

> We currently have a problem with containers : it is very difficult to implement them in a way that is compliant type qualifiers. To restate the problem shortly, let's imagine we want to implement array's as a library type.
>
> struct Array(T) {
>      size_t length;
>      T* ptr;
>      // Methods
> }
>
> Now we have several problems with type qualifiers. For instance, Array!T won't implicitly cast to Array!const(T), and const(Array!T) now become a completely useless type, as ptr will become const by transitivity which will create internal inconsistencies.
>
> This problem is known and makes it hard to provide a nice container library for D. This also imply a lot of circuitry in the compiler that can (and should) be provided as library.
>
> So I propose to introduce the inout template parameter type. We declare as follow :
>
> Array(inout T) {
>      size_t length;
>      T* ptr
> }
>
> The inout template parameter is a type parameter, and so don't overload on them (if both Array(T) and Array(inout T) the instantiation is ambiguous).
>
> Within the template, T is always seen as inout, and only one instantiation occur for all top type qualifiers. Array!T and Array!const(T) refers to the same instance of Array. As a result, this makes it impossible to specialize Array on T's type qualifier.

The one problem I see here is the case where you want to have Array mutate its data.

For example, let's say Array makes ptr and length private overloads opIndex AND opIndexAssign to prevent taking the the address of its data.

How do you do opIndexAssign with your mechanism?  There is no "mutable" type qualifier, so there isn't a way to say, "only allow calling this function if T is mutable," like we have with const and immutable member functions.

Another issue here is, what if you don't want Array to be a template?  That is, you want:

struct IntArray {
    size_t length;
    int *ptr;
}

How do you make this tail-const-able?

> The real type qualifier is determined from the outside. A user of Array!T consider inout as meaning mutable, when a user of Array!const(T) consider it as meaning const, for anything related to Array.
>
> Implicit cast is allowed for instances of Array with different inout parameter's type qualifier, as long as implicit conversion between such qualifier is allowed. Array!T implicitly convert to Array!const(T) but not the other way around.
>
> Finally, Array's type qualifier turtle down to inout parameters's type qualifier. alias A = Array!T; static assert(is(const(A)) == const(Array!const(T))); alias B = Array!immutable(T); static assert(is(const(A)) == const(Array!immutable(T)));
>
> The idea popped in my mind yesterday, so it is not really super fleshed out, and I'm not sure if some horrible dark corner case makes it completely worthless. But it seems super promising to me, so I want to share. The current situation isn't satisfying and we desperately need a solution.

It's a very good start, and very close to the solution I have.  I'm going to finish my article and post it hopefully next week.

-Steve

I knew you'll have interesting feddback.

On Wednesday, 12 June 2013 at 15:33:13 UTC, Steven Schveighoffer wrote:
> The one problem I see here is the case where you want to have Array mutate its data.
>

I admit that this won't solve the issue. But I don't know if this is an issue to be fair.

> For example, let's say Array makes ptr and length private overloads opIndex AND opIndexAssign to prevent taking the the address of its data.
>

I'm not sure what is the use case. But that is clearly impossible with the solution I proposed.

> Another issue here is, what if you don't want Array to be a template?  That is, you want:
>
> struct IntArray {
>     size_t length;
>     int *ptr;
> }
>
> How do you make this tail-const-able?
>

By making it a template. I'm also not sure what is the use case here, but still this is a limitation.

> It's a very good start, and very close to the solution I have.  I'm going to finish my article and post it hopefully next week.
>

Could you give a quick executive summary ?