It looks like an Appender field of a struct doesn't work when its element type is the containing struct, i.e.

struct Foo
{
   Appender!(Foo[]) fooAppender;
}

My guess is the problem is that the Appender needs to know how big "Foo" is since it would be storing each element by value, but since Foo's size depends on the size of the appender, we have a circular dependency on knowing the storage size.

This theory is reinforced because modifying the element type of fooAppender to "Foo*" fixes the problem.
struct Foo
{
   Appender!(Foo*[]) fooAppender; // Works fine
}

One odd thing is that this error doesn't manifest until the appender is used.  It compiles just fine so long as you don't actually use the appender, but once you put something in it or access the data, you will get a compiler error at the location where it was used, and not at the place where the appender was defined.  I'm not sure how it does this, but from the error messages I've seen, it looks like the appender may be setting the element type to "void" and silently continuing on as if this isn't a problem.

Does anything know if this is a bug, or if it is as designed?  If it turns out that the functionality can't be supported, I think that not throwing an error at the location of the appender definition is a problem that should be addressed.

On 11/13/16 11:29 AM, Jonathan Marler wrote:
> It looks like an Appender field of a struct doesn't work when its
> element type is the containing struct, i.e.
>
> struct Foo
> {
>    Appender!(Foo[]) fooAppender;
> }
>
> My guess is the problem is that the Appender needs to know how big "Foo"
> is since it would be storing each element by value, but since Foo's size
> depends on the size of the appender, we have a circular dependency on
> knowing the storage size.
>
> This theory is reinforced because modifying the element type of
> fooAppender to "Foo*" fixes the problem.
> struct Foo
> {
>    Appender!(Foo*[]) fooAppender; // Works fine
> }
>
> One odd thing is that this error doesn't manifest until the appender is
> used.  It compiles just fine so long as you don't actually use the
> appender, but once you put something in it or access the data, you will
> get a compiler error at the location where it was used, and not at the
> place where the appender was defined.  I'm not sure how it does this,
> but from the error messages I've seen, it looks like the appender may be
> setting the element type to "void" and silently continuing on as if this
> isn't a problem.

This is probably because Appender's functions are all templates.

A Foo[] can be stored in a Foo, because it doesn't need the size. But yes, as soon as you start needing Appender functions, then the compiler chokes.

It is a forward reference bug, but still a bug IMO. If you can store the appender, then the compiler knows how big it has to be. So it should be fine at that point.

Paging Timon, I'm betting your front end handles this just fine ;)

-Steve

On 14.11.2016 00:32, Steven Schveighoffer wrote:
>
> A Foo[] can be stored in a Foo, because it doesn't need the size. But
> yes, as soon as you start needing Appender functions, then the compiler
> chokes.
>
> It is a forward reference bug, but still a bug IMO. If you can store the
> appender, then the compiler knows how big it has to be. So it should be
> fine at that point.
>
> Paging Timon, I'm betting your front end handles this just fine ;)
>
> -Steve

It does. :)

Minimal example:

struct Appender(A){
    alias T = typeof({ A a; return a[0]; }());
    T[] data;
    void put(T item){ data~=item; }
}
struct Foo{ Appender!(Foo[]) fooAppender; }
Foo[] test(){
    Foo f;
    f.fooAppender.put(Foo());
    return f.fooAppender.data;
}
static assert(test().length==1);


Error with DMD, works with my front end.

On 11/14/2016 12:26 PM, Timon Gehr wrote:

> Error with DMD, works with my front end.

What are the steps to use your front end instead of dmd's? Is the awesome combo of Timon front-end plus ldc back-end possible?

Ali

On 14.11.2016 21:26, Timon Gehr wrote:
> On 14.11.2016 00:32, Steven Schveighoffer wrote:
>>
>> A Foo[] can be stored in a Foo, because it doesn't need the size. But
>> yes, as soon as you start needing Appender functions, then the compiler
>> chokes.
>>
>> It is a forward reference bug, but still a bug IMO. If you can store the
>> appender, then the compiler knows how big it has to be. So it should be
>> fine at that point.
>>
>> Paging Timon, I'm betting your front end handles this just fine ;)
>>
>> -Steve
>
> It does. :)
>
> Minimal example:
>
> struct Appender(A){
>     alias T = typeof({ A a; return a[0]; }());
>     T[] data;
>     void put(T item){ data~=item; }
> }
> struct Foo{ Appender!(Foo[]) fooAppender; }
> Foo[] test(){
>     Foo f;
>     f.fooAppender.put(Foo());
>     return f.fooAppender.data;
> }
> static assert(test().length==1);
>
>
> Error with DMD, works with my front end.

Coincidentally, I've tried compiling your front end with latest dmd just yesterday. There is one file missing, though: cent_.d. Could you please commit this, too?

The cent code commented out, I noticed your code is suffering from the same issue as the OP (also happens for AAs, e.g. T[int]). A workaround is to use void[] and wrap it in property functions.

I've fixed/worked around a number of issues in dmd with respect to incomplete semantic analysis that happen with template mixins, but more still pop up afterwards.

Are there limitations to what the front end understands? Is it able to digest itself? That would be pretty impressive ;-)

On 15.11.2016 10:10, Ali Çehreli wrote:
> On 11/14/2016 12:26 PM, Timon Gehr wrote:
>
>> Error with DMD, works with my front end.
>
> What are the steps to use your front end instead of dmd's?


The first step is I need to implement all remaining language features. :)

> Is the
> awesome combo of Timon front-end plus ldc back-end possible?
>
> Ali
>

Hopefully it is possible eventually.

On 15.11.2016 14:11, Rainer Schuetze wrote:
>
>
> On 14.11.2016 21:26, Timon Gehr wrote:
>> On 14.11.2016 00:32, Steven Schveighoffer wrote:
>>>
>>> A Foo[] can be stored in a Foo, because it doesn't need the size. But
>>> yes, as soon as you start needing Appender functions, then the compiler
>>> chokes.
>>>
>>> It is a forward reference bug, but still a bug IMO. If you can store the
>>> appender, then the compiler knows how big it has to be. So it should be
>>> fine at that point.
>>>
>>> Paging Timon, I'm betting your front end handles this just fine ;)
>>>
>>> -Steve
>>
>> It does. :)
>>
>> Minimal example:
>>
>> struct Appender(A){
>>     alias T = typeof({ A a; return a[0]; }());
>>     T[] data;
>>     void put(T item){ data~=item; }
>> }
>> struct Foo{ Appender!(Foo[]) fooAppender; }
>> Foo[] test(){
>>     Foo f;
>>     f.fooAppender.put(Foo());
>>     return f.fooAppender.data;
>> }
>> static assert(test().length==1);
>>
>>
>> Error with DMD, works with my front end.
>
> Coincidentally, I've tried compiling your front end with latest dmd just
> yesterday. There is one file missing, though: cent_.d. Could you please
> commit this, too?
> ...

Oops. Thanks for pointing this out, I have committed it. (It's just a stub anyway: It's a wrapper around a long.)

> The cent code commented out, I noticed your code is suffering from the
> same issue as the OP (also happens for AAs, e.g. T[int]). A workaround
> is to use void[] and wrap it in property functions.
>
> I've fixed/worked around a number of issues in dmd with respect to
> incomplete semantic analysis that happen with template mixins, but more
> still pop up afterwards.
> ...

DMD 2.060 is the only version of DMD that builds the code.

> Are there limitations to what the front end understands?

Yup. There are many features missing that are quite easy to implement but require work, and a few that are somewhat messy to specify (e.g. 'protected'). An incomplete list:

* UDA's
* Built-in members (init, stringof, min, max, ...)
* various forms of import statements
  - static, selective, renaming, ...
* Initialization of union fields
* anonymous structs & unions
  - Analysis & Lowering
* additional import paths & implicit object.d
* implicit inheritance from Object
* version declarations
* Associative arrays/Associative array literals
* module declarations
* (implicit) super constructor calls
  - default constructor call insertion
  - flow analysis
* Destructor and postblit calls
  - use flow analysis to optimize moves
* with statements
* associative arrays
* foreach statements
  - automatic decoding for string types
  - foreach over associative arrays
  - foreach over delegates
  - foreach over AliasSeq
* pattern matching in old-style template constraints
* explicit casts from/to class references
  - eg. to/from void* and to bool
* try-catch-finally statements
* scope guards
* initialization crossing check
* multi-argument struct constructors.
* struct postblit & destructors
* finish operator overloading support
* opDispatch
* member alias declarations aliasing members
  - correctly provide a this pointer
* visibility
  - package, protected
* alias this


> Is it able to digest itself?

Not yet. (Mostly because of missing language features.)

> That would be pretty impressive ;-)

I want to get there eventually. :)
Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.

On 2016-11-16 01:11, Timon Gehr wrote:

> Yup. There are many features missing that are quite easy to implement
> but require work, and a few that are somewhat messy to specify (e.g.
> 'protected'). An incomplete list:
>
> * UDA's
> * Built-in members (init, stringof, min, max, ...)
> * various forms of import statements
>    - static, selective, renaming, ...
> * Initialization of union fields
> * anonymous structs & unions
>    - Analysis & Lowering
> * additional import paths & implicit object.d
> * implicit inheritance from Object
> * version declarations
> * Associative arrays/Associative array literals
> * module declarations
> * (implicit) super constructor calls
>    - default constructor call insertion
>    - flow analysis
> * Destructor and postblit calls
>    - use flow analysis to optimize moves
> * with statements
> * associative arrays
> * foreach statements
>    - automatic decoding for string types
>    - foreach over associative arrays
>    - foreach over delegates
>    - foreach over AliasSeq
> * pattern matching in old-style template constraints
> * explicit casts from/to class references
>    - eg. to/from void* and to bool
> * try-catch-finally statements
> * scope guards
> * initialization crossing check
> * multi-argument struct constructors.
> * struct postblit & destructors
> * finish operator overloading support
> * opDispatch
> * member alias declarations aliasing members
>    - correctly provide a this pointer
> * visibility
>    - package, protected
> * alias this

Is it "only" semantic analysis that is missing for these features or lexing and/or parsing as well?

-- 
/Jacob Carlborg

On 16.11.2016 19:17, Jacob Carlborg wrote:
> On 2016-11-16 01:11, Timon Gehr wrote:
>
>> Yup. There are many features missing that are quite easy to implement
>> but require work, and a few that are somewhat messy to specify (e.g.
>> 'protected'). An incomplete list:
>>
>> * UDA's
>> * Built-in members (init, stringof, min, max, ...)
>> * various forms of import statements
>>    - static, selective, renaming, ...
>> * Initialization of union fields
>> * anonymous structs & unions
>>    - Analysis & Lowering
>> * additional import paths & implicit object.d
>> * implicit inheritance from Object
>> * version declarations
>> * Associative arrays/Associative array literals
>> * module declarations
>> * (implicit) super constructor calls
>>    - default constructor call insertion
>>    - flow analysis
>> * Destructor and postblit calls
>>    - use flow analysis to optimize moves
>> * with statements
>> * associative arrays
>> * foreach statements
>>    - automatic decoding for string types
>>    - foreach over associative arrays
>>    - foreach over delegates
>>    - foreach over AliasSeq
>> * pattern matching in old-style template constraints
>> * explicit casts from/to class references
>>    - eg. to/from void* and to bool
>> * try-catch-finally statements
>> * scope guards
>> * initialization crossing check
>> * multi-argument struct constructors.
>> * struct postblit & destructors
>> * finish operator overloading support
>> * opDispatch
>> * member alias declarations aliasing members
>>    - correctly provide a this pointer
>> * visibility
>>    - package, protected
>> * alias this
>
> Is it "only" semantic analysis that is missing for these features or
> lexing and/or parsing as well?
>

Mostly semantic analysis, but parsing is very easy anyway. I think UDAs are the only listed feature still missing in the parser, but that's just because they were introduced after I had already written the parser.

On Wednesday, 16 November 2016 at 09:11:50 UTC, Timon Gehr wrote:
>
> I want to get there eventually. :)
> Unfortunately I haven't had a lot of time to spend on this lately. Also, DMD 2.060 has quite many annoying bugs that slow down development.

Sounds to me like you should rewrite the compiler a little to get it working under 2.07x.