https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #10 from Kenji Hara <k.hara.pg@gmail.com> 2014-01-23 01:29:55 PST ---
(In reply to comment #9)
> (In reply to comment #8)
> > There's no lie. The instantiated function _always_ needs valid runtime context, *regardless* the context is really used or not inside function body.
> 
> What?!?!?
> 
> This makes absolutely no sense!
> 
> There is nothing in the D spec about this! Templated functions must behave in the same way as free functions (whether the function itself is templated or it is inside an explicit template declaration), and "static" is meaningless on free functions!
>
> We are not talking about template mixins! The instantiation of non-mixin templates uses the template declaration's context!

Yes, I'm talking about normal templates. Even if the template is defined in module level, if it has alias parameter, instantiated function may have implicit context.

Indeed it is part of the implementation detail, but today it is mostly unavoidable issue. If you interest, you can read issue 5710 discussion.

> > Unfortunately current D has _no_ language feature to remove unused context pointer (I call it "nested-ness inference").
> 
> As far as I can see, this is just a compiler implementation detail!
> 
> Can you provide an example where this "feature" (passing context pointer from template instantiation site to a free function) is actually used?

  // This is module level template function that have alias parameter.
  int foo(alias sym)() { return sym; }

  class C
  {
    int n;
    void test()
    {
      this.n = 10;
      alias fooN = foo!(C.n);
      // fooN has implicit context to access field variable n in class C.
      static assert(is(typeof(&fooN) == delegate));
      assert(fooN() == 10);
    }
  }

  void main()
  {
    int n = 10;
    alias fooN = foo!n;
    // fooN has implicit context to access local variable n in main.
    static assert(is(typeof(&fooN) == delegate));
    assert(fooN() == 10);

    new C().test();
  }

foo(alias sym) is module level template, but in both case instantiated functions will have implicit context pointer to access runtime data of 'sym'.

> > It's not a bug. If the symbol X needs runtime context ('this' object or
> > function frame),
> 
> It doesn't! .stringof does not need a gosh-darn context pointer!

Again, current compiler does not consider that the context is not really used inside function.

> > f() will be instantiated like as member function or nested
> > function.
> 
> WHY?!?
> 
> It is a free function!

No. It's a template function that is defined in module level.

> > To remove the implicit context pointer, 'static' will work.
> 
> ...
> 
> > After fixing issue 11533, these two declarations have different meanings.
> > 
> > template f(alias X) { static string f() { return null; } }  // A
> > static template f(alias X) { string f() { return null; } }  // B
> > 
> > In A, 'static' attribute is always added to the instantiated function 'f'. So
> > Even if X requires runtime context, 'f' still cannot access it.
> > In B, 'static' attribute is added _if_ X has no runtime context. If X needs
> > runtime context, instantiated 'f' also take the context to access valid runtime
> > storage of X.
> 
> This looks like absurd overcomplication. Do you really expect D users to have to understand the difference?

In many cases, learning it would not be necessary. But in your case the
difference is necessary.
Honestly, the alias parameter usage in your code does not fit the expected
usage in language design.
At least from D1 age, template alias parameter is designed to use the given
symbol in runtime at instantiated code.

> > It would be a not trivial language enhancement that still not yet designed.
> 
> Then I suggest that you postpone all breaking changes until all that is figured out. Breaking code to clean up compiler internals is inexcusable, IMO!
> 
> > To explain the behavior, I'll update webside documentation and add a section in release note.
> 
> The more I think about it the less sense it makes to me. Maybe it makes sense from the point of view of a compiler developer (to use Chen Raymond's phraseology, you're looking at the world through "compiler-tinted glasses").
>
> But as a D user, your explanation reads like one giant WTF to me. It is completely unintuitive and to me it seems that things are broken at a much more fundamental level if you defend the current logic in this way.

I agree that it is hard to understand behavior. That's why I had tried to fix the issue at first.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #11 from Kenji Hara <k.hara.pg@gmail.com> 2014-01-23 07:50:17 PST ---
(In reply to comment #8)
> To explain the behavior, I'll update webside documentation and add a section in release note.

I opened a document improvement for the existing "Nested Templates" feature. https://github.com/D-Programming-Language/dlang.org/pull/479

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #12 from Vladimir Panteleev <thecybershadow@gmail.com> 2014-01-23 20:24:12 EET ---
(In reply to comment #10)
>   // This is module level template function that have alias parameter.
>   int foo(alias sym)() { return sym; }
> 
>   class C
>   {
>     int n;
>     void test()
>     {
>       this.n = 10;
>       alias fooN = foo!(C.n);
>       // fooN has implicit context to access field variable n in class C.
>       static assert(is(typeof(&fooN) == delegate));
>       assert(fooN() == 10);
>     }
>   }

Thank you, I finally understand what's going on. But the situation looks really bad. What if you want to pass the "n" variable from both the struct and the function, to the same template? Or if the template already has a context pointer (it is a templated method)? The context pointers should be tied to the variables, not the same "this" pointer that is used in calling methods. Doing it that way just produces absurd error messages.

In either case, at the very least the error message needs to be improved. The simplest solution would be to just change the wording for such cases to make it Google-able. But of course ideally the problem should be solved at its root.

> Again, current compiler does not consider that the context is not really used inside function.

Well, this is another bug then.

> > It is a free function!
> 
> No. It's a template function that is defined in module level.

I do not believe that the distinction needs to be forced upon the users.

> In many cases, learning it would not be necessary. But in your case the
> difference is necessary.
> Honestly, the alias parameter usage in your code does not fit the expected
> usage in language design.
> At least from D1 age, template alias parameter is designed to use the given
> symbol in runtime at instantiated code.

I just tested another bit of code:

struct O
{
    int f(A...)() { return 0; }

    struct S
    {
        int x;

        void g()
        {
            auto y = f!x();
        }
    }
}

With this one, we get:

test3.d(11,13): Error: template instance f!(x) cannot use local 'x' as
parameter to non-global template f(A...)()
test3.d(11,16): Error: this for f needs to be type O not type S

IMHO the correct solution would be to pass the context pointers of alias parameters separately, so as to not interfere with any existing context pointers of the template.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #13 from Vladimir Panteleev <thecybershadow@gmail.com> 2014-01-24 04:48:25 EET ---
After some thinking:

"static" must not affect whether alias parameters receive context pointers. Currently, there can only be one context pointer per function, but this limitation will hopefully be lifted in the future. When that happens, there will be no way to decide whether each alias parameter carries along a context pointer or not. Therefore, "static" must only untie the function from its immediate surrounding declaration context (struct, class or function frame).

As I understand, currently, if a declaration CAN have a context pointer, the compiler seems to demand it at all times when passing it as an alias parameter.

Instead, it should pass a context pointer if and only if if it is actually available at the scope of the template instantiation.

For example, it will be available when calling a templated function from a non-static method, but it will not be available when declaring an enum, or calling from a static method.

Later, if the template function's code attempts to do anything with the alias parameter which ACTUALLY requires its context, for example reading/writing it or taking its address, then it should raise an error. So, the context pointer check needs to stop being eager and be made as late as possible.

Additionally, if you want to explicitly choose whether you want to pass an alias parameter WITH or WITHOUT a context pointer, I think the language already has the necessary syntax, and for class/struct fields it is quite intuitive: C.x (no context pointer) vs. c.x (c is the context pointer), or this.x vs. typeof(this).x etc. It is more complicated with local variables, but I think __traits(getMember, __traits(parent, n), "n") would work. However, I believe this should not be necessary at all, since the compiler should not demand a context pointer without even bothering to check if the template actually needs it.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #14 from Kenji Hara <k.hara.pg@gmail.com> 2014-01-23 19:51:25 PST ---
(In reply to comment #13)
> After some thinking:
> 
> "static" must not affect whether alias parameters receive context pointers.
[snip]
> "static" must only untie the function from its
> immediate surrounding declaration context (struct, class or function frame).

This is just same with the behavior that was implemented by issue 11533. In git-head, static attribute on template does only unite instantiated functions and aggregate types from its immediate surrounding declaration context.

void main()
{
    int x;

    // Adding 'static' on local template foo unlinks instantied function
    // from the enclosing main() function context.
    static void foo(alias fun)()
    {
        // x = 10; // NG
        fun();
    }
    static void bar() {}
    foo!bar();
    pragma(msg, is(typeof(&foo!bar) == delegate));  // prints false

    // And nested-ness of the instantiated function only depends on
    // the passed argument on the alias parameter
    void nested() {}
    pragma(msg, is(typeof(&foo!nested) == delegate));  // prints true
}

So in the OP code:

static int f(A...)() { return 0; }
struct S { int x; enum y = f!x(); }

f!x is now tied to the runtime object S, and calling it on context-less scope is rejected.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #15 from Kenji Hara <k.hara.pg@gmail.com> 2014-01-23 19:52:56 PST ---
(In reply to comment #14)
>  static attribute on template does only unite instantiated functions ...

static attribute on template does only >>untied<< instantiated functions ...

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #16 from Vladimir Panteleev <thecybershadow@gmail.com> 2014-01-24 05:54:06 EET ---
OK... but then why does your suggestion in comment#1 work? If it is as you say, then "static" should have no effect on free functions, regardless if they are in a template or not, right?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #17 from Kenji Hara <k.hara.pg@gmail.com> 2014-01-23 20:16:30 PST ---
(In reply to comment #16)
> OK... but then why does your suggestion in comment#1 work? If it is as you say, then "static" should have no effect on free functions, regardless if they are in a template or not, right?

Again, 'f' is not a free function, it is a function template that declared in module level. Therefore, instantiated function f!x will become nested if inner static attribute does not exist.

// static int f(A...)() { return 0; }  is equivalent with
static template f(A...) { int f() { return 0; } }
struct S { int x;
    enum y = f!x // f!x is equivalent with member function of S.
}

On the other hand:

template f(A...) { static int f() { return 0; } }
struct S { int x;
    enum y = f!x // f!x is equivalent with 'static' member function of S.
}

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #18 from Vladimir Panteleev <thecybershadow@gmail.com> 2014-01-24 07:34:14 EET ---
OK, but when I said:

> "static" must not affect whether alias parameters receive context pointers.
[snip]
> "static" must only untie the function from its
> immediate surrounding declaration context (struct, class or function frame).

I specifically meant this case. It should not matter whether the function is in a template or not, if it is not declared in a struct/class/function, then "static" should have no effect (or even be forbidden, as the compiler now forbids many senseless attributes). With your change, "static" is still used as a mechanism to tell the compiler whether the function needs the context at the INSTANTIATION site, which should not be necessary, as I said in comment#13 the compiler should instantiate/evaluate the function regardless of whether a context pointer is needed, and if the function's code actually needs the context pointer then the compiler will complain at that point.

But to get back on topic, you said that you're not sure if this problem will ever be solved. But there have been a few unanswered suggestions here and in the other discussion you mentioned, do you think they are all unfeasible?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

https://d.puremagic.com/issues/show_bug.cgi?id=11946



--- Comment #19 from Kenji Hara <k.hara.pg@gmail.com> 2014-01-23 22:47:53 PST ---
(In reply to comment #18)
> OK, but when I said:
> 
> > "static" must not affect whether alias parameters receive context pointers.
> [snip]
> > "static" must only untie the function from its
> > immediate surrounding declaration context (struct, class or function frame).
> 
> I specifically meant this case. It should not matter whether the function is in a template or not, if it is not declared in a struct/class/function, then "static" should have no effect (or even be forbidden, as the compiler now forbids many senseless attributes).

Maybe you're misunderstanding the effect of 'static' attribute. As far as I know, it means that:

- If 'static' attribute is specified to a nested declaration, it will make the nested declaration 'unnested'.

In above description, of course 'nested declaration' contains the instantiated functions that implicitly made nested, like f!x.

That's why static attribute on module level declaration is meaningless - module level declaration is always non-nested, so 'unnesting' has no effect, and adding static attribute to the instantiated nested function is meaningful.

> With your change, "static" is still used as
> a mechanism to tell the compiler whether the function needs the context at the
> INSTANTIATION site

As I explained above, It is expected behavior of 'static' attribute.

> which should not be necessary, as I said in comment#13
[snip]

Even if it will become possible in the future, but currently it isn't. I have no opinion about non existing feature.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------