On Tue, 18 Aug 2009 13:48:23 -0400, Robert Jacques <sandford@jhu.edu> wrote:

> On Tue, 18 Aug 2009 10:38:50 -0700, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
>
>> On Tue, 18 Aug 2009 13:34:36 -0400, bearophile <bearophileHUGS@lycos.com> wrote:
>>
>>> Steven Schveighoffer:
>>>> Another way is to perform escape analysis, but Walter has expressed that
>>>> he doesn't want to do that.  It would require an intermediate interface
>>>> language for imports where annotations could be added by the compiler.
>>>
>>> Why is that bad?
>>
>> I don't think it's bad, but definitely a lot of work.  I would be all for it.
>>
>> -Steve
>
> Actually, it's really bad. Escape analysis requires whole program analysis. It would be impossible to do incremental compilation or to ship/sell D libraries in binary format. I'd recomend checking out http://en.wikipedia.org/wiki/Escape_analysis for an overview of the issues involved. You can avoid doing whole program analysis by introducing ownership types and being a bit conservative in what you allow. There's a (bit confusing) wiki page proposal an how to implement it at http://www.prowiki.org/wiki4d/wiki.cgi?OwnershipTypesInD.

Admitting I didn't read any of that, I think incremental analysis is possible as long as import files are generated by the compiler post-analysis.  i.e. the compiler is able to alter the function signature indicating escape analysis information.

With something like that, you could still ship in binary format, along with generated import files that describe the function signatures (if one requires building against your product).  In fact, the import files could be a part of the binary files, similar to how java works.

-Steve

Steven Schveighoffer Wrote:

> remember, scope is a storage class, not a type modifier. "scopeness" is only a hint to the compiler of where to store it originally, the hint is not passed on to other variables which point to the same data.  I'm surprised it's actually an error to try and return a scope variable.
> 
> One way to get scope to work as you desire is to make it a type modifier and define rules about assignment, but I'm not sure that's a good answer. Another way is to perform escape analysis, but Walter has expressed that he doesn't want to do that.  It would require an intermediate interface language for imports where annotations could be added by the compiler.


It seems that scopeness as-is is not a "hint to the compiler of where to store it originally" because scope reference accepts object which is not allocated at there.
Here is a example:

class C
{
	int i = 1;
	~this() { writeln("~C"); }
	void foo() {}
}

int* f(C arg) {
	scope c = arg;
	return &(c.i);
}

void main()
{
	auto c = new C;
	auto p = f(c);    // destructor is called after f()
	writeln(*p);      // success, at first grance
	c.foo;            // runtime error: access violation
}

The instance have been collected, hence escaped pointer probably points a garbage.
The 'scope' is misleading rather than a hint in this case, since resource had been acquired far away from the 'scope'.

Current behaviors of 'scope' are:
1. Prohibit to return the scope references (as you said, it is a storage class and not transitive). (compile time)
2. Calling destructor for the referenced instance when the reference goes out of scope. (runtime)

I think RAII has similarity to const/immutable, which is fully cared in D2:
* It is safe to reference values/objects from multi-thread, etc. if the referenced object will not change.
* It is safe to destruct a object when reference goes out of scope if the object is referenced only from the scope.


Consequently, the best answer is presumably make it a transitive type modifier.
At least adding rules about assignment, right-hand value of 'scope' reference must be just allocated by new operator.
It is too dangerous to allow referencing any object, especially parameter of the function.
I agree that escape analysis is bad solution because it makes compiler implementation too hard.

Robert Jacques wrote:
> ship/sell D libraries in binary format.

Does anyone sell static libraries anymore? There are too many problems with static linking for that to be very viable. Most libraries I've seen are sold as DLLs.

On Tue, 18 Aug 2009 11:54:27 -0700, Robert Fraser <fraserofthenight@gmail.com> wrote:

> Robert Jacques wrote:
>> ship/sell D libraries in binary format.
>
> Does anyone sell static libraries anymore? There are too many problems with static linking for that to be very viable. Most libraries I've seen are sold as DLLs.

I don't know about selling, but (for instance) NVIDIA's CUDA shipped as a pair of static and dynamic libraries. For a long time, runtime linking wasn't possible with however they did the DLLs so you had to use the static libs.

But, whole program/escape analysis also has to have access to the DLL's source code, to work, since it has to check/know what each function does with its input arguments.

On Tue, 18 Aug 2009 10:57:50 -0700, Steven Schveighoffer <schveiguy@yahoo.com> wrote:

> On Tue, 18 Aug 2009 13:48:23 -0400, Robert Jacques <sandford@jhu.edu> wrote:
>
>> On Tue, 18 Aug 2009 10:38:50 -0700, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
>>
>>> On Tue, 18 Aug 2009 13:34:36 -0400, bearophile <bearophileHUGS@lycos.com> wrote:
>>>
>>>> Steven Schveighoffer:
>>>>> Another way is to perform escape analysis, but Walter has expressed that
>>>>> he doesn't want to do that.  It would require an intermediate interface
>>>>> language for imports where annotations could be added by the compiler.
>>>>
>>>> Why is that bad?
>>>
>>> I don't think it's bad, but definitely a lot of work.  I would be all for it.
>>>
>>> -Steve
>>
>> Actually, it's really bad. Escape analysis requires whole program analysis. It would be impossible to do incremental compilation or to ship/sell D libraries in binary format. I'd recomend checking out http://en.wikipedia.org/wiki/Escape_analysis for an overview of the issues involved. You can avoid doing whole program analysis by introducing ownership types and being a bit conservative in what you allow. There's a (bit confusing) wiki page proposal an how to implement it at http://www.prowiki.org/wiki4d/wiki.cgi?OwnershipTypesInD.
>
> Admitting I didn't read any of that, I think incremental analysis is possible as long as import files are generated by the compiler post-analysis.  i.e. the compiler is able to alter the function signature indicating escape analysis information.
>
> With something like that, you could still ship in binary format, along with generated import files that describe the function signatures (if one requires building against your product).  In fact, the import files could be a part of the binary files, similar to how java works.
>
> -Steve
>

*sigh* That doesn't work. From the wikipedia article:
> In traditional static compilation, method overriding can make escape analysis impossible, as any called method might be overridden by a version that allows a pointer to escape.
For example: Let's take a 3rd party pre-compiled library with class A and function f(A a), neither of which have any escapes. Now create a subclass of A, B, which does contain escapes. Does f(B) escape or not?

Now, introducing some ownership types, (scope, stack, heap, shared, mobile), gives the compiler / methods the guarantees they need to get over this problem.

On Tue, 18 Aug 2009 18:08:56 -0400, Robert Jacques <sandford@jhu.edu> wrote:

> *sigh* That doesn't work. From the wikipedia article:
>> In traditional static compilation, method overriding can make escape analysis impossible, as any called method might be overridden by a version that allows a pointer to escape.
> For example: Let's take a 3rd party pre-compiled library with class A and function f(A a), neither of which have any escapes. Now create a subclass of A, B, which does contain escapes. Does f(B) escape or not?

It also says that Java 6, a language compiled as I proposed D could be, has escape analysis.

I don't think it's easy, but it's definitely possible.

Besides, this entire argument is moot if Walter doesn't want to do it...

-Steve

Steven Schveighoffer wrote:
> It also says that Java 6, a language compiled as I proposed D could be, has escape analysis.

Java's escape analysis is done at runtime (during JIT compilation) AFAIK.

LDC can compile to bitcode and link-time codegen could be used to deal with escape analysis... this doesn't help (much) in generating errors, but it allows better codegen.