On 20 April 2014 14:33, via Digitalmars-d <digitalmars-d@puremagic.com>wrote:

> On Sunday, 20 April 2014 at 00:59:26 UTC, Manu via Digitalmars-d wrote:
>
>> Interpretation: this struct and any data directly reachable from it is
>>> guaranteed to not be GC allocated. Then a precise collector could avoid scanning those and pointers to them.
>>>
>>
>> Why wouldn't precise GC be able to do this anyway? It already has data about everything it scans. It can easily produce a 'don't bother scanning this' bit at the start of that data without programmer assistance?
>>
>
> It doesn't know what can be reached through a node in a graph. It doesn't know what is on the GC heap.
>
>
>  In a visual realtime app, the GC will only be acceptable when it will not
>> interrupt for more than 1ms or so (and I consider that quite generous, I'd be more comfortable with < 500µs). Otherwise you'll lose frames anyway; if
>>
>
> No, because the @nogc thread will not be interrupted.
>
> Think MVC: the model is under GC, the view/controller is under @nogc.
>

I don't really see why a proposed @nogc thread wouldn't hold references to
GC allocated objects... what would such a thread do if it didn't have any
data to work with?
@nogc just says the thread won't allocate, it can still be holding all the
references it likes, and still needs to be scanned.

On Sunday, 20 April 2014 at 05:21:48 UTC, Manu via Digitalmars-d wrote:
> I don't really see why a proposed @nogc thread wouldn't hold references to
> GC allocated objects... what would such a thread do if it didn't have any
> data to work with?

Approach 1: Nogc thread can hold references to the gc heap that is known to be under bookkeeping or that is reachable from objects that are under book keeping. No need to scan the @nogc thread then.

Approach 2: separate scan and freeing in the GC by pushing collected item onto a list of pending objects that are waiting for @nogc to complete an iteration before they are put on the freelist. This should work for standard culling.

Ola.

On Saturday, 19 April 2014 at 23:44:45 UTC, Marco Leise wrote:
> Am Wed, 16 Apr 2014 20:32:20 +0000
> schrieb "Peter Alexander" <peter.alexander.au@gmail.com>:
>
>> On Wednesday, 16 April 2014 at 20:29:17 UTC, bearophile wrote:
>> > Peter Alexander:
>> >
>> >> (I assume that nothrow isn't meant to be there?)
>> >
>> > In D nothrow functions can throw errors.
>> 
>> Of course, ignore me :-)
>> 
>> 
>> >> You could do something like this:
>> >>
>> >> void foo() @nogc
>> >> {
>> >>    static err = new Error();
>> >>    if (badthing)
>> >>    {
>> >>        err.setError("badthing happened");
>> >>        throw err;
>> >>    }
>> >> }
>> >
>> > To be mutable err also needs to be __gshared.
>> 
>> But then it isn't thread safe. Two threads trying to set and throw the same Error is a recipe for disaster.
>
> Also: As far as I remember from disassembling C++, static
> variables in functions are initialized on first access and
> guarded by a bool. The first call to foo() would execute
> "err = new Error();" in that case. This code should not
> compile under @nogc.

D static initialization doesn't work the same way. Everything is initialized as the program is loaded, and everything must have a statically known value. EG: An Error is allocated *somewhere* (may or may not actually be the GC), and then the static value of the Error is a pointer to that.

It's what allows us to do things like:

class A{}

struct S
{
    A a = new A();
}

This is legal. All S.a will point the *same* A. S.init will be initialized to point to that a.

Also, just doing this is good enough:

//----
void foo() @nogc
{
   static err = new Error("badthing happened");
   if (badthing)
       throw err;
}
//----

It does require the message be known before hand, and not custom "built". But then again, where were you going to allocate the message, and if allocated, who would clean it up?

--------

That said, while the approach works, there could be issues with re-entrance, or chaining exceptions.

On 2014-04-19 15:40, monarch_dodra wrote:

> Nonsense. It still works 99% of the time (I think only a subset of 100
> letters in all of Unicode are affect, and even then, another 100 of them
> *shrink* on toUpper). It is really useful. It avoids *needles*
> allocations. Removing it would be more harmful than useful.

I'm implicitly referring to toLowerInPlace as well.

> I'm pretty confident that most of the time it is used, people don't care
> *that* much that *absolutely* no allocation takes place. They just don't
> want to be wasteful.

It still has a confusing name.

> Rename "toUpperMaybeInPlace".

Actually, the functionality is useful, it's just the name that is confusing.

> Then, for those that absolutely *can't* allocate provide a better
> interface. For example:
> `void toUpper(S, O)(S s, ref O o);`
>
> Burden on the caller to make it "inplace" from that (or to allocate
> accordingly when inplace is not possible).


-- 
/Jacob Carlborg

On Wednesday, 16 April 2014 at 02:14:18 UTC, Walter Bright wrote:
> On 4/15/2014 6:57 PM, Mike wrote:
>> I suspect some of the motivation for this is to give customers "faster horses".
>> I would be surprised if a @nogc attribute increased D's appeal, and I think
>> efforts would be better allocated to some form of the above.
>
> Asking for @nogc comes up *constantly*.

How bout this!

Why not allow one to define their own attributes from a generalized subset and then define a few standard ones like @nogc.

i.e., instead of having to define specific attributes every few years to satisfy some new thing, why not just abstract the process.

Attributes, I believe, are essentially relationships between parts of code?

If so, then one simply has to implement some generic way to specify the attributes and properties of the relationship to the compiler. Then anyone would have the tools to define and use these attributes as they wish. (in fact, I think it would just involve enhancing the current attribute support, probably just need to rewrite it all so that the same code is used for built in attributes(@safe, @pure, etc...) and user define attributes.


So, we just need to define the attribute name and the properties it has such as:

Assume Y uses X in some way(function call) and X has an attribute A defined on it:

Inheritance - Y inherits attribute A.

Exclusion - If Y has attribute B and B is mutually excluded from A then error

Composition - If Y also uses Z and Z has attribute B then Y has the compound attribute (A:B). Compound attributes can be rewritten to other attributes using a grammar/reduction scheme. Some compositions can be invalid. E.g., @nogc and @gc, @pure and @notpure, etc...

Duality - If an attribute A is not specified for a block of code then it's inverse attribute is implicitly specified always. e.g., @gc and @!gc = @nogc are duals and one or the other always is specified, even if implicit.

etc... [Note, I'm not saying all attributes have these properties, just that these the possible properties they can have]



By coming up with a general system(I'm sure there is some mathematical structure that describes attributes) it would be very easy to add attributes in the future and there would be a consistent code backing for them. It would also be easier for CT reflection on attributes.


Anyways, just a thought, sounds easy in theory...

On Sunday, 20 April 2014 at 14:38:47 UTC, Frustrated wrote:
> On Wednesday, 16 April 2014 at 02:14:18 UTC, Walter Bright wrote:
>> On 4/15/2014 6:57 PM, Mike wrote:
>>> I suspect some of the motivation for this is to give customers "faster horses".
>>> I would be surprised if a @nogc attribute increased D's appeal, and I think
>>> efforts would be better allocated to some form of the above.
>>
>> Asking for @nogc comes up *constantly*.
>
> How bout this!
>
> Why not allow one to define their own attributes from a generalized subset and then define a few standard ones like @nogc.
>
> i.e., instead of having to define specific attributes every few years to satisfy some new thing, why not just abstract the process.
>
> Attributes, I believe, are essentially relationships between parts of code?
>
> If so, then one simply has to implement some generic way to specify the attributes and properties of the relationship to the compiler. Then anyone would have the tools to define and use these attributes as they wish. (in fact, I think it would just involve enhancing the current attribute support, probably just need to rewrite it all so that the same code is used for built in attributes(@safe, @pure, etc...) and user define attributes.
>
>
> So, we just need to define the attribute name and the properties it has such as:
>
> Assume Y uses X in some way(function call) and X has an attribute A defined on it:
>
> Inheritance - Y inherits attribute A.
>
> Exclusion - If Y has attribute B and B is mutually excluded from A then error
>
> Composition - If Y also uses Z and Z has attribute B then Y has the compound attribute (A:B). Compound attributes can be rewritten to other attributes using a grammar/reduction scheme. Some compositions can be invalid. E.g., @nogc and @gc, @pure and @notpure, etc...
>
> Duality - If an attribute A is not specified for a block of code then it's inverse attribute is implicitly specified always. e.g., @gc and @!gc = @nogc are duals and one or the other always is specified, even if implicit.
>
> etc... [Note, I'm not saying all attributes have these properties, just that these the possible properties they can have]
>
>
>
> By coming up with a general system(I'm sure there is some mathematical structure that describes attributes) it would be very easy to add attributes in the future and there would be a consistent code backing for them. It would also be easier for CT reflection on attributes.
>
>
> Anyways, just a thought, sounds easy in theory...

Sounds like a neat idea, now for some code examples? Because it
sounds like we would need an entirely different notation
mechanism or something crazy.
Like:

struct MyPureFunction(alias MYFUNC) {
     shared static this() {
          registerFunc!(MYFUNC);
     }

     __annotation() {
         static if (!is(ReturnType!MYFUNC == void)) {
             return Tuple!(__annotation(pure),
__annotation(property));
         } else {
             return Tuple!(__anotation(pure));
         }
     }
}

@MyPureFunction
string mypurefunc() {
    return "hi";
}

pragma(msg, mypurefunc);

I added the constructor in there because being able to run code
dependent on it would enable registering of certain types (useful
for e.g. Cmsed so users don't have to).
This would add a new keyword (__annotation) in the same style as
__traits.
__annotation function would be called post constructor meaning
you could negate what you would normally return.
Perhaps another function !__annotation to remove current ones.

Not quite sure how this would relate to @nogc but.. Maybe it
means we can fine tune it per attribute/compiler or something.

But hey just my take.

On Sunday, 20 April 2014 at 15:04:28 UTC, Rikki Cattermole wrote:
> On Sunday, 20 April 2014 at 14:38:47 UTC, Frustrated wrote:
>> On Wednesday, 16 April 2014 at 02:14:18 UTC, Walter Bright wrote:
>>> On 4/15/2014 6:57 PM, Mike wrote:
>>>> I suspect some of the motivation for this is to give customers "faster horses".
>>>> I would be surprised if a @nogc attribute increased D's appeal, and I think
>>>> efforts would be better allocated to some form of the above.
>>>
>>> Asking for @nogc comes up *constantly*.
>>
>> How bout this!
>>
>> Why not allow one to define their own attributes from a generalized subset and then define a few standard ones like @nogc.
>>
>> i.e., instead of having to define specific attributes every few years to satisfy some new thing, why not just abstract the process.
>>
>> Attributes, I believe, are essentially relationships between parts of code?
>>
>> If so, then one simply has to implement some generic way to specify the attributes and properties of the relationship to the compiler. Then anyone would have the tools to define and use these attributes as they wish. (in fact, I think it would just involve enhancing the current attribute support, probably just need to rewrite it all so that the same code is used for built in attributes(@safe, @pure, etc...) and user define attributes.
>>
>>
>> So, we just need to define the attribute name and the properties it has such as:
>>
>> Assume Y uses X in some way(function call) and X has an attribute A defined on it:
>>
>> Inheritance - Y inherits attribute A.
>>
>> Exclusion - If Y has attribute B and B is mutually excluded from A then error
>>
>> Composition - If Y also uses Z and Z has attribute B then Y has the compound attribute (A:B). Compound attributes can be rewritten to other attributes using a grammar/reduction scheme. Some compositions can be invalid. E.g., @nogc and @gc, @pure and @notpure, etc...
>>
>> Duality - If an attribute A is not specified for a block of code then it's inverse attribute is implicitly specified always. e.g., @gc and @!gc = @nogc are duals and one or the other always is specified, even if implicit.
>>
>> etc... [Note, I'm not saying all attributes have these properties, just that these the possible properties they can have]
>>
>>
>>
>> By coming up with a general system(I'm sure there is some mathematical structure that describes attributes) it would be very easy to add attributes in the future and there would be a consistent code backing for them. It would also be easier for CT reflection on attributes.
>>
>>
>> Anyways, just a thought, sounds easy in theory...
>
> Sounds like a neat idea, now for some code examples? Because it
> sounds like we would need an entirely different notation
> mechanism or something crazy.
> Like:
>
> struct MyPureFunction(alias MYFUNC) {
>      shared static this() {
>           registerFunc!(MYFUNC);
>      }
>
>      __annotation() {
>          static if (!is(ReturnType!MYFUNC == void)) {
>              return Tuple!(__annotation(pure),
> __annotation(property));
>          } else {
>              return Tuple!(__anotation(pure));
>          }
>      }
> }
>
> @MyPureFunction
> string mypurefunc() {
>     return "hi";
> }
>
> pragma(msg, mypurefunc);
>
> I added the constructor in there because being able to run code
> dependent on it would enable registering of certain types (useful
> for e.g. Cmsed so users don't have to).
> This would add a new keyword (__annotation) in the same style as
> __traits.
> __annotation function would be called post constructor meaning
> you could negate what you would normally return.
> Perhaps another function !__annotation to remove current ones.
>
> Not quite sure how this would relate to @nogc but.. Maybe it
> means we can fine tune it per attribute/compiler or something.
>
> But hey just my take.

The way I see it is that attributes, in general, are simply meta tags applied to things. Just like tags used in audio files.  You apply tags, do things with them. You can sort them based on tag, only play certain song with certain tags, etc...

Hence, the compiler does not to be coded for specific attributes because attributes, in and of themselves do not need code at the compiler level to handle them except for possibly optimizations. In any case, there should then be a common base for attributes(built in or user defined) that generalizes what they are now which would also make them more powerful with a little work.

As far as @nogc goes, I think it's probably one of the more simpler tags in that it it only uses inheritance and logical and composition. i.e., if all sub-attribute uses are @nogc then the the attribute use becomes @nogc.

E.g., Q uses X, Y, Z which each have attributes x, y, z

Then Q's attribute q = (x == y == z == @nogc) ? @nogc : @gc

[To start off we could say Q's attribute q = x:y:z, a compound attribute... and use a "grammar" to simplify q if possible. If q's simplified compound attribute exists in the defined attributes then processing continues until a atomic attribute is found... of course one has to be careful with ambiguity but probably won't ever be much of a problem]

Simple expressions and statements can also have implicitly defined attributes.

e.g., all of D is, by default @nogc except slicing, new, etc.. The GC module then introduces the @gc attribute on some of it's functions(not that it really matters cause usually you don't use the gc directly but through the core features of D(slicing, new, etc...).

If @nogc is defined as above then everything should just *work*. The compiler will form compound attributes and simplify them and @nogc will propagate. You would rarely ever have to explicitly use @nogc or @gc.

Same thing would happen with @pure, @safe, etc... Using the rules and possible some deduction the compiler could cute out most of the work.

e.g., @pure could be automatically deduced in almost all cases(99% of use cases) by marking everything that is no an assignment in D as pure. Assignments may or may not be pure depending on if they modify parent scope. The compiler determines if they do, and if they do then it marks that assignment @impure... then uses the same rule for combining all the attributes as @nogc.

If the compound attribute can't be reduced. e.g., some assignments in D can't be determined as pure or not by D, then the pureness is unknown which results in a compound attribute that can't be reduced... and hence the user must explicitly mark. (e.g., using a pointer assignment which can't guarantee pureness unless, maybe, static analysis is used to try and figure it out.

So, in D, some assignments are pure, some are impure, some are not provably pure and some are not provably impure. In the case of not probably pure/impure one might allow the user force the attribute to be one or the other so the compound attribute can be resolved. If that is not enough the user could tell the compiler to assume the compound attribute resolves to something.

Anyways, really feels like there is some simple mathematical structure going on here but I can't quite put my finger on it. We have a set of attributes, a set of operators on the set of attributes(probably just need symmetric binary operators but would be nice to be able to handle the more general case), and some "code" to do things with them so user defined attributes are more useful.


For example, suppose we have a complex program with two functions. Depending on something one function may or may not be pure. But lets suppose there is an external reason why the other function must always have the same pureness as the first function. This seems like a difficult problem because it depends on context.

If we have CT reflection of attributes and the ability to define attributes in cool ways then

void foo() { } // pure
void bar() @@forceSame(foo, @#pureness)
{ }


where @@forceSame is an attribute meta function that takes a symbol and a attribute group, which is the group of related attributes, in this case @#pureness = {@pure, @impure}, and forces bar to have the same attribute as foo that is within that group.

D being as cool as it is, we could even execute user defined code:

void foo() { } // pureness unknown
void bar() @@execute({ if (foo:attributes.is(@pure)) @@set(bar, @disable)}
{ }


(just pseudo code so don't get your panties in a wad, but should be obvious, if foo is pure then bar is disabled, else it is enabled)

Of course one can take it as far as one wants... The point is, that we have attributes but no underlying common system which to easily build on. Seems like every attribute added to D requires re-implementing the wheel more or less.

If such a system was already in D, there would not be any discussion about @nogc except possibly should it become a built in attribute which the compiler could use for optimization purposes.

On Saturday, 19 April 2014 at 18:41:39 UTC, Walter Bright wrote:
> On 4/19/2014 11:05 AM, Dicebot wrote:
>> I feel like the origin of the discussion has been completely lost here and we
>> don't speak the same language right now. The very point I have made initially is
>> that @nogc in a way it is defined in your DIP is too restrictive to be
>> effectively used in Phobos.
>>
>> In lot of standard library functions you may actually need to allocate as part
>> of algorithm, strict @nogc is not applicable there. However, it is still
>> extremely useful that no _hidden_ allocations happen outside of weel-defined
>> user API and this is something that less restrictive version of @nogc could help
>> with.
>>
>> The fact that you propose me to use unit tests to verify same guarantees hints
>> that I have completely failed to explain my proposal but I can't really rephrase
>> it any better without some help from your side to identify the point of confusion.
>
> The way I understood your idea, was that a template could be marked @nogc, and yet still allow template arguments that themselves may gc.
>
> This can be accomplished by creating a unit test that passes non-allocating template parameters, and then verifying the instantiation is @nogc.

The only way that works is if the unittest has coverage of all possible currently non-GC-using instantiations of all templates all the way down the call-tree.*

Imagine the case where some function deep down the call-tree has a `static if(T == NastyType) doGCStuff();`.

In order to protect against this, you have to check the internals of the entire call-tree in order to write the required unittest, and verify manually that you haven't missed a case every time anything changes.

*alright, technically only those that can be instantiated by the function your testing, but this still blows up pretty fast.

On Sunday, 20 April 2014 at 14:38:47 UTC, Frustrated wrote:
> Why not allow one to define their own attributes from a generalized subset and then define a few standard ones like @nogc.
>
> i.e., instead of having to define specific attributes every few years to satisfy some new thing, why not just abstract the process.
>
> Attributes, I believe, are essentially relationships between parts of code?
>
> If so, then one simply has to implement some generic way to specify the attributes and properties of the relationship to the compiler. Then anyone would have the tools to define and use these attributes as they wish.

This could be very powerful, but I think you would need to put attributes in namespaces if you expand their usage.

You could also get better concurrency warnings/optimization if you could distinguish between reads/writes to specific types, type members, globals, and function calls and tie it to mutexes.

Unfortunately, it isn't sufficient, since you often want to tag sets of instances, not sets of definitions (which is what attributes do, it creates sets of defs and give the set a name).

Timon Gehr:

> In which case? In case some version of LDC2 is able to avoid the heap allocation using full optimizations? :o)

Please take a look, I have just added one more note in the optimizations section:

http://wiki.dlang.org/DIP60#Behaviour_in_presence_of_optimizations

Bye,
bearophile