On Saturday, 3 May 2014 at 11:12:56 UTC, Michel Fortin wrote:
> On 2014-05-01 17:35:36 +0000, "Marc Schütz" <schuetzm@gmx.net> said:
>
>> Maybe the language should have some way to distinguish between GC-managed and manually-managed objects, preferably in the type system. Then it could be statically checked whether an object is supposed to be GC-managed, and consequentially shouldn't have a destructor.
>
> Or turn the rule on its head: make it so having a destructor makes the heap memory block reference counted. With this adding a destructor always cause deterministic destruction.
>
> The compiler knows statically whether a struct has a destructor. For a class you need a runtime trick because the root object which can be either. Use a virtual call or a magic value in the reference count field to handle the reference count management. You also need a way to tag a class to be guarantied it has no derived class with a destructor (to provide a static proof for the compiler it can omit ARC code), perhaps @disable ~this().
>
> Then remains the problem of cycles. It could be a hard error if the destructor is @safe (error thrown when the GC collects it). The destructor could be allowed to run (in any thread) if the destructor is @system or @trusted.
>
> The interesting thing with this is that the current D semantics are preserved, destructors become deterministic (except in the presence of cycles, which the GC will detect for you), and if you're manipulating pointers to pure memory (memory blocks having no destructor) there's no ARC overhead. And finally, no new pointer attributes; Walter will like this last one.

This is certainly also an interesting idea, but I suspect it is bound to fail, simply because it involves ARC. Reference counting always makes things so much more complicated... See for example the cycles problem you mentioned: If you need a GC for that, you cannot guarantee that the objects will be collected, which was the reason to introduce ARC in the first place. Then there are the problems with shared vs. thread-local RC (including casting between the two), and arrays/slices of RC objects. And, of course, Walter doesn't like it ;-)

On 2014-05-04 09:00:45 +0000, "Marc Schütz" <schuetzm@gmx.net> said:

> On Saturday, 3 May 2014 at 11:12:56 UTC, Michel Fortin wrote:
>> Or turn the rule on its head: make it so having a destructor makes the heap memory block reference counted. With this adding a destructor always cause deterministic destruction.
>> 
>> The compiler knows statically whether a struct has a destructor. For a class you need a runtime trick because the root object which can be either. Use a virtual call or a magic value in the reference count field to handle the reference count management. You also need a way to tag a class to be guarantied it has no derived class with a destructor (to provide a static proof for the compiler it can omit ARC code), perhaps @disable ~this().
>> 
>> Then remains the problem of cycles. It could be a hard error if the destructor is @safe (error thrown when the GC collects it). The destructor could be allowed to run (in any thread) if the destructor is @system or @trusted.
>> 
>> The interesting thing with this is that the current D semantics are preserved, destructors become deterministic (except in the presence of cycles, which the GC will detect for you), and if you're manipulating pointers to pure memory (memory blocks having no destructor) there's no ARC overhead. And finally, no new pointer attributes; Walter will like this last one.
> 
> This is certainly also an interesting idea, but I suspect it is bound to fail, simply because it involves ARC. Reference counting always makes things so much more complicated... See for example the cycles problem you mentioned: If you need a GC for that, you cannot guarantee that the objects will be collected, which was the reason to introduce ARC in the first place. Then there are the problems with shared vs. thread-local RC (including casting between the two), and arrays/slices of RC objects. And, of course, Walter doesn't like it ;-)

Arrays/slices of RC objects would have to be reference counted too. That was part of Andrei's proposal.

Cycles break deterministic destruction and break the guaranty the destructor will be called because we're relying on the GC to free them, true. But today we have no guaranty at all, even when there's no cycle. It is possible to avoid cycles, and not that hard to do in a safe way when you have weak (auto-nulling) pointers (which D should have regardless of the memory management scheme). It's even easier if you use the GC to detect them. I'm proposing that GC calls to a @safe destructor be a hard error so you can use that error to fix your cycles. (Note that cycles with no destructor are perfectly fine though.)

As for shared vs. thread-local RC, Andrei proposed a solution for that earlier.
http://forum.dlang.org/post/ljrr8l$2dt1$1@digitalmars.com

I know Walter doesn't like to have pointers variables do magic stuff. This is why I'm confining ARC behaviour to objects, structs, and arrays having destructors. Andrei's initial proposal was to completely remove destructors for objects; all the pointers that would have survived that proposal are still RC-free in mine except for non-final classes when you haven't explicitly disabled the destructor. (An alternative to disabling the destructor would be to use a different root object for RC, I just find that less desirable.)

Frankly, if we want destructors to work safely when on the heap, I don't see another way.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On Sat, May 03, 2014 at 10:48:47PM -0500, Caligo via Digitalmars-d wrote: [...]
> Last but not least, currently there are two main ways for new features to make it into D/Phobos: you either have to belong to the inner circle, or have to represent some corporation that's doing something with D.

I'm sorry, but this is patently false. I am neither in the inner circle, nor do I represent any corporation, yet I've had many changes pulled into Phobos (including brand new code).

I can't say I'm perfectly happy with the D development process either, but this kind of accusation is bordering on slander, and isn't helping anything.


T

-- 
Why are you blatanly misspelling "blatant"? -- Branden Robinson

On Sun, May 4, 2014 at 12:22 AM, Andrei Alexandrescu via Digitalmars-d < digitalmars-d@puremagic.com> wrote:
>
>
> Mostly good points, but the bountysource program is an experiment by Facebook, not by myself. And (without me trying to speak on Facebook's behalf) it would be difficult to argue that Facebook doesn't understand FOSS or is out there to insult contributors. We're just experimenting with various angles.


If the bounty system was such a great idea, then every FOSS project would be using it.  Now, hiring full-time engineers to work on a FOSS project, that's an entirely different issue.  Besides, if someone is trying to figure out how FOSS teams manage to become successful in regards to development and all the associated technical and social complexities, then all they have to do is study one of the million different FOSS projects out there.  Many well known FOSS contributors have actually documented their experience and knowledge of managing FOSS projects.


>
> The on/off switch may be a nice idea in the abstract but is hardly the perfect recipe to good language feature development; otherwise everybody would be using it, and there's not overwhelming evidence to that. (I do know it's been done a few times, such as the (in)famous "new scoping rule of the for statement" for C++ which has been introduced as an option by VC++.)
>
>
No, it's nothing abstract, and it's very practical and useful.  Rust has
such a thing, #![feature(X,Y,Z)].  So does Haskell, with {-# feature #-}.
 Even Python has __future__, and many others.


> I wonder how you've gotten the perception that one needs to be a member of the inner circle mafia to get things into D. Today's major contributors to D came from all over, without any preexisting relationship to anyone else, and their admission ticket has been getting work done. Could you please get into detail on how you view things? (I tried to look over your past posts to see a pattern of rejected contributions, but didn't find such.)
>
>
>
I wasn't trying to imply that contributions are rejected if contributors are not members of a certain group.  I was just trying to say that it's more likely for a _new_ feature to make it into D/Phobos if they are proposed by members of the inner circle or someone representing a corporation, probably because they become more noticeable, or because they get lost in all the forum noise, I don't know.  I could be wrong, but that's just how I perceive it.



>
> Actually I'd love to get you started so I'd understand your angle better. I'm sure we can do a lot better. One good thing Phobos reviews have done since we initiated them has been to prevent bad artifacts to make it into the library. We'd love to make it better. From what I saw witnessing similar processes (C++, Boost, Python, Scala) - they all have some sense of awkward self-importance to them upon the first look. I think that's the way such things work.
>
>
>
Here is an idea:  include new features in DMD/Phobos as soon as they arrive, and make them part of the official binary release so that the average D user can try them out.  Make sure they are marked as unstable, and put a on/off switch on them (something like what Rust/Haskell have; not a compiler switch).  If the feature receives no implementation bug reports for X consecutive days AND no design bug reports for Y consecutive days, then the feature is marked stable and officially becomes part of DMD/Phobos.  The X and the Y can be decreased as D's number of users increases over the years.  The whole idea is very much like farming: you are planting seeds.  As the plants grow, some of them will not survive, others will be destroyed, and some of them will take years to grow.  In any case, you harvest the fruits when they are ready.

 Here are good starting values for X and Y:
X = 90 days
Y = 180 days

On 5/4/14, 5:38 PM, Caligo via Digitalmars-d wrote:
> On Sun, May 4, 2014 at 12:22 AM, Andrei Alexandrescu via Digitalmars-d
> <digitalmars-d@puremagic.com <mailto:digitalmars-d@puremagic.com>> wrote:
>
>
>     Mostly good points, but the bountysource program is an experiment by
>     Facebook, not by myself. And (without me trying to speak on
>     Facebook's behalf) it would be difficult to argue that Facebook
>     doesn't understand FOSS or is out there to insult contributors.
>     We're just experimenting with various angles.
>
>
> If the bounty system was such a great idea, then every FOSS project
> would be using it.

As I said: experiment.

> Now, hiring full-time engineers to work on a FOSS
> project, that's an entirely different issue.  Besides, if someone is
> trying to figure out how FOSS teams manage to become successful in
> regards to development and all the associated technical and social
> complexities, then all they have to do is study one of the million
> different FOSS projects out there.  Many well known FOSS contributors
> have actually documented their experience and knowledge of managing FOSS
> projects.

Great, a few representative links would be most welcome.

> Here is an idea:  include new features in DMD/Phobos as soon as they
> arrive, and make them part of the official binary release so that the
> average D user can try them out.  Make sure they are marked as unstable,
> and put a on/off switch on them (something like what Rust/Haskell have;
> not a compiler switch).  If the feature receives no implementation bug
> reports for X consecutive days AND no design bug reports for Y
> consecutive days, then the feature is marked stable and officially
> becomes part of DMD/Phobos.  The X and the Y can be decreased as D's
> number of users increases over the years.  The whole idea is very much
> like farming: you are planting seeds.  As the plants grow, some of them
> will not survive, others will be destroyed, and some of them will take
> years to grow.  In any case, you harvest the fruits when they are ready.
>
>   Here are good starting values for X and Y:
> X = 90 days
> Y = 180 days

This is nice, but on the face of it it's just this: an idea on how other people should do things on their free time. I'd have difficulty convincing people they should work that way. The kind of ideas that I noticed are successful are those that actually carry the work through and serve as good examples to follow.


Andrei

On Sun, May 4, 2014 at 11:09 PM, Andrei Alexandrescu via Digitalmars-d < digitalmars-d@puremagic.com> wrote:
>
>
> Great, a few representative links would be most welcome.
>
>
>
Here is a good starting point (it's a classic):

http://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar


 Here is an idea:  include new features in DMD/Phobos as soon as they
>> arrive, and make them part of the official binary release so that the average D user can try them out.  Make sure they are marked as unstable, and put a on/off switch on them (something like what Rust/Haskell have; not a compiler switch).  If the feature receives no implementation bug reports for X consecutive days AND no design bug reports for Y consecutive days, then the feature is marked stable and officially becomes part of DMD/Phobos.  The X and the Y can be decreased as D's number of users increases over the years.  The whole idea is very much like farming: you are planting seeds.  As the plants grow, some of them will not survive, others will be destroyed, and some of them will take years to grow.  In any case, you harvest the fruits when they are ready.
>>
>>   Here are good starting values for X and Y:
>> X = 90 days
>> Y = 180 days
>>
>
> This is nice, but on the face of it it's just this: an idea on how other people should do things on their free time. I'd have difficulty convincing people they should work that way. The kind of ideas that I noticed are successful are those that actually carry the work through and serve as good examples to follow.
>
>
> Andrei
>
>
*sigh*

On 5/4/14, 9:58 PM, Caligo via Digitalmars-d wrote:
> On Sun, May 4, 2014 at 11:09 PM, Andrei Alexandrescu via Digitalmars-d
> <digitalmars-d@puremagic.com <mailto:digitalmars-d@puremagic.com>> wrote:
>
>
>     Great, a few representative links would be most welcome.
>
>
> Here is a good starting point (it's a classic):
>
> http://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar

Of course I've read that, and multiple times! I thought of mentioning it but concluded it would be trite. -- Andrei

On Sunday, 4 May 2014 at 22:56:41 UTC, H. S. Teoh via
Digitalmars-d wrote:
> On Sat, May 03, 2014 at 10:48:47PM -0500, Caligo via Digitalmars-d wrote:
> [...]
>> Last but not least, currently there are two main ways for new features
>> to make it into D/Phobos: you either have to belong to the inner
>> circle, or have to represent some corporation that's doing something
>> with D.
>
> I'm sorry, but this is patently false. I am neither in the inner circle,
> nor do I represent any corporation, yet I've had many changes pulled
> into Phobos (including brand new code).
>
> I can't say I'm perfectly happy with the D development process either,
> but this kind of accusation is bordering on slander, and isn't helping
> anything.
>
>
> T

There is a lot of truth in what Caligo has said, but I would word
that part of it differently.

A couple years ago I submitted std.rational, but it didn't go
anywhere.  About a year later I discovered that someone else had
done a similar thing, but it never made it into Phobos either.
Of course, it's not because we didn't belong to some "inner
circle", but I think it has to do with the fact that D has a very
poor development process.  The point being, something as simple
as a Rational library shouldn't take years for it to become part
of Phobos, specially when people are taking the time to do the
work.

--Arlen

On 05/05/14 00:55, H. S. Teoh via Digitalmars-d wrote:

> I'm sorry, but this is patently false. I am neither in the inner circle,
> nor do I represent any corporation, yet I've had many changes pulled
> into Phobos (including brand new code).

I think he's referring to language changes. Things that will require a DIP, i.e. not just adding a new __trait.

-- 
/Jacob Carlborg

On Mon, May 05, 2014 at 06:16:34AM +0000, Arlen via Digitalmars-d wrote:
> On Sunday, 4 May 2014 at 22:56:41 UTC, H. S. Teoh via Digitalmars-d wrote:
> >On Sat, May 03, 2014 at 10:48:47PM -0500, Caligo via Digitalmars-d wrote: [...]
> >>Last but not least, currently there are two main ways for new features to make it into D/Phobos: you either have to belong to the inner circle, or have to represent some corporation that's doing something with D.
> >
> >I'm sorry, but this is patently false. I am neither in the inner circle, nor do I represent any corporation, yet I've had many changes pulled into Phobos (including brand new code).
> >
> >I can't say I'm perfectly happy with the D development process either, but this kind of accusation is bordering on slander, and isn't helping anything.
[...]
> There is a lot of truth in what Caligo has said, but I would word that part of it differently.
> 
> A couple years ago I submitted std.rational, but it didn't go anywhere.  About a year later I discovered that someone else had done a similar thing, but it never made it into Phobos either. Of course, it's not because we didn't belong to some "inner circle", but I think it has to do with the fact that D has a very poor development process.  The point being, something as simple as a Rational library shouldn't take years for it to become part of Phobos, specially when people are taking the time to do the work.
[...]

This wording is much more acceptable. ;-) While I think accusations of an "elite inner circle" are unfounded (and unfair), I do agree with the sentiment. I think some time ago there was some talk about very old pull requests that have been stuck at the bottom of the queue for months or even years, and nobody was looking at them. I don't know what came out of that talk, though -- apparently not very much. :-(

OTOH, I did find that stubbornness and persistence help. If you keep pestering everybody about your new contribution, and keep pushing it even if people seem to ignore/dislike it, keep updating your pull even if it seems nobody cares, eventually somebody will take notice and do something about it. Of course, this is not ideal -- open source projects really should be actively welcoming new contributions, not merely passively accepting them -- but that's the way it is right now, and I'm not sure how to change that.  Perhaps stubborn and persistent pestering of the PTBs until they change?  Might help, you never know. ;-)


T

-- 
Unix was not designed to stop people from doing stupid things, because that would also stop them from doing clever things. -- Doug Gwyn