On 3 May 2014 18:49, Benjamin Thaut via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> Am 30.04.2014 22:21, schrieb Andrei Alexandrescu:
>>
>> Walter and I have had a long chat in which we figured our current offering of abstractions could be improved. Here are some thoughts. There's a lot of work ahead of us on that and I wanted to make sure we're getting full community buy-in and backup.
>>
>> First off, we're considering eliminating destructor calls from within the GC entirely. It makes for a faster and better GC, but the real reason here is that destructors are philosophically bankrupt in a GC environment. I think there's no need to argue that in this community.
>>
>> The GC never guarantees calling destructors even today, so this decision would be just a point in the definition space (albeit an extreme one).
>>
>> That means classes that need cleanup (either directly or by having fields that are structs with destructors) would need to garner that by other means, such as reference counting or manual. We're considering deprecating ~this() for classes in the future.
>>
>> Also, we're considering a revamp of built-in slices, as follows. Slices of types without destructors stay as they are.
>>
>> Slices T[] of structs with destructors shall be silently lowered into RCSlice!T, defined inside object.d. That type would occupy THREE words, one of which being a pointer to a reference count. That type would redefine all slice primitives to update the reference count accordingly.
>>
>> RCSlice!T will not convert implicitly to void[]. Explicit cast(void[]) will be allowed, and will ignore the reference count (so if a void[] extracted from a T[] via a cast outlives all slices, dangling pointers will ensue).
>>
>> I foresee any number of theoretical and practical issues with this approach. Let's discuss some of them here.
>>
>>
>> Thanks,
>>
>> Andrei
>
>
> Honestly, that sounds like the entierly wrong apporach to me. Your approaching the problem in this way:
>
> "We can not implement a propper GC in D because the language design prevents us from doing so. So lets remove destructors to migate the issue of false pointers."
>
> While the approach should be.
>
> "The language does not allow to implement a propper GC (anything else then dirty mark & sweep), what needs to be changed to allow a implementation of a more sophisticated GC."

Couldn't agree more.
Abandoning destructors is a disaster.
Without destructors, you effectively have manual memory management, or
rather, manual 'resource' management, which is basically the same
thing, even if you have a GC.
It totally undermines the point of memory management as a foundational
element of the language if most things are to require manual
release/finalisation/destruction or whatever you wanna call it.


> Also let me tell you that at work we have a large C# codebase which heavily relies on resource management. So basically every class in there inherits from C#'s IDisposable interface which is used to manually call the finalizer on the class (but the C# GC will also call that finalizer!). Basically the entire codebase feels like manual memory management. You have to think about manually destroying every class and the entire advantage of having a GC, e.g. not having to think about memory management and thus beeing more productive, vanished. It really feels like writing C++ with C# syntax. Do we really want that for D?

This is interesting to hear someone else say this. I have always found
C# - an alleged GC language - to result in extensive manual memory
management in practise too.
I've ranted enough about it already, but I have come to the firm
conclusion that the entire premise of a mark&sweep GC is practically
corrupt. Especially in D.
Given this example that you raise with C#, and my own experience that
absolutely parallels your example, I realise that GC's failure extends
into far more cases than just the ones I'm usually representing.

I also maintain that GC isn't future-proof in essence. Computers grow
exponentially, and GC performance inversely tracks the volume of
memory in the system. Anything with an exponential growth curve is
fundamentally not future-proof.
I predict a 2025 Wikipedia entry: "GC was a cute idea that existed for
a few years in the early 2000's while memory ranged in the 100's mb -
few gb's, but quickly became unsustainable as computer technology
advanced".


> And what if I want unsafe slices of structs with destructors, for performance? Maybe I perfectly know that the memory behind the slice will outlive the slice, and I don't want the overhead of all the reference counthing behind it?
>
> If you actually deprecate ~this, there would be two options for me.
> 1) Migrate my entire codebase to some user defiend finalizer function (which
> doesn't have compiler support), which would be a lot of work.

Does ~this() actually work, or just usually work?
Do you call your destructors manually like C#?

> 2) Quit D. (which is becomeing more and more an option when reading the
> recent news group discussions.)

I'm starting to fear the same outcome for myself.
I don't have any idea how to reconcile this problem in my working
environment, and very little community sympathy. I'm not seeing real
solutions emerging, and the only one I can imagine that's even
theoretically possible is wildly unpopular (ARC).
For years, I just (naively?) assumed that the GC was immature, and
would improve with time. Never gave it much thought; assumed there
were people much smarter than me with a plan...

I can't imagine any way out of this without significant breaking changes to the type system. Probably a new pointer type at least.


This thread is starting to convince me that the GC should probably be
repositioned as a convenience library provided *beside* the language,
rather than a foundation of the language. It should be exclusively
opt-in, not opt-out, and upon opting-in, you accept the associated
problems.
The revelation that established GC languages like C# are actually
broken too hadn't occurred to me until now, but if I had to nominate
the single biggest disaster in C# from my experience, that's certainly
it; it's built on a GC, but isn't really compatible with it either.
I'm constantly cleaning up manually in C#, which leaves very little
value in the feature, and a definite tendency to produce unreliability
by users who aren't experts on garbage collection and presume it
should 'just work'.

I support the notion that if the GC isn't removed as a foundational feature of D, then destructors should probably be removed from D. That said, I really want my destructors, and would be very upset to see them go. So... ARC?

> That said, I really want my destructors, and would be very upset to
> see them go. So... ARC?

Manu, can you direct me what is ARC? This abbreviation is very
misgooglly.

On 6 May 2014 13:51, HaraldZealot via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> That said, I really want my destructors, and would be very upset to see them go. So... ARC?
>
>
> Manu, can you direct me what is ARC? This abbreviation is very misgooglly.

Automatic reference counting, and solution used by Apple in Obj-C. There has been massive debate on the topic already, but it's generally been dismissed.

On 5/5/14, 8:19 PM, Manu via Digitalmars-d wrote:
> On 5 May 2014 14:09, Andrei Alexandrescu via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> This is nice, but on the face of it it's just this: an idea on how other
>> people should do things on their free time. I'd have difficulty convincing
>> people they should work that way. The kind of ideas that I noticed are
>> successful are those that actually carry the work through and serve as good
>> examples to follow.
>
> There's imperfect but useful pull requests hanging around for years,
> extern(Obj-C) for instance, which may be useful as an experimental
> feature to many users, even if it's not ready for inclusion in the
> official feature list and support.
> I suspect it's (experimental) presence would stimulate further
> contribution towards D on iOS for instance; it may be an enabler for
> other potential contributors.

So it would be nice if you reviewed that code.

> What about AST macros? It seems to me that this is never going to be
> explored and there are competing proposals, but I wonder if there's
> room for experimental implementations that anyone in the community can
> toy with?

There would be of course room as long as there's be one or more champions for it. Would that be something you'd be interested in?

> UDA's are super-useful, but they're still lacking the thing to really
> set them off, which is the ability to introduce additional boilerplate
> code at the site of the attribute.

Interesting. Have you worked on a related proposal?

> I reckon there's a good chance that creating a proper platform for
> experimental features would also have an advantage for community
> building and increase contribution in general. If new contributors can
> get in, have some fun, and start trying their ideas while also being
> able to share them with the community for feedback without fear
> they'll just be shot down and denied after all their work... are they
> not more likely to actually make a contribution in the first place?

I'd say so, but we'd need initiative and quite a bit of work for such a platform. Would you be interested?

> Once they've made a single contribution of any sort, are they then
> more likely to continue making other contributions in the future
> (having now taken the time to acclimatise themselves with the
> codebase)?

I agree - and that applies to you, too.

> I personally feel the perceived unlikeliness of any experimental
> contribution being accepted is a massive deterrence to making compiler
> contributions in the first place by anyone other than the most serious
> OSS advocates.

Contributions make it into the compiler and standard library if and they are properly motivated, well done, and reviewed by the core team which is literally self-appointed. The key to being on the core team is just reviewing contributions. Have you considered looking at submissions that are "hanging around for years"?

> I have no prior experience with OSS, and it's certainly
> a factor that's kept me at arms length.

It's as easy as just reviewing stuff. Acta, non verba.


Andrei

Am Mon, 05 May 2014 17:24:38 +0000
schrieb "Dicebot" <public@dicebot.lv>:

> > That experimental package idea that was discussed months ago comes to my mind again. Add that thing as exp.rational and have people report bugs or shortcomings to the original author. When it seems to be usable by everyone interested it can move into Phobos proper after the formal review (that includes code style checks, unit tests etc. that mere users don't take as seriously).
> 
> And same objections still remain.

Sneaky didn't work this time.

-- 
Marco

I have to say that all this discussion (more precisely the understanding on the side of key developers) make me very upset.

It's good that Andrei agreed with impossibility of the harebrained disallowing of the class destructors. But I was very surprise, that so thought go to such head, because such solution contradicts the D spirit totally. There are many language which are very popular and have many dark moments in their design. I (and I think not only me) go to the D not for its popularity, but for its clarity, power and SANITY (that bases on strong guaranties). The strong solutions found on the strong decision makes D itself. (Examples of such strong solutions: immutabilities, default unshareness, struct and class as distinct being). And way that leads us in state where stucts have dtors and classes haven't but allow struct with dtor as member and people have to call struct dtor manually, isn't D way. Because such way relies on programmers discipline, but how Andrei has written "If there one thing that decades of computing have taught us, it must be that discipline-oriented programming does not scale."[TDPL, p. 399].

Our negative filling flood out may be sane from psychologically view, but neither sane nor constructive for D future. For solving problem it's need its formulate. We have to state that current state (lack of structs' dtors call guaranty) is insane, the harebrained disallowing of the class destructors is insane too. And what is sane? If I properly understand philosophy of D, we need semiautomated (not full) resource manager with strong guaranty and good performance, and which automated mode covers the most part of use-case. It is the target. Garbage collection or reference counting or any possible third way is a detail therefor task and mean not a target. And one task, that lays on the way to target, is minimal rape of D2 language (even if solution will be D3), so IMO dtors (perhaps only for structs) must survive.

I notice that I view only part of problem, can anybody link or describe me completely state and problems of current garbage collection and other resource management? It help me in finding of existence solution (at least theoretical).

---
Alaksiej Stankievič

On 06/05/14 05:51, HaraldZealot wrote:

> Manu, can you direct me what is ARC? This abbreviation is very
> misgooglly.

Automatic Reference Counting. Like regular RC but the compiler automatically inserts calls to release/free.

-- 
/Jacob Carlborg

On 06/05/14 08:07, HaraldZealot wrote:

> I notice that I view only part of problem, can anybody link or describe
> me completely state and problems of current garbage collection and other
> resource management? It help me in finding of existence solution (at
> least theoretical).

The major issue with the garbage collector is that it's not guaranteed to run a collection. When a collection is run the GC will call the destructors for the objects it collects. If there's no guarantee a collection is run there can be no guarantee that destructors are called. A collection is usually run when allocating new memory and there's not enough memory available.

-- 
/Jacob Carlborg

On Tuesday, 6 May 2014 at 06:07:41 UTC, HaraldZealot wrote:
> I notice that I view only part of problem, can anybody link or describe me completely state and problems of current garbage collection and other resource management? It help me in finding of existence solution (at least theoretical).

A precise scanning GC is the only robust general solution. RC with weak pointers can only account for a subset of all possible models.

But I agree with you, the language should be redesigned to have a GC friendly set of D constructs where FFI is followed by programmer guaranteed postconditions (specified by library authors).

In other words the @nogc appoach is not sufficient, a @gc approach is needed.

On 6 May 2014 14:09, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 5/5/14, 8:19 PM, Manu via Digitalmars-d wrote:
>>
>> On 5 May 2014 14:09, Andrei Alexandrescu via Digitalmars-d
>>
>> <digitalmars-d@puremagic.com> wrote:
>>>
>>> This is nice, but on the face of it it's just this: an idea on how other
>>> people should do things on their free time. I'd have difficulty
>>> convincing
>>> people they should work that way. The kind of ideas that I noticed are
>>> successful are those that actually carry the work through and serve as
>>> good
>>> examples to follow.
>>
>>
>> There's imperfect but useful pull requests hanging around for years,
>> extern(Obj-C) for instance, which may be useful as an experimental
>> feature to many users, even if it's not ready for inclusion in the
>> official feature list and support.
>> I suspect it's (experimental) presence would stimulate further
>> contribution towards D on iOS for instance; it may be an enabler for
>> other potential contributors.
>
>
> So it would be nice if you reviewed that code.

I don't really know anything about it... and that's not the point. I'm just suggesting by my prior email that some steps like creating an experimental space with a lower barrier to entry might encourage growth in the number of overall contributors, which I think was the basic flavour of the emails leading up to it.

>> What about AST macros? It seems to me that this is never going to be explored and there are competing proposals, but I wonder if there's room for experimental implementations that anyone in the community can toy with?
>
>
> There would be of course room as long as there's be one or more champions for it. Would that be something you'd be interested in?

I have no horse in that race, but I see it come up all the time, and it is something I am passively interested in.

There's at least one DIP which received little attention afaict, it's
an example of something that I think would probably manifest into code
in an experimental space, but clearly couldn't be accepted as a
language feature without lots of field time.
In lieu of an experimental space, there will be no action.

It's an interesting example actually. I think lots feel the DIP isn't really an effective solution, but nobody has the motivation or ideas to refine it. The DIP author clearly has no motivation to test it experimentally, but perhaps that's what it needs to progress? The DIP's shortcomings might be discovered by experimental users in the field? It's hard to know, but it's an example of the sort of things that may have a stifling effect on progress and contribution.

>> UDA's are super-useful, but they're still lacking the thing to really set them off, which is the ability to introduce additional boilerplate code at the site of the attribute.
>
>
> Interesting. Have you worked on a related proposal?

Not really, I've initiated numerous discussions which always seems to
end at AST macros.
The only other semi-reasonable idea I've had is the concept that
tagging mixin templates as UDA's might be a practical angle, but it
doesn't really make clean sense, creates a syntactic special case and
also doesn't seem powerful enough, so I'm not with any solid proposal
that I can imagine within the current language framework.

There are presently bigger issues that keep me awake at night.

>> I reckon there's a good chance that creating a proper platform for experimental features would also have an advantage for community building and increase contribution in general. If new contributors can get in, have some fun, and start trying their ideas while also being able to share them with the community for feedback without fear they'll just be shot down and denied after all their work... are they not more likely to actually make a contribution in the first place?
>
>
> I'd say so, but we'd need initiative and quite a bit of work for such a platform. Would you be interested?

Well, in phobos, just approve 'exp' which has been raised countless
times. I've got contributions that should be in exp, but instead,
they're in limbo, and I've lost momentum and motivation since their
completion is blocked by other issues, and I'm receiving no feedback
from field testing.
What happened to std.serislisation? There was motion there a year or
so back... I was looking forward to it, and did some minor reviewing
at the time. I wonder if that's an interesting case study? (I haven't
looked)

In the compiler... I may be interested, but I don't have any such
compiler feature in mind to motivate the effort.
I have no idea what an experimental feature platform should look like
in the compiler, and if it were to exist, I have no such feature in
mind to make use of it, but I have raised examples of others that
have.

>> Once they've made a single contribution of any sort, are they then more likely to continue making other contributions in the future (having now taken the time to acclimatise themselves with the codebase)?
>
>
> I agree - and that applies to you, too.

Sure, but my point... is below.

>> I personally feel the perceived unlikeliness of any experimental contribution being accepted is a massive deterrence to making compiler contributions in the first place by anyone other than the most serious OSS advocates.
>
>
> Contributions make it into the compiler and standard library if and they are properly motivated, well done, and reviewed by the core team which is literally self-appointed. The key to being on the core team is just reviewing contributions. Have you considered looking at submissions that are "hanging around for years"?

Perhaps you misunderstood the point of my post. I've watched people
make solid contributions that haven't gotten through. That is
discouraging to others considering starting their own work, and for
the person who has already put in the effort to continue to do so in
the future.
The Obj-C thing as an example. Granted, it's a huge feature and has
extensive implications. The Authors have said themselves that they
agree it's not 'ready' for inclusion... so, what? It sits and rots?
I think it needs an experimental place to live and have people make
use of it for what it is. If it's blocked by other unpopular issues
that aren't receiving attention, perhaps it's presence will manifest
the appropriate motivation to see those other unpopular issues
resolved at some point?
My point is that successful OSS seems to be about enabling the
lowest-friction contribution, and my feeling right now, is that the
barrier to entry is high. Whether that's true or not I can't really
comment, but it's a perception, and the mental barrier inhibiting the
first step is perhaps the most significant barrier of all.

I can't review the Obj-C patch. 1, it's huge, 2, I don't know anything about it, other than I'd really like to use D on iOS and that's a major hurdle. Also, the authors themselves said they recognise it's not 'ready'. But is it 'experimental'?

>> I have no prior experience with OSS, and it's certainly
>> a factor that's kept me at arms length.
>
>
> It's as easy as just reviewing stuff. Acta, non verba.

I've never felt I have any particular authority to comment on pulls that I have no experience or vested interest in. (I do occasionally comment on pull requests that I have some interest or knowledge in) I've also had some (hopefully useful) commentary in features that did make it; Win64, UDA's, some traits extensions, and lots of bug reports and fix confirmations.

I'm plenty vocal and active on things I do feel I know about, but
they're often pretty radical, unpopular, and rarely come even close to
turning into code. I'm pretty certain that nothing left on my short
list that I personally *really* care about will ever get pulled, even
if I did do the work.
There's a perfectly good pull there for not-virtual-by-default. No
amount of beating will get that horse through, despite almost
unanimous community support. That was... extremely discouraging, to
say the least.