December 22, 2016
On Thursday, 22 December 2016 at 03:57:10 UTC, Chris Wright wrote:

> You can implement write barriers as runtime calls, but omit them in @nogc code.

That means redefining what @nogc means. Currently it basically means "does not GC-allocate" and you want to change it to "does not mutate GC-allocated objects" which is very different and hardly possible to check in compiler without further changing the language.

> However, this would be costly -- it's an expensive technique in general;

Yep, that's what I'm saying. Fast GC has a price on the rest code speed. Fast code has a price on GC. Unless you separate them very clearly by language means.


December 22, 2016
> Library Standardization:
> ------------------------
> 
> Some of the proposals sounds very correct. The library needs to be split. Every module needs its own GIT. People need to be able to add standard modules ( after approval ).

I can't agree with you there. There are a lot of dependencies between modules.

> No offense but where is the standard database library for D? There is none. That is just a load of bull. Anybody who wants to program in any language expect the language to have a standard database library! Not that you need to search the packages for a standard library. I have seen one man projects that have more standard library support then D.

Go, Java, and C# each have database _interfaces_ in their standard libraries. Most other languages don't.

These interfaces don't let you connect to any database. You still need a library to connect to a specific database.

With the rise of document databases and alternate query systems, it's harder to define a generic database library. It's still possible to write one for SQL databases.

> I do not use 3th party packages

The standard library needs a rigorous approval process. That takes time and human attention for the review and to address concerns identified during review.

D is marginal, and that means we simply don't have the time to get these things done.

> Documentation:
> --------------
> 
> I do not use it. Its such a mess to read with long paragraphs

Long paragraphs describing the details of what a thing does is generally a good thing. MSDN's documentation is like that.

The "mess" part is when we have five hundred identifiers documented in one web page. dpldocs.info is much better about this.

> and a LOT of stuff not even documented.

There's no excuse for that.

> This automated documentation generation is the same **** i see in other "new" languages.

The documentation is hand-written. We don't have a tool capable of annotating, say, std.string.detab with "Replace each tab character in s with the number of spaces necessary to align the following character at the next tab stop." without human intervention. That would be impressive, though.

The HTML is generated by a tool.

> Editor support:
> ---------------
> 
> What a struggle. Visual Studio C is probably the editor with the best 3th party support.

VS Code isn't terrible. It's got the disadvantage that it tries to autocomplete every possible thing -- so you get __monitor and __vptr first, and the other options include things you can't access.

> Too many need 3th party to do something that D needs to support from itself:
> 
> dcd - Used for auto completion
> dfmt - Used for code formatting
> dscanner - Used for static code linting ...
> 
> This needs to be in the default installation of dmd! It makes no sense that these are not included.

From a usability standpoint, I agree.

From a political standpoint, it's unsurprising that they're not included.

> Future:
> --------
> 
> You want D to have traction.

That's *a* goal, but it's not the goal that's closest to most of our hearts, I think. Popularity doesn't exactly bring enjoyment -- it can remove some annoyances, but it's easier to work around those annoyances. It isn't fulfilling. And it doesn't pay the bills.

> Marketing, more library support, less focus
> on adding even more advanced features, fixing issues (
> like better GC ),

There are huge barriers to bringing D's GC to the state of the art. Some improvements could be made, though. For instance, the GC currently scans heap objects scattered about the whole heap. Using separate virtual address ranges for objects with pointers and objects without might improve efficiency somewhat.

> CTFE ( Stefan is dealing with that ), Documentation,
> better Editor support...

I think code-d could potentially be extended to install its dependencies, which would improve the situation there.

> Walter / Andrei:
> ----------------
> 
> No offense guys, just something that i see in a lot of posts. The hinting at people to add more to the standard libraries. That little push. But frankly, its annoying when nothing gets done.
> 
> People complain about x feature. You tell people to add to the standard library or bring the project in. But anybody who has ever read this forum sees how adding things to the language is LONG process and a lot of times idea's get shot down very fast.

The language is *very* conservative. The standard library is merely cautious. But since there's no cost to adding a package to dub, that doesn't prevent code from being written and reused. Except by those who refuse to go outside the standard library, and for those people, I would recommend Java or C#.

> For the standard library there is no process as far as i can tell. Experimental at best, where code seems to have a nice long death.

It could be much better documented.

The process for small changes:

* Make the change.
* Make a pull request.

The process for large changes:

* Propose it here, with your initial design. Get feedback.
* Implement it as a dub package.
* Ask for a formal review here.
* Address feedback.
* Make a pull request.

> Give people the ability to add more to std.experimental. Give it its own forum part.

Nothing stops you from adding dub packages using the namespace std.experimental. The tradition previously has been to use stdx for modules that are intended for the standard library.

An incubator organization might be appropriate as an intermediate between standard library and individuals' dub packages -- easier entry than the standard library, but multiple people are on hand to handle bugs.
December 21, 2016
On 12/21/2016 3:36 AM, thedeemon wrote:
> Bad news: without complete redesign of the language and turning into one more
> C++/CLI (where you have different kinds of pointers in the language for GC and
> non-GC), having C performance and Go-style low-pause GC is not really possible.
> You have to choose one. Go chose GC with short pauses but paid with slow speed
> overall and slow C interop. D chose C-level performance but paid for it with a
> slow GC.

The trouble with a better GC is it usually entails changing the code generator to emit a "write gate" that goes along with assignments via a pointer. This write gate signals the GC that a particular block is being written to, so that block can be marked as "dirty". (Paging virtual memory systems do this automatically.)

What this implies is better GC performance comes at a cost of worse performance of the non-GC code. This strategy is effective for a language that makes very heavy use of the GC (like Java does), but for a language like D that uses the GC lightly, it's a much more elusive benefit.
December 21, 2016
On 12/21/2016 6:50 AM, thedeemon wrote:
> Have you seen this one?
> http://www.infognition.com/blog/2014/the_real_problem_with_gc_in_d.html

Although I had called them write gates, write barriers are the same thing. Yes, that's the problem with implementing a generational collector in D.

I once tried to implement write barriers by using the hardware VM system. I'd mark the old generation pages as read-only. When the program would write to those pages, a seg fault happened. This would then run a handler in the GC code which would mark that page as "dirty", then write-enable the page, and restart the program at the point where it seg faulted.

This worked great. The only trouble was that the seg faulting path at runtime was so slow it ruined the speed advantage of not have write barriers. So I had to abandon it.

But that was long ago. Maybe the tradeoff is better these days with modern hardware. But I suspect that if other GC developers are not using this technique, it is still too slow.
December 21, 2016
On 12/21/2016 7:57 PM, Chris Wright wrote:
> You can implement write barriers as runtime calls, but omit them in @nogc
> code.

@nogc code is code that doesn't allocate from the gc. It can still write to gc allocated objects, however, so that idea won't work.


> However, this would be costly -- it's an expensive technique in
> general; the current GC mallocs each object instead of mmaping a range of
> memory; and in D you can't move heap objects safely,

You can if using a "mostly copying" generational collector, which is what I did long ago. It works.

> so you can't
> distinguish generations based on pointers (you'd have to mark GC data
> structures, and it's O(log n) to find the right one).
>
> You can implement write barriers with mprotect. However, this won't give
> you good granularity. You just know that someone wrote something to an 8
> kilobyte block of memory that has a pointer in it somewhere. This
> requires the GC to use mmap instead of malloc, and it is strongly
> encouraged not to put pointer-free objects in the same page as objects
> with pointers.

Using mprotect works, and I wrote a generational collector using it long ago, but it is even slower.

December 25, 2016
On Thursday, 22 December 2016 at 04:47:06 UTC, Chris Wright wrote:
>> CTFE ( Stefan is dealing with that ), Documentation,
>> better Editor support...
>
> I think code-d could potentially be extended to install its dependencies, which would improve the situation there.

It does already do that though :/
December 26, 2016
On 12/24/16 5:11 PM, WebFreak001 wrote:
> On Thursday, 22 December 2016 at 04:47:06 UTC, Chris Wright wrote:
>>> CTFE ( Stefan is dealing with that ), Documentation,
>>> better Editor support...
>>
>> I think code-d could potentially be extended to install its
>> dependencies, which would improve the situation there.
>
> It does already do that though :/

Will it auto-update them as new versions are released, or when code-d is updated?
December 26, 2016
On Monday, 26 December 2016 at 19:37:34 UTC, David Gileadi wrote:
> On 12/24/16 5:11 PM, WebFreak001 wrote:
>> On Thursday, 22 December 2016 at 04:47:06 UTC, Chris Wright wrote:
>>>> CTFE ( Stefan is dealing with that ), Documentation,
>>>> better Editor support...
>>>
>>> I think code-d could potentially be extended to install its
>>> dependencies, which would improve the situation there.
>>
>> It does already do that though :/
>
> Will it auto-update them as new versions are released, or when code-d is updated?

it will only update workspace-d and only when the plugin updates and requires a new version
December 27, 2016
On Monday, 19 December 2016 at 23:02:59 UTC, Benjiro wrote:
> D has not market:
> -----------------
>
> A lot of times people complain that D has no real audience / market. Is D the perfect system. No... But lets compare to those other languages shall we? Now this is my opinion, so take it with a bit of salt.

People who are complaining that D has no market aren't using the language on the whole - or are using it for private projects and would like to use it for work but don't know of any professional opportunities where they can do so.  The latter is a high quality problem to have for an emerging language because it means people find some reason to want to use the language, and those who care about using good tools tend to be better programmers than those who don't.

Still, it's not true that D has no real market, because the userbase is demonstrably growing.  A complex creature matures more slowly than a simple one - what's significant is not that D does not have an official corporate sponsor of the kind that Go has (Sociomantic plays a different role, as I understand it), but that it is flourishing in spite of not having one and in spite of the absence of any kind of marketing orientation.  That says quite a lot about its intrinsic merits, and we're in an era where the pendulum is shifting from shiny things sponsored by large corporations to giving a greater chance to more grass-roots organic developments.

When you have a small market share, it's easy to win.  You don't need to appeal to "a lot of C++ people" - just a few of them, a few frustrated Python guys, and so on.  D fits quite well the Innovator's Dilemma described by Clayton Christensen.  It's a positive not a negative that many prefer to dismiss it since it makes it easier for it to gain a hold in certain niches, draw energy from such and then spread into adjacent niches.

> I see a lot of people arguing a lot about D and sorry to say but at times it sounds like a kindergarten.

People who care about excellence will tend to argue a lot - see Andy Grove's 'constructive confrontation', Charles Koch's 'market based management', and Ray Dalio's 'Principles'.  The key thing is what happens in response to that.  The language and its documentation are better now than a year ago, and a year ago were better then the previous year.  It's not perfect, but life never is.

> Maybe its because people who use D are more into proprietary software, that
> there is less community response with work going into the library.

If that's true (and I think it might be), that's a very positive thing, because most software is proprietary software, and adoption by companies when successful will tend to provide a future source of support and energy for the ecosystem.  You can see this already with Sociomantic's sponsorship, but they aren't the only ones.  I've paid for some small improvements in dub's shared library support myself (-fPIC did not propagate which made using dub to build shared libraries on linux a bit tricky, but it does now, or will do once PRs are accepted), and you get to excellence from many little improvements of this sort.

So if you want to improve the language and its ecosystem, the best way is to contribute pull requests or $$$s - the Foundation now accepts individual donations, and it's also open to corporate sponsorship, I believe.  One should be a bit patient in expecting changes from the creation of the Foundation - it's an awful lot of work to set something up, and once that's done to figure out how to deploy resources to efficiently make a difference.  It's quite impressive what has been done already - very smart approach to start with institutions one has a personal/national connection with and where great people are much more available than in some other countries.

And on the other hand, pull requests don't need to take much energy.  I don't have much time, but I noticed the curl examples were out of date - string vs char[] if I recall right, and they were annoying me, so I fixed them and they were pulled pretty much immediately.

Most D users don't spend much time on the forums or IRC because they haven't time for it since they are actually using D to get stuff done.  Look at the list of corporate users, and look at the small share of the people working on D in these companies in forum discussions.  And that's only a subset of commercial D users.

Languages reach explosive takeoff not when they make a big change in strategy but when external conditions change in such a way that the problem the language solves suddenly becomes much more important.  Just as Japanese cars didn't used to be very good - and who would want a rinky dink thing like they used to be.  But then energy prices exploded in the 70s, and peoples' priorities changed - and the poor US auto-makers, complacent and bloated in their prior success didn't know what had hit them.  I've written a bit elsewhere about such changing conditions that might be relevant today.  Close to 250k people read it and so far I didn't get a good argument to the contrary.  Though it's hard to make predictions, especially about the future...

> If you are a low level programmer, sure, you can write you way around it. Despite my PHP handicap i am writing a Sqlite wrapper for my work. I do not use 3th party packages because a) i do not know the code b) the fear that the work will be abandoned. I can excuse (a), when i know its the standard library because there will always be people willing to work on  the standard library.

It's not exactly much work to read the code for a basic sqlite wrapper and to become quite familiar with it.  (And it wouldn't after that be much work to improve the docs just a little bit).  Isn't everyone reinventing the wheel somewhat part of the problem?  And if you're familiar with it, the work being abandoned isn't really such a risk.  If you force an enormous step change in the size of the standard library, it doesn't necessarily automatically bring in more resources to work on parts of it people didn't care about before. So I agree with Andrei's vision of favouring a larger standard library, but it's probably also the right approach to have an organic approach as we do.

> I do not use it. Its such a mess to read with long paragraphs and a LOT of stuff not even documented. Like the whole C wrappers etc. My personal bible is simple: Google search for examples and Ali's book for some rare cases.

Ali's book is pretty good, and I have found it easy enough to use the C bindings (not sure what you mean by wrappers) - just pull up the C API in one window, and the source code for the binding in another.  Yes - a one-off upfront price in the beginning to become more familiar with it, but it pays off over time, at least in many uses.

> Editor support:

Sublime text and sometimes vim work well enough for me, though these things are very personal.  For the others, have you contributed anything - time or money in making them better?  If wishes were horses, beggars would ride.  And contribution of either has a higher value than just the thing itself, because it also tends to energise the project - look at the frustration Basil experienced regarding his IDE project.  It's good to have high standards, but one should have some appreciation also for the gift that people make of their time, work, and energy in ways that don't always lead to the gratitude that one might expect.

You wouldn't believe how little money or support, intelligently applied, it can take to indirectly lead to quite big changes...

> Try remote editing D and see how much fun it is.
vim has always worked well enough for me.  or sublime on occasion.

> Seb posted a massive list of modules that can be standard candidates. And the response is more or less ignore it. People who work on Standard libraries are more motivated. Bring  them into the fold. But it seems that they simple get ignored.

Rome wasn't built in a year.  Great things are achieved by taking baby steps, compounded over time.  And if one does what little one can, others are inspired by it.  Enthusiasm and a constructive attitude are infectious in my experience.

> ;) ). Sorry guys but it feels like you are too technical focused and not thinking about the bigger picture. Suggesting things does not work when nobody gives people the chance to prove themselves.
Yes - but languages are founded on different principles.  D is an organic ecosystem oriented to people who care about technical things.  Technical excellence brings in people with resources who find technical questions more important than marketing.  It takes time sometimes.



Laeeth.

December 28, 2016
On Tuesday, 27 December 2016 at 16:36:10 UTC, Laeeth Isharc wrote:
> On Monday, 19 December 2016 at 23:02:59 UTC, Benjiro wrote:
> So if you want to improve the language and its ecosystem, the best way is to contribute pull requests or $$$s - the Foundation now accepts individual donations, and it's also open to corporate sponsorship, I believe.
>
>> Editor support:
>
> Sublime text and sometimes vim work well enough for me, though these things are very personal.  For the others, have you contributed anything - time or money in making them better?  If wishes were horses, beggars would ride.  And contribution of either has a higher value than just the thing itself, because it also tends to energise the project - look at the frustration Basil experienced regarding his IDE project.  It's good to have high standards, but one should have some appreciation also for the gift that people make of their time, work, and energy in ways that don't always lead to the gratitude that one might expect.

There's only so much time and money someone can give. It isn't that appealing when virtually no other language out there suffers from this problem cause they have an actual market behind them. Those markets fuel money being poured into the tools of the lanugage. It doesn't really matter how many users you have, it depends on the demographic of those users. If they are all students still in school, then you haven't really created a market.

Anyways most of the IDEs out there are made by a small team or only one person. Not only that but they almost all (if not all) rely on the same projects to get the features you would expect in an IDE. The DCD, DScanner, DFix, DFmt etc... All those tools also seem to be developed primarily by the same single person. Rust seems to be in a similar situation but at least it seems the rust team has plans for adding IDE support into the compiler itself. Something that is probably unrealistic for D.

>> Seb posted a massive list of modules that can be standard candidates. And the response is more or less ignore it. People who work on Standard libraries are more motivated. Bring  them into the fold. But it seems that they simple get ignored.
>
> Rome wasn't built in a year.  Great things are achieved by taking baby steps, compounded over time.  And if one does what little one can, others are inspired by it.  Enthusiasm and a constructive attitude are infectious in my experience.

D isn't a year old though. If the steps you take are too small, you can also end up being left behind.