On Wednesday, 27 January 2016 at 09:15:27 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 27 January 2016 at 06:17:44 UTC, Laeeth Isharc wrote:
>> On Tuesday, 26 January 2016 at 22:48:23 UTC, Ola Fosheim Grøstad wrote:
>
>>> I am not sure if that is the right motivation. Sounds like recipe for bloat. Good libraries evolve from being used in real applications. Many applications.
>>
>> sayeth a low-level guy (if I understand correctly), which will certainly create a distinct perspective about what you would like to see in the standard library, and yet this may not be the right thing for the language as a whole.
>
> I am both low-level and high level, but D's primary advantage is that it allows low level programming.

Surely, that is C's primary advantage!  The whole point of D is that it doesn't just have one primary advantage, and that is why the front page no longer describes it is as a systems language, even though it's approaching suitable as such.

>> fwiw, people that do use D on a serious scale have remarked that the richness of the standard library (even as it stands today) was a major advantage - in bioinformatics, at a London hedge fund, and I think AdRoll.
>
> Do you consider Angular to be low level? It was used in 100s if not 1000s of applications, but was considered inadequate and scrapped in favour of Angular2. This is the typical pattern for libraries and frameworks.

I really don't see how this relates to the point at hand.  You seemed to suggest that the standard library should be small, and I pointed out that many serious users in industries that are likely to be key markets for D do say that they find what's provided in the standard library to be quite appealing.  Nothing lasts forever, and all is change - that's something I agree with.  But it doesn't seem to have much bearing on the decision about what to put in the standard library.  Your suggestions that because some cloud environments don't have a conventional file system, D perhaps shouldn't even have this in the standard library really seemed to me to be a reductio ad absurdum of your own argument.  Of course there will be lots there that one doesn't need and can't use.  But over time things that were once cutting edge become bog standard, and it makes sense to have coherence and convenience rather than to have to search out the best library for the purpose each time.

On Wednesday, 27 January 2016 at 16:52:53 UTC, Laeeth Isharc wrote:
> your own argument.  Of course there will be lots there that one doesn't need and can't use.  But over time things that were once cutting edge become bog standard, and it makes sense to have coherence and convenience rather than to have to search out the best library for the purpose each time.


I agree about coherence being desirable, but evolution tends to lead to the opposite. Just look at C++ streams vs iterators vs ranges vs ... Coherence is a good reason to have independent libraries that are replaced as a whole.

The standard library should primarily cover stable types you need to describe APIs. Then you can have other recommended independent libraries with no mutual dependencies.

On Tuesday, 26 January 2016 at 12:38:09 UTC, Andrei Alexandrescu wrote:
> including things that some people argue shouldn't be part of a standard library: archives and compression, cryptography, databases, character encodings (including json and xml!), html templating, image processing, suffix arrays (sic), logging, networking, regexp, testing, tokenization.

See my answer below, most of these are standard solutions that you need on a daily basis (except for the suffix array).

> I do agree with Dub's importance. What's unclear to me is what are reasonable criteria for including some given functionality within the stdlib vs. an external one.

A good criteria is whether some area has an established and hard to debate solution, then it can go into the standard library. But if there are many different ways around the same topic you should leave the decision to the users.

So for example there are 3 established ways to parse something, dom, stax and event based parsers. So you can add those parsers as sth. like std.xml or std.json.

There are about 500 different configuration file formats, so anything that isn't as established as xml or json should be left for libraries.

Likewise there are plenty of different GUI toolkits (taking imperative or declarative approaches). Leave it to people to pick one that suites their need.

You could discuss endlessly about the syntax of html templating, but how such a library should work is clear. This is at the edge of standardizable, b/c by putting it into std, you're making a decision for all language users. It's albeit possible, just like declaring a particular code style as standard.

Anything that is highly innovative or experimental should never go into standard libraries.

On Monday, 25 January 2016 at 16:26:36 UTC, Russel Winder wrote:
> PyPI has is an highly opinionated metric that helps you decide what is good and what is dross.

Could you help us on designing one for code.dlang.org as well?

On Wed, 27 Jan 2016 21:01:47 +0000, Martin Nowak wrote:

> On Monday, 25 January 2016 at 16:26:36 UTC, Russel Winder wrote:
>> PyPI has is an highly opinionated metric that helps you decide what is good and what is dross.
> 
> Could you help us on designing one for code.dlang.org as well?

I'd want to use the number of releases, how recent the most recent release was, the longevity of the project, and the number of other projects that depend on it. If people are willing to let dub submit anonymous usage information, the number of builds depending on a project is also an indicator of its value.

Longevity is an awkward one. If I create a project, do nothing with it for four years, then make a release, will that count as well as making a release once a month for four years? But the other metrics should account for that.

If possible, I'd include when the project last compiled and whether it passes its unittests. This requires setting up a continuous integration service for untrusted code. I've seen other people's designs for that, and I think I could replicate it without too much trouble.

On Wednesday, 27 January 2016 at 22:36:37 UTC, Chris Wright wrote:
>
> Longevity is an awkward one. If I create a project, do nothing with it for four years, then make a release, will that count as well as making a release once a month for four years? But the other metrics should account for that.
>

Code I wrote below incorporates some of what you're talking about.

For simplicity, I represented the date of each release as a dynamic array of doubles. I applied a weighting function to each and then calculated the entropy. The score is 1 if you only have one release. The score is higher, the more releases you have. If you have one release five years ago and one release recently, then your score is only marginally higher than 1. But if you have lots of recent releases, then the score is much more favorable.

You could probably adjust this in a number of ways. For instance, adjusting the denominator in
return weight.map!(a => a / weight.sum());
you could make it so that a release today is better than a release five years ago.


auto weight(double[] x, double lambda)
{
	import std.algorithm : sum, map;
	import std.math : exp;
	import std.array : array;
	
	auto weight = x.map!(a => lambda * exp(-lambda * a));	// weights given by
								// exponential
								// distribution
	return weight.map!(a => a / weight.sum());
}

double entropy(T)(T x)
{
	import std.algorithm : sum, map;
	import std.math : exp, log;
	
	auto e = x.map!(a => a * log(a));
	return exp(-sum(e));
}

double calc_score(double[] x, double lambda)
{
	return x.weight(lambda).entropy();
}

void main()
{
	import std.stdio : writeln;
	
	double lambda = 0.5; // controls decay in weighting function
	double[] x;	     // represents years since last update
	x = [0.1, 0.25, 1, 2, 5];
	
	auto score = calc_score(x, lambda);

	writeln(score);
}

Parkinson's Law: work expands so as to fill the time available for its completion.

Having a set of vague tasks to finish within 6 months is great, but having a weekly more specific priority list to go along with it would be better. If, in addition, there was some accountability (not with negative implications, just a designation of responsibility) that would further narrow focus and ramp up productivity.

As a draft proposal, something like a weekly sticky TODO thread with clear goals could work. You'd list a few high priority items and ask for a volunteers for each and list them as they come in. You'd have bigger medium priority item list and ask for volunteers but only half of them need to be assigned to someone and completed. Then you'd have a much longer lower priority list that can be a bit more vague that people are encouraged to work on but that they don't need to volunteer for, just submit their pull requests (but they can flag an item and have it ticked off if they want). On top of that have a weekly pull request goal that updates daily (with a time stamp) and an overview of pull request numbers from previous weeks.

If an item isn't completed it gets rolled over to the next week with an asterisk (if it's medium or high priority) for each time it's been rolled over and the previous volunteer listed as such (and they can volunteer for the same item the next week giving them the perfect opportunity to say where they're at, what needs to be done, if the task should be broken down further, etc). It'll make it clear if whoever volunteered for a task needs some help or encouragement, or if certain high priority tasks need a volunteer or to be broken down into more manageable chunks even if someone drops off the face of the earth. With a one item at a time rule you're going to cut down on stagnation.

I'm sure there are lots of people who'd like to help out but they're not really sure what needs to be done or who else is doing it or if they'll step on toes, etc. Anyone wondering what's being worked on can take a look and help out themselves rather than getting upset in the forums about feature x never seeing the light of day. If they want feature x done they can take a look at the priority list and pick where it should go in the scheme of things, maybe even break it down - turn negative energy into productive energy :)

On Thursday, 28 January 2016 at 00:28:26 UTC, Twenty Two wrote:
> Parkinson's Law: work expands so as to fill the time available for its completion.
>
> [...]

I agree.  Some of the core team uses trello for this:

https://trello.com/b/XoFjxiqG/active

However, that's not really meant for noobs and a lot of the core team, including Walter, doesn't really use it.

On Thursday, 28 January 2016 at 02:36:55 UTC, Joakim wrote:
> On Thursday, 28 January 2016 at 00:28:26 UTC, Twenty Two wrote:
>> Parkinson's Law: work expands so as to fill the time available for its completion.
>>
>> [...]
>
> I agree.  Some of the core team uses trello for this:
>
> https://trello.com/b/XoFjxiqG/active
>
> However, that's not really meant for noobs and a lot of the core team, including Walter, doesn't really use it.

Judging by the activity log, it's mostly just Martin... poor dawg.

Putting something together people will actually use is pretty hard, almost as hard as keeping the momentum to get people to keep using whatever you've put in place. Trello's not a bad tool but it's useless if there's no traction and no one's being reminded to use it. It's hard to attract contributors if you don't spell out who/what/when/where/how and funnel them all to the one place.

Maybe the core team could discuss among themselves what kind of workflow they think would really bring out their potential productivity and attract more hands to lighten the load. It seems unfair that some people get lumped with doing big jobs purely because there isn't really a good way to ask someone else to do pieces of it for them.

I'm certain there are lots more people who would get involved if only there was a lower barrier to entry. Something not over complicated and visible would really move things along, I think.

On Wednesday, 27 January 2016 at 21:00:41 UTC, Martin Nowak wrote:
> A good criteria is whether some area has an established and hard to debate solution, then it can go into the standard library. But if there are many different ways around the same topic you should leave the decision to the users.

I don't think this is the right criteria. Take a good hard look at the C++ standard library.
Plenty of things in there that nobody use, but which one would hope that almost everyone would use. Like for numerics.

When a good external numerics library emerge it will displace whatever numerics functionality the standard library provides.

D has too few users to see high quality external libraries, but that is not a good reason to bloat the standard library.

The argument that some functionality is needed on a daily basis is also misguided. Different applications have different needs.

What the standard library should support is essential interfacing with the system and what you need to interface with libraries, like basic types/iterators.

Once you have something in the standard library you have to maintain it forever. If something is in the standard library and it turns out to be impossible to implement it efficiently on a specific hardware platform then all libraries using that functionality will be dog slow on that platform.

> So for example there are 3 established ways to parse something, dom, stax and event based parsers. So you can add those parsers as sth. like std.xml or std.json.

There is no reason for xml or json to be in the standard library. You can have a set of recommended libraries, allowing for more efficient APIs to emerge over time.

Only functionality where you get unbreakable interdependencies between standard modules need to be in standard library.

Parsers and compressors have low levels of inter-library dependencies. DSP functionality has only internal dependencies. No need for such things in the standard library.


Scripting languages like Python is a horrible example to follow. Scripting languages require all C-implemented functionality to be in the standard library for the sake of being able to deploy pure python code on machines where you don't get to install binaries.