On Mon, 05 May 2014 11:26:29 +0000
bearophile via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> Jonathan M Davis:
> > Such code should be kept separate IMHO.
>
> This means that you now have two modules, so to download them atomically you need some kind of packaging, like a zip. If your project is composed by many modules this is not a problem. But if you have a single module project (and this happens often in Python), going from 1 to 2 files is not nice. I have written tens of reusable D modules, and some of them have a demo or are usable stand-alone when you have simpler needs.

Honestly, I wouldn't even consider distributing something that was only a single module in size unless it were on the scale of std.datetime, which we've generally agreed is too large for a single module. So, a single module wouldn't have enough functionality to be worth distributing. And even if I were to distribute such a module, I'd let its documentation speak for itself and otherwise just expect the programmer to read the code.

Regardless, the version specifier makes it easy to have a version where main is defined for demos or whatever else you might want to do with it. So, I'd suggest just using that. I highly doubt that you'd be able to talk either Walter or Andrei into supporting a separate feature for this. At this point, we're trying to use what we already have to implement new things rather than adding new features to the language, no matter how minor it might seem. New language features are likely be restricted to things where we really need them to be language features. And this doesn't fit that bill.

- Jonathan M Davis

Jonathan M Davis:

> Honestly, I wouldn't even consider distributing something that was only a
> single module in size unless it were on the scale of std.datetime, which we've
> generally agreed is too large for a single module.
> So, a single module
> wouldn't have enough functionality to be worth distributing.

This reasoning style is similar to the Groucho Marx quote:
"I don't care to belong to any club that will have me as a member"

In the Python world online you can find thousands of single-module projects (few of them are mine). I have plenty of single D modules that encapsulate a single functionality. In Haskell cabal you can find many single modules that add functionality (plus larger projects like Diagrams).

And I think D has to strongly encourage the creation of such ecosystem of modules that you download and use in your programs. You can't have everything in the standard library, it's not wise to re-write them (like 2D vectors, I have already seen them implemented ten different times in the D world), and there are plenty of useful things that can be contained in single modules, especially if such modules can import the functionality of one or more other modules.


> And even if I were to distribute such a module, I'd let its
> documentation speak for itself
> and otherwise just expect the programmer to read the code.

A demo and the documentation are both useful. And the documentation can't replace stand-alone functionality.


> Regardless, the version specifier makes it easy to have a version where main is defined for demos or whatever else
> you might want to do with it.

Bye,
bearophile

However, the community is starting to standardize around Dub as the standard package manager. Dub makes downloading a package as easy as editing a JSON file (and it scales such that you can download a project of any size this way). Did Python have a proper package manager before this idiom arose?

On Thursday, 1 May 2014 at 16:08:23 UTC, Andrei Alexandrescu wrote:
>> It got full because of tests (surprise!). Your actions?
>
> Fix the machine and reduce the output created by the unittests. It's a simple engineering problem. -- Andrei

You can't. You have not control over that machine you don't even exactly know that test has failed because of full /tmp/ - all you got is a bug report that can't be reproduced on your machine.

It is not that simple already and it can get damn complicated once you get to something like network I/O

On Thursday, 1 May 2014 at 19:22:36 UTC, Andrei Alexandrescu wrote:
> On 5/1/14, 11:49 AM, Jacob Carlborg wrote:
>> On 2014-05-01 17:15, Andrei Alexandrescu wrote:
>>
>>> That's all nice, but I feel we're going gung ho with overengineering
>>> already. If we give unittests names and then offer people a button
>>> "parallelize unittests" to push (don't even specify the number of
>>> threads! let the system figure it out depending on cores), that's a good
>>> step to a better world.
>>
>> Sure. But on the other hand, why should D not have a great unit testing
>> framework built-in.
>
> It should. My focus is to get (a) unittest names and (b) parallel testing into the language ASAP.
>
> Andrei

It is wrong approach. Proper one is to be able to define any sort of test running system in library code while still being 100% compatible with naive `dmd -unittest`. We are almost quite there, only step missing is transferring attributes to runtime unittest block reflection.

On 5/5/14, 8:11 AM, Dicebot wrote:
> On Thursday, 1 May 2014 at 16:08:23 UTC, Andrei Alexandrescu wrote:
>>> It got full because of tests (surprise!). Your actions?
>>
>> Fix the machine and reduce the output created by the unittests. It's a
>> simple engineering problem. -- Andrei
>
> You can't. You have not control over that machine you don't even exactly
> know that test has failed because of full /tmp/ - all you got is a bug
> report that can't be reproduced on your machine.
>
> It is not that simple already and it can get damn complicated once you
> get to something like network I/O

I know, incidentally the hhvm team has had the same problem two weeks ago. They fixed it (wthout removing file I/O from unittests). It's fixable. That's it.

This segment started with your claim that unittests should do no file I/O because they may fail with a full /tmp/. I disagree with that, and with framing the full /tmp/ problem as a problem with the unittests doing file I/O.


Andrei

On 5/5/14, 8:16 AM, Dicebot wrote:
> On Thursday, 1 May 2014 at 19:22:36 UTC, Andrei Alexandrescu wrote:
>> On 5/1/14, 11:49 AM, Jacob Carlborg wrote:
>>> On 2014-05-01 17:15, Andrei Alexandrescu wrote:
>>>
>>>> That's all nice, but I feel we're going gung ho with overengineering
>>>> already. If we give unittests names and then offer people a button
>>>> "parallelize unittests" to push (don't even specify the number of
>>>> threads! let the system figure it out depending on cores), that's a
>>>> good
>>>> step to a better world.
>>>
>>> Sure. But on the other hand, why should D not have a great unit testing
>>> framework built-in.
>>
>> It should. My focus is to get (a) unittest names and (b) parallel
>> testing into the language ASAP.
>>
>> Andrei
>
> It is wrong approach. Proper one is to be able to define any sort of
> test running system in library code while still being 100% compatible
> with naive `dmd -unittest`. We are almost quite there, only step missing
> is transferring attributes to runtime unittest block reflection.

Penalizing unittests that were bad in the first place is pretty attractive, but propagating attributes properly is even better. -- Andrei

Meta:

> However, the community is starting to standardize around Dub as the standard package manager. Dub makes downloading a package as easy as editing a JSON file (and it scales such that you can download a project of any size this way).

Having package manager(s) in Python doesn't make single module Python projects less popular or less appreciated. Most Python projects are very small, thanks to both standard library and code succinctness (that allows a small program to do a lot), and the presence of an healthy ecology of third party modules that you can import to avoid re-writing things already done by other people. All this should become more common in the D world :-)


> Did Python have a proper package manager before this idiom arose?

Both are very old, and I am not sure, but I think the main module idiom is precedent.

Bye,
bearophile

On Monday, 5 May 2014 at 15:36:19 UTC, Andrei Alexandrescu wrote:
> On 5/5/14, 8:11 AM, Dicebot wrote:
>> On Thursday, 1 May 2014 at 16:08:23 UTC, Andrei Alexandrescu wrote:
>>>> It got full because of tests (surprise!). Your actions?
>>>
>>> Fix the machine and reduce the output created by the unittests. It's a
>>> simple engineering problem. -- Andrei
>>
>> You can't. You have not control over that machine you don't even exactly
>> know that test has failed because of full /tmp/ - all you got is a bug
>> report that can't be reproduced on your machine.
>>
>> It is not that simple already and it can get damn complicated once you
>> get to something like network I/O
>
> I know, incidentally the hhvm team has had the same problem two weeks ago. They fixed it (wthout removing file I/O from unittests). It's fixable. That's it.

It is possible to write a unit test which provides graceful failure reporting for such issues but once you get there it becomes hard to see actual tests behind boilerplate of environmental verification and actual application code behind tests.

Any tests that rely on I/O need some sort of commonly repeated initialize-verify-test-finalize pattern, one that is simply impractical to do with unit tests.

> This segment started with your claim that unittests should do no file I/O because they may fail with a full /tmp/. I disagree with that, and with framing the full /tmp/ problem as a problem with the unittests doing file I/O.

It was just a most simple example. "Unittests should do no I/O because any sort of I/O can fail because of reasons you don't control from the test suite" is an appropriate generalization of my statement.

Full /tmp is not a problem, there is nothing broken about system with full /tmp. Problem is test reporting that is unable to connect failure with /tmp being full unless you do environment verification.

On 5/5/14, 8:55 AM, Dicebot wrote:
>
> It was just a most simple example. "Unittests should do no I/O because
> any sort of I/O can fail because of reasons you don't control from the
> test suite" is an appropriate generalization of my statement.
>
> Full /tmp is not a problem, there is nothing broken about system with
> full /tmp. Problem is test reporting that is unable to connect failure
> with /tmp being full unless you do environment verification.

Different strokes for different folks. -- Andrei