On 04/14/2011 03:50 PM, David Nadlinger wrote:
> On 4/14/11 3:44 PM, spir wrote:
>> On 04/14/2011 02:40 PM, David Nadlinger wrote:
>>> On 4/14/11 2:26 PM, spir wrote:
>>>>> Not having any imports makes for a faster compile, too.
>>>>
>>>> ... and helps in having safe sandboxes.
>>>
>>> In which way exactly do I need an import to write »extern(C) int
>>> system(in char
>>> *); system(…);«?
>>
>> Did I write "it provides safe sandboxes"?
>>
>> Denis
>
> No, but you wrote that it »helps in having safe sandboxes«, and I'm curious how
> you think it would.

I mean imports usually bring in many more tools for naughty code. And I guess in some languages, naughty actions can only be performed via such imported modules (system control, direct memory access,...), thus forbidding import is an easy way of creating a sandbox. At the very minimum, forbidding import or limiting it to a predefined set of modules is a necessary first step, I guess.

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On 04/14/2011 04:03 PM, Steven Schveighoffer wrote:
> Sometimes, I worry that my unit tests or asserts aren't running.  Every once in
> a while, I have to change one to fail to make sure that code is compiling (this
> is especially true when I'm doing version statements or templates).  It would
> be nice if there was a -assertprint mode which showed asserts actually running
> (only for the module compiled with that switch, of course).

Man, I'm very pleased to read someone else advocating for optionally verbose assertions.
This could use 2 arguments instead of a predicate:
    assert(expressions, value);
Example use:

unittest {
    auto args = [0, 1, -1, 999, -999];
    auto vals = [a, b, c, d, e]; // letters here represent expected values
    foreach (i ; 0..args.length)
        assert(f(args[i]), vals[i]);
}

During a test session, assert-mode is set to 'verbose'. This would print for instance, in case f(999) fails:
"
assert: f(0) --> a
assert: f(1) --> b
assert: f(-1) --> c
********************************
Assertion Error:
    expression  : f(999)
    expected    : d
    outcome     : whatever
********************************
assert: f(-999) --> e
"
Then, we get all we need to comfortably debug, I guess. Also note /other/ unittests (eg one testing g called by f) may also bring in valuable feedback data on how the program actually runs.

As is discussed in this thread, this verbose mode is also very nice for code samples (introduction to D, manuals, tutorials, howtos, and even exchanges on mailing lists...).

For a plain regression test, after some change has been made to a piece of code, assert-mode is set to 'silent'. This would only print out failure cases:
"
********************************
Assertion Error:
    expression  : f(999)
    expected    : d
    outcome     : whatever
********************************
"
Then, we just have to turn assert-mode back to verbose, and happily start debugging ;-)

Note: 2 friendly complementary features are (1) expected outcome may be an exception, (2) outcome may be expression via a string representation. This is the reason why output shows as '-->', not '=='.

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

Am 14.04.2011 16:56, schrieb spir:
> On 04/14/2011 03:50 PM, David Nadlinger wrote:
>> On 4/14/11 3:44 PM, spir wrote:
>>> On 04/14/2011 02:40 PM, David Nadlinger wrote:
>>>> On 4/14/11 2:26 PM, spir wrote:
>>>>>> Not having any imports makes for a faster compile, too.
>>>>>
>>>>> ... and helps in having safe sandboxes.
>>>>
>>>> In which way exactly do I need an import to write »extern(C) int
>>>> system(in char
>>>> *); system(…);«?
>>>
>>> Did I write "it provides safe sandboxes"?
>>>
>>> Denis
>>
>> No, but you wrote that it »helps in having safe sandboxes«, and I'm
>> curious how
>> you think it would.
> 
> I mean imports usually bring in many more tools for naughty code. And I guess in some languages, naughty actions can only be performed via such imported modules (system control, direct memory access,...), thus forbidding import is an easy way of creating a sandbox. At the very minimum, forbidding import or limiting it to a predefined set of modules is a necessary first step, I guess.
> 
> Denis

As long as any C function can be called just by defining it, this is just snake oil. For other languages this may work, but it'd make more sense to just use a modified standard-lib that asserts when forbidden functions are used.

For D (and probably any other system programming language) it makes more sense to restrict on OS level, for example with SELinux, and assume that the user code is pure evil.

Cheers,
- Daniel

On 04/14/2011 04:22 PM, Andrei Alexandrescu wrote:
> On 4/14/11 6:52 AM, spir wrote:
>> On 04/14/2011 01:11 AM, bearophile wrote:
>>>> * Since most of them don't actually output anything, the program
>>>> > now detects output.length == 0 and says "Program run
>>>> > successfully." upon completion to give some feedback.
>>> This is a problem. Unittest code is not meant to produce an output,
>>> while online snippets to try are meant to nearly always produce a
>>> meaningful output.
>>
>> All my unittest blocks produce meaningful output; there exist firstly
>> for that. Am I an heretic?
>
> No, you're just doing it wrong. Heretic might be sometimes good.
>
> http://www.linfo.org/rule_of_silence.html

Right, I know that and totally disagree. This rule is IMO the exact opposite of human-friendliness (numerous authors on usability and friends share this opinion, indeed).
Also, this is not what I'm talking about: I'm not here expecting just "all fine, brother", but various data helping me and understand how things actually worked; to be short, data useful to the coder during development or debug. See also reply to Steven's post.

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On 04/14/2011 04:24 PM, Andrei Alexandrescu wrote:
> On 4/14/11 7:25 AM, spir wrote:
>> On 04/14/2011 03:04 AM, Adam D. Ruppe wrote:
>>> bearophile wrote:
>>>> > This is a problem. Unittest code is not meant to produce an output,
>>>> > while online snippets to try are meant to nearly always produce a
>>>> > meaningful output.
>>> I don't know - I like the approach Andrei took in the docs, writing
>>> a series of true assertations.
>>>
>>> assert(sort([4, 2]) == [2, 4]);
>>>
>>> does look pretty neat. I guess the alternative is:
>>>
>>> writeln(sort[4, 2]); // prints "[2, 4]"
>>
>> What about
>>
>> // sort([4, 2]) == [2, 4])
>>
>> ? Looks to me both simpler & clearer.
>>
>> Denis
>
> I don't like that one bit. What is it trying to achieve or improve?

assert(sort([4, 2]) == [2, 4]);

What is it trying to achieve or improve?
plus: is this bebug code / error catching; or information to the reader?

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On Thu, 14 Apr 2011 11:28:39 -0400, spir <denis.spir@gmail.com> wrote:

> On 04/14/2011 04:03 PM, Steven Schveighoffer wrote:
>> Sometimes, I worry that my unit tests or asserts aren't running.  Every once in
>> a while, I have to change one to fail to make sure that code is compiling (this
>> is especially true when I'm doing version statements or templates).  It would
>> be nice if there was a -assertprint mode which showed asserts actually running
>> (only for the module compiled with that switch, of course).
>
> Man, I'm very pleased to read someone else advocating for optionally verbose assertions.
> This could use 2 arguments instead of a predicate:
>      assert(expressions, value);
> Example use:

[snip]

I don't think we can change assert syntax now.  What I was looking was for something more like:

assert(x == y);

prints out

"asserting x == y: true"

for asserts that pass when you have the 'verbose assert' flag turned on.  This should be easy to do in the compiler by just translating

assert(expr);

to something like:

auto result = evaluateAssert(expr);
print("asserting expr: ", result ? "true" : "false");
assert(result);

-Steve

Am 14.04.2011 17:47, schrieb Steven Schveighoffer:
> On Thu, 14 Apr 2011 11:28:39 -0400, spir <denis.spir@gmail.com> wrote:
> 
>> On 04/14/2011 04:03 PM, Steven Schveighoffer wrote:
>>> Sometimes, I worry that my unit tests or asserts aren't running.
>>> Every once in
>>> a while, I have to change one to fail to make sure that code is
>>> compiling (this
>>> is especially true when I'm doing version statements or templates).
>>> It would
>>> be nice if there was a -assertprint mode which showed asserts
>>> actually running
>>> (only for the module compiled with that switch, of course).
>>
>> Man, I'm very pleased to read someone else advocating for optionally
>> verbose assertions.
>> This could use 2 arguments instead of a predicate:
>>      assert(expressions, value);
>> Example use:
> 
> [snip]
> 
> I don't think we can change assert syntax now.  What I was looking was for something more like:
> 
> assert(x == y);
> 
> prints out
> 
> "asserting x == y: true"
> 
> for asserts that pass when you have the 'verbose assert' flag turned on.  This should be easy to do in the compiler by just translating
> 
> assert(expr);
> 
> to something like:
> 
> auto result = evaluateAssert(expr);
> print("asserting expr: ", result ? "true" : "false");
> assert(result);
> 
> -Steve

Another possibility are named unittests and printing out "running test
'foobar' was successful".
One message for every assert is too verbose IMHO.

Cheers,
- Daniel

On 4/14/11 9:03 AM, Steven Schveighoffer wrote:
> Sometimes, I worry that my unit tests or asserts aren't running. Every
> once in a while, I have to change one to fail to make sure that code is
> compiling (this is especially true when I'm doing version statements or
> templates). It would be nice if there was a -assertprint mode which
> showed asserts actually running (only for the module compiled with that
> switch, of course).

Could this be achieved within the language?

Andrei

On Thu, 14 Apr 2011 12:35:49 -0400, Daniel Gibson <metalcaedes@gmail.com> wrote:

> Am 14.04.2011 17:47, schrieb Steven Schveighoffer:
>> On Thu, 14 Apr 2011 11:28:39 -0400, spir <denis.spir@gmail.com> wrote:
>>
>>> On 04/14/2011 04:03 PM, Steven Schveighoffer wrote:
>>>> Sometimes, I worry that my unit tests or asserts aren't running.
>>>> Every once in
>>>> a while, I have to change one to fail to make sure that code is
>>>> compiling (this
>>>> is especially true when I'm doing version statements or templates).
>>>> It would
>>>> be nice if there was a -assertprint mode which showed asserts
>>>> actually running
>>>> (only for the module compiled with that switch, of course).
>>>
>>> Man, I'm very pleased to read someone else advocating for optionally
>>> verbose assertions.
>>> This could use 2 arguments instead of a predicate:
>>>      assert(expressions, value);
>>> Example use:
>>
>> [snip]
>>
>> I don't think we can change assert syntax now.  What I was looking was
>> for something more like:
>>
>> assert(x == y);
>>
>> prints out
>>
>> "asserting x == y: true"
>>
>> for asserts that pass when you have the 'verbose assert' flag turned
>> on.  This should be easy to do in the compiler by just translating
>>
>> assert(expr);
>>
>> to something like:
>>
>> auto result = evaluateAssert(expr);
>> print("asserting expr: ", result ? "true" : "false");
>> assert(result);
>>
>> -Steve
>
> Another possibility are named unittests and printing out "running test
> 'foobar' was successful".
> One message for every assert is too verbose IMHO.

I'm thinking of the code examples that have asserts in them.  I totally like the idea of code examples in ddoc being compiled as unit tests.  This ensures your examples don't get out of date or are not invalid (as many of phobos' examples are).  Nothing is worse when learning a language to encounter errors or unexpected results from the sanctioned examples.  This means they have to look like unit tests, but also be useful for learning (see my above issue).

A compromise might be to be able to name unit tests, and then run a specific unit test by name on execution only.  This allows you to ensure a specific unit test (and specific asserts in that unit test) are running without having to see all the asserts from the other unit tests, but also allows using asserts for general example code/testing.

This also could mean that you could simply directly compile a phobos module with -assertverbose -unittest and do:

./algorithm --run-unit-test std.algorithm.sort_example

to see all the asserts working for that example.

-Steve

On Thu, 14 Apr 2011 12:48:26 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 4/14/11 9:03 AM, Steven Schveighoffer wrote:
>> Sometimes, I worry that my unit tests or asserts aren't running. Every
>> once in a while, I have to change one to fail to make sure that code is
>> compiling (this is especially true when I'm doing version statements or
>> templates). It would be nice if there was a -assertprint mode which
>> showed asserts actually running (only for the module compiled with that
>> switch, of course).
>
> Could this be achieved within the language?

I think you need to do it at the compiler level to make it useful.  For example, an assert like:

assert(container.length == 5);

To print this out properly, I'd want to see that the assert passed, but also what the test was.  I think there is already some work being commissioned (if not already in progress) on dissecting the expression into string form in the compiler.

I don't know how you would do this in a DRY fashion without compiler help.  You can't use a mixin string because mixins are compiled in the context where you mix them in.

I outlined a possible solution in a response to Daniel that I believe:

1. allows one to 'see' an example working that just uses asserts
2. allows one to run specific unit tests to make sure they are working (via names)
3. does not require any changes to any asserts in existance.

I think this would be an awesome feature.  One of the cool things about the new way phobos runs unit tests (one module at a time) is that if a particular test fails, I can just build/run it over and over again until it passes.  This is huge since to run the full phobos unit tests can take a while.

To just be able to select on the command line how to run unit tests (I think overtaking the command line args for unittest-enabled code is reasonable) would be a great tool for debugging code via unittests and assert printouts.

It also gives us a smooth transition into using ddoc examples as unit tests.

-Steve