Several CI pipelines are failing for https://github.com/dlang/phobos/pull/7638, so I moseyed to what I thought would be a simple one, the doc build:

http://dtest.dlang.io/results/d42adbc33e821c4223291e64541d57ffbb5b3b0d/eb9e6443fd3231b8d086509d950283587ac6ee9e/build.log

I spent some time trying to make heads and tails of it. A few things that could be improved:

* It's 2942 lines long. Most of these lines are just chaff, e.g. do we really need to log every single cp operation, or would a "Copied all files" line suffice?

* The word '\Werror\W' appears in the log 25 times. Although I assume most are to be ignored, some occur in quite dire contexts, such as: "Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('". Is it possible to massage error messages known to be benign?

* Some of the command lines being run are printed like this:

[2020-09-20 02:25:55.606583] dautotest: Running: 'timeout' '1800' 'make' '-f' 'posix.mak' 'DMD=/home/dtest/DAutoTest/work/build/bin/dmd' '-j' '8' 'MODEL=64'

Command lines are definitely something people would want to copy and paste from the log. They should be printed like this:

[2020-09-20 02:25:55.606583] dautotest: Running: 'timeout 1800 make -f posix.mak DMD=/home/dtest/DAutoTest/work/build/bin/dmd -j 8 MODEL=64'

The lines that carry actual information appear like this:

src/dmd/frontend.d(142): Error: template std.algorithm.iteration.each cannot deduce function from argument types !(addGlobalIdent)(const(string[])), candidates are:
/dev/shm/dtest/work/repo/dmd/generated/linux/release/64/../../../../../phobos/std/algorithm/iteration.d(805):        each(alias fun = "a", Range)(auto ref Range range)
  with fun = addGlobalIdent,
       Range = const(string[])
  must satisfy one of the following constraints:
       isInputRange!Range
       isStaticArray!Range
       hasMember!(Range, "opApply")

They have escape sequences for pretty printing with colors. Whatever system is producing that output should detect it's not running in a console and omit the escape sequences.

Thanks in advance!

On Sunday, 20 September 2020 at 03:16:57 UTC, Andrei Alexandrescu wrote:
> Several CI pipelines are failing for https://github.com/dlang/phobos/pull/7638, so I moseyed to what I thought would be a simple one, the doc build:
>
> http://dtest.dlang.io/results/d42adbc33e821c4223291e64541d57ffbb5b3b0d/eb9e6443fd3231b8d086509d950283587ac6ee9e/build.log
>
> I spent some time trying to make heads and tails of it. A few things that could be improved:
>
> * It's 2942 lines long. Most of these lines are just chaff, e.g. do we really need to log every single cp operation, or would a "Copied all files" line suffice?
>
> * The word '\Werror\W' appears in the log 25 times. Although I assume most are to be ignored, some occur in quite dire contexts, such as: "Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('". Is it possible to massage error messages known to be benign?
>
> * Some of the command lines being run are printed like this:
>
> [2020-09-20 02:25:55.606583] dautotest: Running: 'timeout' '1800' 'make' '-f' 'posix.mak' 'DMD=/home/dtest/DAutoTest/work/build/bin/dmd' '-j' '8' 'MODEL=64'
>
> Command lines are definitely something people would want to copy and paste from the log. They should be printed like this:
>
> [2020-09-20 02:25:55.606583] dautotest: Running: 'timeout 1800 make -f posix.mak DMD=/home/dtest/DAutoTest/work/build/bin/dmd -j 8 MODEL=64'
>
> The lines that carry actual information appear like this:
>
> src/dmd/frontend.d(142): Error: template std.algorithm.iteration.each cannot deduce function from argument types !(addGlobalIdent)(const(string[])), candidates are:
> /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/../../../../../phobos/std/algorithm/iteration.d(805):        each(alias fun = "a", Range)(auto ref Range range)
>   with fun = addGlobalIdent,
>        Range = const(string[])
>   must satisfy one of the following constraints:
>        isInputRange!Range
>        isStaticArray!Range
>        hasMember!(Range, "opApply")
>
> They have escape sequences for pretty printing with colors. Whatever system is producing that output should detect it's not running in a console and omit the escape sequences.
>
> Thanks in advance!

Well, your PRs break the world and this time the CIs actually manage to catch it.
Have a look at the Buildkite output of your PR: https://buildkite.com/dlang/phobos/builds/3903 where it breaks the build of many real-world projects:

---
make[1]: Entering directory '/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize'
---> Running tsv-summarize unit tests
dmd -I/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/common/src/tsv_utils/common /var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/common/src/tsv_utils/common/utils.d /var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/common/src/tsv_utils/common/numerics.d /var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/common/src/tsv_utils/common/fieldlist.d /var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/common/src/tsv_utils/common/getopt_inorder.d /var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/common/src/tsv_utils/common/unittest_utils.d /var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/common/src/tsv_utils/common/tsvutils_version.d  -unittest -main -run /var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize/src/tsv_utils/tsv-summarize.d
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize/src/tsv_utils/tsv-summarize.d(815): Error: template `std.algorithm.iteration.each` cannot deduce function from argument types `!((x) => x.processHeaderLine(lineFields))(DList!(Operator))`, candidates are:
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/distribution/bin/../imports/std/algorithm/iteration.d(805):        `each(alias fun = "a", Range)(auto ref Range range)`
  with `fun = __lambda2,
       Range = DList!(Operator)`
  must satisfy one of the following constraints:
`       isInputRange!Range
       isStaticArray!Range
       hasMember!(Range, "opApply")`
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize/src/tsv_utils/tsv-summarize.d(842): Error: template instance `tsv_utils.tsv_summarize.SummarizerBase!(BufferedOutputRange!(File))` error instantiating
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize/src/tsv_utils/tsv-summarize.d(559):        instantiated from here: `NoKeySummarizer!(BufferedOutputRange!(File))`
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize/src/tsv_utils/tsv-summarize.d(815): Error: template `std.algorithm.iteration.each` cannot deduce function from argument types `!((x) => x.processHeaderLine(lineFields))(DList!(Operator))`, candidates are:
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/distribution/bin/../imports/std/algorithm/iteration.d(805):        `each(alias fun = "a", Range)(auto ref Range range)`
  with `fun = __lambda2,
       Range = DList!(Operator)`
  must satisfy one of the following constraints:
`       isInputRange!Range
       isStaticArray!Range
       hasMember!(Range, "opApply")`
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize/src/tsv_utils/tsv-summarize.d(842): Error: template instance `tsv_utils.tsv_summarize.SummarizerBase!(Appender!(char[]))` error instantiating
/var/lib/buildkite-agent/builds/ci-agent-f16edc43-2dc3-4961-b9ae-3fc8efb9b382-4/dlang/phobos/build/eBay-tsv-utils/tsv-summarize/src/tsv_utils/tsv-summarize.d(1091):        instantiated from here: `NoKeySummarizer!(Appender!(char[]))`
../makeapp.mk:48: recipe for target 'unittest' failed
---

> * It's 2942 lines long. Most of these lines are just chaff,

Well, extra verbosity is nice when things fail.
Why don't you scroll to the bottom?

> I spent some time trying to make heads and tails of it.

Huh, it yields a fairly nice nice error message: (it's from the link you posted)

---
The following operation failed:
Dependencies:
 -> dmd
 -> /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/VERSION
 -> /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/SYSCONFDIR.imp

Sources:
 -> /dev/shm/dtest/work/repo/dmd/src/dmd/frontend.d

Targets:
 -> /dev/shm/dtest/work/repo/dlang.org/web/phobos-prerelease/dmd_frontend.html

Command: /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/dmd -o- -c -Dd/dev/shm/dtest/work/repo/dlang.org -J/dev/shm/dtest/work/repo/dmd/src/dmd/res -I/dev/shm/dtest/work/repo/dmd/src/dmd /dev/shm/dtest/work/repo/dmd/src/project.ddoc -Df/dev/shm/dtest/work/repo/dlang.org/web/phobos-prerelease/dmd_frontend.html src/dmd/frontend.d -version=MARS -w -de -fPIC -m64 -J/dev/shm/dtest/work/repo/dmd/generated/linux/release/64 -I/dev/shm/dtest/work/repo/dmd/src -dip25 -g -color=on
---

> Command lines are definitely something people would want to copy and paste from the log. They should be printed like this:

I am sorry, but I don't see your point.
1) The command as run must be displayed (no modification)
2) They actually work as displayed. Try `'timeout' '2s' 'sleep' '5s'` in your shell.

> > They have escape sequences for pretty printing with colors.
> Whatever system is producing that output should detect it's not running in a console and omit the escape sequences.

https://issues.dlang.org/show_bug.cgi?id=21266

On Sunday, 20 September 2020 at 03:16:57 UTC, Andrei Alexandrescu wrote:
> Several CI pipelines are failing for https://github.com/dlang/phobos/pull/7638, so I moseyed to what I thought would be a simple one, the doc build:
>
> [...]

I must say I really sympathize with your efforts to clean things up. Code in general just grows if you don't actively control it along the way.

On Sunday, 20 September 2020 at 06:35:22 UTC, Seb wrote:
> On Sunday, 20 September 2020 at 03:16:57 UTC, Andrei Alexandrescu wrote:
>> > They have escape sequences for pretty printing with colors.
>> Whatever system is producing that output should detect it's not running in a console and omit the escape sequences.
>
> https://issues.dlang.org/show_bug.cgi?id=21266

I've never had a problem with the output of build.d with any CI or any other use case I've come across. This should really be a dautotest bug. This is the burden that occurs when you roll your own implementation for everything.

On Sunday, 20 September 2020 at 03:16:57 UTC, Andrei Alexandrescu wrote:
> src/dmd/frontend.d(142): Error: template std.algorithm.iteration.each cannot deduce function from argument types !(addGlobalIdent)(const(string[])), candidates are:
> /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/../../../../../phobos/std/algorithm/iteration.d(805):        each(alias fun = "a", Range)(auto ref Range range)
>   with fun = addGlobalIdent,
>        Range = const(string[])
>   must satisfy one of the following constraints:
>        isInputRange!Range
>        isStaticArray!Range
>        hasMember!(Range, "opApply")

Yes observed too, but very rarily too in the IDE output when I particpated to DMD devel. DMD was build using an IDE tool that called make -f posix.mak. So build.d is called indirectly, dont know if this is the cause. But I can confirm that this is not a CI problem.

On 9/20/20 2:35 AM, Seb wrote:
> Well, your PRs break the world and this time the CIs actually manage to catch it.
> Have a look at the Buildkite output of your PR: https://buildkite.com/dlang/phobos/builds/3903 where it breaks the build of many real-world projects:

Yes, that's great. Thanks. Some of the logs are easier to get into than others.

>> * It's 2942 lines long. Most of these lines are just chaff,
> 
> Well, extra verbosity is nice when things fail.
> Why don't you scroll to the bottom?

Extra verbosity is best when informative.

I understand and agree with this and all of your points. In fact I could have written a similar reply to my own post. It's all a matter of where we come from - if I came from the vantage point "Here's another baseless complaint about the logs that work just fine, let's deconstruct it" the reply would come one way. The problem being, of course, that that sets up the suggester and the maintainer on reciprocally defensive positions, whereby energy goes in the back-and-forth instead of productive improvements.

If we look at things as "logs are good and could be better, let's see how we can get information from here to improve the lot of contributors" then the answer comes very differently. From the same person!

> Huh, it yields a fairly nice nice error message: (it's from the link you posted)
> 
> ---
> The following operation failed:
> Dependencies:
>   -> dmd
>   -> /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/VERSION
>   -> /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/SYSCONFDIR.imp
> 
> Sources:
>   -> /dev/shm/dtest/work/repo/dmd/src/dmd/frontend.d
> 
> Targets:
>   -> /dev/shm/dtest/work/repo/dlang.org/web/phobos-prerelease/dmd_frontend.html
> 
> Command: /dev/shm/dtest/work/repo/dmd/generated/linux/release/64/dmd -o- -c -Dd/dev/shm/dtest/work/repo/dlang.org -J/dev/shm/dtest/work/repo/dmd/src/dmd/res -I/dev/shm/dtest/work/repo/dmd/src/dmd /dev/shm/dtest/work/repo/dmd/src/project.ddoc -Df/dev/shm/dtest/work/repo/dlang.org/web/phobos-prerelease/dmd_frontend.html src/dmd/frontend.d -version=MARS -w -de -fPIC -m64 -J/dev/shm/dtest/work/repo/dmd/generated/linux/release/64 -I/dev/shm/dtest/work/repo/dmd/src -dip25 -g -color=on
> ---

A couple of things come to mind.

* The line where the build failed in frontend.d would be good to have.

* The word "error" does not appear anywhere. "Failed" is good, but it would be great to distinguish these particular lines as the cause of failure. This is because there are several other places that one may legitimately think are problematic, such as:

magic.d(9:39)[error]: Declaration expected
magic.d(76:1)[error]: Declaration expected
magic.d(15:1)[error]: Declaration expected
... about a dozen like these ...

and:

No type found for dmd.parse.Parser.Parser.NeedDeclaratorId.
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
Function object.ModuleInfo.tlsctor has non-function type: Primitive
Error parsing type 'const(void function()* function() pure nothrow @property @nogc)': Missing ')' for 'const('
... about three dozen like these ...

I mean these don't look like something a build would come back from. In fact I legit thought there was some breakage in the CI system and wanted to ask about it. The clue that stopped me from doing this was there was a lot more output after that.

>> Command lines are definitely something people would want to copy and paste from the log. They should be printed like this:
> 
> I am sorry, but I don't see your point.
> 1) The command as run must be displayed (no modification)
> 2) They actually work as displayed. Try `'timeout' '2s' 'sleep' '5s'` in your shell.

Of course. However, workable does not necessarily mean adequate. That's not how people read and write shell commands. I wouldn't want to edit such a command or have it in my .history file. We shouldn't make it unnecessarily difficult for people to read and edit such commands. Most commands appear in a different manner, this is not something one would want to go out of one's way to differentiate.

I was thinking, for example, we could add to std.process a function shellQuote that, given a string, returns that string with minimal additions of escapes to be run in a shell. In fact I think rdmd had such a function at a point, because it's still in one comment: https://github.com/dlang/tools/blob/master/rdmd.d#L698.

>> > They have escape sequences for pretty printing with colors.
>> Whatever system is producing that output should detect it's not running in a console and omit the escape sequences.
> 
> https://issues.dlang.org/show_bug.cgi?id=21266

That's the spirit. Thanks! Probably -color=off in the cmdline would suffice?

On 9/19/2020 8:16 PM, Andrei Alexandrescu wrote:
> Several CI pipelines are failing for https://github.com/dlang/phobos/pull/7638,

I've run into so many problems with the CI, I added a keyword `TestSuite` for them in bugzilla:

https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&keywords=TestSuite%2C%20&keywords_type=allwords&list_id=233243&query_format=advanced

Currently, the most annoying one is:

https://issues.dlang.org/show_bug.cgi?id=21265

because it causes maybe 25% of my test runs to fail, and when it fails I have to restart the entire test suite, usually costing me a half hour of waiting. Who knows what time it is costing others.

Anyhow, please add test suite issues you find to this list.

On 9/19/2020 11:35 PM, Seb wrote:
> https://issues.dlang.org/show_bug.cgi?id=21266

Thank you. Tagged that with "TestSuite" keyword.

On Sunday, 20 September 2020 at 20:36:37 UTC, Walter Bright wrote:
> On 9/19/2020 8:16 PM, Andrei Alexandrescu wrote:
>> Several CI pipelines are failing for https://github.com/dlang/phobos/pull/7638,
>
> I've run into so many problems with the CI, I added a keyword `TestSuite` for them in bugzilla:
>
> https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&keywords=TestSuite%2C%20&keywords_type=allwords&list_id=233243&query_format=advanced
>
> Currently, the most annoying one is:
>
> https://issues.dlang.org/show_bug.cgi?id=21265
>
> because it causes maybe 25% of my test runs to fail, and when it fails I have to restart the entire test suite, usually costing me a half hour of waiting. Who knows what time it is costing others.

As a workaround, why don't you just comment out that test. Run the rest of the test suite. Then run only that test. If it fails, you can rerun just that test and not the whole test suite.

--
/Jacob Carlborg