Thread overview
[dmd-internals] The proposed -Xcc dmd option would be quite useful for PLplot for the -pthread case
Sep 25, 2019
Alan W. Irwin
Sep 25, 2019
Mike Franklin
Sep 25, 2019
Alan W. Irwin
Sep 25, 2019
Alan W. Irwin
Sep 26, 2019
Mike Franklin
Sep 26, 2019
Alan W. Irwin
Sep 26, 2019
Johan Engelen
Sep 26, 2019
Mike Franklin
Sep 26, 2019
Johan Engelen
September 24, 2019
To dmd developers:

I have found a number of bug reports (e.g.,
<https://issues.dlang.org/show_bug.cgi?id=7678> and
<https://issues.dlang.org/show_bug.cgi?id=17390>) concerning the issue
of implementing for the dmd compiler the equivalent of the existing
ldc2 -Xcc option to allow communicating options directly to the C
compiler that is actually used by dmd (according to its -v option) for
linking.

I would like to add my opinion that it would be good to get this
requested -Xcc option implemented for dmd because of the following use
case concerning -pthread (a common linking option used by many
compilers to help arrange all the details correctly for linking the
thread library).

That use case is the PLplot project CMake-based build system for the
static PLplot libraries case necessarily links its D plotting examples
by linking a mixture of C-compiled code from the static plplot core C
library, D-compiled code from the static plplotdmd library that
implements the D binding for that C library, and the D-compiled code
from the D example.  This all works well for our CMake-based build
system except for a fairly common configuration of our C library where
the C code of that library must be linked with the -pthread compiler
option when that C compiler is *used for linking*.  So this all works
well for the gdc compiler when it is used to link the D examples
because it understands that -pthread option.  And it also works well
for ldc2 if the -Xcc-pthread option is used to inform the underlying C
compiler used for linking of that -pthread option.

However, that -Xcc option is currently not available for dmd (as far
as I can tell from the above bug reports and from my attempts to use
that option with a dmd compiler that I built myself on Linux) so I am
pretty much stuck for this use case and dmd until the -Xcc option
becomes available for the dmd case as well.

Anyhow, thanks for listening to my request to implement the -Xcc
option for the dmd compiler, and best wishes for your on-going
development of dmd.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


September 25, 2019
 This issue has been resolved with the `-preview=noXlinker` switch:  https://dlang.org/changelog/pending.html#noxlinker

It will allow users to pass options directly to the linker driver without DMD trying to be clever.
It should arrive in the next release.
See also https://issues.dlang.org/show_bug.cgi?id=6952
    On Wednesday, September 25, 2019, 12:17:42 PM GMT+9, Alan W. Irwin via dmd-internals <dmd-internals@puremagic.com> wrote:

 To dmd developers:

I have found a number of bug reports (e.g., <https://issues.dlang.org/show_bug.cgi?id=7678> and <https://issues.dlang.org/show_bug.cgi?id=17390>) concerning the issue of implementing for the dmd compiler the equivalent of the existing ldc2 -Xcc option to allow communicating options directly to the C compiler that is actually used by dmd (according to its -v option) for linking.

I would like to add my opinion that it would be good to get this requested -Xcc option implemented for dmd because of the following use case concerning -pthread (a common linking option used by many compilers to help arrange all the details correctly for linking the thread library).

That use case is the PLplot project CMake-based build system for the static PLplot libraries case necessarily links its D plotting examples by linking a mixture of C-compiled code from the static plplot core C library, D-compiled code from the static plplotdmd library that implements the D binding for that C library, and the D-compiled code from the D example.  This all works well for our CMake-based build system except for a fairly common configuration of our C library where the C code of that library must be linked with the -pthread compiler option when that C compiler is *used for linking*.  So this all works well for the gdc compiler when it is used to link the D examples because it understands that -pthread option.  And it also works well for ldc2 if the -Xcc-pthread option is used to inform the underlying C compiler used for linking of that -pthread option.

However, that -Xcc option is currently not available for dmd (as far as I can tell from the above bug reports and from my attempts to use that option with a dmd compiler that I built myself on Linux) so I am pretty much stuck for this use case and dmd until the -Xcc option becomes available for the dmd case as well.

Anyhow, thanks for listening to my request to implement the -Xcc option for the dmd compiler, and best wishes for your on-going development of dmd.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


September 24, 2019
On 2019-09-25 03:52-0000 Mike Franklin via dmd-internals wrote:

> This issue has been resolved with the `-preview=noXlinker` switch:  https://dlang.org/changelog/pending.html#noxlinker
>
> It will allow users to pass options directly to the linker driver without DMD trying to be clever.
> It should arrive in the next release.

Hi Mike:

I was very glad to hear that news for the reason I stated in my
initial post.  Thanks to your hint, I now see the relevant commit
(v2.088.0-rc.1-142-gd6183e9ff) on the master branch so I plan to build
and use that (or more likely the current master branch tip version)
for my further tests until 2.088.0 is released.

Those different -L semantics for the -preview=noXlinker dmd option
likely mean I will need to change at least some of the dmd -L options
I use now in my fork of the cmake-d project that currently provides
PLplot with CMake D language support.  Aside from this -pthread issue,
that support was perfect for dmd 2.087.0 on Linux, and there was also
a good preliminary result on Darwin that someone reported for the
MacPorts version of dmd.  So I anticipate with some additional work
this support will be perfect (this time with no -pthread issue) on
Linux and also Darwin for dmd 2.088.0.

By the way, I discovered for 2.087.0 that the Linux install after the
build did not provide a working dmd because certain *.d source files
(I have forgotten which ones) were missing from the install.  So I
ended up using the build-tree version of dmd instead which worked
because it could find those necessary *.d files within the build tree.
Anyhow, if I discover this "missing *.d files within the install tree"
issue again for current master tip, I will report this as a dmd bug
(unless you or someone else lurking on this list beats me to it as a
result of this informal bug report).

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals
September 25, 2019
Hi Mike:

On 2019-09-24 22:42-0700 Alan W. Irwin via dmd-internals wrote:

> So I anticipate with some additional work
> this support will be perfect (this time with no -pthread issue) on
> Linux and also Darwin for dmd 2.088.0.

That, and everywhere else I referred to 2.088.0 should obviously be 2.089.0 since
2.088.0 is already released.

> By the way, I discovered for 2.087.0 that the Linux install after the
> build did not provide a working dmd because [... object.d]
> was missing from the install.

That issue (now fixed) was caused by my dmd build script not properly installing
the druntime component.

So the result of running that script today as follows:

./dmd_git_build.sh HEAD

(where HEAD updates all local repositories to the server master HEAD version before building)

is that

install/linux/bin64/dmd -preview=noXlinker

is so far performing exactly as advertised (i.e., with different -L semantics).

So I now appear to have everything I need to move forward again with
implementing CMake D language support for dmd (with
-preview=noXlinker), and thanks very much for your key help in making that
possible.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


September 26, 2019
 Cool! Glad to have helped.  Best of luck with your work.
Mike

   On Thursday, September 26, 2019, 8:38:04 AM GMT+9, Alan W. Irwin via dmd-internals <dmd-internals@puremagic.com> wrote:

 Hi Mike:

On 2019-09-24 22:42-0700 Alan W. Irwin via dmd-internals wrote:

> So I anticipate with some additional work
> this support will be perfect (this time with no -pthread issue) on
> Linux and also Darwin for dmd 2.088.0.

That, and everywhere else I referred to 2.088.0 should obviously be 2.089.0 since 2.088.0 is already released.

> By the way, I discovered for 2.087.0 that the Linux install after the
> build did not provide a working dmd because [... object.d]
> was missing from the install.

That issue (now fixed) was caused by my dmd build script not properly installing the druntime component.

So the result of running that script today as follows:

./dmd_git_build.sh HEAD

(where HEAD updates all local repositories to the server master HEAD version before building)

is that

install/linux/bin64/dmd -preview=noXlinker

is so far performing exactly as advertised (i.e., with different -L semantics).

So I now appear to have everything I need to move forward again with
implementing CMake D language support for dmd (with
-preview=noXlinker), and thanks very much for your key help in making that
possible.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


September 25, 2019
On 2019-09-26 00:38-0000 Mike Franklin via dmd-internals wrote:

> Cool! Glad to have helped.  Best of luck with your work.

Thanks.

I spoke too soon (sigh) about this new -preview=noXlinker option
actually working, see <https://issues.dlang.org/show_bug.cgi?id=20244>.

I will be happy to test the dmd git master branch again once the fix for this
bug lands there.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals
September 26, 2019
On Thu, Sep 26, 2019 at 4:13 AM Alan W. Irwin via dmd-internals < dmd-internals@puremagic.com> wrote:

>
> I spoke too soon (sigh) about this new -preview=noXlinker option actually working, see <https://issues.dlang.org/show_bug.cgi?id=20244>.
>

Reading that bug report, it appears to me that this is not a simple "preview" option: dmd.conf may need adjustment and thus the user cannot simply switch between the option being on or off. Why not implement LDC's -Xcc as well?

-Johan


September 26, 2019
 Johan,
Yes, users opting into the option with `-preview=noXlinker` will need to fully opt into it.  So it will be best for them to just add `-preview=noXlinker` and other adjustments to their dmd.conf file.
The reason I chose not to implement `-Xcc` is because we have too many ways of doing the same thing. The code is becoming difficult to learn and maintain as we try to carry forward the design decisions of the past indefinitely.  I'm sure you and others will disagree, but I think it is best to invest with the hope that someday we can delete legacy code, not continue to add to it.
Mike

    On Thursday, September 26, 2019, 2:39:16 PM GMT+9, Johan Engelen via dmd-internals <dmd-internals@puremagic.com> wrote:

 On Thu, Sep 26, 2019 at 4:13 AM Alan W. Irwin via dmd-internals <dmd-internals@puremagic.com> wrote:


I spoke too soon (sigh) about this new -preview=noXlinker option actually working, see <https://issues.dlang.org/show_bug.cgi?id=20244>.


Reading that bug report, it appears to me that this is not a simple "preview" option: dmd.conf may need adjustment and thus the user cannot simply switch between the option being on or off. Why not implement LDC's -Xcc as well?
-Johan

 _______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

September 26, 2019
I agree with striving to remove legacy code.
The problem I see with the current solution is that there is no migration
path. The preview switch has the appearance of an optional thing, but it
isn't. Editing dmd/ldc2.conf is not a viable hard requirement in order to
use plplot.

The bigger problem that I see is that we are making D compilation compile flags even more awkward. The noXlinker option means the user has to pass to dmd a gcc option ("-Xlinker") in order to pass a flag to the linker....?!

It's nice there is some sort of solution for the plplot issue. But I'm happy I don't have to implement and maintain the logic of figuring out in a cmake script which dmd version is used, or ldc, or gdc, or what flag is put in dmd.conf that changes cmdline behavior,.............

-Johan


On Thu, Sep 26, 2019, 07:46 Mike Franklin via dmd-internals < dmd-internals@puremagic.com> wrote:

> Johan,
>
> Yes, users opting into the option with `-preview=noXlinker` will need to fully opt into it.  So it will be best for them to just add `-preview=noXlinker` and other adjustments to their dmd.conf file.
>
> The reason I chose not to implement `-Xcc` is because we have too many ways of doing the same thing. The code is becoming difficult to learn and maintain as we try to carry forward the design decisions of the past indefinitely.  I'm sure you and others will disagree, but I think it is best to invest with the hope that someday we can delete legacy code, not continue to add to it.
>
> Mike
>
> On Thursday, September 26, 2019, 2:39:16 PM GMT+9, Johan Engelen via dmd-internals <dmd-internals@puremagic.com> wrote:
>
>
> On Thu, Sep 26, 2019 at 4:13 AM Alan W. Irwin via dmd-internals < dmd-internals@puremagic.com> wrote:
>
>
> I spoke too soon (sigh) about this new -preview=noXlinker option actually working, see <https://issues.dlang.org/show_bug.cgi?id=20244
> >.
>
>
> Reading that bug report, it appears to me that this is not a simple "preview" option: dmd.conf may need adjustment and thus the user cannot simply switch between the option being on or off. Why not implement LDC's -Xcc as well?
>
> -Johan
>
>
>
> _______________________________________________
> dmd-internals mailing list
> dmd-internals@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
> _______________________________________________
> dmd-internals mailing list
> dmd-internals@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals