On Fri, Jun 09, 2017 at 10:34:00PM +0000, Seb via Digitalmars-d wrote:
> On Friday, 9 June 2017 at 22:14:30 UTC, H. S. Teoh wrote:
> > I'm trying to track down an autotester failure in a DMD PR, and getting very frustrated that the DMD test suite does not run on Debian/Linux because of the whole PIE-by-default fiasco that necessitates compiling everything with -fPIC.
> > 
> > (1) Currently, there is no way to customize how test/Makefile compiles d_do_tests.d, so there is no way to insert -fPIC into the offending compile line.
> 
> Once this PR is merged, it will at least use the default conf file:
> 
> https://github.com/dlang/dmd/pull/6870
> 
> (the config file is now in generated and this lead to nice errors on
> my machine)

But you see, that's the problem. What happens if the auto-generated config file needs further adjustment?


> > (2) d_do_tests itself runs compile commands with `-conf=`, so putting -fPIC inside dmd.conf has no effect (where otherwise it does the right thing when dmd is invoked "normally"). There's no obvious way to make customize the compile command used for each test: I tried setting the environment variable ARGS but that just messed up the test runner's expected outputs. Tried to add -fPIC to the DMD environment variable but that blew up in a different way.
> 
> Do you mean DFLAGS with DMD environment variable?

There is no DFLAGS defined in d_do_tests.d.  The only configurable parameter I can see is the DMD variable, so I tried to bluff d_do_tests by setting DMD="dmd -fPIC", but that also didn't work, maybe because it doesn't go through shell interpolation so it tries to find a non-existent executable named "dmd -fPIC".

Anyway, I eventually went back to my trusty old anti-makefile workaround: create a wrapper shell script that does this:

	#!/bin/sh
	/path/to/dmd -fPIC -I/path/to/druntime/import -I/path/to/phobos "$@"

(Yes, the -I's are necessary, I don't know why and don't really want to
at this point.)

Save this to /path/to/dmdwrapper, and then run make with DMD=/path/to/dmdwrapper, and lo and behold, finally the miserable test suite is runnable.

That was 2 hours' worth of frustration just to get the testsuite running.  I dunno about you, but being unable to run the dmd testsuite seems to me to be a big turnoff for potential dmd contributors. I surmise not every potential dmd contributor is as persistent as I am. :-D


> > So basically, I'm stuck up PIE creek without a paddle, and have no way to locally test DMD changes without hogging more load on the autotester.
> 
> Don't worry about that. It's just a computer. As a temporary workaround, docker might be worth considering.

Wow.  Way to bring out the shotgun to kill an ant!  I think installing docker would be total overkill for something that really ought to work out-of-the-box...!


> > Why is something so simple so hard to do?!
> 
> I think it's simply because not many people use the DMD test suite and those who do are used to the pain or have their custom-built tricks. Also I guess that there are two groups of people:
> 
> A) wow that's complicated and frustrating -> move on to the next great thing in life B) eh what the heck. I have seen worse...
> 
> (And not many in-between).

Ha, so now I've invented my own custom-built trick to make it work, and so that's no help to the next would-be dmd contributor that comes along and can't figure out how to make it work. :-/

//

On a tangential note, now that I finally got the miserable test suite running, I've managed to figure out that the problem in my dmd PR was caused by src/ddmd/globals.h not being auto-generated from src/ddmd/globals.d, so when I added new fields to the Global struct, the C++ code that use globals.h with the old definition crashed. I suppose this is just a relic from the pre-self-hosting days, but it would be nice if these last .h files were gotten rid of in the near future.

That, or we should (ab)use the .di generation facility or some sed script to generate these .h files, so that they won't go out-of-sync inadvertently.


T

-- 
LINUX = Lousy Interface for Nefarious Unix Xenophobes.

H. S. Teoh wrote:

> On a tangential note, now that I finally got the miserable test suite
> running, I've managed to figure out that the problem in my dmd PR was
> caused by src/ddmd/globals.h not being auto-generated from
> src/ddmd/globals.d, so when I added new fields to the Global struct, the
> C++ code that use globals.h with the old definition crashed. I suppose
> this is just a relic from the pre-self-hosting days, but it would be
> nice if these last .h files were gotten rid of in the near future.
>
> That, or we should (ab)use the .di generation facility or some sed
> script to generate these .h files, so that they won't go out-of-sync
> inadvertently.

we can use UDAs to do that! just make the generator import the relevand dmd parts, and then use compile-time introspection to build tables, so generator can just write 'em to file in runtime. ;-)

On Sat, Jun 10, 2017 at 02:27:13AM +0300, ketmar via Digitalmars-d wrote:
> H. S. Teoh wrote:
> 
> > On a tangential note, now that I finally got the miserable test suite running, I've managed to figure out that the problem in my dmd PR was caused by src/ddmd/globals.h not being auto-generated from src/ddmd/globals.d, so when I added new fields to the Global struct, the C++ code that use globals.h with the old definition crashed. I suppose this is just a relic from the pre-self-hosting days, but it would be nice if these last .h files were gotten rid of in the near future.
> > 
> > That, or we should (ab)use the .di generation facility or some sed script to generate these .h files, so that they won't go out-of-sync inadvertently.
> 
> we can use UDAs to do that! just make the generator import the relevand dmd parts, and then use compile-time introspection to build tables, so generator can just write 'em to file in runtime. ;-)

Sure!  But ... uhm... what has UDAs gotta do with that? A straight
`foreach (memb; __traits(allMembers, T))` should do the trick.


T

-- 
A mathematician is a device for turning coffee into theorems. -- P. Erdos

On Saturday, 10 June 2017 at 00:16:13 UTC, H. S. Teoh wrote:
> On Sat, Jun 10, 2017 at 02:27:13AM +0300, ketmar via Digitalmars-d wrote:
>> H. S. Teoh wrote:
>> 
>> > [...]
>> 
>> we can use UDAs to do that! just make the generator import the relevand dmd parts, and then use compile-time introspection to build tables, so generator can just write 'em to file in runtime. ;-)
>
> Sure!  But ... uhm... what has UDAs gotta do with that? A straight
> `foreach (memb; __traits(allMembers, T))` should do the trick.
>
>
> T

Are you aware of this excellent PR?

https://github.com/dlang/dmd/pull/5082

H. S. Teoh wrote:

> On Sat, Jun 10, 2017 at 02:27:13AM +0300, ketmar via Digitalmars-d wrote:
>> H. S. Teoh wrote:
>> 
>>> On a tangential note, now that I finally got the miserable test
>>> suite running, I've managed to figure out that the problem in my dmd
>>> PR was caused by src/ddmd/globals.h not being auto-generated from
>>> src/ddmd/globals.d, so when I added new fields to the Global struct,
>>> the C++ code that use globals.h with the old definition crashed. I
>>> suppose this is just a relic from the pre-self-hosting days, but it
>>> would be nice if these last .h files were gotten rid of in the near
>>> future.
>>> That, or we should (ab)use the .di generation facility or some sed
>>> script to generate these .h files, so that they won't go out-of-sync
>>> inadvertently.
>> we can use UDAs to do that! just make the generator import the
>> relevand dmd parts, and then use compile-time introspection to build
>> tables, so generator can just write 'em to file in runtime. ;-)
>
> Sure!  But ... uhm... what has UDAs gotta do with that? A straight
> `foreach (memb; __traits(allMembers, T))` should do the trick.

i thought about situations where you may need to exclude something from generated header... but this surely has no sense, as one cannot exclude fields or virtual methods from generated classes, so `extern(C++)` is surely enough. well, i like to overengineer! ;-)

On Sat, Jun 10, 2017 at 12:33:39AM +0000, Seb via Digitalmars-d wrote:
> On Saturday, 10 June 2017 at 00:16:13 UTC, H. S. Teoh wrote:
> > On Sat, Jun 10, 2017 at 02:27:13AM +0300, ketmar via Digitalmars-d wrote:
> > > H. S. Teoh wrote:
> > > 
> > > > [...]
> > > 
> > > we can use UDAs to do that! just make the generator import the relevand dmd parts, and then use compile-time introspection to build tables, so generator can just write 'em to file in runtime. ;-)
> > 
> > Sure!  But ... uhm... what has UDAs gotta do with that? A straight
> > `foreach (memb; __traits(allMembers, T))` should do the trick.
> > 
> > 
> > T
> 
> Are you aware of this excellent PR?
> 
> https://github.com/dlang/dmd/pull/5082

Whoa. This is exactly what I've been looking for!


T

-- 
The best way to destroy a cause is to defend it poorly.