Hi,

I would like to generate gdc compiler on msys2/mingw-w64 platform.
The build system on msys2 uses archlinux build architecture and the build file is here:
https://github.com/vrichomme/MINGW-packages/blob/newfastrelease/mingw-w64-gcc/PKGBUILD

To generate it I followed the following steps:

1) Install https://msys2.github.io/
2)

$ update-core
$ pacman -Su
$ git clone https://github.com/vrichomme/MINGW-packages.git
$ cd MINGW-packages
$ git fetch origin
$ git checkout -b newfastrelease origin/newfastrelease
$ cd mingw-w64-gcc
$ export MINGW_INSTALLS=mingw64
$ makepkg-mingw -sLf

But of course I get an error because the platform is not supported:

/home/Vincent/tmp/MINGW-packages/mingw-w64-gcc/src/build-x86_64-w64-mingw32/./gcc/gdc -B/home/Vincent/tmp/MINGW-packages/mingw-w64-gcc/src/build-x86_64-w64-mingw32/./gcc/ -L/mingw64/x86_64-w64-mingw32/lib -L/mingw64/lib -isystem /mingw64/x86_64-w64-mingw32/include -isystem /mingw64/include -B/mingw64/x86_64-w64-mingw32/bin/ -B/mingw64/x86_64-w64-mingw32/lib/ -isystem /mingw64/x86_64-w64-mingw32/include -isystem /mingw64/x86_64-w64-mingw32/sys-include    -o gcc/gthreads/posix.o -Wall  -g -frelease -O2 -nostdinc -pipe -Wno-deprecated -I ../../../../gcc-5.3.0/libphobos/libdruntime -I ./x86_64-w64-mingw32 -I .  -c ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d
../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:29:1: error: undefined identifier pthread_key_t, did you mean alias gthread_key_t?
 alias gthread_key_t   = pthread_key_t;
 ^
../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:30:1: error: undefined identifier pthread_once_t, did you mean alias gthread_once_t?
 alias gthread_once_t  = pthread_once_t;
 ^
../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:31:1: error: undefined identifier pthread_mutex_t, did you mean alias gthread_mutex_t?
 alias gthread_mutex_t = pthread_mutex_t;
 ^
../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:32:1: error: undefined identifier pthread_mutex_t, did you mean alias gthread_mutex_t?
 alias gthread_recursive_mutex_t = pthread_mutex_t;
 ^
../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:34:27: error: undefined identifier PTHREAD_MUTEX_INITIALIZER
 enum GTHREAD_MUTEX_INIT = PTHREAD_MUTEX_INITIALIZER;
                           ^
../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:35:27: error: undefined identifier PTHREAD_ONCE_INIT, did you mean variable GTHREAD_ONCE_INIT?
 enum GTHREAD_ONCE_INIT  = PTHREAD_ONCE_INIT;
                           ^
../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:36:37: error: undefined identifier PTHREAD_MUTEX_INITIALIZER
 enum GTHREAD_RECURSIVE_MUTEX_INIT = PTHREAD_MUTEX_INITIALIZER;

My question is how does Version is defined when compiling core.sys.posix.pthread ?
Because I can see some block corresponding to each platform :

version( linux )
{
...
}
else version( OSX )
{
...
}

But how is defined this value ? Is it based on uname function ?
Can I trace the value of version when compiling the file ?

Thanks

On 24 March 2016 at 10:52, Vincent R via D.gnu <d.gnu@puremagic.com> wrote:

> Hi,
>
> I would like to generate gdc compiler on msys2/mingw-w64 platform.
> The build system on msys2 uses archlinux build architecture and the build
> file is here:
>
> https://github.com/vrichomme/MINGW-packages/blob/newfastrelease/mingw-w64-gcc/PKGBUILD
>
> To generate it I followed the following steps:
>
> 1) Install https://msys2.github.io/
> 2)
>
> $ update-core
> $ pacman -Su
> $ git clone https://github.com/vrichomme/MINGW-packages.git
> $ cd MINGW-packages
> $ git fetch origin
> $ git checkout -b newfastrelease origin/newfastrelease
> $ cd mingw-w64-gcc
> $ export MINGW_INSTALLS=mingw64
> $ makepkg-mingw -sLf
>
> But of course I get an error because the platform is not supported:
>
>
>
How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.


>
> But how is defined this value ? Is it based on uname function ? Can I trace the value of version when compiling the file ?
>
>
What versions that are internally defined depend on the target triplet passed when configuring the build.  This directly affects what target headers are included, which contain macros we call when initializing the compiler.

It has been requested even in DMD to have some command-line option to print all turned on version conditions set by the compiler.  Itself is a very trivial addition.  In the absence of that though, in a debugger you can set a breakpoint on VersionCondition::addPredefinedGlobalIdent.

Iain.

On Thursday, 24 March 2016 at 10:12:20 UTC, Iain Buclaw wrote:
> How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.
>
>
>>
>> But how is defined this value ? Is it based on uname function ? Can I trace the value of version when compiling the file ?
>>
>>
> What versions that are internally defined depend on the target triplet passed when configuring the build.  This directly affects what target headers are included, which contain macros we call when initializing the compiler.
>
> It has been requested even in DMD to have some command-line option to print all turned on version conditions set by the compiler.  Itself is a very trivial addition.  In the absence of that though, in a debugger you can set a breakpoint on VersionCondition::addPredefinedGlobalIdent.
>
> Iain.


> How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.

winpthread is defined here:
https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-libraries/winpthreads/include/
So winpthread is just a normal pthread.h include file and mingw compiler include a corresponding library.


Thanks for the information about --enable-threads=win32 and that's what I used to do but now I don't want to use that kind of trick and I would like to generate a posix version because as long as it doesn't support it, it will never be considered as a valid package from msys2/mingw-w64 maintainers.

Am Thu, 24 Mar 2016 11:12:20 +0100
schrieb "Iain Buclaw via D.gnu" <d.gnu@puremagic.com>:

> On 24 March 2016 at 10:52, Vincent R via D.gnu <d.gnu@puremagic.com> wrote:
> 
> > Hi,
> >
> > I would like to generate gdc compiler on msys2/mingw-w64 platform. The build system on msys2 uses archlinux build architecture and the build file is here:
> >
> > https://github.com/vrichomme/MINGW-packages/blob/newfastrelease/mingw-w64-gcc/PKGBUILD
> > [...]
> > 
> How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.

It supports most of the pthreads API IIRC but there are sometimes complaints about performance of the wrapper code. It basically only wraps windows primitives to the pthread API. It's even legal to mix winpthreads and windows API code.

Some new C++11 (or 14) features are only supported by libstdc++ with the pthread API. This is the main reason why MinGW compilers use the winpthread code, nobody has written a native std::thread implementation for MinGW yet.

However, I think for D the best solution is to always use the windows API. Upstream druntime takes care of this anyway so this means less work for us.

> 
> >
> > But how is defined this value ? Is it based on uname function ? Can I trace the value of version when compiling the file ?
> >
> > 
> What versions that are internally defined depend on the target triplet passed when configuring the build.  This directly affects what target headers are included, which contain macros we call when initializing the compiler.
> 

And the details are kinda complicated. In this specific case though, the interesting part is this:

* --enable-threads=X sets the GNU_Thread_Model=X variable in gcc.config
* For windows / posix, we use version(MinGW) or
  version(Windows) / version(Posix).


So in this case you end up with the combinations:
version (Windows)
{
    import gcc.config;
    static if(GNU_Thread_Model == ThreadModel.Posix)
       //winpthreads
    else static if(GNU_Thread_Model == ThreadModel.Win32)
       //native
}

version(Posix) is not set and should not be set!




I also had another look at MinGW-W64 some weeks ago but then got distracted :-)

Here's a quick patch to get winpthreads working: https://paste.gnome.org/pl3pqzcke


This only hooks up the gthreads API with winpthreads. But when implementing this quick workaround, I realized there are two open questions:

* We could probably use the standard windows gthread implementation,
  even if the GCC model is set to pthreads. AFAICS the only place we
  use gthreads is in rt.monitor for the class monitors and it doesn't
  matter if we use the windows API directly or the wrapper. We could
  probably even merge gcc/gthreads into the rt module, much of
  gcc.gthreads isn't even used anymore.
* This patch doesn't expose the fact that a pthread emulation layer is
  available to user code (i.e. you can't use the core.sys.posix.*
  pthread modules). I don't think we need to expose this as we have
  cross-platform wrappers in core.thread/core.sync anyway.

So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?

On Thursday, 24 March 2016 at 13:07:00 UTC, Johannes Pfau wrote:
> I also had another look at MinGW-W64 some weeks ago but then got distracted :-)
>
> Here's a quick patch to get winpthreads working: https://paste.gnome.org/pl3pqzcke
>
>
> This only hooks up the gthreads API with winpthreads. But when implementing this quick workaround, I realized there are two open questions:
>
> * We could probably use the standard windows gthread implementation,
>   even if the GCC model is set to pthreads. AFAICS the only place we
>   use gthreads is in rt.monitor for the class monitors and it doesn't
>   matter if we use the windows API directly or the wrapper. We could
>   probably even merge gcc/gthreads into the rt module, much of
>   gcc.gthreads isn't even used anymore.
> * This patch doesn't expose the fact that a pthread emulation layer is
>   available to user code (i.e. you can't use the core.sys.posix.*
>   pthread modules). I don't think we need to expose this as we have
>   cross-platform wrappers in core.thread/core.sync anyway.
>
> So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?

Thanks for all this information and it seems your knowledge is far beyond mine so I cannot answer your last question but from what you explain maybe it could be a better option.
What I can tel you for now is your patch works and allowed me to generate a gdc compilier.
Now I let you decide what is the best option.

Am Fri, 25 Mar 2016 09:15:06 +0000
schrieb Vincent R <lol@dlang.org>:

> > So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?
> 
> Thanks for all this information and it seems your knowledge is
> far beyond mine so I cannot answer your last question but from
> what you explain maybe it could be a better option.
> What I can tel you for now is your patch works and allowed me to
> generate a gdc compilier.
> Now I let you decide what is the best option.

Great! We've simply removed gthreads now: https://github.com/D-Programming-GDC/GDC/commit/096991b5817ac21ff6c81ea3759ad4a253deff59

It might take a few days till this is merged into the gdc-X branches though.


BTW: When I tested MinGW I found a serious garbage collector bug: TLS variables are currently not scanned correctly. This means if you have GC objects only referenced from thread local memory these might get collected.

I know how to fix this but it's not simple so it will take some time. If you run into this bug you could use DMD or LDC instead (IIRC you ran into an optlink bug when using DMD, but you could probably use the -mscoff32 or m64 options to avoid optlink).

On Friday, 25 March 2016 at 10:11:46 UTC, Johannes Pfau wrote:
> Am Fri, 25 Mar 2016 09:15:06 +0000
> schrieb Vincent R <lol@dlang.org>:
>
>> > So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?
>> 
>> Thanks for all this information and it seems your knowledge is
>> far beyond mine so I cannot answer your last question but from
>> what you explain maybe it could be a better option.
>> What I can tel you for now is your patch works and allowed me to
>> generate a gdc compilier.
>> Now I let you decide what is the best option.
>
> Great! We've simply removed gthreads now: https://github.com/D-Programming-GDC/GDC/commit/096991b5817ac21ff6c81ea3759ad4a253deff59
>
> It might take a few days till this is merged into the gdc-X branches though.
>
>
> BTW: When I tested MinGW I found a serious garbage collector bug: TLS variables are currently not scanned correctly. This means if you have GC objects only referenced from thread local memory these might get collected.
>
> I know how to fix this but it's not simple so it will take some time. If you run into this bug you could use DMD or LDC instead (IIRC you ran into an optlink bug when using DMD, but you could probably use the -mscoff32 or m64 options to avoid optlink).

I have some linker errors when I try to use the gdc compiler :

Start a mingw64 shell, remove wxwidgets3 if installed, generate wxwidgets28 and installs it:

$ pacman -R wxwidgets
$ export MINGW_INSTALLS=mingw64
$ git clone https://github.com/vrichomme/MINGW-packages.git
$ cd mingw-w64-wxwidgets28
$ makepkg-mingw -sLf
$ pacman -U mingw-w64-x86_64-wxWidgets-2.8.12-1-any.pkg.tar.xz

$ git clone https://github.com/vrichomme/wxd.git
$ cd src
$ make
$ $ cd Samples/
$ make
make -C Controls
make[1]: Entering directory '/home/19013692/tmp/wxd/src/Samples/Controls'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/19013692/tmp/wxd/src/Samples/Controls'
make -C Dialogs
make[1]: Entering directory '/home/19013692/tmp/wxd/src/Samples/Dialogs'
gdc -o ../../bin/Dialogs Dialogs.o -L../.. -lwxd  -lwxc `wx-config --libs` -lstdc++ -lws2_32 -lwsock32
C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o): In function `_D3std6socket8Protocol17getProtocolByNameMFxAaZb':
C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/src/std/socket.d:461: undefined reference to `getprotobyname'
C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o): In function `_D3std6socket8Protocol17getProtocolByTypeMFE3std6socket12ProtocolTypeZb':
C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/src/std/socket.d:474: undefined reference to `getprotobynumber'
...

As you can see all the missing symbols are from the socket library and that's why I have added locally -lws2_32 -lwsock32 to the config.gcc but it doesn't solve the problem...

Am Wed, 30 Mar 2016 08:27:27 +0000
schrieb Vincent R <lol@dlang.org>:

> make[1]: Entering directory
> '/home/19013692/tmp/wxd/src/Samples/Controls'
> make[1]: Nothing to be done for 'all'.
> make[1]: Leaving directory
> '/home/19013692/tmp/wxd/src/Samples/Controls'
> make -C Dialogs
> make[1]: Entering directory
> '/home/19013692/tmp/wxd/src/Samples/Dialogs'
> gdc -o ../../bin/Dialogs Dialogs.o -L../.. -lwxd  -lwxc
> `wx-config --libs` -lstdc++ -lws2_32 -lwsock32
> C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o):
> In function `_D3std6socket8Protocol17getProtocolByNameMFxAaZb':
> C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/src/std/socket.d:461:
> undefined reference to `getprotobyname'
> C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o):
> In function
> `_D3std6socket8Protocol17getProtocolByTypeMFE3std6socket12ProtocolTypeZb':
> C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/src/std/socket.d:474:
> undefined reference to `getprotobynumber' ...
> 
> As you can see all the missing symbols are from the socket library and that's why I have added locally -lws2_32 -lwsock32 to the config.gcc but it doesn't solve the problem...
> 
> 

Known problem, I'll probably push a fix to master for this soon. Linking -lws2_32 -lwsock32 is the correct workaround, but the libraries must be specified in the correct order regarding libphobos IIRC.

Try
gdc -nophoboslib -lws2_32 -lwsock32 -lgphobos2
or
gdc -nophoboslib -lgphobos2 -lws2_32 -lwsock32

One of these should work ;-)

On Wednesday, 30 March 2016 at 11:44:41 UTC, Johannes Pfau wrote:
> Am Wed, 30 Mar 2016 08:27:27 +0000
> schrieb Vincent R <lol@dlang.org>:
>
>> [...]
>
> Known problem, I'll probably push a fix to master for this soon. Linking -lws2_32 -lwsock32 is the correct workaround, but the libraries must be specified in the correct order regarding libphobos IIRC.
>
> Try
> gdc -nophoboslib -lws2_32 -lwsock32 -lgphobos2
> or
> gdc -nophoboslib -lgphobos2 -lws2_32 -lwsock32
>
> One of these should work ;-)

Thanks the second option works.

On Wednesday, 30 March 2016 at 12:05:27 UTC, Vincent R wrote:
> On Wednesday, 30 March 2016 at 11:44:41 UTC, Johannes Pfau wrote:
>> Am Wed, 30 Mar 2016 08:27:27 +0000
>> schrieb Vincent R <lol@dlang.org>:
>>
>>> [...]
>>
>> Known problem, I'll probably push a fix to master for this soon. Linking -lws2_32 -lwsock32 is the correct workaround, but the libraries must be specified in the correct order regarding libphobos IIRC.
>>
>> Try
>> gdc -nophoboslib -lws2_32 -lwsock32 -lgphobos2
>> or
>> gdc -nophoboslib -lgphobos2 -lws2_32 -lwsock32
>>
>> One of these should work ;-)
>
> Thanks the second option works.

Is there any progress for mingw ? Have you fixed what you wanted to about tls , link order, ... ?
I am asking because as long as it's not done it cannot be push upstream and cannot be tested.