Jump to page: 1 2
Thread overview
Linux, Mono-D, GtkD and linking
Aug 29, 2014
Bad Sector
Aug 30, 2014
Baz
Aug 30, 2014
Bad Sector
Aug 30, 2014
Baz
Aug 30, 2014
Orvid King
Aug 30, 2014
Bad Sector
Aug 31, 2014
Bad Sector
Aug 31, 2014
Orvid King
Sep 02, 2014
Bruno Medeiros
Sep 03, 2014
Bad Sector
Oct 12, 2014
olivier henley
Nov 19, 2014
Bruno Medeiros
Re: Linux, Mono-D, GtkD and linking (DDT performance bug)
Nov 28, 2014
Bruno Medeiros
Dec 12, 2014
Bruno Medeiros
August 29, 2014
Hi all,

I'm trying to evaluate D for my uses (games and game tools development) and see how it compares to my current languages (C for engine, Lazarus/FreePascal for tools at home and C++ for engine and tools at work). For now it is me trying to see how it fares and maybe switching to it for my own stuff at the beginning.

However i have already stumbled on a very fundamental issue: i cannot convince DMD to compile and link a GtkD program under MonoD.

I'm using Ubuntu so MonoD seems to be the best solution so far, even if the MonoDevelop environment feels very out of place under Ubuntu/Unity (i also tried DDT for Eclipse since i used Eclipse at the past, but DDT was freezing constantly and it didn't had good auto completion support).

I want to have a setup where i can have a few shared libraries (a core library, a rendering library, a utility library) and two applications (the editor and the engine which will work with data produced by the editor - note that unlike some popular engines, i prefer to keep these two as separate as possible), one with GC enabled (the editor) and one with GC disabled (engine). Both need to link against Derelict (for the OpenGL API and SDL2 for the engine) and the editor needs to link against GtkD (for its UI).

For starters i tried to use DUB, but i quickly found out that Mono-D tries to import all projects that the dependencies expose (so i had one project - my initial test - and 10+ projects i didn't care looking at). In addition no settings about the project layout seemed to be saved (i made an "External" solution folder to put all the GtkD and Derelict projects inside to not occupy visual space, but once i reopened the projec in MonoDevelop, the folder wasn't there). Finally just having all the projects there seemed to slow down MonoDevelop considerably. So i decided to drop DUB, at least for development purposes, and use Mono-D's building functionality.

Mono-D by itself provides much more organization options and feels mostly fine (i still dislike MonoDevelop forcing its own theme on me, but i can live with that). However i have several issues with its own builder.

First of all, i had to add all the DUB includes and libraries for GtkD and Derelict manually in the global includes and libraries settings in order for the syntax completion to discover them, despite having them being part of DUB (i expected since i have DUB installed and configured that Mono-D will see it automatically... or at least provide some configuration options to choose which of the installed DUB projects i want to use). Note that i added these in the global settings to avoid placing anything platform-specific in the project's own settings since i'll want to test and compile the project under Windows and OS X too.

However even with the libraries and includes configured, i had linker errors with "dlsym" and "dlerror" missing, which seem to come from the libdl. After googling for a while to find how to fix that, i managed to make it work by adding 'version(Posix) pragma (lib, "dl");' at the top of my test code.

This worked, but i quickly discovered that building and linking is *terribly slow*. One of the reasons i like C is that it has very fast compilation and compilation speed is important for my workflow. However even C++, which i generally avoid with one of the reasons being compilation speed, compiles much faster. Note that my computer only has SSD so I/O performance shouldn't be the case.

I noticed that Mono-D's settings have a single step compilation mode, which *may* have to do with this. I tried to disable it but then i got back the dlsym errors. I tried to put a -L-ldl in almost every possible place in the settings (both global and project) but nothing seemed to work. I cannot make the program link against the library. Also considering that my test program is still only a single file (i'm just trying out GtkD for now) and i read in an old post that pragma lib doesn't propagate to other files, it sounds that this wouldn't work with my intended setup anyway.

So at this point i'm not sure how to continue. TBH what i'd like is to be able to have Mono-D provide me with a nice UI to select DUB dependencies without fussing with library paths, linker flags, etc (unless i want to add some extra flags for optimization or debugging, of course) and use DUB behind the scenes by itself (and keep the DUB file around so i can build the project without having Mono-D installed in another D installation and/or operating system) but without polluting my project tree with external things i didn't explicitly add (i don't consider dependencies something i explicitly added, especially when said dependencies - like Derelict - add a ton of things i don't even plan using).

Is there any tips, suggestions or any other help for the above?
August 30, 2014
Just a question related to the compilation speed: are your libraries pre-build (I speak about *.a) or are the whole sources passed each time you compile a test program ?
August 30, 2014
Probably they weren't built. I was trying to fix the issue by trying several things and one of them was going to the package directories and typing `make all` to build them (in hopes that somehow this would generate some file that told mono-d or dmd the library's own dependencies). It seems that after the linking step became much faster (it seemed to be that the linking was the slow part).

Btw, my CPU provides 8 threads and compiling with `make -j 8` was *much* faster than just by `make` which makes me wonder if mono-d can also run simultaneous builds.

I also managed to find a workaround for the linking issue by adding the full path to the .so files directly to the project settings. Of course this makes the project harder to move to another system (even with the same OS), but it was the only thing that would work. I'd like to see something like Eclipse's "libraries" where you specify some alias for a project's library dependency (you can have multiple of them) and configure the aliases for each Eclipse installation. F.e. in GtkD's case you'd have a "gtkd" alias and Mono-D maps the gtkd alias to some include and library directories (this would also help Mono-D to know which includes exactly to parse).

In any case for now it seems to work, for development at least.
August 30, 2014
On 8/29/2014 5:32 AM, Bad Sector wrote:
> Hi all,
>
> I'm trying to evaluate D for my uses (games and game tools development)
> and see how it compares to my current languages (C for engine,
> Lazarus/FreePascal for tools at home and C++ for engine and tools at
> work). For now it is me trying to see how it fares and maybe switching
> to it for my own stuff at the beginning.
>
> However i have already stumbled on a very fundamental issue: i cannot
> convince DMD to compile and link a GtkD program under MonoD.
>
> I'm using Ubuntu so MonoD seems to be the best solution so far, even if
> the MonoDevelop environment feels very out of place under Ubuntu/Unity
> (i also tried DDT for Eclipse since i used Eclipse at the past, but DDT
> was freezing constantly and it didn't had good auto completion support).
>
> I want to have a setup where i can have a few shared libraries (a core
> library, a rendering library, a utility library) and two applications
> (the editor and the engine which will work with data produced by the
> editor - note that unlike some popular engines, i prefer to keep these
> two as separate as possible), one with GC enabled (the editor) and one
> with GC disabled (engine). Both need to link against Derelict (for the
> OpenGL API and SDL2 for the engine) and the editor needs to link against
> GtkD (for its UI).
>
> For starters i tried to use DUB, but i quickly found out that Mono-D
> tries to import all projects that the dependencies expose (so i had one
> project - my initial test - and 10+ projects i didn't care looking at).
> In addition no settings about the project layout seemed to be saved (i
> made an "External" solution folder to put all the GtkD and Derelict
> projects inside to not occupy visual space, but once i reopened the
> projec in MonoDevelop, the folder wasn't there). Finally just having all
> the projects there seemed to slow down MonoDevelop considerably. So i
> decided to drop DUB, at least for development purposes, and use Mono-D's
> building functionality.
>
> Mono-D by itself provides much more organization options and feels
> mostly fine (i still dislike MonoDevelop forcing its own theme on me,
> but i can live with that). However i have several issues with its own
> builder.
>
> First of all, i had to add all the DUB includes and libraries for GtkD
> and Derelict manually in the global includes and libraries settings in
> order for the syntax completion to discover them, despite having them
> being part of DUB (i expected since i have DUB installed and configured
> that Mono-D will see it automatically... or at least provide some
> configuration options to choose which of the installed DUB projects i
> want to use). Note that i added these in the global settings to avoid
> placing anything platform-specific in the project's own settings since
> i'll want to test and compile the project under Windows and OS X too.
>
> However even with the libraries and includes configured, i had linker
> errors with "dlsym" and "dlerror" missing, which seem to come from the
> libdl. After googling for a while to find how to fix that, i managed to
> make it work by adding 'version(Posix) pragma (lib, "dl");' at the top
> of my test code.
>
> This worked, but i quickly discovered that building and linking is
> *terribly slow*. One of the reasons i like C is that it has very fast
> compilation and compilation speed is important for my workflow. However
> even C++, which i generally avoid with one of the reasons being
> compilation speed, compiles much faster. Note that my computer only has
> SSD so I/O performance shouldn't be the case.
>
> I noticed that Mono-D's settings have a single step compilation mode,
> which *may* have to do with this. I tried to disable it but then i got
> back the dlsym errors. I tried to put a -L-ldl in almost every possible
> place in the settings (both global and project) but nothing seemed to
> work. I cannot make the program link against the library. Also
> considering that my test program is still only a single file (i'm just
> trying out GtkD for now) and i read in an old post that pragma lib
> doesn't propagate to other files, it sounds that this wouldn't work with
> my intended setup anyway.
>
> So at this point i'm not sure how to continue. TBH what i'd like is to
> be able to have Mono-D provide me with a nice UI to select DUB
> dependencies without fussing with library paths, linker flags, etc
> (unless i want to add some extra flags for optimization or debugging, of
> course) and use DUB behind the scenes by itself (and keep the DUB file
> around so i can build the project without having Mono-D installed in
> another D installation and/or operating system) but without polluting my
> project tree with external things i didn't explicitly add (i don't
> consider dependencies something i explicitly added, especially when said
> dependencies - like Derelict - add a ton of things i don't even plan
> using).
>
> Is there any tips, suggestions or any other help for the above?

It is possible to have a Mono-D solution file with a dub project in it, although it's not exactly a straight-forward process to get it into the solution in the first place. You have to open both solutions at the same time, and drag the dub project into the Mono-D solution and ignore any errors it gives you (if any), then you can close the dub solution.

In terms of compile-speed, as I hinted at on the IRC, compiling each file individually is usually going to be a lot slower than compiling them all at once.
August 30, 2014
On Saturday, 30 August 2014 at 09:36:01 UTC, Bad Sector wrote:
> Probably they weren't built. I was trying to fix the issue by trying several things and one of them was going to the package directories and typing `make all` to build them (in hopes that somehow this would generate some file that told mono-d or dmd the library's own dependencies). It seems that after the linking step became much faster (it seemed to be that the linking was the slow part).
>
> Btw, my CPU provides 8 threads and compiling with `make -j 8` was *much* faster than just by `make` which makes me wonder if mono-d can also run simultaneous builds.
>
> I also managed to find a workaround for the linking issue by adding the full path to the .so files directly to the project settings. Of course this makes the project harder to move to another system (even with the same OS), but it was the only thing that would work. I'd like to see something like Eclipse's "libraries" where you specify some alias for a project's library dependency (you can have multiple of them) and configure the aliases for each Eclipse installation. F.e. in GtkD's case you'd have a "gtkd" alias and Mono-D maps the gtkd alias to some include and library directories (this would also help Mono-D to know which includes exactly to parse).
>
> In any case for now it seems to work, for development at least.

Ok. My answer was related to my Coedit baby. If you are a bit ok with D libraries, then with Coedit it's simple: you put the source folder and the lib file in the manager and in your project you just have to use the alias...
https://github.com/BBasile/Coedit/wiki#library-manager-widget

August 30, 2014
On Saturday, 30 August 2014 at 15:11:29 UTC, Orvid King wrote:
> In terms of compile-speed, as I hinted at on the IRC, compiling each file individually is usually going to be a lot slower than compiling them all at once.

I'd expect that to be the case with a few files, but what about multiple files? Wouldn't compiling multiple files in parallel be better in terms of performance? Or does dmd do that already if you pass multiple files?

I have another problem now: i have set up one library project and one executable project that depends on the library (it is set so in the project's settings), but when i modify a source file in the library's code, save it and hit F5 (or F8), the executable isn't relinked (the library is rebuilt, but the executable's binary isn't considered as out of date so it isn't relinked - i assume Mono-D doesn't check the dependent library files?).

Is there something i'm missing or this is a bug? (i tried "link in static/shared libraries from nested dependencies" which sounded somewhat relevant but didn't do the trick and cannot find any other option).
August 31, 2014
Well, i give up, i spent too much time on this without any result. Mono-D doesn't seem to even be able to provide syntax highlighting for more than one `src<blah>` directory in the GtkD package and i had to combine them together just for it to work (that didn't fix the inability to link to a static library that uses GtkD).

It seems to me that stuff build better with DUB, feel better to work with Mono-D but the two are not integrated at all except a small nod from Mono-D that DUB exists :-/
August 31, 2014
On 8/30/2014 5:49 PM, Bad Sector wrote:
> On Saturday, 30 August 2014 at 15:11:29 UTC, Orvid King wrote:
>> In terms of compile-speed, as I hinted at on the IRC, compiling each
>> file individually is usually going to be a lot slower than compiling
>> them all at once.
>
> I'd expect that to be the case with a few files, but what about multiple
> files? Wouldn't compiling multiple files in parallel be better in terms
> of performance? Or does dmd do that already if you pass multiple files?
>
> I have another problem now: i have set up one library project and one
> executable project that depends on the library (it is set so in the
> project's settings), but when i modify a source file in the library's
> code, save it and hit F5 (or F8), the executable isn't relinked (the
> library is rebuilt, but the executable's binary isn't considered as out
> of date so it isn't relinked - i assume Mono-D doesn't check the
> dependent library files?).
>
> Is there something i'm missing or this is a bug? (i tried "link in
> static/shared libraries from nested dependencies" which sounded somewhat
> relevant but didn't do the trick and cannot find any other option).

I believe that's actually a bug, so I'd poke alex about it.
September 02, 2014
On 29/08/2014 11:32, Bad Sector wrote:
> (i also tried DDT for Eclipse since i used Eclipse at the past, but DDT
> was freezing constantly and it didn't had good auto completion support)

Indeed, DDT's auto complete support is currently not as good as Mono-D (I hope to improve that in the future).
However, I do not know of any issues that makes DDT freeze, if you have the time, I would welcome some bug reports with more info on these problems. (especially with the new DDT version)

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
September 03, 2014
On Tuesday, 2 September 2014 at 13:51:09 UTC, Bruno Medeiros wrote:
> However, I do not know of any issues that makes DDT freeze, if you have the time, I would welcome some bug reports with more info on these problems. (especially with the new DDT version)

Out of frustration i uninstalled everything D :-P but it was very easy to reproduce: Go to http://www.dsource.org/projects/gtkd/wiki/CodeExamples and grab some simple example, make a DUB json file for it and have GtkD and Derelict as dependencies. Then try to use auto completion - the IDE hangs for some time, probably trying to parse the code from GtkD and Derelict.

I may try DDT again with the new version since i think i tried it just before you made the announcement.
« First   ‹ Prev
1 2