View mode: basic / threaded / horizontal-split · Log in · Help
October 14, 2010
Re: Visual D Build + DMD Bugginess = Bad
On Wednesday 13 October 2010 23:58:16 Jonathan M Davis wrote:
> The bugs forum is not for posting to.

Ack. I meant the bug mailing list, not forum. Oh well, it should be fairly 
obvious what I meant.

- Jonathan M Davis
October 14, 2010
Re: Visual D Build + DMD Bugginess = Bad
Am 13.10.2010 21:45, schrieb dsimcha:
> I've noticed that, when passed multiple files at once, DMD is generally buggy
> in ways that I can't reproduce in small test cases.  This includes things like
> magically ignoring __gshared variables, and not being able to convert a type
> to an alias to the same type, for example, not being able to convert a size_t
> to a uint on 32-bit, or not being able to convert a float[] to an R where R is
> a template parameter instantiated as float[].
>
> I have two questions:
>
> 1.  Has anyone experienced similar things and if so are they further along
> than me at creating decent test cases?
>
> 2.  I'm used to using Code::Blocks, but want to switch to Visual D because it
> seems to work a lot better for the most part.  However, by default it seems to
> pass all project files to the compiler at once, where Code::Blocks compiles
> one file at a time.  This means I have to deal with DMD's bugginess when
> passed mutliple files at once.  Does anyone know how to make Visual D work
> Code::Blocks style, i.e. compile each file to an object file and link them
> afterwords?

Right now, I'm in exactly the same situation. However, the problems 
might be a bit more obvious: Sometimes symbols are missing at link time 
that should not and sometimes some symbols (most of the time phobos 
symbols) are defined twice but with exactly the same module path.

(Error: multiple definition of conv_208_199f: 
_D53C:\D\dmd2\windows\bin\..\..\src\phobos\std\conv.d.52012__ModuleInfoZ 
and conv: 
_D53C:\D\dmd2\windows\bin\..\..\src\phobos\std\conv.d.52012__ModuleInfoZ)

On Mac OS, using all-at-once compilation I get defective .lib files (the 
ar tool complains that the .o file is smaller than it reports or 
something similar) so I have to compile the code of all static libs 
together with the application code.

The problem is that it is really difficult (almost impossible) to 
isolate the problem in any way because sometimes appears erratically and 
the code base is quite large. And also I cannot share the source code 
for debugging.
October 14, 2010
Re: Visual D Build + DMD Bugginess = Bad
I'll push the issue up the todo list. Maybe I'm not getting a lot of 
these errors because I have applied the dmd patches in the bug reports 
190 and 4753...


dsimcha wrote:
> == Quote from Rainer Schuetze (r.sagitario@gmx.de)'s article
>> Could it be related to any of these?
>> http://d.puremagic.com/issues/show_bug.cgi?id=190
>> http://d.puremagic.com/issues/show_bug.cgi?id=3979
> 
> Yep, these look like at least part of the problem.  Glad someone else has already
> reduced them to sane test cases so I don't have to.  Reducing compiler bugs that
> only seem to occur in non-trivial, multi-module projects to decent test cases is a
> huge PITA.
> 
> Anyhow, the project in question is a very messy codebase because it grew very
> organically.  It's basically a haphazard collection of research prototype
> algorithms for predicting gene expression from DNA sequence, and every time I
> think of a new idea, I tend to just put it wherever I can make it fit and almost
> never bother with non-trivial refactoring.  Therefore, there are cyclic imports
> **everywhere**.
> 
>> I have that option on my todo list, but didn't implement it so far
>> because I was not aware of any problems with compiling multiple files.
>> I don't know if this is really an option for a larger project, but what
>> you can do is select "Custom Build Tool" for all files in the "Common
>> Properties" setting of the project property dialog and enter your own
>> command line and output file. But be warned: you will not get any
>> automatic dependency detection.
>> As editing the settings for a lot of files might be very annoying, you
>> might want to set it for one file and then edit the project file with a
>> text editor to copy the settings to other files.
>> Rainer
> 
> Since build process automation is by far the biggest reason why I use an IDE
> instead of a plain old editor, I'd rather just stick with Code::Blocks for now.
> Could you please bump this up the todo list, given that building multiple files
> simultaneously is buggy in ways that probably aren't going to get fixed too soon?
October 17, 2010
Re: Visual D Build + DMD Bugginess = Bad
Here's a preliminary new version that includes an option on the general 
project configuration page to switch to single file compilation:

http://www.dsource.org/projects/visuald/browser/downloads/VisualD-v0.3.18rc1.exe

I've used ddmd as a medium sized project for testing. It has more than 
400 files with most modules mutually importing each other. Some remarks:

- compilation speed is a lot worse, even when compared to what you would 
expect from a similar C/C++ project. It takes almost 8 minutes to 
compile ddmd on my system with single file compilation, while the 
standard compilation takes about 10 seconds.

- the dependency files written by dmd are so bloated that the check that 
selects the files to compile takes almost 20 seconds. This is because 
the size of each dependency file is about 700 kB, summing up to 220 MB 
to scan for imported files. (I had to do some optimizations and file 
operation caching to get the time down from several minutes).

- I consider the single compilation mode only sensible for a large 
collection of modules that have only limited dependencies.

- probably, compilation speed could be way better if semantic analysis 
would only be done on symbols that are actually directly or indirectly 
referred to by the modules passed on the command line. This would need a 
slight modification of the semantic analysis, but would also put the 
promise of independence of lexical order to the test - and would expose 
more of the problems you are currently getting.

This new version also includes some experimental debug/version statement 
highlighting, which has not yet been extensively tested. If it causes 
too many troubles, it can be disabled on the global property page 
Tools->Options->TextEditor->D->Colorizer.

Rainer


dsimcha wrote:
> == Quote from Rainer Schuetze (r.sagitario@gmx.de)'s article
>> Could it be related to any of these?
>> http://d.puremagic.com/issues/show_bug.cgi?id=190
>> http://d.puremagic.com/issues/show_bug.cgi?id=3979
> 
> Yep, these look like at least part of the problem.  Glad someone else has already
> reduced them to sane test cases so I don't have to.  Reducing compiler bugs that
> only seem to occur in non-trivial, multi-module projects to decent test cases is a
> huge PITA.
> 
> Anyhow, the project in question is a very messy codebase because it grew very
> organically.  It's basically a haphazard collection of research prototype
> algorithms for predicting gene expression from DNA sequence, and every time I
> think of a new idea, I tend to just put it wherever I can make it fit and almost
> never bother with non-trivial refactoring.  Therefore, there are cyclic imports
> **everywhere**.
> 
>> I have that option on my todo list, but didn't implement it so far
>> because I was not aware of any problems with compiling multiple files.
>> I don't know if this is really an option for a larger project, but what
>> you can do is select "Custom Build Tool" for all files in the "Common
>> Properties" setting of the project property dialog and enter your own
>> command line and output file. But be warned: you will not get any
>> automatic dependency detection.
>> As editing the settings for a lot of files might be very annoying, you
>> might want to set it for one file and then edit the project file with a
>> text editor to copy the settings to other files.
>> Rainer
> 
> Since build process automation is by far the biggest reason why I use an IDE
> instead of a plain old editor, I'd rather just stick with Code::Blocks for now.
> Could you please bump this up the todo list, given that building multiple files
> simultaneously is buggy in ways that probably aren't going to get fixed too soon?
October 17, 2010
Re: Visual D Build + DMD Bugginess = Bad
Cool stuff, Rainer. Btw, I've noticed there's a new debugger installed
with newer versions of VisualD (mago?). Do I need to set it up to be
used in VS over the default one or is it selected by default after
installation?

On 10/17/10, Rainer Schuetze <r.sagitario@gmx.de> wrote:
> Here's a preliminary new version that includes an option on the general
> project configuration page to switch to single file compilation:
>
> http://www.dsource.org/projects/visuald/browser/downloads/VisualD-v0.3.18rc1.exe
>
> I've used ddmd as a medium sized project for testing. It has more than
> 400 files with most modules mutually importing each other. Some remarks:
>
> - compilation speed is a lot worse, even when compared to what you would
> expect from a similar C/C++ project. It takes almost 8 minutes to
> compile ddmd on my system with single file compilation, while the
> standard compilation takes about 10 seconds.
>
> - the dependency files written by dmd are so bloated that the check that
> selects the files to compile takes almost 20 seconds. This is because
> the size of each dependency file is about 700 kB, summing up to 220 MB
> to scan for imported files. (I had to do some optimizations and file
> operation caching to get the time down from several minutes).
>
> - I consider the single compilation mode only sensible for a large
> collection of modules that have only limited dependencies.
>
> - probably, compilation speed could be way better if semantic analysis
> would only be done on symbols that are actually directly or indirectly
> referred to by the modules passed on the command line. This would need a
> slight modification of the semantic analysis, but would also put the
> promise of independence of lexical order to the test - and would expose
> more of the problems you are currently getting.
>
> This new version also includes some experimental debug/version statement
> highlighting, which has not yet been extensively tested. If it causes
> too many troubles, it can be disabled on the global property page
> Tools->Options->TextEditor->D->Colorizer.
>
> Rainer
>
>
> dsimcha wrote:
>> == Quote from Rainer Schuetze (r.sagitario@gmx.de)'s article
>>> Could it be related to any of these?
>>> http://d.puremagic.com/issues/show_bug.cgi?id=190
>>> http://d.puremagic.com/issues/show_bug.cgi?id=3979
>>
>> Yep, these look like at least part of the problem.  Glad someone else has
>> already
>> reduced them to sane test cases so I don't have to.  Reducing compiler
>> bugs that
>> only seem to occur in non-trivial, multi-module projects to decent test
>> cases is a
>> huge PITA.
>>
>> Anyhow, the project in question is a very messy codebase because it grew
>> very
>> organically.  It's basically a haphazard collection of research prototype
>> algorithms for predicting gene expression from DNA sequence, and every
>> time I
>> think of a new idea, I tend to just put it wherever I can make it fit and
>> almost
>> never bother with non-trivial refactoring.  Therefore, there are cyclic
>> imports
>> **everywhere**.
>>
>>> I have that option on my todo list, but didn't implement it so far
>>> because I was not aware of any problems with compiling multiple files.
>>> I don't know if this is really an option for a larger project, but what
>>> you can do is select "Custom Build Tool" for all files in the "Common
>>> Properties" setting of the project property dialog and enter your own
>>> command line and output file. But be warned: you will not get any
>>> automatic dependency detection.
>>> As editing the settings for a lot of files might be very annoying, you
>>> might want to set it for one file and then edit the project file with a
>>> text editor to copy the settings to other files.
>>> Rainer
>>
>> Since build process automation is by far the biggest reason why I use an
>> IDE
>> instead of a plain old editor, I'd rather just stick with Code::Blocks for
>> now.
>> Could you please bump this up the todo list, given that building multiple
>> files
>> simultaneously is buggy in ways that probably aren't going to get fixed
>> too soon?
>
October 17, 2010
Re: Visual D Build + DMD Bugginess = Bad
Awesome.  I frankly don't care about compile speeds because the projects in
question are fairly small, so even with a horribly inefficient build process build
times are negligible.  FWIW I use the one at a time builds in CodeBlocks, too,
though I parallelize them on my quad core.

I only wanted the one at a time compilation feature as a temporary workaround for
compiler bugs that will hopefully be fixed by the time people start using D2 for
larger projects.  I'll try Visual D again tomorrow and provide some feedback.
From my limited playing around with it, I've already noticed that the debugger
works a ton better than any other D debugger I've tried, i.e. possibly good enough
that I'll stop using writeln as my main debugging method.

== Quote from Rainer Schuetze (r.sagitario@gmx.de)'s article
> Here's a preliminary new version that includes an option on the general
> project configuration page to switch to single file compilation:
> http://www.dsource.org/projects/visuald/browser/downloads/VisualD-v0.3.18rc1.exe
> I've used ddmd as a medium sized project for testing. It has more than
> 400 files with most modules mutually importing each other. Some remarks:
> - compilation speed is a lot worse, even when compared to what you would
> expect from a similar C/C++ project. It takes almost 8 minutes to
> compile ddmd on my system with single file compilation, while the
> standard compilation takes about 10 seconds.
> - the dependency files written by dmd are so bloated that the check that
> selects the files to compile takes almost 20 seconds. This is because
> the size of each dependency file is about 700 kB, summing up to 220 MB
> to scan for imported files. (I had to do some optimizations and file
> operation caching to get the time down from several minutes).
> - I consider the single compilation mode only sensible for a large
> collection of modules that have only limited dependencies.
> - probably, compilation speed could be way better if semantic analysis
> would only be done on symbols that are actually directly or indirectly
> referred to by the modules passed on the command line. This would need a
> slight modification of the semantic analysis, but would also put the
> promise of independence of lexical order to the test - and would expose
> more of the problems you are currently getting.
> This new version also includes some experimental debug/version statement
> highlighting, which has not yet been extensively tested. If it causes
> too many troubles, it can be disabled on the global property page
> Tools->Options->TextEditor->D->Colorizer.
> Rainer
> dsimcha wrote:
> > == Quote from Rainer Schuetze (r.sagitario@gmx.de)'s article
> >> Could it be related to any of these?
> >> http://d.puremagic.com/issues/show_bug.cgi?id=190
> >> http://d.puremagic.com/issues/show_bug.cgi?id=3979
> >
> > Yep, these look like at least part of the problem.  Glad someone else has already
> > reduced them to sane test cases so I don't have to.  Reducing compiler bugs that
> > only seem to occur in non-trivial, multi-module projects to decent test cases is a
> > huge PITA.
> >
> > Anyhow, the project in question is a very messy codebase because it grew very
> > organically.  It's basically a haphazard collection of research prototype
> > algorithms for predicting gene expression from DNA sequence, and every time I
> > think of a new idea, I tend to just put it wherever I can make it fit and almost
> > never bother with non-trivial refactoring.  Therefore, there are cyclic imports
> > **everywhere**.
> >
> >> I have that option on my todo list, but didn't implement it so far
> >> because I was not aware of any problems with compiling multiple files.
> >> I don't know if this is really an option for a larger project, but what
> >> you can do is select "Custom Build Tool" for all files in the "Common
> >> Properties" setting of the project property dialog and enter your own
> >> command line and output file. But be warned: you will not get any
> >> automatic dependency detection.
> >> As editing the settings for a lot of files might be very annoying, you
> >> might want to set it for one file and then edit the project file with a
> >> text editor to copy the settings to other files.
> >> Rainer
> >
> > Since build process automation is by far the biggest reason why I use an IDE
> > instead of a plain old editor, I'd rather just stick with Code::Blocks for now.
> > Could you please bump this up the todo list, given that building multiple files
> > simultaneously is buggy in ways that probably aren't going to get fixed too soon?
October 17, 2010
Re: Visual D Build + DMD Bugginess = Bad
Andrej Mitrovic wrote:
> Cool stuff, Rainer. Btw, I've noticed there's a new debugger installed
> with newer versions of VisualD (mago?). Do I need to set it up to be
> used in VS over the default one or is it selected by default after
> installation?
> 

If you did not uncheck the mago debugger when installing, you can simply 
select the debug engine to be used on the project property page 
"Debugging" (visual studio internal debugger or mago).
October 17, 2010
Re: Visual D Build + DMD Bugginess = Bad
I've just tried it. It doesn't seem to be able to find any locals,
even if I add a variable to a watchlist it can't find the symbol. I
guess this is still experimental. Anyway, thanks for the info.

On 10/17/10, Rainer Schuetze <r.sagitario@gmx.de> wrote:
>
> Andrej Mitrovic wrote:
>> Cool stuff, Rainer. Btw, I've noticed there's a new debugger installed
>> with newer versions of VisualD (mago?). Do I need to set it up to be
>> used in VS over the default one or is it selected by default after
>> installation?
>>
>
> If you did not uncheck the mago debugger when installing, you can simply
> select the debug engine to be used on the project property page
> "Debugging" (visual studio internal debugger or mago).
>
October 17, 2010
Re: Visual D Build + DMD Bugginess = Bad
That's an unfortunate regression in the linker that comes with dmd 
2.049. There's already an updated linker available 
(http://ftp.digitalmars.com/link.8.00.8.zip). See also the project page 
http://dsource.org/projects/mago_debugger for more info.

Andrej Mitrovic wrote:
> I've just tried it. It doesn't seem to be able to find any locals,
> even if I add a variable to a watchlist it can't find the symbol. I
> guess this is still experimental. Anyway, thanks for the info.
> 
> On 10/17/10, Rainer Schuetze <r.sagitario@gmx.de> wrote:
>> Andrej Mitrovic wrote:
>>> Cool stuff, Rainer. Btw, I've noticed there's a new debugger installed
>>> with newer versions of VisualD (mago?). Do I need to set it up to be
>>> used in VS over the default one or is it selected by default after
>>> installation?
>>>
>> If you did not uncheck the mago debugger when installing, you can simply
>> select the debug engine to be used on the project property page
>> "Debugging" (visual studio internal debugger or mago).
>>
October 17, 2010
Re: Visual D Build + DMD Bugginess = Bad
Ah, ok. I'll have a look, thanks.

On 10/17/10, Rainer Schuetze <r.sagitario@gmx.de> wrote:
> That's an unfortunate regression in the linker that comes with dmd
> 2.049. There's already an updated linker available
> (http://ftp.digitalmars.com/link.8.00.8.zip). See also the project page
> http://dsource.org/projects/mago_debugger for more info.
>
> Andrej Mitrovic wrote:
>> I've just tried it. It doesn't seem to be able to find any locals,
>> even if I add a variable to a watchlist it can't find the symbol. I
>> guess this is still experimental. Anyway, thanks for the info.
>>
>> On 10/17/10, Rainer Schuetze <r.sagitario@gmx.de> wrote:
>>> Andrej Mitrovic wrote:
>>>> Cool stuff, Rainer. Btw, I've noticed there's a new debugger installed
>>>> with newer versions of VisualD (mago?). Do I need to set it up to be
>>>> used in VS over the default one or is it selected by default after
>>>> installation?
>>>>
>>> If you did not uncheck the mago debugger when installing, you can simply
>>> select the debug engine to be used on the project property page
>>> "Debugging" (visual studio internal debugger or mago).
>>>
>
1 2 3
Top | Discussion index | About this forum | D home