On Friday, 23 November 2018 at 08:57:57 UTC, Chris Katko wrote:
> D is supposed to compile fast.

You didnt read the fine print. It compiles simple code fast. Also compilation is separate step from linking and your program might spend half of "compilation" time in link phase.

On Saturday, 24 November 2018 at 20:44:57 UTC, welkam wrote:
> On Friday, 23 November 2018 at 08:57:57 UTC, Chris Katko wrote:
>> D is supposed to compile fast.
>
> You didnt read the fine print. It compiles simple code fast. Also compilation is separate step from linking and your program might spend half of "compilation" time in link phase.

Wait wait wait wait wait.

So 1) I have to compile manually, then link. Except that also runs the files every time even if they're up-to-date. Is that normal behavior for C/C++?

Two questions/topics/issues:

-------------------------------------------------------

#1 How to I only build files that are dirty? Do I actually need a build program like DUB, MAKE, or CMAKE to do that? (Can make, cmake be used?) How do they recognize files are out-dated if DMD can't? Is that just an industry-standard specialization/separation-of-responsibilities to not have the compiler auto-detect up-to-date builds?



I have the simplest project ever. Less than 10 files and my non-VStudio build-scripts have always been simple. A few lines or one long line running GCC/Clang/etc. I don't want to learn a make program's huge syntax just to compile a program if I can avoid it! (I've still got so many D and networking topics to learn on the back-burner!)

I've heard "just use dub" but I've also heard that dub have flaws/gotchas that I can't remember when it comes to say, dynamic linking DLLs/SOs. So I've been hesitant to learn it.




#2 I ran individual file times. They're pretty shocking.
-------------------------------------------------------

std.regex is still, the Devil (TM), clocking in at almost FOUR SECONDS for a very simple set of code that simply matches lines for a poor-man's INI file parser (with a custom feature that allows tab indents to be nested sections). I was considering ripping it out and replacing it with JSON and this has REALLY motivated me to rip out the regex.

Here's the file times:

novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c molto.d
Class 4 - Scaled Rotated
hello
hello -- it matches!

real	0m0.377s
user	0m0.344s
sys	0m0.028s
novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c helper.d

real	0m0.118s
user	0m0.096s
sys	0m0.020s
novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c editor.d

real	0m0.626s
user	0m0.536s
sys	0m0.072s
novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c common.d

real	0m0.755s
user	0m0.636s
sys	0m0.092s
novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c map.d

real	0m1.045s
user	0m0.904s
sys	0m0.112s
novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c object_t.d

real	0m0.359s
user	0m0.336s
sys	0m0.024s
novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c animation.d

real	0m0.365s
user	0m0.280s
sys	0m0.068s
novous@saturn:~/Desktop/bitbucket/ss14$ time dmd -c ini.d

real	0m3.672s <--- WOWZA
user	0m3.292s
sys	0m0.332s



I have to tell you that, as an outsider (who is VERY interested in D), this is very frustrating. "Compile times are fast" != "build times" is a huge misconception that borders on being a clever lie or twisting of words. When people hear "compile times", they think "time to compile the whole project" not "time to compile a simple test case that doesn't use any typical D features--also, it's not linking." Second, as shown here, it's not fast even for compiling! Because the second you touch std.regex (which has NO WARNINGS in the documentation), you're greeted with another clever lie-by-omission: a 10x explosion of build time over some modules.

Now let's stop for a moment. I'm not accusing anyone, and "lie" is a strong word with heavy guilt implications--like people are intentionally being very sneaky to deceive new-comers. I'm not saying any of that, so you can relax and put down the pitchfork. I'm not attacking you or your group personally. However, I can't think of any better word.

So my point is, I keep running into either misconceptions that conveniently make D look good, and other gotchas with NO DOCUMENTATION that make the language much slower to work with than expected.

And if I'm experiencing this, there are dozens (hundreds?) who hit the same roadblocks and gotchas and many people are much less forgiving/understanding than I am and simply just "give up" without ever telling you. So I'm trying to say this with the best of intentions. You can't have standard modules with no warning documentation that explode your RAM usage and compile times orders-of-a-magnitude more than other ones. You can have an "alpha" or "beta" or "Addon" or "external" module. But putting it in your standard framework implies that it works well with the other modules (::cough::std.variant and threads::cough::), and that it's not incredibly slower or more resource intensive. Having it in your standard library implies it meets a certain _STANDARD_.

I mean, can you think of any module in the Python/Javascript/C# standard library that simply including it will swell your program to the point it can't compile? I'm not an omnipotent master programmer, but as a professional, I can't recall ever having this situation in another library or language.

And it's very frustrating for someone who _wants_ to learn and use D.

Thank you for your time, and I hope you had a great Thanksgiving.

On Sunday, 25 November 2018 at 22:00:21 UTC, Chris Katko wrote:
> On Saturday, 24 November 2018 at 20:44:57 UTC, welkam wrote:
>> On Friday, 23 November 2018 at 08:57:57 UTC, Chris Katko wrote:
>>> D is supposed to compile fast.
>>
>> You didnt read the fine print. It compiles simple code fast. Also compilation is separate step from linking and your program might spend half of "compilation" time in link phase.
>
> Wait wait wait wait wait.
>
> So 1) I have to compile manually, then link. Except that also runs the files every time even if they're up-to-date. Is that normal behavior for C/C++?

"runs the files every time"?  If that means "compiles the files every time", then no. D  works exactly like C/C++ - you only need to compile-to-object-code source files in the project that have changed since the last time they were compiled.

On Sunday, 25 November 2018 at 22:00:21 UTC, Chris Katko wrote:


>
> So 1) I have to compile manually, then link. Except that also runs the files every time even if they're up-to-date. Is that normal behavior for C/C++?

Yes, C and C++ compilers behave the same way.


> #1 How to I only build files that are dirty? Do I actually need a build program like DUB, MAKE, or CMAKE to do that? (Can make, cmake be used?) How do they recognize files are out-dated if DMD can't? Is that just an industry-standard specialization/separation-of-responsibilities to not have the compiler auto-detect up-to-date builds?


cmake doesn't actually build anything. It generates make files or IDE project files. I don't know if DUB currently has an option to build only dirty files. Make does it by default.

To do something like that requires looking for a source file's corresponding object file in the output directory and only compiling the source if the object file doesn't exist or if it has an older timestamp.

But consider what happens when you change the command line options, say enable a version on a build that wasn't enabled earlier. With make, you have to run `make clean` first, otherwise only dirty files will get the new options. It doesn't track build options per file from run to run. What should the compiler do? Have a -clean command line switch? Maintain a database of command line options per file? That's the realm of build systems. The compiler just compiles.


>
> I've heard "just use dub" but I've also heard that dub have flaws/gotchas that I can't remember when it comes to say, dynamic linking DLLs/SOs. So I've been hesitant to learn it.

DUB is easy. I've been using it for years and I do full compilation of every file on every invocation in a code->build->run cycle. Some aren't happy with it because it doesn't support some of the things they want to use it for, but it has plenty of satisfied users. I'm unaware of any gotchas about linking shared libraries.


>
>
>
>
> #2 I ran individual file times. They're pretty shocking.
> -------------------------------------------------------
>
> I have to tell you that, as an outsider (who is VERY interested in D), this is very frustrating. "Compile times are fast" != "build times" is a huge misconception that borders on being a clever lie or twisting of words. When people hear "compile times", they think "time to compile the whole project" not "time to compile a simple test case that doesn't use any typical D features--also, it's not linking." Second, as shown here, it's not fast even for compiling! Because the second you touch std.regex (which has NO WARNINGS in the documentation), you're greeted with another clever lie-by-omission: a 10x explosion of build time over some modules.
>
> Now let's stop for a moment. I'm not accusing anyone, and "lie" is a strong word with heavy guilt implications--like people are intentionally being very sneaky to deceive new-comers. I'm not saying any of that, so you can relax and put down the pitchfork. I'm not attacking you or your group personally. However, I can't think of any better word.
>
> So my point is, I keep running into either misconceptions that conveniently make D look good, and other gotchas with NO DOCUMENTATION that make the language much slower to work with than expected.


There is no misleading or misconception or lying or misdirection here. DMD has fast compile times. Heavily templated code is going to slow it down, but if you compile the same sort of heavily templated code in C++, you'll get slow downs there as well. And in your case, you'll find that if you add many more files to your project and they aren't heavily templated, the increase in build time will be very small.

If someone can dig into the compilers and optimize how templates are handled, they might be able to shave a bit of time off, but when you are using a code base that does a lot of work at compile time, then there's no avoiding increasing the length of that compile time.

But that by no means is the same as saying it's not fast, because overall compile times in D are still fast.


>
> I mean, can you think of any module in the Python/Javascript/C# standard library that simply including it will swell your program to the point it can't compile? I'm not an omnipotent master programmer, but as a professional, I can't recall ever having this situation in another library or language.

Have you reached the point in D where you can't compile?

On Monday, 26 November 2018 at 03:09:03 UTC, Mike Parker wrote:
>> I mean, can you think of any module in the Python/Javascript/C# standard library that simply including it will swell your program to the point it can't compile? I'm not an omnipotent master programmer, but as a professional, I can't recall ever having this situation in another library or language.
>
> Have you reached the point in D where you can't compile?

I thought people had reached that point already: https://forum.dlang.org/post/mailman.3134.1517944133.9493.digitalmars-d@puremagic.com

I remember reading about a never ending for loop as well. Because of the way ctfe has to be handled "purely" without mutations, every change is a new allocation (but i could be remembering wrong). So just a large enough for loop because "uncompliable". And this is just time, the memory consumption is also quite large no?

I think there are some caveats which are not mentioned when it comes to the fastness of the compiler. I remember last year's advent of code, I wanted to do everything at compile time, but at some point I thought, no, this small piece of code is taking way too long. Not worth it. And me mate's Go code was "consistently" fast.

On Sunday, 25 November 2018 at 22:00:21 UTC, Chris Katko wrote:.
>
> So 1) I have to compile manually, then link. Except that also runs the files every time even if they're up-to-date. Is that normal behavior for C/C++?

Well you dont have to use separate commands but yes compiling and linking are two steps and its normal behavior for all native languages. Each OS have their own linker and we dont control that.

> Two questions/topics/issues:
>
> -------------------------------------------------------
>
> #1 How to I only build files that are dirty? Do I actually need a build program like DUB, MAKE, or CMAKE to do that? (Can make, cmake be used?) How do they recognize files are out-dated if DMD can't? Is that just an industry-standard specialization/separation-of-responsibilities to not have the compiler auto-detect up-to-date builds?

Yes its separation of responsibilities and there are many tools to automate that. Take a look at rdmd

"rdmd recompiles files only on a needed basis, e.g. two invocations of rdmd in sequence without an intervening change to any relevant source file does not produce the executable again."

https://dlang.org/rdmd.html

If dub or rdmd doesnt satisfy your needs then you will need to learn other build system.

>
> I have to tell you that, as an outsider (who is VERY interested in D), this is very frustrating. "Compile times are fast" != "build times" is a huge misconception that borders on being a clever lie or twisting of words. When people hear "compile times", they think "time to compile the whole project" not "time to compile a simple test case that doesn't use any typical D features--also, it's not linking." Second, as shown here, it's not fast even for compiling! Because the second you touch std.regex (which has NO WARNINGS in the documentation), you're greeted with another clever lie-by-omission: a 10x explosion of build time over some modules.

Yes D have some rough spots thats for sure. For compile times these include std.regex, std.format and heavy CTFE use. There is newCTFE engine that is faster but its not ready yet. That said D code compiles faster than C++ and Rust. To really put fast into perspective read this https://news.ycombinator.com/item?id=18442941


> So my point is, I keep running into either misconceptions that conveniently make D look good, and other gotchas with NO DOCUMENTATION that make the language much slower to work with than expected.

People who advertise languages talk about positives and leave out negatives. One thing you need to know - D doesnt have huge sponsor and some places might be rough because of it. Currently core dev team focus more on stability and bug fixes than build speeds. I think you rather have compiler that works than compiler that crashes 1 sec faster.

Its not all negatives with regex. Current regex implementation is one of the fastest in the world. Thats classic D trade of - you spend compile time for better runtime.

> I mean, can you think of any module in the Python/Javascript/C# standard library that simply including it will swell your program to the point it can't compile?

Have you tried C++ boost?

On Friday, 23 November 2018 at 08:57:57 UTC, Chris Katko wrote:
> Any time I see people mention the benefits of D, I see "compile times" "compile times" "compile times" over and over.
>
> I'm using very modest amounts of templates, for a fairly small sized program (very early work toward a game), and I'm hitting ~15 seconds compile time in LDC and ~7 seconds in DMD. And I'm not even compiling with optimizations!
>
> ldc2 -w -ofextra  extra.d molto.d helper.d editor.d common.d map.d object_t.d animation.d ini.d  -L-L. $@    -gc -d-debug=3  -de -fdmd-trace-functions
>
> dmd -w -ofextra extra.d molto.d helper.d editor.d common.d map.d object_t.d animation.d ini.d -profile=gc  -profile  -g -debug -color -L-L.
>
> I keep putting stuff into new files, but it feels like it's compiling everything from scratch / not getting faster the way C++ does.
>
> And I'm not even bringing up the 800MB of RAM required because I dared to import std.regex. (On a laptop with 2 GB of RAM. RIP. If I dare to have tabs open, the compile time goes into the minutes thanks to swapping.)

I've profiled your example, unsurprisingly allmost all of the compiletime is eaten by regex, which is executed at compile-time.
Most of the time there is actually taken by the template-instantiation and the templates which are involved in building the IR and such.

newCTFE helps out a little, but for this to be fast, std.regex would have to be rewritten with CTFE in mind.

Maybe you can generate a finite automaton for your regex offline using some other engine like which supports D or C.

That alone should cut a few seconds of your compile-times and prevent out-of-memory errors,

I hope this helps.

Cheers,

Stefan