January 03, 2019
On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis wrote:
> *nix has the same distinction. It's a fundamentally different situation from linking your executable against the library. You're really dynamically loading rather than dynamically linking (though unfortunately, the terminology for the two is not particularly distinct, and they're often referred to the same way even though they're completely different). Loading libraries that way is what you do when you do stuff like plugins, because those aren't known when you build your program. But it makes a lot less sense as an alternative to linking your program against the library if you don't actually need to load the library like that. The COFF vs OMF mess on Windows makes it make slightly more sense on Windows (at least with D, where dmd uses OMF by default, unlike most of the C/C++ world at this point), because then it doesn't matter whether COFF or OMF was used (e.g. IIRC, Derelict is designed to be loaded that way for that reason), but in general, it's an unnecessarily complicated way to use a library. And if Windows' eccentricities make it more desirable than it is on *nix systems, then that's just yet another black mark against how Windows does dynamic linking IMHO.
>
> - Jonathan M Davis
>
> From the last time I worked with Windows dlls, I remember quite distinctly
> that doing anything like adding a symbol to the library meant that it was incompatible with executables previously built with it (which is not true for shared libraries on *nix - they only break if the ABI for the existing symbols changes). So, if that's not the case, I don't know what we were doing wrong, but I have an extremely sour taste in my mouth from dealing with Windows dlls. It was my experience that Linux stuff generally just worked, whereas we kept having to deal with junk on Windows to make them work (e.g. adding special attributes to functions just so that they would be exported), and I absolutely hated it. I have nothing good to say about dlls on Windows. It's quite possible that some of it isn't as bad if you know more about it than the team I was working on did, but it was one part of why making our libraries cross-platform was not at all fun.
>
> - Jonathan M Davis

Yah you should just stop and never talk about how Windows DLLs work again until you've actually learned how they actually work. "I used some piece of software that I have no idea how it works, I condemn it cause it works this way even though I'm sure if it even works that way". I already know the D team has no idea what they are doing on Windows, but just stop.
January 03, 2019
On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis wrote:
>From the last time I worked with Windows dlls, I remember quite distinctly
> that doing anything like adding a symbol to the library meant that it was incompatible with executables previously built with it (which is not true for shared libraries on *nix - they only break if the ABI for the existing symbols changes).

That would surely break if you were looking up symbols by ordinal instead of by name. I didn't even know you could do that until I read the documentation for GetProcAddress closely - turns out the name parameter can be used as an ordinal.

All exportable symbols are exported by their mangled name in to the DLL's export table. Resolving by mangle will give you a similar experience to *nix there.
January 03, 2019
On 1/3/2019 11:26 AM, Timon Gehr wrote:
> On 01.01.19 07:21, Walter Bright wrote:
>> On 12/31/2018 2:28 PM, Timon Gehr wrote:
>>> Welcome to D, where 'enum' means 'const', 'const' means 'readonly', 'lazy' means 'by name', 'assert' means 'assume' and 'real' does not mean 'real' (in fact, I really like the 'ireal' and 'creal' keywords, pity they are being phased out). :)
>>
>> D's "by name" are the template alias parameters.
> 
> I think alias parameters are not "by name" they are something like "by symbol". ("by name" is a bit confusing of a name, the argument expression need not really have a "name".)
> 
> The standard PL jargon is this:
> 
> https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_name
> https://en.wikipedia.org/wiki/Lazy_evaluation

Thanks for the info. It seems the explanations are a little slippery, and D's usage doesn't precisely fit.

For example, D's lazy parameters can be used to implement control structures, as it says lazy evaluation is for, but then seems a bit ambiguous about whether the lazy evaluation is done once or many times.
January 04, 2019
On Thursday, 3 January 2019 at 21:22:03 UTC, Ethan wrote:
> On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis wrote:
>>From the last time I worked with Windows dlls, I remember quite distinctly
>> that doing anything like adding a symbol to the library meant that it was incompatible with executables previously built with it (which is not true for shared libraries on *nix - they only break if the ABI for the existing symbols changes).
>
> That would surely break if you were looking up symbols by ordinal instead of by name. I didn't even know you could do that until I read the documentation for GetProcAddress closely - turns out the name parameter can be used as an ordinal.
>
> All exportable symbols are exported by their mangled name in to the DLL's export table. Resolving by mangle will give you a similar experience to *nix there.

Even if you used ordinal values instead, you need to assign the symbol a specific ordinal value. The only reason you would have to relink the executable is if you changed the ordinal values to be incompatible with the previous layout, eg assigning a new function to have a ordinal value the same to one that was previously assigned to different function. The equivalent to removing a symbol resolving by name on *nix/Windows.
January 03, 2019
On Thursday, January 3, 2019 2:22:03 PM MST Ethan via Digitalmars-d wrote:
> On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis
>
> wrote:
> >From the last time I worked with Windows dlls, I remember quite distinctly
> >
> > that doing anything like adding a symbol to the library meant that it was incompatible with executables previously built with it (which is not true for shared libraries on *nix - they only break if the ABI for the existing symbols changes).
>
> That would surely break if you were looking up symbols by ordinal instead of by name. I didn't even know you could do that until I read the documentation for GetProcAddress closely - turns out the name parameter can be used as an ordinal.
>
> All exportable symbols are exported by their mangled name in to the DLL's export table. Resolving by mangle will give you a similar experience to *nix there.

That would be a significant improvement. Thanks for pointing that out. If/when I have to deal with Windows dlls again, I'll have to look into that - though honestly, in general, I don't see any reason to use shared libraries for anything other than plugins (whether you're talking about *nix or Windows). If you're dealing with system libraries that are shared by a ton of programs on the system (e.g. openssl on *nix systems), then using shared libraries can definitely be worth it (especially when the library is likely to need security updates), but if you're distributing a program, static libraries are far easier, and if a library isn't used by many programs, then I think that the simplicity of static libraries is worth far more than the small space gain of using a shared library. It's just with plugins that you really don't have any choice.

Regardless, D should have full shared library support on all of its supported platforms, and for those who care particularly about Windows, hopefully, Benjamin Thaut's work can be finished in a timely manner. IIRC, he did make a post about it a few months back looking for folks to try out his branch of dmd so that any issues could be worked out. I have no clue how long it will be before that work gets merged, but I expect that it will happen at some point.

- Jonathan M Davis



January 04, 2019
On Friday, 4 January 2019 at 03:48:49 UTC, Jonathan M Davis wrote:
> though honestly, in general, I don't see any reason to use shared libraries for anything other than plugins (whether you're talking about *nix or Windows). If you're dealing with system libraries that are shared by a ton of programs on the system (e.g. openssl on *nix systems), then using shared libraries can definitely be worth it (especially when the library is likely to need security updates), but if you're distributing a program, static libraries are far easier, and if a library isn't used by many programs, then I think that the simplicity of static libraries is worth far more than the small space gain of using a shared library. It's just with plugins that you really don't have any choice.

Here we go again. Really? You don't think there is any reason to using shared libraries in anything other than plugins? You then go on to list two other valid reasons in your next run on sentence. I'll add some more to your list: treating code as data is another reason. Hot loading code so you don't have to restart your program which can be time consuming. Would not work without a shared library. Modifying applications, though I guess from your perspective this might be niche. It would not be as elegant without shared libraries. You keep bringing up this size saving rationale, no one has said this other than you.

January 04, 2019
On Monday, 31 December 2018 at 13:20:35 UTC, Atila Neves wrote:
> Blog:
>
> https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-triples-in-c-d-and-rust/

Isn't the main problem with performance of the Timon's range loop that it uses arbitrary-sized integers (BigInts)? I took his example, and modified it to this:

import std.experimental.all;
import std.datetime.stopwatch : AutoStart, StopWatch;

alias then(alias a)=(r)=>map!a(r).joiner;
void main(){
    auto sw = StopWatch(AutoStart.no);

    if (true)
    {   sw.start;
        scope (success) sw.stop;
        auto triples=recurrence!"a[n-1]+1"(1.BigInt)
            .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z))))
            .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2)
            .until!(t=>t[2] >= 500);
        triples.each!((x,y,z){ writeln(x," ",y," ",z); });
    }

    writefln("Big int time is %s microseconds", sw.peek.total!"usecs");
    sw.reset;

    if (true)
    {   sw.start;
        scope (success) sw.stop;
        auto triples=recurrence!"a[n-1]+1"(1L)
            .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z))))
            .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2)
            .until!(t=>t[2] >= 500);
        triples.each!((x,y,z){ writeln(x," ",y," ",z); });
    }

    writefln("Long int time is %s microseconds", sw.peek.total!"usecs");
    return;
}

The output, LDC version being 1.11.0-beta2, with:

dub --compiler=ldc2 --build=release

was:

3 4 5
6 8 10
5 12 13
9 12 15
8 15 17
[...snip...]
155 468 493
232 435 493
340 357 493
190 456 494
297 396 495
Big int time is 4667925 microseconds
3 4 5
6 8 10
5 12 13
9 12 15
8 15 17
[...snip...]
155 468 493
232 435 493
340 357 493
190 456 494
297 396 495
Long int time is 951821 microseconds

That is almost five times as fast. Assuming the factor would be the same in your blog, doesn't that account for most of the difference between performance of D ranged and other versions? The remaining difference might be explained with bounds-checking.


January 04, 2019
On 1/4/19 9:00 AM, Dukc wrote:
> On Monday, 31 December 2018 at 13:20:35 UTC, Atila Neves wrote:
>> Blog:
>>
>> https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-triples-in-c-d-and-rust/ 
>>
> 
> Isn't the main problem with performance of the Timon's range loop that it uses arbitrary-sized integers (BigInts)?

Atila's version of that code doesn't use bigints: https://github.com/atilaneves/pythagoras/blob/master/range.d#L24

The major problem with the D range implementation is that the compiler isn't able to find the optimization of hoisting the multiplication of the outer indexes out of the inner loop.

See my responses to Atila in this thread.

-Steve
January 04, 2019
On Fri, 04 Jan 2019 12:54:12 +0000, Rubn wrote:
> Here we go again. Really? You don't think there is any reason to using shared libraries in anything other than plugins? You then go on to list two other valid reasons in your next run on sentence. I'll add some more to your list

I'll give another: linking times. With GTKD, static linking costs something like 6-10 seconds more than dynamic linking, which is huge.
January 04, 2019
On Friday, 4 January 2019 at 16:28:32 UTC, Neia Neutuladh wrote:
> On Fri, 04 Jan 2019 12:54:12 +0000, Rubn wrote:
>> Here we go again. Really? You don't think there is any reason to using shared libraries in anything other than plugins? You then go on to list two other valid reasons in your next run on sentence. I'll add some more to your list
>
> I'll give another: linking times. With GTKD, static linking costs something like 6-10 seconds more than dynamic linking, which is huge.

And to add to this. *ANY* large project benefits from reducing to smaller DLLs and a corresponding static .lib to load them.

The MSVC team has come around in the last five years to love game developers specifically because we create massive codebases that tax the compiler. Remember how I said our binary sizes were something up near 100MB on Quantum Break during one of my DConf talks? Good luck linking that in one hit.