Am 10.11.2013 13:39, schrieb Timon Gehr:
> On 11/10/2013 01:06 PM, Benjamin Thaut wrote:
>> Am 10.11.2013 13:03, schrieb Timon Gehr:
>>> On 11/10/2013 12:42 PM, Benjamin Thaut wrote:
>>>> Am 08.11.2013 20:32, schrieb Walter Bright:
>>>>  > It looks pretty good, except I have serious reservations about the
>>>> -lib switch proposed behavior:
>>>> I'm glad you like the proposal.
>>>>
>>>>  >
>>>>  > 1. It's too blunt. A user could conceivably want to export some
>>>> symbols and not others. This is all or nothing.
>>>> A user is already able to control which symbols are exported and which
>>>> are not by using the "export" attribute. ...
>>>
>>> Using the export attribute on some member exports the entire module
>>> info. If I understand this right, the module info will contain
>>> information also about non-exported members of the module and therefore
>>> even non-exported members will be accessible from outside. Is this
>>> correct?
>>
>> AFAIK the module info only contains information about the (shared)
>> module constructors and destructors.  And no, non exported symbols will
>> not be acessible because the neccessary accessors are not generated for
>> them (at least on windows).
>
> What about Object.factory?

That would obviously work. But why would you want to restrict that for shared libraries? Its not restricted for static libraries either. And esentially a shared library should behave exactly the same as a static one.

Am 08.11.2013 20:32, schrieb Walter Bright:
>
> And:
>
> Exporting of TLS variables is a minor issue. Frankly, I think people who export global variables should be burned at the stake anyway, why should we make it even easier? Anyhow, if the user really, really wants to get himself immolated and export a TLS variable, he can always manually write the thunk (it is, after all, trivial).

In my opinion it should be possible to add "export:" to the top of every source file and then compiler a dynamic library which exatcly behaves like a static library. If you don't want TLS variables in interfaces this should also be forbidden for interfaces of static libraries, because this isn't really any difference. This really should be consistent. Either TLS variabels are not allowed at all within library interfaces or they are allowed and supported always.

Kind Regards
Benjamin Thaut

On 11/10/2013 4:13 AM, Benjamin Thaut wrote:
> Addition: It is necessary to export the module info. Because as soon as you
> import a module and use any part of it, the compiler will reference the module
> info.

Actually, it isn't entirely clear to me why the moduleinfo needs to be exported.

The moduleinfo is needed to run the static ctors/dtors, etc., but that is needed by the code internal to the dll that initializes the dll.

On 11/10/2013 6:52 AM, Benjamin Thaut wrote:
> Am 08.11.2013 20:32, schrieb Walter Bright:
>  >
>  > And:
>  >
>  > Exporting of TLS variables is a minor issue. Frankly, I think people who
> export global variables should be burned at the stake anyway, why should we make
> it even easier? Anyhow, if the user really, really wants to get himself
> immolated and export a TLS variable, he can always manually write the thunk (it
> is, after all, trivial).
>
> In my opinion it should be possible to add "export:" to the top of every source
> file and then compiler a dynamic library which exatcly behaves like a static
> library. If you don't want TLS variables in interfaces this should also be
> forbidden for interfaces of static libraries, because this isn't really any
> difference. This really should be consistent. Either TLS variabels are not
> allowed at all within library interfaces or they are allowed and supported always.

I understand the consistency argument, but I'll also argue that if one raises consistency as an overriding requirement, everything else suffers. Everything is a tradeoff.

It's long been recognized that using global variables to communicate between interfaces is a bad idea. And it isn't even supported for dlls, because the proposed solution to making them work is to wrap them with a function (much like D properties).

We'd be going out of our way to support a recognized bad paradigm. There is currently no existing D code that requires this. If we add it, then we'd be stuck with supporting it for backwards compatibility. If we don't add it, and it becomes some sort of crisis that it isn't supported, we can add it in later without breaking things.

Am 10.11.2013 20:33, schrieb Walter Bright:
> On 11/10/2013 4:13 AM, Benjamin Thaut wrote:
>> Addition: It is necessary to export the module info. Because as soon
>> as you
>> import a module and use any part of it, the compiler will reference
>> the module
>> info.
>
> Actually, it isn't entirely clear to me why the moduleinfo needs to be
> exported.
>
> The moduleinfo is needed to run the static ctors/dtors, etc., but that
> is needed by the code internal to the dll that initializes the dll.

Well if the dll initializes all the modules, it has to initialize _all_ the modules because it can't know which are used and which are not. This is most certenly the way to go when loading a D dll with the LoadLibrary windows api function(s).

But when you link against a D dll using a genereated import library there would be the option to make it behave exactly like static libraries where only modules get initiliazed, which are actually used.

Also lets assume you do not export module infos, how does the compiler know that the module is inside a dll so that the module info is not referenced? That goes against the basic design of this DIP. The entire idea was to make static / shared liraries more similar and require less effort from the user. And not the other way around. If this is not wanted we can go right back to the C/C++ way of defining both dllexport and dllimport.

Kind Regards
Benjamin Thaut

Am 10.11.2013 20:46, schrieb Walter Bright:
> On 11/10/2013 6:52 AM, Benjamin Thaut wrote:
>
> I understand the consistency argument, but I'll also argue that if one
> raises consistency as an overriding requirement, everything else
> suffers. Everything is a tradeoff.
>
> It's long been recognized that using global variables to communicate
> between interfaces is a bad idea. And it isn't even supported for dlls,
> because the proposed solution to making them work is to wrap them with a
> function (much like D properties).
>
> We'd be going out of our way to support a recognized bad paradigm. There
> is currently no existing D code that requires this. If we add it, then
> we'd be stuck with supporting it for backwards compatibility. If we
> don't add it, and it becomes some sort of crisis that it isn't
> supported, we can add it in later without breaking things.
>

You completely ignored my inlining argument. Lets assume there is some small function that qualifies for inlining but accesses a TLS variable marked with the export attribute. The compiler now wants to cross-module inline it. How does it do that, if there is no way to access TLS variables across DLL boundaries? The obvious solution would be, not do it at all. That means to disqualify all functions that do TLS access for cross-module inlining, because the compiler can't know if they are linked in from a static or shared library.

With D beeing a language that aims for performance, do we really want to do that?

Kind Regards
Benjamin Thaut

On 11/10/2013 1:25 PM, Benjamin Thaut wrote:
> Am 10.11.2013 20:33, schrieb Walter Bright:
>> On 11/10/2013 4:13 AM, Benjamin Thaut wrote:
>>> Addition: It is necessary to export the module info. Because as soon
>>> as you
>>> import a module and use any part of it, the compiler will reference
>>> the module
>>> info.
>>
>> Actually, it isn't entirely clear to me why the moduleinfo needs to be
>> exported.
>>
>> The moduleinfo is needed to run the static ctors/dtors, etc., but that
>> is needed by the code internal to the dll that initializes the dll.
>
> Well if the dll initializes all the modules, it has to initialize _all_ the
> modules because it can't know which are used and which are not. This is most
> certenly the way to go when loading a D dll with the LoadLibrary windows api
> function(s).

Yes, and that's also the way shared library support has been implemented in druntime.


> But when you link against a D dll using a genereated import library there would
> be the option to make it behave exactly like static libraries where only modules
> get initiliazed, which are actually used.

I don't believe it is necessary to support this. I don't think the C or C++ standard shared runtime library does. The general idea with DLLs is they become an indivisible unit - if you want to load only part of a DLL, you should split it into multiple DLLs.


> Also lets assume you do not export module infos, how does the compiler know that
> the module is inside a dll so that the module info is not referenced? That goes
> against the basic design of this DIP. The entire idea was to make static /
> shared liraries more similar and require less effort from the user. And not the
> other way around. If this is not wanted we can go right back to the C/C++ way of
> defining both dllexport and dllimport.

The trouble with dllexport and dllimport is the same module is both depending on how it is used - I don't think that is very viable (C/C++ deal with this using macros). The idea is to use export for both, and then use the context to figure out if it is dllimport or dllexport.

On 11/10/2013 1:50 PM, Benjamin Thaut wrote:
> You completely ignored my inlining argument. Lets assume there is some small
> function that qualifies for inlining but accesses a TLS variable marked with the
> export attribute. The compiler now wants to cross-module inline it. How does it
> do that, if there is no way to access TLS variables across DLL boundaries? The
> obvious solution would be, not do it at all.

Right.

> That means to disqualify all
> functions that do TLS access for cross-module inlining, because the compiler
> can't know if they are linked in from a static or shared library.

The compiler would benefit from knowing which modules are from a shared library, and it needs to anyway in order to distinguish between dllimport and dllexport.


> With D beeing a language that aims for performance, do we really want to do that?

The trouble is, the proposed solution for exporting TLS variables is to wrap the access in a function that is not inline-able anyway. For performance oriented code, the user would be better off to write his own wrapper, and then hoist the call out of the hot spot, and use a pointer to the TLS variable in the hot spot.

As a general rule, if code has a "hot spot" that crosses a DLL boundary, there is going to be performance problems with it regardless. I don't think this is a problem that the language can solve without some effort from the user.

Am 11.11.2013 05:04, schrieb Walter Bright:
>
>
> The trouble with dllexport and dllimport is the same module is both
> depending on how it is used - I don't think that is very viable (C/C++
> deal with this using macros). The idea is to use export for both, and
> then use the context to figure out if it is dllimport or dllexport.
>

Still, how do you want to stop the compiler from referencing the module infos without knowing the module is part of a dll?

Am 11.11.2013 05:17, schrieb Walter Bright:
> On 11/10/2013 1:50 PM, Benjamin Thaut wrote:
>
> The compiler would benefit from knowing which modules are from a shared
> library, and it needs to anyway in order to distinguish between
> dllimport and dllexport.
>

Then you didn't understand the DIP correctly. One of the main points of this DIP is to provide a implementation of the export attribute that does _not_ require the compiler to know anything about, wether certain modules will end up in a DLL or not. With the proposed behaviour you could compile a module into an object file and then link it both into a DLL and static lirary. Both would just work. When using a library the compiler also doesn't have to know if the lirary is shared or static. Both will work and it will be decided at link time which path is coosen (static or shared). Generally this behaviour makes the export attribute easier to use and less error prone.

In an early version of this DIP did involve giving the compiler the information which modules end up in an shared library, but it was rejected in the discussion. A solution which does not require this information was preferred by almost all users taking part in the discussion.

>
> The trouble is, the proposed solution for exporting TLS variables is to
> wrap the access in a function that is not inline-able anyway. For
> performance oriented code, the user would be better off to write his own
> wrapper, and then hoist the call out of the hot spot, and use a pointer
> to the TLS variable in the hot spot.
>
> As a general rule, if code has a "hot spot" that crosses a DLL boundary,
> there is going to be performance problems with it regardless. I don't
> think this is a problem that the language can solve without some effort
> from the user.

Correct. But the compiler would still be able do other optimizations given the function is sligtly bigger then a simple getter.