Am 08.09.2013 06:19, schrieb deadalnix:
> On Saturday, 7 September 2013 at 21:41:01 UTC, Benjamin Thaut wrote:
>> I don't agree if that statement. If you have three libs. lib1, lib2
>> and lib3. And lib2 and lib3 link statically against lib1 you are going
>> to get linker errors because both lib2 and lib3 contain the symbols of
>> lib1. And you don't have any options to avoid that (unless you got
>> into the source of lib1 and remove all export attributes, which might
>> not even be possible because you don't have the source)
>
> And you should have an error, especially is you pass object from one
> side to the other.
>
> typeid won't match (which will create a mess in the runtime), and worse,
> struct layout may not match (hello memory corruption).
>
> If the problem is that big, we can still have a -noexport flag or
> something, but that doesn't seems really safe to me.

But only assuming that you are passing around objects of that type. I see a dll as a module of encapsulation. If the library is only unsed internally nothing of its dependencies should get exported. Because if they are your users might rely on the fact that they are, and from that point on you have to garantuee that all those additional exported functions from your dependencies stay in your library, even if you don't need the dependency anymore.

Am 27.08.2013 12:12, schrieb Benjamin Thaut:
> The current behaviour of export is not sufficient to create a shared
> runtime on windows. Its alos not working very well in a few other cases.
> For background information please read the links provided in the DIP.
> This DIP tries to solve the problem by imitating the proven preprocessor
> techniques used in C/C++ to create windows dlls.
>
> Destroy.
>
> Kind Regards
> Benjamin Thaut

There is yet another problem to be solved. What should happen with tempaltes? Does it make sense to export templates? What happens if a template class marked with export gets instanciated in different DLLs using the same template arguments? Should the TypeInfo objects match? Should templates be exportable in generall?

In C++ the only thing I needed so far was exporting static members of templated classes. So the different instances of the template can work on the same data.

Kind Regards
Benjamin Thaut

Am 08.09.2013 09:33, schrieb Benjamin Thaut:
> Am 27.08.2013 12:12, schrieb Benjamin Thaut:
>> The current behaviour of export is not sufficient to create a shared
>> runtime on windows. Its alos not working very well in a few other cases.
>> For background information please read the links provided in the DIP.
>> This DIP tries to solve the problem by imitating the proven preprocessor
>> techniques used in C/C++ to create windows dlls.
>>
>> Destroy.
>>
>> Kind Regards
>> Benjamin Thaut
>
> There is yet another problem to be solved. What should happen with
> tempaltes? Does it make sense to export templates? What happens if a
> template class marked with export gets instanciated in different DLLs
> using the same template arguments? Should the TypeInfo objects match?
> Should templates be exportable in generall?
>
> In C++ the only thing I needed so far was exporting static members of
> templated classes. So the different instances of the template can work
> on the same data.
>
> Kind Regards
> Benjamin Thaut


Good question. Not sure how Ada, Modula-3, Eiffel, Delphi, MLton solve the issue, even though their generic capabilities are not as powerfull.

--
Paulo

>> I also kind of feel that we need to export lib1 as well, as you may not
>> know what lib2 does with what it gets from lib1 (passing objects around
>> for instance). So certainly at least some part of lib1 need to be
>> exported.
For sure lib2 might forward to some lib1 symbols.
Not sure what might accidentally require linkage.

> I don't agree if that statement. If you have three libs. lib1, lib2 and
> lib3. And lib2 and lib3 link statically against lib1 you are going to
> get linker errors because both lib2 and lib3 contain the symbols of lib1.

A: Why do two DLLs which export the same symbols cause linker errors? Because the corresponding import libraries contain the same symbols?
B: You shouldn't mix static with dynamic linking, ODR issues are what you get in return.
C: What would happen if you statically link against lib2 and lib3?

On 09/07/2013 02:47 PM, Benjamin Thaut wrote:
> If you compile lib1 into a static library and then copmpile lib2 into a
> DLL which statically links against lib1 both symbols lib1Func and
> lib2Func will be exported from the dll
That might indeed cause some issues, but when you make lib1 part of your DLL by statically linking against it that's what you get.
If you just want to use it, you should make lib1 a DLL as well, which also avoids ODR problems.
Also could a .DEF file control which symbols are reexported?

Can we deal with vtable entries pointing to imported functions on Windows? The kind of thing that happens when you inherit across a DLL boundary.

On 09/07/2013 02:47 PM, Benjamin Thaut wrote:
>
> If you compile lib1 into a static library and then copmpile lib2 into a
> DLL which statically links against lib1 both symbols lib1Func and
> lib2Func will be exported from the dll because lib1Func will get marked
> as dllexport when compiled into a static library and the static library
> transports that information into the linking process of lib2. This
> happens both on win32 and win64. But this is something you really don't
> want. Lets imagine you have a DLL that statically links against phobos.
> Suddenly you will have all phobos exported symbols in your dll. This can
> also lead to double defined symbols in case a user links against your
> dll and against the shared phobos dll. So we have to fix this somehow.

I had another thought about this and tried some examples.

If you do transitively link against lib1.lib symbols when using lib2.dll you might get ODR issues if there is another copy of lib1 in your process. This is to be expected because there are two copies of the same library. The safe choice is to make lib1 a DLL too.

If you still want to statically link lib1 into lib2.dll you have to be very cautious. An actual problem only occurs when lib1 symbols get used from outside of lib2.dll.
Of course a mean to prevent bugs from accidental linking lib1 symbols would be nice.
One way of doing this would be to mark all lib1 symbols as PRIVATE in lib2.def.

EXPORTS
D4lib18lib1FuncFZi PRIVATE

Another way would be to write a small tool that turns all exported symbols of a library into non-exported symbols.

I'm not sure we should do this in the compiler, because it simply hands over the libs to the linker. We'd have to make a copy of the library first and then do what a tool would do.

Another note for the phobos case. You should really ask yourself why does your dll need a private copy of phobos. And if so why can phobos symbols leak out of your dll.

On 11.09.2013 00:39, Martin Nowak wrote:
> On 09/07/2013 02:47 PM, Benjamin Thaut wrote:
>>
>> If you compile lib1 into a static library and then copmpile lib2 into a
>> DLL which statically links against lib1 both symbols lib1Func and
>> lib2Func will be exported from the dll because lib1Func will get marked
>> as dllexport when compiled into a static library and the static library
>> transports that information into the linking process of lib2. This
>> happens both on win32 and win64. But this is something you really don't
>> want. Lets imagine you have a DLL that statically links against phobos.
>> Suddenly you will have all phobos exported symbols in your dll. This can
>> also lead to double defined symbols in case a user links against your
>> dll and against the shared phobos dll. So we have to fix this somehow.
>
> I had another thought about this and tried some examples.
>
> If you do transitively link against lib1.lib symbols when using lib2.dll
> you might get ODR issues if there is another copy of lib1 in your
> process. This is to be expected because there are two copies of the same
> library. The safe choice is to make lib1 a DLL too.
>
> If you still want to statically link lib1 into lib2.dll you have to be
> very cautious. An actual problem only occurs when lib1 symbols get used
> from outside of lib2.dll.
> Of course a mean to prevent bugs from accidental linking lib1 symbols
> would be nice.
> One way of doing this would be to mark all lib1 symbols as PRIVATE in
> lib2.def.
>
> EXPORTS
> D4lib18lib1FuncFZi PRIVATE
>

That's probably not feasible for a library like phobos. (At least optlink stumbles if the def file gets too large). Also, compressed symbol names cause troubles for OMF.

> Another way would be to write a small tool that turns all exported
> symbols of a library into non-exported symbols.
>
> I'm not sure we should do this in the compiler, because it simply hands
> over the libs to the linker. We'd have to make a copy of the library
> first and then do what a tool would do.
>
> Another note for the phobos case. You should really ask yourself why
> does your dll need a private copy of phobos. And if so why can phobos
> symbols leak out of your dll.
>

AFAIU the discussed scenario is that the phobos library exports a lot of symbols, and this library is used both for static and dynamic linking. An executable statically linked against this library will have all the exports. That still works but is ugly.

A DLL to be used as a plugin with a C interface might also want to link statically against phobos to avoid DLL hell. Consider multiple plugins built against different versions of phobos. These would also leak also the symbols.

On Tuesday, 3 September 2013 at 07:02:33 UTC, Rainer Schuetze wrote:
> To initialize data with a pointer into another DLL, you cannot do that with a direct relocation, you have to run some code to copy the value from the import table into the data.

Well, yes, static initialization is problematic, but you can have statically initialized pointer to an imported function, which is the same problem.

On 09/11/2013 08:21 AM, Rainer Schuetze wrote:
> AFAIU the discussed scenario is that the phobos library exports a lot of
> symbols, and this library is used both for static and dynamic linking.
> An executable statically linked against this library will have all the
> exports. That still works but is ugly.

Yes and it also has some runtime/memory overhead because the symbol table is much bigger.

> A DLL to be used as a plugin with a C interface might also want to link
> statically against phobos to avoid DLL hell. Consider multiple plugins
> built against different versions of phobos. These would also leak also
> the symbols.

Right, that's the valid use-case. But here the additionally exported symbols are way less of a problem because your loading not linking.
And as I wrote before if you link a program with two copies of the same library you're asking for trouble.

Anyhow I realized there is even less a use-case for statically linking a lib into a DLL (shared library) while expecting the symbols to be exported. We use that to put druntime into libphobos2.so though.
Also one doesn't need exported (visible) symbols for static linking.

So how about going into the opposite direction?
When the compiler creates a lib it strips all the exports (marks all symbols as hidden) by default. For the very special use-case where one wants to preserve the exports we can add a -libexports flag, only to be used with -lib.
Another nice thing is, that dmd writes libs itself and already processes all object files so we could easily integrate the striping in the compiler.