On 15.02.2013 23:34, Rainer Schuetze wrote:
>
> In contrast to the version in the bugzilla report (which linked the DM C
> runtime library snn.lib into the DLL, but exported all its symbols) I
> now used the shared C runtime library snd.lib which is the import
> library for sccrt70.dll). I think that this is the better approach as it
> allows sharing the C runtime with non-D DLLs. In addition it is easier
> to translate to other C-runtimes.
>
> Now I realize that this library and the DLL are not found in the free
> dmc compiler distribution, but only in the commercial distribution.
> Walter, is there a problem with also adding snd.lib and sccrt70.dll to
> the DMD release?


Ping. Walter, does it make sense to assume that snd.lib and sccrt70.dll will be distributed with dmd?

_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On 2/20/2013 12:16 AM, Rainer Schuetze wrote:
> On 15.02.2013 23:34, Rainer Schuetze wrote:
>>
>> In contrast to the version in the bugzilla report (which linked the DM C
>> runtime library snn.lib into the DLL, but exported all its symbols) I
>> now used the shared C runtime library snd.lib which is the import
>> library for sccrt70.dll). I think that this is the better approach as it
>> allows sharing the C runtime with non-D DLLs. In addition it is easier
>> to translate to other C-runtimes.
>>
>> Now I realize that this library and the DLL are not found in the free
>> dmc compiler distribution, but only in the commercial distribution.
>> Walter, is there a problem with also adding snd.lib and sccrt70.dll to
>> the DMD release?
>
>
> Ping. Walter, does it make sense to assume that snd.lib and sccrt70.dll will be distributed with dmd?


They need updating first. But I suspect users would prefer that there just be one dll (your first approach).
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On 20.02.2013 09:39, Walter Bright wrote:
>
> On 2/20/2013 12:16 AM, Rainer Schuetze wrote:
>> On 15.02.2013 23:34, Rainer Schuetze wrote:
>>>
>>> In contrast to the version in the bugzilla report (which linked the DM C
>>> runtime library snn.lib into the DLL, but exported all its symbols) I
>>> now used the shared C runtime library snd.lib which is the import
>>> library for sccrt70.dll). I think that this is the better approach as it
>>> allows sharing the C runtime with non-D DLLs. In addition it is easier
>>> to translate to other C-runtimes.
>>>
>>> Now I realize that this library and the DLL are not found in the free
>>> dmc compiler distribution, but only in the commercial distribution.
>>> Walter, is there a problem with also adding snd.lib and sccrt70.dll to
>>> the DMD release?
>>
>>
>> Ping. Walter, does it make sense to assume that snd.lib and
>> sccrt70.dll will be distributed with dmd?
>
>
> They need updating first. But I suspect users would prefer that there
> just be one dll (your first approach).

I was afraid you would say that ;-) If you are in DLL hell, I don't think it makes a big difference whether you have to deal with one or two libraries. Also, sccrt70.dll is probably not a fast moving target.

What you lose with compiling the C runtime into the D DLL is sharing it with a C host or DLL. If this gets ported to Win64 and the MS runtime, it might be more important because it seems more common to use the shared MS-CRT.

Also, the build process is a lot cleaner with snd.lib, because snn.lib has to be teared apart. I'm also a bit worried that not all objects in snn.lib are prepared for being used from multiple client DLLs.
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On 2/20/2013 10:45 AM, Rainer Schuetze wrote:
>
>> They need updating first. But I suspect users would prefer that there
>> just be one dll (your first approach).
>
> I was afraid you would say that ;-) If you are in DLL hell, I don't think it makes a big difference whether you have to deal with one or two libraries. Also, sccrt70.dll is probably not a fast moving target.
>
> What you lose with compiling the C runtime into the D DLL is sharing it with a C host or DLL.

Right, but I don't think that's an issue here.

> If this gets ported to Win64 and the MS runtime, it might be more important because it seems more common to use the shared MS-CRT.

I agree that the situation for Win64 is different, and the shared MS-CRT dll should be used.

>
> Also, the build process is a lot cleaner with snd.lib, because snn.lib has to be teared apart. I'm also a bit worried that not all objects in snn.lib are prepared for being used from multiple client DLLs.
>
>

I don't think there's an issue there.
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On Feb 6, 2013, at 4:56 PM, David Nadlinger <code@klickverbot.at> wrote:
> 
> But the situation is a *bit* more complex than that. For example, currently the GC root ranges for static and TLS data are just determined once at program startup by taking the address of the respective marker symbols. This clearly can't work in the presence of multiple images that can also be loaded and unloaded during execution, and having a shared druntime doesn't magically change that.

Runtime.loadLibrary etc can be extended to handle the additional tasks.  As Brad said, it's really just a matter of someone taking ownership of the task.  I don't use dynamic libraries much myself or I'd have probably pushed this one farther already.
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On Feb 20, 2013, at 10:45 AM, Rainer Schuetze <r.sagitario@gmx.de> wrote:

> What you lose with compiling the C runtime into the D DLL is sharing it with a C host or DLL. If this gets ported to Win64 and the MS runtime, it might be more important because it seems more common to use the shared MS-CRT.

This.  I'd prefer if the library construction and application build process were consistent between Win32 and Win64.  Why bundle the DLL in 32 bit and leave it out in 64?
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On 15.02.2013 23:34, Rainer Schuetze wrote:
> On 08.02.2013 14:00, Andrei Alexandrescu wrote:
>> On 2/8/13 2:18 AM, Rainer Schuetze wrote:
>>> On 06.02.2013 23:15, Walter Bright wrote:> This would go a long ways
>>> towards solving the dll issue, as then the gc
>>>  > and thread instances will automatically be shared.
>>>
>>> Time to dig up http://d.puremagic.com/issues/show_bug.cgi?id=4071 again.
>>> I had implemented a working shared phobos.dll at that time. I can try to
>>> bring it up to date if the approach seems appropriate.
>>
>> Yes please, that would be awesome.
>>
>> Andrei
>
> I've spent some time on it and have created a Win32 druntime DLL. Stay
> tuned for some pull requests to come after a few cleanups...

Sorry for the delay, I couldn't work on it for a couple of days and faced an unexpected problem with relocation to data:

IMO using the import functionality of "export" to support data imports through the import table is not an option as it pollutes the code (you might even need two different sources when building against a shared or static library) and introduces restrictions (like not being able to use the address of an imported symbol in static data initializers). That's why I am following the idea to patch code to refer to data in the shared library.

- The old patch in bugzilla walked the relocation information in the executable and analyzed the data: if it refers to some entry in the import table it gets patched with the actual address in the shared library.

- Unfortunately this does not work when there is an offset applied to the relocation, because you can no longer identify the entry in the import table. I implemented another indirection through some extra data to detect offsets within the referred data, but that's both ugly and unsafe in case the optimizer generates code with offsets outside the original data range.

- Instead, I patched dmd to emit extra relocation information to be collected in some data segment. This works for references from data, but as functions are generated into comdats the only sensible way to add this data is to use comdats aswell. Unfortunately again, comdats cannot be assigned to a separate segment according to the OMF spec, so they are merged with other data and you cannot identify the relocation information in the image anymore.

- For COFF, there is the concept of "associated" segments (also used by ".pdata" sections), that are added to the binary if the other segment is linked in. Implementing this for Win64 revealed another problem: code is generated with PC-relative data accesses, so patching the code to refer to a DLL far away in the 64-bit address space does not work.

So I ended up with only Win32-COFF working with this concept so far...

Questions:
- Is there a way to have something similar as "associated" segments with the OMF object format?

- Is it possible to force generation of absolute accesses to external data in Win64?

- What is happening on linux? I guess there is no import table there so relocations are pointing directly into the shared library, but what about PC relative adressing?

- Is there something wrong with the whole concept?

_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On 3/1/2013 9:38 AM, Rainer Schuetze wrote:
> On 15.02.2013 23:34, Rainer Schuetze wrote:
>> On 08.02.2013 14:00, Andrei Alexandrescu wrote:
>>> On 2/8/13 2:18 AM, Rainer Schuetze wrote:
>>>> On 06.02.2013 23:15, Walter Bright wrote:> This would go a long ways
>>>> towards solving the dll issue, as then the gc
>>>>  > and thread instances will automatically be shared.
>>>>
>>>> Time to dig up http://d.puremagic.com/issues/show_bug.cgi?id=4071 again.
>>>> I had implemented a working shared phobos.dll at that time. I can try to
>>>> bring it up to date if the approach seems appropriate.
>>>
>>> Yes please, that would be awesome.
>>>
>>> Andrei
>>
>> I've spent some time on it and have created a Win32 druntime DLL. Stay
>> tuned for some pull requests to come after a few cleanups...
>
> Sorry for the delay, I couldn't work on it for a couple of days and faced an unexpected problem with relocation to data:
>
> IMO using the import functionality of "export" to support data imports through the import table is not an option as it pollutes the code (you might even need two different sources when building against a shared or static library) and introduces restrictions (like not being able to use the address of an imported symbol in static data initializers). That's why I am following the idea to patch code to refer to data in the shared library.
>
> - The old patch in bugzilla walked the relocation information in the executable and analyzed the data: if it refers to some entry in the import table it gets patched with the actual address in the shared library.
>
> - Unfortunately this does not work when there is an offset applied to the relocation, because you can no longer identify the entry in the import table. I implemented another indirection through some extra data to detect offsets within the referred data, but that's both ugly and unsafe in case the optimizer generates code with offsets outside the original data range.
>
> - Instead, I patched dmd to emit extra relocation information to be collected in some data segment. This works for references from data, but as functions are generated into comdats the only sensible way to add this data is to use comdats aswell. Unfortunately again, comdats cannot be assigned to a separate segment according to the OMF spec, so they are merged with other data and you cannot identify the relocation information in the image anymore.
>
> - For COFF, there is the concept of "associated" segments (also used by ".pdata" sections), that are added to the binary if the other segment is linked in. Implementing this for Win64 revealed another problem: code is generated with PC-relative data accesses, so patching the code to refer to a DLL far away in the 64-bit address space does not work.
>
> So I ended up with only Win32-COFF working with this concept so far...
>
> Questions:
> - Is there a way to have something similar as "associated" segments with the OMF object format?
>
> - Is it possible to force generation of absolute accesses to external data in Win64?
>
> - What is happening on linux? I guess there is no import table there so relocations are pointing directly into the shared library, but what about PC relative adressing?
>
> - Is there something wrong with the whole concept?

Yes. I'd just stick with the way DLL imports are designed to work, which is the extra level of indirection, and just deal with any restrictions that come out of that.
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On 03/01/13 18:38, Rainer Schuetze wrote:
> - What is happening on linux? I guess there is no import table there so relocations are pointing directly into the shared library, but what about PC relative adressing?

http://www.iecc.com/linker/linker10.html

artur
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On 05.03.2013 16:35, Artur Skawina wrote:
> On 03/01/13 18:38, Rainer Schuetze wrote:
>> - What is happening on linux? I guess there is no import table there so relocations are pointing directly into the shared library, but what about PC relative adressing?
>
> http://www.iecc.com/linker/linker10.html
>
> artur
>

Thanks for the link. The article seems to be a bit old, though (1999, no 64-bit, seems to compare to XP). Not sure how much has changed since.

It isn't clear from the initial description, but the comparison at the end gives me the impression that ELF allows direct data relocations into another shared library very much like what I've tried to implement for Windows.

I've tried generating some ELF object files (under windows by compiling for TARGET_LINUX), but I don't see any code generated to support 64-bit data addresses. How is this going to work?

_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime