On 28 September 2012 17:43, Rob T <rob@ucora.com> wrote:
> I am trying out gdc 4.7 branch with -fPIC, but the info I'm getting is that even with a successulf PIC build I will still not be able to reliably execute D functions directly from C/C++ code.
>
>
>> Old patch, which shows what needs to be done:
>>
>> https://bitbucket.org/goshawk/gdc/issue/166/add-shared-lib-support#comment-648329
>
>
> Does anyone know why patches like this are taking so [bleeping] long to get recognized and implemented?
>
> -rt
>

Because the [bleeping] runtime does not support running from a shared library.

-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';

On Friday, 28 September 2012 at 17:07:16 UTC, Iain Buclaw wrote:
>
> Because the [bleeping] runtime does not support running from a shared library.

I suppose the answer is very complicated, but why can't the runtime execute as a shared lib? Is it a design limitation of the runtime model? This sort of problem should have been dealt with from the ground up, not now. What can be done to fix it, or is it a major redesign + re-write effort?

-rt

On Thursday, 27 September 2012 at 17:10:07 UTC, Jacob Carlborg wrote:
> Actually, I seriously doubt everything is working as expected.

I tried to check how TLS, EX, etc. (mostly exposed to dll issue) are working and here is some kind of test: https://github.com/mxfm/sharedtest. Unfortunately scope(exit) isn't executed when it is situated in a shared library and which calls some throwing function from another shared library. Unittests aren't working either. Regarding other parts - they seem to work.

On 28 September 2012 18:54, Rob T <rob@ucora.com> wrote:
> On Friday, 28 September 2012 at 17:07:16 UTC, Iain Buclaw wrote:
>>
>>
>> Because the [bleeping] runtime does not support running from a shared library.
>
>
> I suppose the answer is very complicated, but why can't the runtime execute as a shared lib? Is it a design limitation of the runtime model? This sort of problem should have been dealt with from the ground up, not now. What can be done to fix it, or is it a major redesign + re-write effort?
>
> -rt

The big implementation issue is that the runtime only passes two data ranges to the GC to scan. Global data segment and TLS data segment of the resultant binary/executable.  So any static data stored in shared libraries aren't scanned / recognised as data we want to keep. So the use of them may cause sudden unwanted collections.

The way I intend to address it is to have each module handle it's own gshared/thread data and pass the information to the D runtime during the module construction stage (.ctor) - there is already something done this way for _Dmodule_ref - so it may end up being that two new fields will be tacked onto it; void[] tlsdata, void[] gsharedata;


Regards
-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';

On 2012-09-28 20:25, Maxim Fomin wrote:

> I tried to check how TLS, EX, etc. (mostly exposed to dll issue) are
> working and here is some kind of test:
> https://github.com/mxfm/sharedtest. Unfortunately scope(exit) isn't
> executed when it is situated in a shared library and which calls some
> throwing function from another shared library. Unittests aren't working
> either. Regarding other parts - they seem to work.

That's a fairly uninteresting test. You are linking to the dynamic library. What's interesting is loading a dynamic library using dlopen, or similar. What's the point of using dynamic libraries if you're linking with them?

-- 
/Jacob Carlborg

On 2012-09-28 19:54, Rob T wrote:

> I suppose the answer is very complicated, but why can't the runtime
> execute as a shared lib? Is it a design limitation of the runtime model?
> This sort of problem should have been dealt with from the ground up, not
> now. What can be done to fix it, or is it a major redesign + re-write
> effort?

See one of my other posts:

http://forum.dlang.org/thread/k3vfm9$1tq$1@digitalmars.com?page=3#post-k4219f:24uft:241:40digitalmars.com

-- 
/Jacob Carlborg

On Saturday, 29 September 2012 at 13:19:01 UTC, Jacob Carlborg wrote:
> That's a fairly uninteresting test.

I am not a D developer which means I have no incentive in blindly portraying D as a language having shared libraries support when in fact it has some issues. I am a D user which has incentive to gladly report which part of D does work (or seems to work) and which doesn't when making shared libraries.

> You are linking to the dynamic library. What's interesting is loading a dynamic library using dlopen, or similar. What's the point of using dynamic libraries if you're linking with them?

I was not taking about dynamic loading, but about dynamic linking. If I understand topic right, the issue is the former, not the latter.

BTW, in majority cases dynamic loading gives no advantages over dynamic linking (just the opposite: doing unnecessary job which can be done by linker and loader). In most cases, when an app is being written, it is known which functional from which third-party libraries would be used. The only exceptions I can name are pluggin support and hacker's binary ELF/PE tools. I can count few apps in windows and linux which are used often, which dynamically load something, and if they do, they likely load pluggins. This was made for cases when app developers by definition cannot know beforehand full list of used shared libraries. So, the question is just opposite: "What's the point of using dynamic loading if you know beforehand which libraries you use, which happens almost in all cases?".

But I certainly agree that dynamic loading is essential for some applications, if not irreplaceable, and support of it also needed to be inspected and improved.

On 2012-09-29 17:40, Maxim Fomin wrote:

> I was not taking about dynamic loading, but about dynamic linking. If I
> understand topic right, the issue is the former, not the latter.

The title of the thread says "... and loading".

> BTW, in majority cases dynamic loading gives no advantages over dynamic
> linking (just the opposite: doing unnecessary job which can be done by
> linker and loader). In most cases, when an app is being written, it is
> known which functional from which third-party libraries would be used.

I agree, in these cases their no point in using dynamic loading.

> The only exceptions I can name are pluggin support and hacker's binary
> ELF/PE tools. I can count few apps in windows and linux which are used
> often, which dynamically load something, and if they do, they likely
> load pluggins. This was made for cases when app developers by definition
> cannot know beforehand full list of used shared libraries.

Exactly, pluggins are one of the main reasons for using dynamic loading. An other reason is license issues. There's a D library called Derelict which uses dynamic loading to load C libraries (OpenGL, SDL, OpenAL and others). One of the many reason for Derelict using dynamic loading is due to license issues/complications with these libraries.

http://dsource.org/projects/derelict/

> So, the question is just opposite: "What's the point of using dynamic loading if
> you know beforehand which libraries you use, which happens almost in all
> cases?".

There's no point in doing that. But what I'm saying is if you know beforehand the libraries you will use you can get quite far with static libraries.

-- 
/Jacob Carlborg

On Saturday, 29 September 2012 at 16:09:14 UTC, Jacob Carlborg wrote:
> There's no point in doing that. But what I'm saying is if you know beforehand the libraries you will use you can get quite far with static libraries.

There are plenty of cases where you have to use a dynamically loaded lib even if you know before hand what will be loaded. I think you understand this if I read your posts correctly.

In my case I have a pre-existing C++ app that is designed to load user defined C++ plugins. I wanted to interface D to one of my own C++ plugins, but I apparently cannot do it reliably because of initialization issues with the GC and possibly some other obscure items.

If I can figure out what needs to be done to resolve the problem in enough detail, then maybe I can hack the runtime source and roll out a solution.

The GC always seems to pop up as a source of problems. For long term solution, the GC should be made 100% optional (in practice as opposed to in theory), the standard library ought to be made to work with or wothout a GC (or simply without), and the GC itself should be easily replaceable with alternate versions. I think this idea has already been discussed elsewhere, and is on the TODO list (I hope!).

--rt

On Friday, 28 September 2012 at 21:26:47 UTC, Iain Buclaw wrote:
> The way I intend to address it is to have each module handle it's own
> gshared/thread data and pass the information to the D runtime during
> the module construction stage (.ctor) - there is already something
> done this way for _Dmodule_ref - so it may end up being that two new
> fields will be tacked onto it; void[] tlsdata, void[] gsharedata;

I re-built libgphobos and libgdruntime with -fPIC and I can now successfully create dynamically loaded D libs. I have successfully linked a dynamic D lib to a C++ app, executing some test code successfully. I have not yet tried to dlopen a dynamic D lib from C++, but I will try maybe tomorrow.

My simple dynamic lib test seems to run fine, but I understand that there may be problems, such as the GC failing to work properly, and perhaps more can/will go wrong as per Jacob's post
http://forum.dlang.org/post/k4219f$uft$1@digitalmars.com

Any idea when/if you will get around to implementing a fix? Wish I could help but I've only just started looking at the source code, so whatever I try to fix will probably cause more harm than good for a while (but it's a start).

--rt