View mode: basic / threaded / horizontal-split · Log in · Help
August 17, 2012
Re: Fragile ABI
Paulo -

Surely there are ways to work around the OS's native ABI, though, 
even from within compiled code?

Rewriting object vtables (etc.) might be a bit more involved than 
simple name mangling, but I'm almost certain it would be possible 
to implement it as part of druntime. The blog post which I linked 
describes one way to go about it.

Michael -

As far as I can tell, most of the performance cost in your idea 
comes from the additional level of indirection, since you store 
vtable offsets in a global variable which must be read before 
each function call.

This might just be wishful thinking, but would it be possible for 
druntime to write new vtable offsets directly into the program's 
machine code, at dynamic-link time? That would remove all of the 
run-time performance overhead, but, as I say, I'm not sure 
whether it's actually possible.

I think the biggest obstacle is that it would require the .text 
section (or equivalent) of the executable to be writable. I know 
that's possible on Linux, but I'm not sure whether the same is 
true for most other operating systems. I guess it might also be a 
security risk?
August 17, 2012
Re: Fragile ABI
On 2012-08-17 12:36:56 +0000, "R Grocott" <rgrocottbugzilla@gmail.com> said:

> Michael -
> 
> As far as I can tell, most of the performance cost in your idea comes 
> from the additional level of indirection, since you store vtable 
> offsets in a global variable which must be read before each function 
> call.
> 
> This might just be wishful thinking, but would it be possible for 
> druntime to write new vtable offsets directly into the program's 
> machine code, at dynamic-link time? That would remove all of the 
> run-time performance overhead, but, as I say, I'm not sure whether it's 
> actually possible.
> 
> I think the biggest obstacle is that it would require the .text section 
> (or equivalent) of the executable to be writable. I know that's 
> possible on Linux, but I'm not sure whether the same is true for most 
> other operating systems. I guess it might also be a security risk?

It's certainly doable if you put it at the right place. But druntime is 
not the one in charge of dynamic linking: the dynamic linker from the 
OS is, so that's the ideal place to do such a thing. If you want to do 
it in druntime it'll be a huge hassle, if doable at all it'll probably 
break easily. But on the OS side you hit a rock because of different 
approach to dynamic linking (Windows does not really have a dynamic 
linker).

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca/
August 17, 2012
Re: Fragile ABI
On Friday, 17 August 2012 at 12:36:57 UTC, R Grocott wrote:
> Paulo -
>
> Surely there are ways to work around the OS's native ABI, 
> though, even from within compiled code?
>
> Rewriting object vtables (etc.) might be a bit more involved 
> than simple name mangling, but I'm almost certain it would be 
> possible to implement it as part of druntime. The blog post 
> which I linked describes one way to go about it.
>
> Michael -
>
> As far as I can tell, most of the performance cost in your idea 
> comes from the additional level of indirection, since you store 
> vtable offsets in a global variable which must be read before 
> each function call.
>
> This might just be wishful thinking, but would it be possible 
> for druntime to write new vtable offsets directly into the 
> program's machine code, at dynamic-link time? That would remove 
> all of the run-time performance overhead, but, as I say, I'm 
> not sure whether it's actually possible.
>
> I think the biggest obstacle is that it would require the .text 
> section (or equivalent) of the executable to be writable. I 
> know that's possible on Linux, but I'm not sure whether the 
> same is true for most other operating systems. I guess it might 
> also be a security risk?

It's possible to modify the vtable at runtime. Just access 
through .__vtable in ClassInfo (I think) and treat it like a 
regular array. Don't know if that's what you mean.

--
/Jacob Carlborg
August 17, 2012
Re: Fragile ABI
On Thursday, 16 August 2012 at 20:19:27 UTC, R Grocott wrote:

> Likewise, WinRT actually hides a huge amount of complexity 
> inside its language bindings. Inheriting from a WinRT object 
> using only the underlying COM interfaces involves a lot of 
> hassle, and is a prime example of what I'm talking about. See 
> here:
>
> http://www.interact-sw.co.uk/iangblog/2011/09/25/native-winrt-inheritance

Someone has written a piece of code making it easy to use WinRT 
from D. I don't have the link but it should be possible to find 
in some of the newsgroups.

--
/Jacob Carlborg
August 17, 2012
Re: Fragile ABI
On Friday, 17 August 2012 at 13:42:27 UTC, Michel Fortin wrote:
> On 2012-08-17 12:36:56 +0000, "R Grocott" [..] (Windows does 
> not really have a dynamic linker).

Why? What is it lacking?

From my experience, I don't have much to complain about it.

Just wanting to know, not trolling.

--
Paulo
August 17, 2012
Re: Fragile ABI
On Friday, 17 August 2012 at 13:42:27 UTC, Michel Fortin wrote:
> It's certainly doable if you put it at the right place. But 
> druntime is not the one in charge of dynamic linking: the 
> dynamic linker from the OS is, so that's the ideal place to do 
> such a thing. If you want to do it in druntime it'll be a huge 
> hassle, if doable at all it'll probably break easily. But on 
> the OS side you hit a rock because of different approach to 
> dynamic linking (Windows does not really have a dynamic linker).

So basically, any system to create a more sensible ABI needs to 
built on top of the C ABI, name mangling, editable vtables, and 
nothing else? Ouch.

If druntime can't detect whether symbols were dynamically or 
statically linked, does that mean that your system would add an 
overhead to *all* virtual function calls, not just those which 
call into a dynamic library? If so, that'd be a pretty strong 
argument against implementing anything like it, unless it's 
something which has to be explicitly switched on (using, say, an 
"extern(D_FlexibleABI)" statement).
August 17, 2012
Re: Fragile ABI
On 2012-08-17 14:13:10 +0000, "R Grocott" <rgrocottbugzilla@gmail.com> said:

> On Friday, 17 August 2012 at 13:42:27 UTC, Michel Fortin wrote:
>> It's certainly doable if you put it at the right place. But druntime is 
>> not the one in charge of dynamic linking: the dynamic linker from the 
>> OS is, so that's the ideal place to do such a thing. If you want to do 
>> it in druntime it'll be a huge hassle, if doable at all it'll probably 
>> break easily. But on the OS side you hit a rock because of different 
>> approach to dynamic linking (Windows does not really have a dynamic 
>> linker).
> 
> So basically, any system to create a more sensible ABI needs to built 
> on top of the C ABI, name mangling, editable vtables, and nothing else? 
> Ouch.

That's pretty much it yes.

> If druntime can't detect whether symbols were dynamically or statically 
> linked, does that mean that your system would add an overhead to *all* 
> virtual function calls, not just those which call into a dynamic 
> library?

Yes. Neither druntime nor the compiler knows whether you're creating an 
executable. The typical compilation process is to convert D code to an 
object file containing machine code. Then that object file is linked 
either inside an executable or inside a shared library. But the machine 
code is written before the compiler knows where it'll go.

> If so, that'd be a pretty strong argument against implementing anything 
> like it, unless it's something which has to be explicitly switched on 
> (using, say, an "extern(D_FlexibleABI)" statement).

Exactly my thoughts.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca/
August 17, 2012
Re: Fragile ABI
On 2012-08-17 14:00:42 +0000, "Paulo Pinto" <pjmlp@progtools.org> said:

> On Friday, 17 August 2012 at 13:42:27 UTC, Michel Fortin wrote:
>> On 2012-08-17 12:36:56 +0000, "R Grocott" [..] (Windows does not really 
>> have a dynamic linker).
> 
> Why? What is it lacking?
> 
>  From my experience, I don't have much to complain about it.
> 
> Just wanting to know, not trolling.

Quote from <http://xenophilia.org/winvunix.html>:

> In Unix, a shared object (.so) file contains code to be used by the 
> program, and also the names of functions and data that it expects to 
> find in the program. When the file is joined to the program, all 
> references to those functions and data in the file's code are changed 
> to point to the actual locations in the program where the functions and 
> data are placed in memory. This is basically a link operation.
> In Windows, a dynamic-link library (.dll) file has no dangling 
> references. Instead, an access to functions or data goes through a 
> lookup table. So the DLL code does not have to be fixed up at runtime 
> to refer to the program's memory; instead, the code already uses the 
> DLL's lookup table, and the lookup table is modified at runtime to 
> point to the functions and data.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca/
August 17, 2012
Re: Fragile ABI
On Friday, 17 August 2012 at 17:42:24 UTC, Michel Fortin wrote:
> On 2012-08-17 14:00:42 +0000, "Paulo Pinto" 
> <pjmlp@progtools.org> said:
>
>> On Friday, 17 August 2012 at 13:42:27 UTC, Michel Fortin wrote:
>>> On 2012-08-17 12:36:56 +0000, "R Grocott" [..] (Windows does 
>>> not really have a dynamic linker).
>> 
>> Why? What is it lacking?
>> 
>> From my experience, I don't have much to complain about it.
>> 
>> Just wanting to know, not trolling.
>
> Quote from <http://xenophilia.org/winvunix.html>:
>
>> In Unix, a shared object (.so) file contains code to be used 
>> by the program, and also the names of functions and data that 
>> it expects to find in the program. When the file is joined to 
>> the program, all references to those functions and data in the 
>> file's code are changed to point to the actual locations in 
>> the program where the functions and data are placed in memory. 
>> This is basically a link operation.
>> In Windows, a dynamic-link library (.dll) file has no dangling 
>> references. Instead, an access to functions or data goes 
>> through a lookup table. So the DLL code does not have to be 
>> fixed up at runtime to refer to the program's memory; instead, 
>> the code already uses the DLL's lookup table, and the lookup 
>> table is modified at runtime to point to the functions and 
>> data.

For this just looks at two different ways to implement dynamic 
loading.

Having Windows experience since the Windows 3.1 days, alongside 
with
many other OS besides the typical Linux fan, I always find sad 
this vision
of Linux == UNIX, right, Windows wrong.

The author also recognises that Aix, which is UNIX, also has 
strange
dynamic loading rules, according to his understanding what 
dynamic loading
is.

--
Paulo
August 18, 2012
Re: Fragile ABI
On 2012-08-17 20:49:21 +0000, "Paulo Pinto" <pjmlp@progtools.org> said:

> On Friday, 17 August 2012 at 17:42:24 UTC, Michel Fortin wrote:
>> Quote from <http://xenophilia.org/winvunix.html>:
>> 
>>> In Unix, a shared object (.so) file contains code to be used by the 
>>> program, and also the names of functions and data that it expects to 
>>> find in the program. When the file is joined to the program, all 
>>> references to those functions and data in the file's code are changed 
>>> to point to the actual locations in the program where the functions and 
>>> data are placed in memory. This is basically a link operation.
>>> In Windows, a dynamic-link library (.dll) file has no dangling 
>>> references. Instead, an access to functions or data goes through a 
>>> lookup table. So the DLL code does not have to be fixed up at runtime 
>>> to refer to the program's memory; instead, the code already uses the 
>>> DLL's lookup table, and the lookup table is modified at runtime to 
>>> point to the functions and data.
> 
> For this just looks at two different ways to implement dynamic loading.
> 
> Having Windows experience since the Windows 3.1 days, alongside with
> many other OS besides the typical Linux fan, I always find sad this vision
> of Linux == UNIX, right, Windows wrong.

I never intended to mean Windows == wrong. What I am saying is that 
Windows does not have a dynamic linker capable of changing pointers to 
symbols in the code it loads (which is what a linker does). Code 
compiled for Windows need instead to refer to a separate lookup table 
when accessing symbols from others DLLs. Neither approach is wrong; 
each has different tradeoffs.

Implementing a non-fragile ABI would be possible using a lookup table 
of runtime calculated values for field and vtable offsets, and those 
could be implemented both through a dynamic linker with DLLs, but 
accessing the lookup table adds some runtime overhead each time you 
need to access this symbol in the code because of the indirection.

In the original linked article (which I wrote) what was proposed was to 
have the dynamic linker calculate offsets for fields and vtable entries 
and insert those offsets directly in the code (just like a linker does 
when it resolves symbols). But for that you'd need a custom linker 
(both static and dynamic), and probably a custom shared library format. 
So it's a huge task, especially when you consider that it should run on 
multiple platforms. But this same approach could make the C++ ABI 
non-fragile too.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca/
1 2 3 4 5
Top | Discussion index | About this forum | D home