View mode: basic / threaded / horizontal-split · Log in · Help
January 16, 2012
Re: output minimal .di files?
On 16-01-2012 21:08, H. S. Teoh wrote:
> On Mon, Jan 16, 2012 at 11:38:15AM -0800, Adam Wilson wrote:
> [...]
>> I would say the main reason for using .h/.di files in libraries is
>> that the library designer does not want his implementation public
>> viewable. And in D, unlike C/C++, .di files are pretty much exclusive
>> to the concept of libraries. I'd say that, based on how many questions
>> are raised about .di files, almost no one expects the current
>> behavior, I certainly didn't, hence my patch. The DI generation patch
>> currently implements the C++ paradigm, where templated function
>> implementations are publicly viewable, but non-templated function
>> implementations are not. I feel that this paradigm, being the
>> currently accepted convention, is the best path for D to take.
> [...]
>
> But if you remove function bodies from inline-able functions, then your
> library loses out on potential optimization by the compiler. Besides,
> all your templates are still world-readable, which, depending on what
> your library is, may pretty much comprise your entire library anyway.
>
> To *truly* have separation of API from implementation, interface files
> shouldn't even have templated functions. It should list ONLY public
> declarations, no private members, no function bodies, no template
> bodies, etc..  All function bodies, including inline functions, template
> bodies, private members, etc., should be in a binary format readable
> only by the compiler.
>
> One way to implement this is to store template/inline function bodies
> inside the precompiled object files as extra info that the compiler
> loads in order to be able to expand templates/inline functions, compute
> the size of structs/classes (because private members are not listed in
> the API file), and so on. How this is feasible to implement, I can't
> say; some platforms may not allow arbitrary data inside object files, so
> the compiler may not be able to store the requisite information in them.
>
>
> T
>

I... don't think the error messages from expanding raw object code would 
be very pleasant to read, if you used a template incorrectly...

-- 
- Alex
January 16, 2012
Re: output minimal .di files?
On Mon, Jan 16, 2012 at 09:32:57PM +0100, Alex Rønne Petersen wrote:
[...]
> I... don't think the error messages from expanding raw object code
> would be very pleasant to read, if you used a template incorrectly...
[...]

It doesn't have to be *executable* object code; the compiler may store
extra info (perhaps as debugging data?) so that it can generate nicer
error messages.

But like I said, this assumes the compiler is allowed to store arbitrary
data inside object files, which may not be the case on some platforms.


T

-- 
Which is worse: ignorance or apathy? Who knows? Who cares? -- Erich Schubert
January 16, 2012
Re: output minimal .di files?
On Mon, Jan 16, 2012 at 12:32:01PM -0800, Adam Wilson wrote:
> On Mon, 16 Jan 2012 12:08:53 -0800, H. S. Teoh
> <hsteoh@quickfur.ath.cx> wrote:
[...]
> >One way to implement this is to store template/inline function bodies
> >inside the precompiled object files as extra info that the compiler
> >loads in order to be able to expand templates/inline functions,
> >compute the size of structs/classes (because private members are not
> >listed in the API file), and so on. How this is feasible to
> >implement, I can't say; some platforms may not allow arbitrary data
> >inside object files, so the compiler may not be able to store the
> >requisite information in them.
[...]
> Not a bad idea, it's similar in function to .NET's Metadata.
> unfortunately to be useful, other linkers would have to be taught how
> to read that data...
[...]

That depends on how you do it.

If we assume, for argument's sake, that we are allowed to store
arbitrary data inside an object file (say inside a debug section or
something), then the compiler could for example store things like parsed
function bodies, partial syntax trees, etc., that allows it to treat
templates and inline functions as though they actually were embedded in
the interface file. When asked to compile a source file that imports the
library, the compiler would read the object file, extract this info and
use it to do whatever it needs to do (expand templates, inline
functions, emit non-inlined function bodies, etc.).

The generated object file can then be linked by the system's usual
linker, assuming that the extra info in the library's object file is
marked such that the linker simply ignores it. The linker doesn't have
to know anything about it because the compiler has already done whatever
needs to be done with it when it compiled the source file that imported
the library.

In fact, if a particular platform doesn't support such extra data inside
object files, the compiler can simply save the data in its own internal
format in another file, and the library writer just ships this file
along with the human-readable API file and any precompiled library
object files. As long as the customer's compiler knows to look for this
file when compiling source that imports the library, it will have enough
info to do what it needs to do.


T

-- 
Everybody talks about it, but nobody does anything about it!  -- Mark Twain
January 16, 2012
Re: output minimal .di files?
On 01/16/2012 09:40 PM, H. S. Teoh wrote:
> On Mon, Jan 16, 2012 at 09:32:57PM +0100, Alex Rønne Petersen wrote:
> [...]
>> I... don't think the error messages from expanding raw object code
>> would be very pleasant to read, if you used a template incorrectly...
> [...]
>
> It doesn't have to be *executable* object code; the compiler may store
> extra info (perhaps as debugging data?) so that it can generate nicer
> error messages.
>
> But like I said, this assumes the compiler is allowed to store arbitrary
> data inside object files, which may not be the case on some platforms.
>
>
> T
>

How would your proposal help hiding implementation details of templates 
anyway? All information still needs to be stored. Anyone could write a 
object file -> source file compiler for template implementations.
January 17, 2012
Re: output minimal .di files?
On Tue, Jan 17, 2012 at 12:17:18AM +0100, Timon Gehr wrote:
> On 01/16/2012 09:40 PM, H. S. Teoh wrote:
> >On Mon, Jan 16, 2012 at 09:32:57PM +0100, Alex Rønne Petersen wrote:
> >[...]
> >>I... don't think the error messages from expanding raw object code
> >>would be very pleasant to read, if you used a template incorrectly...
> >[...]
> >
> >It doesn't have to be *executable* object code; the compiler may store
> >extra info (perhaps as debugging data?) so that it can generate nicer
> >error messages.
> >
> >But like I said, this assumes the compiler is allowed to store arbitrary
> >data inside object files, which may not be the case on some platforms.
> >
> >
> >T
> >
> 
> How would your proposal help hiding implementation details of
> templates anyway? All information still needs to be stored. Anyone
> could write a object file -> source file compiler for template
> implementations.

It depends on what you understand by "hiding implementation details".

I see it from the point of view of encapsulation taken to its logical
conclusion: users of the library only need to know the API of the
library and nothing else. When they read the interface file, it should
only contain the API and no implementation at all. In this sense,
implementation details are completely hidden -- that's what
encapsulation is all about.

Of course, the *compiler* (and/or linker) obviously needs to know the
full implementation details, otherwise it couldn't possibly produce the
final executable. So this information has to come from somewhere, object
files, compiler internal representation files, what-have-you. I don't
think it's possible, even in theory, to prevent reverse-engineering of
these sources of compiler information. Given enough determination,
*anything* can be reverse engineered.

And neither is prevention of reverse engineering the point.  The point
is that these are sources of *compiler* information, rather than
information for library users. In that sense it's useful to separate
information meant for the compiler, and information meant for users of
the library.


T

-- 
All problems are easy in retrospect.
Next ›   Last »
1 2
Top | Discussion index | About this forum | D home