On 12/10/2014 4:15 AM, Paulo Pinto wrote:
> I prefer the model used by the referred languages, where binary libraries and
> metadata is used, instead of the C toolchain model.
>
> For example, just shipping the .TPU/.DCU libraries in the Object Pascal world.

If the metadata had enough info in it to do inlining, it might as well be the source code.

On 10 Dec 2014 18:30, "H. S. Teoh via Digitalmars-d" <digitalmars-d@puremagic.com> wrote:
>
> On Wed, Dec 10, 2014 at 06:15:48PM +0000, Paulo Pinto via Digitalmars-d wrote:
> > On Wednesday, 10 December 2014 at 16:56:24 UTC, Iain Buclaw via Digitalmars-d wrote:
> [...]
> > >In D, this should be akin to:
> > >
> > >// Package header
> > >module functions;
> > >void Swap(T)(out T x, out T y);
> > >
> > >// Package body
> > >module functions;
> > >void Swap(T)(out T x, out T y)
> > >{
> > >  // Implementation
> > >}
> > >
> > >// Importing it
> > >import functions : Swap;
> > >void main()
> > >{
> > >  int x = 1;
> > >  int y = 2;
> > >  Swap(x, y);
> > >}
> > >
> > >Iain
> >
> > But the current object model doesn't support it, right?
> >
> > At least my understanding is that you need to have the full body visible.
> [...]
>
> Yeah, the compiler cannot instantiate the template without access to the full body. It *could*, though, if we were to store template body IR in object files, perhaps under specially-dedicated object file sections. It wouldn't prevent reverse-engineering (which is moot anyway when templates are involved), but it *would* work as an "opaque" library interface file.
>

So long as it's instantiated somewhere in the provided by the library object shipped with the module interface, then all symbols will resolve at link-time.

I can't imagine Ada being much different at the object level. To even quote from a book of Ada that covers information hiding in a section (changing the function names to be relevant for this discussion).

"""
In the above example, the full definition of Swap can be indeed
deferred until the package body.  The reason, of course, is that
nearly all current machines have a uniform addressing structure, so
that an access value always looks the same regardless of what it is
designating.

To summarise, the logical interface corresponds to the visible part; the physical interface corresponds to the complete package specification, that is, to both the visible part and the private part.

As long as a package specification is not changed, the package body
that implements it can be defined and redefined without affecting
other units that use this specification as an interface to the
package.  Hence it is possible to compile a package body separately
from its package specification.
"""

Now if you swap 'package specification' for 'function/template signature', you've got yourself more or less describing how D modules/packages work.

Iain.

On Thu, Dec 11, 2014 at 10:17:29AM +1100, Daniel Murphy via Digitalmars-d wrote:
> "H. S. Teoh via Digitalmars-d"  wrote in message news:mailman.3042.1418240846.9932.digitalmars-d@puremagic.com...
> 
> >Also, storing a full AST is probably overkill -- lexing and parsing the source generally doesn't take up too much of the compiler's time, so we might as well just use the source code instead.
> 
> Exactly
> 
> >What makes it more worthwhile is if the AST has already been somewhat processed, e.g., constants have been folded, etc.. Probably after semantic1 and semantic2 have been run (though I'm not sure how far one can get if the template hasn't been instantiated yet).
> 
> Not very far at all.
> 
> >This way, work that has already been done doesn't need to be repeated again.
> 
> When it's templates that haven't been instantiated, you haven't done any of the work yet.

Well, I was thinking more of how far we *could* go, rather than how far dmd *actually* goes currently (which is nowhere at all, since all it does right now is to parse the template, until you instantiate it).

But I suspect you can't go too far -- at least, not if the code actually depends on the template arguments in any meaningful way. As an extreme case, Dicebot's example is one where you can't go anywhere at all:

	auto myFunc(string code)() {
		return mixin(code);
	}

On the other extreme, you have templates that really shouldn't be templates because they don't actually depend on their template arguments:

	auto myFunc(A...)() {
		return 1+2*3/4-5;
	}

You could basically already compile the entire function without caring for the template arguments.

Real-life use cases, of course, generally fall somewhere in between these two extremes. So I'd expect they would have some parts that cannot be processed any further than parsing, and other parts that can ostensibly go quite far, depending on how independent they are of the template arguments.

Hmm, on second thoughts, this seems to be an interesting direction to explore, because template code that you *can* get quite far on, also represents code that is quite independent of template arguments, which means a large part of them should be identical across different template arguments. That makes them candidates for being merged, which would help reduce template bloat. By attempting analysis of template bodies, the compiler might be able to automatically identify these "mostly-independent" pieces of code, and perhaps apply some strategies for automatic code merging.


T

-- 
Windows 95 was a joke, and Windows 98 was the punchline.

On 12/10/2014 10:28 AM, H. S. Teoh via Digitalmars-d wrote:
> Yeah, the compiler cannot instantiate the template without access to the
> full body. It *could*, though, if we were to store template body IR in
> object files, perhaps under specially-dedicated object file sections. It
> wouldn't prevent reverse-engineering (which is moot anyway when
> templates are involved), but it *would* work as an "opaque" library
> interface file.

Storing it as body IR accomplishes nothing practical over storing it as source file, i.e. .di files.

On Wed, 10 Dec 2014 17:17:11 -0800
Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> On 12/10/2014 10:28 AM, H. S. Teoh via Digitalmars-d wrote:
> > Yeah, the compiler cannot instantiate the template without access to the full body. It *could*, though, if we were to store template body IR in object files, perhaps under specially-dedicated object file sections. It wouldn't prevent reverse-engineering (which is moot anyway when templates are involved), but it *would* work as an "opaque" library interface file.
> 
> Storing it as body IR accomplishes nothing practical over storing it as source file, i.e. .di files.
except that there's no need to parse source code over and over again, which is good for other tools (like completion suggesting, intelligent code browsing and so on).

On Thursday, 11 December 2014 at 08:05:13 UTC, ketmar via Digitalmars-d wrote:
> On Wed, 10 Dec 2014 17:17:11 -0800
> Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
>> On 12/10/2014 10:28 AM, H. S. Teoh via Digitalmars-d wrote:
>> > Yeah, the compiler cannot instantiate the template without access to the
>> > full body. It *could*, though, if we were to store template body IR in
>> > object files, perhaps under specially-dedicated object file sections. It
>> > wouldn't prevent reverse-engineering (which is moot anyway when
>> > templates are involved), but it *would* work as an "opaque" library
>> > interface file.
>> 
>> Storing it as body IR accomplishes nothing practical over storing it as source file, i.e. .di files.
> except that there's no need to parse source code over and over again,
> which is good for other tools (like completion suggesting, intelligent
> code browsing and so on).

Yes tooling is a big part of it.

--
Paulo

>> 
>> Storing it as body IR accomplishes nothing practical over storing it as source file, i.e. .di files.
> except that there's no need to parse source code over and over again,
> which is good for other tools (like completion suggesting, intelligent
> code browsing and so on).

Which usually hold an AST in memory anyway. We have a fast parser, parsing even a big codebase once is really not a problem, see DCD for example.

If the only advantage is to skip a parsing stage here or there, it does not justify the work that would be needed.

On Thu, 11 Dec 2014 08:57:56 +0000
Tobias Pankrath via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> >> 
> >> Storing it as body IR accomplishes nothing practical over storing it as source file, i.e. .di files.
> > except that there's no need to parse source code over and over
> > again,
> > which is good for other tools (like completion suggesting,
> > intelligent
> > code browsing and so on).
> 
> Which usually hold an AST in memory anyway. We have a fast parser, parsing even a big codebase once is really not a problem, see DCD for example.
> 
> If the only advantage is to skip a parsing stage here or there, it does not justify the work that would be needed.
as we have a fast compiler too, i can't see any sense in producing machine code files at all. the only advantage is to skip a parsing and compiling stages here or there, it does not justify the work that would be needed.

>> 
>> Which usually hold an AST in memory anyway. We have a fast parser, parsing even a big codebase once is really not a problem, see DCD for example.
>> 
>> If the only advantage is to skip a parsing stage here or there, it does not justify the work that would be needed.
> as we have a fast compiler too, i can't see any sense in producing
> machine code files at all. the only advantage is to skip a parsing and
> compiling stages here or there, it does not justify the work that would
> be needed.

Come on, that is not even a half decent analogy.

On Thu, 11 Dec 2014 09:18:05 +0000
Tobias Pankrath via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> >> Which usually hold an AST in memory anyway. We have a fast parser, parsing even a big codebase once is really not a problem, see DCD for example.
> >> 
> >> If the only advantage is to skip a parsing stage here or there, it does not justify the work that would be needed.
> > as we have a fast compiler too, i can't see any sense in
> > producing
> > machine code files at all. the only advantage is to skip a
> > parsing and
> > compiling stages here or there, it does not justify the work
> > that would
> > be needed.
> Come on, that is not even a half decent analogy.
it is. you can't see any uses of (semi)compiled module files (and i
can; it's essential for component framework, for example). i can't see
any uses of compiled binaries (i don't need that in component
framework).