On Wednesday, 10 December 2014 at 08:43:49 UTC, Kagamin wrote:
> On Tuesday, 9 December 2014 at 20:55:51 UTC, Dicebot wrote:
>> Because you don't really create a template that way but workaround broken function behavior. It is not the usage of empty templates that is bad but the fact that plain functions remain broken => not really a solution.
>
> You can compile against phobos sources instead of interface files.

As far as I understand gdc/ldc problem the way it currently works there is no difference between .di and .d files - if function is not a template its binary is expected to be found in matching object file - and if that object file belongs to prebuild static lib it is completely out of question, even if the sources are completely available.

I have never understood exactly what in frontend makes it that much of a problem though

On Wednesday, 10 December 2014 at 14:16:47 UTC, Paulo  Pinto wrote:
> The libraries contain the required metadata for symbol tables and code locations that need to be extracted into the executable/library.
>
> Package definition files contain the minimum information the compiler needs to know to search for the remaining information.
>
> Example,
>
> ...


Example shows generics, not templates. Full blown template/mixin support is impossible without anything else but full source access in one form or another.

On Wed, Dec 10, 2014 at 07:00:24PM +0000, Tobias Pankrath via Digitalmars-d wrote:
> >>// my code
> >>foo!(ArcaneType1, DubiousType2)(a, d);
> >
> >That's why the current object file model doesn't work very well.
> >
> >You'd have to extend the object file format to include compiler IR for templates, then the compiler can instantiate templates from that IR without needing access to the source. Which is a feature I've brought up several times, but nobody seems to be interested in doing anything about it.
> >
> >
> >T
> 
> What information would / could that IR contain besides an AST?

It could include additional attributes computed by the compiler that could help in later optimization, such as whether the function escapes references, any inferred function attributes, etc.. The compiler could recompute all this from the IR, of course, but if it's already computed before, why not just reuse the previous results.

Also, storing a full AST is probably overkill -- lexing and parsing the source generally doesn't take up too much of the compiler's time, so we might as well just use the source code instead. What makes it more worthwhile is if the AST has already been somewhat processed, e.g., constants have been folded, etc.. Probably after semantic1 and semantic2 have been run (though I'm not sure how far one can get if the template hasn't been instantiated yet). This way, work that has already been done doesn't need to be repeated again.


T

-- 
In theory, software is implemented according to the design that has been carefully worked out beforehand. In practice, design documents are written after the fact to describe the sorry mess that has gone on before.

On Wed, Dec 10, 2014 at 08:33:22PM +0100, Jacob Carlborg via Digitalmars-d wrote:
> On 2014-12-10 18:43, H. S. Teoh via Digitalmars-d wrote:
> 
> >That's why the current object file model doesn't work very well.
> >
> >You'd have to extend the object file format to include compiler IR for templates, then the compiler can instantiate templates from that IR without needing access to the source. Which is a feature I've brought up several times, but nobody seems to be interested in doing anything about it.
> 
> Can't you just put it in a custom section? Or perhaps that what you're saying. Although, I'm not sure if OMF supports custom sections.
[...]

That *is* what I'm saying. But so far, it seems people aren't that interested in doing that. *shrug*


T

-- 
The volume of a pizza of thickness a and radius z can be described by the following formula: pi zz a. -- Wouter Verhelst

On Wednesday, 10 December 2014 at 19:24:17 UTC, Dicebot wrote:
> On Wednesday, 10 December 2014 at 14:16:47 UTC, Paulo  Pinto wrote:
>> The libraries contain the required metadata for symbol tables and code locations that need to be extracted into the executable/library.
>>
>> Package definition files contain the minimum information the compiler needs to know to search for the remaining information.
>>
>> Example,
>>
>> ...
>
>
> Example shows generics, not templates. Full blown template/mixin support is impossible without anything else but full source access in one form or another.

That was just an example, I could have written lots of other stuff.

-
Paulo

On Wednesday, 10 December 2014 at 18:16:54 UTC, Paulo Pinto wrote:
> Simple, by dropping C based linker model as I state on my comment.
>

Ho please, that is a salesman answer, not an engineer one.

On Wednesday, 10 December 2014 at 21:59:57 UTC, deadalnix wrote:
> On Wednesday, 10 December 2014 at 18:16:54 UTC, Paulo Pinto wrote:
>> Simple, by dropping C based linker model as I state on my comment.
>>
>
> Ho please, that is a salesman answer, not an engineer one.

I was talking how the toolchains for other programming languages work.

It is complete clear to me this wouldn't work for D, besides there are lots of more important areas to improve on.

As a language geek that just lurks and works in other languages, I don't have anything to sell.

--
Paulo

On Wednesday, 10 December 2014 at 21:39:42 UTC, Paulo Pinto wrote:
> That was just an example, I could have written lots of other stuff.

Then please show something that actually helps and is applicable to D template system. There is not much value in vague references with "imagine rest yourself" flavor.

To be specific I am interested how would it handle pattern like this (very common for D code and not present in Ada at all AFAIK):

void foo(T)(T t)
{
    mixin(generator!T());
}

What exactly would one put in IR for plain uninstantiated foo?

On Wednesday, 10 December 2014 at 22:34:50 UTC, Dicebot wrote:
> Then please show something that actually helps and is applicable to D template system. There is not much value in vague references with "imagine rest yourself" flavor.
>
> To be specific I am interested how would it handle pattern like this (very common for D code and not present in Ada at all AFAIK):
>
> void foo(T)(T t)
> {
>     mixin(generator!T());
> }
>
> What exactly would one put in IR for plain uninstantiated foo?

Maybe you should agree on what a high level IR is first? Here are some alternatives:

1. Regular IR: A high level IR basically is a flattened AST where you do substitutions, perform partial evaluation and strip out symbols. In a high level IR that supports templating you might preserve several partial instances using incomplete types. If it is non-transformable to begin with… then the compiler won't transform it.

2. Search optimizing IR: What the compiler does could be based on heuristics and statistical information. Think term rewriting systems working on "possible programs" with heuristics as a guide so you build up a database of precompiled segments for "possible programs" and produce guiding datastructures that speed the optimization search. The basic idea being that you spend a lot of time precompiling libraries, and cut down on compiling concrete instances of programs.

3. Generating IR: The compiler could build JITable compiler code for templates, e.g. the templates are stored as a VM program that is executed by the compiler and is allowed to make calls into the compiler code. Basically "compile-time-functions" with inside compiler know-how.

"H. S. Teoh via Digitalmars-d"  wrote in message news:mailman.3042.1418240846.9932.digitalmars-d@puremagic.com...

> Also, storing a full AST is probably overkill -- lexing and parsing the
> source generally doesn't take up too much of the compiler's time, so we
> might as well just use the source code instead.

Exactly

> What makes it more
> worthwhile is if the AST has already been somewhat processed, e.g.,
> constants have been folded, etc.. Probably after semantic1 and semantic2
> have been run (though I'm not sure how far one can get if the template
> hasn't been instantiated yet).

Not very far at all.

> This way, work that has already been done
> doesn't need to be repeated again.

When it's templates that haven't been instantiated, you haven't done any of the work yet.