View mode: basic / threaded / horizontal-split · Log in · Help
May 03, 2012
GSOC Linker project
Hi!

I'm interested in starting a project to make a linker besides 
optlink for dmd on windows. If possible it'd be cool to run it as 
a gsoc-project, but if that's not an option I'll try to get it 
admitted as a soc-project at my university.

Anyway, the project would aim to be a replacement or alternative 
to optlink on windows. I've personally encountered quite a few 
seemingly random problems with optlink, and the error messages 
are not exactly friendly. My vision is to create a linker in a 
relatively modern language (D) and to release the project as open 
source.

So, I'm curious about some things; Is it too late to get this 
accepted as a summer of code project? Are there any current 
alternative linkers for dmd on windows, or any current projects 
aiming to create one? And do any of you know of a "Everything you 
need to know to write the best linker ever" resource center? ;]

/Pierre
May 03, 2012
Re: GSOC Linker project
On 04-05-2012 00:47, Pierre LeMoine wrote:
> Hi!
>
> I'm interested in starting a project to make a linker besides optlink
> for dmd on windows. If possible it'd be cool to run it as a
> gsoc-project, but if that's not an option I'll try to get it admitted as
> a soc-project at my university.

Absolutely possible, though too late for this year's GSoC. If you're 
still interested in working on it for GSoC 2013 (if Google decides to do 
another GSoC (which they most likely will)), then be sure to submit a 
proposal!

>
> Anyway, the project would aim to be a replacement or alternative to
> optlink on windows. I've personally encountered quite a few seemingly
> random problems with optlink, and the error messages are not exactly
> friendly. My vision is to create a linker in a relatively modern
> language (D) and to release the project as open source.

Sounds like a good idea to me. Though in my personal opinion, you should 
try to make the linker as platform-agnostic as possible, so it's easy to 
adapt for new platforms / file formats.

>
> So, I'm curious about some things; Is it too late to get this accepted
> as a summer of code project? Are there any current alternative linkers
> for dmd on windows, or any current projects aiming to create one? And do
> any of you know of a "Everything you need to know to write the best
> linker ever" resource center? ;]

Too late for this year's GSoC.

For another linker option, see Unilink.

As for resources on linkers, I think your best bet is reading the LLVM 
and GCC source code. I think someone also started an LLVM (machine code) 
linker project recently, but I don't know where to find it.

>
> /Pierre

-- 
- Alex
May 03, 2012
Re: GSOC Linker project
> I'm interested in starting a project to make a linker besides optlink  
> for dmd on windows.

Imho changing dmd to use COFF (incl. 64 support) instead of that crappy  
OMF would be more beneficial than yet another linker.


> My vision is to create a linker in a relatively modern language (D) and  
> to release the project as open source.

If you do write a linker then make it cross-platform right from the start;  
and modular so it can support all object file formats.
May 03, 2012
Re: GSOC Linker project
On Fri, May 04, 2012 at 12:53:02AM +0200, Alex Rønne Petersen wrote:
> On 04-05-2012 00:47, Pierre LeMoine wrote:
[...]
> >Anyway, the project would aim to be a replacement or alternative to
> >optlink on windows. I've personally encountered quite a few seemingly
> >random problems with optlink, and the error messages are not exactly
> >friendly. My vision is to create a linker in a relatively modern
> >language (D) and to release the project as open source.
> 
> Sounds like a good idea to me. Though in my personal opinion, you
> should try to make the linker as platform-agnostic as possible, so
> it's easy to adapt for new platforms / file formats.
[...]

The problem with writing linkers is that they are usually closely tied
to implementation details on the host OS. At the very least, they must
play nice with the OS's runtime dynamic linker (or *be* the dynamic
linker themselves, like ld on the *nixes). They must also play nice with
object files produced by other compilers on that platform, since
otherwise it sorta defeats the purpose of rewriting optlink in the first
place. This means that they must understand all the intimate details of
every common object file and executable format on that OS.

The basic concept behind a linker is very simple, really, but it's the
implementation where details get ugly.

To be frank, I question the wisdom of not just using ld on Posix
systems... but OTOH, the world *needs* better linker technology than we
currently have, so this projects like this one is a good thing.

Linkers date from several decades ago, where programs can be broken up
into separate, self-contained source files in a simple way. Things have
changed a lot since then.  Nowadays, we have template functions, virtual
functions, dynamic libraries, etc., which require hacks like weak
symbols to work properly. And we're *still* missing a sound conceptual
framework for things like cross-module dead code elimination,
cross-module template instantiation, duplicate code merging (like
overlapping immutable arrays), etc.. These things _sorta_ work right
now, but they're sorta hacked on top of basic 30-year-old linker
technology, rather than being part of a sound, conceptual linker
paradigm.


T

-- 
There are 10 kinds of people in the world: those who can count in binary, and those who can't.
May 04, 2012
Re: GSOC Linker project
On 04-05-2012 01:57, H. S. Teoh wrote:
> On Fri, May 04, 2012 at 12:53:02AM +0200, Alex Rønne Petersen wrote:
>> On 04-05-2012 00:47, Pierre LeMoine wrote:
> [...]
>>> Anyway, the project would aim to be a replacement or alternative to
>>> optlink on windows. I've personally encountered quite a few seemingly
>>> random problems with optlink, and the error messages are not exactly
>>> friendly. My vision is to create a linker in a relatively modern
>>> language (D) and to release the project as open source.
>>
>> Sounds like a good idea to me. Though in my personal opinion, you
>> should try to make the linker as platform-agnostic as possible, so
>> it's easy to adapt for new platforms / file formats.
> [...]
>
> The problem with writing linkers is that they are usually closely tied
> to implementation details on the host OS. At the very least, they must
> play nice with the OS's runtime dynamic linker (or *be* the dynamic
> linker themselves, like ld on the *nixes). They must also play nice with
> object files produced by other compilers on that platform, since
> otherwise it sorta defeats the purpose of rewriting optlink in the first
> place. This means that they must understand all the intimate details of
> every common object file and executable format on that OS.

I know, but that doesn't mean you can't write the linker so that it 
actually *is* portable at *all* (unlike a certain other linker ;).

>
> The basic concept behind a linker is very simple, really, but it's the
> implementation where details get ugly.
>
> To be frank, I question the wisdom of not just using ld on Posix
> systems... but OTOH, the world *needs* better linker technology than we
> currently have, so this projects like this one is a good thing.

Well, there's currently an LLVM linker in the works. If anything, that's 
probably the way forward. But seeing as DMD is not using LLVM...

>
> Linkers date from several decades ago, where programs can be broken up
> into separate, self-contained source files in a simple way. Things have
> changed a lot since then.  Nowadays, we have template functions, virtual
> functions, dynamic libraries, etc., which require hacks like weak
> symbols to work properly. And we're *still* missing a sound conceptual
> framework for things like cross-module dead code elimination,
> cross-module template instantiation, duplicate code merging (like
> overlapping immutable arrays), etc.. These things _sorta_ work right
> now, but they're sorta hacked on top of basic 30-year-old linker
> technology, rather than being part of a sound, conceptual linker
> paradigm.
>
>
> T
>

-- 
- Alex
May 04, 2012
Re: GSOC Linker project
On Fri, May 04, 2012 at 02:43:34AM +0200, Alex Rønne Petersen wrote:
> On 04-05-2012 01:57, H. S. Teoh wrote:
[...]
> >The problem with writing linkers is that they are usually closely
> >tied to implementation details on the host OS.
[...]
> I know, but that doesn't mean you can't write the linker so that it
> actually *is* portable at *all* (unlike a certain other linker ;).

True, you could have a properly designed generic framework that makes
plugging in new OS-dependent code very easy. I believe something like
this is done by GNU BFD (binutils & family, probably subsuming ld as
well).

But then, you might as well just use binutils in the first place. :-)
The only catch is that windows has its own conventions on stuff, and
binutils is (AFAIK) tied to Posix.


[...]
> >To be frank, I question the wisdom of not just using ld on Posix
> >systems... but OTOH, the world *needs* better linker technology than
> >we currently have, so this projects like this one is a good thing.
> 
> Well, there's currently an LLVM linker in the works. If anything,
> that's probably the way forward. But seeing as DMD is not using
> LLVM...
[...]

As long as LDC is an option, I think all is well. :-)


T

-- 
Only boring people get bored. -- JM
May 04, 2012
Re: GSOC Linker project
On 2012-05-04 01:57, H. S. Teoh wrote:

> To be frank, I question the wisdom of not just using ld on Posix
> systems... but OTOH, the world *needs* better linker technology than we
> currently have, so this projects like this one is a good thing.

He can start with a version for Windows. If as much as possible of the 
code is generic and modular designed it should be easy to add support 
for new formats and platforms.

> Linkers date from several decades ago, where programs can be broken up
> into separate, self-contained source files in a simple way. Things have
> changed a lot since then.  Nowadays, we have template functions, virtual
> functions, dynamic libraries, etc., which require hacks like weak
> symbols to work properly. And we're *still* missing a sound conceptual
> framework for things like cross-module dead code elimination,
> cross-module template instantiation, duplicate code merging (like
> overlapping immutable arrays), etc.. These things _sorta_ work right
> now, but they're sorta hacked on top of basic 30-year-old linker
> technology, rather than being part of a sound, conceptual linker
> paradigm.

That would be really nice.

-- 
/Jacob Carlborg
May 04, 2012
Re: GSOC Linker project
On Thu, 03 May 2012 19:47:24 -0400, Trass3r <un@known.com> wrote:

>> I'm interested in starting a project to make a linker besides optlink  
>> for dmd on windows.
>
> Imho changing dmd to use COFF (incl. 64 support) instead of that crappy  
> OMF would be more beneficial than yet another linker.

+1

-Steve
May 04, 2012
Re: GSOC Linker project
On Thursday, 3 May 2012 at 23:47:26 UTC, Trass3r wrote:
>> I'm interested in starting a project to make a linker besides 
>> optlink for dmd on windows.
>
> Imho changing dmd to use COFF (incl. 64 support) instead of 
> that crappy OMF would be more beneficial than yet another 
> linker.
>
>
>> My vision is to create a linker in a relatively modern 
>> language (D) and to release the project as open source.
>
> If you do write a linker then make it cross-platform right from 
> the start; and modular so it can support all object file 
> formats.

How about augmenting the object format so that libraries would be 
self contained and would not require additional .di files? Is 
this possible optlink by e.g. adding special sections that would 
be otherwise ignored?

I think that's what Go did in their linker but I don't know what 
format they use, if it's something specific to Go or general.
May 04, 2012
Re: GSOC Linker project
On Fri, 04 May 2012 18:57:44 +0200, foobar <foo@bar.com> wrote:

> On Thursday, 3 May 2012 at 23:47:26 UTC, Trass3r wrote:
>>> I'm interested in starting a project to make a linker besides optlink  
>>> for dmd on windows.
>>
>> Imho changing dmd to use COFF (incl. 64 support) instead of that crappy  
>> OMF would be more beneficial than yet another linker.
>>
>>
>>> My vision is to create a linker in a relatively modern language (D)  
>>> and to release the project as open source.
>>
>> If you do write a linker then make it cross-platform right from the  
>> start; and modular so it can support all object file formats.
>
> How about augmenting the object format so that libraries would be self  
> contained and would not require additional .di files? Is this possible  
> optlink by e.g. adding special sections that would be otherwise ignored?
>
> I think that's what Go did in their linker but I don't know what format  
> they use, if it's something specific to Go or general.
>

http://dsource.org/projects/ddl
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home