Am Thu, 7 Mar 2013 23:35:52 +1100
schrieb "Daniel Murphy" <yebblies@nospamgmail.com>:

> As a side note I actually wrote an omf (dmd-emitted subset) linker last year that gets through almost all of the dmd test suite.  It has no debug info support, which sucks, but it may work in some cases where optlink can't cope.  It is fairly naive performance-wise, but is written in a much friendlier language.

I wonder what would happen if someone plugged that linker right into the dmd D port to skip the external linking phase alltogether...

-- 
Marco

"Marco Leise" <Marco.Leise@gmx.de> wrote in message news:20130307154744.206d9bb0@marco-leise...
> Am Thu, 7 Mar 2013 23:35:52 +1100
> schrieb "Daniel Murphy" <yebblies@nospamgmail.com>:
>
>> As a side note I actually wrote an omf (dmd-emitted subset) linker last
>> year
>> that gets through almost all of the dmd test suite.  It has no debug info
>> support, which sucks, but it may work in some cases where optlink can't
>> cope.  It is fairly naive performance-wise, but is written in a much
>> friendlier language.
>
> I wonder what would happen if someone plugged that linker right into the dmd D port to skip the external linking phase alltogether...
>
> -- 
> Marco
>

Performance++
Memory_Usage++

Even optlink has/once had a library version.

llvm's lld is an interesting project, and adding an omf reader could solve our linker problems once it's more mature.

Linkers are fun, corrupt executables are really in their own special class when it comes to nasty bugs.

07.03.2013 14:28, Jacob Carlborg пишет:
> On 2013-03-07 11:12, Moritz Maxeiner wrote:
>
>> Sorry if this has been answered before/is common knowledge, but is
>> porting functions at a time to C wanted for optlink in general, or only
>> for finding segfaults? (e.g. are pull-requests for that welcome)
>
> Yes, in general. I think the idea is to port the whole Optlink to C and
> then to D. It's easier to port from C to D than from assembly to D. This
> is because you can use a C version that doesn't use the runtime or
> standard library to get the generate assembly as close as possible to
> the original one. I think it was something like that.
>

Didn't get. You don't have to use D with druntime. Just don't link it and everything will be OK - you will just get "better C" (i.e. with D structs and other good stuff).

-- 
Денис В. Шеломовский
Denis V. Shelomovskij

On 2013-03-07 16:53, Denis Shelomovskij wrote:

> Didn't get. You don't have to use D with druntime. Just don't link it
> and everything will be OK - you will just get "better C" (i.e. with D
> structs and other good stuff).

I was referring to using C with out the C runtime or C standard library. Walter talked about this is some thread. He basically reimplemented printf.

-- 
/Jacob Carlborg

On 3/7/2013 4:35 AM, Daniel Murphy wrote:
> Maybe I'm missing something, but I don't understand why this is necessary.
> It should be trivial to look at the map file for the closest preceding
> label.  On modern toolchains it should even be possible to compile the asm
> with debug line information.  Or is grepping more effective than it sounds?

Modern operating systems load programs at different base addresses each time, so the addresses in the map file don't match up with the addresses at run time.

The older assembler I was using didn't emit symbolic debug info. I haven't tried out ML that way.


> This sounds completely insane. (But also fun, in an extremely frustrating
> way)
> I suspect it would be easier to reverse-engineer microsoft's (or borland's?)
> old linker from the disassembly, if they wrote it in C.

You'd be incorrect if you thought MS's or Borland's linker was better. Optlink sold very, very well against those linkers back in the day, because Optlink was far better.


> As a side note I actually wrote an omf (dmd-emitted subset) linker last year
> that gets through almost all of the dmd test suite.  It has no debug info
> support, which sucks, but it may work in some cases where optlink can't
> cope.  It is fairly naive performance-wise, but is written in a much
> friendlier language.
>
> If anyone is interested I'll put it up on github.

There's much more to making a linker. There's the debug info (which you mentioned), then there are the resource files, the module definition files, and lastly the myriad of rather complex switches that are expected. Almost none of that is tested by the dmd test suite.

On Thursday, 7 March 2013 at 19:26:49 UTC, Walter Bright wrote:
>> If anyone is interested I'll put it up on github.
>
> There's much more to making a linker. There's the debug info (which you mentioned), then there are the resource files, the module definition files, and lastly the myriad of rather complex switches that are expected. Almost none of that is tested by the dmd test suite.

Nevertheless having a linker written in D goes hand-in-hand with having a compiler written in D. I'd be interested in seeing his work. I'd also be interested in having something that's documented and that isn't bound by licensing issues.

We also have the abandoned DDL[1] project which had a linker that supports OMF, and Eric Anderton was nice enough to document his process of implementing it: http://www.dsource.org/forums/viewtopic.php?t=959

[1] http://dsource.org/projects/ddl/

On 2013-03-07 13:35, Daniel Murphy wrote:

> If anyone is interested I'll put it up on github.

I would say put it there to see how much interest there is.

-- 
/Jacob Carlborg

On 3/7/2013 11:32 AM, Andrej Mitrovic wrote:
> On Thursday, 7 March 2013 at 19:26:49 UTC, Walter Bright wrote:
>>> If anyone is interested I'll put it up on github.
>>
>> There's much more to making a linker. There's the debug info (which you
>> mentioned), then there are the resource files, the module definition files,
>> and lastly the myriad of rather complex switches that are expected. Almost
>> none of that is tested by the dmd test suite.
>
> Nevertheless having a linker written in D goes hand-in-hand with having a
> compiler written in D. I'd be interested in seeing his work. I'd also be
> interested in having something that's documented and that isn't bound by
> licensing issues.
>
> We also have the abandoned DDL[1] project which had a linker that supports OMF,
> and Eric Anderton was nice enough to document his process of implementing it:
> http://www.dsource.org/forums/viewtopic.php?t=959
>
> [1] http://dsource.org/projects/ddl/

Sure, having a complete linker written in D would be preferable. I also began a project to write such. However, getting such to be a complete, viable replacement for optlink will require a determined, sustained effort.

I also believe that even if Optlink were completely converted to C, it will still be intractably difficult to refactor it to do things like support other object file formats. This is why I had started a new linker project.

However, Optlink does bring considerable value to any linker replacement project, in that embedded in it is an enormous amount of lore about all those twinkie little things that matter, and how things really need to work.

And, of course, consider that we don't have a thorough test suite for a linker. This is a serious problem. Trying to make symbolic debug info work without such is a big issue. Heck, I spent quite a bit of time getting dmd to generate correct CV8 info, given that what MS's debugger accepts does not always match the (sparse and confusing) documentation.

What we *can* do with Optlink is fix the occasional bug that crops up in it.

On Thursday, 7 March 2013 at 20:01:58 UTC, Walter Bright wrote:
> What we *can* do with Optlink is fix the occasional bug that crops up in it.

I hope so. :)

On Thursday, 7 March 2013 at 20:01:58 UTC, Walter Bright wrote:
> I also believe that even if Optlink were completely converted to C, it will still be intractably difficult to refactor it to do things like support other object file formats.

I really wish Unilink[1] was open-source. It's apparently a student project, and it's closed source but freeware. It works with OMF but can also link OMF and COFF objects together, which is a boon for win32 programmers who don't want to go through OBJCONV (object file conversion, doesn't always work) or COFF2OMF (import libs). It also has pretty good error messages.

Of course there's no telling how big its test-suite is. It's fairly easy to set up though[2].

[1] ftp://ftp.styx.cabel.net/pub/UniLink/
[2] http://forum.dlang.org/thread/jbvugp$8l3$1@digitalmars.com#post-mailman.1414.1323538292.24802.digitalmars-d:40puremagic.com

On Thursday, 7 March 2013 at 20:01:58 UTC, Walter Bright wrote:
> This is why I had started a new linker project.

Interesting. But if it's made behind-the-scenes with no input or work from the community I won't be holding my hopes up too much, I know you have very limited time for side-projects like that.

On 3/7/2013 12:15 PM, Andrej Mitrovic wrote:
> Interesting. But if it's made behind-the-scenes with no input or work from the
> community I won't be holding my hopes up too much, I know you have very limited
> time for side-projects like that.

It didn't get much past the conceptual part. I wanted to make a linker where:

1. all the supported file formats were completely encapsulated and modularized

For some idea of how that can work, see libmscoff.c and scanmscoff.c, where the object file format and library file format is completely separated.

2. used ranges to their fullest extend