Hi everyone,

To keeps things short: There shall be a extended debugging feature integrated into Mono-D / VisualD later on. As you may see on http://mono-d.alexanderbothe.com/wordpress/wp-content/uploads/2012/09/2ylpkqg.jpg , there already is debugging functionality possible for windows programs (when it's arrived a pretty stable status it'll be released as a second D addin, that's for sure)

Anyway, we'd like to replace all those 'struct main@helper' value strings with actual values returned by the object's toString function.
So the debug engine shall execute the toString() function/overload of an object
1) via a D DLL that has been injected into the program run-time or
2) directly via CreateRemoteThread(), whereas it should be possible to allocate some code memory and write binary code right into it

I kept experimenting with all the injection, assembler and program hacking stuff quite a while, and these are my primary perceptions:

- It seems that one cannot inject D DLLs into D programs without crashing the actual program (it's always an exception thrown by RTLMultiPool::SelectFree), whereas:
    --One may write a main() function instead of the DllMain() callback
    and then start a normal WindowMessage-loop in order to
    prevent both DLL and Program unload/crash - but that's not really it, because it's just caught in a loop, and nothing less.
    --It's possible to call LoadLibrary with the DLLs file path inside the DllMain() to hook into the program without letting it crash - but then it seems impossible to access the dll from the outside (probably via named pipes, then)
    --It doesn't seem to make sense to load in a C dll - because from there, it's practically impossible to call D functions.

- I've created an export toString(int pointerToObject) method inside the
D program - and it's not possible to invoke it via CreateRemoteThread().
So even if I did it to successfully inject a D Dll into the D program,
there's no guarantee that it's possible to call that toString function even in the D Dll.

extern(C) export string toSt(int p)
{
	return (cast(Object)cast(void*)p).toString();	
}
the 'p' paramter comes from the debugger engine then - so it knows the object address.

- Another approach was to put in raw assembler code into the program's
virtual memory space and try to execute it from there - so 'just'
put the assembler code (I've built it already lol) into the
program run-time, and execute it somehow. But I definitely do not know how to create real working assembler etc.

Or: I've tried Winject yesterday, too, and there it worked to load in the DLL just at launching the program - this is something which could be realized with the debug engine, I guess.
But then again the question of having execution access to the exported functions of the client dll .. named pipes?

Okay, these are my explenanations so far - and I think it would be really interesting to have such debugger-debugee communication in D.
1) So to anyone who's got richer experiences in programming assembler and hacking/'debugging' programs than I - how would you do it?
2) And why can't I inject a D DLL right into the program? I tried it with a C DLL, it's working with that one, but then I don't have access to D-specific functions..
Looking at that, would it make a difference to use dmc to build/link a dll as a D/C++ hybrid or something?

Thanks in advance for any ideas, recommendations etc.!

Oh and the debugger addin project: https://github.com/aBothe/monodevelop-win32-debugger

On Sep 14, 2012, at 10:34 AM, alex <info@alexanderbothe.com> wrote:

> 1) So to anyone who's got richer experiences in programming assembler and hacking/'debugging' programs than I - how would you do it?
> 2) And why can't I inject a D DLL right into the program? I tried it with a C DLL, it's working with that one, but then I don't have access to D-specific functions..
> Looking at that, would it make a difference to use dmc to build/link a dll as a D/C++ hybrid or something?

You might want to check the madCodeHook library.  It works well, and there's a version that includes source code.  Though now that I check the site, it doesn't look like you can get the source-included one for free any more.

On Friday, 14 September 2012 at 18:35:53 UTC, Sean Kelly wrote:
> On Sep 14, 2012, at 10:34 AM, alex <info@alexanderbothe.com> wrote:
>
>> 1) So to anyone who's got richer experiences in programming assembler and hacking/'debugging' programs than I - how would you do it?
>> 2) And why can't I inject a D DLL right into the program? I tried it with a C DLL, it's working with that one, but then I don't have access to D-specific functions..
>> Looking at that, would it make a difference to use dmc to build/link a dll as a D/C++ hybrid or something?
>
> You might want to check the madCodeHook library.  It works well, and there's a version that includes source code.  Though now that I check the site, it doesn't look like you can get the source-included one for free any more.

The primary injection routine and how it works is not the problem - I can successfully insert a MSVC++ Dll without any problems into a D program. (I've used this https://github.com/jeffora/extemory library btw - it's for c# :-))
Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately crashes or unloads the dll when returning false in the DllMain()

So how to insert a D DLL into a D program?

On 2012-09-14 20:53, alex wrote:

> The primary injection routine and how it works is not the problem - I
> can successfully insert a MSVC++ Dll without any problems into a D
> program. (I've used this https://github.com/jeffora/extemory library btw
> - it's for c# :-))
> Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
> crashes or unloads the dll when returning false in the DllMain()
>
> So how to insert a D DLL into a D program?

I don't know the status on Windows but in general D dynamic libraries don't properly work. The problems are TLS, exception handing tables and module infos. There might be other problems as well.

-- 
/Jacob Carlborg

On 14.09.2012 20:53, alex wrote:
> On Friday, 14 September 2012 at 18:35:53 UTC, Sean Kelly wrote:
>> On Sep 14, 2012, at 10:34 AM, alex <info@alexanderbothe.com> wrote:
>>
>>> 1) So to anyone who's got richer experiences in programming assembler
>>> and hacking/'debugging' programs than I - how would you do it?
>>> 2) And why can't I inject a D DLL right into the program? I tried it
>>> with a C DLL, it's working with that one, but then I don't have
>>> access to D-specific functions..
>>> Looking at that, would it make a difference to use dmc to build/link
>>> a dll as a D/C++ hybrid or something?
>>
>> You might want to check the madCodeHook library.  It works well, and
>> there's a version that includes source code.  Though now that I check
>> the site, it doesn't look like you can get the source-included one for
>> free any more.
>
> The primary injection routine and how it works is not the problem - I
> can successfully insert a MSVC++ Dll without any problems into a D
> program. (I've used this https://github.com/jeffora/extemory library btw
> - it's for c# :-))
> Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
> crashes or unloads the dll when returning false in the DllMain()
>
> So how to insert a D DLL into a D program?

I recently implemented an injection DLL aswell, in D, but without any runtime library: https://github.com/rainers/visuald/blob/master/tools/filemonitor.d

The command line to build it is
dmd -offilemonitor.dll -defaultlib=user32.lib -L/ENTRY:_DllMain@12 filemonitor.d

It also crashed when using the standard DLL framework, my suspicion is that the druntime-initialization isn't properly run for the injected DLL. If you are creating the process suspended, the main thread cannot run, so initialization might have to be done differently. But I haven't investigated any further, I was fine with just using the Windows API.

For C/C++ the VS debugger can already call functions in the debuggee as a side effect to watch expressions, so it should work for D aswell. It might be a little complicated to specify the correct symbols, though.

To improve debugging experience in VS I think the better approach would be to extend the capabilities of mago, the debug engine explicitely built for dmd generated executables. Unfortunately development of it has stalled.

15.09.2012 16:03, Jacob Carlborg пишет:
> On 2012-09-14 20:53, alex wrote:
>
>> The primary injection routine and how it works is not the problem - I
>> can successfully insert a MSVC++ Dll without any problems into a D
>> program. (I've used this https://github.com/jeffora/extemory library btw
>> - it's for c# :-))
>> Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
>> crashes or unloads the dll when returning false in the DllMain()
>>
>> So how to insert a D DLL into a D program?
>
> I don't know the status on Windows but in general D dynamic libraries
> don't properly work. The problems are TLS, exception handing tables and
> module infos. There might be other problems as well.
>

TLS is the problem on Windows 5.x (XP, Server 2003), not 6.x. D DLL will use incomplete ugly hacks to fix it and will be unable to unload because of hacks incompleteness. By the way I have created a complete hack version to solve TLS problems in XP forever for all programs independently of its source availability that injects itself into running process, fixes everything and injects itself to all it's children, etc. Sorry for such a long story about my just another project nobody needs.

Lets return to the toppic.

The common practice of DLL-s injection is creating a new thread in remote process and calling LoadLibrary from there but it is incorrect because:
* it wastes resources (requires a copy of TLS local variables, calls of module thread local constructors (for D) and all loading DLL-s DllMains to attach new thread (and then to detach of course))
* injecting DLL's DllMain will be called not in main thread but the DLL may expect to be in main one
* for D DLL it will just fail because of Digital Mars C runtime library (honestly I'm not sure here but everything looks like it is so)

Again, Digital Mars C runtime library is the problem for everything in D language including DLL-s.

-- 
Денис В. Шеломовский
Denis V. Shelomovskij

On Saturday, 15 September 2012 at 13:02:32 UTC, Denis Shelomovskij wrote:
>....
> Again, Digital Mars C runtime library is the problem for everything in D language including DLL-s.

Lol okay I think I've also seen it. I've tried to build a hybrid dll with mixed C and D code (just compiled with dmc+dmd), and it's just not working, even if it's raw C exclusively..


I'll try an other approach now that is probably way more elegant and doesn't need any LoadLibrary calls:
I simply create a code cave in the debuggee and inject some assembler into it.
The method I'll be executing then takes a variable address (that has been stored in an other space), makes an object pointer out of it, and calls the virtual toString() overload - whereas the pointer to that function is stored at a fixed offset, fortunately. The returned string struct/pointer whatever will be stored to the variable address then (so I don't have to allocate another variable space), and the debug engine will finally read out the string.
That's my theory so far, I hope it'll work somehow :)

@Rainer I'll try my method first, and if that's not working at all, or if it's working, I'll contact you ;)

On Saturday, 15 September 2012 at 13:34:02 UTC, alex wrote:
> On Saturday, 15 September 2012 at 13:02:32 UTC, Denis Shelomovskij wrote:
>>....
>> Again, Digital Mars C runtime library is the problem for everything in D language including DLL-s.
>
> Lol okay I think I've also seen it. I've tried to build a hybrid dll with mixed C and D code (just compiled with dmc+dmd), and it's just not working, even if it's raw C exclusively..
>
>
> I'll try an other approach now that is probably way more elegant and doesn't need any LoadLibrary calls:
> I simply create a code cave in the debuggee and inject some assembler into it.
> The method I'll be executing then takes a variable address (that has been stored in an other space), makes an object pointer out of it, and calls the virtual toString() overload - whereas the pointer to that function is stored at a fixed offset, fortunately. The returned string struct/pointer whatever will be stored to the variable address then (so I don't have to allocate another variable space), and the debug engine will finally read out the string.
> That's my theory so far, I hope it'll work somehow :)
>
> @Rainer I'll try my method first, and if that's not working at all, or if it's working, I'll contact you ;)

It's absolutely frickin' awesome - it works! I could inject the assembler code, call the object's toString() method, do everything as I've just explained...it's awesome!! FUCK YEAH I did it..now I can go to sleep :D

@Rainer I'll share it so we both may integrate it into the debuggers then :)

Ah sorry that I've forgotten to post a link to the final project - I was just too excited by the fact that it worked ;)

https://github.com/aBothe/monodevelop-win32-debugger/tree/master/DInject
(The most interesting part is located in Inject.cs)

It is a problem cause by snn.lib , you can edit it with a hex editor

find :
83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 53 e8
replace to :
83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 eb 07



On Friday, 14 September 2012 at 17:34:37 UTC, alex wrote:
> Hi everyone,
>
> To keeps things short: There shall be a extended debugging feature integrated into Mono-D / VisualD later on. As you may see on http://mono-d.alexanderbothe.com/wordpress/wp-content/uploads/2012/09/2ylpkqg.jpg , there already is debugging functionality possible for windows programs (when it's arrived a pretty stable status it'll be released as a second D addin, that's for sure)
>
> Anyway, we'd like to replace all those 'struct main@helper' value strings with actual values returned by the object's toString function.
> So the debug engine shall execute the toString() function/overload of an object
> 1) via a D DLL that has been injected into the program run-time or
> 2) directly via CreateRemoteThread(), whereas it should be possible to allocate some code memory and write binary code right into it
>
> I kept experimenting with all the injection, assembler and program hacking stuff quite a while, and these are my primary perceptions:
>
> - It seems that one cannot inject D DLLs into D programs without crashing the actual program (it's always an exception thrown by RTLMultiPool::SelectFree), whereas:
>     --One may write a main() function instead of the DllMain() callback
>     and then start a normal WindowMessage-loop in order to
>     prevent both DLL and Program unload/crash - but that's not really it, because it's just caught in a loop, and nothing less.
>     --It's possible to call LoadLibrary with the DLLs file path inside the DllMain() to hook into the program without letting it crash - but then it seems impossible to access the dll from the outside (probably via named pipes, then)
>     --It doesn't seem to make sense to load in a C dll - because from there, it's practically impossible to call D functions.
>
> - I've created an export toString(int pointerToObject) method inside the
> D program - and it's not possible to invoke it via CreateRemoteThread().
> So even if I did it to successfully inject a D Dll into the D program,
> there's no guarantee that it's possible to call that toString function even in the D Dll.
>
> extern(C) export string toSt(int p)
> {
> 	return (cast(Object)cast(void*)p).toString();	
> }
> the 'p' paramter comes from the debugger engine then - so it knows the object address.
>
> - Another approach was to put in raw assembler code into the program's
> virtual memory space and try to execute it from there - so 'just'
> put the assembler code (I've built it already lol) into the
> program run-time, and execute it somehow. But I definitely do not know how to create real working assembler etc.
>
> Or: I've tried Winject yesterday, too, and there it worked to load in the DLL just at launching the program - this is something which could be realized with the debug engine, I guess.
> But then again the question of having execution access to the exported functions of the client dll .. named pipes?
>
> Okay, these are my explenanations so far - and I think it would be really interesting to have such debugger-debugee communication in D.
> 1) So to anyone who's got richer experiences in programming assembler and hacking/'debugging' programs than I - how would you do it?
> 2) And why can't I inject a D DLL right into the program? I tried it with a C DLL, it's working with that one, but then I don't have access to D-specific functions..
> Looking at that, would it make a difference to use dmc to build/link a dll as a D/C++ hybrid or something?
>
> Thanks in advance for any ideas, recommendations etc.!
>
> Oh and the debugger addin project: https://github.com/aBothe/monodevelop-win32-debugger