October 29, 2009
So I have this optimization in which I jump from function to function to avoid the overhead of setting up and tearing down identical stack frames. Now I want to be able to jump from a member function of one object to a member function of another object. I accomplished this with g++ by replacing the this pointer that is passed in as a hidden parameter. After some difficulty with dmd I realized that the this pointer is actually stored in EAX. But simply updating EAX to hold the new this pointer doesn't seem to be working. I've verified that EAX contains the address of the destination object, but it is still behaving as if the this pointer had not changed...

class Test()
	static byte* jumpTo;
	void* nextObject;

	int count;

	void Run()
		if (count++ > 5)
	                mov ECX, EAX;
			mov EAX, nextObject[ECX];
	                jmp dword ptr jumpTo;

int main(char[][] args)
	//14 is offset for parameterless member function with no stack variables
	Test.jumpTo = cast(byte*)(&Test.Run) + 14;

	auto t1 = new Test();
	auto t2 = new Test();

	t1.nextObject = cast(void*)t2;
	t2.nextObject = cast(void*)t1;


	Stdout(t1.count, t2.count);

	return 0;

...This gives me the result "7, 0". I don't have the actual disassembled code in front of me but IIRC member variables are referenced via offset from the location stored in EAX. I'm not sure what else to try.