Thread overview
shared not working!
Jul 04, 2016
Mike Parker
Jul 04, 2016
ag0aep6g
Jul 04, 2016
ag0aep6g
July 03, 2016
I have thread. It locks up. If I changed from a bool val it is using from shared to __gshared it works. I checked the address inside and outside of the thread and they are different for shared and same for __gshared.

I thought shared was essentially __gshared with additional semantics? That a shared variable was the same across all threads?

The only difference is that the thread is a windows CreateThread. I guess D doesn't know about such threads and hence shared doesn't extend across to them?

If so, the docs should be updated to mention this.

My code goes something like this:

struct WinThread
{
   shared bool isPaused = false;
   ...

   // Windows threaded callback uses isPaused
}


// use isPaused in main thread

// Crashes in WinThread on use expressions involving isPaused.

Checking pointers between the two shows they are different.

changing to _gshared gives the same address of isPaused for both threads.





July 04, 2016
On Sunday, 3 July 2016 at 23:20:35 UTC, Hiemlick Hiemlicker wrote:

>
> The only difference is that the thread is a windows CreateThread. I guess D doesn't know about such threads and hence shared doesn't extend across to them?

The runtime doesn't know about external threads, no. You have to tell it about them with a call to thread_attachThis() [1]. Still, AFAIK, shared variables should not be dependent upon runtime threads. Do you have a minimum working example to share?

[1] https://dlang.org/phobos/core_thread.html#.thread_attachThis
July 04, 2016
On Monday, 4 July 2016 at 01:36:00 UTC, Mike Parker wrote:
> On Sunday, 3 July 2016 at 23:20:35 UTC, Hiemlick Hiemlicker wrote:
>
>>
>> The only difference is that the thread is a windows CreateThread. I guess D doesn't know about such threads and hence shared doesn't extend across to them?
>
> The runtime doesn't know about external threads, no. You have to tell it about them with a call to thread_attachThis() [1]. Still, AFAIK, shared variables should not be dependent upon runtime threads. Do you have a minimum working example to share?
>
> [1] https://dlang.org/phobos/core_thread.html#.thread_attachThis



Basically the sExtThread struct, changed __gshared to shared or remove.

You can check the and see that the values are correct in the thread by writeln(&paused) type of thing(or store address in global variable).


import std.stdio, std.datetime, std.algorithm, std.string;
import core.thread, core.sync.mutex, core.atomic;


version (Windows)
{
    import core.sys.windows.windows;

    void makeExecutable(void[] code)
    {
        DWORD old;
        VirtualProtect(code.ptr, code.length, PAGE_EXECUTE_READWRITE, &old);
    }
}
else
version (linux)
{
    import core.sys.posix.sys.mman;
    import core.sys.posix.unistd;

    static if (!is(typeof(&mprotect)))
        extern(C) int mprotect(void*, size_t, int);

    void makeExecutable(ubyte[] code)
    {
        auto pageSize = sysconf(_SC_PAGE_SIZE);
        auto address = ((cast(size_t)code.ptr) & ~(pageSize-1));
        int pageCount = (address/pageSize == (address+code.length)/pageSize) ? 1 : 2;
        mprotect(cast(void*)address, pageSize * pageCount, PROT_READ | PROT_WRITE | PROT_EXEC);
    }
}
else
    static assert(0, "TODO");

// Creates a thunk by creating a function(functionTemplate), which the compiler generates code for
// then manipulates the function's code indirectly so the actual architecture asm is not required.

// Creates a thunk by creating a function(functionTemplate), which the compiler generates code for
// then manipulates the function's code indirectly so the actual architecture asm is not required.
R function(A) Delegate2Function(R, A...)(R delegate(A) d)
{
	// Memory placeholders for d.ptr and d.funcptr. Stored per template `instance`.
    shared enum size_t TEMPLATE1 = cast(size_t)0x01234567_01234567;
    shared enum size_t TEMPLATE2 = cast(size_t)0x89ABCDEF_89ABCDEF;
	
	// Thunk callback. This is the function called by Win32, which creates the delegate and points it
	// to the desired callback.  It stores the callback this and function ptr in TEMPLATE1 & TEMPLATE2 resp.
	extern(Windows) // No good?
    static R functionTemplate(A args)
    {
        R delegate(A) d;
        d.ptr     = cast(typeof(d.ptr))TEMPLATE1;
        d.funcptr = cast(typeof(d.funcptr))TEMPLATE2;
        return d(args);
    }

	// A placeholder to help get the size of functionTemplate
    static void functionTemplateEnd() {asm {naked; nop;} }

	// searches a and replaces matches from and rewrites to to
    static void replaceWord(void[] a, size_t from, size_t to)
    {
        foreach (i; 0..a.length - size_t.sizeof + 1)
        {
            auto p = cast(size_t*)(a.ptr + i);
            if (*p == from)
            {
                *p = to;
                return;
            }
        }
        assert(0);
    }


	// Gets the length, in bytes, of the thunk and a slice to it's instructions
	auto functionTemplatePtr = cast(size_t*)&functionTemplate;
	auto length = cast(size_t)((cast(size_t*)&functionTemplateEnd - functionTemplatePtr));
    auto functionTemplateInstructions = functionTemplatePtr[0..length];

    // must allocate type with pointers, otherwise GC won't scan it
    auto newFunctionInstructions = new size_t[length + 3 + 10];
	
    // store context in ptr-aligned boundary, so the GC can find it. This is just so delegate won't be released.
    newFunctionInstructions[0] = cast(size_t)d.ptr;
    newFunctionInstructions[1] = cast(size_t)d.funcptr;
    newFunctionInstructions = newFunctionInstructions[2..$];

	// Copy old instructions to new instructions
	newFunctionInstructions[0..length] = functionTemplateInstructions[0..length];
		
	// Stores this in "thunk data"(local space in function)
    replaceWord(newFunctionInstructions, TEMPLATE1, cast(size_t)d.ptr);
    replaceWord(newFunctionInstructions, TEMPLATE2, cast(size_t)d.funcptr);

	// Make code executable
    makeExecutable(newFunctionInstructions);

	// return thunk(which wraps original delegate to be called)
    return cast(typeof(return)) newFunctionInstructions.ptr;
}





static struct sExtThread
{

	private __gshared
	{
		bool paused = false;
		bool terminated = false;	
		uint _ID = 0;
		HANDLE _Handle;
	
	
		Duration Call_Delay = dur!("msecs")(100);
		Duration Paused_Wait = dur!("msecs")(1000);
		void function() callback;	
		void delegate() originalCallback;
	}

	
		
	alias callbackType = extern (Windows) uint function(void*);	
	
	public void Create(void delegate() c, bool suspended = false, uint stackSize = 16000)
	{	
		originalCallback = c;
		callback = Delegate2Function(c);
				
		auto del = cast(callbackType)Delegate2Function((void*)
		{
		 	import core.thread;
			do
			{
				if (!paused)
					do				
					{
						Thread.sleep(Call_Delay);
						if (!terminated)
							callback();
						else
							paused = true;
					} while(!paused && !terminated);
				else
				{
					Thread.sleep(Paused_Wait);
				}
			} while(!terminated);

			return 0;

		}
		);

		_Handle = CreateThread(cast(SECURITY_ATTRIBUTES*)null, stackSize, del, null, (suspended) ? CREATE_SUSPENDED : 0, &_ID);
	}

	public @property HANDLE Handle() { return _Handle; }
	public @property int ID() { return _ID; }

	public void Pause()
	{
		paused = true;		
	}

	public void Resume()
	{
		paused = false;
	}

	// Terminates the thread when it is finished executing at least one cycle.
	public void Terminate()
	{
		terminated = true;
	}

	public void Now()
	{
		originalCallback();		
	}

	~this()
	{
		if (!terminated)
			TerminateThread(Handle, 0);
		terminated = true;
		
	}
}


private sExtThread bThread;




static class Data
{
	static __gshared
	{

	}

	static void Init()
	{

		
	}

	static double getCpu()
	{
		return 0;	
	}

	static ulong getMem()
	{
		return 0;	
	}

	static ulong getVMem()
	{
		return 0;	
	}
}

	private void RTStatsCollector()
	{
		Data.Init();
		
		bThread.Create(
		(){
				UpdateStats();
		});
		
   	}




	public static void UpdateStats()	
	{
			
			with(Data)
			{

			}
	}


void main()
{
    RTStatsCollector();
	
	import std.random;

	while(getchar() != EOF)
	{
		auto x = new int[std.random.uniform(1000000, 10000000)];

		writeln("--------");
		bThread.Now();

	}

	getchar();
}

July 04, 2016
On 07/04/2016 01:20 AM, Hiemlick Hiemlicker wrote:
> I have thread. It locks up. If I changed from a bool val it is using
> from shared to __gshared it works. I checked the address inside and
> outside of the thread and they are different for shared and same for
> __gshared.
>
[...]
>     shared bool isPaused = false;

Needs to be static shared. __gshared implies static, shared doesn't.
July 04, 2016
On Monday, 4 July 2016 at 05:08:34 UTC, ag0aep6g wrote:
> On 07/04/2016 01:20 AM, Hiemlick Hiemlicker wrote:
>> I have thread. It locks up. If I changed from a bool val it is using
>> from shared to __gshared it works. I checked the address inside and
>> outside of the thread and they are different for shared and same for
>> __gshared.
>>
> [...]
>>     shared bool isPaused = false;
>
> Needs to be static shared. __gshared implies static, shared doesn't.

But static isn't per instance, is it?

That means every thread created will have the same value of Paused. Calling Pause will pause all threads. That can't be right.

Shared should have no implication of static. If that was the case, all shared types would be static, would make them uses to have and to use a this.

Unless static shared is different than static + shared?



July 04, 2016
On 07/04/2016 07:16 AM, Hiemlick Hiemlicker wrote:
> But static isn't per instance, is it?

Sure isn't. __gshared isn't either.

> That means every thread created will have the same value of Paused.
> Calling Pause will pause all threads. That can't be right.

I don't know what exactly you're doing and if static would be correct or not. Just saying that __gshared is closer to static shared than to non-static shared.

> Shared should have no implication of static. If that was the case, all
> shared types would be static, would make them uses to have and to use a
> this.

shared doesn't imply static. But __gshared does. If your program works correctly with __gshared, it seems to depend on the implied static.

> Unless static shared is different than static + shared?

No. It's just that.