View mode: basic / threaded / horizontal-split · Log in · Help
November 18, 2010
Shared pain
I had D code that provided a basis for creation of Windows services, 
which I have just tried to get working with the latest D2.

No dice.

The point of failure was in this method

static void StartService()
{
  if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *) &_sta[0]))
  {
     ...
  }
}

Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code 
that install and remove the service worked fine, but the service control 
dispatcher was clearly not happy about these structs being in TLS, and 
the service crashed immediately when I tried to start it.

The class also has a couple of methods with signatures like:

extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}

That's probably not relevant, but I think it contributed to a host of 
errors that finally made be take a big step back.

I re-implemented the thing in the style used for OOP in C.

struct ServiceBase
{
  SERVICE_TABLE_ENTRY[2] _sta;
  ...
}

__gshared ServiceBase __sb;

void initialize(ServiceBase* sb) { ... }
void whatever(ServiceBase* sb ...) { ... }

and used the __sb global within the functions with the Windows signature, 
which just became

extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}

Now it's working again, but I want to change it into idiomatically sound 
D code.

Style suggestions please!

Steve
November 18, 2010
Re: Shared pain
I'm not familiar with the API, but are you able to declare your original _sta as immutable and call it a day? Immutable data should have the same protection as __gshared but with no implications of bypassing the type system.

If that's not helpful, can you give more details like calling patterns? Is the function called from one thread or from many? My quick read of the windows documentation makes it seem like that's true. Probably unimportant, but I'd use _sta.ptr instead of &_sta[0] because the former more clearly implies array pointer to me, but that may be my own quirky style. 

Steve Teale Wrote:

> I had D code that provided a basis for creation of Windows services, 
> which I have just tried to get working with the latest D2.
> 
> No dice.
> 
> The point of failure was in this method
> 
> static void StartService()

> {

>    if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *) &_sta[0]))

>    {
>       ...

>    }

> }
> 
> Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code 
> that install and remove the service worked fine, but the service control 
> dispatcher was clearly not happy about these structs being in TLS, and 
> the service crashed immediately when I tried to start it.
> 
> The class also has a couple of methods with signatures like:
> 
> extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}
> 
> That's probably not relevant, but I think it contributed to a host of 
> errors that finally made be take a big step back.
> 
> I re-implemented the thing in the style used for OOP in C.
> 
> struct ServiceBase
> {
>    SERVICE_TABLE_ENTRY[2] _sta;
>    ...
> }
> 
> __gshared ServiceBase __sb;
> 
> void initialize(ServiceBase* sb) { ... }
> void whatever(ServiceBase* sb ...) { ... }
> 
> and used the __sb global within the functions with the Windows signature, 
> which just became
> 
> extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}
> 
> Now it's working again, but I want to change it into idiomatically sound 
> D code.
> 
> Style suggestions please!
> 
> Steve
November 18, 2010
Re: Shared pain
On Thu, 18 Nov 2010 06:26:39 -0500, Steve Teale  
<steve.teale@britseyeview.com> wrote:

> I had D code that provided a basis for creation of Windows services,
> which I have just tried to get working with the latest D2.
>
> No dice.
>
> The point of failure was in this method
>
> static void StartService()
>
> {
>
>    if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *)  
> &_sta[0]))
>
>    {
>       ...
>
>    }
>
> }
>
> Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code
> that install and remove the service worked fine, but the service control
> dispatcher was clearly not happy about these structs being in TLS, and
> the service crashed immediately when I tried to start it.

I think this has something to do with how services start.  From this page:  
http://msdn.microsoft.com/en-us/library/ms685990(v=VS.85).aspx
I see that StartServiceCtrlDispatcher creates its own thread to run  
ServiceMain, which most likely does not call the D thread initialization  
routines (not sure).  TLS may not be properly set up inside your thread  
that is running ServiceMain, and probably the static this() module  
functions have not been called.

>
> The class also has a couple of methods with signatures like:
>
> extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}
>
> That's probably not relevant, but I think it contributed to a host of
> errors that finally made be take a big step back.
>
> I re-implemented the thing in the style used for OOP in C.
>
> struct ServiceBase
> {
>    SERVICE_TABLE_ENTRY[2] _sta;
>    ...
> }
>
> __gshared ServiceBase __sb;
>
> void initialize(ServiceBase* sb) { ... }
> void whatever(ServiceBase* sb ...) { ... }
>
> and used the __sb global within the functions with the Windows signature,
> which just became
>
> extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}
>
> Now it's working again, but I want to change it into idiomatically sound
> D code.
>
> Style suggestions please!

I think using a class in D is a good idea.  But what you may need is  
global functions which forward to your class methods.  This is how I would  
probably do it.

-Steve
November 18, 2010
Re: Shared pain
On Thu, 18 Nov 2010 08:33:12 -0500, Steven Schveighoffer wrote:

> 
> I think using a class in D is a good idea.  But what you may need is
> global functions which forward to your class methods.  This is how I
> would probably do it.
> 
> -Steve

Steve,

Thanks. I kind of went that way. Having done the laborious C OOP style 
thing, I put most of the methods back into the struct, just leaving the 
extern(Windows) functions at global scope - they referred to the 
__gshare'd object explicitly - I hate that.

Then I hid all of the crap behind a class that sneakily referred to the 
__gshared struct, but at least now to an external user of the module the 
interface looks much the same as it did before.

Now it works OK with the test case example - a service that just writes 
entries in the event log. However, the real service I'm trying to update 
works fine if I run it in debug mode (i.e. basically as just an 
executable), but if I run it as a service it seems to crash the first 
time I use a RegExp.find().

As you can imagine, this is terrible to debug - printf or similar is bad 
enough, but for the service I'm pushing entries into the Windows event 
logger system. Fortunately my D class for doing that has survived the D2 
changes. I'm going to try tweaking the service so it will interact with 
the desktop, then at least I can revert to printf.

But first I think I'll stop and do some carpentry for a couple of days. 
Then eventually something might dawn on me.

Steve
November 18, 2010
Re: Shared pain
On Thu, 18 Nov 2010 13:53:53 -0500, Steve Teale  
<steve.teale@britseyeview.com> wrote:

> On Thu, 18 Nov 2010 08:33:12 -0500, Steven Schveighoffer wrote:
>
>>
>> I think using a class in D is a good idea.  But what you may need is
>> global functions which forward to your class methods.  This is how I
>> would probably do it.
>>
>> -Steve
>
> Steve,
>
> Thanks. I kind of went that way. Having done the laborious C OOP style
> thing, I put most of the methods back into the struct, just leaving the
> extern(Windows) functions at global scope - they referred to the
> __gshare'd object explicitly - I hate that.
>
> Then I hid all of the crap behind a class that sneakily referred to the
> __gshared struct, but at least now to an external user of the module the
> interface looks much the same as it did before.
>
> Now it works OK with the test case example - a service that just writes
> entries in the event log. However, the real service I'm trying to update
> works fine if I run it in debug mode (i.e. basically as just an
> executable), but if I run it as a service it seems to crash the first
> time I use a RegExp.find().

As I said before, I don't know if the thread being created by the windows  
service procedure is properly initializing the D modules of the  
library/runtime.  Try as the first line of ServiceMain to initialize the  
current thread:

auto mythread = thread_attachThis();

see http://www.digitalmars.com/d/2.0/phobos/core_thread.html

-Steve
November 19, 2010
Re: Shared pain
> As I said before, I don't know if the thread being created by the
> windows service procedure is properly initializing the D modules of the
> library/runtime.  Try as the first line of ServiceMain to initialize the
> current thread:
> 
> auto mythread = thread_attachThis();
> 
> see http://www.digitalmars.com/d/2.0/phobos/core_thread.html
> 
> -Steve

Steve,

I don't think it gets as far as ServiceMain. Anyway, I tried it there, 
and at the point just before the first WINAPI call, but it still crashes.

I can fix it, by making my statically initialized RegExp objects __gshared 
or immutable, but the latter involves inserting a slew of tedious casts 
into my XML parser, since RexExp calls are used all over the place.

See my separate post for my thoughts on that.

Thanks
Steve
November 19, 2010
Re: Shared pain
On Thu, 18 Nov 2010 11:26:39 +0000, Steve Teale wrote:

> I had D code that provided a basis for creation of Windows services,
> which I have just tried to get working with the latest D2.
> 
> No dice.
> 
I have made some progress in understanding this. It appears that any D 
static data structures containing function pointers will now cause a 
Windows service to crash.

For happiness, these must be forced into the global data segment.

I have demonstrated this within a vestigial service for a couple of cases 
- plain old struct containing an int and a function pointer, and for 
Regexp objects by using and not using __gshared on the offending objects.

Now that I'm reasonably sure what's happening, I ought to be able to make 
these things work by making such static members immutable, and 
initializing them in static this().

However, I then run into a slew of complier errors. Lets start with this 
simple framework:

import std.stdio;
import std.regexp;

class ThingWithStatic
{
  static RegExp rex;
  string ns;

  static this()
  {
     rex = RegExp("ab+a");
  }

  this()
  {
     ns = "abbaabba";
  }

  int getFromStatic() { return rex.find(ns); }
}

void main()
{
  ThingWithStatic tws = new ThingWithStatic();
  writefln("%d", tws.getFromStatic());
}

This compiles fine, but if I include this class in a service, and call 
getFromStatic(), the service will crash. If I change to:

__gshared RegExp rex;

It compiles fine, and the service does not crash.

If I change to:

  static immutable RegExp rex;

Then I get errors:

triv.d(11): Error: cannot implicitly convert expression (opCall("ab
+a",null)) of type std.regexp.RegExp to immutable(RegExp)
triv.d(19): Error: function std.regexp.RegExp.find (string string) is not 
callable using argument types (string) immutable
steve@Ubuntu:~/scratch$

So then I use a cast:

rex = cast(immutable(RegExp)) RegExp("ab+a");

and get down to:

triv.d(19): Error: function std.regexp.RegExp.find (string string) is not 
callable using argument types (string) immutable

as somewhat wierd message in itself. So I use a cast in the call to find
():

return (cast(RegExp) rex).find(ns);

At this point it compiles, and if it's used in the service, the service 
does not crash.

BUT - all these complicated casts will be extremely tedious to 
administer to code that uses RexExp methods all  over the place, so 
__gshared is a great temptation. The compiler should try to eliminate 
such temptations in the interests of safety.

Also why are these casts necessary. If I am assigning to something that 
is the same type and was declared as immutable should not at least the 
first cast be done implicitly. In the second case, I'm using a Regexp 
object, wherever it may be stored, so why the cast? If find was pure 
would the cast still be required?

Thanks
Steve
November 19, 2010
Re: Shared pain
On Fri, 19 Nov 2010 01:49:30 -0500, Steve Teale  
<steve.teale@britseyeview.com> wrote:

>
>> As I said before, I don't know if the thread being created by the
>> windows service procedure is properly initializing the D modules of the
>> library/runtime.  Try as the first line of ServiceMain to initialize the
>> current thread:
>>
>> auto mythread = thread_attachThis();
>>
>> see http://www.digitalmars.com/d/2.0/phobos/core_thread.html
>>
>> -Steve
>
> Steve,
>
> I don't think it gets as far as ServiceMain. Anyway, I tried it there,
> and at the point just before the first WINAPI call, but it still crashes.
>
> I can fix it, by making my statically initialized RegExp objects  
> __gshared
> or immutable, but the latter involves inserting a slew of tedious casts
> into my XML parser, since RexExp calls are used all over the place.
>
> See my separate post for my thoughts on that.

OK, I'm out of ideas, sorry :(

It certainly looks like there is an issue with TLS.  I wouldn't suggest  
band-aiding things how you are doing, because some modules in  
phobos/druntime use TLS also.  It's reasonable to assume that if your uses  
of TLS are failing, those may fail as well.

-Steve
November 19, 2010
Re: Shared pain
A cast to immutable is required when constructing immutable objects. The type system can't prove that's safe. It's a design limitation to keep complexity low.

http://www.digitalmars.com/d/2.0/phobos/std_regexp.html
Looking at the Regexp docs, find isn't marked const or pure. If that's an oversite, file it in bugzilla. If it can't be const, then you should find another approach.

When this stuff crashes for you, is it accessing a different thread's TLS or is it simply unable to handle TLS at all?

Steve Teale Wrote:

> On Thu, 18 Nov 2010 11:26:39 +0000, Steve Teale wrote:
> 
> > I had D code that provided a basis for creation of Windows services,
> > which I have just tried to get working with the latest D2.
> > 
> > No dice.
> > 
> I have made some progress in understanding this. It appears that any D 
> static data structures containing function pointers will now cause a 
> Windows service to crash.
> 
> For happiness, these must be forced into the global data segment.
> 
> I have demonstrated this within a vestigial service for a couple of cases 
> - plain old struct containing an int and a function pointer, and for 
> Regexp objects by using and not using __gshared on the offending objects.
> 
> Now that I'm reasonably sure what's happening, I ought to be able to make 
> these things work by making such static members immutable, and 
> initializing them in static this().
> 
> However, I then run into a slew of complier errors. Lets start with this 
> simple framework:
> 
> import std.stdio;

> import std.regexp;

> 

> class ThingWithStatic

> {

>    static RegExp rex;

>    string ns;

> 

>    static this()

>    {

>       rex = RegExp("ab+a");

>    }

> 

>    this()

>    {

>       ns = "abbaabba";

>    }

> 

>    int getFromStatic() { return rex.find(ns); }

> }

> 

> void main()

> {

>    ThingWithStatic tws = new ThingWithStatic();

>    writefln("%d", tws.getFromStatic());

> }
> 
> This compiles fine, but if I include this class in a service, and call 
> getFromStatic(), the service will crash. If I change to:
> 
> __gshared RegExp rex;
> 
> It compiles fine, and the service does not crash.
> 
> If I change to:
> 
>    static immutable RegExp rex;
> 
> Then I get errors:
> 
> triv.d(11): Error: cannot implicitly convert expression (opCall("ab
> +a",null)) of type std.regexp.RegExp to immutable(RegExp)
> triv.d(19): Error: function std.regexp.RegExp.find (string string) is not 
> callable using argument types (string) immutable
> steve@Ubuntu:~/scratch$
> 
> So then I use a cast:
> 
> rex = cast(immutable(RegExp)) RegExp("ab+a");
> 
> and get down to:
> 
> triv.d(19): Error: function std.regexp.RegExp.find (string string) is not 
> callable using argument types (string) immutable
> 
> as somewhat wierd message in itself. So I use a cast in the call to find
> ():
> 
>  return (cast(RegExp) rex).find(ns);
>  
>  At this point it compiles, and if it's used in the service, the service 
> does not crash.
>  
>  BUT - all these complicated casts will be extremely tedious to 
> administer to code that uses RexExp methods all  over the place, so 
> __gshared is a great temptation. The compiler should try to eliminate 
> such temptations in the interests of safety.
>  
> Also why are these casts necessary. If I am assigning to something that 
> is the same type and was declared as immutable should not at least the 
> first cast be done implicitly. In the second case, I'm using a Regexp 
> object, wherever it may be stored, so why the cast? If find was pure 
> would the cast still be required?
> 
> Thanks
> Steve
November 19, 2010
windows debug [Shared pain]
18.11.2010 21:53, Steve Teale пишет:
> As you can imagine, this is terrible to debug - printf or similar is bad
> enough, but for the service I'm pushing entries into the Windows event
> logger system. [...]
>
> But first I think I'll stop and do some carpentry for a couple of days.
> Then eventually something might dawn on me.
Have you tried OutputDebugString()? It's WinAPI function that sends 
strings to the 'debugger'. DebugView application from SysInternals suite 
nicely prints those strings, and also times them.
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home