Jump to page: 1 2
Thread overview
Shared pain
Nov 18, 2010
Steve Teale
Nov 18, 2010
Jason House
Nov 18, 2010
Steve Teale
Nov 19, 2010
Steve Teale
windows debug [Shared pain]
Nov 19, 2010
Stanislav Blinov
Nov 19, 2010
Steve Teale
Nov 19, 2010
Stanislav Blinov
Nov 19, 2010
Steve Teale
Nov 19, 2010
Jason House
Nov 19, 2010
Steve Teale
Nov 19, 2010
bearophile
Nov 19, 2010
Steve Teale
Nov 19, 2010
Fawzi Mohamed
Nov 19, 2010
bearophile
Nov 19, 2010
Jason House
Nov 20, 2010
Steve Teale
Feb 19, 2014
HeiHon
November 18, 2010
I had D code that provided a basis for creation of Windows services, which I have just tried to get working with the latest D2.

No dice.

The point of failure was in this method

static void StartService()
{
   if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *) &_sta[0]))
   {
      ...
   }
}

Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code that install and remove the service worked fine, but the service control dispatcher was clearly not happy about these structs being in TLS, and the service crashed immediately when I tried to start it.

The class also has a couple of methods with signatures like:

extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}

That's probably not relevant, but I think it contributed to a host of errors that finally made be take a big step back.

I re-implemented the thing in the style used for OOP in C.

struct ServiceBase
{
   SERVICE_TABLE_ENTRY[2] _sta;
   ...
}

__gshared ServiceBase __sb;

void initialize(ServiceBase* sb) { ... }
void whatever(ServiceBase* sb ...) { ... }

and used the __sb global within the functions with the Windows signature, which just became

extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}

Now it's working again, but I want to change it into idiomatically sound D code.

Style suggestions please!

Steve
November 18, 2010
I'm not familiar with the API, but are you able to declare your original _sta as immutable and call it a day? Immutable data should have the same protection as __gshared but with no implications of bypassing the type system.

If that's not helpful, can you give more details like calling patterns? Is the function called from one thread or from many? My quick read of the windows documentation makes it seem like that's true. Probably unimportant, but I'd use _sta.ptr instead of &_sta[0] because the former more clearly implies array pointer to me, but that may be my own quirky style.

Steve Teale Wrote:

> I had D code that provided a basis for creation of Windows services, which I have just tried to get working with the latest D2.
> 
> No dice.
> 
> The point of failure was in this method
> 
> static void StartService()

> {

>    if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *) &_sta[0]))

>    {
>       ...

>    }

> }
> 
> Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code that install and remove the service worked fine, but the service control dispatcher was clearly not happy about these structs being in TLS, and the service crashed immediately when I tried to start it.
> 
> The class also has a couple of methods with signatures like:
> 
> extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}
> 
> That's probably not relevant, but I think it contributed to a host of errors that finally made be take a big step back.
> 
> I re-implemented the thing in the style used for OOP in C.
> 
> struct ServiceBase
> {
>    SERVICE_TABLE_ENTRY[2] _sta;
>    ...
> }
> 
> __gshared ServiceBase __sb;
> 
> void initialize(ServiceBase* sb) { ... }
> void whatever(ServiceBase* sb ...) { ... }
> 
> and used the __sb global within the functions with the Windows signature, which just became
> 
> extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}
> 
> Now it's working again, but I want to change it into idiomatically sound D code.
> 
> Style suggestions please!
> 
> Steve


November 18, 2010
On Thu, 18 Nov 2010 06:26:39 -0500, Steve Teale <steve.teale@britseyeview.com> wrote:

> I had D code that provided a basis for creation of Windows services,
> which I have just tried to get working with the latest D2.
>
> No dice.
>
> The point of failure was in this method
>
> static void StartService()
>
> {
>
>    if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *) &_sta[0]))
>
>    {
>       ...
>
>    }
>
> }
>
> Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code
> that install and remove the service worked fine, but the service control
> dispatcher was clearly not happy about these structs being in TLS, and
> the service crashed immediately when I tried to start it.

I think this has something to do with how services start.  From this page: http://msdn.microsoft.com/en-us/library/ms685990(v=VS.85).aspx
I see that StartServiceCtrlDispatcher creates its own thread to run ServiceMain, which most likely does not call the D thread initialization routines (not sure).  TLS may not be properly set up inside your thread that is running ServiceMain, and probably the static this() module functions have not been called.

>
> The class also has a couple of methods with signatures like:
>
> extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}
>
> That's probably not relevant, but I think it contributed to a host of
> errors that finally made be take a big step back.
>
> I re-implemented the thing in the style used for OOP in C.
>
> struct ServiceBase
> {
>    SERVICE_TABLE_ENTRY[2] _sta;
>    ...
> }
>
> __gshared ServiceBase __sb;
>
> void initialize(ServiceBase* sb) { ... }
> void whatever(ServiceBase* sb ...) { ... }
>
> and used the __sb global within the functions with the Windows signature,
> which just became
>
> extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}
>
> Now it's working again, but I want to change it into idiomatically sound
> D code.
>
> Style suggestions please!

I think using a class in D is a good idea.  But what you may need is global functions which forward to your class methods.  This is how I would probably do it.

-Steve
November 18, 2010
On Thu, 18 Nov 2010 08:33:12 -0500, Steven Schveighoffer wrote:

> 
> I think using a class in D is a good idea.  But what you may need is global functions which forward to your class methods.  This is how I would probably do it.
> 
> -Steve

Steve,

Thanks. I kind of went that way. Having done the laborious C OOP style thing, I put most of the methods back into the struct, just leaving the extern(Windows) functions at global scope - they referred to the __gshare'd object explicitly - I hate that.

Then I hid all of the crap behind a class that sneakily referred to the __gshared struct, but at least now to an external user of the module the interface looks much the same as it did before.

Now it works OK with the test case example - a service that just writes entries in the event log. However, the real service I'm trying to update works fine if I run it in debug mode (i.e. basically as just an executable), but if I run it as a service it seems to crash the first time I use a RegExp.find().

As you can imagine, this is terrible to debug - printf or similar is bad enough, but for the service I'm pushing entries into the Windows event logger system. Fortunately my D class for doing that has survived the D2 changes. I'm going to try tweaking the service so it will interact with the desktop, then at least I can revert to printf.

But first I think I'll stop and do some carpentry for a couple of days. Then eventually something might dawn on me.

Steve

November 18, 2010
On Thu, 18 Nov 2010 13:53:53 -0500, Steve Teale <steve.teale@britseyeview.com> wrote:

> On Thu, 18 Nov 2010 08:33:12 -0500, Steven Schveighoffer wrote:
>
>>
>> I think using a class in D is a good idea.  But what you may need is
>> global functions which forward to your class methods.  This is how I
>> would probably do it.
>>
>> -Steve
>
> Steve,
>
> Thanks. I kind of went that way. Having done the laborious C OOP style
> thing, I put most of the methods back into the struct, just leaving the
> extern(Windows) functions at global scope - they referred to the
> __gshare'd object explicitly - I hate that.
>
> Then I hid all of the crap behind a class that sneakily referred to the
> __gshared struct, but at least now to an external user of the module the
> interface looks much the same as it did before.
>
> Now it works OK with the test case example - a service that just writes
> entries in the event log. However, the real service I'm trying to update
> works fine if I run it in debug mode (i.e. basically as just an
> executable), but if I run it as a service it seems to crash the first
> time I use a RegExp.find().

As I said before, I don't know if the thread being created by the windows service procedure is properly initializing the D modules of the library/runtime.  Try as the first line of ServiceMain to initialize the current thread:

auto mythread = thread_attachThis();

see http://www.digitalmars.com/d/2.0/phobos/core_thread.html

-Steve
November 19, 2010
> As I said before, I don't know if the thread being created by the windows service procedure is properly initializing the D modules of the library/runtime.  Try as the first line of ServiceMain to initialize the current thread:
> 
> auto mythread = thread_attachThis();
> 
> see http://www.digitalmars.com/d/2.0/phobos/core_thread.html
> 
> -Steve

Steve,

I don't think it gets as far as ServiceMain. Anyway, I tried it there, and at the point just before the first WINAPI call, but it still crashes.

I can fix it, by making my statically initialized RegExp objects __gshared or immutable, but the latter involves inserting a slew of tedious casts into my XML parser, since RexExp calls are used all over the place.

See my separate post for my thoughts on that.

Thanks
Steve

November 19, 2010
On Thu, 18 Nov 2010 11:26:39 +0000, Steve Teale wrote:

> I had D code that provided a basis for creation of Windows services, which I have just tried to get working with the latest D2.
> 
> No dice.
> 
I have made some progress in understanding this. It appears that any D static data structures containing function pointers will now cause a Windows service to crash.

For happiness, these must be forced into the global data segment.

I have demonstrated this within a vestigial service for a couple of cases - plain old struct containing an int and a function pointer, and for Regexp objects by using and not using __gshared on the offending objects.

Now that I'm reasonably sure what's happening, I ought to be able to make these things work by making such static members immutable, and initializing them in static this().

However, I then run into a slew of complier errors. Lets start with this simple framework:

import std.stdio;
import std.regexp;

class ThingWithStatic
{
   static RegExp rex;
   string ns;

   static this()
   {
      rex = RegExp("ab+a");
   }

   this()
   {
      ns = "abbaabba";
   }

   int getFromStatic() { return rex.find(ns); }
}

void main()
{
   ThingWithStatic tws = new ThingWithStatic();
   writefln("%d", tws.getFromStatic());
}

This compiles fine, but if I include this class in a service, and call getFromStatic(), the service will crash. If I change to:

__gshared RegExp rex;

It compiles fine, and the service does not crash.

If I change to:

   static immutable RegExp rex;

Then I get errors:

triv.d(11): Error: cannot implicitly convert expression (opCall("ab
+a",null)) of type std.regexp.RegExp to immutable(RegExp)
triv.d(19): Error: function std.regexp.RegExp.find (string string) is not
callable using argument types (string) immutable
steve@Ubuntu:~/scratch$

So then I use a cast:

rex = cast(immutable(RegExp)) RegExp("ab+a");

and get down to:

triv.d(19): Error: function std.regexp.RegExp.find (string string) is not callable using argument types (string) immutable

as somewhat wierd message in itself. So I use a cast in the call to find
():

 return (cast(RegExp) rex).find(ns);

 At this point it compiles, and if it's used in the service, the service
does not crash.

 BUT - all these complicated casts will be extremely tedious to
administer to code that uses RexExp methods all  over the place, so
__gshared is a great temptation. The compiler should try to eliminate
such temptations in the interests of safety.

Also why are these casts necessary. If I am assigning to something that
is the same type and was declared as immutable should not at least the
first cast be done implicitly. In the second case, I'm using a Regexp
object, wherever it may be stored, so why the cast? If find was pure
would the cast still be required?

Thanks
Steve
November 19, 2010
On Fri, 19 Nov 2010 01:49:30 -0500, Steve Teale <steve.teale@britseyeview.com> wrote:

>
>> As I said before, I don't know if the thread being created by the
>> windows service procedure is properly initializing the D modules of the
>> library/runtime.  Try as the first line of ServiceMain to initialize the
>> current thread:
>>
>> auto mythread = thread_attachThis();
>>
>> see http://www.digitalmars.com/d/2.0/phobos/core_thread.html
>>
>> -Steve
>
> Steve,
>
> I don't think it gets as far as ServiceMain. Anyway, I tried it there,
> and at the point just before the first WINAPI call, but it still crashes.
>
> I can fix it, by making my statically initialized RegExp objects __gshared
> or immutable, but the latter involves inserting a slew of tedious casts
> into my XML parser, since RexExp calls are used all over the place.
>
> See my separate post for my thoughts on that.

OK, I'm out of ideas, sorry :(

It certainly looks like there is an issue with TLS.  I wouldn't suggest band-aiding things how you are doing, because some modules in phobos/druntime use TLS also.  It's reasonable to assume that if your uses of TLS are failing, those may fail as well.

-Steve
November 19, 2010
A cast to immutable is required when constructing immutable objects. The type system can't prove that's safe. It's a design limitation to keep complexity low.

http://www.digitalmars.com/d/2.0/phobos/std_regexp.html
Looking at the Regexp docs, find isn't marked const or pure. If that's an oversite, file it in bugzilla. If it can't be const, then you should find another approach.

When this stuff crashes for you, is it accessing a different thread's TLS or is it simply unable to handle TLS at all?

Steve Teale Wrote:

> On Thu, 18 Nov 2010 11:26:39 +0000, Steve Teale wrote:
> 
> > I had D code that provided a basis for creation of Windows services, which I have just tried to get working with the latest D2.
> > 
> > No dice.
> > 
> I have made some progress in understanding this. It appears that any D static data structures containing function pointers will now cause a Windows service to crash.
> 
> For happiness, these must be forced into the global data segment.
> 
> I have demonstrated this within a vestigial service for a couple of cases - plain old struct containing an int and a function pointer, and for Regexp objects by using and not using __gshared on the offending objects.
> 
> Now that I'm reasonably sure what's happening, I ought to be able to make these things work by making such static members immutable, and initializing them in static this().
> 
> However, I then run into a slew of complier errors. Lets start with this simple framework:
> 
> import std.stdio;

> import std.regexp;

> 

> class ThingWithStatic

> {

>    static RegExp rex;

>    string ns;

> 

>    static this()

>    {

>       rex = RegExp("ab+a");

>    }

> 

>    this()

>    {

>       ns = "abbaabba";

>    }

> 

>    int getFromStatic() { return rex.find(ns); }

> }

> 

> void main()

> {

>    ThingWithStatic tws = new ThingWithStatic();

>    writefln("%d", tws.getFromStatic());

> }
> 
> This compiles fine, but if I include this class in a service, and call getFromStatic(), the service will crash. If I change to:
> 
> __gshared RegExp rex;
> 
> It compiles fine, and the service does not crash.
> 
> If I change to:
> 
>    static immutable RegExp rex;
> 
> Then I get errors:
> 
> triv.d(11): Error: cannot implicitly convert expression (opCall("ab
> +a",null)) of type std.regexp.RegExp to immutable(RegExp)
> triv.d(19): Error: function std.regexp.RegExp.find (string string) is not
> callable using argument types (string) immutable
> steve@Ubuntu:~/scratch$
> 
> So then I use a cast:
> 
> rex = cast(immutable(RegExp)) RegExp("ab+a");
> 
> and get down to:
> 
> triv.d(19): Error: function std.regexp.RegExp.find (string string) is not callable using argument types (string) immutable
> 
> as somewhat wierd message in itself. So I use a cast in the call to find
> ():
> 
>  return (cast(RegExp) rex).find(ns);
> 
>  At this point it compiles, and if it's used in the service, the service
> does not crash.
> 
>  BUT - all these complicated casts will be extremely tedious to
> administer to code that uses RexExp methods all  over the place, so
> __gshared is a great temptation. The compiler should try to eliminate
> such temptations in the interests of safety.
> 
> Also why are these casts necessary. If I am assigning to something that
> is the same type and was declared as immutable should not at least the
> first cast be done implicitly. In the second case, I'm using a Regexp
> object, wherever it may be stored, so why the cast? If find was pure
> would the cast still be required?
> 
> Thanks
> Steve

November 19, 2010
18.11.2010 21:53, Steve Teale пишет:
> As you can imagine, this is terrible to debug - printf or similar is bad
> enough, but for the service I'm pushing entries into the Windows event
> logger system. [...]
>
> But first I think I'll stop and do some carpentry for a couple of days.
> Then eventually something might dawn on me.
Have you tried OutputDebugString()? It's WinAPI function that sends strings to the 'debugger'. DebugView application from SysInternals suite nicely prints those strings, and also times them.
« First   ‹ Prev
1 2