Hi all,

This is a long shot but I'm out of ideas. I ported an app from Windows to Linux and after many issues it is working but I'm left with a strange problem. The app basically reads data streams from a USB device, processes them and outputs real-time graphical data to a browser. There is also some control input from the browser. The interface to the browser is web sockets for which I have written a D web-socket server. Depending on how much of my application I allow to run I get a stream of these errors.

Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
----------------
./DcSdr() [0x80aa2ed]
./DcSdr() [0x806f52b]
./DcSdr() [0x804f752]
./DcSdr() [0x809b422]
./DcSdr() [0x80ae77e]
/lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]

This is the web-socket accept. I seem to fall straight through the accept call even when I am not making any connection attempt. When I do connect the stream stops and I get one of these outputs each time I send data from the browser. I should not even be near the accept at that point as the connection is made.

The app appears to be unaffected and works as expected. The odd think is if I shut down part of the app these errors stop. I can't tie it down to any specific thing but I suspect threading as the number of threads is reduced by stopping parts of the app. The error also seems to indicate threads are involved. I did not get this problem on Windows.

Any ideas would be much appreciated.

Thanks
bob

Bob Cowdery wrote:
> Hi all,
> 
> This is a long shot but I'm out of ideas. I ported an app from Windows to Linux and after many issues it is working but I'm left with a strange problem. The app basically reads data streams from a USB device, processes them and outputs real-time graphical data to a browser. There is also some control input from the browser. The interface to the browser is web sockets for which I have written a D web-socket server. Depending on how much of my application I allow to run I get a stream of these errors.
> 
> Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
> ----------------
> ./DcSdr() [0x80aa2ed]
> ./DcSdr() [0x806f52b]
> ./DcSdr() [0x804f752]
> ./DcSdr() [0x809b422]
> ./DcSdr() [0x80ae77e]
> /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
> /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
> 
> This is the web-socket accept. I seem to fall straight through the accept call even when I am not making any connection attempt. When I do connect the stream stops and I get one of these outputs each time I send data from the browser. I should not even be near the accept at that point as the connection is made.
> 
> The app appears to be unaffected and works as expected. The odd think is if I shut down part of the app these errors stop. I can't tie it down to any specific thing but I suspect threading as the number of threads is reduced by stopping parts of the app. The error also seems to indicate threads are involved. I did not get this problem on Windows.
> 
> Any ideas would be much appreciated.
> 
	Do you have several threads trying to accept connections on the
same socket simultaneously? If that is the case, the first thread
would block on the socket until your client connects and the other
threads will fail in the call to "accept".

		Jerome
-- 
mailto:jeberger@free.fr
http://jeberger.free.fr
Jabber: jeberger@jabber.fr

On 21/12/2010 22:22, "Jérôme M. Berger" wrote:
> Bob Cowdery wrote:
>> Hi all,
>>
>> This is a long shot but I'm out of ideas. I ported an app from Windows to Linux and after many issues it is working but I'm left with a strange problem. The app basically reads data streams from a USB device, processes them and outputs real-time graphical data to a browser. There is also some control input from the browser. The interface to the browser is web sockets for which I have written a D web-socket server. Depending on how much of my application I allow to run I get a stream of these errors.
>>
>> Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
>> ----------------
>> ./DcSdr() [0x80aa2ed]
>> ./DcSdr() [0x806f52b]
>> ./DcSdr() [0x804f752]
>> ./DcSdr() [0x809b422]
>> ./DcSdr() [0x80ae77e]
>> /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
>> /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
>>
>> This is the web-socket accept. I seem to fall straight through the accept call even when I am not making any connection attempt. When I do connect the stream stops and I get one of these outputs each time I send data from the browser. I should not even be near the accept at that point as the connection is made.
>>
>> The app appears to be unaffected and works as expected. The odd think is if I shut down part of the app these errors stop. I can't tie it down to any specific thing but I suspect threading as the number of threads is reduced by stopping parts of the app. The error also seems to indicate threads are involved. I did not get this problem on Windows.
>>
>> Any ideas would be much appreciated.
>>
> 	Do you have several threads trying to accept connections on the
> same socket simultaneously? If that is the case, the first thread
> would block on the socket until your client connects and the other
> threads will fail in the call to "accept".
>
> 		Jerome
The listner is a single thread that creates an instance of my web socket class for each connection. I'm not trying to support lots of users, most of the time just one. The listner code is almost the same as the sockets example. If you or anyone else can spare a few moments the code can be browsed at http://code.google.com/p/dcsdr/source/browse (branches/linux/DcSdr/WsServer/listener.d). The trunk is the Windows version. I'd be very happy if it's something I'm doing wrong.

bob

Bob Cowdery wrote:
> The listner is a single thread that creates an instance of my web socket class for each connection. I'm not trying to support lots of users, most of the time just one. The listner code is almost the same as the sockets example. If you or anyone else can spare a few moments the code can be browsed at http://code.google.com/p/dcsdr/source/browse (branches/linux/DcSdr/WsServer/listener.d). The trunk is the Windows version. I'd be very happy if it's something I'm doing wrong.
> 
	I looked at your code and I don't see what could cause this issue.
Sorry.

	I did however find a bug: when a client connection is closed, you
remove the corresponding socket from reads but you don't remove the
WebSocket from wsset...

		Jerome
-- 
mailto:jeberger@free.fr
http://jeberger.free.fr
Jabber: jeberger@jabber.fr

On 22/12/2010 10:04, "Jérôme M. Berger" wrote:
> Bob Cowdery wrote:
>> The listner is a single thread that creates an instance of my web socket class for each connection. I'm not trying to support lots of users, most of the time just one. The listner code is almost the same as the sockets example. If you or anyone else can spare a few moments the code can be browsed at http://code.google.com/p/dcsdr/source/browse (branches/linux/DcSdr/WsServer/listener.d). The trunk is the Windows version. I'd be very happy if it's something I'm doing wrong.
>>
> 	I looked at your code and I don't see what could cause this issue.
> Sorry.
>
> 	I did however find a bug: when a client connection is closed, you
> remove the corresponding socket from reads but you don't remove the
> WebSocket from wsset...
>
> 		Jerome
Thanks for looking and pointing out the bug. That could be causing some odd connection issues I see when I have to restart things. I didn't expect it to be a simple coding fault as it goes away when I stop unrelated threads. I guess my next step is to try and reproduce with the minimum of code.

bob

Hi Bob!


My guess: You're listener is set to be non-blocking. That means that when you call listener.accept() it will return immediately with an SocketAcceptException, if there's no connection. And you're basically calling listener.accept() over and over again in an infinite loop.

The following example shows it:

import std.concurrency, std.stdio, std.conv, std.socket;
void main()
{
	ushort port = 9999;
	Socket listener = new TcpSocket;
	assert(listener.isAlive);
	listener.blocking = false;
	listener.bind(new InternetAddress(port));
	listener.listen(10);
	writeln("Listening on port: ", port);
	Socket sn;
	while(true){
		try
		{
			writeln ("Accepting");
			sn = listener.accept();
			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
			assert(sn.isAlive);
			assert(listener.isAlive);
			break;
		}
		catch(Exception e)
		{
			writeln("Error accepting: ", e.toString() );
			if(sn)
			sn.close();
		}
	}
	writeln("end");
}

Running the example will print
    Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for instance opening the URL in a browser) socket.accept _does_ return a connection and the program prints "end".

I don't know why the program doesn't do this on Windows. As far as I can tell the endless exceptions _is_ the correct behaviour, right?

Anyway, if you comment out the the line
    listener.blocking = false;
in 'listener.d', does it work as intended then? In the example above this will cause the listener.accept()-call to actually wait until it gets a connection, and thus not spew out all the exceptions.

BR
/heywood




Bob Cowdery Wrote:

> Hi all,
> 
> This is a long shot but I'm out of ideas. I ported an app from Windows to Linux and after many issues it is working but I'm left with a strange problem. The app basically reads data streams from a USB device, processes them and outputs real-time graphical data to a browser. There is also some control input from the browser. The interface to the browser is web sockets for which I have written a D web-socket server. Depending on how much of my application I allow to run I get a stream of these errors.
> 
> Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
> ----------------
> ./DcSdr() [0x80aa2ed]
> ./DcSdr() [0x806f52b]
> ./DcSdr() [0x804f752]
> ./DcSdr() [0x809b422]
> ./DcSdr() [0x80ae77e]
> /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
> /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
> 
> This is the web-socket accept. I seem to fall straight through the accept call even when I am not making any connection attempt. When I do connect the stream stops and I get one of these outputs each time I send data from the browser. I should not even be near the accept at that point as the connection is made.
> 
> The app appears to be unaffected and works as expected. The odd think is if I shut down part of the app these errors stop. I can't tie it down to any specific thing but I suspect threading as the number of threads is reduced by stopping parts of the app. The error also seems to indicate threads are involved. I did not get this problem on Windows.
> 
> Any ideas would be much appreciated.
> 
> Thanks
> bob

Hi Heywood

Thankyou for your time. Yes I agree making the call blocking does stop the exceptions churning. Unfortunately the application stops accepting data now because after the first incoming transfer from the web socket client it sees data on the listening socket and promptly blocks on it and never comes off until I make another connection. I think this is expected behaviour. I could spawn a thread for each connection which I would normally do but shouldn't need to as it's really only a few users and I believe that would only sidestep the real problem.

There are lots of things here that don't make sense.

1. It works for Windows and it should work for Linux (and Mac) unless
Windows is broken and Linux is behaving as it should.
2. I should only see data on the listen socket when I make a new
connection. What I see is data on the listen socket when I send data on
the connected socket and with non-blocking even when I don't make a
connection or send any data at all.
2. The really confusing thing is if I stop my USB and DSP threads it
stops behaving like this and sees data on the correct sockets. In other
words it works as it does on Windows so I have to assume this is the
correct behaviour.
3. I've played around with trying to figure out exactly what it is in
the other threads that causes the behaviour. I got as far as finding
that only the DSP thead is required so that rules out misbehaving USB.
Very oddly there is a loop that is creating some Json data (not very
efficiently) as it's doing a lot of string concatenation. If I comment
out this loop or reduce its iterations the exception slow down to the
point where I just get 2 and then they stop. This points to something
horrible going on.

I can only hope I've done something stupid that I just happen to be getting away with on Windows. If it's a bug in the compiler or libraries I think I'm stuffed as I wouldn't know where to start.

Regards
bob

On 23/12/2010 00:20, Heywood Floyd wrote:
> Hi Bob!
>
>
> My guess: You're listener is set to be non-blocking. That means that when you call listener.accept() it will return immediately with an SocketAcceptException, if there's no connection. And you're basically calling listener.accept() over and over again in an infinite loop.
>
> The following example shows it:
>
> import std.concurrency, std.stdio, std.conv, std.socket;
> void main()
> {
> 	ushort port = 9999;
> 	Socket listener = new TcpSocket;
> 	assert(listener.isAlive);
> 	listener.blocking = false;
> 	listener.bind(new InternetAddress(port));
> 	listener.listen(10);
> 	writeln("Listening on port: ", port);
> 	Socket sn;
> 	while(true){
> 		try
> 		{
> 			writeln ("Accepting");
> 			sn = listener.accept();
> 			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
> 			assert(sn.isAlive);
> 			assert(listener.isAlive);
> 			break;
> 		}
> 		catch(Exception e)
> 		{
> 			writeln("Error accepting: ", e.toString() );
> 			if(sn)
> 			sn.close();
> 		}
> 	}
> 	writeln("end");
> }
>
> Running the example will print
>     Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
> for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for instance opening the URL in a browser) socket.accept _does_ return a connection and the program prints "end".
>
> I don't know why the program doesn't do this on Windows. As far as I can tell the endless exceptions _is_ the correct behaviour, right?
>
> Anyway, if you comment out the the line
>     listener.blocking = false;
> in 'listener.d', does it work as intended then? In the example above this will cause the listener.accept()-call to actually wait until it gets a connection, and thus not spew out all the exceptions.
>
> BR
> /heywood
>
>
>
>
> Bob Cowdery Wrote:
>
>> Hi all,
>>
>> This is a long shot but I'm out of ideas. I ported an app from Windows to Linux and after many issues it is working but I'm left with a strange problem. The app basically reads data streams from a USB device, processes them and outputs real-time graphical data to a browser. There is also some control input from the browser. The interface to the browser is web sockets for which I have written a D web-socket server. Depending on how much of my application I allow to run I get a stream of these errors.
>>
>> Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
>> ----------------
>> ./DcSdr() [0x80aa2ed]
>> ./DcSdr() [0x806f52b]
>> ./DcSdr() [0x804f752]
>> ./DcSdr() [0x809b422]
>> ./DcSdr() [0x80ae77e]
>> /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
>> /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
>>
>> This is the web-socket accept. I seem to fall straight through the accept call even when I am not making any connection attempt. When I do connect the stream stops and I get one of these outputs each time I send data from the browser. I should not even be near the accept at that point as the connection is made.
>>
>> The app appears to be unaffected and works as expected. The odd think is if I shut down part of the app these errors stop. I can't tie it down to any specific thing but I suspect threading as the number of threads is reduced by stopping parts of the app. The error also seems to indicate threads are involved. I did not get this problem on Windows.
>>
>> Any ideas would be much appreciated.
>>
>> Thanks
>> bob

Hi!


I see.

I think my previous answer was a bit naive—I didn't appreciate the full scope of the problem. Sorry for that, but you know, internet is fast, snap snap : )

Ok, for now I'm afraid I don't have any more to add. (An isolated example would of course help greatly!)

All I can say is, in my experience, the D sockets library _does_ behave differently on different platforms. Had a server that just died seemingly random on osx, while working fine on Linux. Turned out to be bug in D sockets. So there you go. (The bug is still there btw... *cough*)

Hm, also, I realize I'm not sure I quite understand the structure of your app, and where in this structure the bug happends. Is this bug happening when your html/5-client tries to connect to the server? Or are the threads using sockets to communicate between eachother?


BR
/heywood



Bob Cowdery Wrote:

> Hi Heywood
> 
> Thankyou for your time. Yes I agree making the call blocking does stop the exceptions churning. Unfortunately the application stops accepting data now because after the first incoming transfer from the web socket client it sees data on the listening socket and promptly blocks on it and never comes off until I make another connection. I think this is expected behaviour. I could spawn a thread for each connection which I would normally do but shouldn't need to as it's really only a few users and I believe that would only sidestep the real problem.
> 
> There are lots of things here that don't make sense.
> 
> 1. It works for Windows and it should work for Linux (and Mac) unless
> Windows is broken and Linux is behaving as it should.
> 2. I should only see data on the listen socket when I make a new
> connection. What I see is data on the listen socket when I send data on
> the connected socket and with non-blocking even when I don't make a
> connection or send any data at all.
> 2. The really confusing thing is if I stop my USB and DSP threads it
> stops behaving like this and sees data on the correct sockets. In other
> words it works as it does on Windows so I have to assume this is the
> correct behaviour.
> 3. I've played around with trying to figure out exactly what it is in
> the other threads that causes the behaviour. I got as far as finding
> that only the DSP thead is required so that rules out misbehaving USB.
> Very oddly there is a loop that is creating some Json data (not very
> efficiently) as it's doing a lot of string concatenation. If I comment
> out this loop or reduce its iterations the exception slow down to the
> point where I just get 2 and then they stop. This points to something
> horrible going on.
> 
> I can only hope I've done something stupid that I just happen to be getting away with on Windows. If it's a bug in the compiler or libraries I think I'm stuffed as I wouldn't know where to start.
> 
> Regards
> bob
> 
> On 23/12/2010 00:20, Heywood Floyd wrote:
> > Hi Bob!
> >
> >
> > My guess: You're listener is set to be non-blocking. That means that when you call listener.accept() it will return immediately with an SocketAcceptException, if there's no connection. And you're basically calling listener.accept() over and over again in an infinite loop.
> >
> > The following example shows it:
> >
> > import std.concurrency, std.stdio, std.conv, std.socket;
> > void main()
> > {
> > 	ushort port = 9999;
> > 	Socket listener = new TcpSocket;
> > 	assert(listener.isAlive);
> > 	listener.blocking = false;
> > 	listener.bind(new InternetAddress(port));
> > 	listener.listen(10);
> > 	writeln("Listening on port: ", port);
> > 	Socket sn;
> > 	while(true){
> > 		try
> > 		{
> > 			writeln ("Accepting");
> > 			sn = listener.accept();
> > 			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
> > 			assert(sn.isAlive);
> > 			assert(listener.isAlive);
> > 			break;
> > 		}
> > 		catch(Exception e)
> > 		{
> > 			writeln("Error accepting: ", e.toString() );
> > 			if(sn)
> > 			sn.close();
> > 		}
> > 	}
> > 	writeln("end");
> > }
> >
> > Running the example will print
> >     Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
> > for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for instance opening the URL in a browser) socket.accept _does_ return a connection and the program prints "end".
> >
> > I don't know why the program doesn't do this on Windows. As far as I can tell the endless exceptions _is_ the correct behaviour, right?
> >
> > Anyway, if you comment out the the line
> >     listener.blocking = false;
> > in 'listener.d', does it work as intended then? In the example above this will cause the listener.accept()-call to actually wait until it gets a connection, and thus not spew out all the exceptions.
> >
> > BR
> > /heywood
> >
> >
> >
> >
> > Bob Cowdery Wrote:
> >
> >> Hi all,
> >>
> >> This is a long shot but I'm out of ideas. I ported an app from Windows to Linux and after many issues it is working but I'm left with a strange problem. The app basically reads data streams from a USB device, processes them and outputs real-time graphical data to a browser. There is also some control input from the browser. The interface to the browser is web sockets for which I have written a D web-socket server. Depending on how much of my application I allow to run I get a stream of these errors.
> >>
> >> Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
> >> ----------------
> >> ./DcSdr() [0x80aa2ed]
> >> ./DcSdr() [0x806f52b]
> >> ./DcSdr() [0x804f752]
> >> ./DcSdr() [0x809b422]
> >> ./DcSdr() [0x80ae77e]
> >> /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
> >> /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
> >>
> >> This is the web-socket accept. I seem to fall straight through the accept call even when I am not making any connection attempt. When I do connect the stream stops and I get one of these outputs each time I send data from the browser. I should not even be near the accept at that point as the connection is made.
> >>
> >> The app appears to be unaffected and works as expected. The odd think is if I shut down part of the app these errors stop. I can't tie it down to any specific thing but I suspect threading as the number of threads is reduced by stopping parts of the app. The error also seems to indicate threads are involved. I did not get this problem on Windows.
> >>
> >> Any ideas would be much appreciated.
> >>
> >> Thanks
> >> bob
> 

Hi
All communication is appreciated. I often find just trying to explain a
problem can lead to fixing it. Sorry this is a bit rambling.

I know D is not done yet and there will be bugs. I don't know yet if its me or D.

The program structure is quite simple.
1. I have 3 pieces of C code. An interface to libusb. A library of
signal processing functions. An embedded Mongoose HTTP server.
2. There is a main module which has the outer loop and it starts 4
threads. One for the module that talks to the USB hardware. One for the
module that talks to the signal processing library. One for the HTTP
server and one the Web Sockets server which is also the back end of the GUI.
3. These threads all register their TID's with a registry module and
then each has a receive loop. Messages are then passed between threads.
The main threads do start some internal threads as some have more than
one task to perform.

There is no connection at all between threads except the messages that are passed. It is pure message passing concurrency which is one of the major reasons I went for D.

On the web side first connection is from the browser to the HTTP server which simply provides the static content, html, css and js. Once the browser kicks itself into life it make a web socket connection to my web socket server and all chit chat goes on over this connection. I am streaming realtime graphics data to the browser and receiving data from it when I scroll digits, click buttons etc.

The only bug that is visible in the whole app occurs in the web socket server as discussed. The thing I was playing with that seems to effect it was firstly i stopped the USB and sig processing threads from being created and commented out a few things so the sockets side still functioned. This seemed to cure the problem completly. I put back the sig processing thread and got the full force of the problem. In the sig processing thread it constructs the grapic data to send to the browser as series of points to plot. Even if I don't actually send the data the problem still occurs. However, by not constructing the last bit which is a Json string it reduces the problem considerably.

Yes. I need to produce a smaller example that demonstrates this.

Regards
bob

On 23/12/2010 17:06, Heywood Floyd wrote:
> Hi!
>
>
> I see.
>
> I think my previous answer was a bit naive—I didn't appreciate the full scope of the problem. Sorry for that, but you know, internet is fast, snap snap : )
>
> Ok, for now I'm afraid I don't have any more to add. (An isolated example would of course help greatly!)
>
> All I can say is, in my experience, the D sockets library _does_ behave differently on different platforms. Had a server that just died seemingly random on osx, while working fine on Linux. Turned out to be bug in D sockets. So there you go. (The bug is still there btw... *cough*)
>
> Hm, also, I realize I'm not sure I quite understand the structure of your app, and where in this structure the bug happends. Is this bug happening when your html/5-client tries to connect to the server? Or are the threads using sockets to communicate between eachother?
>
>
> BR
> /heywood
>
>
>
> Bob Cowdery Wrote:
>
>> Hi Heywood
>>
>> Thankyou for your time. Yes I agree making the call blocking does stop the exceptions churning. Unfortunately the application stops accepting data now because after the first incoming transfer from the web socket client it sees data on the listening socket and promptly blocks on it and never comes off until I make another connection. I think this is expected behaviour. I could spawn a thread for each connection which I would normally do but shouldn't need to as it's really only a few users and I believe that would only sidestep the real problem.
>>
>> There are lots of things here that don't make sense.
>>
>> 1. It works for Windows and it should work for Linux (and Mac) unless
>> Windows is broken and Linux is behaving as it should.
>> 2. I should only see data on the listen socket when I make a new
>> connection. What I see is data on the listen socket when I send data on
>> the connected socket and with non-blocking even when I don't make a
>> connection or send any data at all.
>> 2. The really confusing thing is if I stop my USB and DSP threads it
>> stops behaving like this and sees data on the correct sockets. In other
>> words it works as it does on Windows so I have to assume this is the
>> correct behaviour.
>> 3. I've played around with trying to figure out exactly what it is in
>> the other threads that causes the behaviour. I got as far as finding
>> that only the DSP thead is required so that rules out misbehaving USB.
>> Very oddly there is a loop that is creating some Json data (not very
>> efficiently) as it's doing a lot of string concatenation. If I comment
>> out this loop or reduce its iterations the exception slow down to the
>> point where I just get 2 and then they stop. This points to something
>> horrible going on.
>>
>> I can only hope I've done something stupid that I just happen to be getting away with on Windows. If it's a bug in the compiler or libraries I think I'm stuffed as I wouldn't know where to start.
>>
>> Regards
>> bob
>>
>> On 23/12/2010 00:20, Heywood Floyd wrote:
>>> Hi Bob!
>>>
>>>
>>> My guess: You're listener is set to be non-blocking. That means that when you call listener.accept() it will return immediately with an SocketAcceptException, if there's no connection. And you're basically calling listener.accept() over and over again in an infinite loop.
>>>
>>> The following example shows it:
>>>
>>> import std.concurrency, std.stdio, std.conv, std.socket;
>>> void main()
>>> {
>>> 	ushort port = 9999;
>>> 	Socket listener = new TcpSocket;
>>> 	assert(listener.isAlive);
>>> 	listener.blocking = false;
>>> 	listener.bind(new InternetAddress(port));
>>> 	listener.listen(10);
>>> 	writeln("Listening on port: ", port);
>>> 	Socket sn;
>>> 	while(true){
>>> 		try
>>> 		{
>>> 			writeln ("Accepting");
>>> 			sn = listener.accept();
>>> 			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
>>> 			assert(sn.isAlive);
>>> 			assert(listener.isAlive);
>>> 			break;
>>> 		}
>>> 		catch(Exception e)
>>> 		{
>>> 			writeln("Error accepting: ", e.toString() );
>>> 			if(sn)
>>> 			sn.close();
>>> 		}
>>> 	}
>>> 	writeln("end");
>>> }
>>>
>>> Running the example will print
>>>     Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
>>> for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for instance opening the URL in a browser) socket.accept _does_ return a connection and the program prints "end".
>>>
>>> I don't know why the program doesn't do this on Windows. As far as I can tell the endless exceptions _is_ the correct behaviour, right?
>>>
>>> Anyway, if you comment out the the line
>>>     listener.blocking = false;
>>> in 'listener.d', does it work as intended then? In the example above this will cause the listener.accept()-call to actually wait until it gets a connection, and thus not spew out all the exceptions.
>>>
>>> BR
>>> /heywood
>>>
>>>
>>>
>>>
>>> Bob Cowdery Wrote:
>>>
>>>> Hi all,
>>>>
>>>> This is a long shot but I'm out of ideas. I ported an app from Windows to Linux and after many issues it is working but I'm left with a strange problem. The app basically reads data streams from a USB device, processes them and outputs real-time graphical data to a browser. There is also some control input from the browser. The interface to the browser is web sockets for which I have written a D web-socket server. Depending on how much of my application I allow to run I get a stream of these errors.
>>>>
>>>> Error accepting: std.socket.SocketAcceptException: Unable to accept socket connection: Resource temporarily unavailable
>>>> ----------------
>>>> ./DcSdr() [0x80aa2ed]
>>>> ./DcSdr() [0x806f52b]
>>>> ./DcSdr() [0x804f752]
>>>> ./DcSdr() [0x809b422]
>>>> ./DcSdr() [0x80ae77e]
>>>> /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
>>>> /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
>>>>
>>>> This is the web-socket accept. I seem to fall straight through the accept call even when I am not making any connection attempt. When I do connect the stream stops and I get one of these outputs each time I send data from the browser. I should not even be near the accept at that point as the connection is made.
>>>>
>>>> The app appears to be unaffected and works as expected. The odd think is if I shut down part of the app these errors stop. I can't tie it down to any specific thing but I suspect threading as the number of threads is reduced by stopping parts of the app. The error also seems to indicate threads are involved. I did not get this problem on Windows.
>>>>
>>>> Any ideas would be much appreciated.
>>>>
>>>> Thanks
>>>> bob

Perseverance pays off in the end. Having worked out how the socket class was supposed to work it seemed that the select call was returning when it was not supposed to and giving a false indication in the socket set of who spoke. When I looked at the return code of select() it was -1 (interrupted). Depending on how much of my other code was enabled the frequency of these interruptions changed which is why I couldn't home in on anything in particular. I don't know why the call is getting interrupted. I don't use signals anywhere but I suspect the D library is and possibly something to do with the receive function.

I can avoid the problem simply by sitting on the select until i get a >0 return code which means someone really did speak. I guess this does not happen on Windows because Windows does not use signals. It's only a work round. I would like to know why it's getting interrupted from someone who knows the language and library implementation.

Regards
Bob

On 23/12/2010 18:28, Bob Cowdery wrote:
> Hi
> All communication is appreciated. I often find just trying to explain a
> problem can lead to fixing it. Sorry this is a bit rambling.
>
> I know D is not done yet and there will be bugs. I don't know yet if its me or D.
>
> The program structure is quite simple.
> 1. I have 3 pieces of C code. An interface to libusb. A library of
> signal processing functions. An embedded Mongoose HTTP server.
> 2. There is a main module which has the outer loop and it starts 4
> threads. One for the module that talks to the USB hardware. One for the
> module that talks to the signal processing library. One for the HTTP
> server and one the Web Sockets server which is also the back end of the GUI.
> 3. These threads all register their TID's with a registry module and
> then each has a receive loop. Messages are then passed between threads.
> The main threads do start some internal threads as some have more than
> one task to perform.
>
> There is no connection at all between threads except the messages that are passed. It is pure message passing concurrency which is one of the major reasons I went for D.
>
> On the web side first connection is from the browser to the HTTP server which simply provides the static content, html, css and js. Once the browser kicks itself into life it make a web socket connection to my web socket server and all chit chat goes on over this connection. I am streaming realtime graphics data to the browser and receiving data from it when I scroll digits, click buttons etc.
>
> The only bug that is visible in the whole app occurs in the web socket server as discussed. The thing I was playing with that seems to effect it was firstly i stopped the USB and sig processing threads from being created and commented out a few things so the sockets side still functioned. This seemed to cure the problem completly. I put back the sig processing thread and got the full force of the problem. In the sig processing thread it constructs the grapic data to send to the browser as series of points to plot. Even if I don't actually send the data the problem still occurs. However, by not constructing the last bit which is a Json string it reduces the problem considerably.
>
> Yes. I need to produce a smaller example that demonstrates this.
>
> Regards
> bob