I realize that they aren't exactly identical, but I have a project in mind that should, logically, be done via forked processes...thousands of forked processes.  This is obviously impractical.  I could do two different redesigns:  one, based around threads, would still involve the creation of immense numbers of threads.  The other, based around procedure calls, would be theoretically a much poorer model.  (Well, a neural net *IS* a bunch of processes that execute relatively independently...)

Typically people decompose a neural net into collections of arrays, but I would prefer to model each "cell" as an object.  This is obviously a poor match of the design of the algorithm to the structure of the processor, but I suspect that there will be details revealed in the operation that will only be detected if each cell is an object.  So that's what I need.

The question is, just how inefficient would it be to model each "firing"
of a neuron as the spawning of a thread.  If I can do it this way then
the "cells" can operate pseudo-simultaneously.  If I can't, I'll need to
 add a separate buffer and a bunch of code to emulate synchronous
operation.  Messy.  Necessary?

Charles D Hixson wrote:

> I realize that they aren't exactly identical, but I have a project in mind that should, logically, be done via forked processes...thousands of forked processes.  This is obviously impractical.  I could do two different redesigns:  one, based around threads, would still involve the creation of immense numbers of threads.  The other, based around procedure calls, would be theoretically a much poorer model.  (Well, a neural net *IS* a bunch of processes that execute relatively independently...)
> 
> Typically people decompose a neural net into collections of arrays, but I would prefer to model each "cell" as an object.  This is obviously a poor match of the design of the algorithm to the structure of the processor, but I suspect that there will be details revealed in the operation that will only be detected if each cell is an object.  So that's what I need.
> 
> The question is, just how inefficient would it be to model each "firing"
> of a neuron as the spawning of a thread.  If I can do it this way then
> the "cells" can operate pseudo-simultaneously.  If I can't, I'll need to
>  add a separate buffer and a bunch of code to emulate synchronous
> operation.  Messy.  Necessary?

When you get into thousands, I think you should avoid at least normal threads. Light weight stack threads / fibers / co-routines might be a better solution, especially if you'd like cooperative switching (very nice for simulations). There has been floating a few suggestions around lately, search the news groups for the terms above. I don't think Mikola has released his updated version yet, though. I think it looked very good.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi

You might try breaking the action down into something like this:
It would let you control the number of threads (SMP?) and simulate as many as
you want.

struct dgbox{dgbox[] delegate dg();}


ThreadSafeQueue!(dgbox) queue;


void main()
{
queue.enque(&Root.StartPoint)

for(int i = 1; i <= numThreads; i++)
{
LaunchThread( function void ()
{
dgbox[] tmp;

while(!queue.empty)
{
tmp = (queue.dequeue.dg)();
Foreach(d,tmp) queue.enqueue(t);
}
});
}
WaitForThreads();
}

In article <e4vu80$30l8$1@digitaldaemon.com>, Lars Ivar Igesund says...
>
>Charles D Hixson wrote:
>
>> I realize that they aren't exactly identical, but I have a project in mind that should, logically, be done via forked processes...thousands of forked processes.  This is obviously impractical.  I could do two different redesigns:  one, based around threads, would still involve the creation of immense numbers of threads.  The other, based around procedure calls, would be theoretically a much poorer model.  (Well, a neural net *IS* a bunch of processes that execute relatively independently...)
>> 
>> Typically people decompose a neural net into collections of arrays, but I would prefer to model each "cell" as an object.  This is obviously a poor match of the design of the algorithm to the structure of the processor, but I suspect that there will be details revealed in the operation that will only be detected if each cell is an object.  So that's what I need.
>> 
>> The question is, just how inefficient would it be to model each "firing"
>> of a neuron as the spawning of a thread.  If I can do it this way then
>> the "cells" can operate pseudo-simultaneously.  If I can't, I'll need to
>>  add a separate buffer and a bunch of code to emulate synchronous
>> operation.  Messy.  Necessary?
>
>When you get into thousands, I think you should avoid at least normal threads. Light weight stack threads / fibers / co-routines might be a better solution, especially if you'd like cooperative switching (very nice for simulations). There has been floating a few suggestions around lately, search the news groups for the terms above. I don't think Mikola has released his updated version yet, though. I think it looked very good.
>
>-- 
>Lars Ivar Igesund
>blog at http://larsivi.net
>DSource & #D: larsivi

BCS wrote:
> You might try breaking the action down into something like this:
> It would let you control the number of threads (SMP?) and simulate as many as
> you want.
> 
> struct dgbox{dgbox[] delegate dg();}
> 
> 
> ThreadSafeQueue!(dgbox) queue;
> 
> 
> void main()
> {
> queue.enque(&Root.StartPoint)
> 
> for(int i = 1; i <= numThreads; i++)
> {
> LaunchThread( function void ()
> {
> dgbox[] tmp;
> 
> while(!queue.empty)
> {
> tmp = (queue.dequeue.dg)();
> Foreach(d,tmp) queue.enqueue(t);
> }
> });
> }
> WaitForThreads();
> }
> 
> In article <e4vu80$30l8$1@digitaldaemon.com>, Lars Ivar Igesund says...
>> Charles D Hixson wrote:
>>
>>> I realize that they aren't exactly identical, but I have a project in mind that should, logically, be done via forked processes...thousands of forked processes.  This is obviously impractical.  I could do two different redesigns:  one, based around threads, would still involve the creation of immense numbers of threads.  The other, based around procedure calls, would be theoretically a much poorer model.  (Well, a neural net *IS* a bunch of processes that execute relatively independently...)
>>>
>>> Typically people decompose a neural net into collections of arrays, but I would prefer to model each "cell" as an object.  This is obviously a poor match of the design of the algorithm to the structure of the processor, but I suspect that there will be details revealed in the operation that will only be detected if each cell is an object.  So that's what I need.
>>>
>>> The question is, just how inefficient would it be to model each "firing"
>>> of a neuron as the spawning of a thread.  If I can do it this way then
>>> the "cells" can operate pseudo-simultaneously.  If I can't, I'll need to
>>>  add a separate buffer and a bunch of code to emulate synchronous
>>> operation.  Messy.  Necessary?
>> When you get into thousands, I think you should avoid at least normal threads. Light weight stack threads / fibers / co-routines might be a better solution, especially if you'd like cooperative switching (very nice for simulations). There has been floating a few suggestions around lately, search the news groups for the terms above. I don't think Mikola has released his updated version yet, though. I think it looked very good.
>>
>> -- 
>> Lars Ivar Igesund
>> blog at http://larsivi.net
>> DSource & #D: larsivi
> 
> 
No, I don't think so.
Given the apparent limitations of threads (and of "protothreads") I
think I'll use build a system based around procedure calls with a
"clock" to synchronize the calls.  Call the procedures in "random"
order, and store the resulting state until the next call.  As I said, messy.

Given what I'm doing it doesn't look like I'll be able to use any of the advanced features of the language except variable length arrays (mandatory!).  Efficiency of implementation of the basics will be even more important than speed, and I'll need to carry along enough state so that cells are "well defined" individuals with unique histories.  (I'm not yet sure what that means at an implementation level...but I know it's important.)