The whole porting works have being done almost by hand.
It is still too early for real using.
When all things done, I'll make it open-source.
I'm just busy with porting the modules about winapi.
----------
Zhang <bitworld@qq.com>

I just ran into some odd stdout corruption issue while doing parallel builds, take a look:

http://i.imgur.com/pgC4o.png

I've tried building again and it was gone, it looks like an appearance of a heisenbug. There might be some bug lurking somewhere in std.parallelism, but it's hard to say when I can't recreate it.

On 22.06.2011 23:01, Andrej Mitrovic wrote:
> I just ran into some odd stdout corruption issue while doing parallel
> builds, take a look:
>
> http://i.imgur.com/pgC4o.png
>
> I've tried building again and it was gone, it looks like an appearance
> of a heisenbug. There might be some bug lurking somewhere in
> std.parallelism, but it's hard to say when I can't recreate it.

I'd say that is to be expected, e.g. when multiple processes (in this case two resource compilers) output  to the same stream (console) and has nothing to do with std.parallelism. One cure for that is to have them output to a unique temporary files or pipes.
The only thing that's strange is that I'd expect it to happen in blocks as there should be buffering, but maybe they doesn't use it at all.

-- 
Dmitry Olshansky

On 6/22/11, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:
> I'd say that is to be expected, e.g. when multiple processes (in this
> case two resource compilers) output  to the same stream (console) and
> has nothing to do with std.parallelism. One cure for that is to have
> them output to a unique temporary files or pipes.
> The only thing that's strange is that I'd expect it to happen in blocks
> as there should be buffering, but maybe they doesn't use it at all.

But stdout is marked as shared. I would think that would solve the issue of outputting from multiple threads?

On 6/22/2011 12:01 PM, Andrej Mitrovic wrote:
> I just ran into some odd stdout corruption issue while doing parallel
> builds, take a look:

stdout is shared, but the atomic units of output are where the calls to it are, and are interleaved otherwise. For example, a single call to printf locks stdout for the duration of the printf.

Andrej Mitrovic wrote:
>On 6/22/11, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:
>> I'd say that is to be expected, e.g. when multiple processes (in this
>> case two resource compilers) output  to the same stream (console) and
>> has nothing to do with std.parallelism. One cure for that is to have
>> them output to a unique temporary files or pipes.
>> The only thing that's strange is that I'd expect it to happen in
>> blocks as there should be buffering, but maybe they doesn't use it
>> at all.
>
>But stdout is marked as shared. I would think that would solve the issue of outputting from multiple threads?

That's only true for D functions based on D stdout (D writeln, etc.). It doesn't apply to C printf, or different processes using the same console handle. I guess you use the std.process system function which afaik doesn't redirect the spawned processes output.

-- 
Johannes Pfau

Walter Bright wrote:
>On 6/22/2011 12:01 PM, Andrej Mitrovic wrote:
>> I just ran into some odd stdout corruption issue while doing parallel builds, take a look:
>
>stdout is shared, but the atomic units of output are where the calls to it are, and are interleaved otherwise. For example, a single call to printf locks stdout for the duration of the printf.

C printf is atomic? Didn't know that.
What happens when multiple processes are spawned with system()? Is
their IO output still synchronized?

-- 
Johannes Pfau

On 6/22/2011 1:49 PM, Johannes Pfau wrote:
> Walter Bright wrote:
>> On 6/22/2011 12:01 PM, Andrej Mitrovic wrote:
>>> I just ran into some odd stdout corruption issue while doing parallel
>>> builds, take a look:
>>
>> stdout is shared, but the atomic units of output are where the calls
>> to it are, and are interleaved otherwise. For example, a single call
>> to printf locks stdout for the duration of the printf.
>
> C printf is atomic? Didn't know that.
> What happens when multiple processes are spawned with system()? Is
> their IO output still synchronized?

I'm sure that Windows does discrete write's atomically, but it's up to the individual apps to bundle things together into discrete writes.

On 6/22/11 3:49 PM, Johannes Pfau wrote:
> Walter Bright wrote:
>> On 6/22/2011 12:01 PM, Andrej Mitrovic wrote:
>>> I just ran into some odd stdout corruption issue while doing parallel
>>> builds, take a look:
>>
>> stdout is shared, but the atomic units of output are where the calls
>> to it are, and are interleaved otherwise. For example, a single call
>> to printf locks stdout for the duration of the printf.
>
> C printf is atomic? Didn't know that.
> What happens when multiple processes are spawned with system()? Is
> their IO output still synchronized?

No, printf is atomic across threads. Spilling buffers to disk is a distinct issue.

Andrei

On 6/22/11, Johannes Pfau <spam@example.com> wrote:
> Andrej Mitrovic wrote:
>>On 6/22/11, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:
>>> I'd say that is to be expected, e.g. when multiple processes (in this
>>> case two resource compilers) output  to the same stream (console) and
>>> has nothing to do with std.parallelism. One cure for that is to have
>>> them output to a unique temporary files or pipes.
>>> The only thing that's strange is that I'd expect it to happen in
>>> blocks as there should be buffering, but maybe they doesn't use it
>>> at all.
>>
>>But stdout is marked as shared. I would think that would solve the issue of outputting from multiple threads?
>
> That's only true for D functions based on D stdout (D writeln, etc.). It doesn't apply to C printf, or different processes using the same console handle. I guess you use the std.process system function which afaik doesn't redirect the spawned processes output.
>
> --
> Johannes Pfau
>
>

Ah you're right, I'm using system and writeln() calls, so that might
be causing it.