so I'im doing an expansive operation with a file, to try speed up, i switch to using parallel but keeping in the otuput printing thread-safe. But for some reason, even using synchonized, it doesn't work as expected, for example, it output multiples results on same time, as in the example below.

the code:

>	stdout.flush();
>	foreach(string fn; parallel(files))
>	{
>		auto res = doSomething(fn);
>		synchronized
>		{
>		   stdout.writefln("outjson = %s", res.serializeToJson);
>		   stdout.flush();
>		}
>
>     }


the expeced output is like that (one per line):

> outjson = {"barCode":"XXXX1","ade":"1"}
> outjson = {"barCode":"XXXX2","ade":"2"}
> outjson = {"barCode":"XXXX3","ade":"3"}
// and so on...

But it in the middle of output, I got output like this:

> outjson = {"barCode":"XXXX20","ade":"20"}♪◙outjson = {"barCode":"XXXXX21","ade":"21"}

also there's that extra ♪◙ character. Thos sounds memory violation somewhere.
This only happens when using parallel. Any guess what's possibily happeing?

Does anyone have some tips to try trace the error with debug or so?
I haven't fixed this issue yet... any help is very appreciated

On 09/03/2018 08:13 PM, Dr.No wrote:
> But it in the middle of output, I got output like this:
> 
>> outjson = {"barCode":"XXXX20","ade":"20"}♪◙outjson = {"barCode":"XXXXX21","ade":"21"}
> 
> also there's that extra ♪◙ character. Thos sounds memory violation somewhere.
> This only happens when using parallel. Any guess what's possibily happeing?

If that only ever happens at line breaks, then that doesn't necessarily look like memory corruption to me.

I looked around a bit and found code page 437 [1]. It has those characters at 0xD and 0xA which is where ASCII has \r and \n. So it might be that code page 437 is used when displaying that particular entry. Why that would happen, I have no idea.

To get better help, you should post a complete test case that people can just copy/paste. That includes imports, a `main` function, and the command you use to compile. Also add information about your environment (OS, compiler version).


[1] https://en.wikipedia.org/wiki/Code_page_437

On Saturday, 8 September 2018 at 14:26:45 UTC, ag0aep6g wrote:
> On 09/03/2018 08:13 PM, Dr.No wrote:
>> But it in the middle of output, I got output like this:
>> 
>>> outjson = {"barCode":"XXXX20","ade":"20"}♪◙outjson = {"barCode":"XXXXX21","ade":"21"}
>> 
>> also there's that extra ♪◙ character. Thos sounds memory violation somewhere.
>> This only happens when using parallel. Any guess what's possibily happeing?
>
> If that only ever happens at line breaks, then that doesn't necessarily look like memory corruption to me.

Yes, it does only happens at line breaks. I hadn't realized that until you mentioned. It does gets in place of \r and \n, that's why there's all in one line when this happens. Thankfor for the notice.

> I looked around a bit and found code page 437 [1]. It has those characters at 0xD and 0xA which is where ASCII has \r and \n. So it might be that code page 437 is used when displaying that particular entry. Why that would happen, I have no idea.

So I guessed that something changed the console code page. I've tried to call call chcp 65001 at program's start up. Not worked. Tried inside the main loop. Also not worked.

> To get better help, you should post a complete test case that people can just copy/paste. That includes imports, a `main` function, and the command you use to compile. Also add information about your environment (OS, compiler version).
>

I made a reduced version where you can do just dmd -run hello.d. The application does convert alot of PDF files to text then do some parsing (I've removed this not needed part). In my code, at iteration 28 I got those character (♪◙), which is the error. I hope you can reproduce this error on your machine as well. I got some PDF files and duplicated them just for testing purpose. If for some reason you didn't get the same error immediately, try duplicate the PDFs and try again. I believe this can reproduce the erro.


Here's the resources to compile the application:

source code: https://pastebin.com/RwrUikQS
PDF files + pdf to text application: https://drive.google.com/file/d/1uKjJX4pQIEWVK4vujUsm0ln2yHS7z5ZZ/view?usp=sharing
OS: Windows 10 64-bit
compiler: DMD32 D Compiler v2.080.0 (same issue happens on ldc)
command line: dmd -run hello.d

Thank you very much for your time.

On Monday, 10 September 2018 at 20:30:52 UTC, Dr.No wrote:
> On Saturday, 8 September 2018 at 14:26:45 UTC, ag0aep6g wrote:
>> [...]
>
> Yes, it does only happens at line breaks. I hadn't realized that until you mentioned. It does gets in place of \r and \n, that's why there's all in one line when this happens. Thankfor for the notice.
>
> [...]

I noticied this happens with everything I print in the loop, even using C's printf:


>	__gshared Mutex m;
>	m = new Mutex();
>	foreach(string fn; parallel(files))
>	{
>		try
>		{
>			auto res = f(fn);
>			synchronized(m)
>			{
>				import std.c.stdio : 				printf;
>				printf("hello!\n");
>			}
>
> [...]


Output:

https://imgur.com/a/Mq9X4c3