Thread overview
How do I make this function thread safe?
May 31, 2018
Dr.No
Jun 01, 2018
Paul Backus
Jun 01, 2018
Dr.No
May 31, 2018
My application create some HTML which is then converted to PDF by wkhtmltopdf library. I'm trying to figure out how make the PDF generation run parallel, currently, it's running linearly. My guess is wkhtmltopdf internal variables is preventing parallelization. But I'm new to parallization and I don't know how to solve that by now. I guesses that that function from an external thread would make each wkhtmltopdf initilization run on its memory space, similiar to a process. But since it is not working, I guess this isn't how it's working.
I'm not asking to just give me the code ready (if it's somehow complex) brather some directions, how I may archive that.


Here's my current code:

void genPDFImplt(string htmlFilename, string outputpdfFilename)
{
	import pdf;
	import std.string : toStringz;

	/* Init wkhtmltopdf in graphics less mode */
	wkhtmltopdf_init(0);
	wkhtmltopdf_global_settings *gs = wkhtmltopdf_create_global_settings();
	/* We want the result to be storred in the file called test.pdf */
	wkhtmltopdf_set_global_setting(gs, "out", outputpdfFilename.toStringz);
	wkhtmltopdf_object_settings *os = wkhtmltopdf_create_object_settings();
	/* We want to convert to convert the qstring documentation page */
	wkhtmltopdf_set_object_setting(os, "page", htmlFilename.toStringz);
	/* Create the actual converter object used to convert the pages */
	wkhtmltopdf_converter * c = wkhtmltopdf_create_converter(gs);
	static if(0) {
		/* Call the progress_changed function when progress changes */
		wkhtmltopdf_set_progress_changed_callback(c, &pdf_progress_changed);
		/* Call the phase _changed function when the phase changes */
		wkhtmltopdf_set_phase_changed_callback(c, &pdf_phase_changed);
		/* Call the error function when an error occures */
		wkhtmltopdf_set_error_callback(c, &pdf_error);
		/* Call the waring function when a warning is issued */
		wkhtmltopdf_set_warning_callback(c, &pdf_warning);
	}
	scope(exit) {
		/* Destroy the converter object since we are done with it */
		wkhtmltopdf_destroy_converter(c);
		/* We will no longer be needing wkhtmltopdf funcionality */
		wkhtmltopdf_deinit();
	}
	wkhtmltopdf_add_object(c, os, null);
	/* Perform the actual convertion */
	wkhtmltopdf_convert(c);
}


called from a loop like this:

foreach(string file; parallel(files)) {
   auto res = doSomething(file);
   auto htmlfile = genHtmlFile(res);
   auto pdffile = genTmpPDFFilename();
   genPDFImplt(htmlfiel, pdffile);
}



Running in a dual core CPU, with 4 threads. It's genrating
one PDF per iteration rather 4. I've made sure already it's the genPDFImplt() which is doing that.
	
June 01, 2018
On Thursday, 31 May 2018 at 19:26:12 UTC, Dr.No wrote:
> My application create some HTML which is then converted to PDF by wkhtmltopdf library. I'm trying to figure out how make the PDF generation run parallel, currently, it's running linearly.

It looks like wkhtmltopdf does not support multithreaded use; see here:

https://github.com/wkhtmltopdf/wkhtmltopdf/issues/1711

So, if you want to run the conversions in parallel, you will have to use separate processes.
June 01, 2018
On Friday, 1 June 2018 at 02:30:34 UTC, Paul Backus wrote:
> On Thursday, 31 May 2018 at 19:26:12 UTC, Dr.No wrote:
>> My application create some HTML which is then converted to PDF by wkhtmltopdf library. I'm trying to figure out how make the PDF generation run parallel, currently, it's running linearly.
>
> It looks like wkhtmltopdf does not support multithreaded use; see here:
>
> https://github.com/wkhtmltopdf/wkhtmltopdf/issues/1711
>
> So, if you want to run the conversions in parallel, you will have to use separate processes.

Can I run that function in another memory space within my application, making even __gshared be allocated in that memory space, so that it seems it is another process? it's quite similar to thread but everything is allocated in that new memory space including __gshared.
I started calling the wkhtmltopdf executable by I wanted to see if there's a signficative gain in performance calling the library directly. My application do generate a lot of PDFs so I'd like to optimiza as possible.