It's look that GC in D is really suxx. There is already second toy-project where I am getting stuck on Windows with D for last 3 month.

I'm using 32-bit build, because I can't understand which libs I should use to get OpenSSL 64 bit work with dlang-request.

32-bit version compile and works fine, but it's fail during downloading 300MB file with next error:
core.exception.OutOfMemoryError@src\core\exception.d(696): Memory allocation failed

code:

import std.stdio;
import std.conv;
import std.file;
import std.path;
import std.string;
import std.algorithm;
import std.regex;

import requests;
pragma(lib, "ssl");
pragma(lib, "eay");


void main()
{
    string login = "Suliman1";
    string pass = "Infinity8"; //*
    string url = "https://n5eil01u.ecs.nsidc.org/SMAP/SPL3SMP_E.001/2017.07.11/";
	
	Request request;
	request.addHeaders(["User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0"]);
	request.authenticator = new BasicAuthentication(login, pass);
	auto rs = request.get(url);
	string content = to!string(rs.responseBody);

	string fname = content.matchFirst(`(SMAP_[^"]+)`).hit;

	string h5_file = url ~ fname;
	string isoXML_file = url ~ fname ~ `.iso.xml`;
	string qa_file = url ~ fname.replace(`h5`, `qa`);

	string [] arrayFullURLs;
	arrayFullURLs ~= h5_file;
	arrayFullURLs ~= isoXML_file;
	arrayFullURLs ~= qa_file;

	foreach(f; arrayFullURLs)
	{
		writeln("Loading file: ", f);
		auto rs1 = request.get(f);
		File file = File(baseName(f), "w");
		file.write(rs1.responseBody);
	}
	
}

dub.sdl:
name "parser"
dependency "requests" version="~>0.4.2"
dependency "progress" version="~>4.0.3"


--
* please do now hack account because I publish this pass to get it's reproduce error easier.

So could anybody explain step-by-step how to get OpenSSL 64 work on 64-bit Windows.

On my PC this app eat 500MB of RAM and then failure with error above.

On Friday, 14 July 2017 at 13:16:17 UTC, Suliman wrote:
> It's look that GC in D is really suxx. There is already second toy-project where I am getting stuck on Windows with D for last 3 month.
>
> I'm using 32-bit build, because I can't understand which libs I should use to get OpenSSL 64 bit work with dlang-request.
>
> 32-bit version compile and works fine, but it's fail during downloading 300MB file with next error:
> core.exception.OutOfMemoryError@src\core\exception.d(696): Memory allocation failed

You might wanna read the file in chunks and write in chunks.

On Friday, 14 July 2017 at 14:50:04 UTC, bauss wrote:
> On Friday, 14 July 2017 at 13:16:17 UTC, Suliman wrote:
>> It's look that GC in D is really suxx. There is already second toy-project where I am getting stuck on Windows with D for last 3 month.
>>
>> I'm using 32-bit build, because I can't understand which libs I should use to get OpenSSL 64 bit work with dlang-request.
>>
>> 32-bit version compile and works fine, but it's fail during downloading 300MB file with next error:
>> core.exception.OutOfMemoryError@src\core\exception.d(696): Memory allocation failed
>
> You might wanna read the file in chunks and write in chunks.

It's helped, but it's obviously that D on Windows should have 64-bit linker out of the box. Because GC on 32-bit machine show a lot of bugs.

On Friday, 14 July 2017 at 17:24:05 UTC, Suliman wrote:
> GC on 32-bit machine show a lot of bugs.

such as..?

On 07/15/2017 03:05 PM, tetyys wrote:
> On Friday, 14 July 2017 at 17:24:05 UTC, Suliman wrote:
>> GC on 32-bit machine show a lot of bugs.
>
> such as..?

D's GC is conservative i.e. it cannot assume an integer is not a pointer. There are ways around this such as marking the memory block as not containing pointers but in general, if the language allows you to cast a pointer to integer, it's possible that any integer may be the only reference to a GC memory.

The problem with 32-bit is, it's very likely that an integer value may happen to have a value that equals to an address into GC memory. This happens when you allocate e.g. half gigabyte memory, read a file into it. So far so good. Unfortunately, it's likely that there is some integer in the program that points into this half a gig memory and the GC will never release it.

Ali