string receiveAll(T)(T socket, int segment_size = 1024)
{
	char[segment_size][] data;
	int cnt = 0;

	while(true)
	{
		auto received = socket.receive(data[cnt]);
		if (received < segment_size)
			break;  /* early exit */
		else if (!received)
			break;
                ++cnt;
	}

	return data;
}


This is my theoretical function, it errors at `char[segment_size][] data;` with the painful `app.d(20): Error: variable segment_size cannot be read at compile time` and I recall having an issue similar to this earlier (yesterday) but I don't think any of my solutions seemed valid for this situation.

I understand it's to do with CTFE or some form of compile-time checking but that's really the only thing that annoys me about D, perhaps somebody could link to some resource that explains (not shortly) how to make the D compiler evaluate some things at run-time opposed to compile time.

Perhaps somebody can link some resources e.g. socket servers in D so I can learn how it's implemented by somebody with a bit more experience, or some resources on how to use sockets properly ¯\_(ツ)_/¯

On 12/12/2017 12:10 PM, Unazed Spectaculum wrote:
> string receiveAll(T)(T socket, int segment_size = 1024)
> {
>      char[segment_size][] data;

Unrelated, you most likely want to use ubyte. (char is for UTF-8.)

The problem is, char[segment_size] is a static array, where the length must be known at compile time because length is a part of its type.

So, depending on what you need you have two options:

a) Use dynamic array if the length is known at run time

b) Although (a) will work just fine, use template parameter for length if the length is known at compile time and you want to avoid dynamic allocation.

However, too large arrays won't fit on the stack. (Further however, your 'data' is a slice anyway, just the elements are static.)

The following program shows the two options with one-dimensional arrays:

// Size is known at run time
void foo(T)(T t, size_t size = 1024) {
    auto data = new ubyte[](size);
}

// Size is known at compile time
void bar(size_t size = 1024, T)(T t) {
    ubyte[size] data;
}

void main() {
    int i;
    foo(i, 10);
    bar!20(i);
}

Here is one with two-dimensional arrays:

import std.stdio;

size_t counter = 0;
bool done() {
    return (++counter % 4) == 0;
}

// Size is known at run time
void foo(T)(T t, size_t size = 1024) {
    ubyte[][] data;

    while (!done) {
        data ~= new ubyte[size];
        // Use data[$-1]
        writeln("a) Will read here: ", data[$-1]);
    }
}

// Size is known at compile time
void bar(size_t size = 1024, T)(T t) {
    ubyte[size][] data;

    while (!done) {
        ++data.length;
        writeln("b) Will read here: ", data[$-1]);
    }
}

void main() {
    int i;
    foo(i, 10);
    bar!20(i);
}

Ali

On Tuesday, 12 December 2017 at 20:27:04 UTC, Ali Çehreli wrote:
> On 12/12/2017 12:10 PM, Unazed Spectaculum wrote:
> > string receiveAll(T)(T socket, int segment_size = 1024)
> > {
> >      char[segment_size][] data;
>
> Unrelated, you most likely want to use ubyte. (char is for UTF-8.)
>
> The problem is, char[segment_size] is a static array, where the length must be known at compile time because length is a part of its type.
>
> So, depending on what you need you have two options:
>
> a) Use dynamic array if the length is known at run time
>
> b) Although (a) will work just fine, use template parameter for length if the length is known at compile time and you want to avoid dynamic allocation.
>
> However, too large arrays won't fit on the stack. (Further however, your 'data' is a slice anyway, just the elements are static.)
>
> The following program shows the two options with one-dimensional arrays:
>
> // Size is known at run time
> void foo(T)(T t, size_t size = 1024) {
>     auto data = new ubyte[](size);
> }
>
> // Size is known at compile time
> void bar(size_t size = 1024, T)(T t) {
>     ubyte[size] data;
> }
>
> void main() {
>     int i;
>     foo(i, 10);
>     bar!20(i);
> }
>
> Here is one with two-dimensional arrays:
>
> import std.stdio;
>
> size_t counter = 0;
> bool done() {
>     return (++counter % 4) == 0;
> }
>
> // Size is known at run time
> void foo(T)(T t, size_t size = 1024) {
>     ubyte[][] data;
>
>     while (!done) {
>         data ~= new ubyte[size];
>         // Use data[$-1]
>         writeln("a) Will read here: ", data[$-1]);
>     }
> }
>
> // Size is known at compile time
> void bar(size_t size = 1024, T)(T t) {
>     ubyte[size][] data;
>
>     while (!done) {
>         ++data.length;
>         writeln("b) Will read here: ", data[$-1]);
>     }
> }
>
> void main() {
>     int i;
>     foo(i, 10);
>     bar!20(i);
> }
>
> Ali

string receiveAll(T)(T socket, size_t segment_size = 1024)
{
	ubyte[][] data;
	size_t count = 0;

	while (true)
	{
		data ~= new ubyte[segment_size];

		auto received = socket.receive(data[count]);
		data[count] = data[count][0 .. received];

		if (!received)
			break;
		else if (received < segment_size)
			break;  /* early exit */

		++count;
	}

	char[] stringData;

	foreach (elem; data)
		stringData ~= elem;

	return to!string(stringData);
}


I've decided to go for the run-time approach to this, it works fine with all of my tests so you have my greatest gratitude.

I might have created some weird inefficiencies but don't worry take time telling me about them unless they're going to blow up my program since I think you've explained enough already :D.

Since I'm only a few days into D I wouldn't expect much of my code, I'm moreover from the generic Python and thereabouts C-ish background.

again, thanks.

On Tuesday, 12 December 2017 at 21:03:54 UTC, Unazed Spectaculum wrote:
> I've decided to go for the run-time approach to this, it works fine with all of my tests so you have my greatest gratitude.

the way I'd do it btw is simply:

ubyte[] result;
ubyte[1024] buffer;
auto got = socket.receive(buffer[]);
while(got > 0) {
   result ~= buffer[0 .. got];
}

if(got < 0)
  throw new Exception(lastSocketError());

return result;


so it uses the one static buffer to receive the stuff one block at a time but just copies it over to the dynamic array with the ~= operator

On Tuesday, 12 December 2017 at 21:14:30 UTC, Adam D. Ruppe wrote:
> On Tuesday, 12 December 2017 at 21:03:54 UTC, Unazed Spectaculum wrote:
>> I've decided to go for the run-time approach to this, it works fine with all of my tests so you have my greatest gratitude.
>
> the way I'd do it btw is simply:
>
> ubyte[] result;
> ubyte[1024] buffer;
> auto got = socket.receive(buffer[]);
> while(got > 0) {
>    result ~= buffer[0 .. got];
> }
>
> if(got < 0)
>   throw new Exception(lastSocketError());
>
> return result;
>
>
> so it uses the one static buffer to receive the stuff one block at a time but just copies it over to the dynamic array with the ~= operator

Oh, thanks; it's definitely shorter however I do enjoy the ability to generalize/ambiguate functions by providing optional parameters, however thanks for showing another way; I enjoy knowing multiple ways of performing one task so anything helps :)

On 12/12/2017 01:14 PM, Adam D. Ruppe wrote:
> On Tuesday, 12 December 2017 at 21:03:54 UTC, Unazed Spectaculum wrote:
>> I've decided to go for the run-time approach to this, it works fine
>> with all of my tests so you have my greatest gratitude.
>
> the way I'd do it btw is simply:

I know you normally do it much better. :)

> ubyte[] result;
> ubyte[1024] buffer;
> auto got = socket.receive(buffer[]);
> while(got > 0) {
>     result ~= buffer[0 .. got];
> }

There are a couple of bugs in that code. The same buffer is used for all segments and socket.receive should be inside while.

Ali

On 12/12/2017 01:52 PM, Ali Çehreli wrote:

>  > ubyte[] result;
>  > ubyte[1024] buffer;

>  >     result ~= buffer[0 .. got];

> The same buffer is used for all segments

Not! Ok, I better go take a nap. :)

Ali

On Tuesday, 12 December 2017 at 21:52:57 UTC, Ali Çehreli wrote:
> The same buffer is used for all segments and socket.receive should be inside while.

The buffer is copied by the ~= operator, but indeed you're right that I forgot to receive again inside the loop! That would just spin until it ran out of memory lol.

But I did put receive outside the loop for a reason: it just needs to prime the return value before it gets checked the first time. But then, of course, it should receive again at the end of the loop to advance to the next chunk.

On Tuesday, 12 December 2017 at 22:11:37 UTC, Adam D. Ruppe wrote:
> On Tuesday, 12 December 2017 at 21:52:57 UTC, Ali Çehreli wrote:
>> The same buffer is used for all segments and socket.receive should be inside while.
>
> The buffer is copied by the ~= operator, but indeed you're right that I forgot to receive again inside the loop! That would just spin until it ran out of memory lol.
>
> But I did put receive outside the loop for a reason: it just needs to prime the return value before it gets checked the first time. But then, of course, it should receive again at the end of the loop to advance to the next chunk.

do while would work better.

On 12/12/2017 10:21 PM, bauss wrote:
> On Tuesday, 12 December 2017 at 22:11:37 UTC, Adam D. Ruppe wrote:
>> On Tuesday, 12 December 2017 at 21:52:57 UTC, Ali Çehreli wrote:
>>> The same buffer is used for all segments and socket.receive should be
>>> inside while.
>>
>> The buffer is copied by the ~= operator, but indeed you're right that
>> I forgot to receive again inside the loop! That would just spin until
>> it ran out of memory lol.
>>
>> But I did put receive outside the loop for a reason: it just needs to
>> prime the return value before it gets checked the first time. But
>> then, of course, it should receive again at the end of the loop to
>> advance to the next chunk.
>
> do while would work better.

It's a bummer that the loop variable must be defined outside of do-while:

    bool done = false;    // Can't be inside
    do {
        // ...
    } while(!done);

That's why I think an unconditional loop with an explicit break is better in such situations:

    while (true) {
        // ... everything inside ...
        if (!got) {
            break;
        }
        // ...
    }

Here is a short program that I played with:

import std.stdio;
import std.range;
import std.algorithm;

int[] makeData(int len) {
    return iota(len).array;
}

struct SomeSource {
    int[] data;

    this(int len) {
        this.data = makeData(len);
    }

    // Alternatively, this can return int[]
    size_t read(int[] buffer) {
        const len = min(data.length, buffer.length);
        buffer[0..len] = data[0..len];
        data = data[len..$];    // Alternatively, popFrontN
        return len;
    }
}

void main() {
    enum len = 50;
    auto foo = SomeSource(len);

    int[] result;
    while (true) {    // <-- UNCONDITIONAL LOOP
        int[17] buffer;
        const got = foo.read(buffer[]);
        if (!got) {
            break;
        }
        result ~= buffer[0..got];
    }
    assert(result == makeData(len));
    writeln(result);
}

Ali