I'm trying to receive data from curl request my sample code looks like that:

...
            auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString);
            client.method = HTTP.Method.get;
            client.addRequestHeader("x-amz-date", xamztime);
            client.addRequestHeader("Authorization", authorizationHeader);
            client.onReceive = (ubyte[] data)
            {
                recievedData = data;
                return data.length;
            };
            client.perform();

            return new Result(recievedData);
...

            ubyte[] receivedData;

...

but im getting only last (like from "tail" command in unix systems) part of data which im expecting.

How to receive and save whole data which came as feedback for request? Or is there some other way to access it after usage of client.perform method?

On Sunday, 18 October 2015 at 18:04:53 UTC, holo wrote:
> I'm trying to receive data from curl request my sample code looks like that:
>
> ...
>             auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString);
>             client.method = HTTP.Method.get;
>             client.addRequestHeader("x-amz-date", xamztime);
>             client.addRequestHeader("Authorization", authorizationHeader);
>             client.onReceive = (ubyte[] data)
>             {
>                 recievedData = data;
>                 return data.length;
>             };
>             client.perform();
>
>             return new Result(recievedData);
> ...
>
>             ubyte[] receivedData;
>
> ...
>
> but im getting only last (like from "tail" command in unix systems) part of data which im expecting.
>
> How to receive and save whole data which came as feedback for request? Or is there some other way to access it after usage of client.perform method?

I believe `onReceive` called multiple times with chunks of received data. So, you need to use `Appender` or `~`.

A bit off-topic:
Are you trying to download file from S3? It seems I should really start working on my S3 library...

On Sunday, 18 October 2015 at 18:04:53 UTC, holo wrote:
> I'm trying to receive data from curl request my sample code looks like that:
>
> ...
>             auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString);
>             client.method = HTTP.Method.get;
>             client.addRequestHeader("x-amz-date", xamztime);
>             client.addRequestHeader("Authorization", authorizationHeader);
>             client.onReceive = (ubyte[] data)
>             {
>                 recievedData = data;
>                 return data.length;
>             };
>             client.perform();
>
>             return new Result(recievedData);
> ...
>
>             ubyte[] receivedData;
>
> ...
>
> but im getting only last (like from "tail" command in unix systems) part of data which im expecting.
>
> How to receive and save whole data which came as feedback for request? Or is there some other way to access it after usage of client.perform method?

You also may try to look at http://vibed.org/api/vibe.http.client/requestHTTP
curl maybe a little bit hard in use.

@sigod

Actually im working on ec2 requests. Thank you  for help, it is working right now. I don't know why i was trying "+=" before instead of "~=". Is it good solution to make it such way?

@Suliaman

I need to collect information about my instances and put it to DB. I want to present those datas with with vibe.d, but collector want to make without too many dependencies (i was thinking to make it as separate thread but simplest solution will be just to use cron to run collector i think)

On Sunday, 18 October 2015 at 20:05:24 UTC, holo wrote:
> @sigod
>
> Actually im working on ec2 requests. Thank you  for help, it is working right now. I don't know why i was trying "+=" before instead of "~=". Is it good solution to make it such way?

Not really as it will trigger allocation on every call. Better use [`Appender`][0].

[0]: http://dlang.org/phobos/std_array.html#.Appender

On Sunday, 18 October 2015 at 20:12:42 UTC, sigod wrote:
> On Sunday, 18 October 2015 at 20:05:24 UTC, holo wrote:
>> @sigod
>>
>> Actually im working on ec2 requests. Thank you  for help, it is working right now. I don't know why i was trying "+=" before instead of "~=". Is it good solution to make it such way?
>
> Not really as it will trigger allocation on every call. Better use [`Appender`][0].
>
> [0]: http://dlang.org/phobos/std_array.html#.Appender

I changed it to such code:

...
            auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString);
            client.method = HTTP.Method.get;
            client.addRequestHeader("x-amz-date", xamztime);
            client.addRequestHeader("Authorization", authorizationHeader);
            client.onReceive = (ubyte[] data){receivedData.put(data); return data.length;};
            client.perform();

            return new Result(receivedData.data);
...
     auto receivedData = appender!string();
...

and it is really much more faster than it was - hope it is what you had on your mind.

On Sunday, 18 October 2015 at 21:01:05 UTC, holo wrote:
> On Sunday, 18 October 2015 at 20:12:42 UTC, sigod wrote:
>> [...]
>
> I changed it to such code:
>
> ...
>             auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString);
>             client.method = HTTP.Method.get;
>             client.addRequestHeader("x-amz-date", xamztime);
>             client.addRequestHeader("Authorization", authorizationHeader);
>             client.onReceive = (ubyte[] data){receivedData.put(data); return data.length;};
>             client.perform();
>
>             return new Result(receivedData.data);
> ...
>      auto receivedData = appender!string();
> ...
>
> and it is really much more faster than it was - hope it is what you had on your mind.

Yes, this is exactly what I meant.

On Sunday, 18 October 2015 at 21:11:58 UTC, sigod wrote:
> On Sunday, 18 October 2015 at 21:01:05 UTC, holo wrote:
>> On Sunday, 18 October 2015 at 20:12:42 UTC, sigod wrote:
>>> [...]
>>
>> I changed it to such code:
>>
>> ...
>>             auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString);
>>             client.method = HTTP.Method.get;
>>             client.addRequestHeader("x-amz-date", xamztime);
>>             client.addRequestHeader("Authorization", authorizationHeader);
>>             client.onReceive = (ubyte[] data){receivedData.put(data); return data.length;};
>>             client.perform();
>>
>>             return new Result(receivedData.data);
>> ...
>>      auto receivedData = appender!string();
>> ...
>>
>> and it is really much more faster than it was - hope it is what you had on your mind.
>
> Yes, this is exactly what I meant.

Thank you for help.