Hi All,

  We recently tested the below components and the test results are as below, even though hunt-database is faster than mysql-native it is hard to use this package as it lacks on documentation, non of the example provided in the documentation works, one has to go through the code and find how things works to use this package. We have raised this issue in the github and it been 3 day's still no update and the last documentation (README) was updated 14 month ago, but the latest release of this package was released on 2020-Oct-11, version 2.1.9. May I know whether is there is any plan for updating the documentation if so may know when would the documentation be updated, so that we can explore other features of hunt packages (hunt-framework vs vib.d framework).

Component : mysql-native + asdf
Executable size : 17 MB
Execution time : 10 secs, 189 ms, 919 μs, and 3 hnsecs

Component : hunt-database + asdf
Executable size : 81 MB
Execution time : 5 secs, 916 ms, 418 μs, and 3 hnsecs


From,
Vino.B

On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:

> Component : mysql-native + asdf
> Executable size : 17 MB
> Execution time : 10 secs, 189 ms, 919 μs, and 3 hnsecs
>
> Component : hunt-database + asdf
> Executable size : 81 MB
> Execution time : 5 secs, 916 ms, 418 μs, and 3 hnsecs


Interesting but I think you should provide the test code for representative results?

On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:
> Hi All,
>
>   We recently tested the below components and the test results are as below, even though hunt-database is faster than mysql-native it is hard to use this package as it lacks on documentation, non of the example provided in the documentation works, one has to go through the code and find how things works to use this package. We have raised this issue in the github and it been 3 day's still no update and the last documentation (README) was updated 14 month ago, but the latest release of this package was released on 2020-Oct-11, version 2.1.9. May I know whether is there is any plan for updating the documentation if so may know when would the documentation be updated, so that we can explore other features of hunt packages (hunt-framework vs vib.d framework).
>
> Component : mysql-native + asdf
> Executable size : 17 MB
> Execution time : 10 secs, 189 ms, 919 μs, and 3 hnsecs
>
> Component : hunt-database + asdf
> Executable size : 81 MB
> Execution time : 5 secs, 916 ms, 418 μs, and 3 hnsecs
>
>
> From,
> Vino.B

While doing performance measurements you should do each test multiple time (at least 5 times). There could be for example an effect that executing a db query the first time is a lot slower than afterwards.

Maybe mysql native is slower, but maybe this isn't the case, just the way performance measurements was done was incorrectly.

Kind regards
Andre

On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:
> On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:
>> [...]
>
> While doing performance measurements you should do each test multiple time (at least 5 times). There could be for example an effect that executing a db query the first time is a lot slower than afterwards.
>
> Maybe mysql native is slower, but maybe this isn't the case, just the way performance measurements was done was incorrectly.
>
> Kind regards
> Andre

Hi Andre,

  The benchmark was performed with 100 executions as below

void main()
{
 void f() {
 Array!string[string] data = getdata("TEST");
 foreach(i; data) { writeln(i[]); }
 }
 auto r = benchmark!(f)(100);
 Duration t = r[0];
 writeln(t);
}

On Saturday, 7 November 2020 at 14:16:46 UTC, Vino wrote:
> On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:
>> On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:
>>> [...]
>>
>> While doing performance measurements you should do each test multiple time (at least 5 times). There could be for example an effect that executing a db query the first time is a lot slower than afterwards.
>>
>> Maybe mysql native is slower, but maybe this isn't the case, just the way performance measurements was done was incorrectly.
>>
>> Kind regards
>> Andre
>
> Hi Andre,
>
>   The benchmark was performed with 100 executions as below
>
> void main()
> {
>  void f() {
>  Array!string[string] data = getdata("TEST");
>  foreach(i; data) { writeln(i[]); }
>  }
>  auto r = benchmark!(f)(100);
>  Duration t = r[0];
>  writeln(t);
> }

Hi All,

  Upon further testing, below are the observations, based on different compliers (DMD/LDC), using LDC the size of the executable is reduced but the run time is increased from 5 to 6 sec , the document states that if we use the complier option dub --"build=release --compiler=ldmd2" it would enhance the performance  but as per our analysis we do not see and improvement, below is the runtime

LDC                                      : 6 secs, 136 ms, 97 μs, and 3 hnsecs
LDC(dub --build=release --compiler=ldmd2): 6 secs, 203 ms, 283 μs, and 6 hnsecs


Observations
********************
Component 	:  mysql-native + asdf	
Complier	:  DMD
Size 		:  17MB
Duration	:  10 secs, 189 ms, 919 μs, and 3 hnsecs

Component 	:  mysql-native + asdf	
Complier	:  LDC
Size 		:  20MB
Duration	:  10 secs, 526 ms, 350 μs, and 6 hnsecs

Component 	:  mysql-native + asdf	
Complier	:  LDC(dub --build=release --compiler=ldmd2)
Size 		:  3.7MB
Duration	:  10 secs, 411 ms, 793 μs, and 8 hnsecs
*************************************************************
Component 	:  hunt-database + asdf	
Complier	:  DMD
Size 		:  81MB
Duration	:  5 secs, 916 ms, 418 μs, and 3 hnsecs

Component 	:  hunt-database + asdf	
Complier	:  LDC
Size 		:  50MB
Duration	:  6 secs, 136 ms, 97 μs, and 3 hnsecs

Component 	:  hunt-database + asdf	
Complier	:  LDC(dub --build=release --compiler=ldmd2)
Size 		:  17MB
Duration	:  6 secs, 136 ms, 97 μs, and 3 hnsecs

From,
Vino.B

On Saturday, 7 November 2020 at 14:38:35 UTC, Vino wrote:
> On Saturday, 7 November 2020 at 14:16:46 UTC, Vino wrote:
>> On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:
>>> On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:
>>>> [...]
>>>
>>> While doing performance measurements you should do each test multiple time (at least 5 times). There could be for example an effect that executing a db query the first time is a lot slower than afterwards.
>>>
>>> Maybe mysql native is slower, but maybe this isn't the case, just the way performance measurements was done was incorrectly.
>>>
>>> Kind regards
>>> Andre
>>
>> Hi Andre,
>>
>>   The benchmark was performed with 100 executions as below
>>
>> void main()
>> {
>>  void f() {
>>  Array!string[string] data = getdata("TEST");
>>  foreach(i; data) { writeln(i[]); }
>>  }
>>  auto r = benchmark!(f)(100);
>>  Duration t = r[0];
>>  writeln(t);
>> }
>
> Hi All,
>
>   Upon further testing, below are the observations, based on different compliers (DMD/LDC), using LDC the size of the executable is reduced but the run time is increased from 5 to 6 sec , the document states that if we use the complier option dub --"build=release --compiler=ldmd2" it would enhance the performance  but as per our analysis we do not see and improvement, below is the runtime
>
> LDC                                      : 6 secs, 136 ms, 97 μs, and 3 hnsecs
> LDC(dub --build=release --compiler=ldmd2): 6 secs, 203 ms, 283 μs, and 6 hnsecs
>
>
> Observations
> ********************
> Component 	:  mysql-native + asdf	
> Complier	:  DMD
> Size 		:  17MB
> Duration	:  10 secs, 189 ms, 919 μs, and 3 hnsecs
>
> Component 	:  mysql-native + asdf	
> Complier	:  LDC
> Size 		:  20MB
> Duration	:  10 secs, 526 ms, 350 μs, and 6 hnsecs
>
> Component 	:  mysql-native + asdf	
> Complier	:  LDC(dub --build=release --compiler=ldmd2)
> Size 		:  3.7MB
> Duration	:  10 secs, 411 ms, 793 μs, and 8 hnsecs
> *************************************************************
> Component 	:  hunt-database + asdf	
> Complier	:  DMD
> Size 		:  81MB
> Duration	:  5 secs, 916 ms, 418 μs, and 3 hnsecs
>
> Component 	:  hunt-database + asdf	
> Complier	:  LDC
> Size 		:  50MB
> Duration	:  6 secs, 136 ms, 97 μs, and 3 hnsecs
>
> Component 	:  hunt-database + asdf	
> Complier	:  LDC(dub --build=release --compiler=ldmd2)
> Size 		:  17MB
> Duration	:  6 secs, 136 ms, 97 μs, and 3 hnsecs
>
> From,
> Vino.B

Hi All,

  After further analysis we suspect that the issue is at the package std.net.curl the flow of the program is as below

hunt.database : queries a single table(12 column's and 3 rows) which contains the 3 api url details(getApiconf)

std.net.curl  : call those api's (parallel) and the send the data to json parser

asdf          : Parse the output of the api's and store the data in a array Array!string[string] data

Code of the API calls
import common.GetApiconf;
import core.stdc.stdlib: exit;
import hunt.database;
import std.algorithm: joiner, sort;
import std.container.array;
import std.conv: to;
import std.net.curl : get, HTTP, CurlOption;
import std.parallelism: parallel;
import std.typecons: Tuple, tuple;

void main ()
{
 Array!(Tuple!(int,string)) apidata;
 Row[] result;
 result = getApiconf("TST");
 foreach(i, k; parallel(result,1))
    {

      string apihost = result[i][0].get!(string);
      int apiport = result[i][1].get!(int);
      string apiuser = result[i][2].get!(string);
      string apipass = result[i][3].get!(string);
      int seq = result[i][4].get!int;
      string apiuri = result[i][5].get!(string);
      int connecttimeout = result[i][6].get!(int);
      int tcp_nodelay = result[i][7].get!(int);
      int http_version = result[i][8].get!(int);
      int sslversion = result[i][9].get!(int);
      int use_ssl = result[i][10].get!(int);
      int ssl_verifypeer = result[i][11].get!(int);

      string url = to!string(([apihost, ":",  to!string(apiport), apiuri]).joiner);
      string usrpass = to!string(([apiuser, ":", apipass]).joiner);

      auto https = HTTP();
      https.handle.set(CurlOption.userpwd, usrpass);
      https.handle.set(CurlOption.connecttimeout, connecttimeout);
      https.handle.set(CurlOption.tcp_nodelay, tcp_nodelay);
      https.handle.set(CurlOption.http_version, http_version);
      https.handle.set(CurlOption.sslversion,  sslversion);
      https.handle.set(CurlOption.use_ssl,  use_ssl);
      https.handle.set(CurlOption.ssl_verifypeer, ssl_verifypeer);
      https.handle.set(CurlOption.url, url);
      https.method(HTTP.Method.get);
      https.StatusLine st;
      https.onReceiveStatusLine = (https.StatusLine st) { if (st.code != 200)
                    {
                     throw new Exception(st.reason);
                    }
      };
      ubyte[] content;
      https.onReceive = (ubyte[] data) {
                                         content ~= data; return data.length;
                                       };
      https.perform();
      scope(failure) { https.shutdown; exit(-4); } scope(exit) https.shutdown;
      apidata.insert(tuple(seq, cast(string) content));
 }
 writeln(apidata[].sort);
}

From,
Vino.B

On Saturday, 7 November 2020 at 14:57:39 UTC, Vino wrote:

>
>   After further analysis we suspect that the issue is at the package std.net.curl the flow of the program is as below

Establishing a new connection may vary in duration of +200ms or more.

On Saturday, 7 November 2020 at 15:26:48 UTC, frame wrote:
> On Saturday, 7 November 2020 at 14:57:39 UTC, Vino wrote:
>
>>
>>   After further analysis we suspect that the issue is at the package std.net.curl the flow of the program is as below
>
> Establishing a new connection may vary in duration of +200ms or more.

Hi All,

 Trying to improve the above code hence request your help on how to use array container instead of array, tried as per the example below but not working.

Code:
  ubyte[] content;
  https.onReceive = (ubyte[] data) { content ~= data; return data.length; };


Example
  Array!ubyte content;
  https.onReceive = (Array!ubyte data) { content.insert(data); return data.length; };

From,
Vino.B

On Saturday, 7 November 2020 at 16:37:22 UTC, Vino wrote:
,
>
>  Trying to improve the above code hence request your help on how to use array container instead of array, tried as per the example below but not working.

I think you can just use a static array as ubyte[1024] if you want a fixed buffer - it should convert to the ubyte[] slice and compile.

Try caching your cURL instance if possible since allocating cURL resources have an impact on overall duration.

On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:

> Maybe mysql native is slower, but maybe this isn't the case, just the way performance measurements was done was incorrectly.

I have tried mysql-native and it tooks way to long for simple things.
My test setup is a remote linux vm with a local database and a windows machine with a SSH tunnel connected to that linux database (TCP)

I have tested a very simple SELECT query:

windows (remote):
connect: 50ms
execute: 25ms
-------------
normal since network latency of ~25ms

linux (local):
connect: 1ms
execute: 44ms (should be <1ms)
-------------
abnormal

And if you assume network problems: mysql client -h127.0.0.1 says
1 row in set (0.00 sec)
as expected

Something wrong with this library.