November 20, 2020
On 19/11/20 20:51, Vino wrote:
> Hi Ali,
> 
> Thank you very much, your solution works for my example, but it does not work for the main goal, let me explain what we are trying to perform.
> 
> Nut shell: Try to execute an aws command on sever accounts in parallel to get some data.
> 
> Noe: each account has as separate username and password store in a database table(encrypted).
> 
> Cod Logic
> Fetch the username/ password from the table for each account.
> 
> Get the “awssecrete” key and “accesskey” for each account by calling an aws api using the above username/password.
> 
> Set the fetched key’s as an environment variable.
> 
> Execute the aws command and get the data for each of the account
> 
>    As we have many accounts what we are trying is to get the data in parallel(execute the aws command in parallel for each account and store the result in a array). At present our code is working fine(without parallel), the moment we enable parallelism, it is throwing an error on the SQL part (Fetch the username/ password from the table for each account), as it could not execute the SQL query in parallel for different account. If there is any other logic please do let me know will give it a try. Below is the SQL code.
> 
> @trusted public auto getAwsconf(immutable string account)
> {
>   auto con = new GetConnections();
>   Statement stmt = con.db.prepare("SELECT username,AES_DECRYPT(b.userpass,b.key,b.vector) AS passwd FROM config WHERE account = :account");
>   stmt.setParameter("account", account);
>   RowSet awsaccount = stmt.query();
>   scope(exit) con.db.close();
>   return awsaccount;
> }
> 
> From,
> Vino.B

Hi,

How does the `GetConnections()` constructor work? Is it creating a new connection to the DB each time it's called, or is it getting them from a pool? In any case it should probably return a different connection for each thread (so the pool size should be at least the same as the number of workers).

Otherwise one thread will try to start a new SQL operation while there is another already running. In the best case, if the library is thread-safe, it could enqueue them thus negating the benefit of parallelism... if not, then you'll likely get errors like the ones you're seeing.

Best,

A.
1 2
Next ›   Last »