Hello

I'm trying to contact AWS API with D according to documentation:

http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html

there is written example in python which i want to rewrite to D:

http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html

I want to ask about such 2 things:

Is it equivalent in D:

hexDigest!SHA256("sometexttohash");

for such python method:

hashlib.sha256('sometexttohash').hexdigest()

?

Maybe is there some prettier solution in D eg:

toHexString(sha256Of("sometexttohash"));

or

SHA256 sha;
sha.start();
string data = "sometexttohash";
sha.put(data);
toHexString(sha.finish())

If those are equivalents to python method which should i choose?

Last but not least, how to write such function in D:

def sign(key, msg):
    return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()

?

I can't find in standard libraryt hmac function, is it existing?

//holo

On 03/10/15 4:11 PM, holo wrote:
> Hello
>
> I'm trying to contact AWS API with D according to documentation:
>
> http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
>
> there is written example in python which i want to rewrite to D:
>
> http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
>
>
> I want to ask about such 2 things:
>
> Is it equivalent in D:
>
> hexDigest!SHA256("sometexttohash");
>
> for such python method:
>
> hashlib.sha256('sometexttohash').hexdigest()
>
> ?
>
> Maybe is there some prettier solution in D eg:
>
> toHexString(sha256Of("sometexttohash"));
>
> or
>
> SHA256 sha;
> sha.start();
> string data = "sometexttohash";
> sha.put(data);
> toHexString(sha.finish())
>
> If those are equivalents to python method which should i choose?
>
> Last but not least, how to write such function in D:
>
> def sign(key, msg):
>      return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
>
> ?
>
> I can't find in standard libraryt hmac function, is it existing?
>
> //holo

You could implement it yourself, (it looks pretty easy).
Or go the route of Botan: https://github.com/etcimon/botan/blob/3bdc835d5b9bad7523deaa86fe98b1fbb2f09d6b/source/botan/mac/hmac.d

On Saturday, 3 October 2015 at 03:15:21 UTC, Rikki Cattermole wrote:
>
> You could implement it yourself, (it looks pretty easy).
> Or go the route of Botan: https://github.com/etcimon/botan/blob/3bdc835d5b9bad7523deaa86fe98b1fbb2f09d6b/source/botan/mac/hmac.d


Thank you for answer. I used dub to fetch botan, after that i created project:

dub init projectname botan

and tried to add botan module:

$ cat app.d
#!/usr/bin/rdmd

import std.stdio;
import botan.mac.hmac;

void main()
{
	writeln("Edit source/app.d to start your project.");
}

but when im trying to run my simple app i get:

$ ./app.d
./app.d(4): Error: module hmac is in file 'botan/mac/hmac.d' which cannot be read
import path[0] = .
import path[1] = /usr/include/dlang/dmd
Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]

What am i doing wrong? How to add library with dub package manager? Im really beginner in D (basically in programming) sorry if im asking obvious questions.

According to my other question are that functions equivalent to that python one? If they are not doing exact that same i could not create auth string which is needed to connect to AWS api.

On 03/10/15 4:54 PM, holo wrote:
> On Saturday, 3 October 2015 at 03:15:21 UTC, Rikki Cattermole wrote:
>>
>> You could implement it yourself, (it looks pretty easy).
>> Or go the route of Botan:
>> https://github.com/etcimon/botan/blob/3bdc835d5b9bad7523deaa86fe98b1fbb2f09d6b/source/botan/mac/hmac.d
>>
>
>
> Thank you for answer. I used dub to fetch botan, after that i created
> project:
>
> dub init projectname botan
>
> and tried to add botan module:
>
> $ cat app.d
> #!/usr/bin/rdmd
>
> import std.stdio;
> import botan.mac.hmac;
>
> void main()
> {
>      writeln("Edit source/app.d to start your project.");
> }
>
> but when im trying to run my simple app i get:
>
> $ ./app.d
> ./app.d(4): Error: module hmac is in file 'botan/mac/hmac.d' which
> cannot be read
> import path[0] = .
> import path[1] = /usr/include/dlang/dmd
> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>
> What am i doing wrong? How to add library with dub package manager? Im
> really beginner in D (basically in programming) sorry if im asking
> obvious questions.

By the looks of that error message with paths, botan isn't actually added as a dependency. Can you please paste the dub file?

> According to my other question are that functions equivalent to that
> python one? If they are not doing exact that same i could not create
> auth string which is needed to connect to AWS api.

I can't say for certain if it is not exact. The reason I did not answer it is because I am not familiar with environment or any of what you are doing.
I can unfortunately only point you in the right direction.

On 03/10/15 6:01 PM, Rikki Cattermole wrote:
> On 03/10/15 4:54 PM, holo wrote:
>> On Saturday, 3 October 2015 at 03:15:21 UTC, Rikki Cattermole wrote:
>>>
>>> You could implement it yourself, (it looks pretty easy).
>>> Or go the route of Botan:
>>> https://github.com/etcimon/botan/blob/3bdc835d5b9bad7523deaa86fe98b1fbb2f09d6b/source/botan/mac/hmac.d
>>>
>>>
>>
>>
>> Thank you for answer. I used dub to fetch botan, after that i created
>> project:
>>
>> dub init projectname botan
>>
>> and tried to add botan module:
>>
>> $ cat app.d
>> #!/usr/bin/rdmd
>>
>> import std.stdio;
>> import botan.mac.hmac;
>>
>> void main()
>> {
>>      writeln("Edit source/app.d to start your project.");
>> }
>>
>> but when im trying to run my simple app i get:
>>
>> $ ./app.d
>> ./app.d(4): Error: module hmac is in file 'botan/mac/hmac.d' which
>> cannot be read
>> import path[0] = .
>> import path[1] = /usr/include/dlang/dmd
>> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>>
>> What am i doing wrong? How to add library with dub package manager? Im
>> really beginner in D (basically in programming) sorry if im asking
>> obvious questions.
>
> By the looks of that error message with paths, botan isn't actually
> added as a dependency. Can you please paste the dub file?

Ohhhh wait nevermind.
Use dub to compile/run your program. You are using the shebang line!

$ dub run

>> According to my other question are that functions equivalent to that
>> python one? If they are not doing exact that same i could not create
>> auth string which is needed to connect to AWS api.
>
> I can't say for certain if it is not exact. The reason I did not answer
> it is because I am not familiar with environment or any of what you are
> doing.
> I can unfortunately only point you in the right direction.

On Saturday, 3 October 2015 at 05:02:58 UTC, Rikki Cattermole wrote:
> On 03/10/15 6:01 PM, Rikki Cattermole wrote:
>> On 03/10/15 4:54 PM, holo wrote:
>>> On Saturday, 3 October 2015 at 03:15:21 UTC, Rikki Cattermole wrote:
>>>>
>>>> You could implement it yourself, (it looks pretty easy).
>>>> Or go the route of Botan:
>>>> https://github.com/etcimon/botan/blob/3bdc835d5b9bad7523deaa86fe98b1fbb2f09d6b/source/botan/mac/hmac.d
>>>>
>>>>
>>>
>>>
>>> Thank you for answer. I used dub to fetch botan, after that i created
>>> project:
>>>
>>> dub init projectname botan
>>>
>>> and tried to add botan module:
>>>
>>> $ cat app.d
>>> #!/usr/bin/rdmd
>>>
>>> import std.stdio;
>>> import botan.mac.hmac;
>>>
>>> void main()
>>> {
>>>      writeln("Edit source/app.d to start your project.");
>>> }
>>>
>>> but when im trying to run my simple app i get:
>>>
>>> $ ./app.d
>>> ./app.d(4): Error: module hmac is in file 'botan/mac/hmac.d' which
>>> cannot be read
>>> import path[0] = .
>>> import path[1] = /usr/include/dlang/dmd
>>> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>>>
>>> What am i doing wrong? How to add library with dub package manager? Im
>>> really beginner in D (basically in programming) sorry if im asking
>>> obvious questions.
>>
>> By the looks of that error message with paths, botan isn't actually
>> added as a dependency. Can you please paste the dub file?
>
> Ohhhh wait nevermind.
> Use dub to compile/run your program. You are using the shebang line!
>
> $ dub run
>
>>> According to my other question are that functions equivalent to that
>>> python one? If they are not doing exact that same i could not create
>>> auth string which is needed to connect to AWS api.
>>
>> I can't say for certain if it is not exact. The reason I did not answer
>> it is because I am not familiar with environment or any of what you are
>> doing.
>> I can unfortunately only point you in the right direction.

Thank you for explaining how to use dub.

I find out in master branch of phobos on git hub is included hmac.d module which will be added in next release of phobos so i think that will be best solution for me to stick with standard library for beginning.

https://github.com/D-Programming-Language/phobos/tree/master/std

I downloaded hmac.d and new version of digest.d file and place them in my phobos library. I think it exactly contains what i need but when im trying to test it i have such error:

$ ./app.d
./app.d(9): Error: template std.digest.hmac.hmac cannot deduce function from argument types !(SHA!(512, 256))(immutable(ubyte)[]), candidates are:
/usr/include/dlang/dmd/std/digest/hmac.d(202):        std.digest.hmac.hmac(H) if (isDigest!H && hasBlockSize!H)
/usr/include/dlang/dmd/std/digest/hmac.d(208):        std.digest.hmac.hmac(H, ulong blockSize)(scope const(ubyte)[] secret) if (isDigest!H)
Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]



My test code (based on unint test from module) looks like that:

#!/usr/bin/rdmd

import std.stdio;
import std.digest.sha, std.digest.hmac;
import std.string : representation;

void main()
{
	auto hmac = hmac!SHA256("secretkey".representation);
	auto digest = hmac.put("texttohash".representation).finish;
	writeln(digest);
}

Am i using it correctly?

On 04/10/15 1:09 AM, holo wrote:
> On Saturday, 3 October 2015 at 05:02:58 UTC, Rikki Cattermole wrote:
>> On 03/10/15 6:01 PM, Rikki Cattermole wrote:
>>> On 03/10/15 4:54 PM, holo wrote:
>>>> On Saturday, 3 October 2015 at 03:15:21 UTC, Rikki Cattermole wrote:
>>>>>
>>>>> You could implement it yourself, (it looks pretty easy).
>>>>> Or go the route of Botan:
>>>>> https://github.com/etcimon/botan/blob/3bdc835d5b9bad7523deaa86fe98b1fbb2f09d6b/source/botan/mac/hmac.d
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> Thank you for answer. I used dub to fetch botan, after that i created
>>>> project:
>>>>
>>>> dub init projectname botan
>>>>
>>>> and tried to add botan module:
>>>>
>>>> $ cat app.d
>>>> #!/usr/bin/rdmd
>>>>
>>>> import std.stdio;
>>>> import botan.mac.hmac;
>>>>
>>>> void main()
>>>> {
>>>>      writeln("Edit source/app.d to start your project.");
>>>> }
>>>>
>>>> but when im trying to run my simple app i get:
>>>>
>>>> $ ./app.d
>>>> ./app.d(4): Error: module hmac is in file 'botan/mac/hmac.d' which
>>>> cannot be read
>>>> import path[0] = .
>>>> import path[1] = /usr/include/dlang/dmd
>>>> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>>>>
>>>> What am i doing wrong? How to add library with dub package manager? Im
>>>> really beginner in D (basically in programming) sorry if im asking
>>>> obvious questions.
>>>
>>> By the looks of that error message with paths, botan isn't actually
>>> added as a dependency. Can you please paste the dub file?
>>
>> Ohhhh wait nevermind.
>> Use dub to compile/run your program. You are using the shebang line!
>>
>> $ dub run
>>
>>>> According to my other question are that functions equivalent to that
>>>> python one? If they are not doing exact that same i could not create
>>>> auth string which is needed to connect to AWS api.
>>>
>>> I can't say for certain if it is not exact. The reason I did not answer
>>> it is because I am not familiar with environment or any of what you are
>>> doing.
>>> I can unfortunately only point you in the right direction.
>
> Thank you for explaining how to use dub.
>
> I find out in master branch of phobos on git hub is included hmac.d
> module which will be added in next release of phobos so i think that
> will be best solution for me to stick with standard library for beginning.
>
> https://github.com/D-Programming-Language/phobos/tree/master/std
>
> I downloaded hmac.d and new version of digest.d file and place them in
> my phobos library. I think it exactly contains what i need but when im
> trying to test it i have such error:
>
> $ ./app.d
> ./app.d(9): Error: template std.digest.hmac.hmac cannot deduce function
> from argument types !(SHA!(512, 256))(immutable(ubyte)[]), candidates are:
> /usr/include/dlang/dmd/std/digest/hmac.d(202): std.digest.hmac.hmac(H)
> if (isDigest!H && hasBlockSize!H)
> /usr/include/dlang/dmd/std/digest/hmac.d(208): std.digest.hmac.hmac(H,
> ulong blockSize)(scope const(ubyte)[] secret) if (isDigest!H)
> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>
>
>
> My test code (based on unint test from module) looks like that:
>
> #!/usr/bin/rdmd
>
> import std.stdio;
> import std.digest.sha, std.digest.hmac;
> import std.string : representation;
>
> void main()
> {
>      auto hmac = hmac!SHA256("secretkey".representation);
>      auto digest = hmac.put("texttohash".representation).finish;
>      writeln(digest);
> }
>
> Am i using it correctly?

By the looks of things the problem is with SHA256, I'm guessing it doesn't have its block size defined.
https://github.com/D-Programming-Language/phobos/blob/master/std/digest/digest.d#L394
You'll also need to update sha.d as well. Otherwise it looks good.

On Saturday, 3 October 2015 at 12:22:11 UTC, Rikki Cattermole wrote:
> On 04/10/15 1:09 AM, holo wrote:
>> On Saturday, 3 October 2015 at 05:02:58 UTC, Rikki Cattermole wrote:
>>> On 03/10/15 6:01 PM, Rikki Cattermole wrote:
>>>> On 03/10/15 4:54 PM, holo wrote:
>>>>> [...]
>>>>
>>>> By the looks of that error message with paths, botan isn't actually
>>>> added as a dependency. Can you please paste the dub file?
>>>
>>> Ohhhh wait nevermind.
>>> Use dub to compile/run your program. You are using the shebang line!
>>>
>>> $ dub run
>>>
>>>>> [...]
>>>>
>>>> I can't say for certain if it is not exact. The reason I did not answer
>>>> it is because I am not familiar with environment or any of what you are
>>>> doing.
>>>> I can unfortunately only point you in the right direction.
>>
>> Thank you for explaining how to use dub.
>>
>> I find out in master branch of phobos on git hub is included hmac.d
>> module which will be added in next release of phobos so i think that
>> will be best solution for me to stick with standard library for beginning.
>>
>> https://github.com/D-Programming-Language/phobos/tree/master/std
>>
>> I downloaded hmac.d and new version of digest.d file and place them in
>> my phobos library. I think it exactly contains what i need but when im
>> trying to test it i have such error:
>>
>> $ ./app.d
>> ./app.d(9): Error: template std.digest.hmac.hmac cannot deduce function
>> from argument types !(SHA!(512, 256))(immutable(ubyte)[]), candidates are:
>> /usr/include/dlang/dmd/std/digest/hmac.d(202): std.digest.hmac.hmac(H)
>> if (isDigest!H && hasBlockSize!H)
>> /usr/include/dlang/dmd/std/digest/hmac.d(208): std.digest.hmac.hmac(H,
>> ulong blockSize)(scope const(ubyte)[] secret) if (isDigest!H)
>> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>>
>>
>>
>> My test code (based on unint test from module) looks like that:
>>
>> #!/usr/bin/rdmd
>>
>> import std.stdio;
>> import std.digest.sha, std.digest.hmac;
>> import std.string : representation;
>>
>> void main()
>> {
>>      auto hmac = hmac!SHA256("secretkey".representation);
>>      auto digest = hmac.put("texttohash".representation).finish;
>>      writeln(digest);
>> }
>>
>> Am i using it correctly?
>
> By the looks of things the problem is with SHA256, I'm guessing it doesn't have its block size defined.
> https://github.com/D-Programming-Language/phobos/blob/master/std/digest/digest.d#L394
> You'll also need to update sha.d as well. Otherwise it looks good.

I updated sha.d and now im getting following error:

$ ./app.d
/usr/include/dlang/dmd/std/digest/sha.d(225): Error: pure function 'std.digest.sha.SHA!(512u, 160u).SHA.transform' cannot call impure function 'core.cpuid.ssse3'
/usr/include/dlang/dmd/std/digest/digest.d(285): Error: template instance std.range.primitives.isOutputRange!(SHA!(512u, 160u), const(ubyte)[]) error instantiating
/usr/include/dlang/dmd/std/digest/digest.d(851):        instantiated from here: isDigest!(SHA!(512u, 160u))
/usr/include/dlang/dmd/std/digest/sha.d(1179):        while looking for match for WrapperDigest!(SHA!(512u, 160u))
Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]

On 04/10/15 1:49 AM, holo wrote:
> On Saturday, 3 October 2015 at 12:22:11 UTC, Rikki Cattermole wrote:
>> On 04/10/15 1:09 AM, holo wrote:
>>> On Saturday, 3 October 2015 at 05:02:58 UTC, Rikki Cattermole wrote:
>>>> On 03/10/15 6:01 PM, Rikki Cattermole wrote:
>>>>> On 03/10/15 4:54 PM, holo wrote:
>>>>>> [...]
>>>>>
>>>>> By the looks of that error message with paths, botan isn't actually
>>>>> added as a dependency. Can you please paste the dub file?
>>>>
>>>> Ohhhh wait nevermind.
>>>> Use dub to compile/run your program. You are using the shebang line!
>>>>
>>>> $ dub run
>>>>
>>>>>> [...]
>>>>>
>>>>> I can't say for certain if it is not exact. The reason I did not
>>>>> answer
>>>>> it is because I am not familiar with environment or any of what you
>>>>> are
>>>>> doing.
>>>>> I can unfortunately only point you in the right direction.
>>>
>>> Thank you for explaining how to use dub.
>>>
>>> I find out in master branch of phobos on git hub is included hmac.d
>>> module which will be added in next release of phobos so i think that
>>> will be best solution for me to stick with standard library for
>>> beginning.
>>>
>>> https://github.com/D-Programming-Language/phobos/tree/master/std
>>>
>>> I downloaded hmac.d and new version of digest.d file and place them in
>>> my phobos library. I think it exactly contains what i need but when im
>>> trying to test it i have such error:
>>>
>>> $ ./app.d
>>> ./app.d(9): Error: template std.digest.hmac.hmac cannot deduce function
>>> from argument types !(SHA!(512, 256))(immutable(ubyte)[]), candidates
>>> are:
>>> /usr/include/dlang/dmd/std/digest/hmac.d(202): std.digest.hmac.hmac(H)
>>> if (isDigest!H && hasBlockSize!H)
>>> /usr/include/dlang/dmd/std/digest/hmac.d(208): std.digest.hmac.hmac(H,
>>> ulong blockSize)(scope const(ubyte)[] secret) if (isDigest!H)
>>> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>>>
>>>
>>>
>>> My test code (based on unint test from module) looks like that:
>>>
>>> #!/usr/bin/rdmd
>>>
>>> import std.stdio;
>>> import std.digest.sha, std.digest.hmac;
>>> import std.string : representation;
>>>
>>> void main()
>>> {
>>>      auto hmac = hmac!SHA256("secretkey".representation);
>>>      auto digest = hmac.put("texttohash".representation).finish;
>>>      writeln(digest);
>>> }
>>>
>>> Am i using it correctly?
>>
>> By the looks of things the problem is with SHA256, I'm guessing it
>> doesn't have its block size defined.
>> https://github.com/D-Programming-Language/phobos/blob/master/std/digest/digest.d#L394
>>
>> You'll also need to update sha.d as well. Otherwise it looks good.
>
> I updated sha.d and now im getting following error:
>
> $ ./app.d
> /usr/include/dlang/dmd/std/digest/sha.d(225): Error: pure function
> 'std.digest.sha.SHA!(512u, 160u).SHA.transform' cannot call impure
> function 'core.cpuid.ssse3'
> /usr/include/dlang/dmd/std/digest/digest.d(285): Error: template
> instance std.range.primitives.isOutputRange!(SHA!(512u, 160u),
> const(ubyte)[]) error instantiating
> /usr/include/dlang/dmd/std/digest/digest.d(851): instantiated from here:
> isDigest!(SHA!(512u, 160u))
> /usr/include/dlang/dmd/std/digest/sha.d(1179):        while looking for
> match for WrapperDigest!(SHA!(512u, 160u))
> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]

And one more file (ssse3.d) needs to be updated.
You are going to be playing whack a mole I think for a while.

On Saturday, 3 October 2015 at 12:50:58 UTC, Rikki Cattermole wrote:
> On 04/10/15 1:49 AM, holo wrote:
>> On Saturday, 3 October 2015 at 12:22:11 UTC, Rikki Cattermole wrote:
>>> On 04/10/15 1:09 AM, holo wrote:
>>>> [...]
>>>
>>> By the looks of things the problem is with SHA256, I'm guessing it
>>> doesn't have its block size defined.
>>> https://github.com/D-Programming-Language/phobos/blob/master/std/digest/digest.d#L394
>>>
>>> You'll also need to update sha.d as well. Otherwise it looks good.
>>
>> I updated sha.d and now im getting following error:
>>
>> $ ./app.d
>> /usr/include/dlang/dmd/std/digest/sha.d(225): Error: pure function
>> 'std.digest.sha.SHA!(512u, 160u).SHA.transform' cannot call impure
>> function 'core.cpuid.ssse3'
>> /usr/include/dlang/dmd/std/digest/digest.d(285): Error: template
>> instance std.range.primitives.isOutputRange!(SHA!(512u, 160u),
>> const(ubyte)[]) error instantiating
>> /usr/include/dlang/dmd/std/digest/digest.d(851): instantiated from here:
>> isDigest!(SHA!(512u, 160u))
>> /usr/include/dlang/dmd/std/digest/sha.d(1179):        while looking for
>> match for WrapperDigest!(SHA!(512u, 160u))
>> Failed: ["dmd", "-v", "-o-", "./app.d", "-I."]
>
> And one more file (ssse3.d) needs to be updated.
> You are going to be playing whack a mole I think for a while.

I downloaded whole master branch phobos and tried to use it (changed path in dmd.conf), but there are missing much more files (i think that master branch is not ready yet). So i get back to my original library and tried to updated file you mention. But i can't find it (ssse3.d), its not appearing in my stable lib and even in that master-branch from zip file.