September 01, 2020
On Tuesday, 1 September 2020 at 11:53:25 UTC, Ola Fosheim Grøstad wrote:
> you scale things up, but there are fast noncryptographic hashes. Some are like 10-500x faster than SHA1:

More like 10-50x...
September 01, 2020
On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
> On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
>> Looking for a full-time or part-time gig? Not only is Symmetry Investments hiring D programmers, they are also generously funding two positions for ecosystem work under the D Language Foundation. And they've put up a bounty for a new DUB feature. Read all about it here:
>>
>> https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
>
> As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events.
>
> BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub?
>
> --
> /Jacob Carlborg

Agreed. A server approach would probably scale much better,
if the intent is to speed up the developer's CTR cycle.
But in any case, thanks to Symmetry for doing this! This is huge.
September 01, 2020
On 9/1/20 5:38 AM, Stefan Koch wrote:
> On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
>> BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub?
>>
> 
> We think that not recompiling certain modules which have not changed will improve our build times.
> And the task proposed is actually something that can go in without too much struggle.
> Whereas deeper issues in dub likely take much longer.

I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data?

-Steve
September 01, 2020
On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven Schveighoffer wrote:
> On 9/1/20 5:38 AM, Stefan Koch wrote:
>> On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
>>> BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub?
>>>
>> 
>> We think that not recompiling certain modules which have not changed will improve our build times.
>> And the task proposed is actually something that can go in without too much struggle.
>> Whereas deeper issues in dub likely take much longer.
>
> I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data?
>
> -Steve

git update. on CI.
September 01, 2020
On Tuesday, 1 September 2020 at 13:30:33 UTC, Stefan Koch wrote:
> On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven Schveighoffer wrote:
>> On 9/1/20 5:38 AM, Stefan Koch wrote:
>>> On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
>>>> BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub?
>>>>
>>> 
>>> We think that not recompiling certain modules which have not changed will improve our build times.
>>> And the task proposed is actually something that can go in without too much struggle.
>>> Whereas deeper issues in dub likely take much longer.
>>
>> I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data?
>>
>> -Steve
>
> git update. on CI.

at least I think that was the problem.
I have not looked into this personally.
September 01, 2020
On Tuesday, 1 September 2020 at 12:59:00 UTC, Mathias LANG wrote:
> On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
>> On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
>>> Looking for a full-time or part-time gig? Not only is Symmetry Investments hiring D programmers, they are also generously funding two positions for ecosystem work under the D Language Foundation. And they've put up a bounty for a new DUB feature. Read all about it here:
>>>
>>> https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
>>
>> As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events.
>>
>> BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub?
>>
>> --
>> /Jacob Carlborg
>
> Agreed. A server approach would probably scale much better,
> if the intent is to speed up the developer's CTR cycle.
> But in any case, thanks to Symmetry for doing this! This is huge.

We need both a fs-watcher daemon support and "offline" incremental build support, based on SHA-256 (note that git is moving away from SHA1 to SHA256 [1]). I'd say SHA-256 is cheap enough these days [2] that I don't see a reason not to use it even for "online" fs-watcher daemon compilation.

[1]: https://git-scm.com/docs/hash-function-transition/
[2]: https://bench.cr.yp.to/impl-hash/sha256.html
September 01, 2020
On 9/1/20 7:34 PM, Petar Kirov [ZombineDev] wrote:
> On Tuesday, 1 September 2020 at 12:59:00 UTC, Mathias LANG wrote:
>> On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
>>> On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
>>>> Looking for a full-time or part-time gig? Not only is Symmetry Investments hiring D programmers, they are also generously funding two positions for ecosystem work under the D Language Foundation. And they've put up a bounty for a new DUB feature. Read all about it here:
>>>>
>>>> https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/ 
>>>>
>>>
>>> As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events.
>>>
>>> BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub?
>>>
>>> -- 
>>> /Jacob Carlborg
>>
>> Agreed. A server approach would probably scale much better,
>> if the intent is to speed up the developer's CTR cycle.
>> But in any case, thanks to Symmetry for doing this! This is huge.
> 
> We need both a fs-watcher daemon support and "offline" incremental build support, based on SHA-256 (note that git is moving away from SHA1 to SHA256 [1]). I'd say SHA-256 is cheap enough these days [2] that I don't see a reason not to use it even for "online" fs-watcher daemon compilation.
> 
> [1]: https://git-scm.com/docs/hash-function-transition/
> [2]: https://bench.cr.yp.to/impl-hash/sha256.html

We can easily use the following option:
```
dub build --hash=sha1
dub build --hash=sha256
```
and let the user to make the final choice
September 01, 2020
On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven Schveighoffer wrote:
> On 9/1/20 5:38 AM, Stefan Koch wrote:
>> On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
>>> BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub?
>>>
>> 
>> We think that not recompiling certain modules which have not changed will improve our build times.
>> And the task proposed is actually something that can go in without too much struggle.
>> Whereas deeper issues in dub likely take much longer.
>
> I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data?
>
git checkout branch
git checkout -



September 02, 2020
On Tuesday, 1 September 2020 at 16:45:55 UTC, drug wrote:
> On 9/1/20 7:34 PM, Petar Kirov [ZombineDev] wrote:
>> On Tuesday, 1 September 2020 at 12:59:00 UTC, Mathias LANG wrote:
>>> On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
>>>> [...]
>>>
>>> Agreed. A server approach would probably scale much better,
>>> if the intent is to speed up the developer's CTR cycle.
>>> But in any case, thanks to Symmetry for doing this! This is huge.
>> 
>> We need both a fs-watcher daemon support and "offline" incremental build support, based on SHA-256 (note that git is moving away from SHA1 to SHA256 [1]). I'd say SHA-256 is cheap enough these days [2] that I don't see a reason not to use it even for "online" fs-watcher daemon compilation.
>> 
>> [1]: https://git-scm.com/docs/hash-function-transition/
>> [2]: https://bench.cr.yp.to/impl-hash/sha256.html
>
> We can easily use the following option:
> ```
> dub build --hash=sha1
> dub build --hash=sha256
> ```
> and let the user to make the final choice

I understand your idea to make this configurable, but this will introduce more complexity than necessary because then $HOME/.dub/packages will have build artifacts based on both SHA1 and SHA256, and so dub will need to support mixed mode file integrity checking.
September 02, 2020
On 9/2/20 12:53 PM, Petar Kirov [ZombineDev] wrote:
> 
> I understand your idea to make this configurable, but this will introduce more complexity than necessary because then $HOME/.dub/packages will have build artifacts based on both SHA1 and SHA256, and so dub will need to support mixed mode file integrity checking.

Yesterday I thought about it and now I believe that if you change sha1 to sha256 or vice versa then dub just rebuild all because sha1 and sha256 has different sizes so cached hashes will be different to calculated ones and collisions are impossible. No need to support mixed mode. The drawback is total rebuild on algorithm changing but I suppose it will be seldom operation. Another way is to having several hashes simultaneously and select one of them depending on option value. Implementation of this will be simple.