To Walter

I have posted 130 pulls on github in a year, but about forty ones are
still opened.
I'd like to increase speed of merging, but I don't know the way to realize it.
What is the necessary?

- D1 patch?
  Today only Walter improvements D1 branch. Almost dmd pulls only
consider D2 branch. Should we add D1 patch at the same time?

- Reviewing?
  Almost dmd pulls doesn't have been reviewed by other dmd commiters.

- Conflicting?
  Almost patches uses runnable/xtest46.d, template9.d, aliasthis.d
for test suite. This fact has introduced conflicts between the
patches. I sometimes rebases my many pulls, but the cost of doing it
is much more.

- Others?
???

Please answer this question, Walter. I'd like to improve the bottleneck.

Kenji Hara

Am 11.11.2011, 18:18 Uhr, schrieb kenji hara <k.hara.pg at gmail.com>:

> To Walter
>
> I have posted 130 pulls on github in a year, but about forty ones are
> still opened.
> I'd like to increase speed of merging, but I don't know the way to
> realize it.
> What is the necessary?

The real necessity is giving commit rights to at least you and Don. Your pull requests are almost always accepted without further comments/complaints.


> - D1 patch?
>   Today only Walter improvements D1 branch. Almost dmd pulls only
> consider D2 branch. Should we add D1 patch at the same time?

Might be helpful.

On 11/11/2011 9:18 AM, kenji hara wrote:
> To Walter
>
> I have posted 130 pulls on github in a year, but about forty ones are still opened.

At the moment, I haven't been doing pulling because I'm working on the 64 bit OSX port. When I get that under control, it's back to pulling.

> I'd like to increase speed of merging, but I don't know the way to realize it. What is the necessary?
>
> - D1 patch?
>    Today only Walter improvements D1 branch. Almost dmd pulls only
> consider D2 branch. Should we add D1 patch at the same time?

Merging with D1 hasn't been too difficult; I use a program called "meld" which makes it a snap.

> - Reviewing?
>    Almost dmd pulls doesn't have been reviewed by other dmd commiters.

Not many are that familiar with the internal design.

> - Conflicting?
>    Almost patches uses runnable/xtest46.d, template9.d, aliasthis.d
> for test suite. This fact has introduced conflicts between the
> patches. I sometimes rebases my many pulls, but the cost of doing it
> is much more.

If you want, you can start a new test.d.

> - Others?
> ???

It usually takes a couple hours to merge a patch, if things go smoothly. Most of that is running the test suite. Some more time is spent updating the changelog and bugzilla.

> Please answer this question, Walter. I'd like to improve the bottleneck.

I'd like to compliment you on how prolific, helpful, and correct your patches have been. Thanks!

On Fri, Nov 11, 2011 at 1:22 PM, Walter Bright <walter at digitalmars.com>wrote:

>
> It usually takes a couple hours to merge a patch, if things go smoothly. Most of that is running the test suite. Some more time is spent updating the changelog and bugzilla.


Maybe we need an automated patch tester somewhere, so that if it's green you can merge it without testing it manually.  Someone prototyped this for Phobos, but I don't remember the details or whether it's production quality yet.  At any rate, the same principle should apply for DMD.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-internals/attachments/20111111/79f62ffb/attachment.html>

On 11/11/11 12:22 PM, Walter Bright wrote:
> It usually takes a couple hours to merge a patch, if things go smoothly. Most of that is running the test suite.

That suggests a simple investment of money in bigger iron could have a large positive impact on the development pace. This is NOT the time to be stingy, as being penny wise now means we're negatively impacting D's future - perhaps irreversibly. Walter, I'm ready to pay 50% of whatever machine you want to buy. Please contact me for details.

Kenji's post is symptomatic of a problem that I wished we'd have for a
long time, and that I figured a few months ago it's inevitable: the
bottleneck is not lack of contributions anymore, it's process and logistics.

Walter, we should discuss improving these areas ASAP.

>> Please answer this question, Walter. I'd like to improve the bottleneck.
>
> I'd like to compliment you on how prolific, helpful, and correct your patches have been. Thanks!

I'd like to join Walter, too, but let me also point out that in addition to thanking him, we need to *really* answer Kenji's question with a concerted course of action.


Thanks,

Andrei

On 11/11/2011 10:42 AM, Andrei Alexandrescu wrote:
> On 11/11/11 12:22 PM, Walter Bright wrote:
>> It usually takes a couple hours to merge a patch, if things go smoothly. Most of that is running the test suite.
>
> That suggests a simple investment of money in bigger iron could have a large positive impact on the development pace. This is NOT the time to be stingy, as being penny wise now means we're negatively impacting D's future - perhaps irreversibly. Walter, I'm ready to pay 50% of whatever machine you want to buy. Please contact me for details.
>
> Kenji's post is symptomatic of a problem that I wished we'd have for a
> long time, and that I figured a few months ago it's inevitable: the
> bottleneck is not lack of contributions anymore, it's process and logistics.
>
> Walter, we should discuss improving these areas ASAP.
>
>>> Please answer this question, Walter. I'd like to improve the bottleneck.
>>
>> I'd like to compliment you on how prolific, helpful, and correct your patches have been. Thanks!
>
> I'd like to join Walter, too, but let me also point out that in addition to thanking him, we need to *really* answer Kenji's question with a concerted course of action.
>
>
> Thanks,
>
> Andrei

The bigger issue, imho, is Walter's lack of trust in the automated testers.  If they're insufficient, then lets talk about what it will take to make them sufficient.  They can and will keep way ahead of Walter's individual test runs. Bigger hardware under his desk (or in a closet or in the basement) is the wrong approach, imho.

We've got Daniel's pull tester here: http://yebblies.com/results/

I'm working on incorporating it in to my system so they're will be more parallelism (multiple boxes) and multiple platforms (all of them, ideally).

With trust in the tester, that eliminates hours of time from each pull request.

I basically *always* run test suites (dmd, druntime, and Phobos) with
my patch first in my local PC (Windows), and then post it as a pull
request.
But, unfortunately, to maintain this process is much difficult than a year ago.

I think there are three levels in my pulls .
Level 1. many trivial patches
   Type decoding bug, AST processing bug, ... (almost of them are
front-end issues)
Level 2. some bigger pulls
   relax opAssign signature, rvalue-struct-literal, object not const
correct, opDispatch and IFTI bug, auto return function with
contracts...
Level 3. few enhancements
   Multiple var declaration, typeof(null)

I believe that the level 1 pulls should merged as soon as possible.

Kenji Hara

2011/11/12 Walter Bright <walter at digitalmars.com>:
>
>
> On 11/11/2011 9:18 AM, kenji hara wrote:
>>
>> To Walter
>>
>> I have posted 130 pulls on github in a year, but about forty ones are still opened.
>
> At the moment, I haven't been doing pulling because I'm working on the 64 bit OSX port. When I get that under control, it's back to pulling.
>
>> I'd like to increase speed of merging, but I don't know the way to realize
>> it.
>> What is the necessary?
>>
>> - D1 patch?
>> ? Today only Walter improvements D1 branch. Almost dmd pulls only
>> consider D2 branch. Should we add D1 patch at the same time?
>
> Merging with D1 hasn't been too difficult; I use a program called "meld" which makes it a snap.
>
>> - Reviewing?
>> ? Almost dmd pulls doesn't have been reviewed by other dmd commiters.
>
> Not many are that familiar with the internal design.
>
>> - Conflicting?
>> ? Almost patches uses runnable/xtest46.d, template9.d, aliasthis.d
>> for test suite. This fact has introduced conflicts between the
>> patches. I sometimes rebases my many pulls, but the cost of doing it
>> is much more.
>
> If you want, you can start a new test.d.
>
>> - Others?
>> ???
>
> It usually takes a couple hours to merge a patch, if things go smoothly. Most of that is running the test suite. Some more time is spent updating the changelog and bugzilla.
>
>> Please answer this question, Walter. I'd like to improve the bottleneck.
>
> I'd like to compliment you on how prolific, helpful, and correct your
> patches have been. Thanks!
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
>

On 11/11/2011 10:42 AM, Andrei Alexandrescu wrote:
> On 11/11/11 12:22 PM, Walter Bright wrote:
>> It usually takes a couple hours to merge a patch, if things go smoothly. Most of that is running the test suite.
>
> That suggests a simple investment of money in bigger iron could have a large positive impact on the development pace. This is NOT the time to be stingy, as being penny wise now means we're negatively impacting D's future - perhaps irreversibly. Walter, I'm ready to pay 50% of whatever machine you want to buy. Please contact me for details.
>
> Kenji's post is symptomatic of a problem that I wished we'd have for a
> long time, and that I figured a few months ago it's inevitable: the
> bottleneck is not lack of contributions anymore, it's process and logistics.
>
> Walter, we should discuss improving these areas ASAP.
>
>>> Please answer this question, Walter. I'd like to improve the bottleneck.
>>
>> I'd like to compliment you on how prolific, helpful, and correct your patches have been. Thanks!
>
> I'd like to join Walter, too, but let me also point out that in addition to thanking him, we need to *really* answer Kenji's question with a concerted course of action.
>
>
> Thanks,
>
> Andrei

Also, if bigger iron is the preferred path, I'll pay all the costs for a c1.xlarge ec2 instance.  It's not the fastest box on the market (pre Nehalem), but it's a good price/performance box.  8 cores, with plenty of memory and disk.  Using the spot market and assuming an average of 5 hours a day of usage, it'd only cost about $30/month.  If faster cores are desired, they're available, just cost more.  But paid at hourly actual use rates, would still cost less than buying a box that's not used full time.

On 11/11/11 1:05 PM, Brad Roberts wrote:
> Also, if bigger iron is the preferred path, I'll pay all the costs for a c1.xlarge ec2 instance. It's not the fastest box on the market (pre Nehalem), but it's a good price/performance box. 8 cores, with plenty of memory and disk. Using the spot market and assuming an average of 5 hours a day of usage, it'd only cost about $30/month. If faster cores are desired, they're available, just cost more. But paid at hourly actual use rates, would still cost less than buying a box that's not used full time.

I think that's a terrific offer that we should take. Thanks, Brad! Also it's not about an either-or choice; improvements in both automated testing AND efficient testing should pay off handsomely.

Andrei

Now I know that the Windows auto tester result is breaking, and I
already posted pull request about it.
But there are still not merged few days after posting.
That is the reason I have opened this thread.

I think auto tester is very important component for D's promotion.

Kenji Hara

2011/11/12 Brad Roberts <braddr at puremagic.com>:
> On 11/11/2011 10:42 AM, Andrei Alexandrescu wrote:
>>
>> On 11/11/11 12:22 PM, Walter Bright wrote:
>>>
>>> It usually takes a couple hours to merge a patch, if things go smoothly. Most of that is running the test suite.
>>
>> That suggests a simple investment of money in bigger iron could have a large positive impact on the development pace. This is NOT the time to be stingy, as being penny wise now means we're negatively impacting D's future - perhaps irreversibly. Walter, I'm ready to pay 50% of whatever machine you want to buy. Please contact me for details.
>>
>> Kenji's post is symptomatic of a problem that I wished we'd have for a long time, and that I figured a few months ago it's inevitable: the bottleneck is not lack of contributions anymore, it's process and logistics.
>>
>> Walter, we should discuss improving these areas ASAP.
>>
>>>> Please answer this question, Walter. I'd like to improve the bottleneck.
>>>
>>> I'd like to compliment you on how prolific, helpful, and correct your patches have been. Thanks!
>>
>> I'd like to join Walter, too, but let me also point out that in addition
>> to thanking him, we need to *really* answer
>> Kenji's question with a concerted course of action.
>>
>>
>> Thanks,
>>
>> Andrei
>
> The bigger issue, imho, is Walter's lack of trust in the automated testers. ?If they're insufficient, then lets talk about what it will take to make them sufficient. ?They can and will keep way ahead of Walter's individual test runs. Bigger hardware under his desk (or in a closet or in the basement) is the wrong approach, imho.
>
> We've got Daniel's pull tester here: http://yebblies.com/results/
>
> I'm working on incorporating it in to my system so they're will be more
> parallelism (multiple boxes) and multiple platforms (all of them, ideally).
>
> With trust in the tester, that eliminates hours of time from each pull
> request.
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
>