Although BugZilla has served well for many years, I would like to propose a move to Jira.  I see the following benefits:

* BugZilla is a bug tracking system; Jira is bug tracking + project management
* Jira has a more mature UI, due to commercial licensing paying for development
* Jira is free for OSS!

So why does D need project management?  I would suggest that work on D is somewhat disorganized and haphazard.  That is partly just the nature of OSS/volunteer work, but also partly due to a lack of organizational structure.  Although language changes are neatly encapsulated by D Improvement Proposals, the vast majority of work which would benefit D falls far short of a proposed language change, and is not neatly and clearly tracked.  From what I can tell, this is where someone would look to find out where to make a contribution:

http://wiki.dlang.org/How_You_Can_Help

This is better than nothing, but just barely.  With a proper project management tool, we can capture issues which arise on the newsgroups, informal discussions, conferences, IRC, etc., along with associated communication and any artifacts produced.  We can order work by importance and track progress.  We can set goals and gain visibility into how and where contributions are made.

None of this means that the current D community workflow needs to change, other than using Jira instead of BugTracker.  While this does incur a user cost to switch, I think most people will find that Jira is generally easy to use and relatively painless to pick up. They should also find it much easier to run reports and view dashboards.

If the community agrees to make the switch, I will volunteer to apply for the Jira license and work with the current BugZilla maintainer(s) to migrate the DB (Jira has a tool for BugZilla to do this).  I will also volunteer to capture information from the newsgroups into tasks and other relevant project-tracking data.

Although it is possible to track non-bug issues in BugZilla, this is not the kind of use case for which BugZilla was designed.  Since Atlassian is making their tools available to us for free, I think we should take advantage of this offer.  For the record, I think we should stick with MediaWiki rather than move to Confluence.  And I do not know of anyone who thinks that Stash is better than GitHub.

Dave


P.S. For those less familiar with the Agile methodology, what I propose to do is to create a master backlog of work.  The community can collaborate to define the ordering of the backlog, but the backlog would represent the priority of work, such that someone who wanted to contribute would know that the top item on the backlog is considered the most important task to work on next, and they could just scan from the top down until they find something they think they can work on.

We can debate whether there should be a backlog per work type (website maintenance, compiler maintenance, documentation, etc.) or a global backlog, but these things only make sense to discuss when we have a tool that makes maintaining a backlog easy to do.  The backlog can also capture work which was only partially done.  So someone might pick up a task, work on it for a while, and then update the work item to explain how far they got, even if they couldn't close it out.

The project tracking system can help produce reports on task progress, bug vs. non-bug work rate, etc.  The idea is that by having easy access to this reporting information, the community can have greater visibility into the overall workings and accomplishments of the "D machine" and see the areas which are deemed important but neglected in a quantified way.  Ideally, this would result in self-directed behavior which would optimize towards the high-value work rather than just the easy work.  Even if it only does so in, say, 20% of the contributions, this would still be a significant improvement in return-on-contribution-invested.

Furthermore, contributors could voluntarily report on status, which would give a sense of what people were actively working on, and help answer the question: "Why isn't issue X being addressed?"  Folks could just look at the capacity dashboard and say: "Oh, I see people are working on Y and Z, which is much higher in the backlog."  Then they could either lobby to have the backlog reordered, or they could admit that the highest value-add was being worked on, and pitch in to the tasks that they deem important.

Of course, I have done a lot of hand-waving about "community-ordered backlog".  In reality, I expect this to be a major point of contention and source of more than a few flame wars.  But having a single place to witness the debate over priority unfold should be valuable all by itself.  As long as justifications are provided as to why the backlog is ordered the way it is, I think the community can come together and agree that it is reasonable, even if it doesn't fit every person's exact preferences (nor could such an ordering be produced).  And, of course, the backlog doesn't determine what anyone works on.  It just represents what the community thinks is important, in rank order.  Anyone is still free to pick something from the bottom and work on it.  However, finding that a lot of people are working on low-ranked tasks should communicate something important about the community in and of itself.  That's why these kinds of tools are important.

A unified backlog should also make it easier for people to discover overlapping and/or duplicated work.  This probably doesn't happen often, but may save a lot of time and effort for folks when it does.  At the least it should make people aware of each other's work so they can proceed deliberately rather than in ignorance.  These kinds of things are done informally now via the newsgroups, but are a kind of waste of contributor cycles, because humans do not need to be the web servers for this kind of mundane information.  By offloading these overhead tasks to the proper software, the forums can be optimized for discussions which cannot be automated.  That should make everyone happier and more efficient.

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

I'll be curious what Martin thinks of this.

My thoughts - as one who's thought of this for years and tried various angles - are generally in disagreement with yours, though not vehemently so.

* This looks like a random change of a tool that works, for benefits that seem small and an overhaul that seems large.

* We are trying trello for management. This is the second attempt to introduce it, by Martin; first, by me, was a failure. Sadly this one also doesn't seem to fare very well.

* Don't take this personally, but we've had a pattern of strong up-and-coming folks without a track record and proposing a large change with "trust me it'll be better" written all over it. Invariably those have been flashes in the pan. If I remember correctly, one of such initiatives was from yourself. Again, don't take this personally but it is my opinion that if you or anyone wants to impact the community, the one way is to consistently get work done on issues that matter.

* I believed D has had a management problem for a good while, until I figured it doesn't. We are in need of talent. I understand how being at DConf - it's a good conference and those are inspiring - instills one with the impression that there's just a lot of folks ready to work but are not organized properly. Monitoring the forums may give a similar impression. The reality in the trenches is we have problems that are very well known, often for years, small and large - often unbelievably trivial - and there are very few people out there to work on them. Those who do, do make progress (e.g. look at Martin's work on the GC, Unique, RefCounted, etc., or Walter's work on weaning Phobos off GC). Walter and I have been trying to lead both by example and by simply telling what needs to be done, several time, with virtually no success.


Andrei

On 5/30/15 6:01 PM, David Held via dmd-internals wrote:
> Although BugZilla has served well for many years, I would like to
> propose a move to Jira.  I see the following benefits:
>
> * BugZilla is a bug tracking system; Jira is bug tracking + project
> management
> * Jira has a more mature UI, due to commercial licensing paying for
> development
> * Jira is free for OSS!
>
> So why does D need project management?  I would suggest that work on D
> is somewhat disorganized and haphazard.  That is partly just the nature
> of OSS/volunteer work, but also partly due to a lack of organizational
> structure.  Although language changes are neatly encapsulated by D
> Improvement Proposals, the vast majority of work which would benefit D
> falls far short of a proposed language change, and is not neatly and
> clearly tracked.  From what I can tell, this is where someone would look
> to find out where to make a contribution:
>
> http://wiki.dlang.org/How_You_Can_Help
>
> This is better than nothing, but just barely.  With a proper project
> management tool, we can capture issues which arise on the newsgroups,
> informal discussions, conferences, IRC, etc., along with associated
> communication and any artifacts produced.  We can order work by
> importance and track progress.  We can set goals and gain visibility
> into how and where contributions are made.
>
> None of this means that the current D community workflow needs to
> change, other than using Jira instead of BugTracker.  While this does
> incur a user cost to switch, I think most people will find that Jira is
> generally easy to use and relatively painless to pick up. They should
> also find it much easier to run reports and view dashboards.
>
> If the community agrees to make the switch, I will volunteer to apply
> for the Jira license and work with the current BugZilla maintainer(s) to
> migrate the DB (Jira has a tool for BugZilla to do this).  I will also
> volunteer to capture information from the newsgroups into tasks and
> other relevant project-tracking data.
>
> Although it is possible to track non-bug issues in BugZilla, this is not
> the kind of use case for which BugZilla was designed.  Since Atlassian
> is making their tools available to us for free, I think we should take
> advantage of this offer.  For the record, I think we should stick with
> MediaWiki rather than move to Confluence.  And I do not know of anyone
> who thinks that Stash is better than GitHub.
>
> Dave
>
>
> P.S. For those less familiar with the Agile methodology, what I propose
> to do is to create a master backlog of work.  The community can
> collaborate to define the ordering of the backlog, but the backlog would
> represent the priority of work, such that someone who wanted to
> contribute would know that the top item on the backlog is considered the
> most important task to work on next, and they could just scan from the
> top down until they find something they think they can work on.
>
> We can debate whether there should be a backlog per work type (website
> maintenance, compiler maintenance, documentation, etc.) or a global
> backlog, but these things only make sense to discuss when we have a tool
> that makes maintaining a backlog easy to do.  The backlog can also
> capture work which was only partially done.  So someone might pick up a
> task, work on it for a while, and then update the work item to explain
> how far they got, even if they couldn't close it out.
>
> The project tracking system can help produce reports on task progress,
> bug vs. non-bug work rate, etc.  The idea is that by having easy access
> to this reporting information, the community can have greater visibility
> into the overall workings and accomplishments of the "D machine" and see
> the areas which are deemed important but neglected in a quantified way.
> Ideally, this would result in self-directed behavior which would
> optimize towards the high-value work rather than just the easy work.
> Even if it only does so in, say, 20% of the contributions, this would
> still be a significant improvement in return-on-contribution-invested.
>
> Furthermore, contributors could voluntarily report on status, which
> would give a sense of what people were actively working on, and help
> answer the question: "Why isn't issue X being addressed?"  Folks could
> just look at the capacity dashboard and say: "Oh, I see people are
> working on Y and Z, which is much higher in the backlog."  Then they
> could either lobby to have the backlog reordered, or they could admit
> that the highest value-add was being worked on, and pitch in to the
> tasks that they deem important.
>
> Of course, I have done a lot of hand-waving about "community-ordered
> backlog".  In reality, I expect this to be a major point of contention
> and source of more than a few flame wars.  But having a single place to
> witness the debate over priority unfold should be valuable all by
> itself.  As long as justifications are provided as to why the backlog is
> ordered the way it is, I think the community can come together and agree
> that it is reasonable, even if it doesn't fit every person's exact
> preferences (nor could such an ordering be produced).  And, of course,
> the backlog doesn't determine what anyone works on.  It just represents
> what the community thinks is important, in rank order.  Anyone is still
> free to pick something from the bottom and work on it.  However, finding
> that a lot of people are working on low-ranked tasks should communicate
> something important about the community in and of itself.  That's why
> these kinds of tools are important.
>
> A unified backlog should also make it easier for people to discover
> overlapping and/or duplicated work.  This probably doesn't happen often,
> but may save a lot of time and effort for folks when it does.  At the
> least it should make people aware of each other's work so they can
> proceed deliberately rather than in ignorance.  These kinds of things
> are done informally now via the newsgroups, but are a kind of waste of
> contributor cycles, because humans do not need to be the web servers for
> this kind of mundane information.  By offloading these overhead tasks to
> the proper software, the forums can be optimized for discussions which
> cannot be automated.  That should make everyone happier and more efficient.
>
> _______________________________________________
> dmd-internals mailing list
> dmd-internals@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 05/31/2015 07:16 AM, Andrei Alexandrescu via dmd-internals wrote:
> I'll be curious what Martin thinks of this.
> 
> * This looks like a random change of a tool that works, for benefits that seem small and an overhaul that seems large.

A tool on it's own doesn't fix our workflow, but if we fix our workflow,
then Jira might
be a better tool to integrate issue tracking and planning.

> * We are trying trello for management. This is the second attempt to introduce it, by Martin; first, by me, was a failure. Sadly this one also doesn't seem to fare very well.

Same as above, b/c nobody took the time to participate. A tool can't
magically fix our workflow, we need everyone to take the time, learn the
process and find a better workflow for us.
For example we don't have sprint meetings yet and hardly anyone but me
has created cards.

> * Don't take this personally, but we've had a pattern of strong up-and-coming folks without a track record and proposing a large change with "trust me it'll be better" written all over it. Invariably those have been flashes in the pan. If I remember correctly, one of such initiatives was from yourself. Again, don't take this personally but it is my opinion that if you or anyone wants to impact the community, the one way is to consistently get work done on issues that matter.

> * I believed D has had a management problem for a good while, until I figured it doesn't. We are in need of talent. I understand how being at DConf - it's a good conference and those are inspiring - instills one with the impression that there's just a lot of folks ready to work but are not organized properly.

You're missing a 2 important points.
We're killing a lot of work at the last moment, b/c there is no way to
plan work upfront. And it's difficult to do "important" work b/c it's
hardly possible to get the necessary attention.
Both points are clear management/communication problems.

Take the "AA proof of concept" as an example. https://github.com/D-Programming-Language/druntime/pull/1282

Look at all the recently reverted pulls or the famous const(Object)ref https://github.com/D-Programming-Language/dmd/pull/3 change.

All of these create a high level of frustration and can be easily addressed with very little communication.


Part of the problem is inherent in github's pull request workflow.
I think we should shift a bit towards more upfront planned work by core
contributors and planned collaborations on particular topics with other
contributors. Pull request continue to work fine for small scale
fixes/additions, but are flawed for huge contributions.