On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:
> This a thread to explore whether it would be feasible to do so.
>
> What do you think?
> ------------------
>

One is a compiler the other is a (optional!) library.
merging the two means forcing druntime onto people who want to use dmd.

the seperation of the repositories is a good thing, since it prompt people to think before breaking compatibilty with the runtime.

Which _should_ not happen often anyhow!

druntime and dmd are seperate things.

> - Has the dmd/druntime split being annoying you too?

The split has never anonyned me. I find it to be helpful in fact.

> - Do you have a better suggestion?

Yes, try to implement features in non-breaking, non runtime-dependent way.

> - Would this break your workflow in a drastic way?

I'd have to see the effects, to be definite about this, but likely the answer is yes.

On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:
> This a thread to explore whether it would be feasible to do so.
>
> Motivation
> ----------
>
> [...]

Not much.

> - Do you have a better suggestion?

No.

> - Would this break your workflow in a drastic way?

No, don't really use the official repos anymore, been using the LDC forks instead, so doesn't matter to me how the official git repos for dmd are organized.

LDC splits off the DMD testsuite as its own git repo too, so any merged directory can always be split off to another git repo if wanted:

https://github.com/ldc-developers/dmd-testsuite

On 7/27/18 7:03 AM, Seb wrote:
> This a thread to explore whether it would be feasible to do so.
> 
> Motivation
> ----------
> 
> DRuntime and DMD heavily depend on each other.
> 
> It happens very often that a PR needs to touch both and then a complicated three-step (or sometimes four-step PR series) needs to be done to keep the CIs happy.
> Moreover, as the whole testsuite is at dmd, it happens more often that an error is caught in the testsuite, but would require a druntime PR or the opposite you make a PR to druntime and want its accompanying test to be checked by the CI.

The auto tester is home-grown. We can adjust it if needed to test PRs in tandem.

In fact, I believe we used to use Daniel Murphy's branch to test mutual PRs this way (kind of ad-hoc, but it worked, and was not frequently needed).

I'd MUCH rather not merge these two repos. Note that this sometimes even happens with Phobos too!

If needed, I'd rather there be an interface on the auto-tester to tag which mutual PR is needed from any of the other repos.

-Steve

On Friday, July 27, 2018 6:28:07 AM MDT Seb via Digitalmars-d wrote:
> On Friday, 27 July 2018 at 12:04:18 UTC, Jonathan M Davis wrote:
> > On Friday, July 27, 2018 5:03:50 AM MDT Seb via Digitalmars-d
> >
> > wrote:
> >> What do you think?
> >> ------------------
> >>
> >> - Has the dmd/druntime split being annoying you too?
> >> - Do you have a better suggestion?
> >> - Would this break your workflow in a drastic way?
> >
> > It would break all existing tools and scripts that are used to build the existing repos - many of which are not official tools. So, yes, it would be very annoying.
>
> FWIW I don't think this would be the case. All build scripts I have seen call the Makefile (except for LDC/GDC, of course). The Makefile build would continue to work at any stage.

I'd be _very_ surprised if it doesn't break the tools that I use to build the repos and keep them in sync. The biggest ones to look at though would probably be DVM and digger.

I'd probably be okay with the build process being changed if we were switching to something that was more maintainable than the makefiles, but to rearrange repos? Not so much. In general, the build process is one thing that I'd very much not like to see touched without something that's going to significantly improve it, because pretty every time someone mucks with it, I have to waste time fixing stuff so that it will build for me again, and usually, the changes don't seem like they really improved things, just moved them around.

> > If the repos are merged, then we're forced to either give some of those folks dmd merge rights or make the smaller pool of folks who have merge rights for dmd deal with those PRs.
>
> This shouldn't be a problem either as GitHub's CODEOWNERS files,
> are made for this use case and
> all we need would be sth. this:
>
> * @team-dmd
> src/druntime @team-druntime

Well, that would certainly help, but it still seems like things that should be separate are being shoved together. dmd and druntime are separate beasts and IMHO should be kept as such. Also, even if the codeowners thing deals with the permissions, it still makes it harder for folks to look through the PR list for stuff that's relevant to what they might be merging, since the dmd and druntime PR queues would presumably be the same one at that point. Anyone who works on druntime but not dmd would have to deal with a ton of dmd PRs, whereas currently, they don't.

_Maybe_ it would make sense if we were talking about putting all three repos in one with clear subdirectories for the three projects, since then the repos could be completely in sync at all times, and it would be easier for folks to figure out how to build everything, whereas right now, it's a bit of a pain to deal with it (hence when a number of us have written scripts or tools to deal with the process), but I don't know if that really makes sense when everything is considered - especially with how the PRs would all be in one bucket. We have enough problems with PRs being properly reviewed in a timely manner as it is. Either way, just putting two of them in a single repo seems like it's causing all of the problems that merging them all together would cause without really getting any of the benefits.

> > The dependencies between the three repos do occasionally cause problems, but overall, I think that the separation makes a lot of sense.
>
> It causes A LOT of problems in terms of maintaining the CI and
> build scripts,
> moving things between DMD and adding test for druntime features
> to the dmd testsuite.

Does it really cause that many problems? If so, I'd honestly rather that we look into adjusting how the CI works than merging repos. And if the problem si that we have a lot of PRs where we need a dmd and druntime PR at the same time, then I think that that's something to be concerned about. I'd be worried if the compiler API needs to be changed that frequently.

And from a code cleanliness and maintenance standpoint, I really don't think that it makes sense to merge dmd and druntime. Some pieces are interconnected, yes, but most of druntime is a very separate thing, and for the most part, it should be treated as separate.

- Jonathan M Davis

On Friday, 27 July 2018 at 11:25:37 UTC, Mike Franklin wrote:
> On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:
>
>> - Do you have a better suggestion?
>
> Do we have a rich enough CI API to recognize dependencies between repositories?  For example, if I create a local branch in my dmd repository named `fix_bug` and a local branch in my druntime repository also named `fix_bug` and submit both PRs to their corresponding repositories, can the CI scripts recognize that they have the same branch name and pull them both for testing?
>
> Mike

It's already implemented for branches under the respective dlang repository.
This roughly happens on every CI:

```sh
for proj in druntime phobos; do
    if [ $BRANCH != master ] && [ $BRANCH != stable ] &&
           ! git ls-remote --exit-code --heads https://github.com/dlang/$proj.git $BRANCH > /dev/null; then
        # use master as fallback for other repos to test feature branches
        clone https://github.com/dlang/$proj.git ../$proj master
    else
        clone https://github.com/dlang/$proj.git ../$proj $BRANCH
    fi
done
```

However, this only applies if the PR is targeting the respective branch, which means the workflow is a bit more annoying

* push dmd branch to dlang/druntime-dmd
* open PR at dmd targeting master (from druntime-dmd)
* create druntime-dmd branch at dlang/druntime
* push changes to private branch
* open PR at druntime targetting druntime-dmd
* merge druntime-dmd back to master once the druntime PR has been approved and merged

It's a bit of an overhead for small changes though :/

Checking out the branch from your repository is problematic, because it's not exposed as environment variable and we would need to query the GitHub API to find this. Now the GitHub API gets rate-limited pretty quick, we would have to use our own GitHub API cache layer and ensure the requests coming from the CIs are really ours.

On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:
> This a thread to explore whether it would be feasible to do so.
[...]
> What do you think?
> ------------------
>
> - Has the dmd/druntime split being annoying you too?

I'm not familiar with contributing to the dmd/druntime, but having druntime in a separate repository never made sense to me. If it makes it easier for people to work on dmd/druntime, I'm all up for that.

> - Do you have a better suggestion?

Not at all.

> - Would this break your workflow in a drastic way?

Nope, in my workflow I either use dmd packages from pacman/apt or install.sh script. If these get updated on time I won't even notice that change.

On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:
> This a thread to explore whether it would be feasible to do so.

From my experience, whenever a change in dmd required a change in druntime the protocol was pretty straightforward and not at all annoying:

-> you make the dmd PR and it causes druntime breakage
-> you make the druntime PR to fix the problem

Problem solved. The only overhead here is :

git checkout -b druntime_fix
git push -f origin druntime_fix

Having druntime and dmd as separate repos makes sense because the two are logically separated. If druntime and dmd are merged together based on your motivation, there's only a matter of time until someone, following the same logic as you did, will propose to also merge phobos with dmd + druntime since modifying the latter often requires changing the former.

The only situation where your proposal has any benefit is when a circular dependency arises: you first need the druntime patch to be able to merge the dmd one and at the same time you first need the dmd patch to merge the druntime one. I don't even know if such a situation is possible, but even so, in this case we can just bite the bullet and merge them both at the (pseudo)same time.

RazvanN

On 2018-07-27 13:03, Seb wrote:
> This a thread to explore whether it would be feasible to do so.
>
> Motivation
> ----------
>
> DRuntime and DMD heavily depend on each other.
>
> It happens very often that a PR needs to touch both and then a
> complicated three-step (or sometimes four-step PR series) needs to be
> done to keep the CIs happy.
> Moreover, as the whole testsuite is at dmd, it happens more often that
> an error is caught in the testsuite, but would require a druntime PR or
> the opposite you make a PR to druntime and want its accompanying test to
> be checked by the CI.

Another approach could be to separate the compiler dependent API, i.e. "rt" and the user facing API, i.e. "core". The "rt" API could be moved to the compiler repository while the user facing API could stay.

-- 
/Jacob Carlborg

On 7/27/2018 4:03 AM, Seb wrote:
> This a thread to explore whether it would be feasible to do so.

Since DMD and Druntime require each other, it is the right thing to do.

On Friday, 27 July 2018 at 11:03:50 UTC, Seb wrote:
> This a thread to explore whether it would be feasible to do so.
>
> Motivation
> ----------
>
> [...]

Druntime is not exclusively DMD. For example when porting to other platforms or cruntimes is already hard because of the sync and merge you need to do when you move your changes upstream. Most of the time those changes have nothing to do with dmd.

Having dmd source internwined with druntime will make things even harder.

Better find a solution on the CI server side rather then merge dmd with druntime.