On Wednesday, 14 January 2015 at 04:29:57 UTC, Paul O'Neil wrote:
>
> I put off some documentation updates a few months ago because it wasn't
> worth the investment yet to figure out how to build the website.  The
> wiki pages appeared to be about building the compiler, which I didn't
> realize I should be interested in.  This barrier is too high.

I know the feeling.  I finally figured out how to get a single
ddoc file to html so I could help update the documentation.  I
added a few notes to the dlang.org CONTRIBUTING.md [1] file a
while back to help other users with that.  It's probably not very
noticeable there, but I don't understand the build process well
enough to do a proper job.

[1]
https://github.com/D-Programming-Language/dlang.org/blob/master/CONTRIBUTING.md

Why does documentation need to be "built" anyway.  I understand
the autogenerated stuff from the source code, but not the
website.  Would it be better to wikify it, somehow, in GitHub
pages?

Mike

"Andrei Alexandrescu"  wrote in message news:m94vfk$2gdh$1@digitalmars.com...

> Hmmm... I wonder why the distinction.

OS detection is required to build successfully, a rebase shortcut is not.

> The problem is basic code duplication with its known liabilities. I'm looking at stuff like this:
>
> Three's a charm. That code is relatively verbose (required newlines). I trust it doesn't need updates often, but it occupies real estate in a readily accessible position (beginning of file).
>
> The real issue here is that acquiescing to such duplication discourages attempts at uniformization before they occur. I'm thinking of simpler, clearer naming conventions and uniform ways of doing the same things (such as unittesting or documentation).

The problem here is really that make sucks, right?  And makefiles suck. Maybe we should start seriously looking at replacing them with D scripts.

> Getting back to REBASE (https://github.com/D-Programming-Language/dlang.org/blob/master/posix.mak#L37), it greatly simplifies my building the entire website in a way I trust is nice:
>
>    make rebase -j && make clean && make rsync -j
>
> (Sadly make clean is necessary because of dub, which makes the process take a lot longer than it should; with luck, somebody will improve on that soon.)
>
> Now "make rebase" is really useful for druntime and phobos as well. But between duplicating more code or just doing it by hand, one may as well give up and choose the latter.

Honestly I don't think automating rebase is a good idea, because it discourages actually learning how to do it.

eg if all you're trying to do is get the current branch rebased on top of upstream's master, you can do this:

git pull upstream master --rebase

> I'd also like to define "make rebase-dirty" based on this:
>
> REBASE_DIRTY = MYBRANCH=`git rev-parse --abbrev-ref HEAD` &&\
>   git stash &&\
>   git co master &&\
>   git pull --ff-only upstream master &&\
>   git co $$MYBRANCH &&\
>   git rebase master &&\
>   git stash pop
>
> which is useful because it lets me rebase on top of work in progress. Again the prospect of duplicating this other macro across 3-4 projects is fairly unattractive.

I've managed to get by fine with using shell scripts for stuff like this. 

On Wednesday, 14 January 2015 at 07:55:31 UTC, Mike wrote:

> Would it be better to wikify it, somehow, in GitHub
> pages?

I think wiki is the wrong term here.  I mean, is there a way to leverage Markdown (or some other markup language) with GitHub's source code management features to make it easier for contributors to fork, edit in their browser, verify in their repository, and create pull requests without having to build with a compiler?

Mike

On Tue, 2015-01-13 at 15:20 -0800, Andrei Alexandrescu via Digitalmars-d wrote:
> Hey folks,
> 
> 
> Over the time a number of stuff has become quite duplicated across our makefiles for dmd, druntime, and phobos.

Are these three separate things with separate builds, or are they three components of the same system? What is the relationship between these three, LDC and GDC?

> These include fetching OS and model but (newer) general-purpose macros for e.g. rebasing repos, see https://github.com/D-Programming-Language/dlang.org/blob/master/posix.mak#L37.
> 
> I think it's time to reduce clutter and duplication by migrating such common stuff into one common.mak file. Question is, where should that file sit? One candidate is tools/ but it's not really a tool. Another candidate is dmd/src/ because that would be the first thing anyone depends on.
> 
> Ideas?

Are all three things in the same Git repository? Can I clone the three independently or do I have to have them all? Can any of them be built without building the other or does a DMD build require a new druntime and phobos build.

Basically what I am trying to say here is that I suspect the extraction of commonality is going to create an inappropriate coupling. The solution is to have the common material as a fourth thing that can be brought in to any or all of the three. With SCons and Waf this is achieved by having a tool, easy simple and keeps the coupling low.

I haven't used Make in 12 years, so I have no idea what it can do these days, But unless you mandate GNU Make, I suspect it is going to be awful trying to build across Windows, OSX, Debian, Arch, Fedora, FreeBSD, OpenBSD, etc. without hacking (in the worst sense).

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 1/14/15 12:29 AM, Daniel Murphy wrote:
> The problem here is really that make sucks, right?  And makefiles suck.
> Maybe we should start seriously looking at replacing them with D scripts.

I agree that make sucks etc. but duplication sucks more (and would suck regardless of what tool we use). Here we do have a simple method (include files) that is suitable for addressing that. I'd assume we'd use something similar with whichever tool we'd use.

> Honestly I don't think automating rebase is a good idea, because it
> discourages actually learning how to do it.
>
> eg if all you're trying to do is get the current branch rebased on top
> of upstream's master, you can do this:
>
> git pull upstream master --rebase

Great. As an aside I just ran that and got:

error: unknown option `rebase'
usage: ...

Whether it's a command or a couple, that doesn't change the fact that I need to run the same command four times, and in parallel if at all possible. It's a perfect candidate for automation, and I haven't seen an argument the makefile isn't a good place for that.

>> I'd also like to define "make rebase-dirty" based on this:
>>
>> REBASE_DIRTY = MYBRANCH=`git rev-parse --abbrev-ref HEAD` &&\
>>   git stash &&\
>>   git co master &&\
>>   git pull --ff-only upstream master &&\
>>   git co $$MYBRANCH &&\
>>   git rebase master &&\
>>   git stash pop
>>
>> which is useful because it lets me rebase on top of work in progress.
>> Again the prospect of duplicating this other macro across 3-4 projects
>> is fairly unattractive.
>
> I've managed to get by fine with using shell scripts for stuff like this.

Do you need to upload the dlang.org website quickly and without error?

Overall I have difficulty understanding counterarguments.

Challenge: "We should eliminate some unpleasant duplication."

Response: "Make sucks. Make includes are unreliable for mythical unexplained reasons. You don't know git. You should do scripts instead."

The response just doesn't follow. Care to clarify what you think we should be doing here?


Andrei

One thought because of somethings being said, dmd-fe is being converted to D. Which means to compile dmd you need a D compiler already.
We could make all the build scripts, D instead.

It would mean one less external to D ecosystem dependency.
But it could easily be a nightmare.

"Andrei Alexandrescu"  wrote in message news:m95ds3$2vm4$1@digitalmars.com...

> I agree that make sucks etc. but duplication sucks more (and would suck regardless of what tool we use). Here we do have a simple method (include files) that is suitable for addressing that. I'd assume we'd use something similar with whichever tool we'd use.

I don't mind using include files, I mind adding more dependencies between projects.

> Great. As an aside I just ran that and got:
>
> error: unknown option `rebase'
> usage: ...

Maybe it's 'git pull --rebase upstream master', I didn't try it and I don't type it manually very often because I have a script to do this.

> Whether it's a command or a couple, that doesn't change the fact that I need to run the same command four times, and in parallel if at all possible. It's a perfect candidate for automation, and I haven't seen an argument the makefile isn't a good place for that.

I guess we have different usage patterns, which isn't really a surprise. Anyway putting rebase in the makefile is a secondary issue, that's probably not worth discussing now.

> > I've managed to get by fine with using shell scripts for stuff like this.
>
> Do you need to upload the dlang.org website quickly and without error?

Do enough people need to that it warrants being in the makefile instead of a local shell script?  Does everyone do it the same way?

> Overall I have difficulty understanding counterarguments.
>
> Challenge: "We should eliminate some unpleasant duplication."
>
> Response: "Make sucks. Make includes are unreliable for mythical unexplained reasons. You don't know git. You should do scripts instead."
>
> The response just doesn't follow. Care to clarify what you think we should be doing here?

Apologies for not being clear.

Challenge: "We should eliminate some unpleasant duplication."

Response:  "We should avoid adding more dependencies between projects at all costs." 

On 1/14/15 2:09 AM, Daniel Murphy wrote:
> "Andrei Alexandrescu"  wrote in message
> news:m95ds3$2vm4$1@digitalmars.com...
>
>> I agree that make sucks etc. but duplication sucks more (and would
>> suck regardless of what tool we use). Here we do have a simple method
>> (include files) that is suitable for addressing that. I'd assume we'd
>> use something similar with whichever tool we'd use.
>
> I don't mind using include files, I mind adding more dependencies
> between projects.

Why? Druntime depends on dmd. Phobos depends on dmd and druntime. Dlang.org and tools depend on dmd, druntime, and phobos. It's a fact of life.

>> Great. As an aside I just ran that and got:
>>
>> error: unknown option `rebase'
>> usage: ...
>
> Maybe it's 'git pull --rebase upstream master', I didn't try it and I
> don't type it manually very often because I have a script to do this.
>
>> Whether it's a command or a couple, that doesn't change the fact that
>> I need to run the same command four times, and in parallel if at all
>> possible. It's a perfect candidate for automation, and I haven't seen
>> an argument the makefile isn't a good place for that.
>
> I guess we have different usage patterns, which isn't really a surprise.
> Anyway putting rebase in the makefile is a secondary issue, that's
> probably not worth discussing now.
>
>> > I've managed to get by fine with using shell scripts for stuff like
>> > this.
>>
>> Do you need to upload the dlang.org website quickly and without error?
>
> Do enough people need to that it warrants being in the makefile instead
> of a local shell script?  Does everyone do it the same way?

Yes.

Currently I'm the site build master by default. In the near future there'll be several. I'd rather have anyone run a simple command than telling them to write their scripts.

So if next month I transfer this responsibility to Martin, I can tell: "to update dlang.org, run 'make rebase -j && make rsync -j'" as opposed to "yeah, I have some scripts somewhere, do your own or let me email them to you".

I don't really understand your position here. It's like you reject robust automation over tribal knowledge and folklore.

>> Overall I have difficulty understanding counterarguments.
>>
>> Challenge: "We should eliminate some unpleasant duplication."
>>
>> Response: "Make sucks. Make includes are unreliable for mythical
>> unexplained reasons. You don't know git. You should do scripts instead."
>>
>> The response just doesn't follow. Care to clarify what you think we
>> should be doing here?
>
> Apologies for not being clear.
>
> Challenge: "We should eliminate some unpleasant duplication."
>
> Response:  "We should avoid adding more dependencies between projects at
> all costs."

Why? As an aside, "at all costs" is suspicious in many contexts.


Andrei

"Andrei Alexandrescu"  wrote in message news:m95i3q$1t6$1@digitalmars.com...

>
> > I don't mind using include files, I mind adding more dependencies
> > between projects.
>
> Why? Druntime depends on dmd. Phobos depends on dmd and druntime. Dlang.org and tools depend on dmd, druntime, and phobos. It's a fact of life.

Druntime, phobos and dlang.org don't (currently) depend on dmd's source. 

On 2015-01-14 00:20, Andrei Alexandrescu wrote:
> Hey folks,
>
>
> Over the time a number of stuff has become quite duplicated across our
> makefiles for dmd, druntime, and phobos.
>
> These include fetching OS and model but (newer) general-purpose macros
> for e.g. rebasing repos, see
> https://github.com/D-Programming-Language/dlang.org/blob/master/posix.mak#L37.
>
>
> I think it's time to reduce clutter and duplication by migrating such
> common stuff into one common.mak file. Question is, where should that
> file sit? One candidate is tools/ but it's not really a tool. Another
> candidate is dmd/src/ because that would be the first thing anyone
> depends on.

Oh God, please no. I really, really hate that the makefiles are are looking in folders outside of the repositories.

> Ideas?

If you really want to do something like this I would recommend creating a new repository, acting like a super repository, including the other repositories as submodules. Then you can create a new makefile that builds everything. Although, I still don't want the makefiles in the existing repositories be dependent on the new repository.

-- 
/Jacob Carlborg