https://issues.dlang.org/show_bug.cgi?id=14427

Andrei Alexandrescu <andrei@erdani.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrei@erdani.com

--- Comment #1 from Andrei Alexandrescu <andrei@erdani.com> ---
The following sequence of events has happened:

* After Vladimir got an account, he uploaded the site as user "vladimir".

* I upload files as user "d-programming".

* There is a mismatch in groups and permissions of these two usernames that makes it impossible for me to upload the site. I get errors and freezes in rsync.

* This problem has been known for a while, and Martin tried to help it: https://github.com/D-Programming-Language/dlang.org/pull/959

* This afternoon I tried to update the site with the rsync command as fixed by Martin. Things still didn't work.

* Assuming at least I didn't upload some corrupted files, I gave up and sent emails to Martin, Vladimir, and Sönke asking for help.

Looking into it further I think the issue here is d-programming is not a member of the group dlang.org. I emailed about it to Jan.

The core issue here is twofold: (a) we have no centralized web czar so
everybody believes a failure is someone else's problem; (b) our provider (Jan)
is external so we depend on him being around to get stuff done.

In the meantime I manually synchronized /phobos/. Will keep this open until the rsync issue gets fixed.

--

https://issues.dlang.org/show_bug.cgi?id=14427

Vladimir Panteleev <thecybershadow@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |thecybershadow@gmail.com

--- Comment #2 from Vladimir Panteleev <thecybershadow@gmail.com> ---
I think we need a staging area of sorts - a controlled environment which pulls strictly the website from git, performs a clean build, allows previewing, and only then uploads it to dlang.org.

This would solve a few issues:

- Removes requirement to install software to fully build dlang.org (e.g.
kindlegen)
- Removes requirement to use a Posix OS (completely building the website is
currently impossible on Windows)
- Removes issues due to environment (wrong/incompatible software versions, e.g.
rsync)
- Removes bottlenecks on Jan to resolve permission issues
- Removes upload bandwidth bottleneck for slow connections
- Fixes users having to upload the entire site because timestamps don't match
between users' local copies

I have something like this on dlang.k3.1azy.net (same server which hosts forum.dlang.org). Andrei has access already. The workflow is:

1. If there have been some significant changes in the build infrastructure
which might cause a dirty build, run: `ssh dlang@k3.1azy.net run/clean`. This
will delete all intermediary files and force clean all repositories (incl. dub
stuff).
2. Run `ssh dlang@k3.1azy.net run/update`
3. Browse to http://dlang.k3.1azy.net/ and make sure there are no glaring
problems.
4. Run `ssh dlang@k3.1azy.net run/upload`

--

https://issues.dlang.org/show_bug.cgi?id=14427

Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@dawg.eu

--- Comment #3 from Martin Nowak <code@dawg.eu> ---
The rsync issue has been resolved by now (was a permission problem).
Can we close this?

--

https://issues.dlang.org/show_bug.cgi?id=14427

Andrei Alexandrescu <andrei@erdani.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #4 from Andrei Alexandrescu <andrei@erdani.com> ---
yes thx

--

https://issues.dlang.org/show_bug.cgi?id=14427

--- Comment #5 from Andrei Alexandrescu <andrei@erdani.com> ---
@Vladimir that's pretty rad! I think what would work best is a "view deltas" capability, i.e. detect and make available for previewing the pages that have been changed since the last upload. That way instead of browsing the entire site, one needs only to glance at the pages that actually changed.

Of course, at times the changed list will be bulky - every time the navigation menu changes, a bunch of html pages change as well. But other times only one or a few pages get updated, and it's nice to be able to distinguish those.

In the future it would be even nicer to have an online diff between the live site html and the preview html.

Any chance of that being reasonably easy to implement?

--

https://issues.dlang.org/show_bug.cgi?id=14427

--- Comment #6 from Vladimir Panteleev <thecybershadow@gmail.com> ---
Yeah, that shouldn't be difficult to put together. Just cache uploaded files to a local copy and add a diff Makefile target that diffs built vs. last-uploaded files. I'll add it to my list but not sure when I'll get to this.

--

https://issues.dlang.org/show_bug.cgi?id=14427

--- Comment #7 from Vladimir Panteleev <thecybershadow@gmail.com> ---
The pinnacle of your idea is a GitHub bot that pastes diff links on each dlang.org pull request, BTW.

--

https://issues.dlang.org/show_bug.cgi?id=14427

--- Comment #8 from Andrei Alexandrescu <andrei@erdani.com> ---
(In reply to Vladimir Panteleev from comment #7)
> The pinnacle of your idea is a GitHub bot that pastes diff links on each dlang.org pull request, BTW.

Well it's diffs between built files, not sources.

--

https://issues.dlang.org/show_bug.cgi?id=14427

--- Comment #9 from Vladimir Panteleev <thecybershadow@gmail.com> ---
That's what I meant.

--

https://issues.dlang.org/show_bug.cgi?id=14427

--- Comment #10 from Andrei Alexandrescu <andrei@erdani.com> ---
@Cybershadow: before I archive this, you may want to document this as a tool if you want this to become our standard way of changing the website. Otherwise it will be forgotten.

--