December 17, 2019
On Tue, Dec 17, 2019 at 01:34:16AM +0000, bachmeier via Digitalmars-d-announce wrote: [...]
> Oh, I don't doubt that. My point was that it makes the D language project look like a small-scale open source project relying on volunteers (in this case Sonke) being generous with time and resources. What manager is going to trust a project like that with a major project? It seems unlikely that this would not be considered a major issue.

This looks like something the D Foundation ought to fund.  Buy one of the many VPS's out there to be used as the "official", Foundation-backed dub server, preferably with a hosting provider that has multiple redundant network connections and automatic backup service.  Depending on availability of funds, add a hot failover server as backup in case things go wrong with the primary server.  This ought to alleviate any concerns about downtime.

The current volunteer-provided servers can still act as mirrors for extra availability.

If a volunteer-driven opensource project like Debian can have worldwide multiple redundant FTP mirrors that almost guarantees 100% uptime, why can't we?


T

-- 
INTEL = Only half of "intelligence".
December 17, 2019
On Tuesday, 17 December 2019 at 17:34:07 UTC, H. S. Teoh wrote:
> On Tue, Dec 17, 2019 at 01:34:16AM +0000, bachmeier via Digitalmars-d-announce wrote: [...]
>> Oh, I don't doubt that. My point was that it makes the D language project look like a small-scale open source project relying on volunteers (in this case Sonke) being generous with time and resources. What manager is going to trust a project like that with a major project? It seems unlikely that this would not be considered a major issue.
>
> This looks like something the D Foundation ought to fund.
> ...
> If a volunteer-driven opensource project like Debian can have worldwide multiple redundant FTP mirrors that almost guarantees 100% uptime, why can't we?

I agree.  I sent an email to the Foundation in October offering to help fund some proper infrastructure.  Maybe the email got lost.  The offer still stands.
December 18, 2019
On Monday, 16 December 2019 at 11:04:38 UTC, Sönke Ludwig wrote:
> As you may have already noticed, the main registry server, code.dlang.org got unreachable yesterday. This was caused by an old VPS of mine getting terminated. The registry had already moved to a different server years ago, but, without me realizing it, the DNS entry still pointed to the old one, with a "temporary" HTTP proxy forwarding to the new server being set up.
>
> By now the DNS entry has been corrected, an up-to-date TLS certificate is in place, and the registry is running stable. There are still reports of people not being able to access code.dlang.org, which is apparently caused by intermediate DNS servers still reporting the old IP address and should start working during the next few hours. A temporary workaround is to specify --registry=http://31.15.67.41/ on the dub command line.
>
> Unfortunately both fallback servers have been down for a while now, so that this resulted in a total blackout. I plan to move the main registry to a powerful dedicated server in January, which will fix all memory resource related issues that sometimes show up, and could then keep the current VPS as a relatively reliable fallback server. Both together should guarantee virtually 100% uptime, although more fallback servers are of course highly desirable.
>
> In addition to that, I plan to separate the repository polling process form the web and REST frontend, as the former appears to be the main cause for failures (a GC memory leak of some kind and a possibly codegen related crash when being compiled with DMD being the two known issues, which both need further investigation).

This is still down for me, regardless of using the IP or address. I don't think it's just me either: https://stats.uptimerobot.com/6mQX4Crw2L/783838659
December 18, 2019
On 12/18/19 12:29 PM, John Colvin wrote:
> 
> This is still down for me, regardless of using the IP or address. I don't think it's just me either: https://stats.uptimerobot.com/6mQX4Crw2L/783838659

The same with me. I can't visit code.dlang.org from home pc, but able to do it from work one. (I did it in different times though)
December 18, 2019
On Wednesday, 18 December 2019 at 09:29:50 UTC, John Colvin wrote:
> This is still down for me, regardless of using the IP or address. I don't think it's just me either: https://stats.uptimerobot.com/6mQX4Crw2L/783838659

Anytime you see the metadata working you can add `--registry=https://dub.bytecraft.nl` to dub.

I am really tempted to cache the metadata as well. Although that brings up the question of how to purge it when new versions are released. (I could setup a job to import the dump every now and then, hmm).

This stuff just begs to be fixed.
December 18, 2019
On Wednesday, 18 December 2019 at 09:29:50 UTC, John Colvin wrote:
>
> This is still down for me, regardless of using the IP or address. I don't think it's just me either: https://stats.uptimerobot.com/6mQX4Crw2L/783838659

https://downforeveryoneorjustme.com/code.dlang.org

Connection timeout here.

December 18, 2019
On Wednesday, 18 December 2019 at 11:03:04 UTC, Andrea Fontana wrote:
> On Wednesday, 18 December 2019 at 09:29:50 UTC, John Colvin wrote:
>>
>> This is still down for me, regardless of using the IP or address. I don't think it's just me either: https://stats.uptimerobot.com/6mQX4Crw2L/783838659
>
> https://downforeveryoneorjustme.com/code.dlang.org
>
> Connection timeout here.

Just said. Up&Running now.
December 18, 2019
On Wednesday, 18 December 2019 at 10:18:03 UTC, Sebastiaan Koppe wrote:
> On Wednesday, 18 December 2019 at 09:29:50 UTC, John Colvin wrote:
>> This is still down for me, regardless of using the IP or address. I don't think it's just me either: https://stats.uptimerobot.com/6mQX4Crw2L/783838659
>
> Anytime you see the metadata working you can add `--registry=https://dub.bytecraft.nl` to dub.
>
> I am really tempted to cache the metadata as well. Although that brings up the question of how to purge it when new versions are released. (I could setup a job to import the dump every now and then, hmm).
>
> This stuff just begs to be fixed.

can't get metadata, so no good right now.
January 09, 2020
On 12/17/19 12:34 PM, H. S. Teoh wrote:
> This looks like something the D Foundation ought to fund.  Buy one of
> the many VPS's out there to be used as the "official", Foundation-backed
> dub server, preferably with a hosting provider that has multiple
> redundant network connections and automatic backup service.  Depending
> on availability of funds, add a hot failover server as backup in case
> things go wrong with the primary server.  This ought to alleviate any
> concerns about downtime.
> ...

I am sorry to post on an older (but < 30 days!) thread that I've just seen.

While it would be very reasonable for the Foundation to be running this infra, Sebastiaan's proposal is quite right -- eliminate the need for a server entirely.

I would add one thing to his proposal:
The metadata are already stored in github/bitbucket/gitlab/etc. -- in the form of the dubfile. All we need is a K:V lookup mapping package name to canonical git repository. This K:V store could be served perhaps more cheaply , and more reliably, than the current infra which must serve all metadata (presumably). If dub could be retooled to only query for the git repo, it could then clone and work entirely with metadata stored in the repo, minimizing dependence on a hosted server, whether 6 nines reliable or not.

If, though, the foundation takes over the current infra and hosting, please put out a call for donations -- I will donate.
January 10, 2020
On Friday, 10 January 2020 at 02:36:51 UTC, James Blachly wrote:
> I am sorry to post on an older (but < 30 days!) thread that I've just seen.
>
> While it would be very reasonable for the Foundation to be running this infra, Sebastiaan's proposal is quite right -- eliminate the need for a server entirely.
>
> I would add one thing to his proposal:
> The metadata are already stored in github/bitbucket/gitlab/etc. -- in the form of the dubfile. All we need is a K:V lookup mapping package name to canonical git repository. This K:V store could be served perhaps more cheaply , and more reliably, than the current infra which must serve all metadata (presumably). If dub could be retooled to only query for the git repo, it could then clone and work entirely with metadata stored in the repo, minimizing dependence on a hosted server, whether 6 nines reliable or not.
>
> If, though, the foundation takes over the current infra and hosting, please put out a call for donations -- I will donate.
I can't say I like the idea of requiring users to use the aforementioned or similar services and necessitating the use of git over other VCS solutions.