On Thursday, 10 April 2025 at 18:22:40 UTC, Bastiaan Veelo wrote:
> code.dlang.org has not been responding all day.
>
> https://www.isitdownrightnow.com/code.dlang.org.html
>
> -- Bastiaan.

This too: https://run.dlang.io/

Matheus.

Am 10.04.2025 um 20:22 schrieb Bastiaan Veelo:
> code.dlang.org has not been responding all day.
> 
> https://www.isitdownrightnow.com/code.dlang.org.html
> 
> -- Bastiaan.

I'm not sure what exactly causes it, but the process is in a state where it's mostly busy with garbage collection. It seems like there is maybe a self-reinforcing effect, where incoming connections that time out lead to higher GC pressure.

The other thing is that there appear to be some very aggressive crawlers that go through the whole site at maximum speed with what looks like 8 parallel requests. Maybe it's possible to get that under control through the Cloudflare frontend? Of course they are using a Safari user agent string instead of something that would identify them as bots.

Finally, the fallback server logic for dub doesn't appear to work correctly anymore - at least for me it hangs more or less indefinitely instead of falling back to codemirror.dlang.org.

I don't have a lot of time to look into this right now, but I'll see if I can do something. It would be good if someone with Cloudflare access could look into a possible mitigation there.

On Friday, 11 April 2025 at 09:06:01 UTC, Sönke Ludwig wrote:
> Am 10.04.2025 um 20:22 schrieb Bastiaan Veelo:
>> [...]
>
> I'm not sure what exactly causes it, but the process is in a state where it's mostly busy with garbage collection. It seems like there is maybe a self-reinforcing effect, where incoming connections that time out lead to higher GC pressure.
>
> [...]

https://arstechnica.com/ai/2025/03/devs-say-ai-crawlers-dominate-traffic-forcing-blocks-on-entire-countries

Am 11.04.2025 um 11:06 schrieb Sönke Ludwig:
> Am 10.04.2025 um 20:22 schrieb Bastiaan Veelo:
>> code.dlang.org has not been responding all day.
>>
>> https://www.isitdownrightnow.com/code.dlang.org.html
>>
>> -- Bastiaan.
> 
> I'm not sure what exactly causes it, but the process is in a state where it's mostly busy with garbage collection. It seems like there is maybe a self-reinforcing effect, where incoming connections that time out lead to higher GC pressure.
> 
> The other thing is that there appear to be some very aggressive crawlers that go through the whole site at maximum speed with what looks like 8 parallel requests. Maybe it's possible to get that under control through the Cloudflare frontend? Of course they are using a Safari user agent string instead of something that would identify them as bots.
> 
> Finally, the fallback server logic for dub doesn't appear to work correctly anymore - at least for me it hangs more or less indefinitely instead of falling back to codemirror.dlang.org.
> 
> I don't have a lot of time to look into this right now, but I'll see if I can do something. It would be good if someone with Cloudflare access could look into a possible mitigation there.

It's a little bit better now with one source of GC allocations temporarily eliminated. As a workaround, you can manually configure codemirror.dlang.org to take precedence by putting this in ~/.dub/settings.json:

{
	"registryUrls": ["https://codemirror.dlang.org/"]
}