On 02.06.2017 15:24, Adam D. Ruppe wrote:
> On Friday, 2 June 2017 at 12:33:17 UTC, Steven Schveighoffer wrote:
>> But the perception is going to be that D web frameworks are too fragile -- one miswritten handler, and your whole webserver dies.
> 
> Correction: "vibe.d frameworks" are fragile. This isn't D specific - my cgi.d is resilient to this (and more) and has been since 2008 since it uses a process pool. Simple solution that works very well.
> 
> Might not handle 10,000 concurrent connections... but you very rarely actually have to.

I'm not convinced that public perception is sensitive to such details. ;)

On 05/31/2017 09:04 AM, Steven Schveighoffer wrote:
> I have discovered an annoyance in using vibe.d instead of another web framework. Simple errors in indexing crash the entire application.
> 
> For example:
> 
> int[3] arr;
> arr[3] = 5;
> 
> Compare this to, let's say, a malformed unicode string (exception), malformed JSON data (exception), file not found (exception), etc.
> 
> Technically this is a programming error, and a bug. But memory hasn't actually been corrupted. The system properly stopped me from corrupting memory. But my reward is that even though this fiber threw an Error, and I get an error message in the log showing me the bug, the web server itself is now out of commission. No other pages can be served. This is like the equivalent of having a guard rail on a road not only stop you from going off the cliff but proactively disable your car afterwards to prevent you from more harm.
> 
> This seems like a large penalty for "almost" corrupting memory. No other web framework I've used crashes the entire web server for such a simple programming error. And vibe.d has no choice. There is no guarantee the stack is properly unwound, so it has to accept the characterization of this is a program-ending error by the D runtime.
> 
> I am considering writing a set of array wrappers that throw exceptions when trying to access out of bounds elements. This comes with its own set of problems, but at least the web server should continue to run.
> 
> What are your thoughts? Have you run into this? If so, how did you solve it?

This is a meaningful concern. People use threads instead of processes for serving requests for improving speed and footprint. Threads hardly communicate with one another so they are virtually independent. D already has good mechanisms for isolating threads effectively (the shared qualifier, @safe) so an argument could be made that bringing down the entire process because a thread has had a problem is disproportionate response.

This was a concern about using D on the server for Facebook as well.

Of course, it may be the case that that thread's failure reflects a memory corruption that affects all others, so one could reduce the matter to this and argue the entire process should be brought down. But of course things are never as simple as we'd like them to be.

Array bound accesses should be easy to intercept and have them just kill the current thread. Vibe may want to do that, or allow their users to. The more difficult matter is null pointer dereferences. I recall there has been work in druntime to convert memory violations into thrown Errors at least on Linux. You may want to look into that.

It seems to me we'd do good to improve matters on this front.


Thanks,

Andrei

On Friday, 2 June 2017 at 10:37:09 UTC, aberba wrote:
> On Friday, 2 June 2017 at 02:11:34 UTC, Laeeth Isharc wrote:
>> On Wednesday, 31 May 2017 at 13:34:25 UTC, Steven Schveighoffer wrote:
>>> [...]
>>
>> Hi Steve.
>>
>> Had similar problems early on.  We used supervisord to automatically keep a pool of vibed applications running and put nginx in front as a load balancer. User session info stored in redis.  And a separate process for data communicating with web server over nanomsg.  Zeromq is more mature but I found sometimes socket could get into an inconsistent state if servers crashed midway, and nanomsg doesn't have this problem. So data update either succeeds or fails but no corruption if Web server crashes.
>>
>> Maybe better ways but it seems to be okay for us.
>>
>>
>> Laeeth
>
> How does that setup affect response time? Do you cache large query results in redis?

Our world is very different from web world.  Very few users but incredibly high value.  If we have twenty users then for most things that's a lot.  We don't cache query results as it's fast enough and the data retrieval bit is not where the bottleneck is.


Laeeth

On Thursday, 1 June 2017 at 14:04:40 UTC, Guillaume Piolat wrote:
> Solved by auto-saving, _before_ the crash

That only works for simple applications.

I'm using D to write an RSS reader.

As I understand it, the compiler does not guarantee correct cleanup when an Error is thrown through a nothrow function. Furthermore, it doesn't guarantee that an Error can be caught (though it happens to allow it today).

Do I need to modify the compiler to ignore nothrow and treat all throwables the same so it doesn't corrupt application state when I recover from an Error? Fork vibe.d and every other library I use to remove nothrow? I can't really justify that. My RSS reader is a side project.

Do I accept that writing my code in D will result in a program that will crash unrecoverably in cases where using C# would just show a 503 and log an error to disk? That's a disservice to my users.

Do I increase development time to make up for D's problems in this area, pipe requests through a proxy that will convert crashes to 503 errors, split things out into as many processes as possible? At that point, I'll just use C#. It's less pleasant in a wide variety of ways, but I'd save a lot of work and complexity.

And this practice is to make code marginally more efficient in uncommon cases, because people are conflating "this is a problem that a competent programmer should have been able to avoid" (yeah, okay, I was incautious, we can move on) with "this dependency of yours, probably the runtime, is in an invalid state", and nothrow optimizations assume the latter only.

And it's exacerbated because bounds checking is seen as an option to help with debugging instead of a safety feature to be used in production. Because removing bounds checking is seen as a sensible thing to do instead of a highly unsafe optimization.

It's exacerbated because Walter is in a mindset of writing mission-critical applications where any detectable bug means you need to restart the program. Honestly, if I were writing flight control systems for Airbus, I could modify druntime to raise SIGABRT or call exit(3) when you try to throw an Error. It would be easy, and it would be worthwhile. If you really need cleanup, atexit(3) is available.

On Friday, 2 June 2017 at 23:23:45 UTC, nohbdy wrote:

> It's exacerbated because Walter is in a mindset of writing mission-critical applications where any detectable bug means you need to restart the program. Honestly, if I were writing flight control systems for Airbus, I could modify druntime to raise SIGABRT or call exit(3) when you try to throw an Error. It would be easy, and it would be worthwhile. If you really need cleanup, atexit(3) is available.

The worst thing happened in programming in the last 30 years is just that less and less programmers are adopting Walter mindset...

I'm really really puzzled by why this topic pops up so often...


/Paolo

On Saturday, 3 June 2017 at 06:55:35 UTC, Paolo Invernizzi wrote:
> The worst thing happened in programming in the last 30 years is just that less and less programmers are adopting Walter mindset...

Really?

On the contrary. What is being adopted is robustness and program verification. More and more.

Assuming that a program shouldn't be able to flush its buffers out of some flawed reasoning about program correctness does not support your argument at all.

Even if your program is fully based on event-sourcing and can deal with an immediate shutdown YOU STILL WANT TO FLUSH YOUR EVENT-BUFFERS TO DISK!

The argument Walter is follwing is flawed. If a failed assert means you should not be able to flush to disk, then it also means that you should undo everything the program has ever written to disk.

The incorrect program state could have occured at install.

You have to reason about these things in probabilistic terms and not in absolutes.

On 03.06.2017 08:55, Paolo Invernizzi wrote:
> On Friday, 2 June 2017 at 23:23:45 UTC, nohbdy wrote:
> 
>> It's exacerbated because Walter is in a mindset of writing mission-critical applications where any detectable bug means you need to restart the program. Honestly, if I were writing flight control systems for Airbus, I could modify druntime to raise SIGABRT or call exit(3) when you try to throw an Error. It would be easy, and it would be worthwhile. If you really need cleanup, atexit(3) is available.
> 
> The worst thing happened in programming in the last 30 years is just that less and less programmers are adopting Walter mindset...
> 
> I'm really really puzzled by why this topic pops up so often...
> 
> 
> /Paolo

I don't get why you would /restart/ mission-critical software that has been shown to be buggy. What you need to do instead: Have a few more development teams that create independent implementations of your service. (Completely from scratch, as the available libraries were not developed to the necessary standard.) All of them should run on different hardware produced in different factories by different companies. Furthermore, you need to hire a team of testers and software verification experts vastly exceeding the team of developers in magnitude, etc.

On Saturday, 3 June 2017 at 09:48:05 UTC, Timon Gehr wrote:
> I don't get why you would /restart/ mission-critical software that has been shown to be buggy. What you need to do instead: Have a few more development teams that create independent implementations of your service. (Completely from scratch, as the available libraries were not developed to the necessary standard.) All of them should run on different hardware produced in different factories by different companies.
> Furthermore, you need to hire a team of testers and software verification experts vastly exceeding the team of developers in magnitude, etc.

Yes, mission critical software such as flight control are (and should) be proven correct. There is modelling software for this very narrow field that will generate correct code.

Or as you say, you can implement 3 different versions, running on 3 different hardware platforms and shut down the 1 that disagrees with the others.

But you still have to think in probabilistic terms, because there could be problems with sensors, actuators, human errors etc etc etc..

On Saturday, 3 June 2017 at 07:51:55 UTC, Ola Fosheim Grøstad wrote:
> On Saturday, 3 June 2017 at 06:55:35 UTC, Paolo Invernizzi wrote:
>> The worst thing happened in programming in the last 30 years is just that less and less programmers are adopting Walter mindset...
>
> Really?
>
> On the contrary. What is being adopted is robustness and program verification. More and more.

It doesn't seems to me that the trends to try to handle somehow, that something, somewhere, who knows when, has gone wild it's coherent with the term "robustness".

And the fact that the "nice tries" are done at runtime, in production, is the opposite of what I'm thinking is program verification.

> Assuming that a program shouldn't be able to flush its buffers out of some flawed reasoning about program correctness does not support your argument at all.
>
> Even if your program is fully based on event-sourcing and can deal with an immediate shutdown YOU STILL WANT TO FLUSH YOUR EVENT-BUFFERS TO DISK!

There's a fundamental difference between trying to flush logs and trying to report what's happened, with the scope of gaining more information of what happened, and trying to "automagically" handle the fact that there's an error in the implementation, or in the logic, or in the HW.

> The argument Walter is follwing is flawed. If a failed assert means you should not be able to flush to disk, then it also means that you should undo everything the program has ever written to disk.
>
> The incorrect program state could have occured at install.

The argument Walter is following is not flawed: it's a really beautiful pragmatic balance of risks and engineering way of developing software, IMHO.

> You have to reason about these things in probabilistic terms and not in absolutes.

I'm trying to exactly do that, I like to think myself as a very pragmatic person...

/Paolo