On Tuesday, 16 April 2019 at 05:44:22 UTC, Eugene Wissner wrote:
> I began to write again some JS and PHP and wondered first "How can I write programs without pure?". And now I would say it isn't bad at all without it.

Yeah, pure definitely isn't a must-have, but I also think it is kinda nice... as long as the little leaf functions are easy, and IMO that is where the default shines.

My example is a property getter:

pure nothrow @safe const @nogc
int a() { return a_; }

When the attribute list is longer than the function, I just think that is silly.

> But if there is no way around pure, I would say yes, pure by default would be reasonable even if I would prefer "@pure(false)/@pure(true)".

So a point I have thought of recently: we also need `null` in addition to true and false there. This acts as if the attribute was never given. Why?


@safe:

/* snip a lot of stuff */

T map(alias fn, T)(T[]) {}


That map template right now is considered safe, because of the colon above. This means it MUST also call a @safe fn. But templates are usually more flexible; it will infer the safety based on the `fn` sent to it.

With @safe, we can turn it off with its opposites...

@system T map....

(or @safe(false) same thing)

But this means it can no longer be called by @safe functions! Still not the same result as just leaving the attribute off.


Hence my proposal:

@safe(null) T map...


The null there just erases any matching attribute that was inherited from a group above, allowing the compiler to revert to the default - which for templates, is actually inference.

On Tuesday, 16 April 2019 at 03:59:38 UTC, Mike Franklin wrote:
> I think I may have found a simple migration path to @safe by default.  I'm still thinking it through, but if I can justify it, I will write a DIP.
>
> `@safe` by default is a no-brainer in my opinion, but `pure` and `immutable` by default are less obvious.  With the aforementioned potential DIP, I have an opportunity to correct purity and mutability defaults as well...but is that something we want to do?
>
> Can anyone save me some trouble and articulate why it would be bad to have `pure` and/or `immutable` by default?
>
> Thanks,
> Mike

I think pure by default is a no-brainer as well. It's not nearly as restrictive in D to mark your function as pure as it is to have a pure function in other languages, due to weak purity - that seems to be a wholly unique invention on D's part and is just waiting for an academic paper or two to be written on the subject.

All pure restricts you from doing is accessing global variables in a function scope. Ironically, PHP got this right; you need to declare inside the function which global variables you are accessing to use them (I think this might be the case in python as well, but may be misremembering). You can still throw Errors/Exceptions and modify the arguments passed to the function, and if those arguments are implicitly convertible to immutable, you get all the nice properties of a strongly-pure function. I can't see a downside, OTOH, other than needing pure(false) or @impure or something to turn it off when needed.

On 4/16/19 3:42 AM, RazvanN wrote:

> As for `pure`, as long as we don't have the equivalent of `not pure`, I don't see how it can be the default

Same arguments for immutable by default -- we don't have something that makes immutable data mutable.

I think @safe by default is a good idea. A fabulous idea. Please do it.

-Steve

On Monday, April 15, 2019 9:59:38 PM MDT Mike Franklin via Digitalmars-d wrote:
> I think I may have found a simple migration path to @safe by default.  I'm still thinking it through, but if I can justify it, I will write a DIP.
>
> `@safe` by default is a no-brainer in my opinion, but `pure` and `immutable` by default are less obvious.  With the aforementioned potential DIP, I have an opportunity to correct purity and mutability defaults as well...but is that something we want to do?
>
> Can anyone save me some trouble and articulate why it would be bad to have `pure` and/or `immutable` by default?

pure can be nice when it works, but really basic things like I/O don't work if pure is used. You can get around that for debugging with debug blocks, but if you have a bunch of code, and it turns out that you need to do something in it that isn't pure, you'd be screwed unless you go and mark a ton of code with impure (or whatever the opposite of pure would be). It's not like you can just opt-out in the middle like you can with @safe by using @trusted to use @system code.

Similarly, you can't opt-out of immutable in the middle of a complex type. It's transitive. So, having immutable be the default would cause a lot of problems in aggregate types.

Also, immutable is very much at odds with basic D concepts like ranges. With how they're currently designed, they require mutability. If immutable were the default, you'd have to slap mutable _everywhere_.

immutable and pure both work great in very functional code, but they don't work anywhere near as well in imperative code. And they're particularly bad if you're doing low level stuff. They're the sort of thing that functional languages force on you. D is multiparadigm. I would absolutely hate to have the language effectively try to force everything to be functional and make you opt-out all over the place. The fact that we have immutable and pure allow for useful things in the cases where they make a lot of sense, and you don't need mutability, and you don't need stuff like I/O. I strongly dispute the idea that your average program is written that way or that that's how most programmers are going to want to write their code.

Walter and Andrei have talked before about how the attributes are supposed to be for larger programs where they bring enough benefit to be worth the trouble. And stuff like smaller scripts shouldn't need to worry about it. I don't think that there's any question that attributes like pure and immutable do not fit well in most smaller programs, and if they were the default, any basic scripts written in D would have to opt-out of them all over the place.

@safe has some of the same problems, but it's more flexible, and it's far more reasonable that most code be @safe, since ultimately you want the top-level of all programs to be @safe, and if the higher level constructs that are typically going to be used by smaller programs are @safe, then making @safe the default might be fine. So, I don't know that it's a bad idea to make @safe the default (though Walter and Andrei have been opposed to it before, because they thought that it wasn't appropriate to require that smaller programs worry about it). But I do think that it would a terrible idea to make pure, immutable, or any other attribute that is not currently the default the default. IMHO, @safe is the only that even might make sense. The various attributes are just too restrictive for it to make sense for them to be the default.

- Jonathan M Davis

On Tuesday, 16 April 2019 at 21:33:54 UTC, Jonathan M Davis wrote:
>
> pure can be nice when it works, but really basic things like I/O don't work if pure is used. You can get around that for debugging with debug blocks, but if you have a bunch of code, and it turns out that you need to do something in it that isn't pure, you'd be screwed unless you go and mark a ton of code with impure (or whatever the opposite of pure would be). It's not like you can just opt-out in the middle like you can with @safe by using @trusted to use @system code.
>

I/O doesn't work in pure code, because I/O isn't pure. And if you want an impure statement in pure code, you can just cast purity away same as pureMalloc does it.

pure doesn't make any sense if it isn't default. Plenty of people for a valid reason don't care about the attributes. As soon as you have dependencies, you can't mark your own code as pure because you use some dependencies that may be 100% pure but aren't annotated as such. It  is possible to write pure-annotated code only if you have not-invented-here-syndrom like me and have no dependencies.

Pure should either be default or be completely removed, it is absolutely useless as it is today.

On Tuesday, 16 April 2019 at 21:33:54 UTC, Jonathan M Davis wrote:
> On Monday, April 15, 2019 9:59:38 PM MDT Mike Franklin via Digitalmars-d wrote:
>> I think I may have found a simple migration path to @safe by default.  I'm still thinking it through, but if I can justify it, I will write a DIP.
>>
>> `@safe` by default is a no-brainer in my opinion, but `pure` and `immutable` by default are less obvious.  With the aforementioned potential DIP, I have an opportunity to correct purity and mutability defaults as well...but is that something we want to do?
>>
>> Can anyone save me some trouble and articulate why it would be bad to have `pure` and/or `immutable` by default?
>
> if you have a bunch of code, and it turns out that you need to do something in it that isn't pure, you'd be screwed unless you go and mark a ton of code with impure (or whatever the opposite of pure would be). It's not like you can just opt-out in the middle like you can with @safe by using @trusted to use @system code.

That's a good point that I almost always forget when we talk about making these attributes the default. To follow the safe/trusted/system model we'd need something like pure/almostPure/impure.

Weak purity also might be able to provide some partial respite:

int pure1(int n, ref IOWrapper io) pure
{
    return pure2(n.to!string(), io); //The chain continues
}

int pure2(string s, ref IOWrapper io) pure
{
    //Can't do this because writeln is impure
    //writeln("The value of s is ", s);

    io.writeln("The value of s is ", s);

    int result;
    //Do some other work

    return result;
}

struct IOWrapper
{
    string[] writeQueue;

    void writeln(Args...)(Args args) pure
    {
        foreach (arg; args)
            writeQueue ~= args.to!string();
        writeQueue ~= '\n';
    }

    void writeAll()
    {
        foreach (msg; writeQueue)
            writeln(msg);
    }
}

And if you don't like that, you can instead accept IOWrappers by value and return them along with the result of your calculation in a Tuple. It's not pretty, but it works.

On Wednesday, April 17, 2019 1:25:24 AM MDT Eugene Wissner via Digitalmars-d wrote:
> On Tuesday, 16 April 2019 at 21:33:54 UTC, Jonathan M Davis wrote:
> > pure can be nice when it works, but really basic things like I/O don't work if pure is used. You can get around that for debugging with debug blocks, but if you have a bunch of code, and it turns out that you need to do something in it that isn't pure, you'd be screwed unless you go and mark a ton of code with impure (or whatever the opposite of pure would be). It's not like you can just opt-out in the middle like you can with @safe by using @trusted to use @system code.
>
> I/O doesn't work in pure code, because I/O isn't pure. And if you want an impure statement in pure code, you can just cast purity away same as pureMalloc does it.

Of course, I/O isn't pure, and casting with pure is almost always the wrong thing to do. It's like const. If you cast it away and mutate the value, then the const is a lie, and the assumptions that the compiler makes are wrong, which could result in wrong code. In the case of pure and I/O, casting could easily mean that I/O isn't done which the code expects to be done, because the compiler decided that a function didn't need to be called multiple times, because it was pure, and the result would be the same. Similarly, if the compiler is lied to about pure, that can really screw with immutable, because the compiler is able to make assumptions based on the fact that the code is pure and determine that some data has to be unique, because it could not have possibly be passed into the function and thus had to have been allocated within the function.

pureMalloc is one case where it's arguably okay to cast with regards to pure - but there's been a lot of discussion about that, because it's incredibly easy to screw up the compiler guarantees in the process. It works with the GC, because of the extra control that the compiler has and the fact that programmer isn't the one that has to deal with freeing the memory.

In general, if code is casting with regards to pure, it's almost certainly wrong. Exceptions to that exist, but they're extremely rare, and they must be done _very_ carefully to avoid running afoul of compiler guarantees.

> pure doesn't make any sense if it isn't default. Plenty of people for a valid reason don't care about the attributes. As soon as you have dependencies, you can't mark your own code as pure because you use some dependencies that may be 100% pure but aren't annotated as such. It  is possible to write pure-annotated code only if you have not-invented-here-syndrom like me and have no dependencies.
>
> Pure should either be default or be completely removed, it is absolutely useless as it is today.

If pure were the default, then you would have to turn it off on main on pretty much every program ever written. That should tell you something. The only programs which could avoid having main be pure would be those whose only input is the arguments to main and whose only output was the return value from main or a thrown exception that escapes main. And that's _very_ few programs.

For code to work as pure, it needs to be written with that in mind and then cannot add stuff like I/O or caching later (caching could be added in _some_ cases when the cache is within a variable, but certainly, something like Phobos' memoize wouldn't work). To try and force a program in general to be pure is to be playing the same insane game that languages like Haskell play. Sure, D's pure isn't quite the same thing (it really should be @noglobal at this point, not pure), but the effect at the call site is the same, and many of the same restrictions within a function still exist even if they're more relaxed.

I'll grant you that it can be annoying to use pure when a library you depend on doesn't use it properly, but by forcing it everywhere, the net result will be that code all over the place will have to be marked with impure (or @global) in order to work properly. And it's likely to be very common that you'd then have to go through large portions of code and add @global all over the place in order to add a piece of functionality that you need that relates to I/O or caching or some other use case where you need to interact with mutable data that isn't passed in as an argument.

pure works well when it is in smaller sections of code which were specifically written to be pure, IMHO, it's a disaster if you try to mark your entire program that way. It's just too easy to run into situations where you can't have a piece of code be pure. It's a problem similar to requiring const or immutable everywhere, only instead of restricting itself to specific pieces of data, it invades the entire call stack. It does work in some programs that are specifically written that way, but for a lot of programs it won't - especially programs that are not written in a functional manner. Trying to force an attribute like const, immutable, or pure as the default is effectively trying to force D code to be written in a functional manner instead of treating it as fully multiparadigm, and it's forcing a paradigm that is very much not in line with the languages that D grew from or with how D's standard libraries and common idioms currently work. D code is typically more functional in nature than other languages that stem from C/C++, but it's still very much an imperative, systems-level language.

- Jonathan M Davis

On Wednesday, April 17, 2019 5:39:02 PM MDT Meta via Digitalmars-d wrote:
> On Tuesday, 16 April 2019 at 21:33:54 UTC, Jonathan M Davis wrote:
> > On Monday, April 15, 2019 9:59:38 PM MDT Mike Franklin via
> >
> > Digitalmars-d wrote:
> >> I think I may have found a simple migration path to @safe by default.  I'm still thinking it through, but if I can justify it, I will write a DIP.
> >>
> >> `@safe` by default is a no-brainer in my opinion, but `pure` and `immutable` by default are less obvious.  With the aforementioned potential DIP, I have an opportunity to correct purity and mutability defaults as well...but is that something we want to do?
> >>
> >> Can anyone save me some trouble and articulate why it would be bad to have `pure` and/or `immutable` by default?
> >
> > if you have a bunch of code, and it turns out that you need to do something in it that isn't pure, you'd be screwed unless you go and mark a ton of code with impure (or whatever the opposite of pure would be). It's not like you can just opt-out in the middle like you can with @safe by using @trusted to use @system code.
>
> That's a good point that I almost always forget when we talk about making these attributes the default. To follow the safe/trusted/system model we'd need something like pure/almostPure/impure.

pure is binary in nature, and backdoors with it are seriously problematic. It would be more accurate at this point to call pure @noglobal, because the key thing is that a function is only able to access data through its function arguments. It can't access any kind of globals except through those function arguments, which means that basic stuff like I/O and caching tend to not work with it. The pure functional stuff that we can get in D then stems from the assumptions that the compiler is able to make based on what it can determine from the function arguments and the knowledge that all of the function's data comes from the function arguments. In the extreme case, that can allow for function call elision (though that's pretty rare), but more commonly, it allows for stuff like implicitly casting a function's return value to immutable when the compiler can determine that the memory for it has to be unique (which greatly simplifies constructing immutable objects). At a macro-level, all pure/@noglobal really does is make it so that when you look at a function signature, you know that it's not grabbing data from anywhere but the function arguments, and even that is frequently not very informative, because complex objects can do stuff like access global variables via stored pointers - something which reduces how informative the attribute is while not really making it easy to access globals when you actually need to. So, I'd honestly argue that having pure all over your entire code base would be far more detrimental than beneficial. It just doesn't provide great benefits at the macro level - mostly just within specific pieces of code that can actually take advantage of what the compiler can do based on pure/@noglobal.

But ultimately, what the compiler needs to know when it does anything with pure/@noglobal is that there is no way that the function can access anything except via its arguments. All of its assumptions and optimizations stem from that. So, having _any_ kind of backdoor for that like you would with an @trusted equivalent destroys the guarantee - just like having a backdoor for const that allows for mutating a const object would destroy the compiler's ability to know that const data hasn't changed and thus would make const pretty meaningless as far as compiler guarantees go.

With @safe, we could conceivably treat main as @safe and then _require_ that any code that involves @system then be verified by the programmer as @safe and marked with @trusted (it would be really annoying for anyone wanting to avoid caring about @safe, but there's no technical reason why it's a problem - just the huge risk that programmers will start slapping @trusted all over the place when they haven't actually verified the code but want the compiler to shut up so that they can get their work done). That isn't the same for pure at all. If main were marked with pure, then it musn't access global variables anywhere, and conceptually, running main multiple times with the same data would then always result in exactly the same result, because you can't pass out any other results via the function arguments to main and can't access anything not provided to main unless it was created within main (so, no I/O).

There _are_ rare cases where a piece of code that isn't technically @noglobal is actually able to follow the compiler's guarantees (e.g. std.datetime's LocalTime() is conceptually pure because it always returns exactly the same value every time it's called, but it has to create that value the first time that it's called, because it was determined unacceptable to have static constructors in Phobos - and that means casting to pure). But such functions are extremely rare and have to be done very carefully. And then there's the mess that's pureMalloc. There's been _tons_ of arguments over whether it's safe to have it, because there are real risks that it's going to be elided, and even people who are very knowledgeable about D have been having a hard time agreeing on what's going on there and what the compiler will or won't do. And all of that has to do with ensuring that something can be safely treated as pure when it isn't technically pure. It's _not_ something that your average programmer should even be considering doing. So, stuff like assumePure or an @trusted version of pure would be incredibly risky.

> Weak purity also might be able to provide some partial respite:
>
> int pure1(int n, ref IOWrapper io) pure
> {
>      return pure2(n.to!string(), io); //The chain continues
> }
>
> int pure2(string s, ref IOWrapper io) pure
> {
>      //Can't do this because writeln is impure
>      //writeln("The value of s is ", s);
>
>      io.writeln("The value of s is ", s);
>
>      int result;
>      //Do some other work
>
>      return result;
> }
>
> struct IOWrapper
> {
>      string[] writeQueue;
>
>      void writeln(Args...)(Args args) pure
>      {
>          foreach (arg; args)
>              writeQueue ~= args.to!string();
>          writeQueue ~= '\n';
>      }
>
>      void writeAll()
>      {
>          foreach (msg; writeQueue)
>              writeln(msg);
>      }
> }
>
> And if you don't like that, you can instead accept IOWrappers by value and return them along with the result of your calculation in a Tuple. It's not pretty, but it works.

You're basically getting into monads, which are a complex and difficult topic. It's how languages like Haskell are able to be pure while still having I/O. It's technically possible, but it's a royal pain, and most people end up having a very hard time understanding it. It's also not how much code is written unless programmers are forced to write that way. We do have something similar with output ranges, which allows for specific pieces of code to be written in a way that could involve I/O without explicitly involving I/O, but such code is still frequently not actually pure, because the output range will often output the data as it goes along rather than building a string to output at the end. Templates deal with that though, so the code using the output range is usually able be pure if the output range itself is pure.

D's pure is a great tool in sections of code that are specifically written for it in mind, and we do have some language features and idioms that make it easier for code to be pure that might not be otherwise, but trying to make entire programs pure really doesn't make sense - not for what is supposed to be a multiparadigm language. It's just way too restrictive. So, improving the tools for it and doing a better job of making sure that stuff is pure when it can be would be useful, but IMHO, trying to make it the default would be far too dogmatic and a huge mistake.

- Jonathan M Davis

On Thursday, 18 April 2019 at 08:18:05 UTC, Jonathan M Davis wrote:
> On Wednesday, April 17, 2019 1:25:24 AM MDT Eugene Wissner via Digitalmars-d wrote:
>> On Tuesday, 16 April 2019 at 21:33:54 UTC, Jonathan M Davis wrote:
>> > pure can be nice when it works, but really basic things like I/O don't work if pure is used. You can get around that for debugging with debug blocks, but if you have a bunch of code, and it turns out that you need to do something in it that isn't pure, you'd be screwed unless you go and mark a ton of code with impure (or whatever the opposite of pure would be). It's not like you can just opt-out in the middle like you can with @safe by using @trusted to use @system code.
>>
>> I/O doesn't work in pure code, because I/O isn't pure. And if you want an impure statement in pure code, you can just cast purity away same as pureMalloc does it.
>
> Of course, I/O isn't pure, and casting with pure is almost always the wrong thing to do. It's like const. If you cast it away and mutate the value, then the const is a lie, and the assumptions that the compiler makes are wrong, which could result in wrong code. In the case of pure and I/O, casting could easily mean that I/O isn't done which the code expects to be done, because the compiler decided that a function didn't need to be called multiple times, because it was pure, and the result would be the same. Similarly, if the compiler is lied to about pure, that can really screw with immutable, because the compiler is able to make assumptions based on the fact that the code is pure and determine that some data has to be unique, because it could not have possibly be passed into the function and thus had to have been allocated within the function.
>
> pureMalloc is one case where it's arguably okay to cast with regards to pure - but there's been a lot of discussion about that, because it's incredibly easy to screw up the compiler guarantees in the process. It works with the GC, because of the extra control that the compiler has and the fact that programmer isn't the one that has to deal with freeing the memory.
>
> In general, if code is casting with regards to pure, it's almost certainly wrong. Exceptions to that exist, but they're extremely rare, and they must be done _very_ carefully to avoid running afoul of compiler guarantees.
>
>> pure doesn't make any sense if it isn't default. Plenty of people for a valid reason don't care about the attributes. As soon as you have dependencies, you can't mark your own code as pure because you use some dependencies that may be 100% pure but aren't annotated as such. It  is possible to write pure-annotated code only if you have not-invented-here-syndrom like me and have no dependencies.
>>
>> Pure should either be default or be completely removed, it is absolutely useless as it is today.
>
> If pure were the default, then you would have to turn it off on main on pretty much every program ever written. That should tell you something. The only programs which could avoid having main be pure would be those whose only input is the arguments to main and whose only output was the return value from main or a thrown exception that escapes main. And that's _very_ few programs.
>
> For code to work as pure, it needs to be written with that in mind and then cannot add stuff like I/O or caching later (caching could be added in _some_ cases when the cache is within a variable, but certainly, something like Phobos' memoize wouldn't work). To try and force a program in general to be pure is to be playing the same insane game that languages like Haskell play. Sure, D's pure isn't quite the same thing (it really should be @noglobal at this point, not pure), but the effect at the call site is the same, and many of the same restrictions within a function still exist even if they're more relaxed.
>
> I'll grant you that it can be annoying to use pure when a library you depend on doesn't use it properly, but by forcing it everywhere, the net result will be that code all over the place will have to be marked with impure (or @global) in order to work properly. And it's likely to be very common that you'd then have to go through large portions of code and add @global all over the place in order to add a piece of functionality that you need that relates to I/O or caching or some other use case where you need to interact with mutable data that isn't passed in as an argument.
>
> pure works well when it is in smaller sections of code which were specifically written to be pure, IMHO, it's a disaster if you try to mark your entire program that way. It's just too easy to run into situations where you can't have a piece of code be pure. It's a problem similar to requiring const or immutable everywhere, only instead of restricting itself to specific pieces of data, it invades the entire call stack. It does work in some programs that are specifically written that way, but for a lot of programs it won't - especially programs that are not written in a functional manner. Trying to force an attribute like const, immutable, or pure as the default is effectively trying to force D code to be written in a functional manner instead of treating it as fully multiparadigm, and it's forcing a paradigm that is very much not in line with the languages that D grew from or with how D's standard libraries and common idioms currently work. D code is typically more functional in nature than other languages that stem from C/C++, but it's still very much an imperative, systems-level language.
>
> - Jonathan M Davis

As it is now, pure is not like immutable or const, it is like @safe, it is just an annotation that tells, whether the code does something you don't expect. Nobody knows if pure will mean one day more or something different, probably not, because even official libraries break purity.

I always have to annotate the main function as impure. So what? It is still better to annotate a few functions as impure than the most code as pure.

Casting away purity was meant by me only for debugging and similar things, not as permanent solution. Even Haskell provides perfomUnsafeIO for that. And if a pure function becomes one day impure, the code needs refactoring anyway. But I see, there is no way for someone who doesn't care about purity to disable it. Maybe compiler should provide some switch that disables purity, nogc and so forth for the code and all its dependencies.

I actually agree with you that D is an imperative programming language and as I said above I think pure brings more harm than good and I could live without it at all. It is just there, it doesn't work as is and of course nobody knows, how it is supposed to work.

On Thursday, 18 April 2019 at 08:53:37 UTC, Jonathan M Davis wrote:
> There _are_ rare cases where a piece of code that isn't technically @noglobal is actually able to follow the compiler's guarantees (e.g. std.datetime's LocalTime() is conceptually pure because it always returns exactly the same value every time it's called, but it has to create that value the first time that it's called, because it was determined unacceptable to have static constructors in Phobos - and that means casting to pure).

It isn't conceptually pure, it depends on the state of the word after compilation, it is just impure.