On Thursday, 15 May 2014 at 00:48:52 UTC, Walter Bright wrote:
> On 5/14/2014 5:44 PM, Brian Schott wrote:
>> Can we say that Mallocator failures are not recoverable?
>
> malloc itself does not have that property. But you could design a wrapper for it that did.

I'm concerned specifically with this wrapper: https://github.com/andralex/phobos/blob/allocator/std/allocator.d#L773

We need to make these functions pure if they are going to be usable.

Removing the stdlib import and adding

private extern (C)
{
    void* malloc(size_t) pure nothrow @trusted;
    void free(void*) pure nothrow @trusted;
    void* realloc(void*, size_t) pure nothrow @trusted;
}

as well as a throwing OutOfMemoryError if they fail should be sufficient, correct?

On Thursday, 15 May 2014 at 00:00:37 UTC, Walter Bright wrote:
>
> Because GC failures are not recoverable, so the pure allocation cannot fail.

Is this intentionally the case? I always thought you could handle it with onOutOfMemoryError if you knew that your program could handle running out of memory and free some memory (i.e., cached data) to recover. I've never actually had a use for it myself, so I'm just basing this off of newsgroup discussions I remember (possibly incorrectly) reading.

On 5/14/14, 5:44 PM, Brian Schott wrote:
> On Thursday, 15 May 2014 at 00:00:37 UTC, Walter Bright wrote:
>> On 5/14/2014 3:42 PM, Brian Schott wrote:
>>> If malloc can never be considered pure, even when hidden behind an
>>> allocator,
>>
>> It cannot be pure as long as it can fail.
>>
>>> why can it be considered pure when hidden behind the GC?
>>
>> Because GC failures are not recoverable, so the pure allocation cannot
>> fail.
>
> Can we say that Mallocator failures are not recoverable?

FWIW std.allocator allows constructing a bunch of small-business allocators for which failure to allocate is entirely legit. The methods of such allocator may be weakly pure.

An allocator that ultimately falls back to GCAllocator and "never fails" should be pure - better yet, inferred as such.


Andrei

On Wednesday, 14 May 2014 at 22:50:10 UTC, w0rp wrote:
> I think even C malloc should be considered pure. True, it affects global state by allocating memory, but it never changes existing values, it just allows for new values. free is pure because it isn't side-effecting, it deallocates what you give it. That's just my perspective on it though, others might have other views on it.

`free` is not pure, because if you have a reference to that memory that reference is no longer valid.

On Thursday, 15 May 2014 at 01:33:36 UTC, Idan Arye wrote:
> On Wednesday, 14 May 2014 at 22:50:10 UTC, w0rp wrote:
>> I think even C malloc should be considered pure. True, it affects global state by allocating memory, but it never changes existing values, it just allows for new values. free is pure because it isn't side-effecting, it deallocates what you give it. That's just my perspective on it though, others might have other views on it.
>
> `free` is not pure, because if you have a reference to that memory that reference is no longer valid.

It is weakly pure, i.e., it can only affect the world through the parameters given to it (and the state of the computer's memory... but we should ignore that).

On Wed, 14 May 2014 17:00:39 -0700
Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> On 5/14/2014 3:42 PM, Brian Schott wrote:
> > If malloc can never be considered pure, even when hidden behind an allocator,
>
> It cannot be pure as long as it can fail.
>
> > why can it be considered pure when hidden behind the GC?
>
> Because GC failures are not recoverable, so the pure allocation cannot fail.

Then we should create a wrapper for malloc which throws a MemoryError when malloc fails. Then malloc failures would be the same as GC failures.

- Jonathan M Davis

On 5/14/14, 6:11 PM, Brian Schott wrote:
> On Thursday, 15 May 2014 at 00:48:52 UTC, Walter Bright wrote:
>> On 5/14/2014 5:44 PM, Brian Schott wrote:
>>> Can we say that Mallocator failures are not recoverable?
>>
>> malloc itself does not have that property. But you could design a
>> wrapper for it that did.
>
> I'm concerned specifically with this wrapper:
> https://github.com/andralex/phobos/blob/allocator/std/allocator.d#L773
>
> We need to make these functions pure if they are going to be usable.
>
> Removing the stdlib import and adding
>
> private extern (C)
> {
>      void* malloc(size_t) pure nothrow @trusted;
>      void free(void*) pure nothrow @trusted;
>      void* realloc(void*, size_t) pure nothrow @trusted;
> }
>
> as well as a throwing OutOfMemoryError if they fail should be
> sufficient, correct?

I think so. A more cautious solution would be to define PureMallocator in addition to Mallocator.

Andrei

For my 2 cents,

The more D allows 'pure' functions to diverge from functional purity the less relevant the flag is for compiler optimisations.

I think purity should be defined by what the compiler is allowed to do to a function.  The results of all pure functions should be cacheable. If the compiler can prove that the arguments to a pure function are constant then the function can be run at compile time to get the result.

A more advanced compiler should be able to do a lot of optimisation based on the fact that a function is marked pure.  A compiler can often prove arguments are constant when a human can't show the same easily. Other optimisation parses often expose constants that may be arguments to pure functions.

For example, a foreach loop over a small array could be unwound to remove the overhead of branching.  The iterator then becomes a constant in each segment of the former loop.  If the iterator is the argument to a pure function, the entire function call could be eliminated and replaced with the result.

By the same reasoning cacheability is important.  A pure function might be called within a loop with a parameter that is not altered during the loop.  If the compiler knows the function is pure, it can perform the calculation before the loop and simply reuse the cached result for each iteration.

New and malloc are not really pure.  They return a different result each call, and the state of the PC is changed.   However, if the programmer knows his function behaves in a pure way, it would be nice if a programmer could tell the compiler to go ahead and optimise away the function call anyway.  Some kind of “trusted pure” tag would be required I guess.

A 'trusted pure' function would be free to allocate memory as it wishes, providing it is freed (or left for collection), and the pointer does not escape the function.  From the point of view of the calling software the function is functionally pure.

My point is that if we are going to allow relaxed purity in D, then the definition should be angled towards a performance advantage.  Naturally there are quite a number of other advantages of creating pure functions besides performance.  If we define what compiler is allowed to do to a pure function we make reasoning about pure functions simple.  It even opens the idea of 'trusted pure' functions that compiler can't prove are pure, because they may do something like call a C routine.  From a compilers point of view they could be considered pure because the programmer says the compiler is free to eliminate the function call if it knows the result.


On 15/05/2014 8:42 AM, Brian Schott via Digitalmars-d wrote:
> What is the plan for the "pure"-ity of memory management?
>
> Right now the "new" operator is considered to be pure even though it is
> not, but related functinos like malloc, GC.addRange, GC.removeRange, and
> others are not.
>
> This prevents large portions of std.allocator from being pure. This then
> prevents any containers library built on std.allocator from being pure,
> which does the same for any funcions and data structures written using
> those containers.
>
> If malloc can never be considered pure, even when hidden behind an
> allocator, why can it be considered pure when hidden behind the GC?
>

On Thu, 15 May 2014 01:33:34 +0000
Idan Arye via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> On Wednesday, 14 May 2014 at 22:50:10 UTC, w0rp wrote:
> > I think even C malloc should be considered pure. True, it affects global state by allocating memory, but it never changes existing values, it just allows for new values. free is pure because it isn't side-effecting, it deallocates what you give it. That's just my perspective on it though, others might have other views on it.
>
> `free` is not pure, because if you have a reference to that memory that reference is no longer valid.

But does that really matter from the perspective of pure? That's really more of an @safety issue. There might be some way that that violates purtiy, but I can't think of one at the moment.

free can't be strongly pure, because it's arguments couldn't be immutable (or even const) without violating the type system, but I _think_ that it's fine for free to be weakly pure. It's quite possible that I'm missing something though.

- Jonathan M Davis

On Thu, 15 May 2014 01:25:52 +0000
Kapps via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> On Thursday, 15 May 2014 at 00:00:37 UTC, Walter Bright wrote:
> >
> > Because GC failures are not recoverable, so the pure allocation cannot fail.
>
> Is this intentionally the case? I always thought you could handle it with onOutOfMemoryError if you knew that your program could handle running out of memory and free some memory (i.e., cached data) to recover. I've never actually had a use for it myself, so I'm just basing this off of newsgroup discussions I remember (possibly incorrectly) reading.

It's intended that all Errors be considered unrecoverable. You can catch them when necessary to try and do cleanup, but that's fairly risky, and you should probably only do it when you're very sure of what's going on (which generally means that the Error was thrown very close to where you're catching it). There's no guarantee that automatic cleanup occurs when Errors are thrown (unlike with Exceptions), so when an Error is thrown, the program tends to be in a weird state on top of the weird state that caused the Error to be thrown in the first place. In theory, you could catch an OutOfMemoryError very close to where it was thrown and deal with it, but that's really not what was intended.

- Jonathan M Davis