On Wednesday, 13 July 2016 at 11:39:11 UTC, Lodovico Giaretta wrote:
> On Wednesday, 13 July 2016 at 00:57:38 UTC, Adam Sansier wrote:
>> [...]
>
> You shall use a static per-thread Region allocator[1] backed by Mallocator[2].
> Then you just make[3] exceptions inside it and throw them.
> So you can allocate and chain exceptions until you end the memory established on creation.
> Whenever you don't need the exception chain anymore (i.e.: you catched them and program is back in "normal" mode, you just reset the region allocator, so you have all of your memory again, for the next exception chain).
>
> [1] https://dlang.org/phobos/std_experimental_allocator_building_blocks_region.html
> [2] https://dlang.org/phobos/std_experimental_allocator_mallocator.html
> [3] https://dlang.org/phobos/std_experimental_allocator.html

Am I going to have to do all this myself or is it already done for me somewhere?

I'm writing currently a library, that is 100% @nogc but not nothrow, and I slowly begin to believe that I should publish it already, though it isn't ready yet. At least as example.
std.experimental.allocator doesn't work nicely with @nogc. for example dispose calls destroy, that isn't @nogc.
I wrote a primitive native allocator for linux and some help functions, that replaces phobos functions till they aren't @nogc-ready. For example for throwing the exceptions:

void raise(T : Throwable, A...)(Allocator allocator, auto ref A args)
{
	auto e = make!T(allocator, args);
	throw e;
}


and you can throw then with raise!Exception("bla-bla")


On Wednesday, 13 July 2016 at 16:13:21 UTC, Adam Sansier wrote:
> On Wednesday, 13 July 2016 at 11:39:11 UTC, Lodovico Giaretta wrote:
>> On Wednesday, 13 July 2016 at 00:57:38 UTC, Adam Sansier wrote:
>>> [...]
>>
>> You shall use a static per-thread Region allocator[1] backed by Mallocator[2].
>> Then you just make[3] exceptions inside it and throw them.
>> So you can allocate and chain exceptions until you end the memory established on creation.
>> Whenever you don't need the exception chain anymore (i.e.: you catched them and program is back in "normal" mode, you just reset the region allocator, so you have all of your memory again, for the next exception chain).
>>
>> [1] https://dlang.org/phobos/std_experimental_allocator_building_blocks_region.html
>> [2] https://dlang.org/phobos/std_experimental_allocator_mallocator.html
>> [3] https://dlang.org/phobos/std_experimental_allocator.html
>
> Am I going to have to do all this myself or is it already done for me somewhere?

On Wednesday, 13 July 2016 at 16:13:21 UTC, Adam Sansier wrote:
> On Wednesday, 13 July 2016 at 11:39:11 UTC, Lodovico Giaretta wrote:
>> On Wednesday, 13 July 2016 at 00:57:38 UTC, Adam Sansier wrote:
>>> [...]
>>
>> You shall use a static per-thread Region allocator[1] backed by Mallocator[2].
>> Then you just make[3] exceptions inside it and throw them.
>> So you can allocate and chain exceptions until you end the memory established on creation.
>> Whenever you don't need the exception chain anymore (i.e.: you catched them and program is back in "normal" mode, you just reset the region allocator, so you have all of your memory again, for the next exception chain).
>>
>> [1] https://dlang.org/phobos/std_experimental_allocator_building_blocks_region.html
>> [2] https://dlang.org/phobos/std_experimental_allocator_mallocator.html
>> [3] https://dlang.org/phobos/std_experimental_allocator.html
>
> Am I going to have to do all this myself or is it already done for me somewhere?

It's actually quite easy. Here's the code (untested):

================================================================

import std.experimental.allocator.building_blocks.region;
import std.experimental.allocator.mallocator;
import std.experimental.allocator;


Region(shared(Mallocator)) exception_allocator;
enum EXCEPTION_MEM_SIZE = 256*1024;
static this()
{
    exception_allocator = typeof(exception_allocator)(EXCEPTION_MEM_SIZE);
}

================================================================

And here is an usage example (untested, too):

================================================================

void throwingFunction()
{
    // try to do something, but fail
    throw exception_allocator.make!Exception("my wonderful error message");
}

void throwingThrowingFunction()
{
    try
    {
        // try to call function, which fails
        throwingFunction;
    }
    catch (Exception exc)
    {
        // try to recover from failure, but generate other exception (just to show chaining)
        throw exception_allocator.make!Exception("I love exception chaining");
    }
}

void main()
{
    try
    {
        // try to call function, which fails
        throwingThrowingFunction;
    }
    catch (Exception exc)
    {
        // recover from failure, then deallocate the exceptions no longer needed
        exception_allocator.deallocateAll;
    }
}
================================================================

On Wednesday, 13 July 2016 at 16:28:23 UTC, Lodovico Giaretta wrote:
> On Wednesday, 13 July 2016 at 16:13:21 UTC, Adam Sansier wrote:
>> On Wednesday, 13 July 2016 at 11:39:11 UTC, Lodovico Giaretta wrote:
>>> On Wednesday, 13 July 2016 at 00:57:38 UTC, Adam Sansier wrote:
>>>> [...]
>>>
>>> You shall use a static per-thread Region allocator[1] backed by Mallocator[2].
>>> Then you just make[3] exceptions inside it and throw them.
>>> So you can allocate and chain exceptions until you end the memory established on creation.
>>> Whenever you don't need the exception chain anymore (i.e.: you catched them and program is back in "normal" mode, you just reset the region allocator, so you have all of your memory again, for the next exception chain).
>>>
>>> [1] https://dlang.org/phobos/std_experimental_allocator_building_blocks_region.html
>>> [2] https://dlang.org/phobos/std_experimental_allocator_mallocator.html
>>> [3] https://dlang.org/phobos/std_experimental_allocator.html
>>
>> Am I going to have to do all this myself or is it already done for me somewhere?
>
> It's actually quite easy. Here's the code (untested):
>
> ================================================================
>
> import std.experimental.allocator.building_blocks.region;
> import std.experimental.allocator.mallocator;
> import std.experimental.allocator;
>
>
> Region(shared(Mallocator)) exception_allocator;
> enum EXCEPTION_MEM_SIZE = 256*1024;
> static this()
> {
>     exception_allocator = typeof(exception_allocator)(EXCEPTION_MEM_SIZE);
> }
>
> ================================================================
>
> And here is an usage example (untested, too):
>
> ================================================================
>
> void throwingFunction()
> {
>     // try to do something, but fail
>     throw exception_allocator.make!Exception("my wonderful error message");
> }
>
> void throwingThrowingFunction()
> {
>     try
>     {
>         // try to call function, which fails
>         throwingFunction;
>     }
>     catch (Exception exc)
>     {
>         // try to recover from failure, but generate other exception (just to show chaining)
>         throw exception_allocator.make!Exception("I love exception chaining");
>     }
> }
>
> void main()
> {
>     try
>     {
>         // try to call function, which fails
>         throwingThrowingFunction;
>     }
>     catch (Exception exc)
>     {
>         // recover from failure, then deallocate the exceptions no longer needed
>         exception_allocator.deallocateAll;
>     }
> }
> ================================================================

Doesn't work. identifier expected on shared.

What's the difference of simply using malloc to allocate the memory and creating the exceptions their? Seems like a long and winded way go about it or is there some benefit to using the experimental allocators?

On Wednesday, 13 July 2016 at 20:44:52 UTC, Adam Sansier wrote:
> On Wednesday, 13 July 2016 at 16:28:23 UTC, Lodovico Giaretta wrote:
>>
>> It's actually quite easy. Here's the code (untested):
>>
>> ================================================================
>>
>> import std.experimental.allocator.building_blocks.region;
>> import std.experimental.allocator.mallocator;
>> import std.experimental.allocator;
>>
>>
>> Region(shared(Mallocator)) exception_allocator;
>> enum EXCEPTION_MEM_SIZE = 256*1024;
>> static this()
>> {
>>     exception_allocator = typeof(exception_allocator)(EXCEPTION_MEM_SIZE);
>> }
>>
>> ================================================================
>>
>> And here is an usage example (untested, too):
>>
>> ================================================================
>>
>> void throwingFunction()
>> {
>>     // try to do something, but fail
>>     throw exception_allocator.make!Exception("my wonderful error message");
>> }
>>
>> void throwingThrowingFunction()
>> {
>>     try
>>     {
>>         // try to call function, which fails
>>         throwingFunction;
>>     }
>>     catch (Exception exc)
>>     {
>>         // try to recover from failure, but generate other exception (just to show chaining)
>>         throw exception_allocator.make!Exception("I love exception chaining");
>>     }
>> }
>>
>> void main()
>> {
>>     try
>>     {
>>         // try to call function, which fails
>>         throwingThrowingFunction;
>>     }
>>     catch (Exception exc)
>>     {
>>         // recover from failure, then deallocate the exceptions no longer needed
>>         exception_allocator.deallocateAll;
>>     }
>> }
>> ================================================================
>
> Doesn't work. identifier expected on shared.
>
> What's the difference of simply using malloc to allocate the memory and creating the exceptions their? Seems like a long and winded way go about it or is there some benefit to using the experimental allocators?

`Region(shared(Mallocator))` shall be `Region!(shared Mallocator)` (again, I'm just looking at the code, didn't test it).

The advantages over a simple malloc are:
1) You can change between GC allocation, malloc, mmap and other allocators by changing a single line, instead of changing every throw;
2) you can use very fast allocators, based on your needs; this example uses the Region allocator, which is way faster than a call to malloc;
3) the Region allocator has the added value of working even if there's no more memory available (because it preallocated it).

In general, the allocators library provides facilities that may seem overkill for simple tasks (and in fact they are), but prove very flexible and useful for advanced uses, or to write generic highly customizable code.
Of course, being experimental, this library has still some issues...

On Wednesday, 13 July 2016 at 20:57:49 UTC, Lodovico Giaretta wrote:
> On Wednesday, 13 July 2016 at 20:44:52 UTC, Adam Sansier wrote:
>> On Wednesday, 13 July 2016 at 16:28:23 UTC, Lodovico Giaretta wrote:
>>>
>>> It's actually quite easy. Here's the code (untested):
>>>
>>> ================================================================
>>>
>>> import std.experimental.allocator.building_blocks.region;
>>> import std.experimental.allocator.mallocator;
>>> import std.experimental.allocator;
>>>
>>>
>>> Region(shared(Mallocator)) exception_allocator;
>>> enum EXCEPTION_MEM_SIZE = 256*1024;
>>> static this()
>>> {
>>>     exception_allocator = typeof(exception_allocator)(EXCEPTION_MEM_SIZE);
>>> }
>>>
>>> ================================================================
>>>
>>> And here is an usage example (untested, too):
>>>
>>> ================================================================
>>>
>>> void throwingFunction()
>>> {
>>>     // try to do something, but fail
>>>     throw exception_allocator.make!Exception("my wonderful error message");
>>> }
>>>
>>> void throwingThrowingFunction()
>>> {
>>>     try
>>>     {
>>>         // try to call function, which fails
>>>         throwingFunction;
>>>     }
>>>     catch (Exception exc)
>>>     {
>>>         // try to recover from failure, but generate other exception (just to show chaining)
>>>         throw exception_allocator.make!Exception("I love exception chaining");
>>>     }
>>> }
>>>
>>> void main()
>>> {
>>>     try
>>>     {
>>>         // try to call function, which fails
>>>         throwingThrowingFunction;
>>>     }
>>>     catch (Exception exc)
>>>     {
>>>         // recover from failure, then deallocate the exceptions no longer needed
>>>         exception_allocator.deallocateAll;
>>>     }
>>> }
>>> ================================================================
>>
>> Doesn't work. identifier expected on shared.
>>
>> What's the difference of simply using malloc to allocate the memory and creating the exceptions their? Seems like a long and winded way go about it or is there some benefit to using the experimental allocators?
>
> `Region(shared(Mallocator))` shall be `Region!(shared Mallocator)` (again, I'm just looking at the code, didn't test it).
>
> The advantages over a simple malloc are:
> 1) You can change between GC allocation, malloc, mmap and other allocators by changing a single line, instead of changing every throw;

Ok, I like!
> 2) you can use very fast allocators, based on your needs; this example uses the Region allocator, which is way faster than a call to malloc;

I like too! But I'll have to assume you are right since I have no proof.

> 3) the Region allocator has the added value of working even if there's no more memory available (because it preallocated it).

Well, one could do this with malloc because one can pre-allocate it too. I figure this is why you stated 2 though because it is pre-allocated? So, really only point 1 stands, but that is probably not even valid since one can wrap the allocator in a template. This is probably exactly what is being done..

So, ultimately no real benefit except the implementation details have been removed. That's not a bad thing as long as it works ;)

> In general, the allocators library provides facilities that may seem overkill for simple tasks (and in fact they are), but prove very flexible and useful for advanced uses, or to write generic highly customizable code.
> Of course, being experimental, this library has still some issues...

Well, I will try out the code and see. You've provided an example and if it works then it should be good enough in my case. If it doesn't limit what I need to do then I'm happy ;)

How is phobo's going to deal with such things when it is trying to get off the GC? It surely has to throw exceptions. Similar method or something entirely different?

Thanks.

On Wednesday, 13 July 2016 at 21:12:29 UTC, Adam Sansier wrote:
>> The advantages over a simple malloc are:
>> 1) You can change between GC allocation, malloc, mmap and other allocators by changing a single line, instead of changing every throw;
>
> Ok, I like!
>> 2) you can use very fast allocators, based on your needs; this example uses the Region allocator, which is way faster than a call to malloc;
>
> I like too! But I'll have to assume you are right since I have no proof.
>
>> 3) the Region allocator has the added value of working even if there's no more memory available (because it preallocated it).
>
> Well, one could do this with malloc because one can pre-allocate it too. I figure this is why you stated 2 though because it is pre-allocated? So, really only point 1 stands, but that is probably not even valid since one can wrap the allocator in a template. This is probably exactly what is being done..
>
> So, ultimately no real benefit except the implementation details have been removed. That's not a bad thing as long as it works ;)
>
>> In general, the allocators library provides facilities that may seem overkill for simple tasks (and in fact they are), but prove very flexible and useful for advanced uses, or to write generic highly customizable code.
>> Of course, being experimental, this library has still some issues...
>
> Well, I will try out the code and see. You've provided an example and if it works then it should be good enough in my case. If it doesn't limit what I need to do then I'm happy ;)
>
> How is phobo's going to deal with such things when it is trying to get off the GC? It surely has to throw exceptions. Similar method or something entirely different?
>
> Thanks.

At the end, all memory comes from one of these: GC heap, malloc, mmap, sbrk. All other allocators build on top of these (or on top of user supplied buffers, which come from these as well). What those "wrapper" allocators do is managing the given memory, either using different allocations strategies for different allocation sizes, or keeping lists of free blocks instead of returning them using "free", or other things (have a look at the docs). So at the end they do what you would manually do in C/C++ to "personalize" the allocations, with the aim of reducing waste and/or being faster. They are also arbitrarily composable on top of each other.

I don't know how Phobos will handle exceptions. Maybe use reference counting (coming soon in D, maybe)? Maybe algorithms that already have to allocate will switch from using the GC to using a user-supplied custom allocator, and will use it for exceptions too.

Currently I'm working on a replacement for std.xml and I'm making every component take a template parameter to specify the allocator. Initially the allocators look like a real mess, but after a couple of days playing with them, you start understanding the mechanics and at the end you really enjoy them. Changing a single parameters allows to switch between @safe gc code and @nogc code in the unittests, without changing the implementation.

On Wednesday, 13 July 2016 at 21:27:16 UTC, Lodovico Giaretta wrote:
> On Wednesday, 13 July 2016 at 21:12:29 UTC, Adam Sansier wrote:
>>> [...]
>>
>> Ok, I like!
>>> [...]
>>
>> I like too! But I'll have to assume you are right since I have no proof.
>>
>>> [...]
>>
>> Well, one could do this with malloc because one can pre-allocate it too. I figure this is why you stated 2 though because it is pre-allocated? So, really only point 1 stands, but that is probably not even valid since one can wrap the allocator in a template. This is probably exactly what is being done..
>>
>> So, ultimately no real benefit except the implementation details have been removed. That's not a bad thing as long as it works ;)
>>
>>> [...]
>>
>> Well, I will try out the code and see. You've provided an example and if it works then it should be good enough in my case. If it doesn't limit what I need to do then I'm happy ;)
>>
>> How is phobo's going to deal with such things when it is trying to get off the GC? It surely has to throw exceptions. Similar method or something entirely different?
>>
>> Thanks.
>
> At the end, all memory comes from one of these: GC heap, malloc, mmap, sbrk. All other allocators build on top of these (or on top of user supplied buffers, which come from these as well). What those "wrapper" allocators do is managing the given memory, either using different allocations strategies for different allocation sizes, or keeping lists of free blocks instead of returning them using "free", or other things (have a look at the docs). So at the end they do what you would manually do in C/C++ to "personalize" the allocations, with the aim of reducing waste and/or being faster. They are also arbitrarily composable on top of each other.
>
> I don't know how Phobos will handle exceptions. Maybe use reference counting (coming soon in D, maybe)? Maybe algorithms that already have to allocate will switch from using the GC to using a user-supplied custom allocator, and will use it for exceptions too.
>
> Currently I'm working on a replacement for std.xml and I'm making every component take a template parameter to specify the allocator. Initially the allocators look like a real mess, but after a couple of days playing with them, you start understanding the mechanics and at the end you really enjoy them. Changing a single parameters allows to switch between @safe gc code and @nogc code in the unittests, without changing the implementation.

Ok. Is there a way to bundle allocations so that one free will work?

For example, your exception handling looks good but I need to supply custom messges. If I use sformat I have to create the array for the buffer to create the message in. This means I would have to free both the exception and the string. It would be nice to be able to do this in one go.

Also, could one create a struct or class for the exception so that it is automatically free'ed at the end of a catch block, if caught? Probably not without language help?

Sometimes it's nice to include runtime info in an error message such as an OS error code.

On Wednesday, 13 July 2016 at 22:42:36 UTC, Adam Sansier wrote:
> On Wednesday, 13 July 2016 at 21:27:16 UTC, Lodovico Giaretta wrote:
>> At the end, all memory comes from one of these: GC heap, malloc, mmap, sbrk. All other allocators build on top of these (or on top of user supplied buffers, which come from these as well). What those "wrapper" allocators do is managing the given memory, either using different allocations strategies for different allocation sizes, or keeping lists of free blocks instead of returning them using "free", or other things (have a look at the docs). So at the end they do what you would manually do in C/C++ to "personalize" the allocations, with the aim of reducing waste and/or being faster. They are also arbitrarily composable on top of each other.
>>
>> I don't know how Phobos will handle exceptions. Maybe use reference counting (coming soon in D, maybe)? Maybe algorithms that already have to allocate will switch from using the GC to using a user-supplied custom allocator, and will use it for exceptions too.
>>
>> Currently I'm working on a replacement for std.xml and I'm making every component take a template parameter to specify the allocator. Initially the allocators look like a real mess, but after a couple of days playing with them, you start understanding the mechanics and at the end you really enjoy them. Changing a single parameters allows to switch between @safe gc code and @nogc code in the unittests, without changing the implementation.
>
> Ok. Is there a way to bundle allocations so that one free will work?
>
> For example, your exception handling looks good but I need to supply custom messges. If I use sformat I have to create the array for the buffer to create the message in. This means I would have to free both the exception and the string. It would be nice to be able to do this in one go.
>
> Sometimes it's nice to include runtime info in an error message such as an OS error code.

As you probably saw, sformat still has some problems with @nogc, because it internally uses std.uft.encode, which may throw a GC-allocated exception, but this can be solved.

Not all allocators keep track of the memory ranges they allocated (e.g.: Mallocator). The ones that do (like Region) usually provide a deallocateAll method. So the idea is that you allocate all data needed by your exception (string buffers or whatever) and the exception itself with one of these allocators, and at the end of a catch you deallocateAll.

> Also, could one create a struct or class for the exception so that it is automatically free'ed at the end of a catch block, if caught? Probably not without language help?

I don't think it's feasible without language support for reference counted classes (which may be added sooner or later, as it has been asked and proposals have been made, and would be very useful). Also note that you don't want RAII or scope(exit) in this case but scope(success), as the exception shall not be deallocated if another one is raised in the catch block, because in D the second exception does not "overwrite" the first, but is chained to it, so that an outer catch can see both of them.

On Friday, 13 February 2015 at 19:03:10 UTC, Jonathan Marler wrote:
> This question comes from wanting to be able to throw an exception in code that is @nogc.
>

https://dlang.org/changelog/2.079.0.html#dip1008
```d
void main() @nogc
{
    throw new Exception("I'm @nogc now");
}
```