It isn't really a release announce, maybe a pre-release.

tanya is a general purpose library, used mostly for networking by me. It is an attempt to develop an alternative memory model for D; 100% of the library are usable in @nogc code.

I previously announced that I want to merge my code with another project and that I would stop the development. But it didn't happen. I'm sorry that I wanted to abandon it. It won't happen anymore. I'll continue the development.

tanya contains an event loop, containers, an URL parsing routine and alternative TCP sockets implementation, multiple precision integer. The library is cross plattform, but not thread-safe yet.

The memory management is based on allocators. The allocators are one-way compatible with std.experimental.allocator, it means my allocators can be used with std.exeperimental.allocator but the std.experimental.allocator isn't usable with my lbirary (though it is pretty straightforward to write a wrapper for phobos allocators). tanya's allocators follow Bloomberg Allocator Model, see discussions on BDE Allocator Model for pro and contra.

And sorry once again for a long post...

What's new:
- Bug fixes and performance improvements in the main allocator: MmapPool.
- Container redesign: There is a new container, Vector (similar to std.container.array) with the support of ranges and all the cool stuff. I also started to experiment with the "move" semantics in D based on the D conf talk of Ali Çehreli. There are also a singly linked list, circular buffer and a queue. SList will get range support soon, and some time later the buffers.
- Bug fixes in the epoll event loop and a few new features.
- RefCounted: more similar to C++ shared_ptr than to phobos' RefCounted, but more primitive.

Future plans:
I still need a few more containers: UTF-8 string and a hash table. The string won't be the same as RCString. I'm thinking of a not templated string with UTF-8 support only (with a possibility to initialize it from a string, dstring and wstring).
Big integer bug fixing and optimization has a priority aswell for me.
I'm also planning to add streams but it won't happen very soon since it is a lot of work.

I plan to start a small blog in April based on the library (university project). It will be a SCGI-server, serving only static pages, behind Apache. It will be the first try-out :). From there I can move to a http-server.

The library is still under permanent development. I begin to care about the API stability and proper deprecations, but it isn't always possible.
I intend to make releases regularly, maybe all the 6 weeks and to prepare some new features for each of the releases.
The API docs for the latest version can be found on https://docs.caraus.io/tanya/
If you have questions, need some examples, let me know. I'll write something up and publish it somewhere.

https://github.com/caraus-ecms/tanya
http://code.dlang.org/packages/tanya
https://docs.caraus.io/tanya/

On Saturday, 18 February 2017 at 15:51:59 UTC, Eugene Wissner wrote:
> It isn't really a release announce, maybe a pre-release.
>
> tanya is a general purpose library, used mostly for networking by me. It is an attempt to develop an alternative memory model for D; 100% of the library are usable in @nogc code.

Sounds really cool!


> tanya contains an event loop, containers, an URL parsing routine and alternative TCP sockets implementation, multiple precision integer. The library is cross plattform, but not thread-safe yet.

Do you know about eventcore [1] (event loop abstraction)?
I hope that this will soon be submitted to Phobos as everyone seems to be brewing their own solution and I am _not_ looking forward to manage to find a way to run five event loops when I want to use fix different libraries.

[1] https://github.com/vibe-d/eventcore

> The memory management is based on allocators. The allocators are one-way compatible with std.experimental.allocator, it means my allocators can be used with std.exeperimental.allocator but the std.experimental.allocator isn't usable with my lbirary (though it is pretty straightforward to write a wrapper for phobos allocators). tanya's allocators follow Bloomberg Allocator Model, see discussions on BDE Allocator Model for pro and contra.

From [2] and your API I get that the BDE API is:

    virtual void* allocate(size_type size)
    virtual void deallocate(void *address)

whereas you extended this by:

   int alignment()
   void[] allocate(in size_t size)
   bool deallocate(void[] p)
   bool reallocate(ref void[] p, in size_t size)
   bool reallocateInPlace(ref void[] p, in size_t size)

Well obviously the std.experimental.allocator protocol is more complicated:

    uint alignment()
    void[] allocate(size_t, TypeInfo ti = null)
    void[] alignedAllocate(size_t n, uint a)
    void[] allocateAll()
    bool expand(ref void[], size_t);
    bool reallocate(ref void[], size_t);
    bool alignedReallocate(ref void[] b, size_t size, uint alignment);
    Ternary owns(void[] b);
    Ternary resolveInternalPointer(void* p, ref void[] result);
    bool deallocate(void[] b);
    bool deallocateAll();
    Ternary empty();

However I would be very interested in hearing your reasons for not using it.
Keep in mind that std.experimental.allocator is supposed to be a general purpose library exactly to avoid everyone writing their own, incompatible Allocators ;-)
So please raise your voice as long as it's still in experimental!

[2] https://github.com/bloomberg/bde/wiki/BDE-Allocator-model

On Saturday, 18 February 2017 at 23:08:52 UTC, Seb wrote:
> Do you know about eventcore [1] (event loop abstraction)?
> I hope that this will soon be submitted to Phobos as everyone seems to be brewing their own solution and I am _not_ looking forward to manage to find a way to run five event loops when I want to use fix different libraries.
>
> [1] https://github.com/vibe-d/eventcore
>
Yes. The event loop is the oldest part of tanya. At the time I began to work on it, eventcore was very young and got very seldom updates. I only heard it is the new fast and native event loop for the new generation of vibe.d. There was libasync aswell which was written as a native replacement for libevent, but was never adopted more than just as an option; instead eventcore was written from the ground up. Botan suffered an even worse fate. So I wasn't sure what direction eventcore would take. No of the event loops, I was aware of, had IOCP support on Windows. For me windows wasn't that important, but I implemented it to ensure that the same API can work with both, Posix-style events and the asynchronous Windows completion ports; and I could really fix some design flaws implementing IOCP. I don't know if I really want to have such an abstraction included in phobos, since there are bindings for C event loops and a few native ones, and maybe they will continue to exist, regardless whether there is something in Phobos or not. But yes, if Phobos would have something for asynchronous programming, I would take a look into it again.

> However I would be very interested in hearing your reasons for not using it.
> Keep in mind that std.experimental.allocator is supposed to be a general purpose library exactly to avoid everyone writing their own, incompatible Allocators ;-)
> So please raise your voice as long as it's still in experimental!
>
> [2] https://github.com/bloomberg/bde/wiki/BDE-Allocator-model

Thanks for the question. I had to describe it from the beginning or in the README, but didn't want to blow my message up.

BDE allocators implement an interface, other allocators normally not. It is the main point. It is not about a concrete interface. Phobos allocators are structs, they can't implement anything and there is a CAllocatorImpl, that implements IAllocator and wraps other allocators, so structs can behave as if they would implement IAllocator too. But these structs can be used directly without CallocatorImpl in the templated code.
So if you have a container, you can do:
auto arr = Array!(int, Mallocator)(Mallocator.instance);

EMSIContainers do that and it is the way Phobos allocators and containers will go. (https://github.com/economicmodeling/containers)

If you don't want to template your code or you use phobos default allocator, you do the following (because theAllocator should have some type):
theAllocator = allocatorObject(Mallocator.instance);


Contras of my allocators:
1) they are slower (because of virtual function calls)
2) If the allocator is passed as template parameter, functions that rely on the allocator can infer the attributes like @safe, @nogc, pure (yes, there was a discussion to make malloc pure).

Pros:
1) "is-a" relation. Mallocator "is-a" allocator, MmapAllocator "is-a" allocator. In your code if you use allocators as a template parameter (the faster way), you can't rely that the allocator implements some method. So your code will look like:

static if (allocator has method "expand")
{
    if (allocator.expand(arr, 8))
    {
    }
}

2) Less code bloat. With phobos allocators you would make as much as possible to templates.

3) I see absolute no reason, why Array!(int, Mallocator) and Array!(int, GCAllocator) has different types. Do you want a function that accepts an array:

void myFunc(Array!int arr);

you have to make a template function from it (with a long "if check" that ensures that an Array!int is passed). Code bloat.

4) It just works everywhere. Phobos allocators are structs but there is still IAllocator interface, because the model isn't universal enough.

5) I started by implementing IAllocator and the reason I dropped it then, is IAllocator.expand. If you have an array of type A where A is a struct with a postblit constructor, you can't just call realloc() to resize the array. realloc() can move memory and if an object of type A has references to other objects in the array, the objects will be corrupted. "A" should be a POD-type. Otherwise you have to allocate new memory, initialize it, copy the objects by one and deallocate the old memory. Now there is IAllocator.expand(). If IAllocator.expand() can expand the memory in place, everything is fine, you haven't to copy the objects because the addresses don't change. But libc realloc for example doesn't guarantee that the address doesn't change if you shrink the memory. But IAllocator doesn't have a shrink() method.
And anyway I find this separation expand/shrink or expandArray/shrinkArray inconvenient; I had to check everywhere if the new size is larger or smaller than the old one, then call expandArray or shrinkArray with a delta size (maybe it was only my use case). Therefore I replaced it reallocateInPlace.

On Sunday, 19 February 2017 at 11:41:44 UTC, Eugene Wissner wrote:
> realloc() can move memory and if an object of type A has references to other objects in the array, the objects will be corrupted. "A" should be a POD-type. Otherwise you have to allocate new memory, initialize it, copy the objects by one and deallocate the old memory. Now there is IAllocator.expand().

What's the difference between realloc+postblit and copy one by one? Postblit is called only after copy, it's not a constructor.

On Wednesday, 22 February 2017 at 13:00:19 UTC, Kagamin wrote:
> On Sunday, 19 February 2017 at 11:41:44 UTC, Eugene Wissner wrote:
>> realloc() can move memory and if an object of type A has references to other objects in the array, the objects will be corrupted. "A" should be a POD-type. Otherwise you have to allocate new memory, initialize it, copy the objects by one and deallocate the old memory. Now there is IAllocator.expand().
>
> What's the difference between realloc+postblit and copy one by one? Postblit is called only after copy, it's not a constructor.

The difference is that realloc will deallocate the old memory before postblit.
For the sake of example:

struct Sample
{
	Sample* sample;

	this(this)
	{
		sample.sample = &this;
	}
}

void main()
{
	auto s = new Sample[2];

	s[0].sample = &s[1];
	s[1].sample = &s[0];
}

Now suppose s[0] has the address 7fc8d2717000, s[1]: 7fc8d2717008.
Realloc cannot reallocate in place and after the call to realloc s[0] has the address 7f27771e0000 and s[1]: 7f27771e0000. It is too late to call the postblit here, Sample.sample refers to invalid memory.