On 03/14/2013 04:58 PM, Jesse Phillips wrote:
> On Thursday, 14 March 2013 at 21:40:34 UTC, Ali Çehreli wrote:
>> I had toyed with the idea of making a ForwardRange from an InputRange
>> by caching the elements. Without any guarantees, :) here is the code:
>>
>> http://forum.dlang.org/thread/ifg5ei$2qc7$1@digitalmars.com
>>
>> Ali
>
> I attempted to create a sliceable forward range from an input range. I
> think I ran into a problem of keeping the buffer in sync when doing
> lookahead.
>
> list2 = list1.save();
>
> while(true) {
> list1.popFront();
> list2.popFront();
>
> assert(list1.front == list2.front);
> }
>
> What happens when your buffer needs to fill, how do you get the data in
> list 1 & 2? I'm not posing this question as impossible only something to
> make work.

I had to study that old code of mine to understand how it works. :)

The popFront() on the range itself dispatches the call to ForwardRangeLazyCache by providing the id of this "view". In a sense, it says "the range with this id would like to pop an element":

    @property void popFront()
    {
        lazyCache.popFront(myId);
    }

ForwardRangeLazyCache.popFront simply increments the pointer of the range that is associated with that id:

    void popFront(size_t id)
    {
        ++(viewPtrs[id]);
    }

I am not sure whether this is a good idea but the actual filling of the cache and synchronizing the "views" happen in front():

    @property immutable(E) front(size_t id)
    {
        ensureRead(viewPtrs[id]);
        return *(viewPtrs[id]);
    }

The above says "ensure that the element that this view is trying to access is in the cache", and then returns that element. It looks like there is even some optimization if the array elements have not been moved:

    void ensureRead(immutable(E) * ptr)
    {
        if (ptr == endPtr) {
            immutable oldPtr = cache.ptr;
            cache ~= range.front.idup;
            range.popFront();

            if (cache.ptr != oldPtr) {
                /* The cache has been relocated */
                adjustPtrs(oldPtr, cache.ptr);

            } else {
                ++endPtr;
            }
        }
    }

etc. :) I am really not sure whether it works correctly. :/

Ali

On Thursday, 14 March 2013 at 14:51:42 UTC, monarch_dodra wrote:
> On Thursday, 14 March 2013 at 14:29:59 UTC, Andrea Fontana wrote:
>> On Thursday, 14 March 2013 at 13:58:56 UTC, monarch_dodra wrote:
>>> On Thursday, 14 March 2013 at 13:20:51 UTC, Andrea Fontana wrote:
>>>> On Thursday, 14 March 2013 at 12:29:26 UTC, Andrei Alexandrescu wrote:
>>>>> On 3/14/13 6:45 AM, Andrea Fontana wrote:
>>>>>> On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana wrote:
>>>>>>> I'm trying to implement a db cursor as simple InputRange. I can't
>>>>>>> implement as forward range because using c-api I can't clone/save cursor.
>>>>> Add .save() to it.
>>>>>
>>>>> Andrei
>>>>
>>>> I can't add save and create a forward range: c-api doesn't allow me to clone db cursor.
>>>
>>> But think about it. You say you can't save, but you want pass by value to preserve state. If you can preserve state, doesn't that mean you just saved?
>>>
>>> Once you've passed your "can't preserve state" to a foreach, then your range is consumed. So yes, it is correct behavior.
>>>
>>> Most input ranges should model reference semantics when passed/copied by value. If somebody modifies the range, all ranges are affected. The tough part is usually preserving correct state.
>>
>> I was hoping there's something I didn't know like rewind() to implement :)
>>
>> Ok, I'll pray db developer to add a clone method or I'll use something like
>>
>> auto res = cursor.array(); if needed
>
> You *could* add rewind to your range, but it isn't quite the same as save: Rewind means you can move back and forth in your view of the file (or cursor/db), but it does not mean you can actually *duplicate* (save) your range, to have two distinct ranges with a distinct view of the your db.
>
> For example, while the interface is different, a C "FILE*" is what you'd call a rewindable input range with reference semantics.

Yes it was just a way to avoid dead cursors locked at end of results :)
I add a feature request hoping they add a method to clone cursors (i think/hope it's possible)

In the meanwhile you have to do query again.

On Friday, 15 March 2013 at 03:44:51 UTC, Ali Çehreli wrote:
> On 03/14/2013 04:58 PM, Jesse Phillips wrote:
>> On Thursday, 14 March 2013 at 21:40:34 UTC, Ali Çehreli wrote:
>>> I had toyed with the idea of making a ForwardRange from an InputRange
>>> by caching the elements. Without any guarantees, :) here is the code:
>>>
>>> http://forum.dlang.org/thread/ifg5ei$2qc7$1@digitalmars.com
>>>
>>> Ali
>>
>> I attempted to create a sliceable forward range from an input range. I
>> think I ran into a problem of keeping the buffer in sync when doing
>> lookahead.
>>
>> list2 = list1.save();
>>
>> while(true) {
>> list1.popFront();
>> list2.popFront();
>>
>> assert(list1.front == list2.front);
>> }
>>
>> What happens when your buffer needs to fill, how do you get the data in
>> list 1 & 2? I'm not posing this question as impossible only something to
>> make work.
>
> I had to study that old code of mine to understand how it works. :)
>
> The popFront() on the range itself dispatches the call to ForwardRangeLazyCache by providing the id of this "view". In a sense, it says "the range with this id would like to pop an element":
>
>     @property void popFront()
>     {
>         lazyCache.popFront(myId);
>     }
>
> ForwardRangeLazyCache.popFront simply increments the pointer of the range that is associated with that id:
>
>     void popFront(size_t id)
>     {
>         ++(viewPtrs[id]);
>     }
>
> I am not sure whether this is a good idea but the actual filling of the cache and synchronizing the "views" happen in front():
>
>     @property immutable(E) front(size_t id)
>     {
>         ensureRead(viewPtrs[id]);
>         return *(viewPtrs[id]);
>     }
>
> The above says "ensure that the element that this view is trying to access is in the cache", and then returns that element. It looks like there is even some optimization if the array elements have not been moved:
>
>     void ensureRead(immutable(E) * ptr)
>     {
>         if (ptr == endPtr) {
>             immutable oldPtr = cache.ptr;
>             cache ~= range.front.idup;
>             range.popFront();
>
>             if (cache.ptr != oldPtr) {
>                 /* The cache has been relocated */
>                 adjustPtrs(oldPtr, cache.ptr);
>
>             } else {
>                 ++endPtr;
>             }
>         }
>     }
>
> etc. :) I am really not sure whether it works correctly. :/
>
> Ali

It *seems* to work :) Funny. The only problem is that caching db results maybe it's not a good idea if i read too much data.

I'm not sure if it's fine or not but:
- I've changed AsForwardRange from class to struct
- I've added a method (it's an "hack" to make it works):
this(this)
{
	auto dummy = lazyCache.newView(lazyCache.cache.ptr);
	myId = dummy.myId;
}
- I've add @property to save() -> isForwardRange gives false without this.

I did this to make it similar to other ranges (original topic!)
Shouldn't a view unregister pointer on its d-tor?

I'm not sure it's correct, is it?

On 03/15/2013 02:57 AM, Andrea Fontana wrote:
> On Friday, 15 March 2013 at 03:44:51 UTC, Ali Çehreli wrote:

>>>> I had toyed with the idea of making a ForwardRange from an InputRange
>>>> by caching the elements. Without any guarantees, :) here is the code:

> It *seems* to work :) Funny. The only problem is that caching db results
> maybe it's not a good idea if i read too much data.

Agreed but that's the only feature of the range. :) The way the data is "uncached" is by moving the slice forward by this code:

        /* Nobody is using the first part of the cache anymore */
        cache = cache[(minPtr - cache.ptr) .. $];

I think it is sufficient for the GC to free the previous first part of the array. The following program keeps running without consuming the memory:

void main()
{
    int[] a;
    while (true) {
        a ~= 42;
        a = a[1 .. $];
    }
}

> I'm not sure if it's fine or not but:
> - I've changed AsForwardRange from class to struct

Good idea. It looks like I had already made it a struct in my local copy of it.

> - I've added a method (it's an "hack" to make it works):
> this(this)
> {
> auto dummy = lazyCache.newView(lazyCache.cache.ptr);
> myId = dummy.myId;
> }

I am not sure why that was necessary. Additionally, some of the member functions like empty() can be made const.

> - I've add @property to save() -> isForwardRange gives false without this.

Agreed.

> I did this to make it similar to other ranges (original topic!)
> Shouldn't a view unregister pointer on its d-tor?

That's a good idea too. :)

Ali

On Sunday, 17 March 2013 at 04:54:58 UTC, Ali Çehreli wrote:
> On 03/15/2013 02:57 AM, Andrea Fontana wrote:
> > On Friday, 15 March 2013 at 03:44:51 UTC, Ali Çehreli wrote:
> > - I've added a method (it's an "hack" to make it works):
> > this(this)
> > {
> > auto dummy = lazyCache.newView(lazyCache.cache.ptr);
> > myId = dummy.myId;
> > }
>
> I am not sure why that was necessary. Additionally, some of the member functions like empty() can be made const.

If not something like:

foreach(e; yourrange)  // <-- here this(this) is called
{
   writeln(e);
}

foreach(e; yourrange) // <-- here this(this) is called
{
   writeln(e);
}

Print:
1 2 3 4 5 6

instead of:
1 2 3 1 2 3  as "expected"

But it's an hack, you should know a better way to do it :)
When a struct is copied like in example above, every copy should retain its own pointer to data (restarted from begin).