an idea to lock data by removing the reference:

class A
{
   Lockable!Data data;
}

The idea is that when the data is going to be used, the user locks the data. The trick here is that data is a pointer to the data and the pointer is set to null when locked so no other data can use it(they see a null reference). To unlock, the data is reassigned:

auto d = data.lock(); // A.data is now null deals with sync issues
//use d
d = data.unlock(d); // restores A.data (A.data = d;) and sets d to null so any future reference is an error(could produce bugs but should mostly be access violations)


Anyone else trying to use data will see that it is null while it is locked.

This basically pushes the standard locking mechanisms in to the Lockable!data(first come first serve) and code that has not captured the data see's it simply as null.

Anyone use know if there exists an idiom like this and what it is called? Maybe some idiomatic code that is efficient?


Ideally I'd want to be able to treat the Lockable!Data as Data. Right now I have to classes(or pointers to structs) and few weird hacks. I think what I'll have to end up doing is having a Locked!Data structure that is returned instead or use an UFCS. The idea being to attach the lock and unlock methods.

On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
> an idea to lock data by removing the reference:
>
> class A
> {
>    Lockable!Data data;
> }
>
> The idea is that when the data is going to be used, the user locks the data. The trick here is that data is a pointer to the data and the pointer is set to null when locked so no other data can use it(they see a null reference). To unlock, the data is reassigned:
>
> auto d = data.lock(); // A.data is now null deals with sync issues
> //use d
> d = data.unlock(d); // restores A.data (A.data = d;) and sets d to null so any future reference is an error(could produce bugs but should mostly be access violations)
>
>
> Anyone else trying to use data will see that it is null while it is locked.
>
> This basically pushes the standard locking mechanisms in to the Lockable!data(first come first serve) and code that has not captured the data see's it simply as null.
>
> Anyone use know if there exists an idiom like this and what it is called? Maybe some idiomatic code that is efficient?
>
>
> Ideally I'd want to be able to treat the Lockable!Data as Data. Right now I have to classes(or pointers to structs) and few weird hacks. I think what I'll have to end up doing is having a Locked!Data structure that is returned instead or use an UFCS. The idea being to attach the lock and unlock methods.

Are you aware of NullableRef?
https://dlang.org/library/std/typecons/nullable_ref.html

Does it fit somehow?

On Tuesday, 22 May 2018 at 22:10:52 UTC, Alex wrote:
> On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
>> an idea to lock data by removing the reference:
>>
>> class A
>> {
>>    Lockable!Data data;
>> }
>>
>> The idea is that when the data is going to be used, the user locks the data. The trick here is that data is a pointer to the data and the pointer is set to null when locked so no other data can use it(they see a null reference). To unlock, the data is reassigned:
>>
>> auto d = data.lock(); // A.data is now null deals with sync issues
>> //use d
>> d = data.unlock(d); // restores A.data (A.data = d;) and sets d to null so any future reference is an error(could produce bugs but should mostly be access violations)
>>
>>
>> Anyone else trying to use data will see that it is null while it is locked.
>>
>> This basically pushes the standard locking mechanisms in to the Lockable!data(first come first serve) and code that has not captured the data see's it simply as null.
>>
>> Anyone use know if there exists an idiom like this and what it is called? Maybe some idiomatic code that is efficient?
>>
>>
>> Ideally I'd want to be able to treat the Lockable!Data as Data. Right now I have to classes(or pointers to structs) and few weird hacks. I think what I'll have to end up doing is having a Locked!Data structure that is returned instead or use an UFCS. The idea being to attach the lock and unlock methods.
>
> Are you aware of NullableRef?
> https://dlang.org/library/std/typecons/nullable_ref.html
>

Yes.

> Does it fit somehow?

Not really. It could be used to wrap the data when used as a struct but it offers none of the locking features which is the ultimate goal.

The idea is simply that one can lock the data to get exclusive access. Normally standard locking tricks are used but in this case the idea is to do all that behind the scenes. Locking the data makes all other accessors see null data(and crash or properly test). This is a first come first serve or single access type of pattern but sort of removes the data from prying eyes while it is being used.

On Tuesday, 22 May 2018 at 22:17:05 UTC, IntegratedDimensions wrote:
> On Tuesday, 22 May 2018 at 22:10:52 UTC, Alex wrote:
>> On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
>>> an idea to lock data by removing the reference:
>>>
>>> class A
>>> {
>>>    Lockable!Data data;
>>> }
>>>
>>> The idea is that when the data is going to be used, the user locks the data. The trick here is that data is a pointer to the data and the pointer is set to null when locked so no other data can use it(they see a null reference). To unlock, the data is reassigned:
>>>
>>> auto d = data.lock(); // A.data is now null deals with sync issues
>>> //use d
>>> d = data.unlock(d); // restores A.data (A.data = d;) and sets d to null so any future reference is an error(could produce bugs but should mostly be access violations)
>>>
>>>
>>> Anyone else trying to use data will see that it is null while it is locked.
>>>
>>> This basically pushes the standard locking mechanisms in to the Lockable!data(first come first serve) and code that has not captured the data see's it simply as null.
>>>
>>> Anyone use know if there exists an idiom like this and what it is called? Maybe some idiomatic code that is efficient?
>>>
>>>
>>> Ideally I'd want to be able to treat the Lockable!Data as Data. Right now I have to classes(or pointers to structs) and few weird hacks. I think what I'll have to end up doing is having a Locked!Data structure that is returned instead or use an UFCS. The idea being to attach the lock and unlock methods.
>>
>> Are you aware of NullableRef?
>> https://dlang.org/library/std/typecons/nullable_ref.html
>>
>
> Yes.
>
>> Does it fit somehow?
>
> Not really. It could be used to wrap the data when used as a struct but it offers none of the locking features which is the ultimate goal.
>
> The idea is simply that one can lock the data to get exclusive access. Normally standard locking tricks are used but in this case the idea is to do all that behind the scenes. Locking the data makes all other accessors see null data(and crash or properly test). This is a first come first serve or single access type of pattern but sort of removes the data from prying eyes while it is being used.

how about something like

 import core.atomic;
 class Lockable!Data
 {
    private __gshared Lockable!Data data;
 }

 struct Locked!Lockable!Data
 {
    private TailShared!Lockable!Data lockedData;
    private Lockable!Data lockableData;

    this(Lockable!Data lockableData)
    {
        this.lockableData = lockableData;
        while( (lockedData= atomicLoad(lockableData)) !is null)
            if(cas(lockedData, lockedData, null))
                break;
    }

    ~this()
    {
        atomicStore(lockedData, lockableData );
    }

    alias lockedData.data this;
 }

With something like this you should be able to do RAII like semantics on your data.
You share a Lockable!Data which you want use, and then you can access it if you instantiate a Locked!Lockable!Data struct with it.
FYI, my atomics isn't all that great so don't expect this to work.

On Tuesday, 22 May 2018 at 23:09:24 UTC, Sjoerd Nijboer wrote:
> On Tuesday, 22 May 2018 at 22:17:05 UTC, IntegratedDimensions wrote:
>> On Tuesday, 22 May 2018 at 22:10:52 UTC, Alex wrote:
>>> On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
>>>> an idea to lock data by removing the reference:
>>>>
>>>> class A
>>>> {
>>>>    Lockable!Data data;
>>>> }
>>>>
>>>> The idea is that when the data is going to be used, the user locks the data. The trick here is that data is a pointer to the data and the pointer is set to null when locked so no other data can use it(they see a null reference). To unlock, the data is reassigned:
>>>>
>>>> auto d = data.lock(); // A.data is now null deals with sync issues
>>>> //use d
>>>> d = data.unlock(d); // restores A.data (A.data = d;) and sets d to null so any future reference is an error(could produce bugs but should mostly be access violations)
>>>>
>>>>
>>>> Anyone else trying to use data will see that it is null while it is locked.
>>>>
>>>> This basically pushes the standard locking mechanisms in to the Lockable!data(first come first serve) and code that has not captured the data see's it simply as null.
>>>>
>>>> Anyone use know if there exists an idiom like this and what it is called? Maybe some idiomatic code that is efficient?
>>>>
>>>>
>>>> Ideally I'd want to be able to treat the Lockable!Data as Data. Right now I have to classes(or pointers to structs) and few weird hacks. I think what I'll have to end up doing is having a Locked!Data structure that is returned instead or use an UFCS. The idea being to attach the lock and unlock methods.
>>>
>>> Are you aware of NullableRef?
>>> https://dlang.org/library/std/typecons/nullable_ref.html
>>>
>>
>> Yes.
>>
>>> Does it fit somehow?
>>
>> Not really. It could be used to wrap the data when used as a struct but it offers none of the locking features which is the ultimate goal.
>>
>> The idea is simply that one can lock the data to get exclusive access. Normally standard locking tricks are used but in this case the idea is to do all that behind the scenes. Locking the data makes all other accessors see null data(and crash or properly test). This is a first come first serve or single access type of pattern but sort of removes the data from prying eyes while it is being used.
>
> how about something like
>
>  import core.atomic;
>  class Lockable!Data
>  {
>     private __gshared Lockable!Data data;
>  }
>
>  struct Locked!Lockable!Data
>  {
>     private TailShared!Lockable!Data lockedData;
>     private Lockable!Data lockableData;
>
>     this(Lockable!Data lockableData)
>     {
>         this.lockableData = lockableData;
>         while( (lockedData= atomicLoad(lockableData)) !is null)
>             if(cas(lockedData, lockedData, null))
>                 break;
>     }
>
>     ~this()
>     {
>         atomicStore(lockedData, lockableData );
>     }
>
>     alias lockedData.data this;
>  }
>
> With something like this you should be able to do RAII like semantics on your data.
> You share a Lockable!Data which you want use, and then you can access it if you instantiate a Locked!Lockable!Data struct with it.
> FYI, my atomics isn't all that great so don't expect this to work.

This is similar to what I am doing... maybe "identical".

On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
> an idea to lock data by removing the reference:
>
> class A
> {
>    Lockable!Data data;
> }
>
> [...]

This sounds like you are looking for is an atomic swap. Afaik it doesn't exist in the standard library. You could use asm for the XCHG, but that would make your code x86 dependent.
I think the easiest way would be to just use a mutex and tryLock.

On 24/05/2018 1:20 AM, Malte wrote:
> On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
>> an idea to lock data by removing the reference:
>>
>> class A
>> {
>>    Lockable!Data data;
>> }
>>
>> [...]
> 
> This sounds like you are looking for is an atomic swap. Afaik it doesn't exist in the standard library. You could use asm for the XCHG, but that would make your code x86 dependent.
> I think the easiest way would be to just use a mutex and tryLock.

What are you talking about? :p

http://dpldocs.info/experimental-docs/core.atomic.cas.1.html

On Wednesday, 23 May 2018 at 13:24:35 UTC, rikki cattermole wrote:
> On 24/05/2018 1:20 AM, Malte wrote:
>> On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
>>> an idea to lock data by removing the reference:
>>>
>>> class A
>>> {
>>>    Lockable!Data data;
>>> }
>>>
>>> [...]
>> 
>> This sounds like you are looking for is an atomic swap. Afaik it doesn't exist in the standard library. You could use asm for the XCHG, but that would make your code x86 dependent.
>> I think the easiest way would be to just use a mutex and tryLock.
>
> What are you talking about? :p
>
> http://dpldocs.info/experimental-docs/core.atomic.cas.1.html

That is Compare-and-set.
To make an exchange using cas I first have to read the value, then write to it expecting to be still the value I read before. That are more instructions than just a swap. If a cas fails, I have to redo everything. An exchange never fails, I just might not get the result I would like to have (null instead of pointer).

On 24/05/2018 1:29 AM, Malte wrote:
> On Wednesday, 23 May 2018 at 13:24:35 UTC, rikki cattermole wrote:
>> On 24/05/2018 1:20 AM, Malte wrote:
>>> On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
>>>> an idea to lock data by removing the reference:
>>>>
>>>> class A
>>>> {
>>>>    Lockable!Data data;
>>>> }
>>>>
>>>> [...]
>>>
>>> This sounds like you are looking for is an atomic swap. Afaik it doesn't exist in the standard library. You could use asm for the XCHG, but that would make your code x86 dependent.
>>> I think the easiest way would be to just use a mutex and tryLock.
>>
>> What are you talking about? :p
>>
>> http://dpldocs.info/experimental-docs/core.atomic.cas.1.html
> 
> That is Compare-and-set.
> To make an exchange using cas I first have to read the value, then write to it expecting to be still the value I read before. That are more instructions than just a swap. If a cas fails, I have to redo everything. An exchange never fails, I just might not get the result I would like to have (null instead of pointer).

So you want a load + store as swap in a single function (that is optimized).
In that case, please create an issue on bugzilla (issues.dlang.org).

On Wednesday, 23 May 2018 at 13:36:20 UTC, rikki cattermole wrote:
> On 24/05/2018 1:29 AM, Malte wrote:
>> On Wednesday, 23 May 2018 at 13:24:35 UTC, rikki cattermole wrote:
>>> On 24/05/2018 1:20 AM, Malte wrote:
>>>> On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:
>>>>> an idea to lock data by removing the reference:
>>>>>
>>>>> class A
>>>>> {
>>>>>    Lockable!Data data;
>>>>> }
>>>>>
>>>>> [...]
>>>>
>>>> This sounds like you are looking for is an atomic swap. Afaik it doesn't exist in the standard library. You could use asm for the XCHG, but that would make your code x86 dependent.
>>>> I think the easiest way would be to just use a mutex and tryLock.
>>>
>>> What are you talking about? :p
>>>
>>> http://dpldocs.info/experimental-docs/core.atomic.cas.1.html
>> 
>> That is Compare-and-set.
>> To make an exchange using cas I first have to read the value, then write to it expecting to be still the value I read before. That are more instructions than just a swap. If a cas fails, I have to redo everything. An exchange never fails, I just might not get the result I would like to have (null instead of pointer).
>
> So you want a load + store as swap in a single function (that is optimized).
> In that case, please create an issue on bugzilla (issues.dlang.org).

No, as I said, that is already one instruction on X86: https://www.felixcloutier.com/x86/XCHG.html
Just being able to use that instruction with the standard library would be good.

You could also use it with compiler intrinsics. Something like
>import ldc.intrinsics;
>T* tryGetPtr(T)(T** a) {
>    return cast(T*)llvm_atomic_rmw_xchg!size_t(cast(shared(size_t)*)a, 0);
>}
>void restorePtr(T)(T** a, T* b) {
>    llvm_atomic_rmw_xchg!size_t(cast(shared(size_t)*)a,cast(size_t)b);
>}

I would just go with mutexes unless your really need to go that low level though, much saner.