On 5/11/2014 1:59 PM, Timon Gehr wrote:
> On 05/11/2014 10:05 PM, Walter Bright wrote:
>> That's clearly an additional benefit of the borrowed pointer notion. But
>> have you examined generated Rust code for the cost of inc/dec? I
>> haven't, but I don't see any way they could avoid this (very expensive)
>> cost without borrowed pointers.
> Sure, but performance is the additional benefit.

One constant theme in this thread, one I find baffling, is the regular dismissal of the performance implications of inc/dec. Borrowed pointers are not necessary to support raw pointers - this can be (and is in some systems) supported by simply wrapping the raw pointer with a dummy reference count.

The reason for borrowed pointers is performance. Rust would be non-viable without them.

I strongly suggest writing a snippet in [[insert your favorite proven technology RC language here]] and disassembling the result, and have a look at what inc/dec entails.


>> The thing is, if the compiler is capable of figuring out these lifetimes
>> by examining the code,
>
> There are explicit lifetime annotations in function signatures.

Yes, because the compiler cannot figure it out itself, so the programmer has to annotate.


> It is simply not true that type systems are inherently restricted to checking
> trivial properties. They can be made as strong as mathematical logic without
> much fuss.

Again, Rust would not need borrowed pointers nor the annotations for them if this knowledge could be deduced by the compiler. Heck, if the compiler can deduce lifetimes accurately, you can get rid of GC and RC, and just have the compiler insert malloc/free in the right spots.

Note that there is a Java version that does this partway, sometimes it will replace a GC object with a stack allocated one if it is successful in deducing that the object lifetime does not exceed the lifetime of the function.


>> Yes, one is & and the other is @.
>
> No, actually currently one is & and the other is RC<T> AFAIK.

Then Rust changed again. The document I read on borrowed pointers was likely out of date, though it had no date on it.


> RC<T> is not more general. It cannot refer to stack-allocated data, for instance.

So there is no general pointer type that has an unbounded lifetime?


> Sure, borrowing is very lightweight, but ultimately what is most important is
> that it solves the problem of multiple incompatible pointer types and makes the
> type system more expressive as well.

Adding more pointer types makes a type system more expressive, by definition.


> A function that uses none of the specific pointer capabilities is more general,
> so what other choice of 'default' makes sense?

A function that doesn't have restrictions on what can be done with the pointers passed to it. Borrowed pointers have restrictions on their usage - this is explicitly stated in the Rust documentation.


> Convenience and reusable functions means using borrowed pointers whenever possible.

Of course. And writing fast code means making your functions fast whenever possible!


>> It seems clear that the decisions of borrow/managed are going to pervade
>> Rust code.
>
> But they are often obvious.

I've written a lot of ref counted code in the past, enough to know that such is a pretty optimistic statement. Dealing with was a significant and ongoing drain on my time, especially when the programs and data structures got more complex.


> They are using Rust to write a safe and performant web browser while developing
> the language.

Sure. But that browser hasn't been released yet. Consider that I've written safe and performant code in D, but others tell me I am unique and that I cannot expect average programmers to get it right.

I repeatedly point out to Manu that he can write performant code in D that does not suffer from GC stalls, and he repeatedly replies that he has to work with average programmers who are not capable of doing this.

So while I have no doubt that the Mozilla team may be very effective at using Rust and making it shine, that may not be transferable to the larger community.


> Borrowed pointers are not even superficially similar to near*. They are
> compatible with everything else, because they can store data that was borrowed
> from anywhere else.

As long as those pointers don't escape. Am I right in that one cannot store a borrowed pointer into a global data structure? The similarity is that there are one way conversions from one to the other, and one of the types is more general. I infer from your other statements about Rust that it doesn't actually have a general pointer type.

On 2014-05-11 21:41:10 +0000, Walter Bright <newshound2@digitalmars.com> said:

> Your proposal still relies on a GC to provide the memory safety, and has no inherent protection against GC pauses. Your idea has a lot of merit, but it is a hybrid ARC/GC system.

If you thread carefully you can disable GC collections at runtime and not suffer GC pauses. If you have no cycles and no gc_only pointers, then you won't run out of memory.

I think if this thread has proven something, it's that people need to be able to choose their memory management policy when the default is unsatisfactory. I'm trying to find a way to do that, a way to disable one side or the other if it is poisonous to your particular application. It is a hybrid system I'm suggesting, no doubt. It'd also be an interesting experiment, if someone wants to take it.


> As long as C++/CX and O-C are brought out here as proven, successful examples for D to emulate here, and there is no acknowledgement that they are not remotely memory safe, I need to continue to point this out.

You should not say that ARC is not safe then, you should say instead that ARC in those languages has to be supplemented with unsafe code to be fast enough. That statement I can agree with. Taking the shortcut saying simply "ARC is unsafe" is misleading.


-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

Am Sun, 11 May 2014 14:41:10 -0700
schrieb Walter Bright <newshound2@digitalmars.com>:

> Your proposal still relies on a GC to provide the memory safety, […] it is a hybrid ARC/GC system.

But I thought ARC cannot be designed without GC to resolve cycles. Or is your comment pure rethoric?

-- 
Marco

On Sunday, 11 May 2014 at 20:45:08 UTC, Rainer Schuetze wrote:
>
>
> On 11.05.2014 22:33, Walter Bright wrote:
>>
>>>> The Boehm collector cannot move objects around, the D one can.
>>> Oh it can? Really?
>>
>> Yes. D, for example, requires that objects not be self-referential for
>> this reason.
>
> I don't think the GC would have problems with fixing up internal pointers to the object itself. self-referential is prohibited to allow moving structures by memcpy, e.g. as return value.

Does this mean that you cannot safely implement something as basic as a circular linked list?

On 5/11/2014 9:47 PM, Marco Leise wrote:
> Am Sun, 11 May 2014 14:41:10 -0700
> schrieb Walter Bright <newshound2@digitalmars.com>:
>
>> Your proposal still relies on a GC to provide the memory safety,
>> […] it is a hybrid ARC/GC system.
>
> But I thought ARC cannot be designed without GC to resolve
> cycles.

It can be, there are various schemes to deal with that, including "don't create cycles". GC is just one of them.

http://en.wikipedia.org/wiki/Reference_counting#Dealing_with_reference_cycles

On 5/11/2014 8:44 PM, Michel Fortin wrote:
>> As long as C++/CX and O-C are brought out here as proven, successful examples
>> for D to emulate here, and there is no acknowledgement that they are not
>> remotely memory safe, I need to continue to point this out.
>
> You should not say that ARC is not safe then, you should say instead that ARC in
> those languages has to be supplemented with unsafe code to be fast enough.

And I did, probably at least a dozen times in this thread, including quotes you replied to specifically: "Again, O-C and C++/CX ARC are not memory safe because in order to make it perform they provide unsafe escapes from it."


I don't know how I could be clearer.


In the spirit of being clear, I am opposed to any replacement for the GC that:

1. makes it impossible to meet or exceed C++ performance levels
2. is not memory safe
3. implies shipping multiple builds of Phobos, each using a different memory management scheme

I am not opposed to ARC as a supplement, even if it is @system only, in particular one that can interact with O-C.

Am Sun, 11 May 2014 17:50:25 -0700
schrieb Walter Bright <newshound2@digitalmars.com>:

> As long as those pointers don't escape. Am I right in that one cannot store a borrowed pointer into a global data structure?

Right, and that's the point and entirely positive-to-do™. Your general purpose function does not know how the memory was allocated, that it receives a pointer to. In particular it must not assume that it is safe to keep a reference to it as there are several memory management schemes that are incompatible with that, like reference counting or stack allocations.

Expanding on these two, Rust can now safely use _more_ allocation schemes with functions that take borrowed pointers than is safely possible in D!

RC pointers:
You cannot pass them as raw pointers in D. In Rust they can
be passed as borrowed.

Stack pointers:
Not allowed in D in @safe code and inherently unsafe in
@system code. Again this is safe to do in Rust due to
borrowing.

> The similarity is that there are one way conversions from one to the other, and one of the types is more general. I infer from your other statements about Rust that it doesn't actually have a general pointer type.

Yes it does: http://static.rust-lang.org/doc/0.10/guide-unsafe.html#raw-pointers

But the design principle in Rust is to only have them in @system code (speaking in D terms), in particular to interface with C.

Turning the argument back to D and assuming you wrote a function that takes a raw pointer because you plan to store it in a global variable. How do you make sure you get a pointer to something with infinite life-time? Let me answer this: You either use GC pointers exclusively or you rely on the convention that the function takes ownership of the memory. The former is impractical and the latter cannot be statically enforced.

Borrowed pointers add an @safe way to deal with the situation
in all contexts where you don't need to store a reference.
But if you _do_ need that capability: ask explicitly for GC
pointers as they can guarantee unlimited life-time.
If that's still too restrictive mark it @system and use raw
pointers (in Rust: unsafe keyword).

Finally, this is not Rust vs. D, because D has had borrowed pointer function arguments since ages as well - maybe even longer than Rust. The semantics of "in/scope" were just never fully implemented. Once this is done we can also write:

@safe void main()
{
	auto stack = 42;
	foo(&stack);
}

@safe void foo(scope int*);

-- 
Marco

On 12.05.2014 06:57, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> On Sunday, 11 May 2014 at 20:45:08 UTC, Rainer Schuetze wrote:
>>
>>
>> On 11.05.2014 22:33, Walter Bright wrote:
>>>
>>>>> The Boehm collector cannot move objects around, the D one can.
>>>> Oh it can? Really?
>>>
>>> Yes. D, for example, requires that objects not be self-referential for
>>> this reason.
>>
>> I don't think the GC would have problems with fixing up internal
>> pointers to the object itself. self-referential is prohibited to allow
>> moving structures by memcpy, e.g. as return value.
>
> Does this mean that you cannot safely implement something as basic as a
> circular linked list?

Depends on your implementation, but I would not expect that the nodes of your linked list could be subject to being moved (mostly happens on the stack AFAICT), as pointers to the node from other nodes would become invalid, too.

On 12 May 2014 10:50, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
>> They are using Rust to write a safe and performant web browser while
>> developing
>> the language.
>
>
> Sure. But that browser hasn't been released yet. Consider that I've written safe and performant code in D, but others tell me I am unique and that I cannot expect average programmers to get it right.
>
> I repeatedly point out to Manu that he can write performant code in D that does not suffer from GC stalls, and he repeatedly replies that he has to work with average programmers who are not capable of doing this.

What? You've never offered me a practical solution.

You tell me I have to sacrifice any dependency on libraries
(ridiculous), and all the modern conveniences and safety of automatic
memory management to do it!
Indeed, average programmers are a real-life practical problem, and
it's not just them, I also appreciate the convenience offered
personally. I only have one life, I *really* appreciate saving time on
mundane and otherwise inconsequential tasks.
Tell me some other reasons why I would be attracted to D? Take that
all away, and what's the point? Automating some boilerplate is nice,
but it's not the motivating reason for a wholesale adoption.

You haven't told me how I can use the GC (or whatever memory
management scheme, I really don't care) in the low frequency code
(again, read: almost all code ever), and not have it interfere with
the high frequency code.
This is the fundamental problem with the GC. I can't use it
***ANYWHERE***, including any libraries I link. You can't isolate a
GC, it's effects are not localised. If you could, maybe it'd be more
workable... but even if it were properly concurrent and didn't halt
the realtime threads when it collected, it's still totally impractical
for any background thread to freeze for 10s-100s of ms while it runs a
collect because I received a network packet which needs to be sent
somewhere for processing or whatever.

What do I do?


> So while I have no doubt that the Mozilla team may be very effective at using Rust and making it shine, that may not be transferable to the larger community.

On Monday, 12 May 2014 at 00:44:54 UTC, Andrei Alexandrescu wrote:
> On 5/11/14, 2:49 PM, ponce wrote:
>> On Sunday, 11 May 2014 at 21:43:06 UTC, sclytrack wrote:
>>>
>>> There is very little use of "@", it's mostly  "&" and "~". Heck I
>>> didn't find any @ while casually browsing the code. It's like they are
>>> not using it at all.
>>>
>>
>> Similarly in current C++ std::shared_ptr is barely there while
>> std::unique_ptr is all over the place.
>
> The right tally here would include bald pointers as well. Also, I'm quite surprised by the confidence of this assertion. -- Andrei

Well I wa'm probably much biased indeed. I have no data.