On Monday, 12 May 2014 at 00:44:54 UTC, Andrei Alexandrescu wrote:
> On 5/11/14, 2:49 PM, ponce wrote:
>> On Sunday, 11 May 2014 at 21:43:06 UTC, sclytrack wrote:
>>>
>>> There is very little use of "@", it's mostly  "&" and "~". Heck I
>>> didn't find any @ while casually browsing the code. It's like they are
>>> not using it at all.
>>>
>>
>> Similarly in current C++ std::shared_ptr is barely there while
>> std::unique_ptr is all over the place.
>
> The right tally here would include bald pointers as well. Also, I'm quite surprised by the confidence of this assertion. -- Andrei

Well I was talking with no data indeed.

On Monday, 12 May 2014 at 07:12:29 UTC, Manu via Digitalmars-d wrote:
> You haven't told me how I can use the GC (or whatever memory
> management scheme, I really don't care) in the low frequency code
> (again, read: almost all code ever), and not have it interfere with
> the high frequency code.

You will like Don's talk this year ;)
Game dev code in not just any low level code though.

On 12 May 2014 17:24, Dicebot via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Monday, 12 May 2014 at 07:12:29 UTC, Manu via Digitalmars-d wrote:
>>
>> You haven't told me how I can use the GC (or whatever memory management scheme, I really don't care) in the low frequency code (again, read: almost all code ever), and not have it interfere with the high frequency code.
>
>
> You will like Don's talk this year ;)

I'm super-disappointed I can't make it this year! We were evicted from our house, have to move, and I can't bail for a week and leave that all on my mrs while she kicks along the fulltime job :(

> Game dev code in not just any low level code though.

It's not just low-level code. It's also very high level code. Gamedev
is a unique (and extremely interesting) field which combines and
tightly integrates almost every imaginable field of software
engineering. I can't think of any that are left out, or any other
industries that also share this property.
I argue, if you can make a modern high-end game in a language, it can
do anything.

On Monday, 12 May 2014 at 00:50:24 UTC, Walter Bright wrote:
> On 5/11/2014 1:59 PM, Timon Gehr wrote:
>> Borrowed pointers are not even superficially similar to near*. They are
>> compatible with everything else, because they can store data that was borrowed
>> from anywhere else.
>
> As long as those pointers don't escape. Am I right in that one cannot store a borrowed pointer into a global data structure?

Perhaps:

struct Test {
    n: &'static int, // [1]
    m: int
}

static val: int = 123;
static mut t: Test = Test { n: &'static val, m: 0 };

fn main() {
    unsafe { // [2]
        let p = &mut t.m;
        *p = 456;
        println!("{} {}", *t.n, t.m); // prints: 123 456
    }
}

[1]: In order to create a static instance of 'Test', the 'n' field (which is a borrowed pointer) must be specified as to be pointing at a static immutable (int) variable.

[2]: Any use of static mutable data requires the use of an 'unsafe' block (similar to @trusted in D)

On Monday, 12 May 2014 at 08:10:43 UTC, Tommi wrote:
> Perhaps: [..]

Somewhat surprisingly to me, you can later on change the borrowed pointer in the mutable static 'Test' to point at a mutable static int:

struct Test {
    n: &'static int
}

static old: int = 111;
static mut new: int = 222;
static mut t: Test = Test { n: &'static old };

fn main() {
    unsafe {
        println!("{}", *t.n); // prints: 111
        t.n = &new; // Can point to a different static
        println!("{}", *t.n); // prints: 222
        // ...but can't point to a local, e.g:
        // let v = 123;
        // t.n = &v; // error: `v` does not live long enough
        new = 333;
        println!("{}", *t.n); // prints: 333
    }
}

On 5/12/2014 12:12 AM, Manu via Digitalmars-d wrote:
> What? You've never offered me a practical solution.

I have, you've just rejected them.


> What do I do?

1. you can simply do C++ style memory management. shared_ptr<>, etc.

2. you can have the non-pausible code running in a thread that is not registered with the gc, so the gc won't pause it. This requires that this thread not allocate gc memory, but it can use gc memory allocated by other threads, as long as those other threads retain a root to it.

3. D allows you to create and use any memory management scheme you want. You are simply not locked into GC. For example, I rewrote my Empire game into D and it did not do any allocation at all - no GC, not even malloc. I know that you'll need to do allocation, I'm just pointing out that GC allocations and pauses are hardly inevitable.

4. for my part, I have implemented @nogc so you can track down gc usage in code. I have also been working towards refactoring Phobos to eliminate unnecessary GC allocations and provide alternatives that do not allocate GC memory. Unfortunately, these PR's just sit there.

5. you can divide your app into multiple processes that communicate via interprocess communication. One of them pausing will not pause the others. You can even do things like turn off the GC collections in those processes, and when they run out of memory just kill them and restart them. (This is not an absurd idea, I've heard of people doing that effectively.)

6. If you call C++ libs, they won't be allocating memory with the D GC. D code can call C++ code. If you run those C++ libs in separate threads, they won't get paused, either (see (2)).

7. The Warp program I wrote avoids GC pauses by allocating ephemeral memory with malloc/free, and (ironically) only using GC for persistent data structures that should never be free'd. Then, I just turned off GC collections, because they'd never free anything anyway.

8. you can disable and enable collections, and you can cause collections to be run at times when nothing is happening (like when the user has not input anything for a while).


The point is, the fact that D has 'new' that allocates GC memory simply does not mean you are obliged to use it. The GC is not going to pause your program if you don't allocate with it. Nor will it ever run a collection at uncontrollable, random, asynchronous times.

On 5/11/2014 10:57 PM, Marco Leise wrote:
> Am Sun, 11 May 2014 17:50:25 -0700
> schrieb Walter Bright <newshound2@digitalmars.com>:
>
>> As long as those pointers don't escape. Am I right in that one cannot store a
>> borrowed pointer into a global data structure?
>
> Right, and that's the point and entirely positive-to-do™.

This means that a global data structure in Rust has to decide what memory allocation scheme its contents must use, and cannot (without tagging) mix memory allocation schemes.

For example, let's say a compiler has internally a single hash table of strings. With a GC, those strings can be statically allocated, or on the GC heap, or anything with a lifetime longer than the table's. But I don't see how this could work in Rust.

Walter Bright:

> But I don't see how this could work in Rust.

Ask it to competent Rust developers/programmers.

Bye,
bearophile

On Monday, 12 May 2014 at 08:45:56 UTC, Walter Bright wrote:
> On 5/12/2014 12:12 AM, Manu via Digitalmars-d wrote:
>> What? You've never offered me a practical solution.
>
> I have, you've just rejected them.
>
>
>> What do I do?
>
> 1. you can simply do C++ style memory management. shared_ptr<>, etc.
>
> 2. you can have the non-pausible code running in a thread that is not registered with the gc, so the gc won't pause it. This requires that this thread not allocate gc memory, but it can use gc memory allocated by other threads, as long as those other threads retain a root to it.
>
> 3. D allows you to create and use any memory management scheme you want. You are simply not locked into GC. For example, I rewrote my Empire game into D and it did not do any allocation at all - no GC, not even malloc. I know that you'll need to do allocation, I'm just pointing out that GC allocations and pauses are hardly inevitable.
>
> 4. for my part, I have implemented @nogc so you can track down gc usage in code. I have also been working towards refactoring Phobos to eliminate unnecessary GC allocations and provide alternatives that do not allocate GC memory. Unfortunately, these PR's just sit there.
>
> 5. you can divide your app into multiple processes that communicate via interprocess communication. One of them pausing will not pause the others. You can even do things like turn off the GC collections in those processes, and when they run out of memory just kill them and restart them. (This is not an absurd idea, I've heard of people doing that effectively.)
>
> 6. If you call C++ libs, they won't be allocating memory with the D GC. D code can call C++ code. If you run those C++ libs in separate threads, they won't get paused, either (see (2)).
>
> 7. The Warp program I wrote avoids GC pauses by allocating ephemeral memory with malloc/free, and (ironically) only using GC for persistent data structures that should never be free'd. Then, I just turned off GC collections, because they'd never free anything anyway.
>
> 8. you can disable and enable collections, and you can cause collections to be run at times when nothing is happening (like when the user has not input anything for a while).
>
>
> The point is, the fact that D has 'new' that allocates GC memory simply does not mean you are obliged to use it. The GC is not going to pause your program if you don't allocate with it. Nor will it ever run a collection at uncontrollable, random, asynchronous times.

The only solutions to the libraries problem that I can see here require drastic separation of calls to said libraries from any even vaguely time critical code. This is quite restrictive.

Yes, calling badly optimised libraries from a hot loop is a bad idea anyway, but the GC changes this from

"well it might take a little more time than usual, but we can spare a few nano-seconds and it'll show up easily in the profiler"

to

"it might, sometimes, cause the GC to run a full collection on our 3.96 / 4.00 GB heap with an associated half-second pause."

And here we go again, "I can't use that library, it's memory management scheme is incompatible with my needs, I'll have to rewrite it myself..."

On Monday, 12 May 2014 at 09:05:39 UTC, John Colvin wrote:
> On Monday, 12 May 2014 at 08:45:56 UTC, Walter Bright wrote:
> The only solutions to the libraries problem that I can see here require drastic separation of calls to said libraries from any even vaguely time critical code. This is quite restrictive.

I think this is more of library writing culture problem than engineering problem. High quality library shouldn't rely on any internal allocations at all, deferring this decision to user code. Otherwise you will eventually have problems, GC or not.