On Friday, 24 May 2013 at 07:48:49 UTC, Vladimir Panteleev wrote:
> I don't know why you say that. The satellite process would be the same executable file as the main process, but started with a special switch (or environment variable), which will be handled by the D runtime. Since the two processes use the same image, the executable code will share the same pages in memory, resulting in a small overhead.

I'm fairly sure windows only shares memory between instances of the same DLL, not executables.

> When implementing WinMain, you have to call the runtime initialization function, which will initialize the GC. GC initialization for the satellite process need not exit.

You're still executing part of the user's code and it's going to be exceedingly hard to satisfactorily explain why the main function will be called twice except the second time it will mysteriously stop half-way through because the runtime init function doesn't return...

> I don't think this is true. Although the image subsystem is auto-detected by the entry point you're using (CONSOLE for main, WINDOWS for WinMain), you can specify it explicitly using the /SUBSYSTEM linker switch (/SUBSYSTEM:WINDOWS).

The only time I've been able to get it to not show a console window is when using WinMain...

> Sorry? Could you provide an example (with a specific security software package)?

Any security software with the active defense type stuff, Commodo does it for example. The things they ask you about vary and it's completely pointless IMO, but they do ask for permission for practically everything and with two processes that's twice as much to ask you about.

Anyway, my main point is that it's not the kind of thing you want to impose as standard on all D applications, there may be other problems it causes depending on what the program is for. If it exists at all it should be opt-in and the default should still be as performant as possible.

> I've tried writing a generational GC for D that used page protection for write barriers a while ago. IIRC, I ran into performance issues (the page faults were rather expensive).

Hmm, it shouldn't really be much slower than the technique Leandro was using - both rely on the exact same number of page faults - and his results looked very promising.

On Friday, 24 May 2013 at 09:56:25 UTC, Diggory wrote:
> The only time I've been able to get it to not show a console window is when using WinMain...

OK, nvm, it seems it's just that the "SUBSYSTEM" setting in VisualD has no effect, you have to set it through a .def file.

On Friday, 24 May 2013 at 09:56:25 UTC, Diggory wrote:
> I'm fairly sure windows only shares memory between instances of the same DLL, not executables.

Oh? I thought Windows treats all modules in the same way. Do you have a source?

> You're still executing part of the user's code and it's going to be exceedingly hard to satisfactorily explain why the main function will be called twice except the second time it will mysteriously stop half-way through because the runtime init function doesn't return...

I don't see that as a problem, considering such a GC would need to be opt-in.

> Any security software with the active defense type stuff, Commodo does it for example. The things they ask you about vary and it's completely pointless IMO, but they do ask for permission for practically everything and with two processes that's twice as much to ask you about.

The reason why I asked for a specific example is that I don't know of any actual case when this will be a problem in practice.

For one, I've never seen security software that reacts differently on two instances of the same executable - usually, any permissions you set are for the .exe file, or more generic rules (like port numbers).

For two, the satellite process does not need any kinds of permissions. Even the requirement of accessing other processes' memory that the straight-forward approach needs can be avoided by allowing the satellite process to inherit a handle to the main process.

> Anyway, my main point is that it's not the kind of thing you want to impose as standard on all D applications, there may be other problems it causes depending on what the program is for. If it exists at all it should be opt-in and the default should still be as performant as possible.

Yes, which is why I also think it should be opt-in.

>> I've tried writing a generational GC for D that used page protection for write barriers a while ago. IIRC, I ran into performance issues (the page faults were rather expensive).
>
> Hmm, it shouldn't really be much slower than the technique Leandro was using - both rely on the exact same number of page faults - and his results looked very promising.

Leandro wasn't using Windows ;)

On 05/24/13 08:52, Diggory wrote:
> After doing some further investigation I think I've found a fairly awesome way of doing garbage collection on windows, hopefully linux has a similar mechanism. It doesn't require memory mapped files or anything special, it can be done retroactively, and it allows a kind of reverse copy on write which is what's actually needed.
> 
> When the GC is run:
> - Use VirtualProtect to mark all mutable memory pages as read-only
> - Add a vectored exception handler to handle the access violation exception
> - Resume the GC thread
> 
> In the exception handler:
> - Copy the page being modified to a new page

A page fault per every page written to between every GC run + a user space callback for every such fault + a page sized copy every time is not going to perform very well...

(There are ways to avoid these costs, but no idea if it's easily doable on
windows...)

artur

On Friday, 24 May 2013 at 10:20:12 UTC, Vladimir Panteleev wrote:
> Leandro wasn't using Windows ;)

True but assuming windows isn't completely degenerate the
overhead of a page fault is going to be pretty close to identical
on the same cpu architecture.

On Friday, 24 May 2013 at 11:15:21 UTC, Artur Skawina wrote:
> A page fault per every page written to between every GC run + a user
> space callback for every such fault + a page sized copy every time is
> not going to perform very well...
>
> (There are ways to avoid these costs, but no idea if it's easily doable on
> windows...)
>
> artur

It's only between the time a GC run is started and the time it
finishes, exactly the same as in the talk. A generational GC
should be able to do complete mark cycles fairly infrequently.

On Monday, 20 May 2013 at 12:50:23 UTC, Andrei Alexandrescu wrote:
> On reddit:
>
> http://www.reddit.com/r/programming/comments/1eovfu/dconf_2013_day_1_talk_6_concurrent_garbage/
>
>
> Enjoy! Discuss!! Vote!!!
>
> Andrei

I know that this is slightly offtopic, since the topic lately seems to be how to make the GC run generationally or with small footprint (don't stop the world, etc.).

I'd like to know if there is interest in a precise garbage collector. Anyways, here is how .NET does it:

http://blogs.msdn.com/b/abhinaba/archive/2009/03/03/back-to-basics-how-does-the-gc-find-object-references.aspx

It uses a mask stored in the type information of a class. D doesn't have this kind of type info in runtime I guess, but since D is on the verge of supporting multiple dlls/so, the time is now for a small modification to be made in the ABI to support this (if it is ever going to be made).

I know that in 64bits there is less of a problem with data as false pointers, but having a precise garbage collector would make two things possible:

  1) Defragmenting the heap by being able to move references.
  2) Easier to make a generational GC

The following link explains (in the first comment) how .NET distinguishes its own stack frames from non-managed stack frames, by adding a "cookie":

http://stackoverflow.com/questions/10669173/how-does-the-gc-update-references-after-compaction-occurs


--jm

On Friday, May 24, 2013 20:30:54 Juan Manuel Cabo wrote:

> I know that this is slightly offtopic, since the topic lately seems to be how to make the GC run generationally or with small footprint (don't stop the world, etc.).
> 
> I'd like to know if there is interest in a precise garbage collector.

There is interest in it, and Rainer Schütze did a talk on it at DConf. At the current pace (assuming that Andrei actually posts one on Monday even though it's a federal holiday in the US), it'll be posted on June 3rd (and if he skips Monday, then it'll probably be June 5th). And actually, the precise GC changes stand a much better chance of making it into druntime in the short term than any concurrency changes do.

- Jonathan M Davis

On Friday, 24 May 2013 at 19:44:19 UTC, Jonathan M Davis wrote:
> On Friday, May 24, 2013 20:30:54 Juan Manuel Cabo wrote:
>
>> I know that this is slightly offtopic, since the topic lately
>> seems to be how to make the GC run generationally or with small
>> footprint (don't stop the world, etc.).
>> 
>> I'd like to know if there is interest in a precise garbage
>> collector.
>
> There is interest in it, and Rainer Schütze did a talk on it at DConf. At the
> current pace (assuming that Andrei actually posts one on Monday even though
> it's a federal holiday in the US), it'll be posted on June 3rd (and if he
> skips Monday, then it'll probably be June 5th). And actually, the precise GC
> changes stand a much better chance of making it into druntime in the short
> term than any concurrency changes do.
>
> - Jonathan M Davis

Thanks, that is great news!

--jm

On 64-bit windows there is also the "GetWriteWatch" function which lets you access the dirty flag in the page table = no page faults = super efficient concurrent generational GC. Just a shame it doesn't exist on 32-bit systems for some reason.

On 5/24/2013 11:12 PM, Diggory wrote:
> On 64-bit windows there is also the "GetWriteWatch" function which lets
> you access the dirty flag in the page table = no page faults = super
> efficient concurrent generational GC. Just a shame it doesn't exist on
> 32-bit systems for some reason.

There's all sorts of interesting stuff in 64 bit windows :)  The user mode thread scheduler is pretty cool.

On the flip side: 32 bit is in its twilight days, and I am reasonably confident the next game I work on will be the last one that even supports 32 bits.  Then I can finally use all the new 64 bit goodies :)   32 bits will be reserved for phones and tablets (and even then tablets will probably be making the switch pretty soon-ish)