Sean Kelly wrote:
> Frits van Bommel wrote:
>> http://www-cs.canisius.edu/~hertzm/bc.html has a nice idea on support the OS could give to garbage collection: they modified Linux to notify the process when it was about to get some of its memory swapped out. Then the process could perform a quick GC (focusing on freeing up pages of memory) and tell the OS which pages to unmap or swap out. The GC freeing pages means the GC can tell the OS memory manager that certain pages can be thrown out without saving them to disk first (no disk writes is faster than disk writes :) ).
> 
> I can think of at least two papers on this topic for anyone interested.  One is "Reducing Garbage Collector Cache Missed" by Hans Boehm, and the other is "Garbage Collection Without Paging" by Hertz, Feng, and Berger.  Both should be available with a bit of googling.

The second one is actually available at the link I posted, but thanks for the first one, I don't think I read that one yet.

Dan wrote:
> Lately a big thing in kernel development world is to move everything possible out of the kernel.

I don't know how lately should be defined in that context, but when I did the research for my diploma thesis (design and implementation of a RTOS-Kernel on a Fujitsu microcontroller) in 2001 there where already
numerous papers to be found on the exokernel subject.
It was a fascinating idea then, and still am. I sure like the approach.

> Just my view.  As I said, I'm interested in helping.

Maybe I'd also be able to throw in a few hours of work... :)

Happy hacking, 0ffh

0ffh Wrote:
> RTOS-Kernel on a Fujitsu microcontroller) in 2001 there where already
> numerous papers to be found on the exokernel subject.
> It was a fascinating idea then, and still....

Yeah, the MIT boys back then did a paper on it, and make Xok, which IMHO demonstrated the concept but otherwise sucked because they used the wrong strategies to 'import' code.

When you have an exokernel, you need to be able to delegate gc, thread/process switching (cpu driver), paging and gc (memory driver), port drivers, dma and such out.  Doing that efficiently IMHO requires using shared memory and priveleges rather than strictly RPC.

I think I'll probably spend some time on this code this weekend; if I *ever* get a URL.

Walter Bright wrote:
> David B. Held wrote:
>> That being said, I think it would be *particularly* clever of you to use your start-from-scratch kernel to explore new ideas in multi-processing and massive multithreading, but this would imply having decent multi-CPU hardware to play with.
> 
> I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data.

(That doesn't exist in Win32, does it?)

L.

Lionello Lunesu wrote:
> Walter Bright wrote:
>> David B. Held wrote:
>>> That being said, I think it would be *particularly* clever of you to use your start-from-scratch kernel to explore new ideas in multi-processing and massive multithreading, but this would imply having decent multi-CPU hardware to play with.
>>
>> I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.
> 
> I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data.
> 
> (That doesn't exist in Win32, does it?)

I don't think so.  If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.


Sean

Sean Kelly wrote:
> Lionello Lunesu wrote:
>> [...]
>> I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data.
>>
>> (That doesn't exist in Win32, does it?)
> 
> I don't think so.  If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.

Well, there's at least two solutions to this.  One is to use infinite memory, in which case you can allocate at addresses that are infinitely far apart, allowing for unlimited growth in data structures.

The second is to use quantum superposition to allow storage of distinct data at the same location.  The trick is getting the state to collapse in a way that makes all the data recoverable, which is nothing more than an engineering detail, really.  We should get an intern to work on these projects over the summer.

Dave

David B. Held wrote:
> Sean Kelly wrote:
>> Lionello Lunesu wrote:
>>> [...]
>>> I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data.
>>>
>>> (That doesn't exist in Win32, does it?)
>>
>> I don't think so.  If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.
> 
> Well, there's at least two solutions to this.  One is to use infinite memory, in which case you can allocate at addresses that are infinitely far apart, allowing for unlimited growth in data structures.
> 
> The second is to use quantum superposition to allow storage of distinct data at the same location.  The trick is getting the state to collapse in a way that makes all the data recoverable, which is nothing more than an engineering detail, really.  We should get an intern to work on these projects over the summer.

And Google's "summer of code" has already stopped taking applications, too.  Darn it!


Sean

"Sean Kelly" <sean@f4.ca> wrote in message news:f09ouq$2ium$1@digitalmars.com...
> Lionello Lunesu wrote:
>> Walter Bright wrote:
>>> David B. Held wrote:
>>>> That being said, I think it would be *particularly* clever of you to use your start-from-scratch kernel to explore new ideas in multi-processing and massive multithreading, but this would imply having decent multi-CPU hardware to play with.
>>>
>>> I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.
>>
>> I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data.
>>
>> (That doesn't exist in Win32, does it?)
>
> I don't think so.  If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.

I don't care if there address changes, just as long as there's no data being copied physically. I mean, a block of continuous memory might already be all-over-the-place in physical memory. That's what I want: a VirtualRealloc that returns the address of a new virtual memory block, but which is using the same physical memory.. 'Free' array resizes, surely this is worth something!

L.

Lionello Lunesu wrote:
> "Sean Kelly" <sean@f4.ca> wrote in message news:f09ouq$2ium$1@digitalmars.com...
>> Lionello Lunesu wrote:
>>> Walter Bright wrote:
>>>> David B. Held wrote:
>>>>> That being said, I think it would be *particularly* clever of you to use your start-from-scratch kernel to explore new ideas in multi-processing and massive multithreading, but this would imply having decent multi-CPU hardware to play with.
>>>> I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.
>>> I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data.
>>>
>>> (That doesn't exist in Win32, does it?)
>> I don't think so.  If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.
> 
> I don't care if there address changes, just as long as there's no data being copied physically. I mean, a block of continuous memory might already be all-over-the-place in physical memory. That's what I want: a VirtualRealloc that returns the address of a new virtual memory block, but which is using the same physical memory.. 'Free' array resizes, surely this is worth something!

Hm, assuming there is nothing else of the page(s) but the data to move, it should be possible to do so without explicit copying.  I have no idea is VirtualAlloc can do this, but it's definitely possible.


Sean

Sean Kelly Wrote:
> Lionello Lunesu wrote:
> > I don't care if there address changes, just as long as there's no data being copied physically. I mean, a block of continuous memory might already be all-over-the-place in physical memory. That's what I want: a VirtualRealloc that returns the address of a new virtual memory block, but which is using the same physical memory.. 'Free' array resizes, surely this is worth something!
> 
> Hm, assuming there is nothing else of the page(s) but the data to move, it should be possible to do so without explicit copying.  I have no idea is VirtualAlloc can do this, but it's definitely possible.

Oh my!

On the x86, this sort of implementation would slaughter your performance, as you'd get massive numbers of cache and page misses.

You could theoretically implement a system on the x86 that would virtualize memory locations arbitrarily, but.... ewww...

I see two effective ways to realloc memory in a space too small for it:

1) move the data you are trying to reallocate to a larger space.

2) move the data that is confining it; which requires updating pointers and such to that data, which you can't realistically track on the x86.