I've been asked for my runtime crash handler with it's CodeView reader earlier in digitalmars.D so here it is:

http://jump.fm/UVYHG

It includes the runtime handler, a PE reader, CodeView reader and crash report window.

So far it supports Windows and some bits are already done for posix platforms. It does a register dump, stack trace with resolved symbol names with their declaration file and line, as well as a listing of all loaded modules.

Sorry for the lack of phobos/tango support, this is part of a runtime project I'm building which should hopefully be released to dsource in a few months. I'm releasing it now so if someone wants to implement a phobos or tango port they're free to do it.

Jeremie Pelletier wrote:
> I've been asked for my runtime crash handler with it's CodeView reader earlier in digitalmars.D so here it is:
> 
> http://jump.fm/UVYHG
> 
> It includes the runtime handler, a PE reader, CodeView reader and crash report window.
> 
> So far it supports Windows and some bits are already done for posix platforms. It does a register dump, stack trace with resolved symbol names with their declaration file and line, as well as a listing of all loaded modules.
> 
> Sorry for the lack of phobos/tango support, this is part of a runtime project I'm building which should hopefully be released to dsource in a few months. I'm releasing it now so if someone wants to implement a phobos or tango port they're free to do it.

Thanks a lot, Jeremie! The code looks really good. I think I'll be stealing the PE && CV parsing parts for Tango :)

I'm curious about the runtime project of yours :o Can you shed a bit of the secret? How will it compare to druntime? What's the target audience? Does it have anything to do with 'meshes and textures'? ;)


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode

Tom S Wrote:

> Jeremie Pelletier wrote:
> > I've been asked for my runtime crash handler with it's CodeView reader earlier in digitalmars.D so here it is:
> > 
> > http://jump.fm/UVYHG
> > 
> > It includes the runtime handler, a PE reader, CodeView reader and crash report window.
> > 
> > So far it supports Windows and some bits are already done for posix platforms. It does a register dump, stack trace with resolved symbol names with their declaration file and line, as well as a listing of all loaded modules.
> > 
> > Sorry for the lack of phobos/tango support, this is part of a runtime project I'm building which should hopefully be released to dsource in a few months. I'm releasing it now so if someone wants to implement a phobos or tango port they're free to do it.
> 
> Thanks a lot, Jeremie! The code looks really good. I think I'll be stealing the PE && CV parsing parts for Tango :)

I'm glad to hear that! Just keep my name in the credits and I'll be happy :)

> I'm curious about the runtime project of yours :o Can you shed a bit of the secret? How will it compare to druntime? What's the target audience? Does it have anything to do with 'meshes and textures'? ;)

Well it started a bit over two years ago, back then there was no druntime, I was digging into phobos to learn more about the language and decided it would be a fun and challenging task to write my own runtime. Then once I had it running stable I found out druntime had been created in the meantime!

In any ways, I learned quite a bit of the language's internals and ABI while coding and debugging it; I began by removing all default paths and libraries from dmd (both in sc.ini and compiling with "--defaultlib= --debuglib=--") and made an empty object.d and built my way up until there was no more compiler crashes (dmd is quite whiny about not having its "intrinsic" symbols declared, mostly the typeinfo's) and all unresolved symbols were added.

Since then the runtime was ported to D2 and kept up to date with every new release, I've been using it for these two years and it works great.

I don't really know how it will compare to druntime, I'm not out there to compete with them really. I use DMD on both windows and posix platforms (Win7 and Ubuntu respectively) so its the only supported compiler right now, haven't tested 64-bit support either. It would be able to support libraries such as phobos and tango with a few changes to the places that call into the compiler runtime or memory manager.

Right now the target audience is me. I haven't really considered who the target audience might be since I touch a bit of everything. I try and make it as generic and as modular as I can, so if someone doesn't need a package they can remove it from their build without getting tons of compile errors. I myself am focusing towards GUI application development with unified 2D/3D rendering and custom controls on top of that rendering as part of the runtime library, among many other things.

So you could say it has to do with meshes and textures, although I haven't gotten to making interfaces for that part yet. The most I did on that level with my runtime was to write a reader for Blizzard's MPQ archives and extract model and texture data to render it in opengl, but apart from the MPQ reader it was all pretty ugly mixing win32 and gl code along with other stuff in the same routine to get the model on screen as fast as possible ;)

I really am just beginning to notice people are interested in my project, I don't know when I'll put it on dsource and/or sourceforge. And given that D2 is still under heavy development it wouldn't be practical right now. I don't mind releasing bits and pieces of it but I would much rather see the first public release of it to be an usable one, although the compiler implementation part is pretty much stable at this point, if there's enough demand for it I might begin with that.

Jeremie

On Sun, 20 Sep 2009 08:25:30 +0300, Jeremie Pelletier <jeremiep@gmail.com> wrote:

>> I'm curious about the runtime project of yours :o Can you shed a bit of
>> the secret? How will it compare to druntime? What's the target audience?
>> Does it have anything to do with 'meshes and textures'? ;)
>
> [snip]

This is cool stuff. I'd be very interested in finding out how you handled memory management / garbage collection. Or have you used the standard GC that comes with Phobos/Tango?

-- 
Best regards,
 Vladimir                          mailto:thecybershadow@gmail.com

Vladimir Panteleev Wrote:

> On Sun, 20 Sep 2009 08:25:30 +0300, Jeremie Pelletier <jeremiep@gmail.com> wrote:
> 
> >> I'm curious about the runtime project of yours :o Can you shed a bit of the secret? How will it compare to druntime? What's the target audience? Does it have anything to do with 'meshes and textures'? ;)
> >
> > [snip]
> 
> This is cool stuff. I'd be very interested in finding out how you handled memory management / garbage collection. Or have you used the standard GC that comes with Phobos/Tango?

I forked the original D1 gc and rewrote it almost entirely. I made numerous speed improvements over it, such as greatly reducing the number of pool lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan' bitfields from pools and rewrote the garbage collector to only use the 'mark' one, I also merged the sweep and freelist rebuild phases together to save on the overhead of yet another loop over the entire working set, also added support for the gc to free memory pools if they're entirely free.

I have it on pastebin if you want to take a look: http://pastebin.com/f7a3b4c4a

Jeremie

Jeremie Pelletier:

>I forked the original D1 gc and rewrote it almost entirely. I made numerous speed improvements over it, such as greatly reducing the number of pool lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan' bitfields from pools and rewrote the garbage collector to only use the 'mark' one, I also merged the sweep and freelist rebuild phases together to save on the overhead of yet another loop over the entire working set, also added support for the gc to free memory pools if they're entirely free.<

That looks like lot of work. Too bad we don't have a suite of GC-heavy benchmarks to test it. (Maybe Lucarella has collected something in the meantime).

Bye,
bearophile

bearophile wrote:
> Jeremie Pelletier:
> 
>> I forked the original D1 gc and rewrote it almost entirely. I made numerous speed improvements over it, such as greatly reducing the number of pool lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan' bitfields from pools and rewrote the garbage collector to only use the 'mark' one, I also merged the sweep and freelist rebuild phases together to save on the overhead of yet another loop over the entire working set, also added support for the gc to free memory pools if they're entirely free.<
> 
> That looks like lot of work. Too bad we don't have a suite of GC-heavy benchmarks to test it. (Maybe Lucarella has collected something in the meantime).
> 
> Bye,
> bearophile

I couldn't agree more about the need for a GC benchmark suite, writing this module involved some of the hardest debugging work I ever did, even with printfs, memory dumps and windbg.

I do believe it to be stable enough for general usage in its current state. Its been some time now since I last had any issues with it.

If you come across such a benchmark suite, let me know, I would love to see how it performs against druntime's GC.

Jeremie

Jeremie Pelletier:

> I couldn't agree more about the need for a GC benchmark suite,

I am slowly creating one (a general one) that later may be used for LDC :-)


> If you come across such a benchmark suite, let me know, I would love to see how it performs against druntime's GC.

Similar GC benchmark suites don't grow on trees. I don't think one exists yet for D (ask to Lucarella too).
You can start testing your runtime against programs that you can find in the "binarytrees" and "gc" subdirectories here:
http://www.fantascienza.net/leonardo/js/slow_d.zip
And I'd like to know the results with both runtimes :-)

Bye,
bearophile

Jeremie Pelletier wrote:
> Tom S Wrote:
> 
>> Jeremie Pelletier wrote:
>>> I've been asked for my runtime crash handler with it's CodeView reader earlier in digitalmars.D so here it is:
>>>
>>> http://jump.fm/UVYHG
>>>
>>> It includes the runtime handler, a PE reader, CodeView reader and crash report window.
>>>
>>> So far it supports Windows and some bits are already done for posix platforms. It does a register dump, stack trace with resolved symbol names with their declaration file and line, as well as a listing of all loaded modules.
>>>
>>> Sorry for the lack of phobos/tango support, this is part of a runtime project I'm building which should hopefully be released to dsource in a few months. I'm releasing it now so if someone wants to implement a phobos or tango port they're free to do it.
>> Thanks a lot, Jeremie! The code looks really good. I think I'll be stealing the PE && CV parsing parts for Tango :)
> 
> I'm glad to hear that! Just keep my name in the credits and I'll be happy :)

Sure thing, I generally stick to the same philosophy :)


>> I'm curious about the runtime project of yours :o Can you shed a bit of the secret? How will it compare to druntime? What's the target audience? Does it have anything to do with 'meshes and textures'? ;)
> 
> Well it started a bit over two years ago, back then there was no druntime, I was digging into phobos to learn more about the language and decided it would be a fun and challenging task to write my own runtime. Then once I had it running stable I found out druntime had been created in the meantime!
> 
> In any ways, I learned quite a bit of the language's internals and ABI while coding and debugging it; I began by removing all default paths and libraries from dmd (both in sc.ini and compiling with "--defaultlib= --debuglib=--") and made an empty object.d and built my way up until there was no more compiler crashes (dmd is quite whiny about not having its "intrinsic" symbols declared, mostly the typeinfo's) and all unresolved symbols were added.
> 
> Since then the runtime was ported to D2 and kept up to date with every new release, I've been using it for these two years and it works great.
> 
> I don't really know how it will compare to druntime, I'm not out there to compete with them really. I use DMD on both windows and posix platforms (Win7 and Ubuntu respectively) so its the only supported compiler right now, haven't tested 64-bit support either. It would be able to support libraries such as phobos and tango with a few changes to the places that call into the compiler runtime or memory manager.

Thanks for the in-depth background story! Looks like you went a similar route that folks trying to do kernels in D choose. Perhaps if you had mentioned the development of this runtime earlier, druntime could be a joint project... But maybe it's not too late and druntime could still benefit from your expertise? There are scarcely few guys who understand the guts of writing D runtimes - I could probably count them all on the fingers of one palm ;) And I don't necessarily mean the concrete implementations but also the top-level structure.


> Right now the target audience is me. I haven't really considered who the target audience might be since I touch a bit of everything. I try and make it as generic and as modular as I can, so if someone doesn't need a package they can remove it from their build without getting tons of compile errors. I myself am focusing towards GUI application development with unified 2D/3D rendering and custom controls on top of that rendering as part of the runtime library, among many other things.

So it's floor wax *and* dessert topping? Sounds like you're set on doing  _everything_ from scratch :o I don't think nowadays I'd be willing to live without Tango's streams...


> So you could say it has to do with meshes and textures, although I haven't gotten to making interfaces for that part yet. The most I did on that level with my runtime was to write a reader for Blizzard's MPQ archives and extract model and texture data to render it in opengl, but apart from the MPQ reader it was all pretty ugly mixing win32 and gl code along with other stuff in the same routine to get the model on screen as fast as possible ;)

Haha, well that's a noble goal. I'll have to get myself some readers for  third-party formats as well. In retrospect, writing a custom 3ds max exporter and supporting only it was a bad idea :P


> I really am just beginning to notice people are interested in my project, I don't know when I'll put it on dsource and/or sourceforge. And given that D2 is still under heavy development it wouldn't be practical right now. I don't mind releasing bits and pieces of it but I would much rather see the first public release of it to be an usable one, although the compiler implementation part is pretty much stable at this point, if there's enough demand for it I might begin with that.

I'm sure people would become interested way earlier, had you mentioned the project anywhere :P I'm still in D1 land, but there are folks trying to use D2 and craving for some code to look into and build upon. The more code we release to the general public, the bigger D feels like and the greater chances it has of attracting new users... at least that's what I like to believe :P


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode

bearophile, el 20 de septiembre a las 22:31 me escribiste:
> Jeremie Pelletier:
> 
> > I couldn't agree more about the need for a GC benchmark suite,
> 
> I am slowly creating one (a general one) that later may be used for LDC
> :-)
> 
> 
> > If you come across such a benchmark suite, let me know, I would love to see how it performs against druntime's GC.
> 
> Similar GC benchmark suites don't grow on trees. I don't think one exists yet for D (ask to Lucarella too).

Unfortunately I had not much success at making a benchmark suite. I think
the main reason is the... "diversity" you find in D. You have D1/Phobos,
D1/Tango and D2/Druntime. Doing a benchmark suite that works with all is
very hard, you have to "port" the benchmark to 3 almost different
languages. And any decent GC benchmark test *real* programs. I was unable
to find any real programs (suitable for a GC benchmarks at least) for
D that are properly maintained except Dil, which is the only program
I plan to use in my benchmarks (I had a lot of problems lately with a bug
in my Naive collector that I'm still trying to find). Picking Dil means to
stay in the D1 Tango world, because I don't have the time to port Dil to
D1/Phobos nor D2/Druntime, so there it goes the "cross-D-version"
benchmark.

I only have that, and a few micro-benchmarks I collected in the 3 or
4 years I've been lurking this newsgroup (there are very few though). I'll
have to make at least a few more micro-benchmarks for multi-threaded
applications, since my plan is to add concurrency to the GC, so I'll have
to measure that scenario.

You can find my benchmark sources in this git repo (is not very interesting for now, really): http://git.llucax.com.ar/w/software/dgc/dgcbench.git

In my blog are a few (meaningless but nice) graphs for the Naive GC =)
http://proj.llucax.com.ar/blog/dgc/blog

I'll posting the results of the tests using the Tango basic collector soon.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
El día que falten los niños, que sobren las mujeres y que se prenda
fuego el último árbol, será el Apocalípsis.
	-- Ricardo Vaporeso. Camino Negro, 1916.