On Thu, 27 Jan 2005 12:42:57 +1100, Matthew wrote:

> 
> "Matthew" <admin.hat@stlsoft.dot.org> wrote in message news:ct9gl9$9k4$1@digitaldaemon.com...
>>    2. many of us do not have any serious problems doing *very successful* (see below) work in C/C++. If D is not a
>> quantum leap forward, _without_ new hassles, then why the hell is anyone ever going to use it? Because it's better
>> than Java?!? Pah!
> 
> btw, Java-phobic hyperbole aside, I should point out that until D can handle scenarios such as outlined in Benji's excellent post, D isn't even fit to kiss the bloated arse of Java. And that's a sad position to be in, to be sure ...

Umm... have you updated your newsreader?  Did you see Walter's recent response in this topic?

It seems the point has been taken. :-)

- John R.

"Walter" <newshound@digitalmars.com> wrote in message news:ct9afo$2l8$1@digitaldaemon.com...
>
> "Benji Smith" <dlanguage@xxagg.com> wrote in message news:ce4gv0dk6v1qgde6o94msecgi4huasdhga@4ax.com...
>> If I were building the same application with D, there would be fifteen different garbage collectors operating in fifteen different heap-spaces, and the objects allocated within one heap space might be referenced by objects in another heap space, each managed by a different garbage collector. It would be much more difficult to develop a heap profiling tool that could successfully allow a developer to navigate through such a fragmented heap space, particularly if the developer needed to figure out which GC was supposed to collect each out-of-reach object. Tracking down and fixing that memory leak probably would have taken a lot longer than an hour and a half.
>>
>> Consequently, I strongly support the development of a model within D that allows for a single GC instance per process. Any other scenario sounds like a development & debugging nightmare.
>
> I agree. I'm working on it.

Cool. I can shut my overbusy chops now! :-)

Please may we have intelligent GCs that can detect each other, within a process space, and connect up in a sensible, and dynamic-lib-unloading proof fashion?

Here's how I think it might work, roughly:

    If the process is written in D, it'll have a GC built-in (statically linked). [I agree with Walter 100% that
processes should have a GC built in, and not have to rely on a DLL. Painful for small process distribution, along with
other DLL issues ...]
    Now any subsequent D dynamic link-unit, whether that be a DDL (i.e. 'exposing' D classes in an analogous fashion to
Java's JARs), or a DLL (i.e. exposing a C API/interface, implemented in D), when loaded, does not create its own GC but
links to that of the process.
    Easy peasy, because its axiomatic that the life of any loaded dynamic link-units cannot exceed the life of the host
process

    If the process is *not* written in D, here's where it gets interesting. IMO, it should be possible to have the same
GC-locating mechanism attach a subsequent D link-unit (whether DDL or DLL). The problem is, how do we keep the first D
link-unit's code and data locked in memory after the application has unloaded it.
    One answer would be to just increment the first link-unit's ref count (e.g. another call to dlopen()/LoadLibrary())
and then discard it. This would work fine, never crash, but would cause issues in long running servers that need to
unload modules to pick up newer versions.
    Another answer would be that each GC-dependent link-unit would add such a lock onto the first-one-in, and release it
when they're released. There may be issues here, however, since one can get into trouble (un)loading libs during lib
(un)loading (albeit that I've never run into this outside C++.NET).
    In either case, a linked-to GC would have to expose a method for either locking it, or passing its instance
name/handle to be locked

Naturally, we have the issue of how a to-be-created GC detects and connects to a prior instance, and how this can be made thread-safe. That's a topic for discussion ...

The Dr .....

"John Reimer" <brk_6502@yahoo.com> wrote in message news:pan.2005.01.27.01.43.39.773589@yahoo.com...
> On Thu, 27 Jan 2005 12:42:57 +1100, Matthew wrote:
>
>>
>> "Matthew" <admin.hat@stlsoft.dot.org> wrote in message news:ct9gl9$9k4$1@digitaldaemon.com...
>>>    2. many of us do not have any serious problems doing *very successful* (see below) work in C/C++. If D is not a
>>> quantum leap forward, _without_ new hassles, then why the hell is anyone ever going to use it? Because it's better
>>> than Java?!? Pah!
>>
>> btw, Java-phobic hyperbole aside, I should point out that until D can handle scenarios such as outlined in Benji's excellent post, D isn't even fit to kiss the bloated arse of Java. And that's a sad position to be in, to be sure ...
>
> Umm... have you updated your newsreader?  Did you see Walter's recent response in this topic?

Been having issues with my cable, and local net.

> It seems the point has been taken. :-)

Coolio. We can get back to telling big-W what a star we think he is, now. <CG>

In article <ct9hki$afi$1@digitaldaemon.com>, Matthew says...
>Here's how I think it might work, roughly:
>
>    If the process is written in D, it'll have a GC built-in (statically linked). [I agree with Walter 100% that
>processes should have a GC built in, and not have to rely on a DLL. Painful for small process distribution, along with other DLL issues ...]
>    Now any subsequent D dynamic link-unit, whether that be a DDL (i.e. 'exposing' D classes in an analogous fashion to
>Java's JARs), or a DLL (i.e. exposing a C API/interface, implemented in D), when loaded, does not create its own GC but links to that of the process.
>    Easy peasy, because its axiomatic that the life of any loaded dynamic link-units cannot exceed the life of the host
>process
>
>    If the process is *not* written in D, here's where it gets interesting. IMO, it should be possible to have the same
>GC-locating mechanism attach a subsequent D link-unit (whether DDL or DLL). The problem is, how do we keep the first D link-unit's code and data locked in memory after the application has unloaded it.
>    One answer would be to just increment the first link-unit's ref count (e.g. another call to dlopen()/LoadLibrary())
>and then discard it. This would work fine, never crash, but would cause issues in long running servers that need to unload modules to pick up newer versions.
>    Another answer would be that each GC-dependent link-unit would add such a lock onto the first-one-in, and release it
>when they're released. There may be issues here, however, since one can get into trouble (un)loading libs during lib (un)loading (albeit that I've never run into this outside C++.NET).
>    In either case, a linked-to GC would have to expose a method for either locking it, or passing its instance
>name/handle to be locked
>
>Naturally, we have the issue of how a to-be-created GC detects and connects to a prior instance, and how this can be made thread-safe. That's a topic for discussion ...
>
>The Dr .....

I didn't quite follow all of that, but here's something that Pragma suggested many moons ago: if external link-units were managed by the "one & only" GC, it could happily reap them when, and only when, there are no more live references.

I know that's a somewhat trivial statement, but it has powerful mojo;

"Kris" <Kris_member@pathlink.com> wrote in message news:ct9k70$cv5$1@digitaldaemon.com...
> In article <ct9hki$afi$1@digitaldaemon.com>, Matthew says...
>>Here's how I think it might work, roughly:
>>
>>    If the process is written in D, it'll have a GC built-in
>> (statically linked). [I agree with Walter 100% that
>>processes should have a GC built in, and not have to rely on a DLL.
>>Painful for small process distribution, along with
>>other DLL issues ...]
>>    Now any subsequent D dynamic link-unit, whether that be a DDL
>> (i.e. 'exposing' D classes in an analogous fashion to
>>Java's JARs), or a DLL (i.e. exposing a C API/interface, implemented
>>in D), when loaded, does not create its own GC but
>>links to that of the process.
>>    Easy peasy, because its axiomatic that the life of any loaded
>> dynamic link-units cannot exceed the life of the host
>>process
>>
>>    If the process is *not* written in D, here's where it gets
>> interesting. IMO, it should be possible to have the same
>>GC-locating mechanism attach a subsequent D link-unit (whether DDL or
>>DLL). The problem is, how do we keep the first D
>>link-unit's code and data locked in memory after the application has
>>unloaded it.
>>    One answer would be to just increment the first link-unit's ref
>> count (e.g. another call to dlopen()/LoadLibrary())
>>and then discard it. This would work fine, never crash, but would
>>cause issues in long running servers that need to
>>unload modules to pick up newer versions.
>>    Another answer would be that each GC-dependent link-unit would add
>> such a lock onto the first-one-in, and release it
>>when they're released. There may be issues here, however, since one
>>can get into trouble (un)loading libs during lib
>>(un)loading (albeit that I've never run into this outside C++.NET).
>>    In either case, a linked-to GC would have to expose a method for
>> either locking it, or passing its instance
>>name/handle to be locked
>>
>>Naturally, we have the issue of how a to-be-created GC detects and
>>connects to a prior instance, and how this can be
>>made thread-safe. That's a topic for discussion ...
>>
>>The Dr .....
>
> I didn't quite follow all of that, but here's something that Pragma
> suggested
> many moons ago: if external link-units were managed by the "one &
> only" GC, it
> could happily reap them when, and only when, there are no more live
> references.
>
> I know that's a somewhat trivial statement, but it has powerful mojo;

I want to be able to write C-API DLLs in D, which I think would be the fly in that mojo.

In article <ct9hki$afi$1@digitaldaemon.com>, Matthew says...
>Here's how I think it might work, roughly:
>
>    If the process is written in D, it'll have a GC built-in (statically linked). [I agree with Walter 100% that
>processes should have a GC built in, and not have to rely on a DLL. Painful for small process distribution, along with other DLL issues ...]
>    Now any subsequent D dynamic link-unit, whether that be a DDL (i.e. 'exposing' D classes in an analogous fashion to
>Java's JARs), or a DLL (i.e. exposing a C API/interface, implemented in D), when loaded, does not create its own GC but links to that of the process.
>    Easy peasy, because its axiomatic that the life of any loaded dynamic link-units cannot exceed the life of the host
>process
>
>    If the process is *not* written in D, here's where it gets interesting. IMO, it should be possible to have the same
>GC-locating mechanism attach a subsequent D link-unit (whether DDL or DLL). The problem is, how do we keep the first D link-unit's code and data locked in memory after the application has unloaded it.
>    One answer would be to just increment the first link-unit's ref count (e.g. another call to dlopen()/LoadLibrary())
>and then discard it. This would work fine, never crash, but would cause issues in long running servers that need to unload modules to pick up newer versions.
>    Another answer would be that each GC-dependent link-unit would add such a lock onto the first-one-in, and release it
>when they're released. There may be issues here, however, since one can get into trouble (un)loading libs during lib (un)loading (albeit that I've never run into this outside C++.NET).
>    In either case, a linked-to GC would have to expose a method for either locking it, or passing its instance
>name/handle to be locked
>
>Naturally, we have the issue of how a to-be-created GC detects and connects to a prior instance, and how this can be made thread-safe. That's a topic for discussion ...
>
>The Dr .....


I should point out all the tricky stuff you're talking about just goes away if the GC is simply a shared-lib (DLL or DDL). May I ask, Matthew: what is your discomfort with shared-libs? I ask because I just don't know why the Microsoft O/S-related version-issue can't be resolved (to an acceptable degree) for one specific instance (the GC).

On a regular basis, I build commercial frameworks that dynamically load/unload mobile code; part of any true solution has to allow for unloading said code, and the mobile code itself should not be bloated out with multiple instances of the GC implementation.

Nor, come to think of it, should it be carrying all the floating-point support that the damned Object.printf() brings in with it. Both go against Walter's "no code bloat!" mantra and, franky, are completely unecessary. Oh, and before anyone say's something about disk-space; the latter is purely about transmission bandwidth & latency.

Still; I'll try not speculate upon the outcome.

- Kris


I said I'd shutup; I guess I lied :-(

"Kris" <Kris_member@pathlink.com> wrote in message news:ct9ltc$es1$1@digitaldaemon.com...
> In article <ct9hki$afi$1@digitaldaemon.com>, Matthew says...
>>Here's how I think it might work, roughly:
>>
>>    If the process is written in D, it'll have a GC built-in
>> (statically linked). [I agree with Walter 100% that
>>processes should have a GC built in, and not have to rely on a DLL.
>>Painful for small process distribution, along with
>>other DLL issues ...]
>>    Now any subsequent D dynamic link-unit, whether that be a DDL
>> (i.e. 'exposing' D classes in an analogous fashion to
>>Java's JARs), or a DLL (i.e. exposing a C API/interface, implemented
>>in D), when loaded, does not create its own GC but
>>links to that of the process.
>>    Easy peasy, because its axiomatic that the life of any loaded
>> dynamic link-units cannot exceed the life of the host
>>process
>>
>>    If the process is *not* written in D, here's where it gets
>> interesting. IMO, it should be possible to have the same
>>GC-locating mechanism attach a subsequent D link-unit (whether DDL or
>>DLL). The problem is, how do we keep the first D
>>link-unit's code and data locked in memory after the application has
>>unloaded it.
>>    One answer would be to just increment the first link-unit's ref
>> count (e.g. another call to dlopen()/LoadLibrary())
>>and then discard it. This would work fine, never crash, but would
>>cause issues in long running servers that need to
>>unload modules to pick up newer versions.
>>    Another answer would be that each GC-dependent link-unit would add
>> such a lock onto the first-one-in, and release it
>>when they're released. There may be issues here, however, since one
>>can get into trouble (un)loading libs during lib
>>(un)loading (albeit that I've never run into this outside C++.NET).
>>    In either case, a linked-to GC would have to expose a method for
>> either locking it, or passing its instance
>>name/handle to be locked
>>
>>Naturally, we have the issue of how a to-be-created GC detects and
>>connects to a prior instance, and how this can be
>>made thread-safe. That's a topic for discussion ...
>>
>>The Dr .....
>
>
> I should point out all the tricky stuff you're talking about just goes
> away if
> the GC is simply a shared-lib (DLL or DDL).

True. But then one has the inordinate hassle of managing a Phobos.DLL for any little utility program.

Note: I use the word 'inordinate' there in context. Having to care about Phobos.DLL for a utility would, for me, rule out D as a language for implementing all the small tools and utilities (as in http://synesis.com.au/systools.html) that I write. That's a 100% certainty. Hence, I agree with Walter, _in this regard_, that Phobos should not have to be dynamically linked.

> May I ask, Matthew: what is your
> discomfort with shared-libs? I ask because I just don't know why the
> Microsoft
> O/S-related version-issue can't be resolved (to an acceptable degree)
> for one
> specific instance (the GC).

I don't want to have to write installers for simple programs / plug-in DLLs (e.g. shell extensions; http://shellext.com/).

Note: I *absolutely* want/must have Phobos in a DLL for serious / large-scale work. To not have it as such would as surely rule out D for my consideration for any large scale project.

Naturally, these provide a contradiction, without a straightforward solution. Therefore, I believe that the only viable solution is that D does something smarter than C/C++, and support both. It requires a degree of sophistication in the runtime arbitration between multiple GC creation events, but I hardly think this is an insurmountable problem. I'd be very surprised if this is something that we cannot all work through, or that will confound big-W's programming skill. In any case, I don't see that there's a (commercially) viable alternative.

> On a regular basis, I build commercial frameworks that dynamically
> load/unload
> mobile code; part of any true solution has to allow for unloading said
> code, and
> the mobile code itself should not be bloated out with multiple
> instances of the
> GC implementation.

Exactly.

> Nor, come to think of it, should it be carrying all the floating-point
> support
> that the damned Object.printf() brings in with it.

This is another issue, but one on which I completely agree.

> Both go against Walter's "no
> code bloat!" mantra and, franky, are completely unecessary.

Agreed.

> Oh, and before
> anyone say's something about disk-space; the latter is purely about
> transmission
> bandwidth & latency.

Not so. There is a more fundamental objection: coupling should be minimised in all circumstances. I might offer something like "anyone exposed to enough software engineering on a commercial scale will come to believe this" but that'd get me involved in YABA, so I'll just say "it has been my unwavering experience, in a multitude of languages, technologies, application domains, that increasing coupling is bad, and decreasing coupling is good".

If someone were to look at D, then look at Phobos, then look at Object, then see the coupling between Object and printf() and the CRT, and follow their own experiential logical flow to come to the conclusion that "D is crap", I would wish to persuade them otherwise, but I could not fault their reasoning.

> Still; I'll try not speculate upon the outcome.

I would think that the recent, singular (?), success of our carping and whineing in getting a movement from big-W on the dynamic-linking issue offers all kinds of encouragment to those who are yet to hit their D sweet spot.

> - Kris
>
> I said I'd shutup; I guess I lied :-(

It's people like you, Kris, who have the knowledge, the general experience, and, perhaps most importantly, the significant real-world D experience, who provide the meat on the bones of the instinctual (some would say half-cocked) mutterings of the likes of me. Please don't shut up.


The Dr .....

In article <ct9mvn$g05$1@digitaldaemon.com>, Matthew says...
>I don't want to have to write installers for simple programs / plug-in DLLs (e.g. shell extensions; http://shellext.com/).
>
>Note: I *absolutely* want/must have Phobos in a DLL for serious / large-scale work. To not have it as such would as surely rule out D for my consideration for any large scale project.
>
>Naturally, these provide a contradiction, without a straightforward solution. Therefore, I believe that the only viable solution is that D does something smarter than C/C++, and support both. It requires a degree of sophistication in the runtime arbitration between multiple GC creation events, but I hardly think this is an insurmountable problem. I'd be very surprised if this is something that we cannot all work through, or that will confound big-W's programming skill. In any case, I don't see that there's a (commercially) viable alternative.

So how about this:

1) the GC lives within a library (as it does now)

2) there is an optional DLL, compiled with the library GC, and there is a seperate library shim to bind said DLL instead

3) the developer makes a choice to either (a) link with the library GC directly
or (b) link with the DLL GC shim

4) the choice is made by adding the DLL-shim library-name to the dmd command-line, causing the linker to select the DLL GC rather than the default statically-linked GC (the linker will not try to bind the static GC instance if those symbols have already been satisfied)

Does that covers all bases? There's probably several other ways to acheive a similar effect. Note that the default, and simple, behaviour is to statically link the GC ...

- Kris

On Wed, 26 Jan 2005 14:19:04 -0800, "Walter" <newshound@digitalmars.com> wrote:

>I agree. I'm working on it.

Fantastic. Thanks, Walter. I really appreciate how receptive you are to the whims (er...I mean, the intelligent and informed opinions) of the people in this ng.

--Benji

"Kris" <Kris_member@pathlink.com> wrote in message news:ct9s2l$lo5$1@digitaldaemon.com...
> In article <ct9mvn$g05$1@digitaldaemon.com>, Matthew says...
>>I don't want to have to write installers for simple programs / plug-in DLLs (e.g. shell extensions; http://shellext.com/).
>>
>>Note: I *absolutely* want/must have Phobos in a DLL for serious /
>>large-scale work. To not have it as such would as surely rule out D
>>for
>>my consideration for any large scale project.
>>
>>Naturally, these provide a contradiction, without a straightforward
>>solution. Therefore, I believe that the only viable solution is that D
>>does something smarter than C/C++, and support both. It requires a
>>degree of sophistication in the runtime arbitration between multiple
>>GC
>>creation events, but I hardly think this is an insurmountable problem.
>>I'd be very surprised if this is something that we cannot all work
>>through, or that will confound big-W's programming skill. In any case,
>>I
>>don't see that there's a (commercially) viable alternative.
>
> So how about this:
>
> 1) the GC lives within a library (as it does now)
>
> 2) there is an optional DLL, compiled with the library GC, and there
> is a
> seperate library shim to bind said DLL instead
>
> 3) the developer makes a choice to either (a) link with the library GC
> directly
> or (b) link with the DLL GC shim
>
> 4) the choice is made by adding the DLL-shim library-name to the dmd
> command-line, causing the linker to select the DLL GC rather than the
> default
> statically-linked GC (the linker will not try to bind the static GC
> instance if
> those symbols have already been satisfied)
>
> Does that covers all bases? There's probably several other ways to
> acheive a
> similar effect. Note that the default, and simple, behaviour is to
> statically
> link the GC ...

What about an application, written in D and statically linked to a GC, that may or may not load a DDL to get some D classes, depending on its cmd-line params?