On 12/07/2014 12:10 p.m., Mike wrote:
> On Friday, 11 July 2014 at 19:46:25 UTC, Walter Bright wrote:
>> I agree that the GC phobia is way, WAY, overblown for practical
>> programming.
>
> I agree with this, as well, but it's a generalization.
>
> There are some applications that the current GC is not suitable for, but
> D provides a plethora of features for disabling the GC or managing
> memory in other ways (very cool), so I don't see a reason to not use D
> even for the most demanding problem.  I love the GC for some things I
> do, but can't use it for other things I do.  For the vast majority of
> applications I see people using D for, however, I see no reason why
> there should be any worry about the GC.
>
> The problem, however, when managing one's own memory is that one cannot
> use some of the built-in types, like Exceptions, that are instantiated
> deep within the runtime.  A solution to this would likely quiet some of
> the clamoring, IMO.
>
> I would be interested in hearing any suggestions for disabling the GC
> and still making use of Exceptions, dynamic arrays, etc... using a
> user-supplied memory manager.  Maybe something like this already exists,
> and people like me just aren't aware of it.
>
> Being a novice still, I don't know what the solution is.  At the moment
> I exploring region-based memory management (nice example at
> http://en.wikipedia.org/wiki/Region-based_memory_management). I also saw
> some proposals for something like
> gc.pushAllocator(myAllocator)/gc.popAllocator(), which would be nice.
>
> Mike

Something I've been thinking about is an overload for with statement.
E.g.

with(new MyAllocator) {
	void*[256] values;
	//...
}

class MyAllocator : IGC {
	private {
		IGC prevGC;
	}
	
	void opWithIn() {
		this.prevGC = GC.getImpl();
		GC.setImpl(this);
	}

	void opWithOut() {
		GC.setImpl(this.prevGC);
	}
}

Of course if we added an overload that allowed for passing root memory blocks that can be freed (ref countered guaranteed at compilation), we could force them to be freed in opWithOut. Otherwise tell the previous GC to use it.

Now I sort of mix GC and allocator up a little bit, but if it is done right. We could swap out the GC for an allocator and make it almost the same.

But in saying this, this would mean we would need to have a rethink of how we do the GC in druntime. At least from the architecture point of view.

This isn't the reason I thought of this, mostly because I was exploring how to do something like GWT nice (not really required but could be useful for caching output).

Its a small addition, that may pay off for this kind of work substantially. Unless of course we already have it? I didn't realize that statements worked in with statements till a couple days ago. So if not, I would be surprised.

On Sat, Jul 12, 2014 at 01:18:05AM +0000, Mike via Digitalmars-d wrote: [...]
>  - D is universal -  I don't know how to articulate this, but I'm sick
> of learning so many languages for different purposes and different
> platforms.  I'm beginning to use D for just about everything, and I
> don't have to worry so much about whether I'm on Windows or Linux.

One of the things I eventually hope to get to, is to write D apps that work on both Windows and Linux.  I'm pretty sure my code can already run on Windows, but I just haven't setup the dev environment yet.


> I'm even using D to write low-level drivers for my micrcontroller.  I use it for my build scripts, automating my builds in the same language I'm building.  D, one language to rule them all.
[...]

Whoa. That's a pretty cool idea: use D to build D!  I think I'm going to start doing that too. Right now I'm using SCons (Python-based), which is pretty great for what I need, but the thought of using D to build D sounds so appealing to me that I'm gonna hafta do just that. :) Thanks for the idea!


T

-- 
"Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next. -- (Stolen from the net)

On 7/11/2014 5:10 PM, Mike wrote:
> The problem, however, when managing one's own memory is that one cannot use some
> of the built-in types, like Exceptions, that are instantiated deep within the
> runtime.  A solution to this would likely quiet some of the clamoring, IMO.

The thing is, Exceptions should be exceptional, not normal. So if you're worried about GC pauses during exception processing, I think it's time to re-examine what exceptions in your code are being used for.

On 7/11/14, 9:10 PM, Walter Bright via Digitalmars-d wrote:
> On 7/11/2014 5:10 PM, Mike wrote:
>> The problem, however, when managing one's own memory is that one cannot use some
>> of the built-in types, like Exceptions, that are instantiated deep within the
>> runtime.  A solution to this would likely quiet some of the clamoring, IMO.
>
> The thing is, Exceptions should be exceptional, not normal. So if you're worried about GC pauses
> during exception processing, I think it's time to re-examine what exceptions in your code are being
> used for.

It's not the try/throw/catch/finally parts, it's the new that's usually involved.  The chance that the gc will kick in and affect every thread, not just the thread dealing with the exception is a big hit that is unacceptable in some scenarios.

On Saturday, 12 July 2014 at 04:10:32 UTC, Walter Bright wrote:
> On 7/11/2014 5:10 PM, Mike wrote:
>> The problem, however, when managing one's own memory is that one cannot use some
>> of the built-in types, like Exceptions, that are instantiated deep within the
>> runtime.  A solution to this would likely quiet some of the clamoring, IMO.
>
> The thing is, Exceptions should be exceptional, not normal. So if you're worried about GC pauses during exception processing, I think it's time to re-examine what exceptions in your code are being used for.

I understand that, but that wasn't my point.  I was just using Exceptions as an example of a built-in type (instantiated in the runtime outside of the users control).  Dynamic arrays are another.

The goal is for the user to be able to be able to control the allocation for all types in D, not just the ones the user creates.  And to be able to continue to use, to the extent possible, roughly the same idioms and patterns they would use if employing the GC.

It looks like you were headed down that path with DIP46 (http://wiki.dlang.org/DIP46).  It's almost a year old.  Do you still feel its worth pursuing?

Mike

On Saturday, 12 July 2014 at 03:59:58 UTC, Rikki Cattermole wrote:
>
> Something I've been thinking about is an overload for with statement.
> E.g.
>
> with(new MyAllocator) {
> 	void*[256] values;
> 	//...
> }
>
> class MyAllocator : IGC {
> 	private {
> 		IGC prevGC;
> 	}
> 	
> 	void opWithIn() {
> 		this.prevGC = GC.getImpl();
> 		GC.setImpl(this);
> 	}
>
> 	void opWithOut() {
> 		GC.setImpl(this.prevGC);
> 	}
> }
>
> Of course if we added an overload that allowed for passing root memory blocks that can be freed (ref countered guaranteed at compilation), we could force them to be freed in opWithOut. Otherwise tell the previous GC to use it.
>
> Now I sort of mix GC and allocator up a little bit, but if it is done right. We could swap out the GC for an allocator and make it almost the same.
>
> But in saying this, this would mean we would need to have a rethink of how we do the GC in druntime. At least from the architecture point of view.
>
> This isn't the reason I thought of this, mostly because I was exploring how to do something like GWT nice (not really required but could be useful for caching output).
>
> Its a small addition, that may pay off for this kind of work substantially. Unless of course we already have it? I didn't realize that statements worked in with statements till a couple days ago. So if not, I would be surprised.

I think that puts convenient syntax around the basic idea.

Region-based memory management is just what I'm currently studying.  There are probably other ways to accomplish the goal:  allow the user control the allocation, lifetime, and deallocation of the built-in types.

Mike

On 7/11/2014 9:35 PM, Mike wrote:
> It looks like you were headed down that path with DIP46
> (http://wiki.dlang.org/DIP46).  It's almost a year old.  Do you still feel its
> worth pursuing?

I haven't thought much about that since. There always seems to be something else needing attention.

On 7/11/2014 9:24 PM, Brad Roberts via Digitalmars-d wrote:
> On 7/11/14, 9:10 PM, Walter Bright via Digitalmars-d wrote:
>> The thing is, Exceptions should be exceptional, not normal. So if you're
>> worried about GC pauses
>> during exception processing, I think it's time to re-examine what exceptions
>> in your code are being
>> used for.
>
> It's not the try/throw/catch/finally parts, it's the new that's usually
> involved.  The chance that the gc will kick in and affect every thread, not just
> the thread dealing with the exception is a big hit that is unacceptable in some
> scenarios.

That's a good point.

On 07/11/2014 09:53 AM, simendsjo wrote:

> On 07/11/2014 06:28 PM, Russel Winder via Digitalmars-d wrote:
>> On Fri, 2014-07-11 at 17:43 +0200, simendsjo via Digitalmars-d wrote:
>> […]
>>> A little anecdote.. I once got a 20% speed increase in Python by
>>> "moving" a variable instantiation outside a tight loop.
>>>    i = 0
>>>    # loop here
>>>      i = something
>>>
>>> rather than
>>>    # loop here
>>>      i = something
>>
>> This is interesting. I can believe there is some performance benefit,
>> but I am not sure I believe 20% improvement. If you can send me the code
>> you were using, I would like to do some benchmarking on this.
>
> Yes, I was very perplexed when I was profiling and finally found the
> main offender. Unfortunately I don't have the code - it was a project
> done for a past employer back in 2006/2007 (Python 2.4 IIRC).
>
>>> The compiler wasn't smart enough to do this.
>>
>> The Python compiler cannot and will never be able to do any such thing.
>> Indeed if it did any such thing, it would be an error since it
>> significantly changes the semantics of the program. Thus not doing this
>> is not the fault of the compiler.  The fact that you were able to do
>> this and it appeared to give you the same results just means that the
>> change in program semantics did not affect your computation. Which is
>> good, but not something the compiler could determine.
>
> I think of this as a fault in the compiler. It was quite obvious (to me)
> that nothing else relied on the value so the value didn't have to be
> created on each iteration.

Can that 'i = something' expression be monkey-patched in Python? If so, it could have side-effects to make the program change semantics like Russel Winder means.

Ali

Ali Çehreli:

> Can that 'i = something' expression be monkey-patched in Python? If so, it could have side-effects to make the program change semantics like Russel Winder means.

Right. And even PyPy isn't a compiler. Python is interpreted. And at best JITted. In most cases you use Cpython, that is an interpreter. And in Python most optimizations are dangerous, because the language is very dynamic. If you look for performance it's better for you to look elsewhere (like Julia).

Bye,
bearophile