January 04, 2012

On 04.01.2012 15:33, Vladimir Panteleev wrote:
> On Wednesday, 4 January 2012 at 14:28:07 UTC, Andrei Alexandrescu wrote:
>> Hmmm, I see that the other way around. D CTFE-generated macros are
>> much easier to debug because you can actually print the code before
>> mixing it in. If it looks like valid code... great.
>
> Paging Don Clugston: would it be feasable to have the compiler remember
> the source position of every single char/string literal or
> compile-time-evaluated string expression?
>
> I'm thinking that if the compiler tracks the source of every string/char
> literal in the source code, all across to any manipulations, debugging
> CTFE-generated code would be a lot easier - the compiler would emit
> error messages pointing inside string literals, and debuggers could step
> inside code in string literals. (The one thing this doesn't allow is
> allowing debuggers to step through a DSL with no D code in it.)
>
> The naive implementation would store the position of every character,
> which would blow up the memory usage by about 13 times or so on 32-bit?
> (For every character, add a struct with 3 fields - char* filename; int
> line, column). A rope-like structure could cut down on that but possibly
> drastically complicating the implementation.

A simpler way to debug CTFE generated code is to dump it to another file and redirect the debugger to this file.
Here is a patch that does just that, but it is probably not up to date anymore: http://d.puremagic.com/issues/show_bug.cgi?id=5051#c4

January 04, 2012
On 01/04/12 10:39, Manu wrote:
> Walter made an argument "The same goes for all those language extensions you mentioned. Those are not part of Standard C. They are vendor extensions. Does that mean that C is not actually a systems language? No."
> This is absurd... are you saying that you expect Iain to add these things to GDC to that people can use them, and then create incompatible D code with the 'standard' compiler?

Some of these things are *already* in GDC... Probably not documented and tested enough [1], but they are there. So you /can/ have function declarations such as:

pragma(GNU_attribute, always_inline, flatten, hot) int fxx(int i) { ... }

Now, this wouldn't be that bad, if we had a preprocessor or some kind of macro facility. But as it is, writing portable code is too expensive. (I may need to add a cpp stage for D because of issues like this, haven't decided yet...)

> Why would you intentionally fragment the compiler support of language features rather than just making trivial (but important) features that people do use part of the language?

There are more issues, that *will* be fixed in time, once (maybe even "if") D matures. A wiki page etc listing all the needed changes ("D next generation") would definitively be helpful. Not only to record what needs fixing, but also what to avoid. Could reduce the inevitable balkanization significantly.


Functions attributes seems like it could be an easy, backward compatible addition:

@attr(attributes...)

then define some obvious generic attributes like "inline" (which is (always|force)_inline, as it's the only one that makes sense), "noinline", "hot", "cold" etc. This lets you write "@attr(inline) int f(i){}" etc, but doesn't help the vendor specific attr case at all, unfortunately. [2]

artur

[1] Like the "target" "tune" (sub)attribute, which is not per function, but global (ie behaves as C #pragma). That might be a gcc bug though. Also, using gcc asm in a function makes the compiler generate worse code (triggered by the /first/ asm use, next ones within a function are free).

[2] The problem is what do you do when you have a lot of function/methods that need to be inlined/flattened, have a different calling convention or otherwise need to be specially marked _and_ it needs to be done differently for different compilers?...
January 04, 2012
Oh, and virtual-by-default... completely unacceptable for a systems language. most functions are NOT virtual, and finding the false-virtuals while optimising will be extremely tedious and time consuming. Worse, if libraries contain false virtuals, there's good chance I may not be able to use said library on certain architectures (PPC, ARM in particular). Terrible decision... completely contrary to modern hardware design and trends. Why invent a 'new' language for 10 year old hardware?

On 4 January 2012 11:39, Manu <turkeyman@gmail.com> wrote:

> This conversation has meandered into one very specific branch, but I just
> want to add my 2c to the OP.
> I agree, I want D to be a useful systems language too. These are my issues
> to that end:
>
>  * __forceinline ... I wasn't aware this didn't exist... and yes, despite
> all this discussion, I still depend on this all the time. People are
> talking about implementing forceinline by immitating macros using mixins...
> crazy? Here's a solid reason I avoid mixins or procedurally generated code
> (and the preprocessor in C for that matter, in favour of __forceinline):
> YOU CAN DEBUG IT. In an inline function, the code exists in the source
> file, just like any other function, you can STEP THE DEBUGGER through it,
> and inspect the values easily. This is an underrated requirement. I would
> waste hours on many days if I couldn't do this. I would only ever use
> string mixins for the most obscure uses, preferring inline functions for
> the sake of debugging 99% of the time.
>
>  * vector type ... D has exactly no way to tell the compiler to allocate
> 128bit vector registers, load/store, and pass then to/from functions. That
> is MOST of the register memory on virtually every modern processor, and D
> can't address it... wtf?
>
>  * inline assembler needs pseudo registers ... The inline assembler is
> pretty crap, immitating C which is out-dated. Registers in assembly code
> should barely ever be addressed directly, they should only be addressed by
> TYPE, allowing the compiler to allocate available registers (and/or manage
> storing the the stack where required) as with any other code. Inline
> assembly without pseudo-registers is almost always an un-optimisation, and
> this is also the reason why almost all C programmers use hardware opcode
> intrinsics instead of inline assembly. There is no way without using
> intrinsics in C to allow the compiler to perform optimal register
> allocation, and this is still true for D, and in my opinion, just plain
> broken.
>
>  * __restrict ... I've said this before, but not being able to hint that
> the compiler ignore possible pointer aliasing is a big performance problem,
> especially when interacting with C libs.
>
>  * multiple return values (in registers) ... (just because I opened a
> topic about it before) This saves memory accesses in common cases where i
> want to return (x, y), or (retVal, errorCode) for instance.
>
> Walter made an argument "The same goes for all those language extensions
> you mentioned. Those are not part of Standard C. They are vendor
> extensions. Does that mean that C is not actually a systems language? No."
> This is absurd... are you saying that you expect Iain to add these things
> to GDC to that people can use them, and then create incompatible D code
> with the 'standard' compiler?
> Why would you intentionally fragment the compiler support of language
> features rather than just making trivial (but important) features that
> people do use part of the language?
>
> This is a great example of why C is shit, and a good example of why I'm interested in D at all...
>
> On 29 December 2011 13:19, Vladimir Panteleev <vladimir@thecybershadow.net
> > wrote:
>
>> On Thursday, 29 December 2011 at 09:16:23 UTC, Walter Bright wrote:
>>
>>> Are you a ridiculous hacker? Inline x86 assembly that the compiler actually understands in 32 AND 64 bit code, hex string literals like x"DE ADB EEF" where spacing doesn't matter, the ability to set data alignment cross-platform with type.alignof = 16, load your shellcode verbatim into a string like so: auto str = import("shellcode.txt");
>>>
>>
>> I would like to talk about this for a bit. Personally, I think D's system programming abilities are only half-way there. Note that I am not talking about use cases in high-level application code, but rather low-level, widely-used framework code, where every bit of performance matters (for example: memory copy routines, string builders, garbage collectors).
>>
>> In-line assembler as part of the language is certainly neat, and in fact coming from Delphi to C++ I was surprised to learn that C++ implementations adopted different syntax for asm blocks. However, compared to some C++ compilers, it has severe limitations and is D's only trick in this alley.
>>
>> For one thing, there is no way to force the compiler to inline a function (like __forceinline / __attribute((always_inline)) ). This is fine for high-level code (where users are best left with PGO and "the compiler knows best"), but sucks if you need a guarantee that the function must be inlined. The guarantee isn't just about inlining heuristics, but also implementation capabilities. For example, some implementations might not be able to inline functions that use certain language features, and your code's performance could demand that such a short function must be inlined. One example of this is inlining functions containing asm blocks - IIRC DMD does not support this. The compiler should fail the build if it can't inline a function tagged with @forceinline, instead of shrugging it off and failing silently, forcing users to check the disassembly every time.
>>
>> You may have noticed that GCC has some ridiculously complicated assembler facilities. However, they also open the way to the possibilities of writing optimal code - for example, creating custom calling conventions, or inlining assembler functions without restricting the caller's register allocation with a predetermined calling convention. In contrast, DMD is very conservative when it comes to mixing D and assembler. One time I found that putting an asm block in a function turned what were single instructions into blocks of 6 instructions each.
>>
>> D's lacking in this area makes it impossible to create language features that are on the level of D's compiler built-ins. For example, I have tested three memcpy implementations recently, but none of them could beat DMD's standard array slice copy (despite that in release mode it compiles to a simple memcpy call). Why? Because the overhead of using a custom memcpy routine negated its performance gains.
>>
>> This might have been alleviated with the presence of sane macros, but no such luck. String mixins are not the answer: trying to translate macro-heavy C code to D using string mixins is string escape hell, and we're back to the level of shell scripts.
>>
>> We've discussed this topic on IRC recently. From what I understood,
>> Andrei thinks improvements in this area are not "impactful" enough, which I
>> find worrisome.
>>
>> Personally, I don't think D qualifies as a true "system programming
>> language" in light of the above. It's more of a compiled language with
>> pointers and assembler. Before you disagree with any of the above, first
>> (for starters) I'd like to invite you to translate Daniel Vik's C memcpy
>> implementation to D: http://www.danielvik.com/2010/**
>> 02/fast-memcpy-in-c.html<http://www.danielvik.com/2010/02/fast-memcpy-in-c.html>. It doesn't even use inline assembler or compiler intrinsics.
>>
>
>


January 04, 2012
On 1/4/2012 10:53 AM, Manu wrote:
> Oh, and virtual-by-default... completely unacceptable for a systems language.
>  most functions are NOT virtual, and finding the false-virtuals while
> optimising will be extremely tedious and time consuming.

The only reason to use classes in D is for polymorphic behavior - and that means
virtual functions. Even so, a class member function will be called directly if
it is private or marked as 'final'.

An easy way to find functions that are not overridden (what you called false virtuals) is to add:

   final:

at the top of your class definition. The compiler will give you errors for any functions that need to be virtual.

If you don't want polymorphic behavior, use structs instead. Struct member
functions are never virtual.


> Worse, if libraries contain false virtuals, there's good chance I may not be
> able to use said library on certain architectures (PPC, ARM in particular).

??
January 04, 2012
On 01/04/2012 07:53 PM, Manu wrote:
> Oh, and virtual-by-default... completely unacceptable for a systems
> language. most functions are NOT virtual, and finding the false-virtuals
> while optimising will be extremely tedious and time consuming. Worse, if
> libraries contain false virtuals, there's good chance I may not be able
> to use said library on certain architectures (PPC, ARM in particular).
> Terrible decision... completely contrary to modern hardware design and
> trends. Why invent a 'new' language for 10 year old hardware?
>

If you don't need virtual functions don't use classes.
January 04, 2012
Walter:

> The only reason to use classes in D is for polymorphic behavior - and that means virtual functions.

I don't agree, in some cases I use final class instances instead of heap-allocated structs even when I don't need polymorphic behaviour just to avoid pointer syntax (there is also a bit higher probability of destructors being called, compared to heap-allocated structs).
In some cases I've used a final class just to be able to use a this() with no arguments :-)

Bye,
bearophile
January 04, 2012
You just missed a big discussion on IRC about this, where I think I made some fair points that people actually agreed with.

On 1/4/2012 10:53 AM, Manu wrote:
>
>> Oh, and virtual-by-default... completely unacceptable for a systems
>> language.
>>  most functions are NOT virtual, and finding the false-virtuals while
>> optimising will be extremely tedious and time consuming.
>>
>
> The only reason to use classes in D is for polymorphic behavior - and that
> means
> virtual functions. Even so, a class member function will be called
> directly if
> it is private or marked as 'final'.
>

Is this true? Surely the REAL reason to use classes is to allocate using
the GC?
Aren't struct's allocated on the stack, and passed to functions by value?
Do I need to start using the ref keyword to use GC allocated structs?



> An easy way to find functions that are not overridden (what you called
> false virtuals) is to add:
>
>   final:
>
> at the top of your class definition. The compiler will give you errors for any functions that need to be virtual.
>
> If you don't want polymorphic behavior, use structs instead. Struct member functions are never virtual.


I have never written a class in any language where the ratio of virtual to
non-virtual functions is more than 1:10 or so... requiring that one
explicitly declared the vastly more common case seems crazy.
The thing I'm most worried about is people forgetting to declare 'final:'
on a class, or junior programmers who DON'T declare final, perhaps because
they don't understand it, or perhaps because they have 1-2 true-virtuals,
and the rest are just defined in the same place... This is DANGEROUS. The
junior programmer problem is one that can NOT be overstated, and doesn't
seem to have been considered in a few design choices.
I'll bet MOST classes result in an abundance of false-virtuals, and this is
extremely detrimental to performance on modern hardware (and getting worse,
not better, as hardware progresses).



>  Worse, if libraries contain false virtuals, there's good chance I may not
>> be
>> able to use said library on certain architectures (PPC, ARM in
>> particular).
>>
>
> ??
>

If a library makes liberal (and completely unnecessary) virtual calls to the point where it performs too poorly on some architecture; lets say ARM, or PPC (architectures that will suffer far more than x86 form virtual calls), I can no longer use this library in my project... What a stupid position to be in. The main strength of any language is its wealth of libraries available, and a bad language decision prohibiting use of libraries for absolutely no practical reason is just broken by my measure.


January 04, 2012
On 5 January 2012 01:17, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 01/04/2012 07:53 PM, Manu wrote:
>
>> Oh, and virtual-by-default... completely unacceptable for a systems language. most functions are NOT virtual, and finding the false-virtuals while optimising will be extremely tedious and time consuming. Worse, if
>>
>> libraries contain false virtuals, there's good chance I may not be able to use said library on certain architectures (PPC, ARM in particular). Terrible decision... completely contrary to modern hardware design and trends. Why invent a 'new' language for 10 year old hardware?
>>
>>
> If you don't need virtual functions don't use classes.
>

Polymorphism isn't the only difference by a long shot. Allocation and referencing patterns are totally different. I don't feel this is a reasonable counter-argument.


January 04, 2012
On 4 January 2012 16:28, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>wrote:

> On 1/4/12 3:39 AM, Manu wrote:
>
>>  * __forceinline ... I wasn't aware this didn't exist... and yes,
>> despite all this discussion, I still depend on this all the time. People
>> are talking about implementing forceinline by immitating macros using
>> mixins.... crazy? Here's a solid reason I avoid mixins or procedurally
>>
>> generated code (and the preprocessor in C for that matter, in favour of __forceinline): YOU CAN DEBUG IT. In an inline function, the code exists in the source file, just like any other function, you can STEP THE DEBUGGER through it, and inspect the values easily. This is an underrated requirement. I would waste hours on many days if I couldn't do this. I would only ever use string mixins for the most obscure uses, preferring inline functions for the sake of debugging 99% of the time.
>>
>
> Hmmm, I see that the other way around. D CTFE-generated macros are much easier to debug because you can actually print the code before mixing it in. If it looks like valid code... great.
>
> I think the deal with inline functions is significantly more complex. Inlining is the first step in a long pipeline of optimizations that often make the code virtually unrecognizable and impossible to map back to source in a way that's remotely understandable.


It's rare to step through optimised code. You tend to debug and step in
debug/unoptimised builds, where inline functions are usually not even
inlined, and code flow still looks natural, and easy to follow.. This saves
lots of time.
C/C++ macros present the same problem of not being able to step and inspect
values. Most industry programmers I work with tend to avoid macros for this
reason above all others.


January 04, 2012
>
> Manu:
>
> >  * vector type ... D has exactly no way to tell the compiler to allocate
> > 128bit vector registers, load/store, and pass then to/from functions.
> That
> > is MOST of the register memory on virtually every modern processor, and D can't address it... wtf?
>
> Currently the built-in vector operations of D are not optimized, their syntax and semantics has some small holes that I'd like to see fixed (it's not just a matter of implementation bugs, I also mean design bugs). So I suggest first to improve them a lot, and only later, if necessary, to introduce intrinsics.
>

I'm not referring to vector OPERATIONS. I only refer to the creation of a type to identify these registers... anything more than that can be done with inline asm, hardware intrinsics, etc, but the language MUST at least expose the type to allow register allocation and parameter passing.

A language defined 128bit SIMD type would be fine for basically all architectures. Even though they support different operations on these registers, the size and allocation patterns are always the same across all architectures; 128 bits, 16byte aligned, etc. This allows at minimum platform independent expression of structures containing simd data, and calling of functions passing these types as args.

SSE, VMX (PPC), VFP (ARM)... they all share the same rules.