Hey all,

I've not posted here in a while, but I've been keeping up to speed with D's progress over the last couple of years and remain consistently impressed with the language.

I'm part of a new computing society in the University of Newcastle, Australia, and am essentially known throughout our Computer Science department as 'the D guy'. At the insistence of my peers, I have decided to give an introductory lecture on the D Programming Language, in order to expose more students to the increasingly amazing aspects of D. I expect to cover the good, the bad, the awesome, and the ugly, in a complement-criticism-complement styled talk, and while I have my own opinions regarding each of these things, I'd like a broader view from the community regarding these aspects, so that I may provide as accurate and as useful information as possible.

So, if you would be so kind, give me a bullet list of the aspects of D you believe to be good, awesome, bad, and/or ugly. If you have the time, some code examples wouldn't go amiss either! Try not to go in-depth to weird edge cases - remain general, yet informative. E.g. I consider D's string mixins to be in the 'awesome' category, but its reliance on the GC for large segments of the standard library to be in the 'ugly' category.

Thanks so much for your time!

On Mon, Jul 07, 2014 at 11:47:25PM +0000, Aerolite via Digitalmars-d-learn wrote: [...]
> So, if you would be so kind, give me a bullet list of the aspects of D you believe to be good, awesome, bad, and/or ugly. If you have the time, some code examples wouldn't go amiss either! Try not to go in-depth to weird edge cases - remain general, yet informative. E.g. I consider D's string mixins to be in the 'awesome' category, but its reliance on the GC for large segments of the standard library to be in the 'ugly' category.
[...]

String mixins are a controversial feature. Many (including myself) think they are definitely awesome, but that power also comes with the price of being harder to maintain, and difficult to integrate with IDE features (though the latter doesn't matter to me 'cos I don't use IDEs). Because of that, some others think they are a misfeature, and have proposed to remove them. But I don't think it's going away anytime soon, since several key features depend on it.

Perhaps I may bring up a representative use case: operator overloading. In C++, if you implement a custom number type, for example, you have to overload operator+, operator*, operator/, operator-. And then you realize you left out operator+=, operator*=, operator/=, ... and then you need operator<, operator>, operator<=, operator>=, ad nauseum. In D?

	// This covers +, -, *, /, +=, -=, *=, /=, etc..
	auto opBinary(string op)(NumType n)
	{
		return NumType(mixin(this.val ~ op ~ n.val));
	}

	// And this covers all the comparison operators:
	int opCmp(NumType n) {
		return ... /* implementation here */
	}

Without string mixins, you'd have to copy-n-paste a ton of boilerplate to get the necessary operator overloadings.

//

About the GC, my opinion is that anti-GC sentiment is largely unfounded. The GC greatly simplifies certain tasks (string manipulation, for one thing, and returning recursive data structures like trees). There *are* cases where the GC can cause trouble, like in game engines, esp. given that the GC implementation in D leaves a lot of room for improvement, but a lot of C/C++ programmers have knee-jerk reactions about GCs for no other reason than unfamiliarity breeding contempt (I speak for myself, since I come from a strong C/C++ background, and had the same reaction), than any truly objective measurements. I'd say a large percentage of applications don't need to manually micro-manage memory, and having a GC eliminates an entire class of bugs and greatly improves productivity.

//

But anyway, not to dwell on a contentious issue, let's move on to something oft overlooked (and sometimes even maligned): unittests. I could sing praises of D's built-in unittest blocks all day. When I was young and foolish, I prided myself on being the macho C/C++ programmer whose code will work the first time after I write it. Who needs tests? They are for the weak-minded, who cannot grasp their code in every last detail of their programs in their head with absolute clarity. Real Programmers can write code in their dreams, and it will work the first time they run it.

Except that they don't. :P  When I first started learning D, I tried my best to ignore unittests, telling myself that it's for weaker programmers, but they just kept sitting their, right in the language, and staring at me, all cute-like, until I was shamed into actually writing a few of them just to quiet that nagging voice within. And lo and behold, I started finding bugs in my "perfect" code -- corner cases I missed (even though I painstakingly took care of all the *other* corner cases), subtle typos that compiled but gave the wrong answers, blatant errors that I missed due to obsession over tiny details, etc.. To my chagrin, I discovered that I was *not* the macho perfect programmer that I imagined, and that these unittests for the weak were singlehandedly improving the quality of my code by leaps and bounds.

Sometimes, the mere act of writing unittests would bring my attention to a corner case that I'd missed, and I'd fix the code, long before my past self would've noticed it (by finding it as a runtime bug much later). When I revised the code later, I suddenly could have peace of mind that if the unittests didn't break, then in all likelihood the new code is still correct (or mostly so). Regressions were instantly noticed, rather than weeks or months down the road when I suddenly needed a corner case that I knew was previously working but had since been broken by a later revision.

Unittests in the language == total win.  External unittest frameworks may be more powerful, more flexible, etc., but I'd never use them, 'cos they are too troublesome. I have to switch out of coding mode, open a new file, and possibly shift gears to a different language, write the test, then switch back, and later on it becomes too trouble to keep switching back and forth (and maintaining the unittests) so I'd just not bother running them, and when I do run them, I'd regard them with loathing that they failed Yet Again!, and so I'd hesitate adding any more tests. D's built-in unittests? Total win. You can call other functions, submodules, etc., from the unittest, which makes it much, much easier to setup scaffolding for testing. Trying to do that in a dedicated testing language is an exercise in pain tolerance.

//

And it's getting late, and this email is getting way too long, but I can't help bringing up metaprogramming. That's probably the one thing that singlehandedly converted me to D.  If you got your impression of metaprogramming from that nasty rats'-nest known as C++ templates, then you have my deepest sympathies, but that is *not* representative of what metaprogramming can be. D's templates, which are perhaps better regarded as compile-time parameters, plus compile-time introspection and CTFE, launch D into new heights of expressiveness and compile-time power. Regretfully I need to go soon, but I would point out Dmitry Olshansky's std.regex.ctRegex as an example of metaprogramming that allows you to write regexes in comfortable, natural syntax (unlike C++'s monstrous *cough*Xpressive*ahem*), yet with full compile-time optimization that outperforms basically every other regex library out there. No small feat, one has to admit!

//

Alright. The warts. The bad. There are a few, unfortunately.

- The current AA implementation leaves a lot to be desired... It has
  been improving, and hope is bright, but it's still rather messy, and
  interacts poorly with other built-in features, like const / immutable,
  dtors, @disabled ctors, etc.. Use string keys, and you're mostly OK.
  But once const starts getting in there, or dtors, and things quickly
  get ugly, real fast. Strange corner cases, weird bugs, and other
  pathological implementation holes start showing up.

- Dtors, in general, are prone to nastiness. (One could argue this is
  where having a GC stinks.) Basic usage, for the most part, is OK.
  Start mixing it with, say, closures, containers, @disabled, etc., and
  you quickly run into a minefield of pitfalls that will, on the
  positive side, *quickly* educate you on the arcane deep aspects of D,
  but on the negative side, will also have you pulling out all your hair
  in short order.

- Using array literals with enums (i.e., as a manifest constant) is a
  Very Nasty Very Bad thing to do. It will silently allocate a new array
  every single time you reference that enum, thereby destroying the
  performance benefits of a compiled language very quickly. Having this
  as default behaviour is a big design mistake IMO, and doesn't jive
  with the rest of the language where the default behaviour is the sane
  behaviour, and to get weird behaviour you have to explicitly ask for
  it.

- @safe still isn't quite usable yet. It's there, and many things
  support it, but there are still fundamental features (including
  library features) that aren't @safe-compatible yet, which makes it
  unusable for me. Not to mention there are currently holes in the @safe
  system.

- Contracts (DbC) are a good idea, but the current implementation is
  disappointing. There are still some glaring holes in it that may or
  may not be easy to fix. As things stand, it still has a ways to go.

- ... OK, there's a lot more stuff I want to write but it's late and I
  have to go now, so maybe I'll chime in again later. Maybe.


T

-- 
Food and laptops don't mix.

On Tue, Jul 8, 2014 at 7:50 AM, H. S. Teoh via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote <quite a wall of text>

Wow, what to add to that? Maybe you scared other from participating ;-)

* I'll second metaprogramming: the alliance between templates, CTFE and mixins is really nice. It's *the* feature (or triplet of features) I think of when Walter says that many D parts are kind of "what for?" in isolation but really grow into something awesome by using one another.

* I'd add static introspection to the mix: using static if, __traits(...) and is(...), clunky as the syntax is (there, one 'ugly' thing for you), is easy and very powerful: the idea to have code selecting its flow or extracting information (is that a static array? Does this aggregate have an '.empty' method?). This is the basis for std.algorithm idiom of 'static interface' which allows us compile-time duck typing, which I find very pleasing.

* unittests are nice of course, H. S. Teoh already said it expressively enough :-)

* I'd add arrays and slice. They are wonderfully simple to use, efficient, etc. I remember learning D by translating the Euler Project problems from C++ to D and it was a joy.

Which brings me to another feature I like: ranges. The idea is nothing new, but I find it quite well-realized in D, far easier than other compiled languages alternatives and since they are pervasive in the library, you can really plug components into one another nicely.

For example: http://wiki.dlang.org/Component_programming_with_ranges#Case_Study:_Formatting_a_Calendar

* dub is good, and you can show code.dlang.org in your presentation.

* Bonus point: different compilers. I like that. I regularly use at least two in parallel while developping (comparing results, speed, etc). Kudos to all the guys involved!



As for the bad and ugly:

* it's frustrating not to have a big collection module in Phobos, but
then I didn't propose one, so I'll shut up.
* there are still some not so nice interaction between
const/shared/inout/ auto ref, though I rarely hit them these days
* The compiler still has some quirks: I find it strange you can put
unused qualifiers in many places.

But very honestly, it was already a joy to use a few years ago, and it's becoming better and better.

On Wed, Jul 09, 2014 at 07:51:24AM +0200, Philippe Sigaud via Digitalmars-d-learn wrote:
> On Tue, Jul 8, 2014 at 7:50 AM, H. S. Teoh via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote <quite a wall of text>
> 
> Wow, what to add to that? Maybe you scared other from participating ;-)

I hope not. :)


[...]
> * I'd add static introspection to the mix: using static if,
> __traits(...) and is(...), clunky as the syntax is (there, one 'ugly'
> thing for you), is easy and very powerful:
[...]

Oh yeah, I forgot about that one. The syntax of is-expressions is very counterintuitive (not to mention noisy), and has too many special-cased meanings that are completely non-obvious for the uninitiated, for example:

	// Assume T = some type
	is(T)		// is T a valid type?
	is(T U)		// is T a valid type? If so, alias it to U
	is(T : U)	// is T implicitly convertible to U?
	is(T U : V)	// is T implicitly convertible to V? If so,
			// alias it to U
	is(T U : V, W)	// does T match the type pattern V, for some
			// template arguments W? If so, alias to U
	is(T == U)	// is T the same type as U?

	// You thought the above is (somewhat) consistent? Well look at
	// this one:
	is(T U : __parameters)
			// is T the type of a function? If so, alias U
			// to the parameter tuple of its arguments.

That last one is remarkably pathological: it breaks away from the general interpretation of the other cases, where T is matched against the right side of the expression; here, __parameters is a magic keyword that makes the whole thing mean something else completely. Not to mention, what is "returned" in U is something extremely strange; it looks like a "type tuple", but it's actually something more than that. Unlike usual "type tuples", in addition to encoding the list of types of the function's parameters, it also includes the parameter names and attributes...  except that you can only get at the parameter names using __traits(name,...). But indexing it like a regular "type tuple" will reduce its elements into mere types, on which __traits(name,...) will fail; you need to take 1-element slices of it in order to preserve the additional information.

This strange, inconsistent behaviour only barely begins to make sense once you understand how it's implemented in the compiler. It's the epitome of leaky abstraction.


T

-- 
Do not reason with the unreasonable; you lose by definition.

On Monday, 7 July 2014 at 23:47:26 UTC, Aerolite wrote:
> Hey all,
>
> I've not posted here in a while, but I've been keeping up to speed with D's progress over the last couple of years and remain consistently impressed with the language.
>
> I'm part of a new computing society in the University of Newcastle, Australia, and am essentially known throughout our Computer Science department as 'the D guy'. At the insistence of my peers, I have decided to give an introductory lecture on the D Programming Language, in order to expose more students to the increasingly amazing aspects of D. I expect to cover the good, the bad, the awesome, and the ugly, in a complement-criticism-complement styled talk, and while I have my own opinions regarding each of these things, I'd like a broader view from the community regarding these aspects, so that I may provide as accurate and as useful information as possible.
>
> So, if you would be so kind, give me a bullet list of the aspects of D you believe to be good, awesome, bad, and/or ugly. If you have the time, some code examples wouldn't go amiss either! Try not to go in-depth to weird edge cases - remain general, yet informative. E.g. I consider D's string mixins to be in the 'awesome' category, but its reliance on the GC for large segments of the standard library to be in the 'ugly' category.
>
> Thanks so much for your time!

opDispatch is a mostly untapped goldmine of potential. Just take a look at this thread, where an (almost, depends on the compiler) no-cost safe dereference wrapper was implemented using it: http://forum.dlang.org/post/mailman.2584.1403213951.2907.digitalmars-d@puremagic.com

opDisptach also allows for vector swizzling, which is really nice for any kind of vector work.

One of the uglier things in D is also a long-standing problem with C and C++, in that comparison of signed and unsigned values is allowed.

On Wednesday, 9 July 2014 at 14:51:41 UTC, Meta wrote:
> One of the uglier things in D is also a long-standing problem with C and C++, in that comparison of signed and unsigned values is allowed.

I would like that, if it would be implemented along this line:

/// Returns -1 if a < b, 0 if they are equal or 1 if a > b.
/// this will always yield a correct result, no matter which numeric types are compared.
/// It uses one extra comparison operation if and only if
/// one type is signed and the other unsigned but the signed value is >= 0
/// (that is what you need to pay for stupid choice of type).
int opCmp(T, U)(const(T) a, const(U) b) @primitive if(isNumeric!T && isNumeric!U)
{
   static if(Unqual!T == Unqual!U)
   {
      // use the standard D implementation
   }
   else static if(isFloatingPoint!T || isFloatingPoint!U)
   {
      alias CommonType!(T, U) C;
      return opCmp!(cast(C)a, cast(C)b);
   }
   else static if(isSigned!T && isUnsigned!U)
   {
      alias CommonType!(Unsigned!T, U) C;
      return (a < 0) ? -1 : opCmp!(cast(C)a, cast(C)b);
   }
   else static if(isUnsigned!T && isSigned!U)
   {
      alias CommonType!(T, Unsigned!U) C;
      return (b < 0) ? 1 : opCmp!(cast(C)a, cast(C)b);
   }
   else // both signed or both unsigned
   {
      alias CommonType!(T, U) C;
      return opCmp!(cast(C)a, cast(C)b);
   }
}

Of course without the ! after opCmp in the several cases.

On Wed, Jul 09, 2014 at 04:24:38PM +0000, Dominikus Dittes Scherkl via Digitalmars-d-learn wrote:
> On Wednesday, 9 July 2014 at 14:51:41 UTC, Meta wrote:
> >One of the uglier things in D is also a long-standing problem with C and C++, in that comparison of signed and unsigned values is allowed.
> 
> I would like that, if it would be implemented along this line:
> 
> /// Returns -1 if a < b, 0 if they are equal or 1 if a > b.
> /// this will always yield a correct result, no matter which numeric types are compared.
> /// It uses one extra comparison operation if and only if
> /// one type is signed and the other unsigned but the signed value is >= 0
> /// (that is what you need to pay for stupid choice of type).
[...]

Yeah, I don't see what's the problem with comparing signed and unsigned values, as long as the result is as expected. Currently, however, this code asserts, which is wrong:

	uint x = uint.max;
	int y = -1;
	assert(x > y);


> {
>    static if(Unqual!T == Unqual!U)

Nitpick: should be:

    static if(is(Unqual!T == Unqual!U))


[...]
>    else static if(isSigned!T && isUnsigned!U)
>    {
>       alias CommonType!(Unsigned!T, U) C;
>       return (a < 0) ? -1 : opCmp!(cast(C)a, cast(C)b);
>    }
>    else static if(isUnsigned!T && isSigned!U)
>    {
>       alias CommonType!(T, Unsigned!U) C;
>       return (b < 0) ? 1 : opCmp!(cast(C)a, cast(C)b);
>    }
[...]

Hmm. I wonder if there's a more efficient way to do this.

For the comparison s < u, where s is a signed value and u is an unsigned value, whenever s is negative, the return value of opCmp must be negative.  Assuming 2's-complement representation of integers, this means we simply copy the MSB of s (i.e., the sign bit) to the result. So we can implement s < u as:

	enum signbitMask = 1u << (s.sizeof*8 - 1); // this is a compile-time constant
	return (s - u) | (s & signbitMask); // look ma, no branches!

which would translate (roughly) to the assembly code:

	mov	eax, [<address of s>]
	mov	ebx, [<address of u>]
	mov	ecx, eax	; save the value of s for signbit extraction
	sub	eax, ebx	; s - u
	and	ecx, #$10000000	; s & signbitMask
	or	eax, ecx	; (s - u) | (s & signbitMask)
	(ret		; this is deleted if opCmp is inlined)

which avoids a branch hazard in the CPU pipeline.

Similarly, for the comparison u < s, whenever s is negative, then opCmp must always be positive. So this means we copy over the *negation* of the sign bit of s to the result. So we get this for u < s:

	enum signbitMask = 1u << (s.sizeof*8 - 1); // as before
	return (u - s) & ~(s & signbitMask); // look ma, no branches!

which translates roughly to:

	mov	eax, [<address of u>]
	mov	ebx, [<address of s>]
	sub	eax, ebx	; u - s
	and	ebx, #$10000000	; s & signbitMask
	not	ebx		; ~(s & signbitMask)
	and	eax, ebx	; (u - s) & ~(s & signbitMask)
	(ret		; this is deleted if opCmp is inlined)

Again, this avoid a branch hazard in the CPU pipeline.

In both cases, the first 2 instructions are unnecessary if the values to be compared are already in CPU registers. The naïve implementation of opCmp is just a single sub instruction (this is why opCmp is defined the way it is, BTW), whereas the "smart" signed/unsigned comparison is 4 instructions long. The branched version would look something like this:

	mov	eax, [<address of u>]
	mov	ebx, [<address of s>]
	cmp	ebx, $#0
	jge	label1		; first branch
	mov	eax, $#FFFFFFFF
	jmp	label2		; 2nd branch
label1:
	sub	eax, ebx
label2:
	(ret)

The 2nd branch can be replaced with ret if opCmp is not inlined, but requiring a function call to compare integers seems excessive, so let's assume it's inlined, in which case the 2nd branch is necessary. So as you can see, the branched version is 5 instructions long, and always causes a CPU pipeline hazard.

So I submit that the unbranched version is better. ;-)

(So much for premature optimization... now lemme go and actually benchmark this stuff and see how well it actually performs in practice. Often, such kinds of hacks often perform more poorly than expected due to unforeseen complications with today's complex CPU's. So for all I know, I could've just been spouting nonsense above. :P)


T

-- 
Debian GNU/Linux: Cray on your desktop.

On Wednesday, 9 July 2014 at 17:13:21 UTC, H. S. Teoh via Digitalmars-d-learn wrote:
> On Wed, Jul 09, 2014 at 04:24:38PM +0000, Dominikus Dittes Scherkl via Digitalmars-d-learn wrote:
>> /// Returns -1 if a < b, 0 if they are equal or 1 if a > b.
>> /// this will always yield a correct result, no matter which numeric types are compared.
>> /// It uses one extra comparison operation if and only if
>> /// one type is signed and the other unsigned but the signed value is >= 0
>> /// (that is what you need to pay for stupid choice of type).
> [...]
>
> Yeah, I don't see what's the problem with comparing signed and unsigned
> values, as long as the result is as expected. Currently, however, this
> code asserts, which is wrong:
>
> 	uint x = uint.max;
> 	int y = -1;
> 	assert(x > y);
Yes, this is really bad.
But last time I got the response that this is so to be compatible with C.
That is what I really thought was the reason why D throw away balast from C,
to fix bugs.

>>    static if(Unqual!T == Unqual!U)
>
> Nitpick: should be:
>
>     static if(is(Unqual!T == Unqual!U))
Of course.

> [...]
>>    else static if(isSigned!T && isUnsigned!U)
>>    {
>>       alias CommonType!(Unsigned!T, U) C;
>>       return (a < 0) ? -1 : opCmp!(cast(C)a, cast(C)b);
>>    }
>>    else static if(isUnsigned!T && isSigned!U)
>>    {
>>       alias CommonType!(T, Unsigned!U) C;
>>       return (b < 0) ? 1 : opCmp!(cast(C)a, cast(C)b);
>>    }
> [...]
>
> Hmm. I wonder if there's a more efficient way to do this.
I'm sure. But I think it should be done at the compiler, not in a library.

> {...]
> opCmp is just a single sub instruction (this is why opCmp is defined the
> way it is, BTW), whereas the "smart" signed/unsigned comparison is 4
> instructions long.
> [...]
> you can see, the branched version is 5 instructions long, and always
> causes a CPU pipeline hazard.
>
> So I submit that the unbranched version is better. ;-)
I don't think so, because the branch will only be taken if the signed
type is >= 0 (in fact unsigned). So if the signed/unsigned comparison
is by accident, you pay the extra runtime. But if it is intentional
the signed value is likely to be negative, so you get a correct result
with no extra cost.
Even better for constants, where the compiler can not only evaluate expressions like (uint.max > -1) correct, but it should optimize them completely away!

>
> (So much for premature optimization... now lemme go and actually
> benchmark this stuff and see how well it actually performs in practice.
Yes, we should do this.

> Often, such kinds of hacks often perform more poorly than expected due
> to unforeseen complications with today's complex CPU's. So for all I
> know, I could've just been spouting nonsense above. :P)
I don't see such a compiler change as a hack. It is a strong improvement IMHO.

On Wednesday, 9 July 2014 at 17:13:21 UTC, H. S. Teoh via
Digitalmars-d-learn wrote:
> For the comparison s < u, where s is a signed value and u is an unsigned
> value, whenever s is negative, the return value of opCmp must be
> negative.  Assuming 2's-complement representation of integers, this
> means we simply copy the MSB of s (i.e., the sign bit) to the result. So
> we can implement s < u as:
>
> 	enum signbitMask = 1u << (s.sizeof*8 - 1); // this is a compile-time constant
> 	return (s - u) | (s & signbitMask); // look ma, no branches!

This is a problem, isn't it:

void main()
{
     assert(cmp(0, uint.max) < 0); /* fails */
}
int cmp(int s, uint u)
{
     enum signbitMask = 1u << (s.sizeof*8 - 1); // this is a
compile-time constant
     return (s - u) | (s & signbitMask); // look ma, no branches!
}