What is your favorite D feature? (page 4)

On Fri, Jun 23, 2017 at 06:51:13PM +0000, Anonymouse via Digitalmars-d wrote: [...]
> Fairly specific, but foreach (member; someStruct.tupleof).
> 
> Part of my work on my IRC bot has been to serialise structs into configuration files (since std.json was *non-trivial* to deal with), and foreaching a someStruct.tupleof or a __traits(allMembers, symbol) allows for really, really interesting stuff.

Yup!  Recently I wrote a module that, given a struct S annotated with certain UDA's, automatically:

1) Fills an instance of S with values extracted from a given range of key/value pairs (with automatic type conversion from string input to the correct field type via the awesome std.conv.to);

2) Displays a usage message listing the name of each field in S, its type, along with a message explaining that the field is for (the message is in a UDA attached to the struct field);

3) Given an instance of S, produces a listing of field names to the current values in the given instance.

Put together, these 3 things can basically replace getopt(): you just declare a struct with UDAs attaching descriptions to struct fields, then just pass the program arguments along to this module and you're all set. It will parse the program arguments and fill in the struct for you, automatically generate a helpful usage message, and even dump the values of the obtained settings if you implement a verbose mode, for example.

But it gets even better. Yesterday I added a new UDA that lets you split up your struct into multiple nested structs, but have all the field names visible at the top-level, i.e., say you have modules A, B, C, each declaring their own configuration structs:

	module A;
	struct Config {
		@Desc("The number of iterations")
		int x;
	}

	module B;
	struct Config {
		@Desc("The starting value")
		float y;
	}

	module B;
	struct Config {
		@Desc("The output label")
		string z;
	}

Then you can combine all of these configuration parameters into a single struct in your main program:

	struct Config {
		@Denest A.Config aCfg;
		@Denest B.Config bCfg;
		@Denest C.Config cCfg;
	}

The @Denest UDA tells the parser that aCfg.x should appear to the user as simply "x", bCfg.y as just "y", and cCfg.z as "z".  So the user doesn't have to know how your program is internally divided into modules; all of the settings appear as top-level settings.

But internally, you pass Config.aCfg to module A, Config.bCfg to module B, etc., thereby avoiding leaking details about module B to module A, which would happen if you put everything into a single struct (you might end up accidentally having code in module A depend on a setting that belongs to module B, which breaks encapsulation and makes the code harder to maintain).

D rawkz, I tell ya.

T

-- 
VI = Visual Irritation

June 23, 2017

Re: What is your favorite D feature?

Posted by H. S. Teoh
in reply to Seb

Permalink

H. S. Teoh

Posted in reply to Seb

Permalink

On Thu, Jun 22, 2017 at 12:48:25AM +0000, Seb via Digitalmars-d wrote:
> Hi,
> 
> I am currently trying to modernize the D code example roulette on the
> dlang.org front page [1]. Hence, I would love to hear about your
> favorite feature(s) in D.
> Ideas:
> - favorite language construct
> - favorite code sample
> - "only possible in D"
[...]

Today, I just got another idea for this: show off std.parallelism, especially parallel foreach!!!

Here's the background: I have a program that takes user input, converts it into D code using a code template, invokes dmd to compile it into a shared library, and then loads the shared library so that the main program can call the compiled code directly.  The output is a series of images that can be assembled into an animation.  The first version of the code looked like this (greatly simplified, of course):

	void main(string[] args) {
		auto input = parseUserInput(args);
		auto libhandle = compile(input);

		foreach (i; 0 .. n) {
			auto outfile = File(...);
			... // do stuff
			libhandle(...);	// call compiled code
			... // more stuff
			outfile.rawWrite(image);
		}
	}

When n is large, of course, this runs rather slowly, because it's generating the images one by one.  Parallel foreach comes to the rescue:

	void main(string[] args) {
		auto input = parseUserInput(args);
		auto libhandle = compile(input);

		foreach (i; parallel(iota(0, n))) {	// ***
			auto outfile = File(...);
			... // do stuff
			libhandle(...);	// call compiled code
			... // more stuff
			outfile.rawWrite(image);
		}
	}

The only change is marked in // *** above. It's just a 1-line change, and boom, now the loop automatically spawns worker threads and generates the images in parallel.  Instant 6x speedup on my AMD Hexacore machine!

This all sounds very obvious, but actually there are some D-specific features at play here that make this even remotely possible:

- First of all, I deliberately described the shared library bit, because
  at first I wasn't sure if std.parallel would Just Work(tm), the reason
  being that the shared object has global variables, and the loop body
  does change the values of those global variables. After reading the
  warning about implicit data sharing between threads in the docs, I was
  a bit wary of writing the loop as a parallel loop to begin with.

  Thankfully, TLS comes to the rescue: because the shared object is
  compiled as D code, the globals sit in the TLS segment, rather than
  C/C++'s default global storage.  So modifying those globals turned out
  to be completely harmless w.r.t. parallel foreach: each thread would
  have its own copy of the global, so they don't interfere with each
  other.  More importantly, this holds not only for the main program,
  but also for the *shared library* loaded via dlopen(). (It was this
  last part that I wasn't 100% sure about; it was already obvious that
  if these globals were in the main program, it would be no problem by
  default. But for this to also extend to manually-loaded shared
  libraries was a big relief for me.)

  That was the most tricky bit, but then other D features come together
  to make it possible to change a linear loop into a parallel loop by
  basically just editing the foreach statement:

- opApply: in spite of being the scorned illegitimate second cousin of
  the beloved range-based counterparts, this is what made it possible to
  keep the only difference between a "native" foreach and a parallel
  foreach just a matter of a simple syntactic change in the foreach
  aggregate (just add a call to `parallel`). Without this, we'd have to
  hoist the loop body out into a delegate or some such, and replace the
  foreach with an awkward call to some obscure function in std.parallel.
  Not horrible, but just lots of fussy little editing details to take
  care of. Thanks to opApply, all we needed to do was to essentially add
  `parallel` to the one line.  The compiler takes care of converting the
  loop body into the requisite delegate, transparently close over the
  required local variables, etc..  No fuss, no muss.

- iota: the underappreciated construct, without which we'd have to
  manually construct some indexing array or some other such thing for
  parallel() to work with. I.e., a lot more tedious typing than just a
  simple 1-line change.

- Finally, parallel() itself: one of the most ingenious API designs I've
  ever seen, it nicely encapsulates the dirty details of managing
  TaskPools manually, spawning worker threads, assigning Tasks to
  threads, synchronizing with threads afterwards, and all of that fussy
  boilerplate that one has to drown in if doing this in a language like
  C or C++.

Thanks to all of the above, D lets you switch from a linear loop to a multithreaded loop just by changing 1 line of code.  I don't know of any other imperative language that can boast this. (Of course, functional languages, esp. the pure ones, have this "by default". But we're talking about a dirty, mutating, imperative language with loop bodies that modify *global* variables -- to do this in spite of said mutation of globals, now *that's* an achievement.)

T

-- 
English is useful because it is a mess. Since English is a mess, it maps well onto the problem space, which is also a mess, which we call reality. Similarly, Perl was designed to be a mess, though in the nicest of all possible ways. -- Larry Wall

Forums