On Sunday, 6 September 2015 at 14:36:53 UTC, chris stevens wrote:
> Hi All,
>
> I am considering using D for my latest project and there are a few features I would like and am not entirely sure at this point whether D has them. They are:
>
> - dynamic creation of classes/structs at runtime (think I can emulate this with variants/dynamic)

At runtime, no. However... if it doesn't actually need to be runtime and is in the code itself you can have code be generated at compile time.

> - dynamic compilation of code files at runtime

Only by running a compiler. But as above, if it doesn't strictly have to be at runtime and everything is specified in the source files, you can compute strings at compile-time and mix them in to generate code.

> - some basic code creation tools

mixin, template mixin, CTFE.

> Are the above possible?

It really depends on what you want to do. A lot of the dynamic code generation that goes on is done at runtime because that's what other languages support. In my experience a lot of those use-cases can be done at compile-time in D. A famous example is compile-time regular expressions. Unless you're writing a tool similar to grep, all regexes in your program are known at compile-time. They don't _need_ to be parsed/interpreted/compiled at run-time, that's just what every other language that isn't D does.

So, if your use-case depends on information from the file-system, user interaction, networking, etc. then your only choice is to generate text files with D code and compile it. If, on the other hand, it's all defined by the code you write, then D has excellent compile-time code generation tools at your disposal.

Atila

On Tuesday, 8 September 2015 at 21:51:03 UTC, wobbles wrote:
> "Previously undefined". As far as I know, this is impossible in D. Thr compiler has to know how much memory to allocate/request and it has to know that at compiletime (else it wouldn't be the compiler!)

http://wiki.dlang.org/Dynamic_typing

This is what I saw that made me think that I could. Have had another closer look and I do believe it's possible.

On Wednesday, 9 September 2015 at 08:24:06 UTC, Kagamin wrote:
> As I understand, you wanted to build an AST tree and format it to string?

Thanks again. So I can build an AST and convert it to full D source from that? I think that's what i'm likely going need to do.

On Wednesday, 9 September 2015 at 09:14:30 UTC, Atila Neves wrote:
> So, if your use-case depends on information from the file-system, user interaction, networking, etc. then your only choice is to generate text files with D code and compile it. If, on the other hand, it's all defined by the code you write, then D has excellent compile-time code generation tools at your disposal.

Thanks for the in depth reply. Unfortunately, yes, my use case depends on user input so definitions will not be known at compiletime. I'm keen to learn D though so I think I'm going to try and make it work with dynamically compiling and loading libraries.

On Tuesday, 8 September 2015 at 21:51:03 UTC, wobbles wrote:
> "Previously undefined". As far as I know, this is impossible in D. Thr compiler has to know how much memory to allocate/request and it has to know that at compiletime (else it wouldn't be the compiler!)

There was also this:

http://rosettacode.org/wiki/Add_a_variable_to_a_class_instance_at_runtime#D

That made me think I could emulate the functionality I need.

On 09/08/2015 12:30 PM, chris stevens wrote:

> create new
> (previously undefined) classes in D at runtime that I could then use
> with Object.factory to create instances of.

SDC is a D compiler as a library but as far as I know, it's not complete yet:

  https://github.com/SDC-Developers/SDC

Ali

On Wednesday, 9 September 2015 at 10:23:55 UTC, chris stevens wrote:
> http://wiki.dlang.org/Dynamic_typing
>
> This is what I saw that made me think that I could. Have had another closer look and I do believe it's possible.

These are things I wrote, so let me explain how they work: they do not attempt to actually run through the D compiler, but instead use tagged memory to check it all at runtime.

This is kinda similar to creating a class, but it has no help from the compiler and no static type checks.


So, let's say the user wanted to create an object with two integers, x and y. Code like the jsvar in the link does it with an associative array: var[string]. var is defined as a struct with a storage variable for the data, and a variable saying what type it is. All operations check that type variable. Then the names are indexed through the hash map. This is like how javascript engines work.

You could also do it a bit differently by making a type definition for the whole object and allocating the memory that way.



You can also go with the run the compiler and load a library option. There's different pros and cons for each approach.



Here's some example code you might go with the second option: a type definition and dynamically allocated memory:

---

struct ObjectHandle {
	immutable(ObjectDefinition) type;
	ubyte[] data;

	ubyte[] getRawDataForField(string field) {
		auto info = type.fields[field];
		auto raw = data[info.offset .. info.offset + info.ti.tsize];
		return raw;
	}

	string toString() {
		string s = type.name ~ "_dynamic(\n";
		foreach(fieldName, fieldDefinition; type.fields) {
			s ~= "\t" ~ fieldName ~ ": " ~ typeToString(fieldDefinition.ti, getRawDataForField(fieldName));
			s ~= "\n";

		}
		return s ~ ")";
	}
}

class ObjectDefinition {
	// OffsetTypeInfo is defined in object.d
	// http://dlang.org/phobos/object.html#OffsetTypeInfo
	//
	// These TypeInfo things are D's built-in runtime reflection.
	// We're borrowing that information and the types to create our
	// own extensions.
	OffsetTypeInfo[string] fields;

	string name;

	this(string name) {
		this.name = name;
	}

	void addField(string type, string name) {
		switch(type) {
			case "int":
				OffsetTypeInfo field;
				field.ti = typeid(int);
				field.offset = dataSize(); // you might also want to do some alignment here...
				fields[name] = field;
			break;
			// you can add other supported types here. You could even loop through a list
			// of them at compile time and generate this code btw.
			default: assert(0, "unsupported type " ~ type);
		}
	}

	int dataSize() const {
		int size = 0;
		foreach(k, v; fields)
			size += v.ti.tsize;
		return size;
	}

	// this is immutable because you don't want
	// the layout to change after you've created it.
	//
	// So it forces the factory to be called only after
	// the definition is set in stone.
	ObjectHandle factory() immutable {
		auto data = new ubyte[](this.dataSize()); // dynamically allocate space for our object's variables...
		foreach(k, field; fields) { // and now initialize them properly
			auto i = cast(const(ubyte)[]) field.ti.init();
			if(i.length)
				data[field.offset .. field.offset + field.ti.tsize] = i[];
		}
		return ObjectHandle(this, data); // and finally return it, along with a reference to this so we can identify the type later

	}
}

// Similarly to addField, doing operations on our field need to dispatch based on
// type. I'll only show int. Others could be added in the if/else chain, or also
// auto-generated by a compile time loop.
string typeToString(const(TypeInfo) t, const(ubyte)[] rawData) {
	import std.conv;
	if(t == typeid(int)) {
		int i = *(cast(int*) rawData.ptr);
		return to!string(i);
	}

	return null;
}



void main() {
	auto definition = new ObjectDefinition("MyRuntimeType");
	definition.addField("int", "x");
	definition.addField("int", "y");

	auto finalDefinition = cast(immutable) definition;
	definition = null; // we can't edit it anymore, so nulling out the mutable original reference


	auto object = finalDefinition.factory();

	auto rawData = object.getRawDataForField("x");
	rawData[0] = 15; // since I know it is an int, I can do this now
	// but in actual practice, you'd want to do some kind of conversions and type checking based on the type field
	// instead of poking raw memory

	import std.stdio;
	writeln(object.toString());
}

---



There's nothing really D specific in there so far, though the compile time loops would be to add more types to support.

Supporting all types would be a lot of code and tricky in points. Tying it into TypeInfo itself and actually making your dynamic class available via static interfaces and other existing facilities like Object.factory gets trickier.... but is doable. You'd need to match the field layouts in your raw data, then fill them in appropriately and poke the appropriate arrays in the runtime.

Should be a learning experience if you actually want to run with it!