Thread overview
A idea to improve diet-ng compile speed!
Aug 12, 2021
workman
Aug 12, 2021
workman
Aug 12, 2021
workman
Aug 12, 2021
workman
Aug 12, 2021
workman
August 12, 2021

diet-ng is great and I love to use it, but it is too slow to compile.

One idea to speedup is use preprocessor like DMD compile C header file with GCC -E.

A diet-ng compiler write in D, translate template file into D source code, then mixin from projects. A runtime diet-ng compiler will easaly speed up 1000 times compare to CTFE version, and you don't get any performance loss.

A runtime diet-ng compiler also possible compile template into dynamic library file, you edit view files and a file monitor will recompile the dynamic library then http server reload it, no need recompile full projects.

Add new feature into diet will be more easy, since developer don't limit to use CTFE code.

D CTFE is great, but not suit for task like dieg-ng.

August 12, 2021

On Thursday, 12 August 2021 at 07:55:08 UTC, workman wrote:

>

D CTFE is great, but not suit for task like dieg-ng.

simple code like this, finish in 0.004s for 20 files. ldc2 take hours to finish in product mode.

import diet.defs;
import diet.dom;
import diet.input;
import diet.parser;
import diet.html;

import std.stdio;
import std.file;
import std.path;
int main(string[] args){
	foreach(filepath; args[1..$]){
		scope filename	= baseName(filepath);
		scope ext	= extension(filepath);
		if( ext != ".dt" ) {
			writefln("%s extension %s is not .dt", filepath, ext);
			break;
		}
		scope dfile	= filepath[0..$-1];
		try {
			scope data	= cast(string) read(filepath);
			scope doc	= parseDiet(data, filename);
			scope code	= getHTMLMixin(doc, dietOutputRangeName, HTMLOutputStyle.compact);
			if( std.file.exists(dfile) && isFile(dfile) ) {
				scope _code = cast(string) read(dfile);
				if(  _code == code ) continue;

			}
			writefln("%s => %s", filepath, dfile);
			if( is_cached ) {
				continue;
			}
			std.file.write(dfile, code);

		} catch(Exception e) {
			writefln("%s throw %s", filepath, e);
			break;
		}
	}
	return 0;
}

dieg not support inherit and layout like https://mozilla.github.io/nunjucks/ or https://jinja.palletsprojects.com/en/3.0.x/, to add more advance function CTFE will crashed to compile.

August 12, 2021

On 8/12/21 3:55 AM, workman wrote:

>

diet-ng is great and I love to use it,  but it is too slow to compile.

One idea to speedup is use preprocessor like DMD compile C header file with GCC -E.

A diet-ng compiler write in D, translate template file into D source code, then mixin from projects.  A runtime diet-ng compiler will easaly speed up 1000 times compare to CTFE version, and you don't get any performance loss.

A runtime diet-ng compiler also possible compile template into dynamic library file,   you edit view files and a file monitor will recompile the dynamic library then http server reload it, no need recompile full projects.

Add new feature into diet will be more easy, since developer don't limit to use CTFE code.

D CTFE is great, but not suit for task like dieg-ng.

diet-ng is really hard to deal with at runtime.

I added a feature that allows any non-code portions of the template to be reinterpreted at runtime, so for instance, if you want to add or remove an html class from an element, that is doable without rebuilding. I also created a diet precompiler to move most of the CTFE heavy-lifting to a pre-compilation step (it still runs a checksum on the files at compile-time, which is annoying to me, but I haven't bothered yet to strip that part out).

But changing the code means recompiling. And that is not always easy, because diet templates are compiled with specific context and aliases, which are not easy to replicate as a subpart of the project. I have considered splitting my routes out into individual subprojects to see if that helps with the build times.

With precompiling templates and live mode, diet-ng is at least tolerable for development.

If I were to do my own template system, with my experience in diet-ng, it would be focused on runtime interpretation (only a change to the input data structure or backend code would require a recompile) with the ability to compile everything for performance.

-Steve

August 12, 2021

On 8/12/21 9:45 AM, workman wrote:

>

On Thursday, 12 August 2021 at 07:55:08 UTC, workman wrote:

>

D CTFE is great, but not suit for task like dieg-ng.

simple code like this, finish in 0.004s for 20 files. ldc2 take hours to finish in product mode.

From experience, I think the 2 hours is probably not because of diet CTFE interpretation. But I could be wrong, I don't know what your application is like.

>

dieg not support inherit and layout like https://mozilla.github.io/nunjucks/ or https://jinja.palletsprojects.com/en/3.0.x/, to add more advance function CTFE will  crashed to compile.

diet supports layout inheritance via the extends directive.

e.g. all my files look like:

extends layout
block extraCss
  // extra css includes
block extraJs
  // extra js includes
block content
  // page content

Where layout.dt provides the overall layout.

-Steve

August 12, 2021

On Thursday, 12 August 2021 at 13:49:32 UTC, Steven Schveighoffer wrote:

>

But changing the code means recompiling. And that is not always easy, because diet templates are compiled with specific context and aliases, which are not easy to replicate as a subpart of the project. I have considered splitting my routes out into individual subprojects to see if that helps with the build times.

Mixed d code with views made vibe.d like old PHP code, it hard to maintain if business logic change fast, and not easy to testing.

I am consider use a javascript server side render template like

On Thursday, 12 August 2021 at 14:04:03 UTC, Steven Schveighoffer wrote:

>

On 8/12/21 9:45 AM, workman wrote:
From experience, I think the 2 hours is probably not because of diet CTFE interpretation. But I could be wrong, I don't know what your application is like.

Not really take hours, but very slow and require a lot memory.

>

diet supports layout inheritance via the extends directive.

e.g. all my files look like:

extends layout
block extraCss
  // extra css includes
block extraJs
  // extra js includes
block content
  // page content

Thanks for the tips, I will try extends. What will be really nice is some thing like:

{% extends "base.html" %}

    {% block sidebar %}
        <h3>Table Of Contents</h3>
        ...
        {{ parent() }}
    {% endblock %}
    ```



August 12, 2021

On 8/12/21 10:11 AM, workman wrote:

>

On Thursday, 12 August 2021 at 13:49:32 UTC, Steven Schveighoffer wrote:

>

But changing the code means recompiling. And that is not always easy, because diet templates are compiled with specific context and aliases, which are not easy to replicate as a subpart of the project. I have considered splitting my routes out into individual subprojects to see if that helps with the build times.

Mixed d code with views made vibe.d like old PHP code, it hard to maintain if business logic change fast, and not easy to testing.

I am consider use a javascript server side render template like

On Thursday, 12 August 2021 at 14:04:03 UTC, Steven Schveighoffer wrote:

>

On 8/12/21 9:45 AM, workman wrote:
From experience, I think the 2 hours is probably not because of diet CTFE interpretation. But I could be wrong, I don't know what your application is like.

Not really take hours, but very slow and require a lot memory.

OK. My experience is that precompiling templates and using the cache feature reduced my build time from 38 seconds to 11, and also reduced memory consumption quite a bit. However, you still must build production without these modes. I would like to introduce a production precompilation mode, where the diet processing is done separately, and the code just imports the file without having to checksum it. But I haven't had the need yet.

If you have memory constraints that prevent you from building production, this might be a viable path.

> >

diet supports layout inheritance via the extends directive.

e.g. all my files look like:

extends layout
block extraCss
  // extra css includes
block extraJs
  // extra js includes
block content
  // page content

Thanks for the tips, I will try extends. What will be really nice is some thing like:

{% extends "base.html" %}

     {% block sidebar %}
         <h3>Table Of Contents</h3>
         ...
         {{ parent() }}
     {% endblock %}
     ```

Yes, that is exactly what extends does. Same keywords even. See docs (they are a bit sparse, but there's an example there to help you understand how to do it).

-Steve

August 12, 2021

On Thursday, 12 August 2021 at 14:51:28 UTC, Steven Schveighoffer wrote:

>

Yes, that is exactly what extends does. Same keywords even. See docs (they are a bit sparse, but there's an example there to help you understand how to do it).

-Steve

Thanks again for the tips.

The extends file(.ext) is not implement yet, and include section.dt is not work like other popular template system ( need super() or parent() ).

I am think of use https://mozilla.github.io/nunjucks/ as a client side solution, it is easy to add a server side render solution with quickjs or a nodejs.

August 12, 2021

On Thursday, 12 August 2021 at 15:48:29 UTC, workman wrote:

>

On Thursday, 12 August 2021 at 14:51:28 UTC, Steven Schveighoffer wrote:

>

Yes, that is exactly what extends does. Same keywords even. See docs (they are a bit sparse, but there's an example there to help you understand how to do it).

-Steve

Thanks again for the tips.

The extends file(.ext) is not implement yet, and include section.dt is not work like other popular template system ( need super() or parent() ).

I am think of use https://mozilla.github.io/nunjucks/ as a client side solution, it is easy to add a server side render solution with quickjs or a nodejs.

extends file(.ext) work for me.