Jump to page: 1 2
Thread overview
Lazily Initialized Variables that are Compatible with const/immutable
Nov 17
razyk
November 17

An uncommon, but not rare, situation that I run into from time to time, is that there are variables used either globally or as a member of a class that are expensive in their resource usage (time to compute, or memory size), but which are only sometimes used, e.g. only when certain functions are called, which depends on the runtime usage of the program.

The concept of lazy initialization is used in these situations to avoid paying this cost up front, and deferring it until the value is used. E.g. if your object has a member named value, then obj.value may refer to a method that returns a cached value, and if the cache is empty, produces the value and caches it.

However, in D, such usage runs into problems when the object/method is either const or immutable. Making such changes requires modifying a variable that technically existed at the time obj was created.

While working on a project, I encountered a solution in Dart which I think could be carried over into D:

https://dart.dev/guides/language/language-tour#late-variables

When you mark a variable as late but initialize it at its declaration, then the initializer runs the first time the variable is used. This lazy initialization is handy in a couple of cases:

    The variable might not be needed, and initializing it is costly.
    You’re initializing an instance variable, and its initializer needs access to this.

In the following example, if the temperature variable is never used, then the expensive readThermometer() function is never called:

    // This is the program's only call to readThermometer().
    late String temperature = readThermometer(); // Lazily initialized.

The already existing keyword lazy could be used for this purpose in D, and then it would be possible to write code such as:

class Thing {
  lazy immutable String piDigits = computePiDigits();

  String computePiDigits() { ... }

  double getTemperature() const {
    // If only this function is used, piDigits is never computed.
  }

  double getCircumferance() const {
    // Some silly usage of piDigits, which causes computePiDigits() to be run.
    return piDigits.to!double * radius * 2.0;
  }
}

Is there another way to perform lazy initialization that is compatible with const/immutable, or is this needed as a language feature?

November 17

On Thursday, 17 November 2022 at 10:57:55 UTC, Vijay Nayar wrote:

>

An uncommon, but not rare, situation that I run into from time to time, is that there are variables used either globally or as a member of a class that are expensive in their resource usage (time to compute, or memory size), but which are only sometimes used, e.g. only when certain functions are called, which depends on the runtime usage of the program.

The concept of lazy initialization is used in these situations to avoid paying this cost up front, and deferring it until the value is used. E.g. if your object has a member named value, then obj.value may refer to a method that returns a cached value, and if the cache is empty, produces the value and caches it.

However, in D, such usage runs into problems when the object/method is either const or immutable. Making such changes requires modifying a variable that technically existed at the time obj was created.

[...]

Is there another way to perform lazy initialization that is compatible with const/immutable, or is this needed as a language feature?

No, this is needed as a bultin feature I'd say. I sometime think that since we can have local static variablbes, we could have local member variables. Similarly hiddden, only accessible through a getter, which replaces the need for a const member.

November 17

On Thursday, 17 November 2022 at 10:57:55 UTC, Vijay Nayar wrote:

>

Is there another way to perform lazy initialization that is compatible with const/immutable, or is this needed as a language feature?

Wouldn't precomputing this const/immutable data at compile time via CTFE be a better idea in general?

Unless it's some sort of a sparse data structure with a huge memory footprint and the goal is to postpone memory allocation. But then you would probably also want to free memory as soon as possible after you are done using it and const/immutable would be a hindrance again.

November 17

On Thursday, 17 November 2022 at 12:16:41 UTC, Siarhei Siamashka wrote:

>

On Thursday, 17 November 2022 at 10:57:55 UTC, Vijay Nayar wrote:

>

Is there another way to perform lazy initialization that is compatible with const/immutable, or is this needed as a language feature?

Wouldn't precomputing this const/immutable data at compile time via CTFE be a better idea in general?

Unless it's some sort of a sparse data structure with a huge memory footprint and the goal is to postpone memory allocation. But then you would probably also want to free memory as soon as possible after you are done using it and const/immutable would be a hindrance again.

CTFE would indeed have better performance for data that is known at compile time, but often, the objects created which need lazy evaluation, are specific to the instance and runtime data.

A few examples include:

  • Initializing costly hardware connections/setup.
  • Creating large objects used to organize or index data that are only needed sometimes, e.g. when serializing for writing to disk.
  • Retrieving information from a network, e.g. a JWT or other authorization information to be used for a session.
November 17
On 11/17/22 02:57, Vijay Nayar wrote:

> Is there another way to perform lazy initialization that is compatible
> with const/immutable, or is this needed as a language feature?

If the variable needs to be shared, there is a library solution:

  https://dlang.org/phobos/std_concurrency.html#.initOnce

And here is a user of initOnce():

  https://github.com/dlang/phobos/blob/master/std/parallelism.d#L3569

Ali

November 17
On 17.11.22 13:04, MorteFeuille123 wrote:
>>
>> Is there another way to perform lazy initialization that is compatible with const/immutable, or is this needed as a language feature?
> 
> No, this is needed as a bultin feature I'd say. I sometime think that since we can have local static variablbes, we could have local member variables. Similarly hiddden, only accessible through a getter, which replaces the need for a const member.

module2.d
```
import std.stdio;

private string _lazy1;

string compute(){
  writeln("computing...");
  return "_lazy1 val";
}

string lazy1(){
  if (_lazy1 == null)
    _lazy1 = compute();
  return _lazy1;
}	
```

module1.d
```
import std.stdio;
import module2;

void main() {
  writeln(lazy1);
  writeln(lazy1);
}
```

November 17
On Thursday, 17 November 2022 at 15:33:47 UTC, razyk wrote:
> On 17.11.22 13:04, MorteFeuille123 wrote:
>>>

While the solutions from razyk and Ali take care of lazy initialization, with Ali's solution also having the benefit of being thread-safe, none of these methods are actually compatible with `const` or `immutable`, meaning that in any case where lazy initialization is needed, you cannot have a const class, you have to pick one or the other in the current setup.
November 17
On 17.11.22 22:49, Vijay Nayar wrote:
> On Thursday, 17 November 2022 at 15:33:47 UTC, razyk wrote:
>> On 17.11.22 13:04, MorteFeuille123 wrote:
>>>>
> 
> While the solutions from razyk and Ali take care of lazy initialization, with Ali's solution also having the benefit of being thread-safe, none of these methods are actually compatible with `const` or `immutable`, meaning that in any case where lazy initialization is needed, you cannot have a const class, you have to pick one or the other in the current setup.

Well, that's what `const` and `immutable` mean. If you need to restrict mutation in some other way, classic encapsulation works.

Note that it is not all that often appropriate to use those qualifiers when writing traditional OO code.
November 18

On Thursday, 17 November 2022 at 22:01:21 UTC, Timon Gehr wrote:

>

On 17.11.22 22:49, Vijay Nayar wrote:

>

On Thursday, 17 November 2022 at 15:33:47 UTC, razyk wrote:

>

On 17.11.22 13:04, MorteFeuille123 wrote:

> >

While the solutions from razyk and Ali take care of lazy initialization, with Ali's solution also having the benefit of being thread-safe, none of these methods are actually compatible with const or immutable, meaning that in any case where lazy initialization is needed, you cannot have a const class, you have to pick one or the other in the current setup.

Well, that's what const and immutable mean. If you need to restrict mutation in some other way, classic encapsulation works.

Note that it is not all that often appropriate to use those qualifiers when writing traditional OO code.

In response to this argument, I would cite the Dart language's implementation of late variables: https://dart.dev/guides/language/language-tour#late-variables

To its core, Dart is fully object oriented, and it makes frequent use of immutable and const values which is used to optimize performance (in Dart, const is closer to D's immutable). The mechanism presented in the link above demonstrates the concept of lazy evaluation while also being compatible with const. The variable is still const, however, the computation of its initial and only value is deferred until its first usage.

November 18
On 18.11.22 10:00, Vijay Nayar wrote:
> On Thursday, 17 November 2022 at 22:01:21 UTC, Timon Gehr wrote:
>> On 17.11.22 22:49, Vijay Nayar wrote:
>>> On Thursday, 17 November 2022 at 15:33:47 UTC, razyk wrote:
>>>> On 17.11.22 13:04, MorteFeuille123 wrote:
>>>>>>
>>>
>>> While the solutions from razyk and Ali take care of lazy initialization, with Ali's solution also having the benefit of being thread-safe, none of these methods are actually compatible with `const` or `immutable`, meaning that in any case where lazy initialization is needed, you cannot have a const class, you have to pick one or the other in the current setup.
>>
>> Well, that's what `const` and `immutable` mean. If you need to restrict mutation in some other way, classic encapsulation works.
>>
>> Note that it is not all that often appropriate to use those qualifiers when writing traditional OO code.
> 
> In response to this argument, I would cite the Dart language's implementation of `late` variables: https://dart.dev/guides/language/language-tour#late-variables
> 
> To its core, Dart is fully object oriented, and it makes frequent use of immutable and const values

I am not sure what the point is. Dart does not support D const/immutable and D does not support Dart final/const. They are different.

D's built-in qualifiers really don't mix well with OO. One reason is the embarrassing limitation that there is no way to tail-qualify a class reference. Another reason is that "absolutely no mutation" is often incompatible with encapsulation as you may want to cache results or lazily initialize class fields.

It is not surprising that with Dart being OO first, they have designed their superficially similar features to fit OO use cases.

> which is used to optimize performance (in Dart, `const` is closer to D's `immutable`).

Actually, it is closer to `static immutable`.
Dart `const` means "compile-time constant".

There is also `final`, but it is not transitive.

> The mechanism presented in the link above demonstrates the concept of lazy evaluation while also being compatible with const. The variable is still const, however, the computation of its initial and only value is deferred until its first usage.

I don't understand why you'd want to do that. A const variable has a const initializer and it is final. What do you gain from late initializing such a variable? Also, Dart designers agree with me to the point where you actually can't even do that: "Members can't be declared to be both 'const' and 'late'"

(Note that in D you are actually less restricted and can initialize immutable globals in a `shared static this` module constructor at program startup.)

You can do "late final", but the meaning of that is equivalent to doing encapsulation manually with runtime checks like I suggested. Maybe create a mixin/mixin template or struct that does the things for you that Dart has built-in if you need those. The built-in qualifiers are probably not what you want if you need late initialization. I guess an alternative would be to create a DIP that adds Dart-like features, but I don't know how well that would be received.

« First   ‹ Prev
1 2