On 2013-11-21 08:38, Jacob Carlborg wrote:
> On 2013-11-21 01:16, Simen Kjærås wrote:
>
>> The result of re-validating is performance loss. The result of missed
>> validation is a bug. Also, in just a few lines, you can make a version
>> that will *not* decay to the original type:
>>
>>    struct Validated(alias fn, T) {
>>        private T _value;
>>        @property inout
>>        T value() {
>>            return _value;
>>        }
>>    }
>>
>>    // validated() is identical to before.
>>
>> Sure, using it is a bit more verbose than using the unadorned type,
>> which is why I chose to make the original version automatically decay.
>> This is a judgment where sensible people may disagree, even with
>> themselves on a case-by-case basis.
>
> It's still accessible via "value".
>

Indeed it is. If we want to make it perfectly impossible to get at the contents, so as to hinder all possible use of the data, I suggest this solution:

struct Validated {}

Validated validate() {
    return Validated.init;
}

--
  Simen

On Wednesday, 20 November 2013 at 18:30:58 UTC, Dmitry Olshansky wrote:
>
> And it decays to the naked type in a blink of an eye. And some function down the road will do the validation again...
>

Not if that function down the road only accepted validated in the first place because that is what it needed. Follow the rule - if you need validated instance only accept validated type - do not try to validate.

On Wednesday, 20 November 2013 at 17:45:43 UTC, Simen Kjærås wrote:
> On 20.11.2013 12:49, Jacob Carlborg wrote:
>> On 2013-11-20 12:16, Jonathan M Davis wrote:
>>
>>> You'd do it the other way around by having something like
>>>
>>> ValidatedString!char s = validateString("hello world");
>>
>> Right.
>>
>>> ValidatedString would then avoid any extra validation when iterating
>>> over the
>>> characters, though I don't know how much of an efficiency gain that would
>>> actually be given that much of the validation occurs naturally when
>>> decoding
>>> or using stride. It would have the downside that any function which
>>> specializes on strings would likely have to then specialize on
>>> ValidatedString
>>> as well. So, while I agree with the idea in concept, I'd propose that we
>>> benchmark the difference in decoding and striding without the checks
>>> and see if
>>> there actually is much difference. Because if there isn't, then I
>>> don't think
>>> that it's worth going to the trouble of adding something like
>>> ValidatedString.
>>
>> If not just if the string is valid UTF-8. There can be many other types
>> of valid strings. Or rather other functions that have additional
>> requirements. Like sanitized filenames, HTML/SQL escaped strings and so on.
>
> May I suggest:
>
> struct Validated(alias fn, T) {
>     private T value;
>     @property inout
>     T get() {
>         return value;
>     }
> }
>
> Validated!(fn, T) validate(alias fn, T)(T value) {
>     Validated!(fn, T) result;
>     fn(value);
>     result.value = value;
>     return result;
> }
>
> void functionThatTakesSanitizedFileNames(Validated!(sanitizeFileName, string) path) {
>    // Do stuff
> }

What if you have more that just one validation, e.g. Positive and LessThan42?
Is Positive!LessThan42!int the same type as LessThan42!Positive!int? Implicitly convertible?

I feel that it might be better to use @attributes here instead. Something like:

@positive int validatePositive(int value) {
  assert(value > 0);
  return value;
}

@lessThan42 validateLessThan42(int value) {
  assert(value < 42);
  return value;
}

Now you can have @positive @lessThan42 int value = validatePositive(validateLessThan42(x));

It also doesn't involve creating new types.

On Thursday, 21 November 2013 at 22:51:43 UTC, inout wrote:
> What if you have more that just one validation, e.g. Positive and LessThan42?
> Is Positive!LessThan42!int the same type as LessThan42!Positive!int? Implicitly convertible?

Allow multiple validation functions. Then a Validated type is only valid if validationFunction1(val) && validationFunction2(val) &&...

Validated!(isPositive, lessThan42, int) validatedInt = validate!(isPositive, lessThan42)(34);
//Do stuff with validatedInt

Or just pass a function that validates that the int is both positive and less than 42, which would be much simpler.

> ...
>
> It also doesn't involve creating new types.

Creating new types is what allows us to provide static, compiler-verified guarantees.

On 22.11.2013 00:50, Meta wrote:
> On Thursday, 21 November 2013 at 22:51:43 UTC, inout wrote:
>> What if you have more that just one validation, e.g. Positive and
>> LessThan42?
>> Is Positive!LessThan42!int the same type as LessThan42!Positive!int?
>> Implicitly convertible?
>
> Allow multiple validation functions. Then a Validated type is only valid
> if validationFunction1(val) && validationFunction2(val) &&...
>
> Validated!(isPositive, lessThan42, int) validatedInt =
> validate!(isPositive, lessThan42)(34);
> //Do stuff with validatedInt

I believe inout's point was this, though:

  Validated!(isPositive, lessThan42, int) i = foo();

  Validated!(isPositive, int) n = i; // Fails.
  Validated!(lessThan42, isPositive, int) r = i; // Fails.

This is of course less than optimal.

If a type such as Validate is to be added to Phobos, these problems need to be fixed first.


> Or just pass a function that validates that the int is both positive and
> less than 42, which would be much simpler.



-- 
  Simen

On 22.11.2013 02:55, Simen Kjærås wrote:
> On 22.11.2013 00:50, Meta wrote:
>> On Thursday, 21 November 2013 at 22:51:43 UTC, inout wrote:
>>> What if you have more that just one validation, e.g. Positive and
>>> LessThan42?
>>> Is Positive!LessThan42!int the same type as LessThan42!Positive!int?
>>> Implicitly convertible?
>>
>> Allow multiple validation functions. Then a Validated type is only valid
>> if validationFunction1(val) && validationFunction2(val) &&...
>>
>> Validated!(isPositive, lessThan42, int) validatedInt =
>> validate!(isPositive, lessThan42)(34);
>> //Do stuff with validatedInt
>
> I believe inout's point was this, though:
>
>    Validated!(isPositive, lessThan42, int) i = foo();
>
>    Validated!(isPositive, int) n = i; // Fails.
>    Validated!(lessThan42, isPositive, int) r = i; // Fails.
>
> This is of course less than optimal.
>
> If a type such as Validate is to be added to Phobos, these problems need to be fixed first.
>
>
>> Or just pass a function that validates that the int is both positive and less than 42, which would be much simpler.

I've created a version of Validated now that takes 1 or more constraints, and where a type whose constraints are a superset of another's, is implicitly convertible to that. Sadly, because of D's lack of certain implicit conversions, there are limits.

Attached is source (validation.d), and some utility functions that are necessary for it to compile (utils.d).

Is this worth working more on? Should it be in Phobos? Other critique?

Oh, sorry about those stupid questions, we have a term for that:

Detroy!

-- 
   Simen

On Sunday, 24 November 2013 at 17:35:51 UTC, Simen Kjærås wrote:
>> I believe inout's point was this, though:
>>
>>    Validated!(isPositive, lessThan42, int) i = foo();
>>
>>    Validated!(isPositive, int) n = i; // Fails.
>>    Validated!(lessThan42, isPositive, int) r = i; // Fails.
>>
>> This is of course less than optimal.
>>
>> If a type such as Validate is to be added to Phobos, these problems need
>> to be fixed first.
>>
>>
>>> Or just pass a function that validates that the int is both positive and
>>> less than 42, which would be much simpler.
>
> I've created a version of Validated now that takes 1 or more
> constraints, and where a type whose constraints are a superset of
> another's, is implicitly convertible to that. Sadly, because of D's lack
> of certain implicit conversions, there are limits.
>
> Attached is source (validation.d), and some utility functions that are
> necessary for it to compile (utils.d).
>
> Is this worth working more on? Should it be in Phobos? Other critique?
>
> Oh, sorry about those stupid questions, we have a term for that:
>
> Detroy!

Awesome, I was messing around with something similar but you beat me to the punch. A couple things:

- The function validated would probably be better named validate, since it actually performs validation and returns a validated type. The struct's name is fine.

- I think it'd be better to change "static if (is(typeof(fn(value)) == bool))" to "static if (is(typeof(fn(value)) : bool))", which rather than checking that the return type is exactly bool, it only checks that it's implicitly convertible to bool, AKA "truthy".

- It might be a good idea to have a version(AlwaysValidate) block in assumeValidated for people who don't care about code speed and want maximum safety, that would always run the validation functions. Also, it might be a good idea to mark assumeValidated @system, because it blatantly breaks the underlying assumptions being made in the first place. Code that wants to be rock-solid @safe will be restricted to using only validate. Or maybe that's going too far.

- Validated doesn't work very well with reference types. The following fails:

class CouldBeNull
{
}

bool notNull(T)(T t)
if (is(T == class))
{
	return t !is null;
}

//Error: cannot implicitly convert expression (this._value) of type inout(CouldBeNull) to f505.CouldBeNull
void takesNonNull(Validated!(CouldBeNull, notNull) validatedT)
{
}

- On the subject of reference types, I don't think Validated handles them quite correctly. This is a problem I ran into, and it's not an easy one. Assume for a second that there's a class FourtyTwo that *does* work with Validated:

	class FortyTwo
	{
		int i = 42;
	}
	
	bool containsFortyTwo(FortyTwo ft)
	{
		return ft.i == 42;
	}
	
	void mutateFortyTwo(Validated!(FortyTwo, containsFortyTwo) fortyTwo)
	{
		fortyTwo.i = 43;
	}
	
	auto a = validated!containsFortyTwo(new FortyTwo());
	auto b = a;
	//Passes
	assert(a.i == 42);
	assert(b.i == 42);
	mutateFortyTwo(a);
	//Fails
	assert(a.i == 43);
	assert(b.i == 43);

This is an extremely contrived example, but it illustrates the problem of using reference types with Validated. It gets even hairier if i itself were a reference type, like a slice:

	void mutateCopiedValue(Validated!(FortyTwo, containsFortyTwo) fortyTwo)
	{
		//We're not out of the woods yet
		int[] arr = fortyTwo.i;
		arr[0] += 1;
	}

        //Continuing from previous example,
        //except i is now an array
	mutateCopiedValue(b);
	assert(a.i[0] == 44);
	assert(b.i[0] == 44);

Obviously in this case you could just .dup i, but what if i were a class itself? It'd be extremely easy to accidentally invalidate every Validated!(FortyTwo, ...) in the program in a single swipe. It gets even worse if i were some class reference to which other, non-validated references existed. Changing those naked references would change i, and possibly invalidate it.

On Monday, 25 November 2013 at 07:24:10 UTC, Meta wrote:
> 	auto a = validated!containsFortyTwo(new FortyTwo());
> 	auto b = a;
> 	//Passes
> 	assert(a.i == 42);
> 	assert(b.i == 42);
> 	mutateFortyTwo(a);
> 	//Fails
> 	assert(a.i == 43);
> 	assert(b.i == 43);

"//Fails" should be "//Passes" as well.

On 2013-11-25 08:24, Meta wrote:
> - The function validated would probably be better named validate, since it actually performs validation and returns a validated type. The struct's name is fine.

Yeah, I was somewhat torn there, but I think you're right. Fixed.


> - I think it'd be better to change "static if (is(typeof(fn(value)) ==
> bool))" to "static if (is(typeof(fn(value)) : bool))", which rather than
> checking that the return type is exactly bool, it only checks that it's
> implicitly convertible to bool, AKA "truthy".

Even better - test if 'if (fn(value)) {}' compiles. Fixed.


> - It might be a good idea to have a version(AlwaysValidate) block in assumeValidated for people who don't care about code speed and want maximum safety, that would always run the validation functions. Also, it might be a good idea to mark assumeValidated @system, because it blatantly breaks the underlying assumptions being made in the first place. Code that wants to be rock-solid @safe will be restricted to using only validate. Or maybe that's going too far.

@safe is only for memory safety, which this is not. I agree it would be nice to mark assumeValidated as 'warning, may not do what it claims', but @safe is not really the correct indicator of that.


> - Validated doesn't work very well with reference types. The following fails:
>
> class CouldBeNull
> {
> }
>
> bool notNull(T)(T t)
> if (is(T == class))
> {
>      return t !is null;
> }
>
> //Error: cannot implicitly convert expression (this._value) of type
> inout(CouldBeNull) to f505.CouldBeNull
> void takesNonNull(Validated!(CouldBeNull, notNull) validatedT)
> {
> }

Yeah, found that. It's a bug in value(), which should return inout(T), not T. Fixed.


> - On the subject of reference types, I don't think Validated handles them quite correctly. This is a problem I ran into, and it's not an easy one. Assume for a second that there's a class FourtyTwo that *does* work with Validated:
>
>      class FortyTwo
>      {
>          int i = 42;
>      }
>
>      bool containsFortyTwo(FortyTwo ft)
>      {
>          return ft.i == 42;
>      }
>
>      void mutateFortyTwo(Validated!(FortyTwo, containsFortyTwo) fortyTwo)
>      {
>          fortyTwo.i = 43;
>      }
>
>      auto a = validated!containsFortyTwo(new FortyTwo());
>      auto b = a;
>      //Passes
>      assert(a.i == 42);
>      assert(b.i == 42);
>      mutateFortyTwo(a);
>      //Fails
>      assert(a.i == 43);
>      assert(b.i == 43);
>
> This is an extremely contrived example, but it illustrates the problem of using reference types with Validated. It gets even hairier if i itself were a reference type, like a slice:
>
>      void mutateCopiedValue(Validated!(FortyTwo, containsFortyTwo)
> fortyTwo)
>      {
>          //We're not out of the woods yet
>          int[] arr = fortyTwo.i;
>          arr[0] += 1;
>      }
>
>          //Continuing from previous example,
>          //except i is now an array
>      mutateCopiedValue(b);
>      assert(a.i[0] == 44);
>      assert(b.i[0] == 44);
>
> Obviously in this case you could just .dup i, but what if i were a class itself? It'd be extremely easy to accidentally invalidate every Validated!(FortyTwo, ...) in the program in a single swipe. It gets even worse if i were some class reference to which other, non-validated references existed. Changing those naked references would change i, and possibly invalidate it.

This is a known shortcoming for which I see no good workaround. It would be possible to use std.traits.hasAliasing to see which types can be safely .dup'ed and only allow those types, but this is not a solution I like.

I guess it could print a warning when used with unsafe types. If I were to do that, I would still want some way to turn that message off. Eh. Maybe there is no good solution.


What else is new?
- Better error messages for invalid constraints (testing if an int is
   null, a string is divisible by 3 or an array has a database
   connection, e.g.)
- Fixed a bug in opCast (I love that word - in Norwegian it [oppkast]
   means puke. ...anyways...) when converting to an incompatible wrapped
   value.

--
   Simen

Am Fri, 22 Nov 2013 02:55:44 +0100
schrieb Simen Kjærås <simen.kjaras@gmail.com>:

> On 22.11.2013 00:50, Meta wrote:
> > On Thursday, 21 November 2013 at 22:51:43 UTC, inout wrote:
> >> What if you have more that just one validation, e.g. Positive and
> >> LessThan42?
> >> Is Positive!LessThan42!int the same type as LessThan42!Positive!int?
> >> Implicitly convertible?
> >
> > Allow multiple validation functions. Then a Validated type is only valid
> > if validationFunction1(val) && validationFunction2(val) &&...
> >
> > Validated!(isPositive, lessThan42, int) validatedInt =
> > validate!(isPositive, lessThan42)(34);
> > //Do stuff with validatedInt
> 
> I believe inout's point was this, though:
> 
>    Validated!(isPositive, lessThan42, int) i = foo();
> 
>    Validated!(isPositive, int) n = i; // Fails.
>    Validated!(lessThan42, isPositive, int) r = i; // Fails.
> 
> This is of course less than optimal.
> 
> If a type such as Validate is to be added to Phobos, these problems need to be fixed first.

Can you write a templated assignment operator that
accepts any Validated!* instance and builds the set difference
of validation functions that are missing on the assigned value?
E.g. in the case of n = i: {isPositive} / {isPositive,
lessThan42} = emtpy set.

-- 
Marco