Am Wed, 20 Nov 2013 16:26:59 -0800
schrieb Walter Bright <newshound2@digitalmars.com>:

> Utf validation isn't the only form of validation for strings. You could, for example, validate that the string doesn't contain SQL injection code, or contains a correctly formatted date, or has a name that is guaranteed to be in your employee database, or is a valid phone number, or is a correct email address, etc.
> 
> Again, validation is not defined by D, it is defined by the constraints YOUR PROGRAM puts on it.

A checked type for database access goes a bit beyond the scope
of the proposal. You'd need to encapsulate a transaction that
needs to be working on a snapshot of the database state and
fail if data changed in another transaction.
Otherwise you could validate a name against the database just
before someone else deletes it and thus invalidates the string.

With a DB transaction wrapped in the validation, assignment between two "validated" strings becomes a pretty sophisticated runtime action, while the original proposals evolved around validation functions that can be pure. /This allows us to assign one validated string type to another with no runtime overhead./

-- 
Marco

On 2013-11-25 13:00, Marco Leise wrote:
> Am Fri, 22 Nov 2013 02:55:44 +0100
> schrieb Simen Kjærås <simen.kjaras@gmail.com>:
>
>> On 22.11.2013 00:50, Meta wrote:
>>> On Thursday, 21 November 2013 at 22:51:43 UTC, inout wrote:
>>>> What if you have more that just one validation, e.g. Positive and
>>>> LessThan42?
>>>> Is Positive!LessThan42!int the same type as LessThan42!Positive!int?
>>>> Implicitly convertible?
>>>
>>> Allow multiple validation functions. Then a Validated type is only valid
>>> if validationFunction1(val) && validationFunction2(val) &&...
>>>
>>> Validated!(isPositive, lessThan42, int) validatedInt =
>>> validate!(isPositive, lessThan42)(34);
>>> //Do stuff with validatedInt
>>
>> I believe inout's point was this, though:
>>
>>     Validated!(isPositive, lessThan42, int) i = foo();
>>
>>     Validated!(isPositive, int) n = i; // Fails.
>>     Validated!(lessThan42, isPositive, int) r = i; // Fails.
>>
>> This is of course less than optimal.
>>
>> If a type such as Validate is to be added to Phobos, these problems need
>> to be fixed first.
>
> Can you write a templated assignment operator that
> accepts any Validated!* instance and builds the set difference
> of validation functions that are missing on the assigned value?
> E.g. in the case of n = i: {isPositive} / {isPositive,
> lessThan42} = emtpy set.

Do you mean this?

Validated!(int, isPositive, lessThan42) a =
    validated!(isPositive, lessThan42)(13);
Validated!(int, isPositive) b = a;
a = b; // Only tests lessThan42

If so, you're mostly right that this should be done. I am however of the opinion that conversions that may throw should be marked appropriately, so this will be the right way:

a = validated!(isPositive, lessThan42)(b); // Only tests lessThan42

New version now available on GitHub:
http://git.io/hEe0MA
http://git.io/QEP-kQ

--
  Simen

On Monday, 25 November 2013 at 08:52:14 UTC, Simen Kjærås wrote:
> @safe is only for memory safety, which this is not. I agree it would be
> nice to mark assumeValidated as 'warning, may not do what it claims',
> but @safe is not really the correct indicator of that.

What about a version flag, then, that can be passed to specify that the user wants assumeValidated() to run the validation functions as well?

> This is a known shortcoming for which I see no good workaround. It would
> be possible to use std.traits.hasAliasing to see which types can be
> safely .dup'ed and only allow those types, but this is not a solution I
> like.

It's a hard problem. This is a case where a Unique!T type would be really useful.

> What else is new?
> - Better error messages for invalid constraints (testing if an int is
>    null, a string is divisible by 3 or an array has a database
>    connection, e.g.)
> - Fixed a bug in opCast (I love that word - in Norwegian it [oppkast]
>    means puke. ...anyways...) when converting to an incompatible wrapped
>    value.
>
> --
>    Simen

Keep up the good work!

On 2013-11-26 06:37, Meta wrote:
> On Monday, 25 November 2013 at 08:52:14 UTC, Simen Kjærås wrote:
>> @safe is only for memory safety, which this is not. I agree it would be
>> nice to mark assumeValidated as 'warning, may not do what it claims',
>> but @safe is not really the correct indicator of that.
>
> What about a version flag, then, that can be passed to specify that the
> user wants assumeValidated() to run the validation functions as well?

That's already in:
http://git.io/EdHw8A

--
  Simen

On Monday, 25 November 2013 at 13:01:43 UTC, Simen Kjærås wrote:
> On 2013-11-25 13:00, Marco Leise wrote:
>> Am Fri, 22 Nov 2013 02:55:44 +0100
>> schrieb Simen Kjærås <simen.kjaras@gmail.com>:
>>
>>> On 22.11.2013 00:50, Meta wrote:
>>>> On Thursday, 21 November 2013 at 22:51:43 UTC, inout wrote:
>>>>> What if you have more that just one validation, e.g. Positive and
>>>>> LessThan42?
>>>>> Is Positive!LessThan42!int the same type as LessThan42!Positive!int?
>>>>> Implicitly convertible?
>>>>
>>>> Allow multiple validation functions. Then a Validated type is only valid
>>>> if validationFunction1(val) && validationFunction2(val) &&...
>>>>
>>>> Validated!(isPositive, lessThan42, int) validatedInt =
>>>> validate!(isPositive, lessThan42)(34);
>>>> //Do stuff with validatedInt
>>>
>>> I believe inout's point was this, though:
>>>
>>>    Validated!(isPositive, lessThan42, int) i = foo();
>>>
>>>    Validated!(isPositive, int) n = i; // Fails.
>>>    Validated!(lessThan42, isPositive, int) r = i; // Fails.
>>>
>>> This is of course less than optimal.
>>>
>>> If a type such as Validate is to be added to Phobos, these problems need
>>> to be fixed first.
>>
>> Can you write a templated assignment operator that
>> accepts any Validated!* instance and builds the set difference
>> of validation functions that are missing on the assigned value?
>> E.g. in the case of n = i: {isPositive} / {isPositive,
>> lessThan42} = emtpy set.
>
> Do you mean this?
>
> Validated!(int, isPositive, lessThan42) a =
>     validated!(isPositive, lessThan42)(13);
> Validated!(int, isPositive) b = a;
> a = b; // Only tests lessThan42
>
> If so, you're mostly right that this should be done. I am however of the opinion that conversions that may throw should be marked appropriately, so this will be the right way:
>
> a = validated!(isPositive, lessThan42)(b); // Only tests lessThan42
>
> New version now available on GitHub:
> http://git.io/hEe0MA
> http://git.io/QEP-kQ
>
> --
>   Simen

I find this to be too verbose to be useful. And you also need to
be very careful not to discard any existing qualifiers on input
and carry them over. This will essentially make any function that
uses them to be templated, while all the instances will be the
same (yet have a different body since no D compiler merges
identical functions).

I still find wrapping int with some type to add a tag to it
without adding any methods is not a great idea - it doesn't scale
well with composition and tag propagation. Any operation that
expects int will essentially discard all the qualifiers.

Am Mon, 25 Nov 2013 14:01:28 +0100
schrieb Simen Kjærås <simen.kjaras@gmail.com>:

> On 2013-11-25 13:00, Marco Leise wrote:
> > Can you write a templated assignment operator that
> > accepts any Validated!* instance and builds the set difference
> > of validation functions that are missing on the assigned value?
> > E.g. in the case of n = i: {isPositive} / {isPositive,
> > lessThan42} = emtpy set.
> 
> Do you mean this?
> 
> Validated!(int, isPositive, lessThan42) a =
>      validated!(isPositive, lessThan42)(13);
> Validated!(int, isPositive) b = a;
> a = b; // Only tests lessThan42
> 
> If so, you're mostly right that this should be done. I am however of the opinion that conversions that may throw should be marked appropriately, so this will be the right way:
> 
> a = validated!(isPositive, lessThan42)(b); // Only tests lessThan42
> 
> New version now available on GitHub:
> http://git.io/hEe0MA
> http://git.io/QEP-kQ
> 
> --
>    Simen

Yes, that is what I had in mind.

-- 
Marco

On 26.11.2013 21:14, inout wrote:
> I find this to be too verbose to be useful.

This I understand. It is actually the best argument I can find in favor of doing constraints checking upon construction, rather than in a separate construction function. This allows you to use one alias instead of two.


> And you also need to
> be very careful not to discard any existing qualifiers on input
> and carry them over. This will essentially make any function that
> uses them to be templated, while all the instances will be the
> same (yet have a different body since no D compiler merges
> identical functions).

Could you give an example of this? It's a bit unclear to me what you mean. Is it this sort of thing:

  auto doPrimeStuff(Validated!(int, isPrime) a){return a;}
  auto doLessThan42Stuff(Validated!(int, lessThan42) a){return a;}

  Validated!(int, isPrime, lessThan42) i = 13;

  i.doPrimeStuff().doLessThan42Stuff();

Where the second chained function call fails due to lessThan42 being removed from the constraints? (There's also the problem that this wouldn't work in the first place due to D's lack of implicit conversions)

> I still find wrapping int with some type to add a tag to it
> without adding any methods is not a great idea - it doesn't scale
> well with composition and tag propagation. Any operation that
> expects int will essentially discard all the qualifiers.

And any operation of the kind you describe is likely to change the value so the constraints need to be checked again. abs(Validated!(int, isNegative)) cannot possibly return the same type it received.

-- 
  Simen

On 11/20/13, 18:45, Lars T. Kyllingstad wrote:
> I think we should use ubyte[] to a greater extent for data which is
> potentially *not* valid UTF.  Examples include interfacing with C
> functions, where I think there is a tendency towards always translating
> C char to D char, when they are in fact not equivalent.  Another example
> is, again, std.file.read(), which currently returns void[].  I guess it
> is a matter of taste, but I think ubyte[] would be more appropriate
> here, since you can actually use it for something without casting it first.

+1

Especially the windows APIs, they never take UTF-8(*) but consistently get translated to taking D char :(

In fact, if we want a good translation from C to D, we should be using D byte. On most platforms I've run into have C char is signed. (To be honest, you don't see 'byte' much in D code, so it would make the ported code stand out even more.)

* except from MultiByteToWideChar

On Tuesday, 26 November 2013 at 20:14:15 UTC, inout wrote:
> Any operation that expects int will essentially discard all the qualifiers.

It isn't surprising that any operation that expects int will get int. To take advantage of Validated, an operation has to expect Validated.