On Tuesday, 26 August 2014 at 09:05:05 UTC, Sönke Ludwig wrote:
> When on the other hand a ubyte/ushort/uint range is used, the lexer should validate all string literals.

Yes, so this will be supported? Because this is what is most useful.

I tried using "-disable-linker-strip-dead", but it had no effect. From the error messages it seems the problem is compile-time and not link-time...



On Tuesday, 26 August 2014 at 07:01:09 UTC, Jacob Carlborg wrote:
> On 25/08/14 21:49, simendsjo wrote:
>
>> So ldc can remove quite a substantial amount of code in some cases.
>
> It's because the latest release of LDC has the --gc-sections falg enabled by default.

Am 26.08.2014 11:11, schrieb "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>":
> On Tuesday, 26 August 2014 at 09:05:05 UTC, Sönke Ludwig wrote:
>> When on the other hand a ubyte/ushort/uint range is used, the lexer
>> should validate all string literals.
>
> Yes, so this will be supported? Because this is what is most useful.

If nobody plays a veto card, I'll implement it that way.

On Tuesday, 26 August 2014 at 07:34:05 UTC, Ola Fosheim Gr wrote:
> On Tuesday, 26 August 2014 at 07:24:19 UTC, Don wrote:
>> Processors from AMD have signalling NaN behaviour which is different from processors from Intel.
>>
>> And the situation is worst on most other architectures. It's a lost cause, I think.
>
> I disagree. AFAIK signaling NaN was standardized in IEEE 754-2008.
> So it receives attention.

It was always in IEEE754. The decision in 754-2008 was simply to not remove it from the spec (a lot of people wanted to remove it). I don't think anything has changed.

The point is, existing hardware does not support it consistently. It's not possible at reasonable cost.

---
real uninitialized_var = real.snan;

void foo()
{
  real other_var = void;
  asm {
     fld uninitialized_var;
     fstp other_var;
  }
}
---

will signal on AMD, but not Intel. I'd love for this to work, but the hardware is fighting against us. I think it's useful only for debugging.

On Thursday, 21 August 2014 at 22:35:18 UTC, Sönke Ludwig wrote:
> Following up on the recent "std.jgrandson" thread [1], I've picked up the work (a lot earlier than anticipated) and finished a first version of a loose blend of said std.jgrandson, vibe.data.json and some changes that I had planned for vibe.data.json for a while. I'm quite pleased by the results so far, although without a serialization framework it still misses a very important building block.
>
> Code: https://github.com/s-ludwig/std_data_json
> Docs: http://s-ludwig.github.io/std_data_json/
> DUB: http://code.dlang.org/packages/std_data_json

Do we have any benchmarks for this yet. Note that the main
motivation for a new json parsers was that std.json is remarkable
slow in comparison to python's json or ujson.

On Tuesday, 26 August 2014 at 10:55:20 UTC, Don wrote:
> It was always in IEEE754. The decision in 754-2008 was simply to not remove it from the spec (a lot of people wanted to remove it). I don't think anything has changed.

It was implementation defined before. I think they specified the bit in 2008.

>      fld uninitialized_var;
>      fstp other_var;

This is not SSE, but I guess MOVSS does not create exceptions either. AVX is quite complicated, but searching for "signaling" gives some hints about the semantics you can rely on.

https://software.intel.com/en-us/articles/introduction-to-intel-advanced-vector-extensions

https://software.intel.com/sites/default/files/managed/c6/a9/319433-020.pdf

Ola.

On Tuesday, 26 August 2014 at 12:37:58 UTC, Ola Fosheim Grøstad wrote:

> either. AVX is quite complicated, but searching for "signaling" gives some hints about the semantics you can rely on.
…
> https://software.intel.com/sites/default/files/managed/c6/a9/319433-020.pdf

(Actually, searching for "SNAN" is better…)

With the danger of being noisy, these instructions are subject to floating point exceptions according to my (perhaps sloppy) reading of Intel Architecture Instruction Set Extensions Programming Reference (2012):

(V)ADDPD, (V)ADDPS, (V)ADDSUBPD, (V)ADDSUBPS, (V)CMPPD, (V)CMPPS, (V)CVTDQ2PS, (V)CVTPD2DQ, (V)CVTPD2PS, (V)CVTPS2DQ, (V)CVTTPD2DQ, (V)CVTTPS2DQ, (V)DIVPD, (V)DIVPS, (V)DPPD*, (V)DPPS*, VFMADD132PD, VFMADD213PD, VFMADD231PD, VFMADD132PS, VFMADD213PS, VFMADD231PS, VFMADDSUB132PD, VFMADDSUB213PD, VFMADDSUB231PD, VFMADDSUB132PS, VFMADDSUB213PS, VFMADDSUB231PS, VFMSUBADD132PD, VFMSUBADD213PD, VFMSUBADD231PD, VFMSUBADD132PS, VFMSUBADD213PS, VFMSUBADD231PS, VFMSUB132PD, VFMSUB213PD, VFMSUB231PD, VFMSUB132PS, VFMSUB213PS, VFMSUB231PS, VFNMADD132PD, VFNMADD213PD, VFNMADD231PD, VFNMADD132PS, VFNMADD213PS, VFNMADD231PS, VFNMSUB132PD, VFNMSUB213PD, VFNMSUB231PD, VFNMSUB132PS, VFNMSUB213PS, VFNMSUB231PS, (V)HADDPD, (V)HADDPS, (V)HSUBPD, (V)HSUBPS, (V)MAXPD, (V)MAXPS, (V)MINPD, (V)MINPS, (V)MULPD, (V)MULPS, (V)ROUNDPS, (V)ROUNDPS, (V)SQRTPD, (V)SQRTPS, (V)SUBPD, (V)SUBPS

(V)ADDSD, (V)ADDSS, (V)CMPSD, (V)CMPSS, (V)COMISD, (V)COMISS, (V)CVTPS2PD, (V)CVTSD2SI, (V)CVTSD2SS, (V)CVTSI2SD, (V)CVTSI2SS, (V)CVTSS2SD, (V)CVTSS2SI, (V)CVTTSD2SI, (V)CVTTSS2SI, (V)DIVSD, (V)DIVSS, VFMADD132SD, VFMADD213SD, VFMADD231SD, VFMADD132SS, VFMADD213SS, VFMADD231SS, VFMSUB132SD, VFMSUB213SD, VFMSUB231SD, VFMSUB132SS, VFMSUB213SS, VFMSUB231SS, VFNMADD132SD, VFNMADD213SD, VFNMADD231SD, VFNMADD132SS, VFNMADD213SS, VFNMADD231SS, VFNMSUB132SD, VFNMSUB213SD, VFNMSUB231SD, VFNMSUB132SS, VFNMSUB213SS, VFNMSUB231SS, (V)MAXSD, (V)MAXSS, (V)MINSD, (V)MINSS, (V)MULSD, (V)MULSS, (V)ROUNDSD, (V)ROUNDSS, (V)SQRTSD, (V)SQRTSS, (V)SUBSD, (V)SUBSS, (V)UCOMISD, (V)UCOMISS

VCVTPH2PS, VCVTPS2PH

So I guess Intel floating point exceptions trigger on computations, but not on moves?

Ola.

On Tuesday, 26 August 2014 at 12:37:58 UTC, Ola Fosheim Grøstad wrote:
> On Tuesday, 26 August 2014 at 10:55:20 UTC, Don wrote:
>> It was always in IEEE754. The decision in 754-2008 was simply to not remove it from the spec (a lot of people wanted to remove it). I don't think anything has changed.
>
> It was implementation defined before. I think they specified the bit in 2008.
>
>>     fld uninitialized_var;
>>     fstp other_var;
>
> This is not SSE, but I guess MOVSS does not create exceptions either.

No, it's more subtle. On the original x87, signalling NaNs are triggered for 64 bits loads, but not for 80 bit loads. You have to read the fine print to discover this. I don't think the behaviour was intentional.

On Monday, 25 August 2014 at 14:04:12 UTC, Sönke Ludwig wrote:
> Am 25.08.2014 15:07, schrieb Don:
>> On Thursday, 21 August 2014 at 22:35:18 UTC, Sönke Ludwig wrote:
>>> Following up on the recent "std.jgrandson" thread [1], I've picked up
>>> the work (a lot earlier than anticipated) and finished a first version
>>> of a loose blend of said std.jgrandson, vibe.data.json and some
>>> changes that I had planned for vibe.data.json for a while. I'm quite
>>> pleased by the results so far, although without a serialization
>>> framework it still misses a very important building block.
>>>
>>> Code: https://github.com/s-ludwig/std_data_json
>>> Docs: http://s-ludwig.github.io/std_data_json/
>>> DUB: http://code.dlang.org/packages/std_data_json
>>>
>>> The new code contains:
>>> - Lazy lexer in the form of a token input range (using slices of the
>>>   input if possible)
>>> - Lazy streaming parser (StAX style) in the form of a node input range
>>> - Eager DOM style parser returning a JSONValue
>>> - Range based JSON string generator taking either a token range, a
>>>   node range, or a JSONValue
>>> - Opt-out location tracking (line/column) for tokens, nodes and values
>>> - No opDispatch() for JSONValue - this has shown to do more harm than
>>>   good in vibe.data.json
>>>
>>> The DOM style JSONValue type is based on std.variant.Algebraic. This
>>> currently has a few usability issues that can be solved by
>>> upgrading/fixing Algebraic:
>>>
>>> - Operator overloading only works sporadically
>>> - No "tag" enum is supported, so that switch()ing on the type of a
>>>   value doesn't work and an if-else cascade is required
>>> - Operations and conversions between different Algebraic types is not
>>>   conveniently supported, which gets important when other similar
>>>   formats get supported (e.g. BSON)
>>>
>>> Assuming that those points are solved, I'd like to get some early
>>> feedback before going for an official review. One open issue is how to
>>> handle unescaping of string literals. Currently it always unescapes
>>> immediately, which is more efficient for general input ranges when the
>>> unescaped result is needed, but less efficient for string inputs when
>>> the unescaped result is not needed. Maybe a flag could be used to
>>> conditionally switch behavior depending on the input range type.
>>>
>>> Destroy away! ;)
>>>
>>> [1]: http://forum.dlang.org/thread/lrknjl$co7$1@digitalmars.com
>>
>>
>> One missing feature (which is also missing from the existing std.json)
>> is support for NaN and Infinity as JSON values. Although they are not
>> part of the formal JSON spec (which is a ridiculous omission, the
>> argument given for excluding them is fallacious), they do get generated
>> if you use Javascript's toString to create the JSON. Many JSON libraries
>> (eg Google's) also generate them, so they are frequently encountered in
>> practice. So a JSON parser should at least be able to lex them.
>>
>> ie this should be parsable:
>>
>> {"foo": NaN, "bar": Infinity, "baz": -Infinity}
>
> This would probably best added as another (CT) optional feature. I think the default should strictly adhere to the JSON specification, though.

Yes, it should be optional, but not a compile-time option.
I think it should parse it, and based on a runtime flag, throw an error (perhaps an OutOfRange error or something, and use the same thing for values that exceed the representable range).

An app may accept these non-standard values under certain circumstances and not others. In real-world code, you see a *lot* of these guys.

Part of the reason these are important, is that NaN or Infinity generally means some Javascript code just has an uninitialized variable. Any other kind of invalid JSON typically means something very nasty has happened. It's important to distinguish these.