I have a user who submitted a bug report for one of my projects. The error is in std\encoding.d on line 2554.

The problem arises when he types "google.com.hk/search?q={{query}}" (exact string) into this function:
https://github.com/spikespaz/search-deflector/blob/master/source/setup.d#L253-L278

Here is the issue on GH:
https://github.com/spikespaz/search-deflector/issues/12

I really can't figure this one out and any help would be appreciated. Thanks.

On Tuesday, 28 August 2018 at 13:38:39 UTC, spikespaz wrote:
> I have a user who submitted a bug report for one of my projects. The error is in std\encoding.d on line 2554.
>
> The problem arises when he types "google.com.hk/search?q={{query}}" (exact string) into this function:
> https://github.com/spikespaz/search-deflector/blob/master/source/setup.d#L253-L278
>
> Here is the issue on GH:
> https://github.com/spikespaz/search-deflector/issues/12
>
> I really can't figure this one out and any help would be appreciated. Thanks.

To add to the parent, I've recompiled with debugging and this is the full traceback. As I suspected, it's an error with curl:

0x00007FF77A162F4D in std.encoding.EncodingScheme std.encoding.EncodingScheme.create(immutable(char)[])
0x00007FF77A14300A in std.net.curl._decodeContent!char._decodeContent at C:\D\ldc2\import\std\net\curl.d(1213)
0x00007FF77A143610 in std.net.curl._basicHTTP!char._basicHTTP at C:\Users\spike\Documents\github.com\spikespaz\search-deflector\source\common.d(1078)
0x00007FF77A142C97 in std.net.curl.get!(std.net.curl.HTTP, char).get at C:\Users\spike\Documents\github.com\spikespaz\search-deflector\source\common.d(562)
0x00007FF77A1411E0 in std.net.curl.get!(std.net.curl.AutoProtocol, char).get at C:\D\ldc2\import\std\net\curl.d(574)
0x00007FF77A146635 in setup.getCustomEngine at C:\Users\spike\Documents\github.com\spikespaz\search-deflector\source\setup.d(264)
0x00007FF77A145C8E in setup.setup at C:\Users\spike\Documents\github.com\spikespaz\search-deflector\source\setup.d(44)
0x00007FF77A145AD0 in D main at C:\Users\spike\Documents\github.com\spikespaz\search-deflector\source\setup.d(26)
0x00007FF77A1AAD8D in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
0x00007FF77A1AAC3A in d_run_main
0x00007FF77A1C9658 in __scrt_common_main_seh at f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl(283)
0x00007FFEC50687E4 in BaseThreadInitThunk
0x00007FFEC701B311 in RtlUserThreadStart

import std.stdio : stdout, writeln;

writeln(__MODULE__, ":", __LINE__); stdout.flush;

Trace it that way. The flush is important.

https://github.com/dlang/phobos/blob/master/std/net/curl.d#L1197

get!ubyte will disable encoding.

On 8/28/18 9:38 AM, spikespaz wrote:
> I have a user who submitted a bug report for one of my projects. The error is in std\encoding.d on line 2554.
> 
> The problem arises when he types "google.com.hk/search?q={{query}}" (exact string) into this function:
> https://github.com/spikespaz/search-deflector/blob/master/source/setup.d#L253-L278 
> 
> 
> Here is the issue on GH:
> https://github.com/spikespaz/search-deflector/issues/12
> 
> I really can't figure this one out and any help would be appreciated. Thanks.

The answer is: the encoding "big5" isn't supported by std.encoding.

You can add a new subclass and register it, to handle the encoding. Maybe this article can help you write one: https://en.wikipedia.org/wiki/Big5

You want to subclass EncodingScheme. See instructions at the top of https://dlang.org/phobos/std_encoding.html

-Steve

We have a reasonably number (but quiet) Chinese user base, we should really support big5 and its extensions out-right.

On Tuesday, 28 August 2018 at 13:38:39 UTC, spikespaz wrote:
> The problem arises when he types "google.com.hk/search?q={{query}}" (exact string)

AFAIK, google has a parameter to output results in utf8.