March 16, 2022
On Monday, 14 March 2022 at 22:20:42 UTC, rikki cattermole wrote:
>
> The recommended solution by Unicode is to use Trie tables for Look Up Tables (LUTs).
>
> https://en.wikipedia.org/wiki/Trie
>
> You can generate these as read only global arrays and are very fast for this.

Thank your for this hint. I'll have to check whether this is applicable for this context. For example this lookup table appears to be a bijection.
March 16, 2022

On Tuesday, 15 March 2022 at 03:01:05 UTC, Salih Dincer wrote:

>

OMG, I gasp at my computer screen and waited for minutes. :)

When you edit the code at the back-end level, you can use system resources in the best way. I think you should start with dlang.dmd.backend.aarray

If we use the following codes with Ali's code by separate the keys and values, it compiles fast on DMD and works correctly:

[snip]

SDB@79

Thank you, but wouldn't using the DMD backend make it so that it won't compile with LDC? I am not that knowledgeable in compiler internals so I don't know.

March 16, 2022

On Wednesday, 16 March 2022 at 18:40:35 UTC, zhad3 wrote:

>

On Tuesday, 15 March 2022 at 03:01:05 UTC, Salih Dincer wrote:

>

OMG, I gasp at my computer screen and waited for minutes. :)

When you edit the code at the back-end level, you can use system resources in the best way. I think you should start with dlang.dmd.backend.aarray

If we use the following codes with Ali's code by separate the keys and values, it compiles fast on DMD and works correctly:

[snip]

SDB@79

Thank you, but wouldn't using the DMD backend make it so that it won't compile with LDC? I am not that knowledgeable in compiler internals so I don't know.

No need for all files. Just aarray.d is enough.

SDB@79

March 17, 2022

On Monday, 14 March 2022 at 09:40:00 UTC, zhad3 wrote:

>

Hey everyone, I am in need of some help. I have written this Windows CP949 encoding table https://github.com/zhad3/zencoding/blob/main/windows949/source/zencoding/windows949/table.d which is used to convert CP949 to UTF-16.

After some research about how to initialize immutable associative arrays people suggested using shared static this(). So far this worked for me, but I recently discovered that DMD cannot compile this in release mode with optimizations.

dub build --build=release or dmd with -release -O fails:

code      windows949
function  zencoding.windows949.fromWindows949!(immutable(ubyte)[]).fromWindows949
code      table
function  zencoding.windows949.table._sharedStaticCtor_L29_C1
dmd failed with exit code -11.

I usually compile my projects using LDC where this works fine, but I don't want to force others to use LDC because of this one problem.

Hence I'd like to ask on how to change the code so that it compiles on DMD in release mode (with optimizations). I thought about having a computational algorithm instead of an encoding table but sadly I could not find any references in that regard. Apparently encoding tables seem to be the standard.

Why not use a simple static array (not an associative array). Where the values are indexed on key - min(keys). Even with the holes in the keys (i.e. keys that do not have corresponding values) it will be smaller that the constructed associative array? The lookup is also faster.

March 17, 2022

On Thursday, 17 March 2022 at 11:36:40 UTC, Patrick Schluter wrote:

>

On Monday, 14 March 2022 at 09:40:00 UTC, zhad3 wrote:

>

Hey everyone, I am in need of some help. I have written this Windows CP949 encoding table https://github.com/zhad3/zencoding/blob/main/windows949/source/zencoding/windows949/table.d which is used to convert CP949 to UTF-16.

After some research about how to initialize immutable associative arrays people suggested using shared static this(). So far this worked for me, but I recently discovered that DMD cannot compile this in release mode with optimizations.

dub build --build=release or dmd with -release -O fails:

code      windows949
function  zencoding.windows949.fromWindows949!(immutable(ubyte)[]).fromWindows949
code      table
function  zencoding.windows949.table._sharedStaticCtor_L29_C1
dmd failed with exit code -11.

I usually compile my projects using LDC where this works fine, but I don't want to force others to use LDC because of this one problem.

Hence I'd like to ask on how to change the code so that it compiles on DMD in release mode (with optimizations). I thought about having a computational algorithm instead of an encoding table but sadly I could not find any references in that regard. Apparently encoding tables seem to be the standard.

Why not use a simple static array (not an associative array). Where the values are indexed on key - min(keys). Even with the holes in the keys (i.e. keys that do not have corresponding values) it will be smaller that the constructed associative array? The lookup is also faster.
Something akin to

auto lookup(ushort key)
{
  return cp949[key-0x8141];
}

immutable ushort[0xFDFE-0x8141+1] cp949 = [
0x8141-0x8141: 0xAC02,
0x8142-0x8141: 0xAC03,
0x8143-0x8141: 0xAC05,
0x8144-0x8141: 0xAC06,
0x8145-0x8141: 0xAC0B,
0x8146-0x8141: 0xAC0C,
0x8147-0x8141: 0xAC0D,
0x8148-0x8141: 0xAC0E,
0x8149-0x8141: 0xAC0F,
0x814A-0x8141: 0xAC18,
0x814B-0x8141: 0xAC1E,
0x814C-0x8141: 0xAC1F,
0x814D-0x8141: 0xAC21,
0x814E-0x8141: 0xAC22,
0x814F-0x8141: 0xAC23,
0x8150-0x8141: 0xAC25,
0x8151-0x8141: 0xAC26,
0x8152-0x8141: 0xAC27,
0x8153-0x8141: 0xAC28,
0x8154-0x8141: 0xAC29,
0x8155-0x8141: 0xAC2A,
0x8156-0x8141: 0xAC2B,
0x8157-0x8141: 0xAC2E,
0x8158-0x8141: 0xAC32,
0x8159-0x8141: 0xAC33,
0x815A-0x8141: 0xAC34,
0x8161-0x8141: 0xAC35,
0x8162-0x8141: 0xAC36,
0x8163-0x8141: 0xAC37,
...
0xFDFA-0x8141: 0x72A7,
0xFDFB-0x8141: 0x79A7,
0xFDFC-0x8141: 0x7A00,
0xFDFD-0x8141: 0x7FB2,
0xFDFE-0x8141: 0x8A70,
];
March 17, 2022

On Thursday, 17 March 2022 at 12:11:19 UTC, Patrick Schluter wrote:

>

On Thursday, 17 March 2022 at 11:36:40 UTC, Patrick Schluter wrote:

> >

[...]
Something akin to

auto lookup(ushort key)
{
  return cp949[key-0x8141];
}

[...]

Takes 165 ms to compile with dmd 2.094.2 -O on godbolt with the whole table generated from the Unicode link.

March 17, 2022

On Thursday, 17 March 2022 at 12:19:36 UTC, Patrick Schluter wrote:

>

On Thursday, 17 March 2022 at 12:11:19 UTC, Patrick Schluter wrote:

>

On Thursday, 17 March 2022 at 11:36:40 UTC, Patrick Schluter wrote:

> >

[...]
Something akin to

auto lookup(ushort key)
{
  return cp949[key-0x8141];
}

[...]

Takes 165 ms to compile with dmd 2.094.2 -O on godbolt with the whole table generated from the Unicode link.

Upps, remove the ] at the end of the link to godbolt

1 2
Next ›   Last »