So, I'm writing a poker AI bot. The idea was to generate a lookup table of all the poker hands using CTFE so runtime can be as quick as possible (as the bot has a very small amount of time to act).

There are a LOT of calculations though, many millions of combinations.

During complation, this is consuming inordinate amounts of memory, 16GB+ (which is all the RAM on my machine + some swap space).

I'm wondering is there any techniques to freeing some memory used by the compiler so it can be reused?
Are malloc and free usable at compile time maybe?

Another possibilty I was looking at was to write a tool that will spit out all combinations at runtime, and then import these back into the bot at compile time to build a lookup table that way.

Thanks!

On Thursday, 9 April 2015 at 09:49:39 UTC, wobbles wrote:
> So, I'm writing a poker AI bot. The idea was to generate a lookup table of all the poker hands using CTFE so runtime can be as quick as possible (as the bot has a very small amount of time to act).
>
> There are a LOT of calculations though, many millions of combinations.
>
> During complation, this is consuming inordinate amounts of memory, 16GB+ (which is all the RAM on my machine + some swap space).

Unfortunately, the current implementation of CTFE is not very memory-friendly. This is being worked on.

> I'm wondering is there any techniques to freeing some memory used by the compiler so it can be reused?
> Are malloc and free usable at compile time maybe?

No, they are not.

> Another possibilty I was looking at was to write a tool that will spit out all combinations at runtime, and then import these back into the bot at compile time to build a lookup table that way.

This is probably the most practical solution for the time being. Some advice:

1. writeln, format and friends print most values (e.g. arrays) in a syntax close to D's, which means you should be able to take the output and paste it into a D program.

2. You can use import expressions and mixin expressions to import constant values from files:

http://dlang.org/expression.html#ImportExpression
http://dlang.org/expression.html#MixinExpression

On 9/04/2015 9:49 p.m., wobbles wrote:
> So, I'm writing a poker AI bot. The idea was to generate a lookup table
> of all the poker hands using CTFE so runtime can be as quick as possible
> (as the bot has a very small amount of time to act).
>
> There are a LOT of calculations though, many millions of combinations.
>
> During complation, this is consuming inordinate amounts of memory, 16GB+
> (which is all the RAM on my machine + some swap space).
>
> I'm wondering is there any techniques to freeing some memory used by the
> compiler so it can be reused?
> Are malloc and free usable at compile time maybe?
>
> Another possibilty I was looking at was to write a tool that will spit
> out all combinations at runtime, and then import these back into the bot
> at compile time to build a lookup table that way.
>
> Thanks!

My recommendation is to pregenerate this with a separate file.
So generator.d -> mydata.d
Since mydata.d won't need to be updated often, only recompile it shouldn't be an issue.

On Thursday, 9 April 2015 at 09:49:39 UTC, wobbles wrote:
> Another possibilty I was looking at was to write a tool that will spit out all combinations at runtime, and then import these back into the bot at compile time to build a lookup table that way.
>
> Thanks!

This is definitely a better idea. I would probably load them at runtime instead, it scales better.

On Thursday, 9 April 2015 at 10:04:09 UTC, John Colvin wrote:
> On Thursday, 9 April 2015 at 09:49:39 UTC, wobbles wrote:
>> Another possibilty I was looking at was to write a tool that will spit out all combinations at runtime, and then import these back into the bot at compile time to build a lookup table that way.
>>
>> Thanks!
>
> This is definitely a better idea. I would probably load them at runtime instead, it scales better.

But if the compiler knows about all the values at compile time, it will be able to optimise things better than at runtime?

Cheers folks!
Will get to writing this now.

On Thursday, 9 April 2015 at 10:23:06 UTC, wobbles wrote:
> On Thursday, 9 April 2015 at 10:04:09 UTC, John Colvin wrote:
>> On Thursday, 9 April 2015 at 09:49:39 UTC, wobbles wrote:
>>> Another possibilty I was looking at was to write a tool that will spit out all combinations at runtime, and then import these back into the bot at compile time to build a lookup table that way.
>>>
>>> Thanks!
>>
>> This is definitely a better idea. I would probably load them at runtime instead, it scales better.
>
> But if the compiler knows about all the values at compile time, it will be able to optimise things better than at runtime?

Well, it depends. Sometimes it makes a big difference, other times there's not much, if any, advantage. The only way to reason about it is to look at your code and think about what the compiler might be able to do given certain information, then check the asm to see if it's actually doing it.

On Thursday, 9 April 2015 at 11:05:22 UTC, John Colvin wrote:
> On Thursday, 9 April 2015 at 10:23:06 UTC, wobbles wrote:
>> On Thursday, 9 April 2015 at 10:04:09 UTC, John Colvin wrote:
>>> On Thursday, 9 April 2015 at 09:49:39 UTC, wobbles wrote:
>>>> Another possibilty I was looking at was to write a tool that will spit out all combinations at runtime, and then import these back into the bot at compile time to build a lookup table that way.
>>>>
>>>> Thanks!
>>>
>>> This is definitely a better idea. I would probably load them at runtime instead, it scales better.
>>
>> But if the compiler knows about all the values at compile time, it will be able to optimise things better than at runtime?
>
> Well, it depends. Sometimes it makes a big difference, other times there's not much, if any, advantage. The only way to reason about it is to look at your code and think about what the compiler might be able to do given certain information, then check the asm to see if it's actually doing it.

or just time it, of course. But even that has pitfalls and
difficulties. Optimisation is difficult.