December 23

On Friday, 22 December 2023 at 22:33:35 UTC, H. S. Teoh wrote:

>

IMNSHO, if I had very large data files to load, I wouldn't use JSON. Precompile the data into a more compact binary form that's already ready to use, and just mmap() it at runtime.

I wondered about that decision as well, especially because this was internal game data that did not have to be user readable.
That's beside the point though; it was a ~10 MB JSON file that took them several minutes to parse. That's really just insane. Turns out it helps if you don't count the length of the entire document for every single value. It also helps if you don't iterate over your entire array of already written values every time you want to insert a new one. :)
In case you didn't know the story, here's a link:
https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/

I think there are several great lessons in there. Rockstar must have noticed how slow the loading is, but apparently just accepted it as a given... for 7+ years. Who needs optimizations on today's great hardware, right? There couldn't possibly be algorithmic problems in something simple like a JSON parser, right?
Second, look at what people suspected as the root cause of the problem, like the P2P architecture. It's funny how speculations about performance problems are always wrong. Only measuring will tell you the truth.

December 23

On Monday, 18 December 2023 at 16:44:11 UTC, Bkoie wrote:

>

just look at this i know this is overdesign im just trying to get a visual on how a api can be design im still new though but the fact you can build an api like this and it not break it is amazing.

but what is with these ppl and the gc?
just dont allocate new memory or invoke,
you can use scopes to temporry do stuff on immutable slices that will auto clean up
the list goes on

and you dont need to use pointers at all.......!!

i honesty see nothing wrong with gc,

I don't think there is any wrong having GC in language either and upcoming languages also show that as a majority of the have some form of GC. GC is here to stay regardless.

So what is the problem with D? The problem with D is that it is limited to what type of GC it can support. Right now D only supports stop the world GC which is quickly becoming unacceptable on modern systems. Sure it was fine when when we had dual core CPUs but today desktop PCs can have 32 execution units (server CPUs can have an insane amount of of them like 128). Stopping 32 execution (potentially even more if you have more threads) units is just unacceptable, which not only takes a lot of time but a very clumsy approach on modern systems.

What GC should D then support? In my opinion, all of them. Memory management is a moving target and I don't know how it will look like in 10 years. Will cache snoop be viable for example, will the cores be clustered so that snoops are only possible within them etc? D needs a more future proof language design when it comes to memory management.

Because of this it is important that D can as seamless as possible support different types of GC types. Exposing raw pointers in the language for GC allocated type was a big mistake in the D language design which I think should be rectified. About all other new languages have opaque pointers/reference types in order to hide the GC mechanism and so that other GC algorithms like reference counting can be used.

This is a an F- in language design.

1 2
Next ›   Last »