On 8/24/22 08:16, Ali Çehreli wrote:
> On 8/22/22 15:51, Chris Piker wrote:
>
>  > So depending on the use case, dxml works quite well.  For my own
>  > purposes I'll need to find/create a ForwardRange adapter for stdin
>
> The 'cached' range adaptor I mentioned on these forums a couple of times
> and in my DConf 2022 lightning talk converts any InputRange to a
> ForwardRange. (It does this by evaluating the elements once; so it would
> be valuable with generators as well; and in fact, a generator use case
> was why I wrote it.)

It is now available:

  https://code.dlang.org/packages/alid

> (Aside: It actually makes a RandomAccessRange because it supports
> opIndex as well but it does not honor O(1): It will grab 'n' elements if
> you say myRange[n] and if those elements are not in the cache yet.)

I realized that it is still O(1) because the seemingly unnecessarily grabbed elements would still count as "amortized" because they are readily available at O(1) for consumption of both this range and all its .save'd ranges.

> Currently it has an assumed performance issue because it uses a regular
> D slice, and the way it uses the slice incurs an allocation cost per
> element. There are different ways of dealing with that issue but I
> haven't finished that yet.

I solved that by writing the `alid.circularblocks` module.

Ali

On Monday, 12 September 2022 at 09:29:11 UTC, Ali Çehreli wrote:

> It is now available:
>
>   https://code.dlang.org/packages/alid
>
> > (Aside: It actually makes a RandomAccessRange because it
> supports
> > opIndex as well but it does not honor O(1): It will grab 'n'
> elements if
> > you say myRange[n] and if those elements are not in the cache
> yet.)
>
> I realized that it is still O(1) because the seemingly unnecessarily grabbed elements would still count as "amortized" because they are readily available at O(1) for consumption of both this range and all its .save'd ranges.

Wow pretty slick, thanks!  I know everyone wants the D community to be larger, but there are some advantages to a tight group.  Heck, I just got help on a ground support project from my favorite computer textbook author.  Outstanding!

As soon as I get back around to working on that project again I'll try out alid.  Neck deep in another sprint right now which depends on dpq2.

Best,

On Monday, 22 August 2022 at 23:30:58 UTC, H. S. Teoh wrote:
> Do you need to parse xml on-the-fly, or would it work to just slurp the entire stdin into a buffer and then parse that?

Thanks for the suggestion, though the purpose of the lib is to support stream based processing of very long time series datasets (> 2 TB occurs occasionally).  Due to data volume, we typically work with binary formats, but there is a supported XML representation and I'd prefer to apply the same mentality when processing it so as not to break user expectations.

On 8/22/22 7:48 AM, solidstate1991 wrote:
> Since the XML parsing library was removed from Phobos, I'm thinking about either getting dlang-community/experimental.xml into a usable state, or write a completely new parser.
> 
> First I'd want some community input, and would like to hear from the users of lodo1995's library. Depending on some circumstances, I'll be losing my job next month, so I'll have some extra time on my hands (no money will be a tough thing), and even without that I'll try to pull it off somehow.

Would be nice to have XSD support; many (most?) XML libraries I've looked at _across all languages_ only support DTD but not XSD schema def.

On 9/13/22 19:00, Chris Piker wrote:

> As soon as I get back around to working on that project again I'll try
> out alid.

Please don't give up but give feedback if it doesn't fit your use case. It desperately needs to be tested in the wild. :)

Ali

On 10/3/22 00:28, tsbockman wrote:

> `CircularBlocks.addExistingBlock_` will reallocate `blocks` and move the
> contents over to the new address,

CircularBlocks does not move elements. It just allocates and adds a new block to its "array of slices" queue.

Algorithmic complexity does get complicated :) if the block size is too small compared to live elements. Then there would be too many small blocks to shuffle around e.g. to the back of the queue to be reused.

Ali