On 7/30/2011 10:27 PM, Jonathan M Davis wrote:
> So, what major functionality which we don't currently have would you like to
> see in either Phobos or in a 3rd party library so that you could use it in
> your D programs?

Better Unicode support.

http://training.perl.com/OSCON2011/index.html

On 31/07/2011 9:51 PM, Peter Alexander wrote:
> My wish list:
>
> - A good allocator model and integration with standard containers
> - Fast vector math library suitable for games.

+1 vector math lib.

Some template/mixin functions which ease generating C or C++ bindings. Same goes for DLLs.

On Jul 31, 11 13:27, Jonathan M Davis wrote:
> I think that it would be useful to query the community for what piece of
> library functionality they don't currently have in D and would most like to
> see. For instance, there is no official logging framework in D or any 3rd party
> libraries which do it AFAIK. So, that could be one type of functionality that
> you may like to see. Now, there is a prospective implementation for std.log
> which shouldn't be all that far away from being reviewed, so listing that here
> wouldn't be all that useful, since it's on its way. But what other major
> functionality do you miss in D that other languages' that you use have
> available in their libraries?
>
> My hope here would be that we could get some good ideas going here such that
> we have can have a better idea what type of functionality it would be
> particularly useful to be working on for Phobos or 3rd party D libraries for
> the community, and maybe it'll even get some people to actually go and work on
> these ideas so that we can improve the libraries that we have to work with in
> D. We can always use more help, and we definitely need a richer library
> ecosystem for D. But even just discussing ideas could be of benefit.
>
> So, what major functionality which we don't currently have would you like to
> see in either Phobos or in a 3rd party library so that you could use it in
> your D programs?
>
> - Jonathan M Davis

1. An officially endorsed XML *and* HTML library that supports Xpath. Preferably with an interface as easy to use as Python's lxml. I don't know how long the current std.xml will stay, if it's upon rewrite please do consider supporting Xpath (or CSS3 selector).

(lxml is based on libxml2 and libxslt which are MIT license and may not be compatible with Boost; lxml's source code itself is in BSD)

2. More containers, including:
    - std.container.HashTable (or allow void[T] in the language).
    - std.container.List (doubly linked list)
    - std.container.Deque
    - std.container.IntervalSet
    - std.container.Trie (or PatriciaTrie)
    - Adaptors to RedBlackTree and HashTable, making it a Set, Dictionary (map) and CountedSet (bag/multiset).
    - Ordered associative array

3. A std.variant.Algebraic that does not rely on typeid (like C++'s Boost.Variant).

4. Wrapper to GSL (= GNU Scientic Library, GPL of course).

5. Unicode normalization (the NFKD stuff) and other Unicode support, preferably as a wrapper to ICU (non-restrictive license: http://source.icu-project.org/repos/icu/icu/trunk/license.html).

6. Rational numbers (std.fraction?).

7. Serialization library (yes I know there's Orange).

8. Support for CSV and plist

9. More pure/@safe/const :)

On Sun, Jul 31, 2011 at 6:13 AM, simendsjo <simendsjo@gmail.com> wrote:

> On 31.07.2011 07:27, Jonathan M Davis wrote:
>
>> I think that it would be useful to query the community for what piece of
>> library functionality they don't currently have in D and would most like
>> to
>> see. For instance, there is no official logging framework in D or any 3rd
>> party
>> libraries which do it AFAIK. So, that could be one type of functionality
>> that
>> you may like to see. Now, there is a prospective implementation for
>> std.log
>> which shouldn't be all that far away from being reviewed, so listing that
>> here
>> wouldn't be all that useful, since it's on its way. But what other major
>> functionality do you miss in D that other languages' that you use have
>> available in their libraries?
>>
>> My hope here would be that we could get some good ideas going here such
>> that
>> we have can have a better idea what type of functionality it would be
>> particularly useful to be working on for Phobos or 3rd party D libraries
>> for
>> the community, and maybe it'll even get some people to actually go and
>> work on
>> these ideas so that we can improve the libraries that we have to work with
>> in
>> D. We can always use more help, and we definitely need a richer library
>> ecosystem for D. But even just discussing ideas could be of benefit.
>>
>> So, what major functionality which we don't currently have would you like
>> to
>> see in either Phobos or in a 3rd party library so that you could use it in
>> your D programs?
>>
>> - Jonathan M Davis
>>
>
> From the top of my head, in no particular order:
> * xml
> * csv
> * ini
> * logging
> * database interface
> * serialization
> * parsing / easier to create external dsl's - antlr port?
> * smtp
> * uri - a complete uri implementation, not the simple validation that
> exists now
> * wsdl - far from everything is REST
> * ftp (this is perhaps included with curl?)
> * globalization - languages, number formats etc.
> * gui - perhaps not best done in phobos, but it's difficult to get up and
> running to say the least..
> * html producer
> * html parser
> * cgi/fastcgi
> * cryptography
> * oauth
>
> And some windows specifics:
> * "complete" win32 bindings - move the win32 project to etc/c/win32?
> * better com integration (like http://dsource.org/projects/**juno<http://dsource.org/projects/juno>
> )
>


FTP, HTTP, and SMTP are the protocols that are included in etc.curl.

On 7/31/2011 5:57 AM, Jacob Carlborg wrote:
>> * Lexing and parsing:
>>
>> Standard facilities for these tasks could be very useful. Perhaps D
>> could get its own dlex and dyacc or some such tools. Personally, I
>> prefer sticking to LL(1), but LALR is generally more convenient and
>> flexible, and thus I'd suggest something YACC/ANTLR-like.
>>
>> (I know this doesn't have much to do with Phobos per se, but I figured
>> I'd mention it.)
>
> I think someone is working on this.

I've started on a port of DMD's lexer (not really a port ;) ):

https://github.com/jmacdonagh/phobos/compare/master...std.lang.d.lexer

Basically, you give it some string (string, wstring, or dstring), and it gives you a range of tokens back. The token has the type, a slice of the input that corresponds to the token, line / column, and a value (e.g. an integer constant).

Some features I'm planning:

1. Support D1 and D2.
2. Warnings and errors returned in the tokens. For example, if you use an octal constant for D2 code, it will correctly return an integer constant token with some kind of warning flag set and a message. In terms of errors, if the lexer hits "0xz012", it will return an error token for the slice "0xz" and then start lexing an integer constant "012". No exceptions, easy peasy.
3. CTFEable. Although I'll probably have to wait till the next DMD release.
4. Support any kind of character range. Not sure if people want to lex something that's not a string/wstring/dstring.

I'm glad this was brought up. I remember Walter's post last year asking for this module, but the conversation seemed to kill the idea. I started on this just for the fun of it, but then doubted whether Phobos wanted it. I feel that a hand written lexer / parser is going to be faster than something generated, but maybe I'm old fashioned.

Anyway, Jim, if you want to do this I can move on to something else. If you want, I can continue on. I didn't see a branch in your repo so I'm not sure what you've done.

Btw why not take DDMD's lexer.d and start from there?

On 7/31/2011 1:27 AM, Jonathan M Davis wrote:
> I think that it would be useful to query the community for what piece of
> library functionality they don't currently have in D and would most like to
> see. For instance, there is no official logging framework in D or any 3rd party
> libraries which do it AFAIK. So, that could be one type of functionality that
> you may like to see. Now, there is a prospective implementation for std.log
> which shouldn't be all that far away from being reviewed, so listing that here
> wouldn't be all that useful, since it's on its way. But what other major
> functionality do you miss in D that other languages' that you use have
> available in their libraries?
>
> My hope here would be that we could get some good ideas going here such that
> we have can have a better idea what type of functionality it would be
> particularly useful to be working on for Phobos or 3rd party D libraries for
> the community, and maybe it'll even get some people to actually go and work on
> these ideas so that we can improve the libraries that we have to work with in
> D. We can always use more help, and we definitely need a richer library
> ecosystem for D. But even just discussing ideas could be of benefit.
>
> So, what major functionality which we don't currently have would you like to
> see in either Phobos or in a 3rd party library so that you could use it in
> your D programs?
>
> - Jonathan M Davis

My top 3:

1. Database interface. Should support sqlite, mysql, postgres, etc...
2. DWT fixed up and merged into Phobos.
3. Some kind of web framework.

On 7/31/2011 9:31 PM, Andrej Mitrovic wrote:
> Btw why not take DDMD's lexer.d and start from there?

I wasn't sure how up-to-date DDMD was when I started. Plus, I thought a fresh look at a lexer using D2 features rather than a straight port would be a little cleaner. For example, instead of that huge switch statement for all the operators, I have a mixin that generates that huge switch statement based on an array of operator strings:

https://github.com/jmacdonagh/phobos/compare/master...std.lang.d.lexer#L0R506

I may end up using "borrowing" some code from DDMD ;)

On Monday 01 August 2011 03:31:41 Andrej Mitrovic wrote:
> Btw why not take DDMD's lexer.d and start from there?

It's a possibility but has 2 downsides to it.

1. Due to licensing issues, you have to get permission from the ddmd developers. The front-end is LGPL IIRC, not Boost. Walter gave permission to port the lexer from the front-end to D and make it Boost, but that doesn't hold for the ddmd code, which was taken from dmd's front-end and ported by other folks. So, their permission would be required. It's not necessarily a big deal, but it is something that needs to be dealt with.

2. To really do it right, you need to understand how it works. Personally, I think that doing all the work of porting it is going to help a _lot_ in gaining that understanding. If all you do is take and adjust the ddmd code, then you don't gain that understanding.

So, ddmd's source code definitely could be used as the base for Phobos' D lexer, but I haven't taken the time to deal with #1, and because of #2, I find it very valuable to do the porting myself. I haven't gotten very far on of it of late though. Taking ddmd and adjusting would probably be faster, but as a maintainer of the result, it would be harder to maintain just because I wouldn't understand it as well.

- Jonathan M Davis