On Sunday, 30 March 2014 at 20:43:52 UTC, Andrej Mitrovic wrote:
> Sounds like wasted effort, why improve tools for parsing C++ instead
> of improving tools for parsing D?

If the lexer is the culprit (though there's no proof for it), improving C lexer can help improve D lexer.

On 3/30/14, 5:01 PM, Leandro Lucarella wrote:
> ixid, el 30 de March a las 20:04 me escribiste:
>> On Sunday, 30 March 2014 at 19:28:20 UTC, Walter Bright wrote:
>>> On 3/30/2014 10:08 AM, Kagamin wrote:
>>>> On Friday, 28 March 2014 at 21:16:29 UTC, Ali Çehreli wrote:
>>>>> It could be useful for me just this past week in a throw-away
>>>>> D program that I
>>>>> wrote (at work! :) ) to parse some C and C++ files very
>>>>> crudely.
>>>>
>>>> As I understand, a preprocessor works on macros only, the rest
>>>> is lexed minimally.
>>>
>>> Yes, it won't help much with the rest.
>>
>> Were those ycombinator performance figures putting warp someway
>> behind clang valid? Perhaps we should unleash a community effort to
>> match clang?
>
> I think that's pretty wasteful, why won't you just use clang? What's the
> point of competing with another opensource project (a very good one,
> that took a lot of men-hour to do a good C/C++ compiler, including the
> preprocessor). I understand Walter did this in a couple of weeks, clang
> have been developed for at least 7 years now, is totally understandable
> that clang outperforms warp, is enough merit for warp to outperform GCC.
> I mean, if someone wants to have fun, go ahead, but putting community
> effort on that where there are so many places that are more important to
> put the effort on seems a bit silly.

It's quite obvious. The D codebase is smaller and simpler than clang pp's and can be taken many places; the next thing I'll work on is multithreaded preprocessing that shares already opened files. One thing that is self-evident but the article could have stressed is that open-sourcing warp is the beginning, not the end of its lifecycle. There's a lot of improvements that are within easy reach for warp, and are easier to realize than for clang.

Andrei

On Monday, 31 March 2014 at 14:46:32 UTC, Andrei Alexandrescu wrote:
>
> It's quite obvious. The D codebase is smaller and simpler than clang pp's and can be taken many places; the next thing I'll work on is multithreaded preprocessing that shares already opened files. One thing that is self-evident but the article could have stressed is that open-sourcing warp is the beginning, not the end of its lifecycle. There's a lot of improvements that are within easy reach for warp, and are easier to realize than for clang.
>
> Andrei

We also have the ability to present it in a different use-case, pre-processing every header in one invocation, allowing warp to do some optimizations that Clang and GCC won't do, such as unconditional pre-processing (stripping comments and things such as #if 0 / #if 1, or conditions that are unconditionally met by things defined within the header, typically used to disable/enable certain code) because they would be of very limited use to GCC and Clang, which are typically invoked once for every source file. Warp could also offer the ability to pass certain #define's that are known to never be #undef'd by the source code (such as compiler capabilities / version identifications), allowing for more extensive unconditional pre-processing.

Am 28.03.2014 19:27, schrieb Andrei Alexandrescu:
> Facebook is open-sourcing warp, a fast C and C++ preprocessor written by
> Walter Bright.

currently any ideas why clang could be 40% faster?

https://news.ycombinator.com/item?id=7489724

On Monday, 31 March 2014 at 17:11:48 UTC, dennis luehring wrote:
> Am 28.03.2014 19:27, schrieb Andrei Alexandrescu:
>> Facebook is open-sourcing warp, a fast C and C++ preprocessor written by
>> Walter Bright.
>
> currently any ideas why clang could be 40% faster?
>
> https://news.ycombinator.com/item?id=7489724

SIMD and virtual-file-system?

Andrei Alexandrescu, el 31 de March a las 07:46 me escribiste:
> On 3/30/14, 5:01 PM, Leandro Lucarella wrote:
> >ixid, el 30 de March a las 20:04 me escribiste:
> >>On Sunday, 30 March 2014 at 19:28:20 UTC, Walter Bright wrote:
> >>>On 3/30/2014 10:08 AM, Kagamin wrote:
> >>>>On Friday, 28 March 2014 at 21:16:29 UTC, Ali Çehreli wrote:
> >>>>>It could be useful for me just this past week in a throw-away
> >>>>>D program that I
> >>>>>wrote (at work! :) ) to parse some C and C++ files very
> >>>>>crudely.
> >>>>
> >>>>As I understand, a preprocessor works on macros only, the rest is lexed minimally.
> >>>
> >>>Yes, it won't help much with the rest.
> >>
> >>Were those ycombinator performance figures putting warp someway behind clang valid? Perhaps we should unleash a community effort to match clang?
> >
> >I think that's pretty wasteful, why won't you just use clang? What's the point of competing with another opensource project (a very good one, that took a lot of men-hour to do a good C/C++ compiler, including the preprocessor). I understand Walter did this in a couple of weeks, clang have been developed for at least 7 years now, is totally understandable that clang outperforms warp, is enough merit for warp to outperform GCC. I mean, if someone wants to have fun, go ahead, but putting community effort on that where there are so many places that are more important to put the effort on seems a bit silly.
> 
> It's quite obvious. The D codebase is smaller and simpler than clang pp's and can be taken many places; the next thing I'll work on is multithreaded preprocessing that shares already opened files. One thing that is self-evident but the article could have stressed is that open-sourcing warp is the beginning, not the end of its lifecycle. There's a lot of improvements that are within easy reach for warp, and are easier to realize than for clang.

Honestly, I'm not so sure, Clang is modern and has been designed with
extensibility and performance from day 0. I think it will be quite hard
to compete with it and I wonder why are you willing to spend that much
effort instead of contributing to Clang. If you said that about GCC,
I could agree, but with Clang, at least for me, is harder to sell that
argument.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
MP: Cómo está, estimado Bellini?   B: Muy bien, Mario, astrologando.
MP: Qué tengo?                     B: Un balcón-terraza.
MP: No, en mi mano, Bellini...     B: Un secarropas!
MP: No, escuche bien, eh. Tiene    B: El circo de Moscú.
    números.
MP: No Bellini. Toma medidas.      B: Un ministro.
MP: No Bellini, eh! Algunas son    B: Una modelo, Mario!
    de plástico y otras de madera.
MP: No, Bellini, no y no!
	-- El Gran Bellini (Mario Podestá con una regla)

justme, el 31 de March a las 03:25 me escribiste:
> On Monday, 31 March 2014 at 00:09:34 UTC, Leandro Lucarella wrote:
> >I think that's pretty wasteful, why won't you just use clang?
> >What's the
> >point of competing with another opensource project (a very good
> >one,
> >that took a lot of men-hour to do a good C/C++ compiler, including
> >the
> >preprocessor). I understand Walter did this in a couple of weeks,
> >clang
> >have been developed for at least 7 years now, is totally
> >understandable
> >that clang outperforms warp, is enough merit for warp to
> >outperform GCC.
> >I mean, if someone wants to have fun, go ahead, but putting
> >community
> >effort on that where there are so many places that are more
> >important to
> >put the effort on seems a bit silly.
> 
> Walter taking 2 weeks to do something comparable to what the clang and gcc guys have done over many years, serves as massive advertising for D.
> 
> Also, here we now have an entire project written by the man himself. That should serve as required reading for anybody who wants to learn how to code in the latest D.
> 
> And it serves as a benchmark for the best C++ coders. They can try to do the same in C++ in two weeks. (I bet by the end of the two weeks the guys are ready to switch languages!)

Don't take the "couple of weeks" too literally, this is just my impression after reading the article! Maybe it would be good if Walter said how much time did it take him to code this.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
Pack and get dressed
before your father hears us,
before all hell breaks loose.

On 3/31/2014 10:50 AM, Leandro Lucarella wrote:
> Don't take the "couple of weeks" too literally, this is just my
> impression after reading the article! Maybe it would be good if Walter
> said how much time did it take him to code this.

I spent 2 weeks on the initial version, and another week tuning it. Since then, I've fixed a handful of bugs, but that didn't amount to much time.

Walter Bright:

> Since then, I've fixed a handful of bugs, but that didn't amount to much time.

Have you kept a list of such bugs/mistakes of yours for warp? It is an interesting list.

Bye,
bearophile

On 3/31/2014 2:06 PM, bearophile wrote:
> Walter Bright:
>
>> Since then, I've fixed a handful of bugs, but that didn't amount to much time.
>
> Have you kept a list of such bugs/mistakes of yours for warp? It is an
> interesting list.

It's on github, though currently in a private repository. They were the usual mix of stupid coding mistakes and adjustments needed to match cpp's behavior.