I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?


-Alex

On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>
>
> -Alex

You're in luck: https://github.com/atilaneves/dpp

On Tuesday, 6 October 2020 at 21:04:20 UTC, Meta wrote:
> On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
>> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>>
>>
>> -Alex
>
> You're in luck: https://github.com/atilaneves/dpp

What part of "built in and not part of a 3rd party library" do you not understand?

On Tuesday, 6 October 2020 at 22:42:11 UTC, 12345swordy wrote:
> On Tuesday, 6 October 2020 at 21:04:20 UTC, Meta wrote:
>> On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
>>> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>>>
>>>
>>> -Alex
>>
>> You're in luck: https://github.com/atilaneves/dpp
>
> What part of "built in and not part of a 3rd party library" do you not understand?

You could easily work it out for yourself by looking at any C parser and the DMD front. But why bother with C, it is trivial to generate C bindings from headers. C++ is where it becomes complicated and where it would be useful.

On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>
>
> -Alex

Not impossible (it's worth saying that you can literally do this as a library via mixins), but having a c parser in the compiler seems like too much work when it's already fairly trivial to do. If D had a team of full time staff on payroll, then it could be made to work but it adds so much surface area to the compiler.

On Wednesday, 7 October 2020 at 01:29:05 UTC, norm wrote:
> On Tuesday, 6 October 2020 at 22:42:11 UTC, 12345swordy wrote:
>> On Tuesday, 6 October 2020 at 21:04:20 UTC, Meta wrote:
>>> On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
>>>> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>>>>
>>>>
>>>> -Alex
>>>
>>> You're in luck: https://github.com/atilaneves/dpp
>>
>> What part of "built in and not part of a 3rd party library" do you not understand?
>
> You could easily work it out for yourself by looking at any C parser and the DMD front.

> But why bother with C, it is trivial to generate C bindings from headers.

It really isn't. Can it be easy? Yes. Is it easy in the general case? No.

I've tried calling a large C codebase from D before. It was a nightmare. It's very hard to do manual wrapping when you have thousands of headers in hundreds of directories and everything depends on everything.

On Wednesday, 7 October 2020 at 10:04:57 UTC, Atila Neves wrote:
> On Wednesday, 7 October 2020 at 01:29:05 UTC, norm wrote:
>> On Tuesday, 6 October 2020 at 22:42:11 UTC, 12345swordy wrote:
>>> On Tuesday, 6 October 2020 at 21:04:20 UTC, Meta wrote:
>>>> On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
>>>>> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>>>>>
>>>>>
>>>>> -Alex
>>>>
>>>> You're in luck: https://github.com/atilaneves/dpp
>>>
>>> What part of "built in and not part of a 3rd party library" do you not understand?
>>
>> You could easily work it out for yourself by looking at any C parser and the DMD front.
>
>> But why bother with C, it is trivial to generate C bindings from headers.
>

Really? I have not had any issues before automatically creating C bindings for D using codegen. Sure, occasionally a macro will bite but it isn't that often nor difficult to fix.  They were not the kernel but they were not small code bases without macros either.

TBH though now I just use dpp so I don't see the value of bolting on a "C" parser to the D front end. C++ I have struggled with that and it requires an annoying amount of manual editing. DPP works well but even it falls over for some C++ code bases I've tried.

On Wednesday, 7 October 2020 at 11:14:02 UTC, norm wrote:
> On Wednesday, 7 October 2020 at 10:04:57 UTC, Atila Neves wrote:
>> On Wednesday, 7 October 2020 at 01:29:05 UTC, norm wrote:
>>> On Tuesday, 6 October 2020 at 22:42:11 UTC, 12345swordy wrote:
>>>> On Tuesday, 6 October 2020 at 21:04:20 UTC, Meta wrote:
>>>>> On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
>>>>>> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>>>>>>
>>>>>>
>>>>>> -Alex
>>>>>
>>>>> You're in luck: https://github.com/atilaneves/dpp
>>>>
>>>> What part of "built in and not part of a 3rd party library" do you not understand?
>>>
>>> You could easily work it out for yourself by looking at any C parser and the DMD front.
>>
>>> But why bother with C, it is trivial to generate C bindings from headers.
>>
>
> Really? I have not had any issues before automatically creating C bindings for D using codegen. Sure, occasionally a macro will bite but it isn't that often nor difficult to fix.  They were not the kernel but they were not small code bases without macros either.

Imagine this and extrapolate to hundreds of headers:

---
// hdr0.h
struct struct0 {
    struct1* s1;
};
---

---
// hdr1.h
#define SILLY_MACRO struct2;
struct struct1 {
    SILLY_MACRO* s2;
}
----

---
// hdr2.h
struct struct2 {
    struct3* s3;
}
---


>
> TBH though now I just use dpp so I don't see the value of bolting on a "C" parser to the D front end. C++ I have struggled with that and it requires an annoying amount of manual editing. DPP works well but even it falls over for some C++ code bases I've tried.

dpp doesn't officially support C++ yet, because... it doesn't work yet. There are several reasons for this.

The first, and most annoying, is that libclang isn't all it's cracked up to be. There's a *lot* of functionality missing for C++ that the compiler frontend itself knows about but doesn't expose via the C API.

The second, and maybe unfixable reason, is how to translate C++-only concepts to D. Consider std::is_reference_v. That can't be written in D, not even by hand. What if that gets used in SFINAE? What about expression templates given how operator overloading in D is a lot different? I still don't know how to get around that.

On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>
>
> -Alex

This would indeed be a total game changer.

I just looked at D in the course of checking out newer programming languages and it turns out that I think none of them will be really successful replacing C (or C++) because there's no direct compatibility with C (Priority 1) and C++ (Priority 2) and ideally not only using the C (and potentially C++) binaries directly, but also creating header files to allow using the D binaries smoothly "on the other side".

There is just some much existing code that you would totally on the winner side with this and your totally out in the desert without it. Just like the biggest argument for Java is it's ecosystem, this would make all C (and maybe even C++) libraries part of the D ecosystem.

For this to be really an argument, it must be an in-built feature of the language toolchain and a priority in all future developments.

Just imagine what this would mean: a clean way forward from existing C code to write bulletproof applications on top of all the existing code. And a way to use the new code from new libraries written in D without any hassle.

A potential external tool would a C to D converter which also allows easy migration of C code. Just imagine converting a complete Linux implementation to D, compile it and having a full D-based Linux.

Then you could start migrating the code base to use the advanced features of D and start a new age for the whole industry.

This would make the world listen.

Actually it looks as if this is not such an ambitious goal for D, so I'm surprised it's not under way. If had time I'd maybe look at it myself, unfortunately I have project work and 3 children, leaving really very little time for extra projects.

On Saturday, 17 April 2021 at 08:36:41 UTC, Alexis Kyaw wrote:
> On Tuesday, 6 October 2020 at 20:59:54 UTC, 12345swordy wrote:
>> I remember reading a reddit comment saying that if d were to import c header files directly without any 3rd party libraries it would be a game changer. The question is: how difficult to implement a C AST parser for the dmd front end?
>>
>>
>> -Alex
>
> This would indeed be a total game changer.
>
> I just looked at D in the course of checking out newer programming languages and it turns out that I think none of them will be really successful replacing C (or C++) because there's no direct compatibility with C (Priority 1) and C++ (Priority 2) and ideally not only using the C (and potentially C++) binaries directly, but also creating header files to allow using the D binaries smoothly "on the other side".
>
> There is just some much existing code that you would totally on the winner side with this and your totally out in the desert without it. Just like the biggest argument for Java is it's ecosystem, this would make all C (and maybe even C++) libraries part of the D ecosystem.
>
> For this to be really an argument, it must be an in-built feature of the language toolchain and a priority in all future developments.
>
> Just imagine what this would mean: a clean way forward from existing C code to write bulletproof applications on top of all the existing code. And a way to use the new code from new libraries written in D without any hassle.
>
> A potential external tool would a C to D converter which also allows easy migration of C code. Just imagine converting a complete Linux implementation to D, compile it and having a full D-based Linux.
>
> Then you could start migrating the code base to use the advanced features of D and start a new age for the whole industry.
>
> This would make the world listen.
>
> Actually it looks as if this is not such an ambitious goal for D, so I'm surprised it's not under way. If had time I'd maybe look at it myself, unfortunately I have project work and 3 children, leaving really very little time for extra projects.

+1

Have been experimenting with converting C(++) to D. It's possible to some extent.

*dream hat on*

I've thought about this, and what would be best for D is if code could be converted. Why? Because otherwise we still just have built a lot of stuff upon the old, meaning that the % of D in the world hasn't really increased much.

Also, any modifications would be hard etc etc.

What's lacking tho is good tooling around D, like static analysis etc etc.

But, if C(++) could be converted to D, *some* parts of the tools might be reusable, and you would get a compounding effect after a while.

For instance, even *if* C(++) and D could be seamlessly mixed in the same source file, you would still end up with much C(++) code.

I still think dpp, dstep etc are 100% needed, and more work should definitely be put in it, but the endgame would be to somehow increase the amount of D, right? (Btw, what's the state of cpp2d?)

Yes, that would mean you end up with two code bases, but that's kind of the point in this case. There are obviously + and - with this solution. But maybe the endgame deluxe would be to be able to reuse the parts that work well and convert the parts that doesn't. Best of both worlds.

*dream hat off* <- note