July 08, 2012
On 2012-07-08 04:59, Andrei Alexandrescu wrote:
> On 7/7/12 5:20 PM, Walter Bright wrote:
>> In fact, we could make it a general facility, where if D sees:
>>
>> import "filename.ext";
>>
>> that it fork/exec's the program:
>>
>> ext_to_D filename.ext tmpfile.d
>>
>> and them imports tmpfile.d.
>
> (Aside) This has an obvious security risk.
>
> Andrei

Don't know if it helps but the tool can be easily used as a library and wrapped in a C API. Then DMD can used the tool directory like a library. No need for creating a new process and writing temporary files.

-- 
/Jacob Carlborg


July 08, 2012
On 7/8/2012 4:15 AM, Jacob Carlborg wrote:
> Don't know if it helps but the tool can be easily used as a library and wrapped
> in a C API. Then DMD can used the tool directory like a library. No need for
> creating a new process and writing temporary files.

Creating a new process has its advantages:

1. Bugs from one process won't propagate to another.

2. The programmer of that process can develop completely independently of the D compiler.


July 08, 2012
On 7/8/2012 4:13 AM, Jacob Carlborg wrote:
> How is this going to work, is it going to be an optional feature? I mean, this
> will add DStep (D and Clang) as dependencies to DMD.

I think that implicitly using the feature will depend on those programs being available. It also means that any 3rd party can supply such a feature, to import a file in any format.


> DStep is built to be used as a library, I can easily create a C API which can be
> used directly by DMD. No need for creating a new process. I can also make DStep
> give back the translate D code, no need for creating temporarily D files.

I think there are many advantages to DStep being a separate program, not the least of which is debugging the output of it. Also, it means DStep could be written in any language. For example, suppose a Go-to-D is proposed. Go provides a Go library to parse Go code - so such a tool might be more easily written in Go than in D.


> BTW, how would you indicate that the header file is an Objective-C file? Since
> both C and Objective-C uses the same extension for header files, this is
> required by Clang, otherwise it will treat the file as a C file.

Since OC is a proper superset of C, this shouldn't be a problem. Just run the OC converter as your "C" compiler.


> In that project I had a tool for converting C/Objective-c headers to D modules.
> This tool was a Ruby script based on BridgeSupport. This is a complete rewrite
> of that tool. The whole project was called DStep and the name fit among other
> Objective-C related names like NeXTSTEP, OpenStep and GNUStep.

The name makes more sense now, but for marketing reasons it should give more of a clue as to what it does.

July 08, 2012
Jacob Carlborg, el  8 de July a las 13:13 me escribiste:
> On 2012-07-07 23:20, Walter Bright wrote:
>
> >I think this is potentially a big deal.
> >
> >If it can be made complete enough, I'd like to add support into D for it, so you could do things like:
> >
> >     import "stdio.h";
> >
> >and the D compile would fork/exec Dstep, generate the corresponding .d file, and import the .d file.
>
> How is this going to work, is it going to be an optional feature? I mean, this will add DStep (D and Clang) as dependencies to DMD.
>
> DStep is built to be used as a library, I can easily create a C API which can be used directly by DMD. No need for creating a new process. I can also make DStep give back the translate D code, no need for creating temporarily D files.

This can also be done by just dumping the generated code to stdout. Then any application can use the output without using any temporary files. Pipes for the win!

--
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Me encanta el éxito; por eso prefiero el estado de progreso constante,
con la meta al frente y no atrás.
	-- Ricardo Vaporeso. Punta del Este, Enero de 1918.
July 08, 2012
On 7/8/12 3:05 PM, Leandro Lucarella wrote:
> Jacob Carlborg, el  8 de July a las 13:13 me escribiste:
>> On 2012-07-07 23:20, Walter Bright wrote:
>>
>>> I think this is potentially a big deal.
>>>
>>> If it can be made complete enough, I'd like to add support into D for
>>> it, so you could do things like:
>>>
>>>      import "stdio.h";
>>>
>>> and the D compile would fork/exec Dstep, generate the corresponding .d
>>> file, and import the .d file.
>>
>> How is this going to work, is it going to be an optional feature? I
>> mean, this will add DStep (D and Clang) as dependencies to DMD.
>>
>> DStep is built to be used as a library, I can easily create a C API
>> which can be used directly by DMD. No need for creating a new
>> process. I can also make DStep give back the translate D code, no
>> need for creating temporarily D files.
>
> This can also be done by just dumping the generated code to stdout.
> Then any application can use the output without using any temporary
> files. Pipes for the win!

Good old date checks (makefile style) can be used for the win.

Andrei


July 08, 2012
On 2012-07-08 20:42, Walter Bright wrote:

> I think that implicitly using the feature will depend on those programs
> being available. It also means that any 3rd party can supply such a
> feature, to import a file in any format.

I see.

> I think there are many advantages to DStep being a separate program, not
> the least of which is debugging the output of it. Also, it means DStep
> could be written in any language. For example, suppose a Go-to-D is
> proposed. Go provides a Go library to parse Go code - so such a tool
> might be more easily written in Go than in D.

DStep is both usable as a program and a library. Both I guess you have a point. On the other hand, C can be used to glue together libraries written in different languages.

> Since OC is a proper superset of C, this shouldn't be a problem. Just
> run the OC converter as your "C" compiler.

That is not completely true if one is picky. The following code is legal C, but not legal Objective-C :

int id;
int nil;

In Objective-C "id" is a type.

> The name makes more sense now, but for marketing reasons it should give
> more of a clue as to what it does.

Do you have a suggestion?

-- 
/Jacob Carlborg


July 08, 2012
On Saturday, 7 July 2012 at 14:47:49 UTC, Jacob Carlborg wrote:
> DStep is a tool for translating C and Objective-C headers to D modules. It uses libclang for lexing/parsing and AST traversal. This means it handles everything that Clang itself can handle, although this doesn't mean it will correctly translate everything.
>
> I would consider this release alpha or beta. I'm releasing this now in hope I get some feedback on what language features the tool can't handle.
>
> The tool is available at github:
> https://github.com/jacob-carlborg/dstep
>
> Binaries are available for Mac OS X and Ubuntu 11.10 32bit:
> https://github.com/jacob-carlborg/dstep/downloads
>
> Unfortunately I haven't been able to successfully compile it on Windows due to Optlink not cooperating. I'll most likely provide Linux binaries with better compatibility later.
>
> Build instructions are available at github.
>
> Usage:
>
> dstep <input-file.h> -o output_file.d
>
> For Objective-C
>
> dstep <input-file.h> -o output_file.d -ObjC
>
> Tests:
>
> DStep uses Cucumber and Aruba (Ruby tools) to run its tests. It will basically run the tool on all *.h files in the "test_files" directory and compare the results to the corresponding *.d files.
>
> Known issues/missing functionality:
>
> * Multiple input files
> * Framework as input file
> * Add module declaration
> * Option for specifying before and after code
> * Option for specifying package
> * Windows support
>
> C:
>     * Self includes
>     * Out of order typedefs of structs
>     * Bitfields
>     * Non-standard extensions
>     * Preprocessor
>     * Arrays with no size marked as "extern".
>
> Objective-C:
>     * Protocols
>     * Properties
>     * Blocks
>     * Categories
>     * Actions
>     * Outlets
>     * Selectors
>
> This is basically what's on the todo list:
>
> https://raw.github.com/jacob-carlborg/dstep/master/todo.taskpaper
>
> There's no point in reporting issues which are listed above.

Jacob,

  I just used your tool to try and convert the GTK 3.0 include files into D stubs, and wanted to say that it worked pretty darn well. As a matter of practice, the single file limitation wasn't too much of a problem - I hacked together a script to just go through all the .h files and put the outputs in the right place. D has become my new favorite scripting language, BTW!

For anybody who cares - here's how I went about it. This is probably the most ignorant and least efficient method possible, but it worked for me to at least display a little window (yay.).

#!/usr/bin/rdmd

import std.stdio;
import std.string;
import std.process;

int main(string[] args)
{	
	foreach(string word; args[1..$])
	{
string cmd = format("dstep /home/jon/devel/gtk-include/%s -o /home/jon/devel/gtk-include/d/%s.d -v -I./ -I./glib-2.0/ -I./gdk/ -I./gdk-pixbuf/ -I/usr/lib/gcc/i686-linux-gnu/4.6/include/ -I/usr/include/pango-1.0/ -I/usr/include/cairo/ -I/usr/include/atk-1.0/", word, chomp(word,".h"));
		writefln("%s",cmd);
		system(cmd);
	}
	return 0;
}

Run with ./convert.d folder/*.h

The only disadvantage to the single-file limitation is that in the case of GTK at least, it has preprocessor directives to keep you from just #include-ing the single file you want to convert, so I just used sed to strip out all the #error directives that come up and force it to do my bidding. I understand DStep doesn't deal with preprocessor yet, but as far as the CLang front-end it uses goes, it might be helpful to find a way to turn off #error-s.

sed -i 's/#error/\/\//g' *.h

The next step was to rename all the D reserved words that GTK used as function arguments - in, out, function, and align are the only ones I can think of off the top of my head. Easy fix for the user (by no means am I complaining), but if you want to streamline the conversion, automatically renaming these kinds of arguments might be a helpful option.

Then, renaming all the duplicate empty struct{} entries in some of the files. You already know about this, but it was probably the most time-consuming part of the process for converting GTK, at least. I couldn't think of an easy way to automate this on my end, because some of the empty structs were necessary to get it to compile.

Finally, putting import statements in all the .d files after I was done. Still a long way to go on this (500 files).

Sorry for the long post, this is probably obvious stuff to everybody else, but I was really impressed with DStep - thank you for creating it!

-Jon
July 08, 2012
On Sunday, 8 July 2012 at 20:01:07 UTC, Jacob Carlborg wrote:
> On 2012-07-08 20:42, Walter Bright wrote:
>> Since OC is a proper superset of C, this shouldn't be a problem. Just
>> run the OC converter as your "C" compiler.
>
> That is not completely true if one is picky. The following code is legal C, but not legal Objective-C :
>
> int id;
> int nil;
>
> In Objective-C "id" is a type.
I suppose this symptom will repeat in the future. I mean, for a particular file extension there may be several code importers. An (exotic?) example might be when some existing code uses one converter, but for some reason new code should use a different one.

What about using the something like this:

mixin convertImport!"header.h";

with ability to specify a particular converter as second template parameter?
July 08, 2012
On Sunday, 8 July 2012 at 22:13:27 UTC, Roman D. Boiko wrote:
> What about using the something like this:
>
> mixin convertImport!"header.h";
>
> with ability to specify a particular converter as second template parameter?

Oh, completely forgot to mention that inside mixin there could be a pragma for compiler instructing it to execute conversion.
July 08, 2012
On 09-Jul-12 02:13, Roman D. Boiko wrote:
> On Sunday, 8 July 2012 at 20:01:07 UTC, Jacob Carlborg wrote:
>> On 2012-07-08 20:42, Walter Bright wrote:
>>> Since OC is a proper superset of C, this shouldn't be a problem. Just
>>> run the OC converter as your "C" compiler.
>>
>> That is not completely true if one is picky. The following code is
>> legal C, but not legal Objective-C :
>>
>> int id;
>> int nil;
>>
>> In Objective-C "id" is a type.
> I suppose this symptom will repeat in the future. I mean, for a
> particular file extension there may be several code importers. An
> (exotic?) example might be when some existing code uses one converter,
> but for some reason new code should use a different one.
>
> What about using the something like this:
>
> mixin convertImport!"header.h";
>

Ineffective even in distant future. Fixed functionality (=compiled, native, etc.) is faster and more practical.
E.g. the above was possible already for something like a year (no less)
the exact magic is:
mixin(translate(import("file.ext"));

But it never scaled to reasonably sized inputs/amounts of files like translating headers.

> with ability to specify a particular converter as second template
> parameter?

However something like :
import "file.ext", FancyImporter; could work and call some 'FancyImporter' for compiler's tools directory to produce file.di

I think extra syntax could be added easily WHEN the need arrives, so far 1:1 converters to extension feels fine.

-- 
Dmitry Olshansky