The more I read about D, the more I fall in love with it. It contains almost everything I ever wanted to
see in a programming language, and I've been coding for 25 years now: from 6502 assembler to C/C+
+ (Zortech! ;-), C#, a bit of Java, Scheme and Prolog, SQL, and "Logix" - an inhouse visual mostly
functional language which my team created for videogame designers/artists. Logix was used to create
some special effects and mini-games on Playstation 3. It was amazing to see that artists with no
programming skills could create incredible stuff given the right visuals / notation...

Anyway, D looks really great, but I'm spoiled with todays popular RAD tools such as integrated debugging, edit-and-continue, code completion, parameter tooltips, refactoring, fast navigation, call graphs, builtin version control, etc... as found in e.g. Visual Studio 2005 + Resharper 2.5 or Eclipse/ IntelliJ IDEA. It's also handy to have a huge standard framework such as DOTNET or J2SE/EE, or even STL/boost. It's not really necessary: my first videogames did not use any code from the OS, it was 100% pure self written assembly code directly talking to the hardware, but that was a century ago ;-)

So as soon as I want to get started with D I find myself stuck (also because of my RSI... I just refuse to type a symbol all over again ;-). It is as if I got this brand new car engine that looks to outperform all others, but I can't find the correct tires, suspension, etc. Frustrating.

One thing I don't like about current IDEs: they still work on the text level, which is horrible for
refactoring in a team (think extreme programming). For example renaming a symbol should be one
change under version control, but it currently means that all source files refering to the symbol (by
name!) must be modified, potentially giving a lot of merge conflicts and co-workers shouting not to
rename a symbol anymore, just leave the bad names... The advantage of a pure drag-drop-connect-
the-dots visual programming language like Logix is that it can work very close to the AST, directly
linking to statements/functions by "pointer/identifier", so a symbolname never matters for the
computer, only for a human, and a rename is just one modification to the symbol and not to its
references. Of course we programmers don't want to work with visual graphs (screen clutter!), we want
to see code, but I think we might also benefit from writing code closer to the AST; after all, code
completion and all those handy code snippets are a bit like that: you insert a foreach loop using a
single keystroke, and fill in the symbols, but its still just text. Why not insert a foreach statetement in a
high-level AST, and regard the text as a representation/tagged navigation of the (high level) AST,
instead of translating the text into the AST... I heared some old LISP editors worked like that, but I never
saw one.

So maybe it would be a good idea to develop and IDE just as (r)evolutionary as D is? Or does it already exist, meaning I just wasted half an hour typing this email ;-)

Keep up the amazing work,
Peter

On Tue, 10 Apr 2007 22:26:40 +0000, Peter Verswyvelen wrote:

> The more I read about D, the more I fall in love with it. It contains almost everything I ever wanted to
> see in a programming language, and I've been coding for 25 years now: from 6502 assembler to C/C+
> + (Zortech! ;-), C#, a bit of Java, Scheme and Prolog, SQL, and "Logix" - an inhouse visual mostly
> functional language which my team created for videogame designers/artists. Logix was used to create
> some special effects and mini-games on Playstation 3. It was amazing to see that artists with no
> programming skills could create incredible stuff given the right visuals / notation...
> 
> Anyway, D looks really great, but I'm spoiled with todays popular RAD tools such as integrated debugging, edit-and-continue, code completion, parameter tooltips, refactoring, fast navigation, call graphs, builtin version control, etc... as found in e.g. Visual Studio 2005 + Resharper 2.5 or Eclipse/ IntelliJ IDEA. It's also handy to have a huge standard framework such as DOTNET or J2SE/EE, or even STL/boost. It's not really necessary: my first videogames did not use any code from the OS, it was 100% pure self written assembly code directly talking to the hardware, but that was a century ago ;-)
> 
> So as soon as I want to get started with D I find myself stuck (also because of my RSI... I just refuse to type a symbol all over again ;-). It is as if I got this brand new car engine that looks to outperform all others, but I can't find the correct tires, suspension, etc. Frustrating.
> 
> One thing I don't like about current IDEs: they still work on the text level, which is horrible for
> refactoring in a team (think extreme programming). For example renaming a symbol should be one
> change under version control, but it currently means that all source files refering to the symbol (by
> name!) must be modified, potentially giving a lot of merge conflicts and co-workers shouting not to
> rename a symbol anymore, just leave the bad names... The advantage of a pure drag-drop-connect-
> the-dots visual programming language like Logix is that it can work very close to the AST, directly
> linking to statements/functions by "pointer/identifier", so a symbolname never matters for the
> computer, only for a human, and a rename is just one modification to the symbol and not to its
> references. Of course we programmers don't want to work with visual graphs (screen clutter!), we want
> to see code, but I think we might also benefit from writing code closer to the AST; after all, code
> completion and all those handy code snippets are a bit like that: you insert a foreach loop using a
> single keystroke, and fill in the symbols, but its still just text. Why not insert a foreach statetement in a
> high-level AST, and regard the text as a representation/tagged navigation of the (high level) AST,
> instead of translating the text into the AST... I heared some old LISP editors worked like that, but I never
> saw one.
> 
> So maybe it would be a good idea to develop and IDE just as (r)evolutionary as D is? Or does it already exist, meaning I just wasted half an hour typing this email ;-)
> 
> Keep up the amazing work,
> Peter

Take a look at this project http://www.dsource.org/projects/codeanalyzer

Knud Soerensen Wrote:

> Take a look at this project http://www.dsource.org/projects/codeanalyzer

@Knud:  not to knock your project, but I think Peter had it more on the mark.

We should move more visual and less textual.  Once Walter releases AST Reflection I think we'll be able to write a "truly" visual IDE with a dynamic debugger, code proveability, symbol recognition (rename a symbol and it renames all cases of it), and the likes.  I'm looking forward to it.

PS:  Anyone know how one might gain access to a complete x86 opcode -> hex table?  All the ones I find only list the mnemonics and description, none list the actual hex.

Thx.

Dan wrote:
> PS:  Anyone know how one might gain access to a complete x86 opcode -> hex table?  All the ones I find only list the mnemonics and description, none list the actual hex.
> 
> Thx.

Have you tried looking in the x86(-64) manuals from Intel or AMD?  They should provide a complete reference for all opcodes, e.g. this (http://www.intel.com/design/intarch/manuals/243191.htm) "Programmer's Manual" for the 286 through PII from Intel (see Appendix A).

Such a code analyzer is a really useful tool indeed.

But as what I actually meant, is more like what Dan said: instead of taking text
as the format of the "source" code, we should use a code-DOM or something
structured like an AST, where symbols are either defined or referenced directly by
pointer/id, not by name.  Code style and formatting is then pure meta data, just
present to reconstruct a textual view in a particular style. Given other
presentation related metadata (e.g. node X,Y positions), the code could just as
well be presented using a graph with nodes and edges. In the fantastic "structure
and interpretation of computer programs" video lectures made in the eighties this
was already mentioned somewhere. If you want a crash course in programming, this
is still amazing stuff:
http://video.google.com/videoplay?docid=5546836985338782440&q=structure+and+interpretation+of+computer+programs

Now this already existed on the Commodore 64, in an assembly language editor: when you typed a line of assembly, it would be parsed immediately, and symbols got resolved lazely on the fly. There was no need to parse the whole program anymore, just linking was needed, so it was really fast (I believe the program's name was "Turbo Assembler", but it had nothing to do with Borland). Also, the code got automatically formatted and syntax highlighted, and symbol lookup was easy. Ofcourse, this is really simple because of the simplicity of assembler, but the principle remains.

Todays refactoring tools and intelligent editors have a really hard time to sync their internal code-DOM with the text; just deleting a curly bracket messes up the whole DOM, making refactoring a fuzzy adnd dangerous concept; clearly a more structured (and faster!) way of entering code could be enforced, but then of course we all should adapt, and we don't like that, the tools should adapt to us no? ;-)

Peter Verswyvelen wrote:
> The more I read about D, the more I fall in love with it. It contains almost everything I ever wanted to
> see in a programming language, and I've been coding for 25 years now: from 6502 assembler to C/C+
> + (Zortech! ;-), C#, a bit of Java, Scheme and Prolog, SQL, and "Logix" - an inhouse visual mostly
> functional language which my team created for videogame designers/artists. Logix was used to create
> some special effects and mini-games on Playstation 3. It was amazing to see that artists with no
> programming skills could create incredible stuff given the right visuals / notation...
> 
> Anyway, D looks really great, but I'm spoiled with todays popular RAD tools such as integrated
> debugging, edit-and-continue, code completion, parameter tooltips, refactoring, fast navigation, call
> graphs, builtin version control, etc... as found in e.g. Visual Studio 2005 + Resharper 2.5 or Eclipse/
> IntelliJ IDEA. It's also handy to have a huge standard framework such as DOTNET or J2SE/EE, or even
> STL/boost. It's not really necessary: my first videogames did not use any code from the OS, it was 100%
> pure self written assembly code directly talking to the hardware, but that was a century ago ;-)
> 
> So as soon as I want to get started with D I find myself stuck (also because of my RSI... I just refuse to
> type a symbol all over again ;-). It is as if I got this brand new car engine that looks to outperform all
> others, but I can't find the correct tires, suspension, etc. Frustrating.
> 
> One thing I don't like about current IDEs: they still work on the text level, which is horrible for
> refactoring in a team (think extreme programming). For example renaming a symbol should be one
> change under version control, but it currently means that all source files refering to the symbol (by
> name!) must be modified, potentially giving a lot of merge conflicts and co-workers shouting not to
> rename a symbol anymore, just leave the bad names... The advantage of a pure drag-drop-connect-
> the-dots visual programming language like Logix is that it can work very close to the AST, directly
> linking to statements/functions by "pointer/identifier", so a symbolname never matters for the
> computer, only for a human, and a rename is just one modification to the symbol and not to its
> references. Of course we programmers don't want to work with visual graphs (screen clutter!), we want
> to see code, but I think we might also benefit from writing code closer to the AST; after all, code
> completion and all those handy code snippets are a bit like that: you insert a foreach loop using a
> single keystroke, and fill in the symbols, but its still just text. Why not insert a foreach statetement in a
> high-level AST, and regard the text as a representation/tagged navigation of the (high level) AST,
> instead of translating the text into the AST... I heared some old LISP editors worked like that, but I never
> saw one.
> 
> So maybe it would be a good idea to develop and IDE just as (r)evolutionary as D is? Or does it already
> exist, meaning I just wasted half an hour typing this email ;-)
> 
> Keep up the amazing work,
> Peter

Wow! You've worded my feelings quite nicely :)

I also hate the way we're writing text files. If the files/modules get large, we split the text files. If we get too many files, we make folders/packages, etc.. It all feels so pre-historic. I wish we could just write code, connect functions, drop-in a pattern and fill in the blanks...

I started with some ideas about using a function repository from which you can drag-n-drop blocks into your project, connecting the parameters with your local variables, possible passing a code block as a delegate to another code block.

Do you have a link to that Logix you've mentioned? I find too many references to it and don't know which one you meant.

L.

Indeed. Actually the whole file system thingy is an ancient leftover, we should be working with an object database as 'file'system by now ;-)

Logix was the codename of proprietary system developed for my previous employer, Playlogic, for creating
realtime 3D effects and procedural content. It is not available.  I even think I'm not allowed to talk about it,
you know how these company policies are... Systems like virtools and quest3D look a bit like it. But if
you're a programmer, you won't like this stuff, because we are trained to read text and symbols (as long as
it contains syntax colors, its amazing how lazy my brain became because of these colorscreens ;-)

Peter Verswyvelen wrote:
> Indeed. Actually the whole file system thingy is an ancient leftover, we should be working with an object
> database as 'file'system by now ;-)
> 
> Logix was the codename of proprietary system developed for my previous employer, Playlogic, for creating
> realtime 3D effects and procedural content. It is not available.  I even think I'm not allowed to talk about it,
> you know how these company policies are... Systems like virtools and quest3D look a bit like it. But if
> you're a programmer, you won't like this stuff, because we are trained to read text and symbols (as long as
> it contains syntax colors, its amazing how lazy my brain became because of these colorscreens ;-)

PlayLogic you say!? I know quite a few guys that work(ed) there :)

L.

Lionello Lunesu escribió:
> Peter Verswyvelen wrote:
>> The more I read about D, the more I fall in love with it. It contains almost everything I ever wanted to
>> see in a programming language, and I've been coding for 25 years now: from 6502 assembler to C/C+
>> + (Zortech! ;-), C#, a bit of Java, Scheme and Prolog, SQL, and "Logix" - an inhouse visual mostly
>> functional language which my team created for videogame designers/artists. Logix was used to create
>> some special effects and mini-games on Playstation 3. It was amazing to see that artists with no
>> programming skills could create incredible stuff given the right visuals / notation...
>>
>> Anyway, D looks really great, but I'm spoiled with todays popular RAD tools such as integrated
>> debugging, edit-and-continue, code completion, parameter tooltips, refactoring, fast navigation, call
>> graphs, builtin version control, etc... as found in e.g. Visual Studio 2005 + Resharper 2.5 or Eclipse/
>> IntelliJ IDEA. It's also handy to have a huge standard framework such as DOTNET or J2SE/EE, or even
>> STL/boost. It's not really necessary: my first videogames did not use any code from the OS, it was 100%
>> pure self written assembly code directly talking to the hardware, but that was a century ago ;-)
>>
>> So as soon as I want to get started with D I find myself stuck (also because of my RSI... I just refuse to
>> type a symbol all over again ;-). It is as if I got this brand new car engine that looks to outperform all
>> others, but I can't find the correct tires, suspension, etc. Frustrating.
>>
>> One thing I don't like about current IDEs: they still work on the text level, which is horrible for
>> refactoring in a team (think extreme programming). For example renaming a symbol should be one
>> change under version control, but it currently means that all source files refering to the symbol (by
>> name!) must be modified, potentially giving a lot of merge conflicts and co-workers shouting not to
>> rename a symbol anymore, just leave the bad names... The advantage of a pure drag-drop-connect-
>> the-dots visual programming language like Logix is that it can work very close to the AST, directly
>> linking to statements/functions by "pointer/identifier", so a symbolname never matters for the
>> computer, only for a human, and a rename is just one modification to the symbol and not to its
>> references. Of course we programmers don't want to work with visual graphs (screen clutter!), we want
>> to see code, but I think we might also benefit from writing code closer to the AST; after all, code
>> completion and all those handy code snippets are a bit like that: you insert a foreach loop using a
>> single keystroke, and fill in the symbols, but its still just text. Why not insert a foreach statetement in a
>> high-level AST, and regard the text as a representation/tagged navigation of the (high level) AST,
>> instead of translating the text into the AST... I heared some old LISP editors worked like that, but I never
>> saw one.
>>
>> So maybe it would be a good idea to develop and IDE just as (r)evolutionary as D is? Or does it already
>> exist, meaning I just wasted half an hour typing this email ;-)
>>
>> Keep up the amazing work,
>> Peter
> 
> Wow! You've worded my feelings quite nicely :)
> 
> I also hate the way we're writing text files. If the files/modules get large, we split the text files. If we get too many files, we make folders/packages, etc.. 

That reminds me of "Divide and conquer". It's not prehistoric, it's just that it's easier for a human to handle small cases. Same as you split a function into smaller functions, you split a file into multiple files to understand the whole easier.

I like the ideas proposed in this thread, but I'd also like to see a screenshot of this visual language, or some more thorough explanation... or a plan to make that kind of IDE, because I can't imagine well how that is going to work.

Peter Verswyvelen wrote:
> Such a code analyzer is a really useful tool indeed.
> 
> But as what I actually meant, is more like what Dan said: instead of taking text
> as the format of the "source" code, we should use a code-DOM or something
> structured like an AST, where symbols are either defined or referenced directly by
> pointer/id, not by name.  Code style and formatting is then pure meta data, just
> present to reconstruct a textual view in a particular style. Given other
> presentation related metadata (e.g. node X,Y positions), the code could just as
> well be presented using a graph with nodes and edges. In the fantastic "structure
> and interpretation of computer programs" video lectures made in the eighties this
> was already mentioned somewhere. If you want a crash course in programming, this
> is still amazing stuff:
> http://video.google.com/videoplay?docid=5546836985338782440&q=structure+and+interpretation+of+computer+programs
> 
> Now this already existed on the Commodore 64, in an assembly language editor: when
> you typed a line of assembly, it would be parsed immediately, and symbols got
> resolved lazely on the fly. There was no need to parse the whole program anymore,
> just linking was needed, so it was really fast (I believe the program's name was
> "Turbo Assembler", but it had nothing to do with Borland). Also, the code got
> automatically formatted and syntax highlighted, and symbol lookup was easy.
> Ofcourse, this is really simple because of the simplicity of assembler, but the
> principle remains.
> 
> Todays refactoring tools and intelligent editors have a really hard time to sync
> their internal code-DOM with the text; just deleting a curly bracket messes up the
> whole DOM, making refactoring a fuzzy adnd dangerous concept; clearly a more
> structured (and faster!) way of entering code could be enforced, but then of
> course we all should adapt, and we don't like that, the tools should adapt to us
> no? ;-)

Representing code as graphs with nodes and edges is IMHO not all too dependent on the underlying programming language. Basically you always have the same three parts in any imperative language (namely sequence, selection and iteration). Representing those graphically removes the need to be coupled to a specific programming language.

Of course you would represent structs and objects too, but behind the scenes they could just as easily be mapped to even a non-OO language, and you'd never know, unless you specifically wanted to alternate between the graphical and the textual representation.

If somebody created such a graphic environment (and wanted "the source code accessible too"), it would not be too much of an additional effort to also have several languages as alternate choices. You'd probably be able to switch on the fly between those programming languages.

After all, you'd be conjuring up the essence, the aspects, the actual logic, and the structure, instead of choosing between the syntactical choices or semantic details.


All this of course forgets the fact that the graphical representation constitutes a Programming Language per se! So, strictly speaking, the "underlying" programming language isn't even necessary. [1]

If you search in the archives, you'll see that I suggested this precisely same thing some three or four years ago in this very newsgroup. At that time it was D related, but today my view has changed, as you see here. :-)

In some sense (in addition to what's been said in this thread), the old Borland GUI editors did much of the same. Essentially the same program created the UI for both C++ and Pascal. It was bundled with both, only with a different language back-end, which probably only was a set of code snippets and rules in a database, from which it picked what it needed. They also had an equivalent code generator for character-UI programs that worked in the same way. Several prominent products were generated with it, for example the Alpha-Four database (a real gem!)

Of course, here you didn't use drag and drop to directly create language constructs, instead you drew up the UI and the input field connections, but the idea is pretty much the same.

Then you of course have the round-trip UML tools, some of which create code for more than one of Java, C++ and Pascal.


Back to the issue at hand, the essential part of such a tool is the actual "graphical programming language". The target audience should be carefully decided on, or the whole thing becomes just "a little for everybody, but not enough for anybody". (I suspect that within the foreseeable future we won't have enough know-how to create a really universal GUI-language. Later, it will become reality, though.)

This being a D newsgroup, the default idea is of course that of "a general purpose" thing, or even "a systems programming" thing. But I have a hard time imagining Linus Torvalds dragging iterative constructs across the screen for the ultimate core in a serious OS kernel project.

I admit that in the long run we seem to be headed towards practically all programming being drag-and-drop (or at that time, grab-and-drop, once the UI hardware becomes way more subtle and hominid centric).

But for the time being, developing, say, this graphical thing for "essentially all things D would be used in" would be a tall order. Rather, things like network programming, web servers, database front ends, or even middle-tier glue SW, robotics, toys (e.g. advanced Tamagotchi SW design, GameBoy apps), point-of-sale cash register SW, medical systems, would seem like a good idea.


It may be possible that for hard-core programming, the keyboard won't go away, not even in a thousand years. Think about it: we have computers, we have (or had) regular typewriters, the ball-point pen, dictaphones (small sound recording devices for notes and letters your secretary later typed for you), mobile phones, calculators, and whatnot. And still today the number of regular pencils around exceeds that of computers, mobile phones and calculators combined.

Speech recognition, GUI programming, AI, and whatever other /plate du jours/ all had their try, but we still bang away like nothing ever happened. Also, the interplay of writing, seeing, reading, and communicating, really is a paradigm that has proven itself in areas vastly more broad than computer programming. Anything where the meaning has to be universally understandable, unambiguous, or thoroughly fixed -- law, procedures, public statements, speaches, in-depth system descriptions, J. Edgar Hoover's files, medical record statements, species descriptions, cake receipes, the US Constitution...

So, "graphical programming" is unlikely to usurp "emacs programming" in our lifetime, in the general sense. But for some niches, it probably will be a killer.


PS, sorry, I didn't intend this to be a bazooka attack on you, or this thread in general. Just aired my 3.14 cents. :-)

----

[1] Upon proofreading, it occurred to me that if there does not exist an "underlying" programming language, then the graphical representation might have a lot more freedom to express ideas. But this is not even a thought, it's merely a hunch. I probably should think some more about it.

Of course, ultimately any such thing is representable by at least an Assembler listing, so one could argue there's _always_ a programming language as the back-end.

But then another could argue that it's having a _specific_ language here that hurts. If we pretend not to have any such language, then we're free to wave hands as we please and decide upon the meaning of such without (at least undue) restrictions.

Extremely interesting. And there'll be a heck of a practical difference between early implementations, I'm sure.