On 11/23/2018 4:11 PM, H. S. Teoh wrote:
>  Though I wouldn't
> mind if he gave a bit more love to the dmd optimizer. ;-)  (When will we
> get loop unrolling?)

It's already there:

https://github.com/dlang/dmd/blob/master/src/dmd/backend/gloop.d#L3767

On Saturday, 24 November 2018 at 00:11:37 UTC, H. S. Teoh wrote:
> On Fri, Nov 23, 2018 at 03:56:31PM -0800, Walter Bright via Digitalmars-d wrote:
>> On 11/23/2018 4:59 AM, Chris wrote:
>> > Then there was the whole issue of ARM. Walter said he had no experience with it and kinda didn't care either,
>>
>> Actually, a person in the D community was working for a couple years on an ARM back end, but he eventually lost interest and it was abandoned.
>
> LDC is already well able to target ARM -- I've been using it to write Android apps, and while it takes a bit of work to set up, once it's setup it works very well.

Care to say a bit more about what you are using it for?  Do you write the gui in D too? Using jni or dlangui if so?

On Friday, 23 November 2018 at 19:51:23 UTC, Paulo Pinto wrote:
> On Friday, 23 November 2018 at 10:25:57 UTC, Joakim wrote:
>> On Wednesday, 21 November 2018 at 14:38:07 UTC, Chris wrote:
>>> On Wednesday, 21 November 2018 at 13:26:34 UTC, Joakim wrote:
>>>
>> [...]
>> Why hasn't ruby/rails, Rust, or Nim gotten backing from big players yet? Most things don't get backing from big players, especially initially. What you hope is to create a superior tool that helps small companies grow into the big players someday.
>> [...]
>
> They have, unless you aren't paying attention to the news it seems.
>
> Sun invested into JRuby and Netbeans support, while they dropped support for
> Netbeans, under Oracle's stewardship, they kept investing into JRuby. In fact JRuby is what drives most of the Graal optimizations regarding compilation of dynamic languages and was the genesis of Project Panama, JNI's replacement project.
>
> Besides being sponsored by Mozilla, Rust is now used in Visual Studio Code, IoT Core native layer and an internal distributed service by Microsoft. There are also ongoing projects from Oracle and Dropbox.
>
> Khronos is working together with Mozilla on a multi-platform 3D API framework, which is built with Rust.
>
> Four game studios, namely Ready at Dawn, Chucklefish, SEED and Embark(former DICE/EA devs) have announced that they are building their future tooling and engine improvement with Rust.
>
> Rust was given the spotlight alongside C and C++ at Chrome Developers Summit 2018 regarding the languages currently mature for WebAssembly development.
>
> GNOME is adopting Rust and collaborating with Mozilla to improve the overall development experience when dealing with the gobject OOP model.
>
> Nim just got some support from Status, one of the companies behind Ethereum, but I guess it isn't a major player.
>
> So there is some support going on from them.
>
> --
> Paulo

I was aware of most of those efforts, but I wouldn't call any of them big players like Chris wanted, nor those Microsoft efforts as much of a "backing." We all pay attention when giant corporations really back an OSS language, such as Apple has done with Swift, because that means we get a lot of free dev tools to play with, ;) and these languages don't have it.

Anyway, my intention was not to put those languages down, only to point out that even fairly successful or known languages like these never got a big player really backing them. As I've pointed out with my Linus quotes before about how Linux was built, I think it's much better for OSS tech to have many small or medium backers putting it to many different kinds of uses rather than one or two big backers who tend to use it in a certain way, as Linus says that tends to make the tech less specialized and more likely to survive over the long haul:

https://yarchive.net/comp/evolution.html

On Sat, Nov 24, 2018 at 01:38:07AM +0000, Laeeth Isharc via Digitalmars-d wrote:
> On Saturday, 24 November 2018 at 00:11:37 UTC, H. S. Teoh wrote:
> > On Fri, Nov 23, 2018 at 03:56:31PM -0800, Walter Bright via Digitalmars-d wrote:
> > > On 11/23/2018 4:59 AM, Chris wrote:
> > > > Then there was the whole issue of ARM. Walter said he had no experience with it and kinda didn't care either,
> > > 
> > > Actually, a person in the D community was working for a couple years on an ARM back end, but he eventually lost interest and it was abandoned.
> > 
> > LDC is already well able to target ARM -- I've been using it to write Android apps, and while it takes a bit of work to set up, once it's setup it works very well.
> 
> Care to say a bit more about what you are using it for?  Do you write the gui in D too? Using jni or dlangui if so?

Right now, it's still an experimental project to test out the waters... Things are looking good and it might turn into a "real" project soon, but as of now, it's still experimental.

Basically I have the Java code handle interacting with Android OS (it's possible to do this via NDK but too troublesome and not necessary at the moment), basically just a thin wrapper to forward most of the logic to the D code via JNI. There's an error popup screen purely written in Java, but it's more a development tool for displaying uncaught D exceptions than an actual end-user UI.

The bulk of the code is in D, and renders geometric models via OpenGL ES.  At some point there will be text, and menus, and other UI elements, but right now, just some animated geometric shapes.  It communicates with the Java code via JNI.  Presently, the GL context creation is still handled by the Java code (haven't found a pressing need to manually do it yet, so why not leverage what the Android Java API already provides).

The geometric models are converted by a D utility running on the host system (i.e., not on Android) that converts a bunch of models I have handy into the required format for OpenGL as a bunch of D arrays that I just embed into the executable.

There's also a bunch of GLSL shaders that are stripped and turned into D string literals and embedded into the executable as well -- no need to load resource files at runtime. (This may or may not change as the project gets bigger, though.)  This is also done by a build-time D utility, which also auto-generates nice D interfaces for binding shader inputs.  So instead of manually writing a bunch of glUniformxxx calls (with the associated high risk of typos that causes crashes at runtime), I can just write:

	useProgram(...);
	myshader.worldmatrix = FMatrix(...);
	myshader.worldposition = FVec(1, 2, 3, 1);
	...
	glDrawArrays(...);

and it automatically issues the correct glUniformxxx calls for me.

Currently vertex attribute bindings still have to be done manually, but with just a little more infrastructure in place, and I could auto-generate those too.  Then I wouldn't have to worry about those crash-prone fiddly details that the OpenGL API requires ever again.

The D code obtains the app-specific data directory from the Java code via JNI, and handles its own saving/restoring of UI state via a home-made made (somewhat hamfisted) serializer.  Eventually it will probably be replaced by a real serialization system / library, but right now, it's still in the initial experimental stage so just a crude homebrew system. But the point is that the D code is well able to handle Android app lifetime events.

One interesting aspect of this project is that I designed the D code to be mostly independent of the Android API -- the API is still more or less structured according to an Android app's lifetime callbacks, but is agnostic to the actual implementation.  The OpenGL code being also a subset of desktop OpenGL APIs, I was able to write an X11 driver that runs the same D backend (without any version blocks!), except on the PC instead of an Android device.  This has been very useful for ironing out silly mistakes by allowing development as a local PC program before deploying to the actual Android device. I suppose I *could* use the Android simulator for this, but it's slow, needlessly resource-hungry, and far too klunky to use for fast development. Plus, it opens up the possibility of deploying this program on PCs too, in addition to Android.

D's malleability really helped to pull this off -- by templating away Android-specific types that can be substituted by PC-specific types / wrappers, I can pretty much just reuse the code as-is without the major refactoring into a labyrinth of interfaces and other boilerplate that would have been required in Java to abstract away OS-specific components.  Like I've already said, I haven't gotten to the point of autogenerating JNI interfaces to be directly callable from D, but if the code turns out to need to cross the JNI boundary much more, that's what I'll do.

I also setup my build system (a custom one based on SCons -- Gradle is far too heavyweight for my tastes) to compile and run unittests on the local PC, which will abort if any unittests fail.  So the same source files are compiled twice -- once with -unittest targeting x86_64 (using dmd, no less! :-P), and once without -unittest targeting ARM (using LDC cross-compiler). If the build succeeds at all, I know that all unittests pass, and the APK can be deployed to the device to be further tested there.

The entire build takes about 10-11 seconds to build everything from a clean workspace (includes generating model data, generating shader D APIs, compiling and running unittests, compiling X11 executables, compiling Java code, cross-compiling D code, and building and signing an APK), and about 5-6 seconds on incremental builds, depending on what was changed.  Not as fast as Atila would like, ;-) but reasonably fast and with absolutely minimal system requirements -- I literally did all this with just a text editor, SCons, and SSH. Didn't even fire up Android Studio once.


tl;dr: Thanks to Joakim's efforts and the LDC team, it is quite possible to write Android apps in D today. This project of mine still has a Java component to it, but I'd wager that doing a completely native app with the NDK would work just as well (Joakim's github repo has simple examples of this).  I won't lie, though -- setting up LDC to cross-compile to Android/ARM does require some effort.  But it's not unduly hard, only has to be done once, and your build system takes care of it thereafter (assuming you use a reasonable build system, that is).


T

-- 
People walk. Computers run.

On Saturday, 24 November 2018 at 02:45:46 UTC, H. S. Teoh wrote:
> On Sat, Nov 24, 2018 at 01:38:07AM +0000, Laeeth Isharc via Digitalmars-d wrote:
>> [...]
>
> Right now, it's still an experimental project to test out the waters... Things are looking good and it might turn into a "real" project soon, but as of now, it's still experimental.
>
> [...]

Interesting, good to hear it's working well. I plan to write up a post for the D blog about the Android port next month: this would make a good demo to link to, particularly if the source is available.

Any chance you'll make it available, at least as a demo app if not the source code? If you have commercial aims for this app and don't want to release either, that's fine and I understand.

On Saturday, 24 November 2018 at 00:11:37 UTC, H. S. Teoh wrote:
> On Fri, Nov 23, 2018 at 03:56:31PM -0800, Walter Bright via Digitalmars-d wrote:
>> On 11/23/2018 4:59 AM, Chris wrote:
>> > Then there was the whole issue of ARM. Walter said he had no experience with it and kinda didn't care either,
>>
>> Actually, a person in the D community was working for a couple years on an ARM back end, but he eventually lost interest and it was abandoned.
>
> LDC is already well able to target ARM -- I've been using it to write Android apps, and while it takes a bit of work to set up, once it's setup it works very well.
>
> Frankly, I would not be particularly interested in an ARM target for dmd: dmd's weak optimizer, sorry to say, makes it a rather unattractive option compared to LDC.  And now that LDC is keeping up with DMD releases, I'm quite tempted to just start using LDC for all of my D projects, or at least all the performance-sensitive ones, since it would be keeping up with the latest features / bugfixes.
>
>
>> Building a backend is something usually teams of people work on exclusively.  It's a little unfair to expect me to do one spending an hour or so a day on it. I can't order someone to work on it, I can't hire someone to work on it.
>> 
>> It had to wait until someone both competent and self-motivated stepped up to do it.
>
> I would much rather Walter spend his time on higher-level, more important D issues than writing another dmd backend.  Though I wouldn't mind if he gave a bit more love to the dmd optimizer. ;-)  (When will we get loop unrolling?)
>
>
> T

I can add that the documentable presence of the LLVM backend in LDC since a long time is considered a positive point from the the customer point of view. LLVM has a strong reputation, and simply add value to delivered product, along with a better acceptance that the source code is written in D

DMD adds us values during the development phase, as it's still faster than LDC (also if it's not so much more faster as used to be years ago).

Paolo

On Thursday, 22 November 2018 at 10:08:06 UTC, Chris wrote:

> Yawn. Same strategy again: "Contribute or shut up!". Groundhog Day.

I have been long enough on this forum/mailing list to have seen the same thing happen over and over again. Just like you say - Groundhog Day but I see different thing. I see people who dont understand how little available resources D have and who think that things are the way they are not because of lack of resources but because people are stupid. They go on forum/mailing list having wrong preconceptions thinking that if things are bad because of stupid people and if I enlighten those stupid people things will get better. So they make post.

But then people from community try to explain to them that its lack of resources that is responsible and if you want for thing to change either contribute your own work or donate money. Sadly this kind of response is minority of post on thread and most people are getting carried by secondary topics.

People dont like this response and get frustrated and leave this forum after few more post of their own, but few begin to understand that D doesnt live to its potential because people rather use D than contribute to this project and if there is a problem they just complain instead of working on solving it. Now here things get funny. Well at least to me. In essence they are complaining that other people act just like them without realizing that they complain about them self.


D project has huge surface area - from low level to web development, from game consoles to mobile. People dont want for things just to run but the whole experience to be excellent. Thats not possible with just $1605 per month. we need more than 10x more than that.

"D foundation expenses for H1 2017 have averaged at $1605 per month."

On Friday, 23 November 2018 at 23:49:16 UTC, Walter Bright wrote:
> On 11/23/2018 4:59 AM, Chris wrote:


> Fixing autodecode will break about everything. But you don't like breaking changes.
>
> What do you suggest?

If you put it this way, as you tend to, of course, I look like a lunatic. The breaking changes I don't like (and no-one does) are changes that seem completely random to the user. Normal, "innocent" code just breaks. It may be, because you're preparing feature X that will be available in a year or so and in order for it to work old code has to be sacrificed. If changes need to be made, let's do them in a sensible manner and not just walk over ordinary users' code. I'm under the impression that a fancy feature counts more than code stability ("Let the rabble clean up their code, we are pursuing a higher goal.")

autodecode, however, is a different beast. It is an essential flaw and thus needs to be tackled (or D will be forever flawed). I've proposed a transition via a dual system (legacy / new). It could be a switch like "-autodecode=off". No code needs to break, but those that are willing to make the transition could go ahead. The transition could be documented and there could be an automated tester that tells users later where / how much of their code would break and how to fix it, if they turn autodecode off.

But autodecode is not even open for discussion anymore.

On Friday, 23 November 2018 at 23:56:31 UTC, Walter Bright wrote:
> On 11/23/2018 4:59 AM, Chris wrote:

> Actually, a person in the D community was working for a couple years on an ARM back end, but he eventually lost interest and it was abandoned.
>
> Building a backend is something usually teams of people work on exclusively. It's a little unfair to expect me to do one spending an hour or so a day on it. I can't order someone to work on it, I can't hire someone to work on it.

> It had to wait until someone both competent and self-motivated stepped up to do it.

Of course you cannot do everything alone. I never expected that. But ARM was never really high on the agenda. It need not be dmd, ldc is fine, but it was never really pushed. Maybe it's not "challenging" enough for the core devs, I don't know. But it is essential enough that it should have gotten a higher priority than just to wait until someone stepped up.

I see a lot of other things happening like the re-implementation of the D compiler in D. Fine. But do I as a user really care if it's written in D or C++? I can see that it's a prestigious thing to have, but when I see where D/ARM is, I just wonder if the priorities are right. Maybe the D Foundation could pay (or could have paid) someone, to set up a LDC-ARM toolchain (it need not be a dmd backend). 6 months to year I think would be a realistic time frame? What do you reckon?

If you develop software for ordinary people to use (not in-house frameworks for ads or time tables), they do ask you if there's an app for Android or iOS. And with D it's still too much of a gamble. It's quite a chore to set it up, and then it might break with the every new compiler release and it might or might not cater for various Android (and iOS) releases. And then it's wait, wait, wait...work around etc.

I don't understand how things are prioritized in D. Basic and important things seem to be at the bottom of the list (XML parser), other things get huge attention while they are of dubious value to many users. This is why I don't completely buy the "we don't have enough resources" argument. The scarce resources you have are not used wisely in my opinion. And it is a pity when I see that D has loads of potential (C/C++ interop, Objective-C interop etc.) but other new languages overtake D because they focus on practical issues too.

On 11/23/18 6:49 PM, Walter Bright wrote:
> On 11/23/2018 4:59 AM, Chris wrote:
>> Then there was the whole issue of string handling and autodecode and the way the leadership / big guys in the community dealt with it. I was of the opinion that such an essential issue had to be fixed immediately (of course with a proper path to fix it). I even offered to be the guinea pig and document the transition. But no.
>>
>> Then there were the dreaded dmd updates. "Shit, what will break now?" this question would be my constant companion.
> Fixing autodecode will break about everything. But you don't like breaking changes.
> 
> What do you suggest?

In fact, fixing autodecode will break very little. It will break some things, and I shudder to think of possible "correct" deprecation paths. But for the most part, code only cares about strings, and not the individual items inside them.

1. foreach(x; someString) stays the same. In fact it gets better, because it will be more consistent with phobos
2. someString.front changes, breaking some code, but most of it is already written to deal with it. Take for example std.algorithm.splitter -- there are already cases to handle ranges of char, it will just move from auto-decoding to... intentional decoding.
3. Probably the WORST part is that char implicitly casts to dchar, so things that expect dchar will still compile with strings whose element types switch to char, but instead of decoding, it will promote. Where you will find problems is where code assumes char[].front is dchar instead of using the Phobos ElementType in constraints.
4. The longer we wait, the worse it will be.

There is no way to fix autodecoding without breaking code. So we have to make that sacrifice to make progress. The most painful thing I think would be a possible deprecation path where you have to be explicit about how you want things decoded in order to avoid messages.

-Steve