October 07, 2012
On 2012-10-07 00:14, Nick Sabalausky wrote:

> I don't know about the rest of DSSS as I only ever used the
> 'rebuild' component. But as for rebuild, there are problems:
>
> For one thing, 0.76 is generally considered to work much better than
> 0.77 and the final version, 0.78 (I forget the details, but a lot
> of people including me have had problems with 0.78 that never showed up
> with 0.76). But despite that, read-made builds aren't available for
> 0.76. As DSSS is dead none of this is likely to get fixed. And
> there's no reason to fix rebuild since RDMD is a superior and non-dead
> alternative to rebuild.

I'm still using 0.75, when I'm using it.

> Also, rebuild is slow, even with "one at a time" disabled. Try
> compiling DVM with the included rebuild-based script and the
> RDMD-based one. It takes a fair amount of time with rebuild
> (even with "one at a time" off), but with RDMD it's almost instant.

I'm well aware that RDMD is a lot faster than Rebuild.

> There's no reason for anyone to use rebuild anymore, and very few
> people do.

Maybe not only Rebuild, but DSSS offers more than RDMD. DSSS supports building libraries, build files, generating documentation and other features. With RDMD you must likely need a shell script for the build flags. Shell scripts aren't cross-platform which means you need a .bat file on Windows. That will result in duplication which is not good.

> So does RDMD, unless I'm mistaken.

Yes, but there are still reasons to use DSSS for D1, see above.

-- 
/Jacob Carlborg
October 07, 2012
On 2012-10-07 10:55, Russel Winder wrote:

> Why only PostgreSQL. Shouldn't it also work with MySQL, Oracle, DB2,
> PervasiveSQL, SQLite3, etc.?
>
>  From the example I assume that this is just a library for managing
> connections and that everything else is just string-based SQL
> statements. Groovy's and Python's lowest level is roughly the same.
> However on top of these are expression languages in Groovy / Python so
> as to remove the reliance on string processing, i.e. use an internal DSL
> to do all the SQL stuff. For Python this is SQLAlchemy, for Groovy it
> will hopefully be GSQL. I am sure Scala and C++ have something similar?

They do.

> So I guess the question is how to ensure this all works with all SQL
> systems and how to put an abstraction layer over this to avoid all the
> error prone string manipulation?

ActiveRecord in Ruby on Rails uses several layers to handle all database related functionality. At the highest level there's a DSL which allows you to write the SQL queries mostly in Ruby. Another library, ARel, is used by ActiveRecord to generate the SQL code from the DSL. ARel handles all the differences among all the supported databases. ARel then passes the SQL code back to ActiveRecord where a lower layer handles the connections to the database and performs the actual query.

Then you have another layer that transforms the response into objects, sets up all the relations and so on.

-- 
/Jacob Carlborg
October 07, 2012
On Sun, 07 Oct 2012 02:51:42 -0700
Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Sunday, October 07, 2012 11:43:21 Peter Alexander wrote:
> > On Saturday, 6 October 2012 at 21:19:58 UTC, Joseph Rushton
> > 
> > Wakeling wrote:
> > > 
> > > Might be worth placing some prominent message on DSource stating that it's being maintained to document all the D1 work and projects, but that the active work is now over at dlang.org?
> > 
> > +1
> > 
> > Getting tired of people heading to dsource and assuming that D is dead.
> 
> Isn't part of the problem that no one can get ahold of the person who runs it? At least, that's what I remember being discussed previously. It was my understanding that that's why we've never been able to get dsource cleaned up or really changed at all.
> 

No, I think he'd just been busy. I've seen him around here a few times lately.


October 07, 2012
Jacob Carlborg wrote:
> On 2012-10-07 10:55, Russel Winder wrote:
>
>> Why only PostgreSQL. Shouldn't it also work with MySQL, Oracle, DB2,
>> PervasiveSQL, SQLite3, etc.?
>>
>>  From the example I assume that this is just a library for managing
>> connections and that everything else is just string-based SQL
>> statements. Groovy's and Python's lowest level is roughly the same.
>> However on top of these are expression languages in Groovy / Python so
>> as to remove the reliance on string processing, i.e. use an internal DSL
>> to do all the SQL stuff. For Python this is SQLAlchemy, for Groovy it
>> will hopefully be GSQL. I am sure Scala and C++ have something similar?
>
> They do.
>
>> So I guess the question is how to ensure this all works with all SQL
>> systems and how to put an abstraction layer over this to avoid all the
>> error prone string manipulation?
>
> ActiveRecord in Ruby on Rails uses several layers to handle all database
> related functionality. At the highest level there's a DSL which allows
> you to write the SQL queries mostly in Ruby. Another library, ARel, is
> used by ActiveRecord to generate the SQL code from the DSL. ARel handles
> all the differences among all the supported databases. ARel then passes
> the SQL code back to ActiveRecord where a lower layer handles the
> connections to the database and performs the actual query.
>
> Then you have another layer that transforms the response into objects,
> sets up all the relations and so on.

Having distinct layers that don't know each other isn't always a good idea.

In my prostgres client one may specify field types at compile time. If I had divided the client into two separate layers it would return a Variant[] at first layer, then convert it to user specified tuple at the second. For example:

auto cmd = new SqlCommand(connection, "SELECT 1, 'abc'");
auto untypedRow = connection.executeRow(); // return DBRow!(Variant[])
auto typedRow = connection.executeRow!(int, string)(); // returns DBRow!(int, string);

Internally executeRow could always take a Variant[], then convert it to Tuple!(int, string), but it's suboptimal. Firstly, it must allocate an array of two Variants, then each Variant must be coerced to the corresponding type.

Instead, the client fills each of the tuple field directly as they come from the socket stream. With binary formatting all it has to do is swapEndian() on integers and floats (no parsing!). Of course, there's one allocation for the string, but if we change field type to char[4], there'll be no allocation at all.

Just wanted to illustrate that "layers" shouldn't always be separate.
October 07, 2012
> Having distinct layers that don't know each other isn't always a good idea.

> Just wanted to illustrate that "layers" shouldn't always be separate.

Actually I'm not sure how separate they are in ActiveRecord. I wanted to mostly point out that generating the SQL was done by a separate library.

-- 
/Jacob Carlborg
October 07, 2012
>> Isn't part of the problem that no one can get ahold of the person who
>> runs it? At least, that's what I remember being discussed previously.
>> It was my understanding that that's why we've never been able to get
>> dsource cleaned up or really changed at all.
>> 
>
> No, I think he'd just been busy. I've seen him around here a few times
> lately.

Are you sure about that? There is a guy named Brad Anderson posting here, but he doesn't seem to be the Brad Anderson that dsource.org is registered to.
October 07, 2012
On 10/07/2012 10:55 AM, Russel Winder wrote:
> Why only PostgreSQL. Shouldn't it also work with MySQL, Oracle, DB2,
> PervasiveSQL, SQLite3, etc.?

I don't have sufficient experience with SQL to be able to really make a judgement here, but is there a case for a std.sql or std.db that would provide a uniform D interface to the arbitrary DB of choice?

Perhaps better as something in Deimos rather than Phobos, as I imagine it would bring in a bunch of external dependencies that the standard library shouldn't really have.  Am I right that there's something in Adam Ruppe's web modules that's heading in this direction?
October 07, 2012
On Sunday, 7 October 2012 at 17:06:31 UTC, Joseph Rushton Wakeling wrote:
> Am I right that there's something in Adam Ruppe's web modules
> that's heading in this direction?

Yeah, though I'm a little biased toward mysql since that's what I use every day, so some of the stuff that should be in generic database.d are instead in mysql.d.

But my stuff indeed does sql strings which can then be passed to a Database class with pretty uniform interface, at least for basic queries, for some major dbs.

https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff


Among the string manipulation stuff is class DataObject, which builds an UPDATE or INSERT query:

auto obj = new DataObject(db, "table_name");
obj.id = 10;
obj.name = "cool";
obj.commitChanges(); /* runs:
  if(db.query("select id from table_name where id = 10").empty)
      db.query("insert into table_name (id, name) values (10, 'cool')");
   else
      db.query("UPDATE table_name set name = 'cool' where id = 10");
*/

and also a build data object subclass from sql create table which kinda works, ugly mixin stuff.


And there's also a SelectBuilder which does basic concat stuff:

auto query = new SelectBuilder();
query.table = "something";
query.fields ~= "something.*";
query.wheres ~= "id > ?0";

db.query(query.toString(), 10);


expands into

select something.* from something where (id > 10);



Nothing super fancy but it works for me.
October 07, 2012
On Sunday, 7 October 2012 at 17:06:31 UTC, Joseph Rushton Wakeling wrote:
> On 10/07/2012 10:55 AM, Russel Winder wrote:
>> Why only PostgreSQL. Shouldn't it also work with MySQL, Oracle, DB2,
>> PervasiveSQL, SQLite3, etc.?
>
> I don't have sufficient experience with SQL to be able to really make a judgement here, but is there a case for a std.sql or std.db that would provide a uniform D interface to the arbitrary DB of choice?
>
> Perhaps better as something in Deimos rather than Phobos, as I imagine it would bring in a bunch of external dependencies that the standard library shouldn't really have.  Am I right that there's something in Adam Ruppe's web modules that's heading in this direction?

There was a std.database proposal from Steve Teale, but it appears to have died.

http://prowiki.org/wiki4d/wiki.cgi?ReviewQueue

It could work like in other languages with OO support.

Everything is interface based and it is up to the respective driver to provide proper implementations. Those implementations can be provided either as static or dynamic libraries.

The important thing are interfaces, as such you're not bringing external dependencies. Unless the D community decides to have the drivers as part of the language (comes with batteries kind of thing).

--
Paulo
October 07, 2012
On Sunday, 7 October 2012 at 12:39:35 UTC, Piotr Szturmaj wrote:
> In my prostgres client one may specify field types at compile time. If I had divided the client into two separate layers it would return a Variant[] at first layer, then convert it to user specified tuple at the second. For example:
>
> auto cmd = new SqlCommand(connection, "SELECT 1, 'abc'");
> auto untypedRow = connection.executeRow(); // return DBRow!(Variant[])
> auto typedRow = connection.executeRow!(int, string)(); // returns DBRow!(int, string);
>
> Internally executeRow could always take a Variant[], then convert it to Tuple!(int, string), but it's suboptimal. Firstly, it must allocate an array of two Variants, then each Variant must be coerced to the corresponding type.
>
> Just wanted to illustrate that "layers" shouldn't always be separate.

It's not a very convincing illustration. In practice the overhead of those operations would likely be completely insignificant compared to performing the actual database query. Avoiding intermediate layers for optimality's sake seems like a bad case of premature optimization to me.