View mode: basic / threaded / horizontal-split · Log in · Help
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Sean Kelly wrote:
> Andrei Alexandrescu (See Website For Email) wrote:
>> Sean Kelly wrote:
>>> Andrei Alexandrescu (See Website For Email) wrote:
>>>>
>>>> I'm not even thinking of stored procedures as logic vehicles. All 
>>>> stored procedures I've dealt with were little more than simply 
>>>> stored views (SELECT statements). If I want to see customer phone 
>>>> numbers and their orders, I'm glad to let the database do that 
>>>> efficiently by doing an inner join and passing me the information in 
>>>> one shot, instead of having me look at the two relevant tables for 
>>>> the sake of object orientedness. To say nothing about data 
>>>> integrity, which is best taken care of at the database level.
>>>
>>> Stored procedures have a few advantages over inline queries:
>>>
>>> - Speed.  Stored procedures are pre-compiled.  This can have a 
>>> tremendous impact on performance for server applications.
>>> - Decoupling.  If the schema changes the app doesn't typically even 
>>> need recompilation, it "just works" so long as the stored procs are 
>>> updated as well.
>>> - Security.  Access to the tables can be restricted to admins so the 
>>> only thing a user can do is run approved stored procs.
>>> - Encapsulation.  This is really an extension of decoupling, but from 
>>> more of an OO perspective.  Hiding data access and manipulation 
>>> behind structs has the same advantages of the same thing in OO 
>>> programming languages.
>>>
>>> By comparison, views are pre-compiled and provide security, but not 
>>> the other two features.  Quite simply, all interaction with a DB in 
>>> applications I design is through stored procedures.  If a particular 
>>> server doesn't support them then it's a toy.
>>
>> One can tell you work in finance. :o)
> 
> :-)
> 
>> Then probably a framework that does what RoR does, plus offers tighter 
>> binding to large-scale databases (in addition to better speed and 
>> checking), could fill a niche that today RoR does not address for a 
>> mix of practical and philosophical reasons?
> 
> Probably.  I'll admit to not being too terribly familiar with RoR, but 
> it's an appealing concept.  I'll really have to read up on it so I can 
> contribute more to the discussion.
> 
> For what it's worth, one of my first jobs was writing systems to do 
> customized telephony processing for a switched long-distance carrier. 
> Much of the actual logic, in addition to the call records, user data, 
> etc, was stored in SQL.  It was a closed system so security wasn't much 
> of an issue, but the necessary performance would have been impossible to 
> achieve without stored procedures.  So I'll admit to being slightly 
> baffled at why many web servers and other popular technologies nowadays 
> don't seem to consider them important (MySQL didn't have them until 
> recently AFAIK).  Is it simply that automated code generation is more 
> valuable from a cost perspective?
> 
> 
> Sean

In my experience, there are several reasons why databases aren't ever used to their fullest potential in web applications:

1) DBA Hegemony in the workplace - devs aren't always given the rights or permissions to create stored procedures and 
views, even in a development environment.  YMMV.  The common attitude in this scenario is: "I'm Oracle 9i certified, 
you're not, it's my machine so it's my rules, and I golf regularly with the VP."  Just imagine trying to write an 
application with the BOFH in the way - you're going to draft a schema in Er-Win, test it 100 ways from Sunday, hand it 
to him once, and pray to Bob you can code around any mistakes.  Office politics and/or plain ol' bureaucracy is the key 
problem here.

2) Lack of experience/education.  Devs don't always know much beyond basic SQL and cartesian joins.  Also SQL is 
*weird*; things like "null != null" is hard for some folks to grasp, type conversion is hackish (or vendor-specific), 
and date/time wrangling is a chore so they stay where they're comfortable instead.

3) Lack of vendor conformance to features post SQL-99.   Really practical things, like Limit/Top, type conversion, and 
so-forth are non-standard.  As the SQL-99 spec ages, the major RDBMS vendors become more divergent in their application 
of new and useful stuff, which only hurts the educational landscape out there.  The result is that more and more, 
developers have to stick to the lowest-common denominator which is often shored up by middleware that can inject 
vendor-specific features as needed.

4) Abundance of solutions to the above.  With so many great middleware solutions for wrangling your database, why would 
you ever need to know more than basic SQL?  Sure, that's like burying your head in the sand, but if it's a time-saver, 
you're likely going to go this route.


-- 
- EricAnderton at yahoo
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Pragma wrote:
> 
> In my experience, there are several reasons why databases aren't ever 
> used to their fullest potential in web applications:
> 
> 1) DBA Hegemony in the workplace - devs aren't always given the rights 
> or permissions to create stored procedures and views, even in a 
> development environment.  YMMV.  The common attitude in this scenario 
> is: "I'm Oracle 9i certified, you're not, it's my machine so it's my 
> rules, and I golf regularly with the VP."  Just imagine trying to write 
> an application with the BOFH in the way - you're going to draft a schema 
> in Er-Win, test it 100 ways from Sunday, hand it to him once, and pray 
> to Bob you can code around any mistakes.  Office politics and/or plain 
> ol' bureaucracy is the key problem here.

I've heard as much but haven't ever found myself in this situation.  DB 
admins who think programmers aren't capable of writing efficient or 
reliable SQL so they accept work orders and implement everything 
themselves.  The official reason seems to be that the DB admin is more 
aware of the performance demands of different departments, etc.  I 
suppose this is a common case for internal applications in large firms.

> 2) Lack of experience/education.  Devs don't always know much beyond 
> basic SQL and cartesian joins.  Also SQL is *weird*; things like "null 
> != null" is hard for some folks to grasp, type conversion is hackish (or 
> vendor-specific), and date/time wrangling is a chore so they stay where 
> they're comfortable instead.

This one seems to pretty common, and I suppose gives rise to #1.  But 
then many programmers don't seem to adequately understand their language 
of choice, either.

> 3) Lack of vendor conformance to features post SQL-99.   Really 
> practical things, like Limit/Top, type conversion, and so-forth are 
> non-standard.  As the SQL-99 spec ages, the major RDBMS vendors become 
> more divergent in their application of new and useful stuff, which only 
> hurts the educational landscape out there.  The result is that more and 
> more, developers have to stick to the lowest-common denominator which is 
> often shored up by middleware that can inject vendor-specific features 
> as needed.
> 
> 4) Abundance of solutions to the above.  With so many great middleware 
> solutions for wrangling your database, why would you ever need to know 
> more than basic SQL?  Sure, that's like burying your head in the sand, 
> but if it's a time-saver, you're likely going to go this route.

And here I thought there was some practical reason why things were done 
this way :-)  But my experience as a customer for various online sites 
and products (MMORPGs come to mind) reinforces what you've said.  I'll 
admit to being occasionally baffled at the problems some seem to have, 
but I guess it's the same everywhere.  Well, good to know I haven't 
missed the boat, I suppose.


Sean
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Robby wrote:
> 
> To play devils advocate for a bit:
> 
> There are several scenarios where writing procs would seem kinda over 
> the top, I mean, mysql handles itself quite well in performance 
> benchmarks (not saying it's the best) while going for years not 
> implementing them, you can even go as far to say that while they're 
> enabled they're not there by default (MyISAM is the default and doesn't 
> have procs?) and considering how many 'big' sites have mysql implemented 
>  at some level it would be fair to say that procs is not the only way to 
> go.

Fair enough.  I'll admit that the performance demands of many websites 
simply aren't at the level I'm accustomed to, but I'd think the impact 
on maintenance would still be a factor.  However, I guess this is where 
middleware and code generation come in.

> So there's causes and cases for both concerns, RoR seems to only work 
> with the 'no procs, I'll control my domain logic', there may be other 
> implementations that sees procs and views as something that is first 
> class I'm not sure. D from a performance and expressive standpoint could 
> implement both. It's possible and should be considered.

Good to know that RoR does indeed generate inline SQL as I suspected.

> Another post mentions something about injects and if RoR handles it, it 
> does. There are conventions built in to sanitize arguments going in to 
> prevent such things.

Yup.  And I'll admit that dynamically generated queries can be very 
useful for some situations.  I suppose I just haven't been involved in 
developing products where this was the most appropriate way to go.


Sean
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Bruno Medeiros wrote:
snipped
> Hum, in the context of the previous discussion of 1.005 features, I too 
> have been trying to understand what makes Rails so special. After 
> reading that article, it is my understanding that most of the goodness 
> of Rails comes from the ability to generate Ruby code from the 
> database's SQL schema, where that Ruby code handles all or most of the 
> ORM logic. Is that correct? If so, is there anything special about Ruby 
> about the language of Rails? Couldn't a similar framework be made for 
> other languages, like Java for example, with similar results as Rails?
> 
In this context, yes. Matter in fact there is several implementations of 
active record (the ORM logic) coming in from other languages. Keep in 
mind that Rails is a MVC stack for the web, written in Ruby. The ORM is 
just one contribution.
> Also, from the article, I've identified two DSLs: SQL and rhtml (the 
> equivalent of Java's JSPs). 
In the interest of accuracy there's also yaml (an xml like language, 
without the cruft), rjs (javascript templating etc). SQL isn't directly 
used as a DSL, the metadata via database apis is used to generate code.

In both cases, Ruby code is generated from
> code in these two DSLs, but that code generation is performed not by the 
> Ruby compiler during compile-time, but by an external tool (similar to 
> parser generators for example).
No, it's done with Ruby during run time, using Ruby's dynamic and 
reflective abilities.

 If so, that would be mean that the D
> 1.005 features are not necessarily required or useful for the "enabling 
> of such applications", which I think was the point kris was making in 
> the Derailed DSL thread ago. Good interoping with DSLs is decisive, but 
> that doesn't mean the interoping needs to be done at compile-time, by 
> the compiler.
> Is this correct?
> 
Yes
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Dave wrote:
> Robby wrote:
>>> - Are there any well defined studies suggesting that RoR developed 
>>> apps. are cheaper than, say, apps. developed with PHP, using a 
>>> traditional development approach?
>> More enterprise speak?
> 
> Guilty as charged <g> Unfortunately, no matter how promising and cool a 
> technology is many managers have to have a solid justification to try 
> something like RoR even for a smallish project. Heck - I don't blame the 
> managers - I'd have to have some solid justification to divert some of 
> my personal time over to learning RoR, because there's so much to keep 
> up with, and much of it ends up being nothing but a temporary phenom 
> (you mentioned Struts for example).
> 
Oh, I know all about managers *knowing more* :), so no worries, just 
didn't want to answer those questions if they were in that context. And 
I agree about a lot to keep up with, which actually falls somewhat 
inline with the current discussion, if you've taken D as a language of 
choice wouldn't you want to be able to work on the web stack with D 
itself? The thing is the web is *so* complicated that there is always 
going to be generational fads and improvements.
>>> - Is RoR here to stay?
>> I'd say the adoption and the momentum is going to keep it in swing for 
>> quite a while, how long? no clue.. I mean, back in the day Struts was 
>> the thing (bleh), and it's slowly becoming defunct. Can you be 
>> productive in RoR, sure, can you improve it? Sure, it's all about 
>> tastes.. but if you want something in the terms of an independent 
>> report on how great it is, I can't help. I've used it, it's worked for 
>> me on a quite large intranet application, but nothing is the be all 
>> end all.
>>
>>>
>>> Then I've got to ask: How could D possibly improve on RoR enough to 
>>> get people to move away from RoR to DeRailed?
>>>
>> It shouldn't be able moving people from, to d's implementation, it's 
>> about providing a strong stack of development tools to allow people 
>> who want to code in d on the client, on the server and indirectly, for 
>> the web.
> 
> You're right - but is DeRailed really what many of the talented [tango] 
> lib. developers should be working on (if they were so inclined to take 
> direction)?

I can't and won't speak for the Tango developers (I'm not one atm). 
However, one can assume that you can approach Tango as a toolkit, and 
considering that Mango's developers are also apart of Tango, so a 
toolkit for the web kinda makes sense.

Also, in my opinion the best part about trying to compete with 
technologies in problem domains allows you to open your eyes about some 
improvements that may help D as a whole. Consider some of the things 
that Ruby does to make it so easy, reflective abilities, dynamic 
generation and runtime type information.

Now obviously D isn't going to turn into a dynamic language, but 
exposing such traits that D could help a static typed langauge approach 
could help us all in the end. - such as runtime type information.
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Sean Kelly wrote:
> Robby wrote:
>>
>> To play devils advocate for a bit:
>>
>> There are several scenarios where writing procs would seem kinda over 
>> the top, I mean, mysql handles itself quite well in performance 
>> benchmarks (not saying it's the best) while going for years not 
>> implementing them, you can even go as far to say that while they're 
>> enabled they're not there by default (MyISAM is the default and 
>> doesn't have procs?) and considering how many 'big' sites have mysql 
>> implemented  at some level it would be fair to say that procs is not 
>> the only way to go.
> 
> Fair enough.  I'll admit that the performance demands of many websites 
> simply aren't at the level I'm accustomed to, but I'd think the impact 
> on maintenance would still be a factor.  However, I guess this is where 
> middleware and code generation come in.
> 
>> So there's causes and cases for both concerns, RoR seems to only work 
>> with the 'no procs, I'll control my domain logic', there may be other 
>> implementations that sees procs and views as something that is first 
>> class I'm not sure. D from a performance and expressive standpoint 
>> could implement both. It's possible and should be considered.
> 
> Good to know that RoR does indeed generate inline SQL as I suspected.
> 
>> Another post mentions something about injects and if RoR handles it, 
>> it does. There are conventions built in to sanitize arguments going in 
>> to prevent such things.
> 
> Yup.  And I'll admit that dynamically generated queries can be very 
> useful for some situations.  I suppose I just haven't been involved in 
> developing products where this was the most appropriate way to go.
> 
> 
> Sean
For the record I've been on both sides of the fence, tri weekly meetings 
with a DBA, and playing DBA myself. So I fully understand where you're 
coming from, just trying to give a point of view from the dark side ;o).
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Robby wrote:
> Also, in my opinion the best part about trying to compete with 
> technologies in problem domains allows you to open your eyes about some 
> improvements that may help D as a whole. Consider some of the things 
> that Ruby does to make it so easy, reflective abilities, dynamic 
> generation and runtime type information.
> 
> Now obviously D isn't going to turn into a dynamic language, but 
> exposing such traits that D could help a static typed langauge approach 
> could help us all in the end. - such as runtime type information.

Definitely. RTTI and reflection are good to have will probably make it 
in D (as I mentioned, as a couple of the many applications of static 
introspection). The runtime code generation issue is much more 
problematic, so a alternative approach is to do compile-time generation 
for select modules. Running the compiler to generate code will be not 
much slower than running the interpreter of a competing language. The 
advantage is that the resulting code will be easier to make correct and 
faster.

Let me put it this way. If all you need to do is recompile a couple of 
modules when the database schema changes (a process that is easy to 
automate), then this might be more attractive than a scheme that does a 
lot of busywork at runtime to adapt itself dynamically to a structure 
that seldom changes, and to fail "late" when inconsistencies in the code 
are revealed.


Andrei
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
BLS wrote:
> Hi Andrei,
> 
> Andrei Alexandrescu (See Website For Email) schrieb:
> 
>> Sorry, I'm not sure I understand. My understanding of the mechanism is 
>> the following:
>>
>> 1. The app runs a SQL-to-target-language parser to build an idea about 
>> the database.
>>
>> 2. The database folk changes the database in any number of ways. This 
>> is not a process that automatically notifies the target language 
>> application.
>>
>> 3. The target language application must undergo some change to 
>> accommodate the change in the database.
>>
>> I did DB/financial work in 1998. This scenario was a total bitch 
>> because we didn't have small and fast test cases for all logic code to 
>> run when the database changed. Basically it was the customer 
>> (financial analysts) who let us know when something bombed, and they 
>> actually got so used to it that they even weren't pissed anymore.
>> Andrei
> 
> I understand and I know about the problem. Due to the fact that this is 
> a serious task, which should not be answered (like before) within a 
> simple statement, and the fact that I am not used to think and argue in 
> english, requires that I have to write down my thoughts in german first 
> and to translate them later. This may take a while, I hope you understand.
> 
> Allow me a few WHYs
> WHY a datastore called database has such an influence/impact  on 
> consumer applications, f.i. your Software ? I mean a database should be
> a Black Box ONLY accessable through your application by using a public 
> key in the sense of (PGP)... Your Application should master the database 
> and not vice versa.
> 
> Since about 24 years I am in the database business and I have seen only 
> one implemantation which keeps you away from this kind of trouble, let 
> me quote /
> Suneido has an integrated client-server relational database. The 
> database is accessed via a language which includes administration 
> requests and update operations as well as actual queries. The query 
> language is based on the relational algebra language in An Introduction 
> to Database Systems by C.J.Date.
> /
> 
> D is young and refreshing: So WHY not implementing this kind of DB 
> System in D ?
> 
> Since I know that you are very busy I hesitate to offer you a this link 
>  but I think you will find some very interesting information worth 
> spending a few minutes...
> http://www.suneido.com/index.php?option=com_content&task=view&id=49&Itemid=1 
> 
> 
> Further. a big step
> I could imagine that it could be a matter of interest to port the new 
> MINIX  kernel into D having a DB system instead of "journaling file 
> system"  (Beside, still wonder why this kind of filesys. is not allready 
> reality) Why not doing  that in D ? This is indeed a  vision and as our 
> old german chanceler "Helmut Schmidt" says : People having visions 
> should see a doctor really quick.

I've seen Tanenbaum talking about Minix. It's a jewel; the kernel is 
only 4000 lines of code, and everything else is modularly implementable 
in user space. Writing Minix's kernel in D would definitely be an 
interesting D project.


Andrei
February 12, 2007
Re: RoR, Judge Judy, and little old ladies
Andrei Alexandrescu (See Website For Email) wrote:
> Robby wrote:
>> Also, in my opinion the best part about trying to compete with 
>> technologies in problem domains allows you to open your eyes about 
>> some improvements that may help D as a whole. Consider some of the 
>> things that Ruby does to make it so easy, reflective abilities, 
>> dynamic generation and runtime type information.
>>
>> Now obviously D isn't going to turn into a dynamic language, but 
>> exposing such traits that D could help a static typed langauge 
>> approach could help us all in the end. - such as runtime type 
>> information.
> 
> Definitely. RTTI and reflection are good to have will probably make it 
> in D (as I mentioned, as a couple of the many applications of static 
> introspection). The runtime code generation issue is much more 
> problematic, so a alternative approach is to do compile-time generation 
> for select modules. Running the compiler to generate code will be not 
> much slower than running the interpreter of a competing language. The 
> advantage is that the resulting code will be easier to make correct and 
> faster.
> 
> Let me put it this way. If all you need to do is recompile a couple of 
> modules when the database schema changes (a process that is easy to 
> automate), then this might be more attractive than a scheme that does a 
> lot of busywork at runtime to adapt itself dynamically to a structure 
> that seldom changes, and to fail "late" when inconsistencies in the code 
> are revealed.
> 
> 
> Andrei
Yeah, fair enough.
I mentioned something similar in the DeRailed thread and posted a very 
very rough draft[1] how something could work with D's implementation 
now. And now that I think about it further, that very setup could even 
use the new mixin/import features.

What I'm currently working on is using derelict's loader setup and 
porting sqlite to use it, this way I'm not confined to where the 
sqlite.dll is on windows (ugh), then I'm going to start working on 
DeActive using sqlite as a test bed.

I'm actually not looking at this whole thing from a RubyOnRails 
perspective, but for now, from a ActiveRecord perspective. I want a nice 
ORM setup for a gui app I'm working on, and in turn an ORM to keep track 
of my poker hands + backtesting for my poker bot. I just happen to have 
extensive experience through my old job on RubyOnRails and its implemention.

[1]http://www.dsource.org/projects/tango/wiki/DeActive
February 13, 2007
Re: RoR, Judge Judy, and little old ladies
Bruno Medeiros wrote:
> Andrei Alexandrescu (See Website For Email) wrote:
>> After yesterday's hubbub, Judge Judy called and punished me to read 
>> about RoR, in addition to the obligatory sentence of helping a little 
>> old lady cross the street five times a week.
>>
>> So I went and read the nice tutorial at:
>>
>> http://www.onlamp.com/pub/a/onlamp/2006/12/14/revisiting-ruby-on-rails-revisited.html 
>>
>>
>> I have a couple of questions that I assume will be easy to answer by 
>> anyone who actually has used RoR.
>>
>> On the second page of the tutorial, the authors describe how they 
>> write SQL code to create tables, and then how they invoke Ruby to 
>> parse that SQL code and generate (I assume) Ruby wrappers for it.
>>
>> Now consider that the database changes: new tables, new fields, 
>> different types for existing fields, etc.
>>
>> 1. Is now the generated Ruby code is out of sync with the database?
>>
>> 2. In case it is out of sync, what is the way to bring it back in 
>> sync? Manual editing of the Ruby code? Editing the SQL and then 
>> regenerating the wrappers? Some automated way?
>>
>> An additional question: most interesting work in databases is done 
>> through views (SELECT statements) and stored procedures. Can Ruby 
>> parse such stuff and generate appropriate wrappers? If so, what 
>> happens when the views and stored procedures change?
>>
>> I'm asking these questions because I want to figure whether automating 
>> the task of keeping in sync with a database, plus the additional type 
>> safety and speed, are significant advantages in the Web/DB domain. In 
>> such a scenario, error messages like the one in Part 2 
>> (http://www.onlamp.com/pub/a/onlamp/2007/01/05/revisiting-ruby-on-rails-revisited-2.html?page=4) 
>> may be avoided; the code simply fails to compile. I know of domains 
>> where such advantages are very important, but I'm not sure how the 
>> Web/DB domain feels about it.
>>
>>
>> Andrei
> 
> Hum, in the context of the previous discussion of 1.005 features, I too 
> have been trying to understand what makes Rails so special. After 
> reading that article, it is my understanding that most of the goodness 
> of Rails comes from the ability to generate Ruby code from the 
> database's SQL schema, where that Ruby code handles all or most of the 
> ORM logic. Is that correct? If so, is there anything special about Ruby 
> about the language of Rails? Couldn't a similar framework be made for 
> other languages, like Java for example, with similar results as Rails?
> 
> Also, from the article, I've identified two DSLs: SQL and rhtml (the 
> equivalent of Java's JSPs). In both cases, Ruby code is generated from 
> code in these two DSLs, but that code generation is performed not by the 
> Ruby compiler during compile-time, but by an external tool (similar to 
> parser generators for example). If so, that would be mean that the D 
> 1.005 features are not necessarily required or useful for the "enabling 
> of such applications", which I think was the point kris was making in 
> the Derailed DSL thread ago. Good interoping with DSLs is decisive, but 
> that doesn't mean the interoping needs to be done at compile-time, by 
> the compiler.
> Is this correct?
> 

That is correct. But everybody is doing the run-time schtick, so doing 
it during compilation would bring an edge (error detection and faster 
execution).

Andrei
1 2 3 4
Top | Discussion index | About this forum | D home