On 4/3/11 3:33 PM, Christian Manning wrote:
> On 03/04/2011 19:30, Andrei Alexandrescu wrote:
>> * What coursework did you complete? As a second-year student this makes
>> it easier for us to assess where you are in terms of expertise. Scores
>> would help as well.
>
> By this do you mean you'd like to see my completed courseworks? Or just
> descriptions and scores (where available)?

Description and score.

Andrei

Am 03.04.2011 20:15, schrieb Fawzi Mohamed:
> On 3-apr-11, at 19:28, Piotr Szturmaj wrote:
> 
>> Fawzi Mohamed wrote:
>>> Looking more maybe I was a bit too harsh, if you define clearly the
>>> goals of your API then yes it might be a good project.
>>> The api doesn't have to be defined yet, but a more detailed definition
>>> of its goals should be there, maybe with code example of some usages.
>>> Questions that should be answered:
>>
>> I know your response is'nt to me, but please let me answer these questions from my point of view, based on my recent work on ddb.
> 
> I think that your responses are very relevant, as it seems to me that your work is nice, and I find that if a GSoC is done in that direction it should definitely work together with the good work that is already done, let's don't create multiple competing projects if people are willing to work together.
> 
>>> * support for static and dynamic types.
>>> how access of dynamic and static types differs, should be as little as
>>> possible, and definitely the access one uses for dynamic types should
>>> work without changes on static types
>>
>> If you mean statically or dynamically typed data row then I can say my DBRow support both.
> 
> yes but as I said I find the support for dynamic data rows weak.
> 
>>> * class or struct for row object
>>
>> I'm using struct, because I think row received from database is a value type rather than reference. If one selects rows from one table then yes, it is possible to do some referencing based on primary key, but anyway I think updates should be done explicitly, because row could be deleted in the meantime. In more complex queries, not all of selected rows are materialized, i.e. they may be from computed columns, view columns, aggregate functions and so on. Allocation overhead is also lower for structs.
>>
>>> * support for table specific classes?
>>
>> Table specific classes may be written by user and somehow wrap underlying row type.
> 
> well with the current approach it is ugly because your calls would be another type, thus either you remove all typing or you can't have generic functions, accepting rows, everything has to be a template, looping on a table or a row you always need a template.
> 
>>> * reference to description of the table (to be able to get also dynamic types by column name, but avoid using too much memory for the structure)
>>
>> My PostgreSQL client already supports that. Class PGCommand has member "fields", which contain information about returned columns. You can even check what columns will be returned from a query, before actually executing it.
> 
> ok that is nice, and my point is that the type that the user sees by default should automatically take advantage of that
> 
>>> * Nice to define table structure, and what happens if the db has another structure.
>>
>> This is a problem for ORM, but at first, we need standard query API.
> 
> I am not so sure about this, yes these (also classes for tables) are part of the ORM, but the normal users will more often be at the ORM level I think, and how exactly we want the things look like that the object level can influence the choice of the best low level interface.
> 
>>> * you want to support only access or also db creation and modification?
>>
>> First, I'm preparing base "traditional" API. Then I want to write simple object-relational mapping. I've already written some code that generated CREATE TABLE for structs at compile time. Static typing of row fields is very helpful here.
> 
> Very good I think that working on getting the API right there and having it nice
> to use is important.
> Maybe you are right and the current DBRow is indeed the best abstraction, but I
> am not yet 100% sure, to me it looks like it isn't the best end user abstraction
> (but it might be an excellent low level object)
> 

I'd hate not having a rows-and-tables view onto the database.
An Object-Relational-Mapper is nice to have of course, but I agree with Piotr
that a traditional view onto the DB is a good start to built an ORM on and I
think that the traditional view should also be available to the user (it'll be
there internally anyway, at least for traditional relational databases).

Also: How are you gonna write queries with only the ORM view? Parse your own
SQL-like-syntax that uses the Object type? Or have the SQL operators as methods?
And then generate the apropriate SQL string?
What about differences in SQL-syntax between different databases?
What about tweaks that may be possible when you write the SQL yourself and not
have it generated from your ORM?

No, being able to write the SQL-queries yourself and having a "low level" view (tables and rows, like it's saved in the DB) is quite important.

However: Since Piotr already seems to have much work done, maybe Christian Manning could polish Piotrs work (if necessary) and create a ORM on top of it?

Oh, and just an Idea: Maybe something like LINQ is feasible for ORM? So you can write a query that includes local containers/ranges, remote Databases (=> part of it will internally be translated to SQL) and maybe even XML (but that could be added later once the std.xml replacement is ready)?

Cheers,
- Daniel

Am 03.04.2011 22:53, schrieb Andrei Alexandrescu:
> On 4/3/11 3:33 PM, Christian Manning wrote:
>> On 03/04/2011 19:30, Andrei Alexandrescu wrote:
>>> * What coursework did you complete? As a second-year student this makes it easier for us to assess where you are in terms of expertise. Scores would help as well.
>>
>> By this do you mean you'd like to see my completed courseworks? Or just descriptions and scores (where available)?
> 
> Description and score.
> 
> Andrei

Put probably in private (a Mail to you, not in this list), right?
I personally wouldn't want to expose these informations to the whole internet..

Cheers,
- Daniel

On 4/3/11 3:56 PM, Daniel Gibson wrote:
> Am 03.04.2011 22:53, schrieb Andrei Alexandrescu:
>> On 4/3/11 3:33 PM, Christian Manning wrote:
>>> On 03/04/2011 19:30, Andrei Alexandrescu wrote:
>>>> * What coursework did you complete? As a second-year student this makes
>>>> it easier for us to assess where you are in terms of expertise. Scores
>>>> would help as well.
>>>
>>> By this do you mean you'd like to see my completed courseworks? Or just
>>> descriptions and scores (where available)?
>>
>> Description and score.
>>
>> Andrei
>
> Put probably in private (a Mail to you, not in this list), right?
> I personally wouldn't want to expose these informations to the whole internet..
>
> Cheers,
> - Daniel

Either way is fine. FWIW many students put such information in resumes available online.

Andrei

Am 03.04.2011 22:57, schrieb Andrei Alexandrescu:
> On 4/3/11 3:56 PM, Daniel Gibson wrote:
>> Am 03.04.2011 22:53, schrieb Andrei Alexandrescu:
>>> On 4/3/11 3:33 PM, Christian Manning wrote:
>>>> On 03/04/2011 19:30, Andrei Alexandrescu wrote:
>>>>> * What coursework did you complete? As a second-year student this makes it easier for us to assess where you are in terms of expertise. Scores would help as well.
>>>>
>>>> By this do you mean you'd like to see my completed courseworks? Or just descriptions and scores (where available)?
>>>
>>> Description and score.
>>>
>>> Andrei
>>
>> Put probably in private (a Mail to you, not in this list), right?
>> I personally wouldn't want to expose these informations to the whole internet..
>>
>> Cheers,
>> - Daniel
> 
> Either way is fine. FWIW many students put such information in resumes available online.
> 
> Andrei

Ok.
At my university they're very reluctant to publish test results online, not even
with the kind-of-anonymous matriculation number.
Workarounds are to either publish it in a private website that only the members
of the corresponding course can access or to let students choose a secret alias
when writing the test, so they can just publish "secret_alias: 5.0" - or even
both (with alias on private website).

Cheers,
- Daniel

On 03/04/2011 21:57, Andrei Alexandrescu wrote:
> On 4/3/11 3:56 PM, Daniel Gibson wrote:
>> Am 03.04.2011 22:53, schrieb Andrei Alexandrescu:
>>> On 4/3/11 3:33 PM, Christian Manning wrote:
>>>> On 03/04/2011 19:30, Andrei Alexandrescu wrote:
>>>>> * What coursework did you complete? As a second-year student this
>>>>> makes
>>>>> it easier for us to assess where you are in terms of expertise. Scores
>>>>> would help as well.
>>>>
>>>> By this do you mean you'd like to see my completed courseworks? Or just
>>>> descriptions and scores (where available)?
>>>
>>> Description and score.
>>>
>>> Andrei
>>
>> Put probably in private (a Mail to you, not in this list), right?
>> I personally wouldn't want to expose these informations to the whole
>> internet..
>>
>> Cheers,
>> - Daniel
>
> Either way is fine. FWIW many students put such information in resumes
> available online.
>
> Andrei

I'm not particularly bothered about it, so I'll probably put them in this thread.
I'll gather all the scores that I can tomorrow, but the last 4 of my assignments have yet to be marked, 2 of them do have preliminary marks though. This is unfortunate as they are the biggest pieces of work I've done thus far.

On 3-apr-11, at 22:54, Daniel Gibson wrote:

> Am 03.04.2011 20:15, schrieb Fawzi Mohamed:
>> On 3-apr-11, at 19:28, Piotr Szturmaj wrote:
>>
>>>> * Nice to define table structure, and what happens if the db has another
>>>> structure.
>>>
>>> This is a problem for ORM, but at first, we need standard query API.
>>
>> I am not so sure about this, yes these (also classes for tables) are part of the
>> ORM, but the normal users will more often be at the ORM level I think, and how
>> exactly we want the things look like that the object level can influence the
>> choice of the best low level interface.
>>
>>>> * you want to support only access or also db creation and modification?
>>>
>>> First, I'm preparing base "traditional" API. Then I want to write simple
>>> object-relational mapping. I've already written some code that generated
>>> CREATE TABLE for structs at compile time. Static typing of row fields is very
>>> helpful here.
>>
>> Very good I think that working on getting the API right there and having it nice
>> to use is important.
>> Maybe you are right and the current DBRow is indeed the best abstraction, but I
>> am not yet 100% sure, to me it looks like it isn't the best end user abstraction
>> (but it might be an excellent low level object)
>>
>
> I'd hate not having a rows-and-tables view onto the database.
> An Object-Relational-Mapper is nice to have of course, but I agree with Piotr
> that a traditional view onto the DB is a good start to built an ORM on and I
> think that the traditional view should also be available to the user (it'll be
> there internally anyway, at least for traditional relational databases).

I fully agree, I probably did not express me clearly enough, a basic table view is a must, but the ORM that one wants to realize might influence how exactly the basic view looks like.
For example it would be nice if a basic row would also somehow be the basic object of the ORM with a dynamic description, and automatically specialized if the db description is available at compiletime.
As I had said before "the object level can influence the choice of the best low level interface", this does not imply that a lower level interface is not needed :).

> Also: How are you gonna write queries with only the ORM view? Parse your own
> SQL-like-syntax that uses the Object type? Or have the SQL operators as methods?
> And then generate the apropriate SQL string?
> What about differences in SQL-syntax between different databases?
> What about tweaks that may be possible when you write the SQL yourself and not
> have it generated from your ORM?
>
> No, being able to write the SQL-queries yourself and having a "low level" view
> (tables and rows, like it's saved in the DB) is quite important.

again I fully agree, but if we want to be able to store business logic in objects that come from the database, and be able to express them easily (for example like ruby does), can be very useful.
At the ORM level one should express at most simple queries, for more complex stuff SQL is a must (there is no point to define another DSL when SQL is already one (but having special methods with common queries can be useful to more easily support non SQL dbs).

> However: Since Piotr already seems to have much work done, maybe Christian
> Manning could polish Piotrs work (if necessary) and create a ORM on top of it?

if accepted I definitely think that Piotrs and Christian will have to coordinate their work

> Oh, and just an Idea: Maybe something like LINQ is feasible for ORM? So you can
> write a query that includes local containers/ranges, remote Databases (=> part
> of it will internally be translated to SQL) and maybe even XML (but that could
> be added later once the std.xml replacement is ready)?

well simple queries, not sure if a full LINQ implementation is too much to ask, but simple queries should be feasible.

Fawzi

Fawzi Mohamed wrote:
> On 3-apr-11, at 19:28, Piotr Szturmaj wrote:
>
>> Fawzi Mohamed wrote:
>>> Looking more maybe I was a bit too harsh, if you define clearly the
>>> goals of your API then yes it might be a good project.
>>> The api doesn't have to be defined yet, but a more detailed definition
>>> of its goals should be there, maybe with code example of some usages.
>>> Questions that should be answered:
>>
>> I know your response is'nt to me, but please let me answer these
>> questions from my point of view, based on my recent work on ddb.
>
> I think that your responses are very relevant, as it seems to me that
> your work is nice, and I find that if a GSoC is done in that direction
> it should definitely work together with the good work that is already
> done, let's don't create multiple competing projects if people are
> willing to work together.

I'm ready to cooperate :)

>>> * support for static and dynamic types.
>>> how access of dynamic and static types differs, should be as little as
>>> possible, and definitely the access one uses for dynamic types should
>>> work without changes on static types
>>
>> If you mean statically or dynamically typed data row then I can say my
>> DBRow support both.
>
> yes but as I said I find the support for dynamic data rows weak.

I've just added row["column"] bracket syntax for dynamic rows.

>
>>> * class or struct for row object
>>
>> I'm using struct, because I think row received from database is a
>> value type rather than reference. If one selects rows from one table
>> then yes, it is possible to do some referencing based on primary key,
>> but anyway I think updates should be done explicitly, because row
>> could be deleted in the meantime. In more complex queries, not all of
>> selected rows are materialized, i.e. they may be from computed
>> columns, view columns, aggregate functions and so on. Allocation
>> overhead is also lower for structs.
>>
>>> * support for table specific classes?
>>
>> Table specific classes may be written by user and somehow wrap
>> underlying row type.
>
> well with the current approach it is ugly because your calls would be
> another type, thus either you remove all typing or you can't have
> generic functions, accepting rows, everything has to be a template,
> looping on a table or a row you always need a template.
>

Could you elaborate? I don't know what do you mean.

>>> * reference to description of the table (to be able to get also dynamic
>>> types by column name, but avoid using too much memory for the structure)
>>
>> My PostgreSQL client already supports that. Class PGCommand has member
>> "fields", which contain information about returned columns. You can
>> even check what columns will be returned from a query, before actually
>> executing it.
>
> ok that is nice, and my point is that the type that the user sees by
> default should automatically take advantage of that
>
>>> * Nice to define table structure, and what happens if the db has another
>>> structure.
>>
>> This is a problem for ORM, but at first, we need standard query API.
>
> I am not so sure about this, yes these (also classes for tables) are
> part of the ORM, but the normal users will more often be at the ORM
> level I think, and how exactly we want the things look like that the
> object level can influence the choice of the best low level interface.

A "defined" DBRow or static one, if used on result which has inequal number of columns or their types aren't convertible to row fields then it's an error. But, if someone uses a static fields, he should also take care that the query result is consistent with those fields.

>>> * you want to support only access or also db creation and modification?
>>
>> First, I'm preparing base "traditional" API. Then I want to write
>> simple object-relational mapping. I've already written some code that
>> generated CREATE TABLE for structs at compile time. Static typing of
>> row fields is very helpful here.
>
> Very good I think that working on getting the API right there and having
> it nice to use is important.
> Maybe you are right and the current DBRow is indeed the best
> abstraction, but I am not yet 100% sure, to me it looks like it isn't
> the best end user abstraction (but it might be an excellent low level
> object)
>

I should state here, that end-user usability is very important to me. I should also clarify that my code isn't completely finished and of course it is a subject to change. Any suggestions and critics are welcome :)

Piotr Szturmaj wrote:
> Any suggestions and critics are welcome :)

Of course I meant critique.

On 4-apr-11, at 02:01, Piotr Szturmaj wrote:

> Fawzi Mohamed wrote:
>> [...]
>> I think that your responses are very relevant, as it seems to me that
>> your work is nice, and I find that if a GSoC is done in that direction
>> it should definitely work together with the good work that is already
>> done, let's don't create multiple competing projects if people are
>> willing to work together.
>
> I'm ready to cooperate :)

great :)

>>>> * support for static and dynamic types.
>>>> how access of dynamic and static types differs, should be as little as
>>>> possible, and definitely the access one uses for dynamic types should
>>>> work without changes on static types
>>>
>>> If you mean statically or dynamically typed data row then I can say my
>>> DBRow support both.
>>
>> yes but as I said I find the support for dynamic data rows weak.
>
> I've just added row["column"] bracket syntax for dynamic rows.

excellent, ideally that should work also for untyped, because one wants to be able to switch to a typed Row without needing to change its code (and it should work exactly the same, so the typed rows will need to wrap things in Variants when using that interface).

>>>> * class or struct for row object
>>>
>>> I'm using struct, because I think row received from database is a
>>> value type rather than reference. If one selects rows from one table
>>> then yes, it is possible to do some referencing based on primary key,
>>> but anyway I think updates should be done explicitly, because row
>>> could be deleted in the meantime. In more complex queries, not all of
>>> selected rows are materialized, i.e. they may be from computed
>>> columns, view columns, aggregate functions and so on. Allocation
>>> overhead is also lower for structs.
>>>
>>>> * support for table specific classes?
>>>
>>> Table specific classes may be written by user and somehow wrap
>>> underlying row type.
>>
>> well with the current approach it is ugly because your calls would be
>> another type, thus either you remove all typing or you can't have
>> generic functions, accepting rows, everything has to be a template,
>> looping on a table or a row you always need a template.
>>
>
> Could you elaborate? I don't know what do you mean.

Well I am not totally sure either, having the row handle better the dynamic case i already a nice step forward, I still fear that we will have problems with the ORM level, I am not 100% sure, that is the reason I would like to try to flesh out the ORM level a bit more.
I would likethat one can loop on all the tables and for each one get the either the generic or the specialized object depending on what is needed.
If one wants to have business logic in the specialized object it should be difficult to bypass them.
Maybe I am asking too much and the ORM level should never expose the rows directly, because if we use structs we cannot have a common type representing a generic row of a DB which might be specialized or not (without major hacking).

>>>> * reference to description of the table (to be able to get also dynamic
>>>> types by column name, but avoid using too much memory for the structure)
>>>
>>> My PostgreSQL client already supports that. Class PGCommand has member
>>> "fields", which contain information about returned columns. You can
>>> even check what columns will be returned from a query, before actually
>>> executing it.
>>
>> ok that is nice, and my point is that the type that the user sees by
>> default should automatically take advantage of that
>>
>>>> * Nice to define table structure, and what happens if the db has another
>>>> structure.
>>>
>>> This is a problem for ORM, but at first, we need standard query API.
>>
>> I am not so sure about this, yes these (also classes for tables) are
>> part of the ORM, but the normal users will more often be at the ORM
>> level I think, and how exactly we want the things look like that the
>> object level can influence the choice of the best low level interface.
>
> A "defined" DBRow or static one, if used on result which has inequal number of columns or their types aren't convertible to row fields then it's an error. But, if someone uses a static fields, he should also take care that the query result is consistent with those fields.

For example doe we want lazy loading of an object from the db? if yes how we represent it with current Rows objects?

>>>> * you want to support only access or also db creation and modification?
>>>
>>> First, I'm preparing base "traditional" API. Then I want to write
>>> simple object-relational mapping. I've already written some code that
>>> generated CREATE TABLE for structs at compile time. Static typing of
>>> row fields is very helpful here.
>>
>> Very good I think that working on getting the API right there and having
>> it nice to use is important.
>> Maybe you are right and the current DBRow is indeed the best
>> abstraction, but I am not yet 100% sure, to me it looks like it isn't
>> the best end user abstraction (but it might be an excellent low level
>> object)
>>
>
> I should state here, that end-user usability is very important to me. I should also clarify that my code isn't completely finished and of course it is a subject to change. Any suggestions and critics are welcome :)

very good :)