Hi!

I've searched and searched for some stuff about web development in D
but have found almost nothing. What is the status of the CGI module[1]
that Marten wrote (the link on that page is broken)? Or should I go
with Mango?

Has anyone done any webapps with D?

[1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI

-- 
Antti Holvikari

Reply to Antti,

> Hi!
> 
> I've searched and searched for some stuff about web development in D
> but have found almost nothing. What is the status of the CGI module[1]
> that Marten wrote (the link on that page is broken)? Or should I go
> with Mango?
> 
> Has anyone done any webapps with D?
> 
> [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
> 

as for CGI:

I have built a local search engine in D (search local finls, v1 was built with find/grep/sed/cat/etc.)

http://www.dsource.org/projects/scrapple/browser/trunk/search
http://www.dsource.org/projects/scrapple/browser/trunk/cgi

and a bare bones NNTP viewer

http://www.dsource.org/projects/scrapple/browser/trunk/nntp

but IIRC that is missing some of the code (I'l try to rectify that if I can find the ret of that app.

Antti Holvikari wrote:
> Hi!
> 
> I've searched and searched for some stuff about web development in D
> but have found almost nothing. What is the status of the CGI module[1]
> that Marten wrote (the link on that page is broken)? Or should I go
> with Mango?
> 
> Has anyone done any webapps with D?
> 
> [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
> 

I wish we'd have a framework similar to Django, however, come to think of it .. I'd rather use Django than wait for a D clone .. which I don't think can be more powerful since D doesn't have any dynamic reflection capabilities.

On 1/22/07, Hasan Aljudy <hasan.aljudy@gmail.com> wrote:
> Antti Holvikari wrote:
> > Hi!
> >
> > I've searched and searched for some stuff about web development in D
> > but have found almost nothing. What is the status of the CGI module[1]
> > that Marten wrote (the link on that page is broken)? Or should I go
> > with Mango?
> >
> > Has anyone done any webapps with D?
> >
> > [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
> >
>
> I wish we'd have a framework similar to Django, however, come to think
> of it .. I'd rather use Django than wait for a D clone .. which I don't
> think can be more powerful since D doesn't have any dynamic reflection
> capabilities.

Hmm, hope I'm not missing something here but why would you need
reflection for that?

-- 
Antti Holvikari <http://phphalo.com>

Antti Holvikari wrote:
> On 1/22/07, Hasan Aljudy <hasan.aljudy@gmail.com> wrote:
>> Antti Holvikari wrote:
>> > Hi!
>> >
>> > I've searched and searched for some stuff about web development in D
>> > but have found almost nothing. What is the status of the CGI module[1]
>> > that Marten wrote (the link on that page is broken)? Or should I go
>> > with Mango?
>> >
>> > Has anyone done any webapps with D?
>> >
>> > [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
>> >
>>
>> I wish we'd have a framework similar to Django, however, come to think
>> of it .. I'd rather use Django than wait for a D clone .. which I don't
>> think can be more powerful since D doesn't have any dynamic reflection
>> capabilities.
> 
> Hmm, hope I'm not missing something here but why would you need
> reflection for that?
> 

Well, We don't need it to make a server in D, but if we want to make a powerful framework for web applications, I'd say reflection is a must.

Maybe I don't understand Reflection very well, but there are things in Django that'd only be possible in a dynamic language, or a language with   good Reflection capabilities.

Hasan Aljudy wrote:
> 
> 
> Antti Holvikari wrote:
>> On 1/22/07, Hasan Aljudy <hasan.aljudy@gmail.com> wrote:
>>> Antti Holvikari wrote:
>>> > Hi!
>>> >
>>> > I've searched and searched for some stuff about web development in D
>>> > but have found almost nothing. What is the status of the CGI module[1]
>>> > that Marten wrote (the link on that page is broken)? Or should I go
>>> > with Mango?
>>> >
>>> > Has anyone done any webapps with D?
>>> >
>>> > [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
>>> >
>>>
>>> I wish we'd have a framework similar to Django, however, come to think
>>> of it .. I'd rather use Django than wait for a D clone .. which I don't
>>> think can be more powerful since D doesn't have any dynamic reflection
>>> capabilities.
>>
>> Hmm, hope I'm not missing something here but why would you need
>> reflection for that?
>>
> 
> Well, We don't need it to make a server in D, but if we want to make a powerful framework for web applications, I'd say reflection is a must.
> 
> Maybe I don't understand Reflection very well, but there are things in Django that'd only be possible in a dynamic language, or a language with   good Reflection capabilities.

I've been doing web development in D for a long time. I use my own framework to do it. If there are enough people interested, I could possibly open source it in a few weeks (it needs a lot of cleaning)

it connects over fastcgi interface (I run lighttpd, but apache can be used), or can use  shttpd's embedded for connections directly. It's quite extensive -- it's parsed templates into bytecode, and quite a bit of other stuff... It's definitely pretty solid, and it's the backend of a social network that's being built.

if you guys are really interested, I'll write some samples up to get feedback before opening it.

kenny wrote:
> Hasan Aljudy wrote:
>>
>>
>> Antti Holvikari wrote:
>>> On 1/22/07, Hasan Aljudy <hasan.aljudy@gmail.com> wrote:
>>>> Antti Holvikari wrote:
>>>> > Hi!
>>>> >
>>>> > I've searched and searched for some stuff about web development in D
>>>> > but have found almost nothing. What is the status of the CGI 
>>>> module[1]
>>>> > that Marten wrote (the link on that page is broken)? Or should I go
>>>> > with Mango?
>>>> >
>>>> > Has anyone done any webapps with D?
>>>> >
>>>> > [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
>>>> >
>>>>
>>>> I wish we'd have a framework similar to Django, however, come to think
>>>> of it .. I'd rather use Django than wait for a D clone .. which I don't
>>>> think can be more powerful since D doesn't have any dynamic reflection
>>>> capabilities.
>>>
>>> Hmm, hope I'm not missing something here but why would you need
>>> reflection for that?
>>>
>>
>> Well, We don't need it to make a server in D, but if we want to make a powerful framework for web applications, I'd say reflection is a must.
>>
>> Maybe I don't understand Reflection very well, but there are things in Django that'd only be possible in a dynamic language, or a language with   good Reflection capabilities.
> 
> I've been doing web development in D for a long time. I use my own framework to do it. If there are enough people interested, I could possibly open source it in a few weeks (it needs a lot of cleaning)
> 
> it connects over fastcgi interface (I run lighttpd, but apache can be used), or can use  shttpd's embedded for connections directly. It's quite extensive -- it's parsed templates into bytecode, and quite a bit of other stuff... It's definitely pretty solid, and it's the backend of a social network that's being built.
> 
> if you guys are really interested, I'll write some samples up to get feedback before opening it.

That sounds amazing!! Examples would be great, though. :)

ok, here is some explanation of the framework I use: Sorry for the length of the post. I promise I will never make one this big again. It took quite some time to write.

1. most of the core is somewhat contract programmed and heavily unittested -- that is, I'm pretty sure it won't break, and the few times I've got a few good ideas, the unittests told me they weren't so good.

2. connecting to the content server can be done one of two ways: 1. over fastcgi (I use lighttpd) or 2. through shttpd embedded. Soon, a third may be added, integrating it directly into the kernel -- similar to the red hat content server, and I'm looking into every possible way to have D load up the various modules in real time. (eg. call a function, and all of the .d files are recompiled and reloaded) This could be done similar to derelict style, or maybe through the mixins idea somewhere else in this newsgroup.

3. it's very fast -- yet has tons of opportunities to be faster (possibly double the speed). Originally, I started with straight output using fastcgi's print functions, and string concats. concatenation is very slow, so I moved to a variadic function to print stuff out (cause templates weren't good enough back then). I now use a template system for design, which parses the template into "panels" then stores the byte code generated for later rendering. This decreased performance by 50% -- but in the future, the D-templates could easily parse it, (if it can ever accept flat files for input) and easily generate machine code to do just that. The beauty of templates though, is templates can change, and they won't be reloaded in memory until I push the button. Also, no downtime either. In every example I had MySQL was ALWAYS the limiting factor. On my laptop, if I cached the data in RAM, I could easily do 6000-14000 requests a second, but using mysql, that number dropped to about 1
000 requests per second (for a dynamic page). Profiling shows that mysql takes about 80%-90% of the time, so improving the speed of templates will yield little results.

4. it's still pretty immature. I will need help getting it super stable and production worthy. There are no bugs that I know of, but that doesn't mean they don't exist.

5. I use custom links and forms, but they are not necessary. This is because the program I'm writing needs everything to go through an iframe or through ajax, so I don't mess with the layers on top. It gets complicated explaining, so I just won't... the standard <a href="?lala=2">link</a> will work quite fine. It's optional. Because of the nature of what I do, the applications must start having more motion. So, one of the directions I'm moving in, is to make it easier to write inline the stuff to mess with the javascript/dom.

6. you will see them... the qry(...) and the prt(...) are hideous... they shouldn't look after a ^ to insert the data. It should be templated version of writef that everything is made on compile time. Later that can be made... for now, who cares... it's fast enough, and old code dies hard.

---------------

first, it searches for main.pnl in the panels dir and starts from there... here are various template parameters that can be seen as examples... HTML is designed to be completely intermingled

<%interface name: "myprofile" %>
<%load Base %>
<%load Me %>
<%if uid == 0 %>
	<b>you are not logged in</b>
<%else%>
	<div>your name is <%=Me.fullname%>
	<%load Users:Friends {friends_gid: $Me.friends_gid } %>
	<div>You have <%=Users.total%> friends</div>
	<%loop Users:Friends %>
	// copy and pasted from one of my projects...
            <div class="friend">
                <div class="izquierda_amigos">
                    <img src="i/60/<%=Users.pic%>" alt="<%=Users.fullname%>" />
                </div>
                <div class="derecha_amigos">
                    <h3><a href="#" class="sub_azul"><%=Users.firstname%> <%=Users.lastname%></a></h3>
                </div>
            </div>
           <%endloop%>
<%endif%>

you can make/set vars in the templates...
<%variable name: "varname" type: "uint" default: "2" %>
<%set name: "varname" value: 3 %>

linking is easy... in any panel, you can define where a sub panel will go:

// ex: main.pnl
<%interface name: "main" %>
html... blah blah blah
<%panel name: "m" default: "home" %>
more html blah blah blah

then in the url, to put the panel "home" inside of the subpanel location "m" you do nothing... however, let's say that you want the panel named "myprofile" in there. That's done with the URL string:

<a href="?m=myprofile">link to my profile</a>

pretty easy.

for those looking for a truly object oriented design, don't look here... lol, it looks like I'm writing C/PHP mix most of the time, cause of my heavy use of assoc arrays and loops for parsing...

The modules are written in D and are quite easy to make as well... There are three types... Loopable, One-shot (no loops), and read only (for feedback to forms) For example, let me show you what it takes to make the above Users:Friends module (it's a loopable type):

static class Users : TemplateStaticImport {
	static:
	char[] firstname;
	char[] lastname;
	char[] fullname;
	char[] pic;
	char[][][] res;
	
	uint page;
	uint size;

	const char[] name = "Users";
	
	static this() {
		// all users in the website
		GenericLoop!(name ~ ":Site", generic_init,
			function () {
				reload = true;
				total = toUint(q1v("SELECT COUNT(*) FROM `users`"));
			},
			function int() {
				if(reload) {
					res = qry("SELECT `uid`, `firstname`, `lastname`, `fullname`, `pic` FROM `users`");
					count = res.length;
				}
				
				return generic_loop;
			});
		
		// You can make as many of these as you want in this class actually... technically, you don't even need it in a class, but it is for now here in a class, because they used to be instantiated, not static...

		// just the group group of people I ask (which happen to be friends of the person)
		GenericLoop!(name ~ ":Friends", generic_init,
			function () {
				assert(friends_gid);
				assert(*friends_gid);
				reload = true;
				total = toUint(q1v("SELECT COUNT(*) FROM `group_members` WHERE `type` = '" ~ s_gid_user_friends ~ "' AND `gid` = '^' AND `id` != '^'", *friends_gid, uid));
			},
			function int() {
				if(reload) {
					assert(friends_gid);
					assert(*friends_gid);
					// I know it's more optimized to do a direct relations table here, but that's not what we need... we need generic groups of people for queries like this all over the place :) plus... mysql 5.1 partitions will solve my problems almost entirely
					res = qry("SELECT u.`uid`, `firstname`, `lastname`, `fullname`, `pic` FROM `group_members` LEFT JOIN `users` u ON (u.`uid` = `id`) WHERE `type` = '" ~ s_gid_user_friends ~ "' AND `gid` = '^' AND `id` != '^'", *friends_gid, uid);
					count = res.length;
				}
				
				return generic_loop;
			});
	}
	
	
	uint* friends_gid;
	uint this_init(PNL* pnl, void*[char[]] args) {
		pnl.registerString(name ~ ".firstname", &firstname);
		pnl.registerString(name ~ ".lastname", &lastname);
		pnl.registerString(name ~ ".fullname", &fullname);
		pnl.registerString(name ~ ".pic", &pic);
		
		if("friends_gid" in args) {
			friends_gid = cast(uint*)args["friends_gid"];
		}
		
		return 0;
	}
	
	void this_set() {
		Base.zid = toUint(res[current][0]);
		firstname = res[current][1];
		lastname = res[current][2];
		fullname = res[current][3];
		pic = res[current][4];
	}
	
	mixin Generic!(this_init, this_set) G;
	alias G.loop generic_loop;
	alias G.pnl_init generic_init;
	alias G.current current;
	alias G.reload reload;
	alias G.total total;
}

The other thing that's written in D are the functions (things that modify the database). Our database doesn't use cluster, it uses replication, so every write has to go through the single master -- for this reason, all qry (full table), q1r (one row), q1c (one column) and q1v (one value) talk to the database at localhost, and q(for writing) talks to the master. q(...) can only be found in functions.d ... which is good to prevent possible failures.

functions are simple. All data coming in comes to the POST/GET associative array, and the function is called with the query or post string func=logout or func=postwcomment

case "logout":
	if(uid) {
		q("UPDATE `session` SET `online` = 0 WHERE `uid` = ^ AND `sid` = ^", uid, sid);
		q("UPDATE `users` SET `last_logout` = ^ WHERE `uid` = ^", request_time, uid);
		uid = zid = 0;
		is_me = false;
	}
break;
case "postwcomment":
	if(uid) {
		char[]* wcomment = ("wcomment" in POST);
		if(wcomment) {
			// There's no injection here, strings are automatically escaped with the query functions... :)
			q("INSERT INTO `profilewall` ( `profile_uid` , `poster_uid` , `wid` , `text`, `time` )"
					"VALUES ('^', '^', '', '^', '^');", zid, uid, *wcomment, request_time);
		}
	}
break;



Still interested? Feedback? Thoughts?

Antti Holvikari wrote:
> On 1/22/07, Hasan Aljudy <hasan.aljudy@gmail.com> wrote:
>> I wish we'd have a framework similar to Django, however, come to think
>> of it .. I'd rather use Django than wait for a D clone .. which I don't
>> think can be more powerful since D doesn't have any dynamic reflection
>> capabilities.
> 
> Hmm, hope I'm not missing something here but why would you need
> reflection for that?
> 

Django uses python's reflections capabilities to bind classes to SQL tables and build administration interfaces in ways that I don't thing even the compile time reflection capabilities of D can implement.

The following code taken from the Django tutorial admin interfaces for this Master/Details model with all the features you will expect: Pagination, searching, basic validation and form field grouping. Adding filtering and custom validation is really easy.

from django.db import models

class Poll(models.Model):
    question = models.CharField(maxlength=200)
    pub_date = models.DateTimeField('date published')

	class Admin:
		fields = (
			(None, {'fields': ('question',)}),
			('Date information', {'fields': ('pub_date',)}),
		)

class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice = models.CharField(maxlength=200)
    votes = models.IntegerField()

    class Admin:
        pass

As you can see, Django uses reflection to build all of this functionality with just some hints from the programmer.

>> if you guys are really interested, I'll write some samples up to get feedback before opening it.
> 
> That sounds amazing!! Examples would be great, though. :)

Yeah, it does sound pretty interesting :)

 - Paul