On Friday, 9 December 2011 at 16:38:01 UTC, Andrei Alexandrescu wrote:
> I think the domain dfeed.kimsufi.thecybershadow.net should be completely invisible to people who browse the news, google, etc. When googling for posts one should /only/ find forum.d-programming-language.org/xxx URLs.

Absolutely. As I stated in the OP, the current address is temporary. I'll change it to a redirect after we create the official one.

> I hope you don't take this the wrong way, but (a) I think having posts under that master domain looks integrated and professional, (b) if you later get hit by a bus or simply move on to other interests the domain doesn't go dark.

I'm all for this (which is also why I'm for providing shell access to D VIPs).
> The second point also raises questions such as who owns the user database etc. Long-term I don't think it's reasonable (and I don't think you want!) to keep user account information on your own server for all installations of dfeed.

I'm not sure what you mean by "all installations of dfeed", and how that affects where data is stored. Since I use SQLite, data is in a file on the filesystem.

> On the other hand, I don't want this "invisibility" to affect negatively letting others know about your product. You should put a notice in e.g. the footer a la "Page generated by dfeed", with a link to the product.

The page accessible via the "Help" link has an "About" section.

> I suggest you package your code in ways that make it easy to deploy to any other website.
> This all leads to the idea that at best you should package dfeed as a classic "product" that you offer via github as an independent project, with releases, updates, bug fixes, and the such. Whenever you update the product you notify your clients that they can update their installations through a classic process (running a script etc). The disadvantage is that clients would run the binaries which means more deployment issues and the such. Also, given that dfeed does quite a few more things than just bridging with NNTP you'd need to do some serious work on packaging.

This goes a bit beyond the project's initial scope, since all I wanted was to create something for D... There would be a good amount of work involved in refactoring it in such a way so that it could be easily set up and used for other projects. It would be much easier if the need appeared before this work is started, so that there would be a clear goal for what needs to be modularized, configurable and customizable.

> Please advise on how to best proceed in the short and long term. I think in the short term we can run off your domain (with the visible URLs using top domain d-programming-language.org), and migrate later to a classic product installation configuration that will preserve the URLs.

Sorry, I'm not following you again. Perhaps there was a misunderstanding regarding how much the program is tied to my server? As it is, all data (newsgroup/mailing list posts, users and user preferences) are stored in a single SQLite database file. Assuming all necessary build tools (of which only D is a requirement) are present, the program otherwise has no other dependencies - it doesn't even need a separate web server (like Apache). So, other users of the program would run it on their own server, and have their own database of everything on their server's filesystem.

On Friday, 9 December 2011 at 17:03:24 UTC, Vladimir Panteleev wrote:
> the program otherwise has no other dependencies

OK, that's a lie. It obviously needs SQLite, and it also uses iconv to decode messages in multibyte (non-UTF) encodings.

On 12/9/11 11:03 AM, Vladimir Panteleev wrote:
> This goes a bit beyond the project's initial scope, since all I wanted
> was to create something for D... There would be a good amount of work
> involved in refactoring it in such a way so that it could be easily set
> up and used for other projects. It would be much easier if the need
> appeared before this work is started, so that there would be a clear
> goal for what needs to be modularized, configurable and customizable.

I understand. So you want to approach this by serving forum content off your own server under the guise of d-programming-language.org. This should be fine.

>> Please advise on how to best proceed in the short and long term. I
>> think in the short term we can run off your domain (with the visible
>> URLs using top domain d-programming-language.org), and migrate later
>> to a classic product installation configuration that will preserve the
>> URLs.
>
> Sorry, I'm not following you again. Perhaps there was a misunderstanding
> regarding how much the program is tied to my server? As it is, all data
> (newsgroup/mailing list posts, users and user preferences) are stored in
> a single SQLite database file. Assuming all necessary build tools (of
> which only D is a requirement) are present, the program otherwise has no
> other dependencies - it doesn't even need a separate web server (like
> Apache). So, other users of the program would run it on their own
> server, and have their own database of everything on their server's
> filesystem.

Great. The remaining mid-term issue is as follows. Right now you own the database, which entails some fiduciary responsibilities. Think "unexpected": if e.g. you get hit by a bus (God forbid), the database is virtually lost. This scenario is solved by your granting e.g. Walter access to the server, so he can copy the database and possibly migrate the installation.

But then say you and Walter have a huge disagreement over a core issue, which makes you furious. Then you could revoke Walter's login and/or destroy the database. And so on.

Also, privacy is not a biggie right now because the database contains public information plus some simple preferences, but later you could store additional information of various degrees of sensitivity. So you would in fact responsible for users' privacy, but d-programming-language.org is the ostensible responsible.

How do you think we could solve this issue?


Andrei

On 12/9/2011 5:58 AM, Vladimir Panteleev wrote:
> This is a complicated issue - mainly due to the possibility of abuse. Reddit has
> complicated heuristic algorithms for detecting misbehavior and balancing votes,
> built up from years of experience. Even though their code is open, it's probably
> not worth the effort. We could make little attempt to police votes, but then
> they would lose meaning as soon as someone bored enough comes along.

Much as I like the voting system, I agree that it may be not worth it due to dealing with abuse.

On 12/9/2011 5:58 AM, Vladimir Panteleev wrote:
> There are also some security considerations with attachments. We need to either
> have a separate subdomain for attachments, or forbid viewing them in the browser
> (which would mean that viewing non-whitelisted attachments, including HTML
> parts, would need to force a download). Gmail uses googleusercontent.com for
> attachments, for example.

I recommend ignoring attachments, including HTML ones. The current D archives ignores them, and it has never been an issue.

One reason is that with attachments people will tend to post their projects and bug reports to the n.g. rather than github (or equivalent) and bugzilla where they belong.

Another is to keep the posts readable by simple NNTP readers.

On 12/9/2011 7:39 AM, Vladimir Panteleev wrote:
> The short version is that HTML attachments may contain malicious JavaScript
> code, which will have access to all forum cookies. For example, it would be
> possible to write a worm which would repost itself whenever anyone opened the
> attachment.

I hadn't thought about that.

I strongly suggest ignoring all sections of posting which are not plain text.

On Friday, 9 December 2011 at 19:48:46 UTC, Walter Bright wrote:
> I hadn't thought about that.
>
> I strongly suggest ignoring all sections of posting which are not plain text.

What if someone with an NNTP client posts a question and attaches their D code?

If security is the sole problem with decoding attachments, it is easily mitigated with separate subdomains (or forced download). Whether it should be possible to post attachments is another question.

On 12/9/2011 11:51 AM, Vladimir Panteleev wrote:
> On Friday, 9 December 2011 at 19:48:46 UTC, Walter Bright wrote:
>> I hadn't thought about that.
>>
>> I strongly suggest ignoring all sections of posting which are not plain text.
>
> What if someone with an NNTP client posts a question and attaches their D code?

They can put it inline as text, the way it is done now.


> If security is the sole problem with decoding attachments, it is easily
> mitigated with separate subdomains (or forced download). Whether it should be
> possible to post attachments is another question.

It's not the sole problem, I mentioned a couple other in another post.

I reiterate that I strongly recommend ignoring attachments and html content. I think that trying to deal with them correctly and securely is a huge distraction & time sink with little added value.

On Friday, 9 December 2011 at 19:54:28 UTC, Walter Bright wrote:
> They can put it inline as text, the way it is done now.

The problem is that they won't know that some users won't be able to view it. I don't suppose you can configure the NNTP server to reject attachments with a sensible error message?

> I reiterate that I strongly recommend ignoring attachments and html content. I think that trying to deal with them correctly and securely is a huge distraction & time sink with little added value.

I think that's an exaggeration. The bulk of the work has already been done.

On 12/9/2011 11:58 AM, Vladimir Panteleev wrote:
> On Friday, 9 December 2011 at 19:54:28 UTC, Walter Bright wrote:
>> They can put it inline as text, the way it is done now.
>
> The problem is that they won't know that some users won't be able to view it. I
> don't suppose you can configure the NNTP server to reject attachments with a
> sensible error message?

I don't know if that can be done - but it is configured to reject posts that are above a certain size. This tends to discourage attachments.