August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to H. S. Teoh | On Friday, 23 August 2013 at 16:50:27 UTC, H. S. Teoh wrote:
[..]
> Frankly, the fact that line counts are used at all has already
> decremented the author's credibility for me.
I agree that LOC is a very poor measure, but I think the intent was to offer some sort of comparison of syntactic complexity or code bloat (think Haskell vs. Java). Counting lines or non white-space characters is a low hanging fruit - anything more sophisticated seems to be quite difficult.
More importantly, LOC is also used in relative mode to show the fraction of code, which had to be modified for parallelism.
|
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to H. S. Teoh | On Friday, 23 August 2013 at 16:50:27 UTC, H. S. Teoh wrote:
> Seriously, I don't understand what's with this obsession with line count metrics.
While LOC isn't a very good metric, you're complaining about things that aren't really there. Yes you can shorten it to 2 lines, but did he? It looks like he had consistent formatting style across the languages.
If we decided that 2 lines was how we do formatting, what is wrong with that? If it is harder to read (and I believe it is) then our code examples aren't going to attract people even though the line count is down.
All that said, I think the delta is a nice statement.
|
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | On Friday, 23 August 2013 at 19:17:01 UTC, Walter Bright wrote:
> On 8/23/2013 6:48 AM, bearophile wrote:
>> The author of the serial language comparison has now created a simple parallel
>> comparison:
>>
>> http://togototo.wordpress.com/2013/08/23/benchmarks-round-two-parallel-go-rust-d-scala-and-nimrod/
>
> And note how well D did in the speed tests!
It gets second no matter how you read it!
|
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jesse Phillips | On 8/23/2013 7:10 PM, Jesse Phillips wrote:
> If we decided that 2 lines was how we do formatting,
In general, I regard a "line of code" as one statement or one declaration. Comments don't count, nor does cramming 3 statements into one line make it one LOC.
Of course, you can still game that, but if you're reasonable then it is a reasonable measure.
|
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jesse Phillips | On Saturday, 24 August 2013 at 02:12:41 UTC, Jesse Phillips wrote:
> It gets second no matter how you read it!
It was also not very idiomatic. It looks like some performance improvements could be made.
|
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | On Fri, Aug 23, 2013 at 08:25:20PM -0700, Walter Bright wrote:
> On 8/23/2013 7:10 PM, Jesse Phillips wrote:
> >If we decided that 2 lines was how we do formatting,
>
> In general, I regard a "line of code" as one statement or one declaration. Comments don't count, nor does cramming 3 statements into one line make it one LOC.
>
> Of course, you can still game that, but if you're reasonable then it is a reasonable measure.
I am still skeptical of LOC as a reliable measure of language complexity. How many LOC does the following code have?
auto formatYear(int year, int monthsPerRow)
{
enum colSpacing = 1;
return
datesInYear(year)
.byMonth()
.chunks(monthsPerRow)
.map!(r =>
r.formatMonths()
.array()
.pasteBlocks(colSpacing)
.join("\n"))
.join("\n\n");
}
It's 14 lines using the above formatting, but the { and } are arguably discountable, which makes it 12 lines of code.
But by your metric of one statement / one declaration, this is only 3 lines of code. Or, if you count the lambda separately, 4 lines of code.
That's a variation by a factor of at least 3, which makes it an unreliable metric.
So what would you do? Declare that each function call should count as 1 LOC? How then would you count the LOC in the following code?
void main(string[] args) {
int x = max(2, min(5, args[1].to!int));
}
Is it 2 lines of code or 4 (since it's 3 function calls + 1 declaration of main)? Again, you have a variation by a factor of 2, which makes any results derived from LOC unreliable at best.
In none of the above examples did I try to deliberately game with the metric. But the metric is still pretty inaccurate, and requires subjective judgment calls.
A far more reliable measure of code complexity is to look at the compressed size of the source code (e.g., with zip), which is an approximation of the Kolgomorov complexity of the text, roughly equivalent to the amount of information content it has.
Say you're comparing two functionally equivalent programs P1 and P2 in two languages L1 and L2, respectively. Then the compressed size of P1 and P2, respectively, measure roughly how complex the source code must be in order to express the program. The smaller the compressed size, the more expressive the language, because it means that you only need to use a small amount of complexity in order to express the program.
Also interesting is the ratio of the uncompressed source code size to its compressed size: this ratio tells you how verbose the language is. The higher the ratio, the more verbose the language: roughly speaking, it requires more characters to represent a given amount of complexity. If this ratio is low, it means that the source code is pretty terse; it only requires a small amount of characters to express a given amount of complexity.
While the above is obviously still an approximate metric, it at least has some scientific basis in information theory, and I would put more trust in this than in LOC. Already, it tells you that there are two related but different factors at play: the inherent expressiveness of the language (the absolute size of the compressed source code of a program of fixed functionality, given a fixed compression algorithm), and the verbosity of its syntax (the ratio of the uncompressed source with the compressed source). It's the inherent expressiveness that's a better measure of a language's expressiveness, because it doesn't unfairly penalize a language for having, say, longer, more descriptive identifiers, but it does penalize a language for requiring more convoluted structures to express a given task. Verbosity may matter more to us human coders, especially those of us who like text editors, but it doesn't fairly measure the inherent expressiveness of the language.
LOC not only doesn't even begin to approximate Kolgomorov complexity, it conflates these two related but ultimately different characteristics of the language. Thus, any conclusions drawn from LOC are bound to be highly unreliable.
T
--
Designer clothes: how to cover less by paying more.
|
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to H. S. Teoh | On 8/23/2013 9:58 PM, H. S. Teoh wrote:
> On Fri, Aug 23, 2013 at 08:25:20PM -0700, Walter Bright wrote:
>> On 8/23/2013 7:10 PM, Jesse Phillips wrote:
>>> If we decided that 2 lines was how we do formatting,
>>
>> In general, I regard a "line of code" as one statement or one
>> declaration. Comments don't count, nor does cramming 3 statements into
>> one line make it one LOC.
>>
>> Of course, you can still game that, but if you're reasonable then it
>> is a reasonable measure.
>
> I am still skeptical of LOC as a reliable measure of language
> complexity. How many LOC does the following code have?
Like I said, you can still game it. I think some common sense applies, not a literal interpretation.
|
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
On Fri, Aug 23, 2013 at 09:58:11PM -0700, H. S. Teoh wrote: [...] > A far more reliable measure of code complexity is to look at the compressed size of the source code (e.g., with zip), which is an approximation of the Kolgomorov complexity of the text, roughly equivalent to the amount of information content it has. [...] Excuse me, I meant *Kolmogorov* complexity. :) T -- WINDOWS = Will Install Needless Data On Whole System -- CompuMan |
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | On Fri, Aug 23, 2013 at 10:13:56PM -0700, Walter Bright wrote: > On 8/23/2013 9:58 PM, H. S. Teoh wrote: > >On Fri, Aug 23, 2013 at 08:25:20PM -0700, Walter Bright wrote: > >>On 8/23/2013 7:10 PM, Jesse Phillips wrote: > >>>If we decided that 2 lines was how we do formatting, > >> > >>In general, I regard a "line of code" as one statement or one declaration. Comments don't count, nor does cramming 3 statements into one line make it one LOC. > >> > >>Of course, you can still game that, but if you're reasonable then it is a reasonable measure. > > > >I am still skeptical of LOC as a reliable measure of language complexity. How many LOC does the following code have? > > Like I said, you can still game it. I think some common sense applies, not a literal interpretation. You conveniently snipped the rest of my post, which postulates a far better metric that's no harder to apply in practice. :) Comparing the compressed size of the source code is far more reliable than LOC. The absolute compressed size gives you an approximation of the Kolmogorov complexity of the source, which approximates the expressiveness of the language if you fix the functionality of the program you're comparing. The ratio of the uncompressed size to the compressed size gives you an approximation of the verbosity of the language's syntax. All it takes is for you to run zip instead of wc -l, and you have a far better metric for measuring language expressiveness. Insisting on using LOC despite this just makes one lose one's credibility. T -- They say that "guns don't kill people, people kill people." Well I think the gun helps. If you just stood there and yelled BANG, I don't think you'd kill too many people. -- Eddie Izzard, Dressed to Kill |
August 24, 2013 Re: Parallel Rogue-like benchmark | ||||
---|---|---|---|---|
| ||||
Posted in reply to H. S. Teoh | On 8/23/2013 10:23 PM, H. S. Teoh wrote: >> Like I said, you can still game it. I think some common sense >> applies, not a literal interpretation. > You conveniently snipped the rest of my post, which postulates a far > better metric that's no harder to apply in practice. :) You can't compress by visually looking at the code, and LOC is a unit that people fairly intuitively understand. > All it takes is for you to run zip instead of wc -l, and you have a far > better metric for measuring language expressiveness. Insisting on using > LOC despite this just makes one lose one's credibility. LOC ain't that far off, if you use it with some common sense rather than literal dogma. |
Copyright © 1999-2021 by the D Language Foundation