On Fri, Jun 21, 2013 at 11:59:12AM +0200, Jacob Carlborg wrote:
> On 2013-06-21 07:57, H. S. Teoh wrote:
> 
> >we got to ask HR to first administer a technical test before any interviews are arranged; test results are reviewed before deciding to interview the candidate
> 
> I done tests like that, they all suck. This is how it usually works:
> 
> You get a problem to solve. They say: "solve it anyway you want, using any language you want".

Heh, at my job, we're looking for skill in specific languages, so candidates are explicitly asked to write C code. And this is done not in an interview setting (which puts too much pressure on the candidate -- can *you* write code in front of interviewers without panicking at least a bit?), but in a classroom setting. Though, granted, when *I* had to go through the procedure, it felt like I was taking my CS final exams all over again. Not exactly a pleasant experience. :-P


> You solve it farily quickly and straight forward. Then they say, "you are not allowed to use that function". Basically they're saying it's cheating. Then you remove that function and change the implementation accordingly. Then they say, "you are not allowed to use that other function". You change the code accordingly and this dance continues. Then you're thinking to yourself, "Am I supposed to reimplement the standard library?".

That's the sign of a poorly-chosen test question. The kind of question I like to use doesn't focus on nitpicky details like that, but on actual algorithms or domain-specific knowledge, like showing that the candidate understands bit operations, pointer arithmetic, and stuff like that. On that note, it's rather shocking how few candidates know how to use malloc() and free() correctly...

That's not to say the tests my employer gives have only "good" questions, though; there are some that basically do what you describe: "please re-implement strstr(), please concatenate strings without using strcat()", and the like. When I was involved in hiring, I usually ignore these questions when evaluating the candidate, or cut them a lot of slack.

But I do have a slightly evil streak when it comes to technical tests... once I wrote a Javascript question that requires the candidate to evaluate what a piece of code does, that involves several nested closures and variable shadowing. We hired one of the (very) few candidates who actually got that question right, and she turned out to be an excellent coder.


> I would say, most times, it's almost irresponsible to _not_ use the standard library. A bunch of people, a lot smarter than I, have had a good 20 years or so to perfect and fine tune the standard library. Not a chance in hell that I'm going to beat that, on an interview, with code written on a whiteboard.

Well, I don't think the point of the test is to prove that you can write code that's *superior* to the standard library, but that you *can* write non-trivial functionality from ground up.


> A side note. You're of curse not allowed to look at any documentation at all and you're not allowed to use a text editor.  You write the code on a whiteboard, yes a _whiteboard_.

Well, that's a new one. :-P  Our tests are administered as pen-and-paper tests. Not unlike CS final exams, unfortunately. :-/


> That's just so stupid. That's not how programming works in the real world. Why the hell do they think IDE's have built in, easy to read/find, documentation.
[...]

Actually, we instruct HR to provide the standard C library manuals if candidates ask for it. There *are* some unfair questions that expect you to know, e.g., the exact order of arguments to select() and stuff like that, but like I said, it's not HR who evaluates the tests, but the hiring managers, so it's up to our discretion how much weight we put on questions like that. We usually disregard minor mistakes like using the wrong order of arguments to qsort(), if the intent is clear. We don't (or at least I don't) just blindly look for the highest-scoring test and hire those candidates; I usually take the time to look at the code and ask, "why did they write it that way?  what does that show about their coding abilities?".


T

-- 
I see that you JS got Bach.

On 2013-06-21 17:14, H. S. Teoh wrote:

> Heh, at my job, we're looking for skill in specific languages, so
> candidates are explicitly asked to write C code. And this is done not in
> an interview setting (which puts too much pressure on the candidate --
> can *you* write code in front of interviewers without panicking at least
> a bit?), but in a classroom setting. Though, granted, when *I* had to go
> through the procedure, it felt like I was taking my CS final exams all
> over again. Not exactly a pleasant experience. :-P

I had four, one hour each, interviews in one afternoon. Afterwards my brain was basically fried.

> That's the sign of a poorly-chosen test question. The kind of question I
> like to use doesn't focus on nitpicky details like that, but on actual
> algorithms or domain-specific knowledge, like showing that the candidate
> understands bit operations, pointer arithmetic, and stuff like that. On
> that note, it's rather shocking how few candidates know how to use
> malloc() and free() correctly...

I guess I managed to avoid what they actually wanted to test.

> That's not to say the tests my employer gives have only "good"
> questions, though; there are some that basically do what you describe:
> "please re-implement strstr(), please concatenate strings without using
> strcat()", and the like. When I was involved in hiring, I usually ignore
> these questions when evaluating the candidate, or cut them a lot of
> slack.

I can also understand that it not that easy to create meaningful tests like these.

> But I do have a slightly evil streak when it comes to technical tests...
> once I wrote a Javascript question that requires the candidate to
> evaluate what a piece of code does, that involves several nested
> closures and variable shadowing. We hired one of the (very) few
> candidates who actually got that question right, and she turned out to
> be an excellent coder.

Hehe.

> Well, I don't think the point of the test is to prove that you can write
> code that's *superior* to the standard library, but that you *can* write
> non-trivial functionality from ground up.

Yeah, I guess it's hard to find meaningful tests.

> Well, that's a new one. :-P  Our tests are administered as pen-and-paper
> tests. Not unlike CS final exams, unfortunately. :-/

I hate pen-and-paper tests too. I remember those from the university. I understand that they don't want you to have access to a computer with access to the internet or a compiler. But come on, can I at least get a simple text editor. Half of the time I spend on erasing code because I cannot insert new code in the middle of existing code. Then also writing those annoying curly braces (the actual symbols) which I never get right.

> Actually, we instruct HR to provide the standard C library manuals if
> candidates ask for it.

That's generous.

> There *are* some unfair questions that expect you
> to know, e.g., the exact order of arguments to select() and stuff like
> that, but like I said, it's not HR who evaluates the tests, but the
> hiring managers, so it's up to our discretion how much weight we put on
> questions like that. We usually disregard minor mistakes like using the
> wrong order of arguments to qsort(), if the intent is clear. We don't
> (or at least I don't) just blindly look for the highest-scoring test and
> hire those candidates; I usually take the time to look at the code and
> ask, "why did they write it that way?  what does that show about their
> coding abilities?".

I guess we have different experiences.

-- 
/Jacob Carlborg

On 2013-06-21 16:56, H. S. Teoh wrote:

> +1, me too! I can say that 85-90% of what I do at work today, I learned
> from my personal coding projects, not from the CS courses I took in
> university. (That's why I like to joke about CS grads knowing more about
> uncomputable problems than computable ones...)

It feels like there's something wrong with the world here :)

-- 
/Jacob Carlborg

On Fri, Jun 21, 2013 at 05:48:25PM +0200, Jacob Carlborg wrote:
> On 2013-06-21 16:56, H. S. Teoh wrote:
> 
> >+1, me too! I can say that 85-90% of what I do at work today, I learned from my personal coding projects, not from the CS courses I took in university. (That's why I like to joke about CS grads knowing more about uncomputable problems than computable ones...)
> 
> It feels like there's something wrong with the world here :)
[...]

Only "something" wrong? ;-)

To be fair, though, I can understand why university programs are that way: their goal is to produce more researchers and professors who may join the faculty and produce more research. So they focus more on the theoretical aspects of computer science in order to produce such candidates, whereas focusing on more practical aspects is something for technical colleges whose goal is to produce industry workers.

Unfortunately, in North America at least, the technical colleges lean more toward stuff like "how to use MS Word", "how to organize your company data in MS SQL", rather than _real_ programming, and universities that actually *teach* real programming are more interested in finding solutions to uncomputable problems than teaching students how to solve computable ones, so there's a gap in the area of producing qualified industry coders who can write functional software. There *are* pockets of competent programming education here and there, of course, but this is the impression I get from the general situation.


T

-- 
If you think you are too small to make a difference, try sleeping in a closed room with a mosquito. -- Jan van Steenbergen

On 6/21/2013 3:04 AM, Jacob Carlborg wrote:
> But I do some researcher about TDD and similar techniques, use
> my common sense, pick some pieces from here and there and use what I think works.

I've been around long enough to have seen an endless parade of magic new techniques du jour, most of which purport to remove the necessity of thought about your programming problem.

In the end they wind up contributing one or two pieces to the collective wisdom, and fade away in the rearview mirror.

But there is one technique that stands head and shoulders above the others in improving code quality, sometimes dramatically so. That's unit testing coupled with a coverage analyzer.

I can't even count the number of times I thought "this is so simple, I couldn't have gotten it wrong, I doan need no steeenkin unit tests." But I write a couple anyway, and you can guess the rest.

Whether you write the unit tests before, during, or after the module is written is irrelevant.

On 6/20/2013 8:50 PM, H. S. Teoh wrote:
> One of my previous supervisors told me that when he gets resumés, as
> soon as he sees "Ph.D" he chucks it straight into the trash.

He'd have missed out on Andrei, then.

On Friday, 21 June 2013 at 19:14:18 UTC, Walter Bright wrote:
> On 6/20/2013 8:50 PM, H. S. Teoh wrote:
>> One of my previous supervisors told me that when he gets resumés, as
>> soon as he sees "Ph.D" he chucks it straight into the trash.
>
> He'd have missed out on Andrei, then.

And a lot of other people who are driven towards new ideas.

There's a fair amount of inverted snobbery about academia here.

Ultimately, a PhD shows the ability to conduct original research and present it. It doesn't make you a great programmer, but then again *it never purports to*. Nor does a computer science degree.

If you want a normal programming job, you need to show more real world experience than a PhD, but just throwing out people who have proved their originality and in depth understanding of a topic through a PhD is nothing short of absurd.

On 6/21/2013 12:49 PM, John Colvin wrote:
> If you want a normal programming job, you need to show more real world
> experience than a PhD, but just throwing out people who have proved their
> originality and in depth understanding of a topic through a PhD is nothing short
> of absurd.

You need a mix of people - young and old, newbies and old hands, academics and mechanics, detail guys and big picture guys, to have a healthy dev organization.

On Fri, 21 Jun 2013 21:11:51 +0200, Walter Bright <newshound2@digitalmars.com> wrote:

> Whether you write the unit tests before, during, or after the module is written is irrelevant.

I mostly agree with this, but there is one reason to write tests before
implementing - you don't know the code yet. This way you don't filter out
stuff that you think is unnecessary because you know the code handles it.

Of course, the converse is also true - write tests after writing the code,
because you then know the code and have a better idea of what the corner
cases are.

-- 
Simen

On Fri, Jun 21, 2013 at 09:49:21PM +0200, John Colvin wrote:
> On Friday, 21 June 2013 at 19:14:18 UTC, Walter Bright wrote:
> >On 6/20/2013 8:50 PM, H. S. Teoh wrote:
> >>One of my previous supervisors told me that when he gets resumés, as soon as he sees "Ph.D" he chucks it straight into the trash.
> >
> >He'd have missed out on Andrei, then.
> 
> And a lot of other people who are driven towards new ideas.
> 
> There's a fair amount of inverted snobbery about academia here.
> 
> Ultimately, a PhD shows the ability to conduct original research and present it. It doesn't make you a great programmer, but then again *it never purports to*. Nor does a computer science degree.

According to my ex-supervisor (and I'm not saying I agree with him), it indicates that one is opinionated enough to originate ideas, and stubborn enough to successfully defend said ideas, which can be detrimental in a team setting if the idea wasn't a good one.  (And since a PhD doesn't purport to make you a great programmer, and he was looking for great programmers rather than researchers, that could be a reason for his views on the matter.)


> If you want a normal programming job, you need to show more real world experience than a PhD, but just throwing out people who have proved their originality and in depth understanding of a topic through a PhD is nothing short of absurd.

Well, my ex-supervisor *did* have a reputation of having many "interesting" (i.e. extreme) ideas about many things. :) I can't say I subscribe to his views on this matter, but what I was trying to get at was the prevalent fallacious fixation on academic achievement (i.e. equating "he has good grades / a degree / a PhD" with "he is a good programmer").  Too many potential employers can't see beyond that, or are not willing / don't have the time and energy to do so, thus resulting in the situation where people are being hired because of their academic achievement, but are expected to have skills not necessarily implied by said achievement. Then when such hires consistently produce sub-par work, some people get provoked to equate "PhD" with "poor programming skills".

This situation wouldn't have developed if employers evaluated candidates based on their *skills* rather than by the credentials on paper.  But in this day and age where time is never enough, it's all too convenient to dismiss a candidate because he has no academic credentials rather than to spend the time / energy to review unlikely candidates on the off-chance that perhaps they might turn out to be hidden programming prodigies. Or to blindly hire a candidate *with* said credentials because someone of high academic standing is likely to be skillful enough to do the job, rather than to spend to time / energy to check if this is actually the case. (Not to mention that all too often, the ones with hiring powers may not necessarily have the ability to discern real skills from a smooth talker.)


T

-- 
"You are a very disagreeable person." "NO."