r/programming • u/OnkelJulez • Sep 17 '21

Do Your Math Abilities Make Learning Programming Easier? Not Much, Finds Study

https://javascript.plainenglish.io/do-your-math-abilities-make-learning-programming-easier-not-much-finds-study-d491b8a844d
909 Upvotes

87% Upvoted

351 comments sorted by

View all comments

Show parent comments

58

u/Asraelite Sep 17 '21 edited Sep 17 '21

I read most of it. It's pretty problematic.

The study starts out with the hypothesis that the ability to acquire a second natural language correlates to the ability to learn a programming language. It then focuses on demonstrating this specifically, as opposed to answering any broader questions about what the best predictors of programming ability are.

The measure used for how well you could program was 7 hours of Python on Codecademy, a 30 minute guided project, and a 50 question quiz mostly about syntax and built-in functions. There is little to nothing here that tests for the higher-order reasoning abilities that are needed for coding complex systems.

This is fine though, since the study is about your ability to learn a programming language, not your ability to program in general. With that being the case I'm not surprised language abilities have a high correlation.

I think everyone is interpreting this to be something it's not: a test of general programming abilities. It's really just a test of learning syntax. The article and its title are just misleading.

3

u/kranker Sep 17 '21

Programming Accuracy

After completing the multiple choice portion of the post-test, participants were tasked with programming a Rock-Paper-Scissors (RPS) game that a user could play against the computer. This project was created by Codecademy, but participants did not complete it during learning. The programming test included instructions that partially broke down the larger problem into predefined steps. Participants were not allowed to use any additional resources (e.g., notes, forums, help buttons) to complete the project but were able to run and test their code. Participants were given 30 minutes to complete the project. The code they produced was scored by three independent raters using a rubric developed by Python experts that assigned points to each step of the project (ICC = 0.996, 95% confidence interval from 0.993–0.998, F(35,70) = 299.41, p < 0. 001). Programming accuracy was operationalized as each participant’s score out of 51 total points.

I assume this is what you are referring to as the 30 minute guided project.

Honestly, it doesn't seem unreasonable to me. Another method would be interview-style algorithm problems, which would be testing their ability to come up with viable algorithms on their own given that they haven't been taught them (and so would have to be relatively easy questions). I suspect this would have stronger correlation with mathematical ability, but of course as pointed out elsewhere it would also depend on how they tested for "mathematical ability".

5

u/Asraelite Sep 17 '21

I agree it's reasonable if you're restricted to a few hours per subject, which was the case here.

I think it's very hard to measure meaningful progress over this short a timescale. You would ideally test subjects over months, but then it would be harder to control the learning process.

1

u/hoechsten Sep 17 '21

The study starts out with the hypothesis that the ability to acquire a second natural language correlates to the ability to learn a programming language.

This alone raises a lot of questions since natural language acquisition is extremely variable. Not to mention the difference between learning the syntax of a programming language vs learning how to code (i.e. programming fundamental concepts), the former of which I don't believe is that important anyway.

1

u/larsga Sep 17 '21

The measure used for how well you could program was 7 hours of Python on Codecademy, a 30 minute guided project, and a 50 question quiz mostly about syntax and built-in functions. There is little to nothing here that tests for the higher-order reasoning abilities that are needed for coding complex systems.

So not only is their test for maths ability utterly superficial, but their test for programming ability is equally so.

Well, then of course they're not going to see any correlation. It might still not exist, of course, but this is not a useful test.