Fluid intelligence doesn't matter! effects of code examples on the usability of crypto APIs

Context: Programmers frequently look for the code of previouslysolved problems that they can adapt for their own problem. Despiteexisting example code on the web, on sites like Stack Overflow,cryptographic Application Programming Interfaces (APIs) are commonly misused. There is little known about what makes exampleshelpful for developers in using crypto APIs. Analogical problemsolving is a psychological theory that investigates how people useknown solutions to solve new problems. There is evidence that thecapacity to reason and solve novel problems a.k.a Fluid Intelligence(Gf ) and structurally and procedurally similar solutions supportproblem solving. Aim: Our goal is to understand whether similarityand Gf also have an effect in the context of using cryptographicAPIs with the help of code examples. Method: We conducted a controlled experiment with 76 student participants developing withor without procedurally similar examples, one of two Java cryptolibraries and measured the Gf of the participants as well as theeffect on usability (effectiveness, efficiency, satisfaction) and security bugs. Results: We observed a strong effect of code exampleswith a high procedural similarity on all dependent variables. Fluidintelligence Gf had no effect. It also made no difference whichlibrary the participants used. Conclusions: Example code must bemore highly similar to a concrete solution, not very abstract andgeneric to have a positive effect in a development task.