Ten-year challenge: Reproduction of An allosteric model of calmodulin [PNAS 105(31), 10768 (2008)]

[MelanieIStefan]

Original article: M.I. Stefan, S.J. Edelstein, N. Le Novère (2008). An allosteric model of calmodulin explains differential activation of PP2B and CaMKII. PNAS 105(31), 10768.

PDF URL: https://github.com/MelanieIStefan/calmodulin_reproduction/blob/master/ReScience_Calmodulin.pdf
Metadata URL:
Code URL: https://github.com/MelanieIStefan/calmodulin_reproduction

Scientific domain: Systems neuroscience
Programming language: SBML, R
Suggested editor: This is part of the 10-year challenge, so should be handled by the challenge organisers

[rougier]

Thanks for your submission, I will edit it.

@apdavison Could you review this submission for the Ten Years Reproducibility Challenge?

[apdavison]

yes, I can review it

[rougier]

@apdavison Wonderful, thanks. No hard deadline for the review but the sooner the better...

[rougier]

@apdavison Gentle reminder

[apdavison]

Hi @MelanieIStefan I'm struggling a bit with this, as I haven't used Copasi before. I've got as far as loading "Figure3.cps" and running a parameter scan, but I can't figure out how to save the results (i.e. to try to recreate Figure3.txt). Please can you help me out with a detailed description of what to click on/select?

[MelanieIStefan]

Hi @apdavison - when you run the parameter scan, a separate window should be created by Copasi where the results are plotted. In that window, you can select "Save Data" from the menu above the plot.

This should bring up a menu allowing you to save the data as a .txt file. Please let me know if this doesn't work.

[apdavison]

This is a well-written article, which serves as an excellent example of why reproducibility of modelling studies requires not only the model specification but also the simulation configuration and data analysis scripts to be stored.

I was able to re-run the simulations, and reproduce the data files in the repository.

Major issues:

• the curves in the manuscript differ substantially from those in the corrections to the original manuscript. The following two graphs compare the curves digitised from the corrected Figure 3 and 4 in Stefan et al. 2008 with the data from the reproduction.

On the basis of this, I don't think that the original parameters have been correctly recovered. I recommend an attempt is made to obtain a closer fit to the original curves (perhaps using automated parameter fitting). This might also result in a successful reproduction of Figure 5. Otherwise, I think this has to be labelled an unsuccessful reproduction and the manuscript amended accordingly.

Minor issues:

• IUPAC recommendations are that the names of elements should not be capitalised, so "Calcium" should be replaced by "calcium" (as in the original article that is being reproduced).
• "an attempt to reproduce the code" should be something like "an attempt to reproduce the results using the original code"
• "Calcium frequencies" - it is not clear what is meant here. Concentrations?

[MelanieIStefan]

Thank you for your thoughtful comments!

[rougier]

@apdavison Thanks for the detailed review.
@MelanieIStefan Did you have time to address the comments ?

[rougier]

@MelanieIStefan Sorry for the delay. Did you have time to take comments into account?

[rougier]

@MelanieIStefan Should we close this submission?

[MelanieIStefan]

Hello! Please don't close it just yet. I know it's been a while, but I would like a chance to figure this out please.

[rougier]

@MelanieIStefan Any update (after 1.5 years :))