Review of the revised article “Not fleeting but lasting: Limited influence of aging on implicit adaptative motor learning and its short-term retention” by Pauline Hermans, Koen Vandevoorde, and Jean-Jacques Orban de Xivry, submitted to PCI Health and Movement Science.
The authors have provided a nice revision of their paper. Again, I think this study is very well done, the findings are interesting and make a nice contribution to the literature, and the authors have adequately addressed the majority of my questions. I only have one more remaining constructive comment below:
I appreciate the authors’ making an effort to emphasize that this is a conceptual replication. I also share their view that such conceptual replications are important, despite the fact that such studies have limitations/weaknesses. I also appreciate the addition of a paragraph in the discussion to further emphasize the conceptual replication point. However, it is worth considering the claim that is made at the end of this paragraph. It assumes that an age difference in spontaneous recovery is a finding that is “likely dependent on the experimental conditions.” By that same logic, the finding of a lack of age difference in spontaneous recovery is likely dependent on experimental conditions. I argue that a better statement would emphasize the need for further research to understand the conditions under which there is, and is not an age difference in spontaneous recovery, and to further explore the implications for those observations for the view that spontaneous recovery specifically reflects implicit learning/memory processes.
In my review of the previous version of this manuscript, I expressed my concerns regarding rationale behind the work, the specificity of the hypotheses, the description of the experimental task, and some of the analysis. These concerns have been resolved in the revised version of the manuscript and the revisions have helped to improve the quality and clarity of the work. I have no remaining comments.
DOI or URL of the preprint: https://www.biorxiv.org/content/10.1101/2023.08.30.555501v1
Version of the preprint: 1
Thank you for your submission titled “Not fleeting but lasting: Limited influence of aging on implicit adaptative motor learning and its short-term retention”. The paper addresses the question of whether aging affects the implicit component of learning . In my own reading of the article (and echoed by the comments from the two reviewers), this a very well-done study with several strengths– the paper addresses an important question and uses a well-established approach to addressing the question. In addition, the methods and analysis are also rigorously done, and presentation of the results is transparent (with robustness checks also being provided).
However, the reviewers have raised a few concerns that need addressing before I can provide a recommendation. As you will see, both reviewers have provided thoughtful reviews and suggestions – so I hope the authors will address these concerns. Both reviewers raise important theoretical and methodological clarifications that will help improve the clarity of the manuscript. I will simply highlight what I see as a main concern here (mentioned by Reviewer 2).
The main concern is the framing of the paper as a conceptual replication. As highlighted in several discussions on the replication crisis (notably by Chambers in “The 7 Deadly Sins of Psychology”), unlike a direct replication, the idea of a “conceptual” replication is subjective and often difficult to get agreement on since it is difficult to determine apriori if the parameters that were changed from the original study are critical to the original result (While the arguments against conceptual replication are typically used when a conceptual replication ‘agrees’ with the original result, it is probably fair to say that they also apply when the conceptual replication ‘fails’, as in this case). Thus, while the current study seems to a fair and rigorous test of the original question (with a section in the Discussion highlighting differences from the original study), the framing at several points in the paper (from the Abstract to the conclusion) often blurs this difference between a conceptual and a direct replication (e.g., “We failed to replicate…”). This could potentially be misleading as it could lead readers to think that this was a direct replication failure.
I would therefore suggest that ‘conceptual replication’ framing needs some re-consideration. One possibility is that the authors could choose to address this as a study of the research question on its own (while of course still comparing the results from the original study). However, if the authors want to retain this framing of a conceptual replication, it might help to be more transparent right from the outset that conceptual replications have some weaknesses (and make sure that the terms such as ‘failed to replicate’ are not used ambiguously in the manuscript)
A minor concern is that the data file in the repository seems to be one big MATLAB file which can only be used with the author’s code as far as I could tell (but this code is provided). However, it would be helpful to have the deposited dataset be “independently readable” (i.e., a datafile accompanied by a codebook that explains what these variables are and how they are organized) so that a potential reader could independently use the dataset without having to rely on the author’s code.
The manuscript presents a comprehensive report on a behavioral study that investigates the effect of age on the contribution of implicit processes to motor adaptation and spontaneous recovery. The introduction presents relevant background information and the data-analysis approach via both frequentist and Bayesian models is an asset to the work. I also appreciate that the authors present individual data points in the figures and I believe that the discussion offers a sound, integrative interpretation of the findings. At the same time, I have several requests/questions related to clarification of the rationale behind the work, the specificity of the hypotheses, the description of the experimental task, and some of the analysis choices. I will provide more detailed comments below, which I hope will be helpful towards revising the work.