diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 5d227a5..1a0de2e 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -14,6 +14,6 @@ jobs: - uses: actions/checkout@v2 - uses: ruby/setup-ruby@v1 with: - ruby-version: 2.6 + ruby-version: 3.3 bundler-cache: true # runs 'bundle install' and caches installed gems automatically - run: bundle exec jekyll build diff --git a/_author_instructions/06_lessons_learned.md b/_author_instructions/06_lessons_learned.md index 10d7888..4d79281 100644 --- a/_author_instructions/06_lessons_learned.md +++ b/_author_instructions/06_lessons_learned.md @@ -3,30 +3,44 @@ layout: single sidebar: nav: authors_lessons_learned.md title: "Lessons Learned" -excerpt: What is a "lesson learned" (i.e. negative results) document? +excerpt: What are "lesson learned" and reproducibility documents? permalink: /authors/lessons_learned/ --- -## What is a "lesson learned" (i.e., negative results) document? +## What are "lesson learned" and reproducibility documents? +This article category will now include two general classes of article types: negative results and reproducibilty studies. These are defined in more detail below, but any article in the category must be of value to a substantial subset of the community. + +First, we address "lessons learned". Research inherently involves failure, but without documentation of negative results, investigators are doomed to repeat prior mistakes. The "Lessons Learned" category will address this gap in the literature in a topic-based way - e.g., "Lessons Learned in Potential of Mean Force Computation." Each article should be broadly useful, with the potential to aid in the planning of a significant number of studies. Thus, results obtained in multiple systems and different conditions are desirable. Authors will be encouraged to add to an existing topic when possible and appropriate (and receive formal authorship credit, as dicsussed elsewhere). +Reproducibility studies are the second article type in the category. +Such a study examines a published method or published application of a method, attempting to reproduce or correct the published material. +Acceptable studies may examine solely the physics or mathematics behind a method, the implementation of that method in a public code, or the application of a method to a particular system. +In the last case, a prior application may have correctly applied a correct method but without sufficient quality controls regarding sampling, for example. +Reproducibility studies may include both positive and/or negative results, i.e., conflicting with or confirming prior findings. +If the results confirm prior findings, authors should provide a compelling argument for why the confirmation is of value to the community and worth publishing. +The complexity or novelty of a method or system might provide such a motivation. + ## Manuscript preparation instructions -Quite simply, a "Lessons Learned" manuscript is a regular scientific article and hence should be prepared with the standard high level of care and documentation. +Quite simply, a "Lessons Learned" or "Reproducibility" manuscript is a regular scientific article and hence should be prepared with the standard high level of care and documentation. The article should contain the usual sections: Introduction, including background, motivation and a brief discussion of related work; Methods/Theory, with all information required for a reader to reproduce the results, including software version numbers and links to any in-house files or scripts; Results; Discussion, highlighting the lessons learned and tips for avoiding future mistakes; and Conclusions. Data analysis should conform to [best practices](https://github.com/dmzuckerman/Sampling-Uncertainty). +In cases where a reproducibility study conflicts with prior findings, authors are strongly encouraged to work with a LiveCoMS editor to engage authors of the original study to collaborate in clarifying problems or misunderstandings. The editors will make every effort to engage all parties diplomatically to yield a consensus joint paper. If a mutual effort proves impossible, the original authors will be invited to write a rebuttal. + ## Additional critera considered in the review of lessons learned articles -Reviewers will consider the following factors in reviewing "lessons learned" document. +Reviewers will consider the following factors in reviewing "lessons learned" or "reproducibility" document. * Is the motivation for the approach clearly described and reasonable given existing literature? * Are the "lessons" provided likely to be of value to a significant number of researchers, based on demonstrated broad applicability? * Is the procedure used fully described (and derived mathematically if appropriate) to enable a reader to reproduce the results? * Are any needed supported files provided? * Has data analysis been performed in conformance with best practices? +* For a reproducibility document, has the prior work been described fairly given publicly available material? ## Revision schedule for lessons learned documents * Once published, unless subsequently there are recognized flaws with or important updates to an already-published negative-result discussion, such article or article section may not require updating.