A peer review is a critical read of your resource from another subject matter expert, useful to identify gaps and inconsistencies in your resource. ‘Peers’ can offer constructive feedback and solutions that will make your work even stronger. It’s also a chance to dispel the notion that “if it’s free, then it can’t be good,” because as we know — the open resources you are creating are certainly high-quality resources. For OER, review signals to prospective adopters that the work has passed through rigorous quality control, and that its content is suitable for use in the classroom. Not only does it give a public indicator of usefulness to potential adopters, but from experience we know that external reviewers in the field will very often end up adopting the text they’ve reviewed themselves, so it’s great ‘advertising.’ Having your work peer reviewed can also act as an additional stamp of quality for promotion and tenure committees when considering your work within the context of your dossier.

Peer review can occur before (pre) or after (post) publication. While post-publication review is common and accepted in open education (e.g., the Open Textbook Library’s very helpful public reviews), VIVA encourages its authors to undergo pre-publication review so that feedback from the reviewers can be integrated to improve the work before publication.

Peer review can also be anonymous, double anonymous, or open. Open peer review presents itself differently depending on the publication, from lack of anonymity to published reviews alongside the publication. Ross-Hellauer (2017) provides a good overview of these options.^[1]

VIVA is happy to offer peer review coordination–from querying readers to paying honorariums–for VIVA Publishing authors. This section provides an overview of what that process looks like and what decisions you may have to make along the way.

Considerations for peer review

Type of Review

One of the biggest considerations you’ll have to make it about the type of review: anonymous, double anonymous, or open.

Anonymous peer review may be the option for you if you want the reviewers to remain anonymous, but you don’t care if they know who you are. Those who choose anonymous peer review often think that it is a more rigorous review since peer reviewers may be more honest given the protection of anonymity. Anonymous peer review requires a third party (like VIVA) to administer the review.

Double anonymous peer review may be the best choice for you if you like anonymous review but don’t have any personal anecdotes and/or important identifying information in the manuscript. It requires an anonymous manuscript be submitted, and the authors name and affiliation will not be shared with reviewers. Double anonymous peer review is often used to maintain a level playing field in an attempt to eliminate bias from the reviewer based on the author’s identity.

Open peer review may work best for you if you’d like more transparency in the review process. Similar to those using anonymous peer review, those who use open peer review often think it is the most honest and collegial peer review because the identities are known and people can’t hide behind anonymity. If you’re interested in open peer review, you also need to consider what open peer review will look like for your manuscript. Will it mean that:

• there is no anonymity in the review process?
• the reviews will be published alongside the manuscript?
• the authors and reviewers will enter into a dialogue to help review and resolve issues for the manuscript?
• or something else?

VIVA does not currently have an open peer review policy, but would be happy to discuss this option with any authors who are interested.

Scope of Review

In addition to the type of review, you’ll have to decide what the reviewers will be reading. In traditional peer review, a reviewer is asked to review a full article or book manuscript. However, reviewers can also be asked to review individual or groups of chapters. The latter can be helpful if you’re worried about specialization or load of the reviewers.

When you’re deciding on the scope of review, consider:

• Are there chapters with a specific focus which might benefit from a close review by a subject matter expert?
• Do certain chapters have similar subjects and could be grouped for a review of multiple chapters by a single reviewer?
• What is the length of the chapters and book? What is a reasonable load for a single reviewer?
• It can be helpful to have at least one person (we recommend two) review the book if reviewing by chapter. This provides feedback on the book as a whole, including:
  • gaps in content
  • repetition between chapters
  • inconsistency in structure or tone across the book

VIVA is happy to discuss what would be the ideal peer review strategy for your work. In general, we recommend either full book peer reviewers (2-3) or a combination of book reviews and multi-chapter reviewers.

Selecting Reviewers

When talking about “peers” in this context, we are referring to another subject matter expert. It is important to emphasize that the term “peer” does not solely mean academics. Depending on the focus of the book, others–such as practitioners or community members–could provide important feedback on the subject. For example, a current k-12 teacher may be a good reviewer for a teacher prep course or a practicing CPA for an accounting course. Consider:

• Instructors who teach similar courses can provide feedback through the lens of similar textbooks and/or their experience working with students in the classroom, in addition to subject matter expertise
• Other academics with similar backgrounds/research focus can provide in-depth theoretical/practical knowledge
• Practitioners can provide expert feedback through the lens of applying the skills/knowledge to on the job experience

In addition to how to define “peer,” other considerations for who to select for peer reviewers include:

• Do you want feedback from your departmental colleagues?
  • This can often be easier to obtain, but will typically not allow for anonymity.
  • Many traditional publishing venues avoid using departmental colleagues or those who have previously published together to eliminate any potential bias.
• Do you want feedback from others in Virginia?
  • VIVA’s goal in supporting the creation of OER is to impact Virginia students and faculty. Getting feedback from Virginia educators can emphasize this focus and highlight potential future adopters.
    • VIVA maintains a Virginia-first reviewer selection system, where we prioritize other Virginia reviewers but do not limit ourselves to those in Virginia.
  • If your discipline is highly specialized, you may not have enough reviewers in Virginia.
  • Looking for reviewers outside of Virginia will also potentially increase the adoption of your resource post publication.

Please note that sometimes students are included in peer review process. We do not include them here, but they are certainly an option as a viable reviewer! Instead of focusing on expert review of content, students can provide feedback on items including:

• Reading level and content (e.g., can a non-expert understand? is it appropriate for the course level? are there any lingering questions/content gaps?)
• Activities (e.g., are they helpful? easy to complete?)
• Pedagogical structural elements (e.g., do they make sense? Are there enough? too many?)

If you do not include a student in the peer review process, we recommend beta-testing with a class to see how the book is received by students.

Honorarium

VIVA strongly supports thanking reviewers for their contribution to your through honorariums if you have the budget to do so.  [If you are applying for a VIVA Open Course Grant and would like VIVA to coordinate peer review on your behalf, do not include honorariums in your budget. This is part of the publishing services provided in-kind to grant recipients.]

That being said, there is no best practices for peer review honorarium compensation. A lot of peer review is completed without compensation. For those reviews that do offer honorarium, expectations can vary depending on the discipline. Honorarium amounts will also vary depending on the length of the review (i.e., short chapters, long chapters, multiple chapters, or whole book).

To balance budgets and compensation, VIVA uses the following framework for peer review honorariums:

• $400 to $500 for review of the whole book, depending on length
• $50/per chapter. This can be for individual chapters or multiplied when requesting review of multiple chapters.
  • When presenting to reviewers, offer the whole amount (e.g., if you are asking them to review 3 chapters, offer $150) but use this number as a framework to calculate the amount offered.

VIVA is happy to work with authors to determine the appropriate honorarium amount based on the length, content, and review needs of your book.

Timeline

You will also need to determine how long reviewers have to complete the peer review. This is the timeline from when they receive the initial inquiry or manuscript to when the review is due.

What’s the ideal timeline? We typically recommend:

• 2-4 weeks for a single chapter
• 4-6 weeks for multiple chapters
• 6-8 weeks for the whole book

Even with these guidelines, we have found that reviewers often complete their peer review at the last minute, rather than spreading the work out of the review period. Thus, the review timeline should be more reliant on your timeline. We suggest establishing a deadline when you would like the reviews in hand so you can begin your revisions. To the date, work backwards to create a rough timeline including:

• If working with VIVA, 1-3 weeks to get the process set-up with our external services provider and finalize all forms and questions
• 1-2 weeks (minimum) to secure reviewers. This may take longer if reviewers are hard to secure.
• 1-2 weeks after the due date for those who ask for extensions or who need reminders to complete and submit their work.
• You may also consider adding a buffer if any of these steps take longer or a review backs out mid-process and we have to secure a new reviewer.
• Review timeline for the reviewer (initial inquiry to due date)

Overall, we recommend budgeting a minimum of 3 months for the review process, although this can be longer or shorter depending on the review timeline. VIVA is happy to work with authors to establish a review process timeline and schedule.

When should I send the manuscript for peer review?

We recommend sending the manuscript for peer review once the content is complete. However, permissions (e.g., for copyrighted images) do not have to be obtained prior to peer review, especially since these may change after the review.

When reviewing a textbook, a big question for authors is whether to send the manuscript to peer review before or after beta-testing with students.

• Sending to subject matter experts prior to beta-testing with students may allow for a more finalized version to be presented to students for feedback.
• Sending to subject matter experts following beta-testing with students will incorporate any student feedback on content, format, or language into the version sent to peer review, allowing for a more finalized version to be reviewed for feedback
• Sending to subject matter experts at the same time as beta-testing with students allows both students and faculty to review the same version of the text and for authors to receive feedback from both sources simultaneously, including feedback on content and format/language

Due to the length of the publication process and the desire for maximum feedback, VIVA recommends sending to peer review at the same time as beta-testing. However, we support whatever method you would like to undertake.

Please note that sometimes students are included in peer review process. Here, we refer beta-testing as using a draft as a class’s texbook rather than involving students in a standard peer review process.

Peer Reviewer Guidance and Questions

Guidance for reviews

When someone agrees to peer review a VIVA Open Publishing work, they are sent the manuscript in Word or PDF, instructions for the review, and a review form. The instructions outline general guidelines for the review, including:

• VIVA provides information on the intended course topic and education level for the book, and reviewers are asked to keep this audience in mind when reviewing.
• Reviewers are not expected to copyedit or mark up the manuscript, although they are welcome to if they choose.
• Reviewers are encouraged to
  • provide examples to illustrate their comments, when able.
  • provide constructive criticism
  • consider language and tone in their feedback.

VIVA also asks reviewers to consider the following when undertaking their review, which is adapted from the Open Education Network’s Review Rubric:

• Comprehensiveness: The text covers all areas and ideas of the subject appropriately.
• Content Accuracy: Content is accurate, error-free and unbiased.
• Relevance Longevity: Content is up-to-date, but not in a way that will quickly make the text obsolete within a short period of time. The text is written and/or arranged in such a way that necessary updates will be relatively easy and straightforward to implement.
• Clarity: The text is written in lucid, accessible prose, and provides adequate context for any jargon/technical terminology used.
• Consistency: The text is internally consistent in terms of terminology and framework.
• Organization Structure Flow: The topics in the text are presented in a logical, clear fashion.
• Grammatical Errors: The text contains no grammatical errors.
• Cultural Relevance: The text is not culturally insensitive or offensive in any way. It should make use of examples that are inclusive of a variety of races, ethnicities, and backgrounds.

VIVA Peer Review Guidance:

• VIVA Peer Review_Book [Word]
• VIVA Peer Review_Chapter [Word]

Peer review Questions

VIVA’s standard peer review questions are intended to assess the quality of the book as well as its usefulness and appropriateness for use in the classroom. While these sample questions were designed for use in all VIVA books, VIVA staff will meet with authors prior to beginning the peer review process to determine if additional questions or adjustments to existing questions are needed to meet the needs of a specific project.

VIVA provides questions for both the full book and for chapters. While these are similar, they are designed with the understanding that chapters reviewers are not looking at the book as a whole.

VIVA-coordinated peer review

VIVA will coordinate peer review for VIVA Open Grant recipients in conjunction with external services providers. Coordination can include:

• Locating peer reviewers
• Querying peer reviewers
• Providing peer review forms and finalizing questions in consultation with authors
• Sending peer review forms
• Following-up with reviewers and ensuring reviews are submitted
• Collecting peer reviews
• Paying honorariums

If you are interested in VIVA coordinating peer review for your work, please reach out to VIVA Open Publishing.

Requirements

When VIVA coordinates peer review for a work, authors should submit:

• The manuscript via Word (preferred) or PDF
  • Images should be inserted in text. Final permissions do not have to be secured prior to review.
  • Send the full manuscript regardless of if the review is being conducted chapter by chapter or as a whole book
• A list of at least 3 suggested peer reviewers, or more depending on the number of reviews you want.
  • You do not have to query them ahead of time
  • If you are doing anonymous peer review, you should explicitly not contact them ahead of time
•  Scope of review (book vs chapters) and honorarium amount
  • Can be finalized through discussion with VIVA Publishing staff
• A description of the book, intended audience, and course topic.
  • To be used when querying peer reviewers.
• Any considerations to be passed along to peer reviewers.

VIVA will work with authors to finalize a list of peer review questions, based on the standard peer review questions and the needs of the project.