A quantified and multi-level scale of authenticity

Compass to Publish uses a criteria-based evaluation to quantify the degree of authenticity of open access journals requiring or hiding article processing charges (APCs).

The tested journal's degree of authenticity is displayed dynamically on a quantified scale ranging from -20 (extremely predatory) to +20 (no suspicion of deceptive behavior or fake character). A journal's score can exceed the scale's boundaries but for practical reasons and ease of use, the actual score is not displayed beyond those boundaries.

The quantified scale comprises 7 shades of color, which are meant to categorize and explain the potential fake and/or deceptive character of tested journals:

Dark red	Your answers indicate that the journal is very likely to be predatory. It is best to avoid it altogether.
Medium red	This journal is highly suspicious. There are many signs indicating it may be a predatory journal.
Light red	This journal looks risky.
Yellow	Your answers make it hard to determine whether or not this may be a predatory journal.
Light green	The risk that this may be a predatory journal is fairly limited.
Medium green	This is most likely not a predatory journal.
Dark green	Rest assured! Everything seems to indicate that this journal is very unlikely to be predatory.

These shades of color are meant to provide a more nuanced picture of the predatory publishing landscape than the binary logic sustained by trusted lists and databases of predatory journals.

What are the criteria?

Compass to Publish uses an evaluation method based on 26 criteria which take the form of questions. These criteria and questions are the result of the critical and analytical work of the team behind Compass to Publish, who have examined the practices of a significant number of predatory journals and publishers. This examination was then followed by a qualitative survey and selection of criteria developed by trusted lists and directories, as well as checklists for the identification of predatory journals, including:

• the "Transparency and best practice" checklist of the Directory of Open Access Journals (DOAJ) and the basic criteria for inclusion in the DOAJ
  
• the list of criteria for the identification of predatory journals developed by Eriksson & Helgesson (2016)
• the v.1.1 criteria version for the identification of predatory journals developed by Cabells (a for-profit company)

Looking at the full range of these criteria, we only retained those that are:

• truly incriminating and easy to check to ensure user-friendliness;
• sufficiently relevant and clear;
  
• easy to use and check for users.

Some information regarding journal policies and procedures can be very hard and/or time-consuming to verify. We deliberately decided not to include this type of criteria in the evaluation process in an effort to ensure user-friendliness.

Weighing method

Every criterion can generate positive or negative points, following the answer given. The weighing of criteria may vary according to:

• their degree of severity (severe, moderate, ...)
  
• their subjective or objetive character
  

Details of points and criteria may be found below in the section "Criteria and Scoring".

Additional penalty points

Depending on the answers given, additional penalty points may apply to certain questions that are part of a specific and pre-determined subgroup of questions.

This pre-determined subgroup only contains questions whose answers may testify to clear misleading behaviors and/or a willingness to deceive. These specific questions are marked with a in the "Criteria and Scoring" section below.

Additional penalty points are applied if the test registers at least two answers to questions of this subgroup which indicate deceptive and/or miseleading behaviors. 

These additional penalty points are calculated as follows:

Condition	Additional penalty points
If  2 registered answers to questions included in the subgroup indicate misleading and/or deceptive behaviors;	-2
If  3 registered answers to questions included in the subgroup indicate misleading and/or deceptive behaviors;	-4
If  4 registered answers to questions included in the subgroup indicate misleading and/or a deceptive behaviors;	-6
If  5 registered answers to questions included in the subgroup indicate misleading and/or deceptive behaviors;	-8
If  6 registered anwsers to questions included in the subgroup indicate misleading and/or deceptive behaviors;	-10

These additional penalty points are meant to:

• make sure that truly predatory journals stand out
  
• put into perspective the potential inclusion of journals in trusted lists AND in databases of alleged predatory journals

Criteria and scoring

1. Trusted lists

2. Lists of alleged predatory journals and publishers (often referred to as blacklists)

3. Hijacked journals

4. Indexing and metrics

5. Editorial board and peer review

6. Content and presentation

7. Communication strategies