An open automation system for predatory journal detection

Abstract

The growing number of online open-access journals promotes academic exchanges, but the prevalence of predatory journals is undermining the scholarly reporting process. Data collection, feature extraction, and model prediction are common steps in tools designed to distinguish between legitimate and predatory academic journals and publisher websites. The authors include them in their proposed academic journal predatory checking (AJPC) system based on machine learning methods. The AJPC data collection process extracts 833 blacklists and 1213 whitelists information from websites to be used for identifying words and phrases that might indicate the presence of predatory journals. Feature extraction is used to identify words and terms that help detect predatory websites, and the system's prediction stage uses eight classification algorithms to distinguish between potentially predatory and legitimate journals. We found that enhancing the classification efficiency of the bag of words model and TF-IDF algorithm with diff scores (a measure of differences in specific word frequencies between journals) can assist in identifying predatory journal feature words. Results from performance tests suggest that our system works as well as or better than those currently being used to identify suspect publishers and publications. The open system only provides reference results rather than absolute opinions and accepts user inquiries and feedback to update the system and optimize performance.

Figure 1

Our proposed academic journal predatory checking (AJPC) system identified the first journal, Antarctic Science, as legitimate, and the second, International Journal for Development of Computer Science and Technology, as potentially predatory. Similarities between the two websites are noted in the color box frames 1a was captured from https://www.cambridge.org/core/journals/antarctic-science# and 1b was captured from http://ijdcst.com/.

Figure 2

Examples of potentially misleading text in invitations sent to scholars to submit manuscripts.

Figure 3

Proposed academic journal predatory checking (AJPC) system architecture.

Figure 4

AJPC system preprocessing steps.

Figure 5

Legitimate and predatory journal query examples.

Figure 6

Legitimate and predatory journal query results returned by the AJPC system.

Figure 7

Legitimate and predatory journals’ queried results returned by AJPC system.

Figure 8

Relationship between K distance and error rate in KNN classifier algorithm.

Figure 9

Recall rate performance data for the eight classifiers examined in this study.

Figure 10

F1-score performance data for the eight classifiers examined in this study.