Distinguishing between Questionable,
Low Quality and Quality Indonesian
Open Access Journals
using DOAJ criteria and analytical tools
Tom Olijhoek DOAJ Editor-in-Chief tom@doaj.org
International Workshop for Indonesian Editors
March 25-27, 2019, Hotel Grand Mega, Kuta, Bali.
organized by HEBII (Indonesian Association of Scientific Journal Editors).
and KEMENTERIAN RISET, TEKNOLOGI, DAN PENDIDIKAN TINGGI
(Indonesian Ministry of Research, Technology, and Higher Education)

You are free to:
Copy, share, adapt, or re-mix;
Blog, live-blog, or post video of;
This presentation. Provided that:
You attribute the work to its author and respect the rights and licenses
associated with its components.
Slide Concept by Cameron Neylon, who has waived all copyright and related or neighbouring rights. This slide only ccZero.
Social Media Icons adapted with permission from originals by Christopher Ross. Original images are available under GPL at;
http://www.thisismyurl.com/free-downloads/15-free-speech-bubble-icons-for-popular-websites
Slides in this presentation by Tom Olyhoek, Judith Barnsby and Ina Smith

Part 1 the basics
• History DOAJ
• The Team, The Structure, The Workflow
• Journals: distribution countries languages
• Differences with Scopus Web of Science
• Quality of publishing versus content

Part 2 Criteria for entering DOAJ
• IMPORTANT: English is NOT requirement, full
Bahassa is OK
• Basic criteria
• Extended non essential criteria
• Recommended policies for specific criteria
• Specific problems (Indonesia)

Part 3 Scientific quality
• Properties of low quality journals
• Properties of questionable journals
• Whitelists versus blacklist

Part 4 Practical session
• Filling out an application

A not-for-profit Community Interest Company
(C.I.C.), registered in the United Kingdom.

What is the DOAJ?
• Directory of Open Access Journals
• THEN: launched in May 2003, Lund University, Sweden
– list of 300 titles
• NOW: Centrally, publicly and internationally available
community-curated list of > 12,000 high quality open
access journal titles across all disciplines
(scientific/scholarly) from 129 countries
• DOAJ Ambassador programme now covers all major
world regions (with exception of Japan)
• Also lists journals publishing in local language only

The DOAJ core team
• Managing Director
• Operations Manager
• Project and Communications Manager
• Editor-in-Chief
• Senior Managing Editor
• 6 Managing Editors
• We are based in Sweden, United Kingdom, the
Netherlands, Italy, Spain, India & Denmark
– And…

Volunteers and
Ambassadors
• 50+ Voluntary Editors/Associate Editors working unpaid a
few hours/week – distributed in editorial groups managing
20+ languages
• 20 Ambassadors recruited to
– Promote DOAJ
– Handle applications of journals to be listed in DOAJ
– Promote best publishing practice and
– Help identifying and spotting questionable and unethical
publishers
• Ambassadors are based in
– China, India, Russia, Egypt, Ethiopia, Burkina Faso, Algeria, South
Africa and Mexico, Indonesia & Korea – covering Asia, Middle
East, Africa and Latin America

three-tier evaluation
proces
Managing
Editor
Associate Editors: reviewing applications, communicate with publishers,
recommend inclusion/rejection
Editors: allocating applications to Associate Editors, recommend
inclusion/rejection
Managing Editors: allocate applications to Editors & decide on
inclusion/rejection

Publisher/Editor
Applies
DOAJ Editor
Assess/ISSN
DOAJ Team
Evaluate &
Recommend
DOAJ Editor
Final Decision
Feedback to
Publisher/Editor
Workflow
1. Reject
2. Approve
3. Approve & Seal

New Governance Model
• Board 8-10 seats, nominated by major
contributors
– tasks: Advice and feedback on Business issues
(Strategy, Budget, End of Year Results, Fundraising
Strategy
• Council: Broader Advisory Group
– 20-25 seats (to allow for geographical and stakeholder
diversity). Based on a general call for nominations
from current supporters.
• Editorial Committee 10 members

DOAJ ambassadors for Indonesia:
Dian Eka Indriani
Ikhwan Arief
Mochammad Tanzil Multazam
Due to the scale of operations in Indonesia, we have also
appointed 2 honorary Ambassadors: Professor Toho
Cholik Mutohir (in Indonesian) and Abdul Wahid
Maktub. Professor Mutohir is a lecturer at Surabaya
State University and Abdul Wahid Maktub is Special
Advisor to the Minister of Research, Technology and
Higher Education of the Republic of Indonesia.
INCREASING THE IMPACT OF OPEN ACCESS IN
INDONESIA

Algeria and Indonesia are the first countries
outside the EU that have government
directives for journals to be included in DOAJ

DOAJ Statistics 2018

TOTAL NR
OF
JOURNALS
IN DOAJ
2018

Position of Indonesia
NEED FOR IMPROVING JOURNAL QUALITY
First rank in journals added to DOAJ in 2018
But also
First rank in journals rejected

• DOAJ Directory of Open Access Journals
OPEN ACCESS JOURNALS ONLY, NO RANKING
• SCOPUS
OPEN ACCESS & SUBSCRIPTION JOURNALS , RANKING
• WEB OF SCIENCE
OPEN ACCESS & SUBSCRIPTION JOURNALS , RANKING
• NATIONAL LISTS
OPEN ACCESS & SUBSCRIPTION JOURNALS, RANKING
INDEXING SERVICES
for quality control

• DOAJ looks at the quality of publishing using the principles of Best
Practice in scholarly publishing
The scientific quality of articles only plays a role in discriminating
between low quality and questionable journals
The journals in DOAJ are not routinely assessed for the quality of
their content
• Citations are not a good measure of article quality and alternatives
are not yet well developed
Assessing Quality of Journals

• The scientific quality of a journal can not be assessed by the
average citation scores of articles
(the Journal Impact Factor JIF and derived methods like Citescore)
• The scientific quality of articles cannot be assessed by where
they are published
• The scientific quality of scientist cannot be assessed by where
or how much they publish
Scientific quality and impact can only be assessed on article level
Assessing Quality of Research

Quality of Scholarly Publishing
The basics

The principles of Transparency and Best
Practice in Scholarly Communication
• The Principles are very much inspired by the initial draft of the new
DOAJ criteria, apply not only for Open Access publishing and has
developed into de-facto standards.
• https://doaj.org/bestpractice

https://doaj.org/

The application form
• The new application form:
• http://doaj.org/application/new

Essential criteria for journals to be listed
• ISSN registered
• Clear and complete instructions for authors
• An Open Access statement
• Comply with the BOAI definition
• Transparency on charges (APC, submission)
• A peer-review process, details of the kind of process
• An editor/editorial board with clearly identifiable members
• Licensing and copyright information
• Aims and scope
• Published a least 5 articles per year to qualify

Instructions for Authors*
• Detailed style guide
• Description of quality control process (review)
• Copyright information
• Licensing information
• Plagiarism policy
• Instructions on how to submit an article
• Contact email address

“Non essential” criteria
• Archiving
• Permanent article identifiers (DOI etc)
• Deposit policy of journal in Sherpa Romeo or other
• Plagiarism control
• Uploading metadata
• Use of Creative Commons licenses
• NOTE all questions need to be answered!
But ‘No” here is also acceptable

Recommendations to journals wanting to be listed
• Unrestricted copyright for the author
• No exclusive publishing rights for the journal
• No transfer of commercial rights
• Use of Creative Commons licensing
• Embedded licensing information with articles
• Permanent article identifiers
• No mention of impact factors

• Openness, Reuse & Remixing rights, Licensing,
Copyrights and Permissions!
Openness

Editorial issues
Specify what kind of review process is applied: Editorial
review, Peer Review, Blind Peer Review, Double Blind
Peer Review, Open Peer Review

Reuse/remix

Licensing

Copyright and
permissions

Archiving/Preservation
• Archiving is important – too many OA-journals do not have an
archiving arrangement

Plagiarism etc

Charges

DOAJ Seal for Open Access Journals
•Best practice recommendations for OA publishing
•Based on 7 criteria relating to accessibility, openness,
discoverability, reuse, and authors rights
•All criteria must be met to be awarded the DOAJ Seal
•Note the Seal does not reflect the academic quality of the
journal

DOAJ Seal

DOAJ Seal criteria
•Copyright
•To allow authors to hold copyright and publishing rights
without restriction
•Copyright must be retained by the author
•Exclusive publishing rights for the publisher do not qualify
•Transfer of commercial rights to the publisher do not qualify

DOAJ Seal criteria
•Deposit policy
•To provide information on authors’ self-archiving rights
•Policies for archiving of pre-prints, post-prints and publisher’s
version
•Covers author’s own web site, institutional and subject
repositories
•SHERPA/RoMEO and others

Depositing Policy

http://www.sherpa.ac.uk/romeo/index.php

• Website security (Malware)
• Links are not to correct information
• Copyright and licensing
• Peer reviewer number and diversity
• APC changes after DOAJ listing
• Editorial board diversity
• Authors = editors
Examples of problems with DOAJ
criteria in Indonesia

Your journal's page has a “Not Secure” Warning in Chrome?
Here's how to make your journal's page more secure
47
Website security (Malware)

We reviewed more than 300 journals in terms of security after
feedback on Malware in Gold Open Access Journals by
Walt Crawford
What does “Non Secure” mean?
48

The latest version of the Google Chrome browser, version 68,
introduced a new “Not Secure” warning in the address bar that
appears anytime you are visiting an insecure web page.
It’s alerting you that information sent and received with that
page is unprotected and it could potentially be stolen, read, or
modified by attackers, hackers, and entities with access to
internet infrastructure.
49

Over the last few years, websites have been transitioning to
HTTPS—note the S appended to the end—which does
provide security and is used by millions of websites including
Google.com, Facebook.com, and Amazon.com, to protect
your information while browsing, logging in, and making
purchases.
50

The “Not Secure” warning does not indicate that your
computer or the site you are visiting is affected by
malware. It only serves to alert you that you do not
have a secure connection with that page.
Learn more:
https://support.google.com/chrome/answer/95617?visit_id=636885998421155131
-3452271450&p=ui_security_indicator&rd=1
51

• All links given in the application form to same
URL
• Links point to wrong pages
• Required information is not in the website
Links are not to correct information

• Copyright Notice
• Authors retain copyright and grant the journal right of first publication with the
work simultaneously licensed under a Creative Commons Attribution-ShareAlike
4.0 International License that allows others to share the work with an
acknowledgment of the work's authorship and initial publication in this journal.
• Every accepted manuscript should be accompanied by Copyright Transfer
Agreement prior to the article publication
Copyright and licensing
•Authors retain copyright
•Copyright Transfer Agreement in place

•Open Access license
•All rights reserved notice with the articles
Copyright and licensing

•Open Access license
•Publisher restricts rights of the authors to fair
use only
Copyright and licensing
Copyright transfer agreement.
Penulis masih mempunyai hak untuk hal-hal berikut:
1. menggandakan seluruh atau sebagian materi yang dipublikasikan untuk digunakan oleh penulis
sendiri sebagai bahan pengajaran di kelas atau bahan presentasi lisan dalam berbagai forum;
2. menggunakan kembali sebagian atau keseluruhan materi sebagai bahan kompilasi bagi karya
tulis penulis selanjutnya;
3. membuat salinan dari bahan yang dipublikasikan untuk didistribusikan di lingkungan institusi
tempat penulis bekerja.

•Use of Open Access license (CC-BY-NC)
•Rights restricted to using articles for
educational purposes only
Copyright and licensing

•Copyright transferred to journal
•Use of Open Access license (CC-BY-SA)
•Exclusive rights for publisher
Copyright and licensing

• Articles reviewed by one reviewer
• Most peer reviewers from same institute or
university
• Peer reviewers from non-relevant disciplines
Peer reviewer number and
diversity

• before listing no APC, after listing APC
• Before listing APC, after listing (much) higher
APC
APC changes after DOAJ listing

• Editors from one Institute or University
• Editors all local in International journal
• Editors not experts in Journal’s scope
Editorial board diversity

• peer reviewers and editors as co-authors in
(too)many of the articles of a journal
(> 25 %)
Authors = editors
4 /6 of the articles authored by
editors / peer reviewers

Differences between
questionable and
low-quality journals

OUR DEFINITION
Questionable publishers are
publishers, who are not living up to
reasonable standards in terms of
content, services, transparency and
business behavior.

OUR DEFINITION
Low Quality Publishers are
publishers, who publish low quality
articles due to inadequate peer
review and or editors / peer
reviewers from one institute only

Questionable Publishing: the Media Picture
• BIG PROBLEM LINKED TO OPEN ACCESS JOURNALS
• OPEN ACCESS IS LOW QUALITY (everything accepted)
• REPUTED SCIENTISTS ALSO PUBLISHED THERE
• DANGER FOR THE REPUTATION OF SCIENCE
• BIG MEDIA ATTENTION WITH BIASED SURVEYS
* http://walt.lishost.org/2015/11/ppppredatory-article-
counts-an-investigation-part-1/
**Shen and Björk. BMC Medicine201513:230
THE STING

Questionable Publishing: the Real Picture
• NOT ONLY IN OPEN ACCESS JOURNALS
• Not AS HIGH AS OFTEN REPORTED
according to a study* by Walt Crawford the number questionable publishers in
2014 was about 3275 publishing about 121,000 articles and not 420,000 (Shen and
Björk ** reported 8000 journals and 420,000 articles!!)
* http://walt.lishost.org/2015/11/ppppredatory-article-
counts-an-investigation-part-1/
**Shen and Björk. BMC Medicine201513:230

Questionable Publishing in Perspective
• Proportion of low quality journals is comparable
between open access and subscription publishing
but it looks worse because Open Access journals are more visible
FACTS
• Not all subscription journals are in Scopus or WoS: only 20-30,000 of 100,000 (data Ulrich’s
Web)
• Not all open access journals are in DOAJ : only 8 -12,000 of 30,000 ***
CONCLUSION Percentage of Quality Journals in Open Access and Subscription
Journals is comparable
*** Walt Crawford
http://citesandinsights.info/civ17i1.pdf

• Indonesia has not many questionable journals
• The problem is more one of low quality publishing and bad
understanding of Good Publishing Practices and the DOAJ criteria for
assessing journal quality
• There is a need for (repeated) nation-wide education
• Useful to get Indonesian translation of the 'DOAJ for publishers
info' page
The good news is

PRACTICAL
Filling out an application online:
https://testdoaj.cottagelabs.com
https://testdoaj.cottagelabs.com/

Information on the DOAJ
• Home: https://doaj.org/
• About: https://doaj.org/about
• Publisher information: https://doaj.org/publishers
• Apply: https://doaj.org/application/new
• FAQs: https://doaj.org/faq
• Best Practice: https://doaj.org/bestpractice
• comprehensive online resource

Thanks to :
All the Library Consortia, Universities and Publishers
and our Sponsors for the financial support to DOAJ!
Speed of Indexing in DOAJ and Indexing as such is independent of sponsor status of applicants
Gold Sponsor

Thank you for your attention!
tom@doaj.org

DOAJ depends entirely on donations
https://doaj.org/membership

https://scoss.org
OR DONATE VIA SCOSS