An open science monitor or open access monitor is a scientific framework that aims to assess the spread of open practices in a scientific context.
Open science monitors or dashboards are built at different scales: from institutional to national or international. They require an accurate assessment of the total scientific output and a further breakdown between open and closed content. They rely on a variety of data sources and methodologies to achieve this end. Consequently, open science monitors have also become relevant tools for bibliometric analysis.
While initially conceived to track publications in academic journals, open science monitor have diversify their scopes and indicators. A recent trend has been to map other major outputs of open science research such as datasets, software or clinical trials.
Definition Open science monitor are a scientific infrastructure that provide a "good knowledge of the state" of scientific outputs and their "open access rate". They are also a policy tool that aims to better assess the discrepancy between actual practices and long-term objectives: they "can inform future strategies at institutional and national levels, provides guidance for policy development and review, helps to assess the effects of funding mechanisms and is crucial to negotiate transformative agreements with traditional subscription publishers."
Open Access Monitors are a specific variant of open science monitors, that is focused on open access publications. They aim to track the share of open access among journal articles, but also "books, book chapters, proceedings, and other publication types". In contrast, generic open science monitors have a more expansive scope and will in effect include all forms of scientific outputs and activities: "By definition, open science concerns the entire cycle of the scientific process, not only open access to publications"
Nearly all the open science monitor have been created at a national scale, as part of a general policy of enhanced visibility of public costs and investments in regards to scientific publications. Major examples include the Baromètre de la science ouverte in France, the Open Access Monitor in Germany, JUULI in Finland, the Open Access Barometer in Denmark, NARCIS and later openaccess.nl in the Netherlands and the Swiss Open Access Monitor.Swiss Open Access Monitor A prototype of open science monitor was also conceived in the United Kingdom in 2017 but "apparently not realized."
International initiatives include the Australian-based Curtin Open Knowledge Initiative (CUKI), the Open Science Monitor of the European Union and OpenAIRE. Yet, the spread of their data is more limited than national monitors, as they do "not offer evaluation options on an institutional level".
History
Context Open science monitors belong to a global ecosystem of open scientific infrastructures. This ecosystem emerged in the first decades of the 21st century as an alternative to the closed infrastructures built by large scientific publishers and analytic companies.
After the Second World War, scientific publishing faced a "periodical crisis": funders, institutions and journals could not keep up with the rapidly increasing scientific output. New infrastructure, tools have to be developed also to keep track of scientific investment. Due to the limited success of public initiatives like SCITEL or MEDLINE in the United States, large private organizations filled this need. In 1963, Eugene Garfield created the Institute for Scientific Information that aimed to transform the projects initially envisioned with the Federal administration into a profitable business. The Science Citation Index and, later, the Web of Science had a massive and lasting influence on global scientific publication in the last decades of the 20th century, as its most important metrics, the Journal Impact Factor, "ultimately came to provide the metric tool needed to structure a competitive market among journal. Consequently funders increasingly relied on analytics created by the Science Citation Index and its main competitors to assess the performance of institutions or individual researchers.
After 1990, leading academic publishers started to diversify their activities beyond publishing and moved "from a content-provision to a data analytics business." By 2019, Elsevier has either acquired or built a large portofolio platforms, tools, databases and indicators covering all aspects and stages of scientific research: "the largest supplier of academic journals is also in charge of evaluating and validating research quality and impact (e.g., Pure, Plum Analytics, Sci Val), identifying academic experts for potential employers (e.g., Expert Lookup5), managing the research networking platforms through which to collaborate (e.g., SSRN, Hivebench, Mendeley), managing the tools through which to find funding (e.g., Plum X, Mendeley, Sci Val), and controlling the platforms through which to analyze and store researchers' data (e.g., Hivebench, Mendeley)." Metrics and indicators are key components of this vertical integration: "Elsevier's further move to offering metrics-based decision making is simultaneously a move to gain further influence in the entirety of the knowledge production process, as well as to further monetize its disproportionate ownership of content." The new market for scientific publication and scientific data has been compared with the business models of social networks, search engines and other forms of platform capitalism While content access is free, it is indirectly paid through data extraction and surveillance.
Early developments
The first open science monitors were created in the 2000s and the early 2010s. They were usually conceived as a natural outgrowth of new national and international policy in favor of open access and open science. The Berlin Declaration from 2003 especially introduced the concept of a global "transition of scientific publishing toward an open access system" which require "information on publication output and on subscription and publication fees."
Additionally the diversification of open science publishing into various publication venues (journals, repositories, overlay journals...) and formats (articles, conferences, datasets...) created unprecedented challenges.
One of the earliest form of open science monitor was the Dutch project NARCIS ("National Academic Research and Collaborations Information System") that started operating in December 2005. NARCIS is primarily a national scientific portal that aims to integrate "all kinds of types of information from scientific institutes in the Netherlands." Yet it also has a special focus on "academic OAI repositories" and publishes global statistics on the rate of open restricted and embargoed scientific works since 2000.
By 2013, Finland pioneered the influential Jyväskylä Model through its national portal JUULI. First experimented at the Open Science Centre of the University of Jyväskyl this approach aims "to centralize all aspects of the self-archiving and open access processes lying within the responsibility of the professionals at the university library" in order to ease the process of data collection: "Researchers do as little as possible and, in some cases, nothing at all."
From open access to open science After 2015, the European Union started to implement ambitious programs and goals within its own funding mechanism like Horizon 2020. This created an unprecedented impetus for the development of monitoring tools and methodologies at a supranational scale: "there has also been a general push for increased monitoring, aiming for both increased transparency to enable each country to see what others are doing" By 2018, 81% of the scientific organizations from Science Europe stated that they "planned to develop Open Access monitoring mechanisms in the future"
In their preparatory work of the Open Science Monitor, Smith et al. underlined that "open science is much more than simply open access, despite the fact that open access tends to dominate discussions at present." Beyond research publications, their approach singled out open research data and a wider range of Communication activities related to open science that included preprints, evaluations, comments and online discussions on social networks.
In May 2018, the European Commission unveiled its plan for a European Open Science Monitor, through a detailed methodological note. While the core features of the Monitor were in line with previous research, it was also announced that Elsevier would be the leading subcontractor for the creation for the platform, despite the past commitments of the academic publisher against open science, and the metrics would combine the metadata of Scopus and Unpaywall to assess the rate of open access publications. The proposal was met with significant backlash, with nearly 1000 researchers and open science activists signing a formal complaint to European Ombudsman. In an oped to the Guardian, Jon Tennant stated that "it is a cruel irony that Elsevier are to be paid to monitor the very system that they have historically fought against."
The European Science Monitor has been subsequently reworked in a different direction. As of 2023, the website only include data only up to 2018. In 2022, the European Council clearly states that "data and bibliographic databases used for research assessment should, in principle, be openly accessible and that tools and technical systems should enable transparency".
The European Open Science Monitor has entailed a significant shift in the objectives and ambitions of similar projects in the member states. In 2018, the French feedback for the Monitor included a detailed plan for the elaboration of open science indicators beyond publications that would prove to have a direct influence over the Barometer of open science
Sources Yet, open science monitors have to deal with different sources of scientific data, since currently "no database provides an easy and complete answer". Consequently, "for most monitoring exercises, data from multiple sources will need to be gathered, aggregated, and reconciled"
The most important sources available for open science monitors include international open science infrastructures, local sources and proprietary platforms. The choice of sources is frequently dictated by policy concerns and technical constraints. The United Kingdom or Germany lack a "pool of data" from local sources and consequently decided to rely significantly on proprietary databases like Dimensions, Wos or Scopus. Conversely, the French Open Science Monitor opted for a "constitutive choice" of open sources.
International Infrastructures Leading open science infrastructures commonly used in open science monitor include, Unpaywall, Crossref and the Directory of Open Access Journals (DOAJ) Crossref is the primary information source of the French Open Science Monitor, as it only considers "publications associated with a Crossref DOI"
Due to significant developments during the 2010s, international infrastructure have a larger scope of "publications, languages and sources" than proprietary databases. Yet "they offer insufficiently standardized metadata, which complicates their collection and processing" and may lack key information for the creation of the open science monitors, such as author affiliations.
Local infrastructures and repositories