Measuring the Concept of PID Literacy: User Perceptions and Understanding of PIDs in Support of Open Scholarly Infrastructure

1.1 Persistent Identification and the PID Graph

A PID provides a unique and persistent reference to an entity which is normally accessible over the Internet (McMurry et al., 2017). Such PIDs not only provide long-term identification for these entities but also actionability by being encoded as a Uniform Resource Identifier (URI) (e.g. Weigel, Kindermann, & Lautenschlager, 2014). A PID registry service – usually designed to assign PID URI prefixes, register new PIDs, and provide the necessary technical infrastructure to manage PIDs – will also typically provide metadata describing the entity, contextualizing its potential (re)use. One common form of PID which has come to typify the identification of scholarly resources on the web is the DOI. The notion underpinning a DOI is that it remains static (“persistent”) over time and, when dereferenced on the web, will always resolve to the identified resource, irrespective of whether its digital location may have changed in the intervening period (International DOI Foundation, 2017). This general PID approach has since been adopted within the digital scholarly infrastructure to persistently identify a wide variety of people, places, and things on the web, from researchers (e.g. ORCID^[1]) to research projects (e.g. Research Activity Identifier [RAiD]^[2]) to cell lines (e.g. Research Resource Identifier [RRID]^[3]), some reusing existing Handle.net infrastructure or devising different but similar service mechanisms to enable the assignation, registration, and resolution of PIDs (Cousijn et al., 2021). Figure 1 provides an example for the following Research Organization Registry (ROR) PID, the European Organization for Nuclear Research (CERN). ROR is used to persistently identify research-related organizational entities, and, in this case, it identifies CERN: https://ror.org/01ggx4157. In the same way that DOI registries gather metadata about the entities to which they are assigning DOIs, similarly, associated structured data for this PID are gathered by the ROR registry, available via the ROR API: https://api.ror.org/organizations/01ggx4157.

Figure 1

Example of the ROR for the PID of the CERN: https://ror.org/01ggx4157. Screen captured from ROR (https://ror.org/), under Creative Commons Attribution International License 4.0 (CC-BY).

Although the concept of source identification in scholarship is not new (Kaplan, 1965), PID approaches under discussion in this work have emerged within the context of digital object management where the issue of “link rot” and persistence in the scholarly record have become important considerations in maintaining a healthy scholarly publication and research ecosystem (Jones et al., 2016). PIDs enable the improved citation and tracking of scholarly entities because their identifiers – whether of people, places, or things – remain persistent over time. This mitigates problems arising from link or reference rot and enables continued reuse of whatever is identified by the PID, either by user or machine. For this reason, PIDs are increasingly considered an important insurance mechanism within scholarship, assuring that scholarly verification, reproducibility, and replicability remain possible within an age of evolving digital scholarly infrastructure (Ivie & Thain, 2018; Meadows, Haak, & Brown, 2019). Additionally, the improvements in tracking made possible by PIDs offer scientometric potential, with services like Crossref becoming valuable datasets in better measuring research impact, knowledge growth, and scholarly communication trends (Hendricks, Tkaczyk, Lin, & Feeney, 2020).

It should be noted, however, that PIDs in themselves do not and cannot guarantee persistence. For example, investigations by Klein and Balakireva (2022) of DOIs – arguably the most ubiquitous PID type – suggest widespread DOI request failures and inconsistent machine responses from organizations using them. Members of the same research team have also proposed their Memento “Robust Links” approach as a means of improving the reliability of Uniform Resource Locator (URL) and URI-based referencing on the web, including with respect to PIDs (Klein, Shankar, & Van de Sompel, 2018). PIDs are therefore only persistent insofar as a PID registration service commits to resolving them, or insofar as a publisher commits to updating a PID registry with the current location of a web resource. Meanwhile, there are explorations of alternative PID models for services like open repositories, where the use of repository Open Archive Initiative (OAI) identifiers – a unique identifier for a metadata record held within a repository or OAI compliant software platform – offers a decentralized mechanism for contributing to scholarly data graphs (Knoth, Budko, Pavlenko, & Cancellieri, 2022). Despite these caveats and developments, PIDs can and do foster greater persistence of scholarly entities and help to unambiguously identify them, thereby encouraging resource discovery and reuse.

The ability to unambiguously identify entities – combined with greater certainty of persistence – has enabled the encoding of relational associations between PIDs and their associated metadata, in turn, generating complex but valuable scholarly graph-based networks (Um, Choi, Kim, & Lee, 2020). By extending many of the conventions established by the Resource Description Framework (RDF) (Macgregor, 2009) and its use within Linked Open Data (LOD) in particular (Treloar, 2011), the so-called “PID graph” presents opportunities for improved resource discovery, inferencing/reasoning, research aggregation, and so forth by establishing connections between different entities within the research landscape. RAiD, for instance, can persistently identify research activities such as an ongoing research project. This RAiD can then be referenced within scholarly article metadata thereby creating a relational association and the colocation of articles arising from the same research project. But the RAiD itself can also be used to reference associated entities within its “metadata envelope,” as in Figure 2. In this diagrammatic example, the RAiD PID references DOIs of research datasets associated with the research activity, an identifier from the Global Research Identifier Database and the International Standard Name Identifier (ISNI) scheme, a group identifier, and even another related RAiD, referenced as a sub-project (Janke, McCafferty, & Duncan, 2017).

Figure 2

Diagrammatic of a RAiD, a metadata envelope for an example research activity (i.e. project), containing associative relations via PID to related research entities. Diagram from Janke et al., 2017, used under Creative Commons Attribution International License 4.0 (CC-BY).

Numerous scholarly graphs have emerged in recent years but not all generate their entity relations entirely using PIDs and instead use knowledge extraction and inferencing techniques, or combinations thereof (Atzori, Manghi, & Bardi, 2018; Manghi, Atzori, De Bonis, & Bardi, 2020; Schirrwagen et al., 2020). Though the PID graph is dependent upon metadata that demonstrates high levels of expressiveness, the graph is simpler because graph construction, maintenance, and relational associations are based entirely on the PIDs themselves. This aids graph scaling as the computational overhead associated with knowledge extraction, inferencing, and de-duplication does not exist (Cousijn et al., 2021). Suffice to state that important contributors to the PID graph include – but is not limited to – DataCite, Crossref, ORCID, ROR, ISNI, FundRef, RRID, and RAiD. The nature of this article is such that a fuller treatment of the PID graph exceeds our scope; however, useful technical background is provided by many others in the literature (Aquino et al., 2017; Cousijn et al., 2021; Dappert et al., 2017; Klump & Huber, 2017; Meadows et al., 2019).

It is also apposite to note that emerging models of scientific communication entail even greater levels of PID specificity. The so-called “nanopublication” models (Bucur, Kuhn, & Ceolin, 2020) and emerging platforms such as Octopus.ac^[4] seek to disaggregate the components of research into verifiable, citable chunks. Such models disaggregate research into persistently identifiable components, such as research problem, hypothesis, method, analysis, and so forth (e.g. Freeman, 2022); however, within other models, disaggregation can extend to specific paragraphs within a scientific paper (Bucur et al., 2020). These models are beyond the indicative cases to be noted later in this article but nevertheless point to an emerging trend which is seeking to expose scholars to the “PID-ification” of almost all aspects of their research.

1.2 User Research and Introducing “PID Literacy”

An extensive body of PID literature has emerged since the early 2000s, especially within digital library, digital repository, and scholarly infrastructure research, but focuses almost exclusively on PID infrastructure, technical governance, PID types, policies, metadata profiles, and so forth (e.g. Chandrakar, 2006; Foulonneau & André, 2008; Koehler, 1999; Nelson & Allen, 2002; Simons & Richardson, 2013; Weigel, Lautenschlager, Toussaint, & Kindermann, 2013). This literature has been critical to the technical evolution of PID-based technologies but does not address the socio-technical side of PID growth. For example, little is known about scholars’ understanding and perception of PIDs, or the extent of users’ “PID literacy” – a concept which we will introduce in this section.

PIDs are largely concerned with citing entities. Their broader efficacy is dependent upon their accurate reproduction by others. Since our direct understanding of scholars’ interaction with PIDs is minimal, it is useful to consider tangential areas of research, especially work exploring scholars’ capacity for identifying scholarly entities, such as their citation habits and competency. Although the competencies required to use and understand PIDs is higher, the routine of citing sources within academic work is somewhat cognate since it requires that sources – i.e. entities – be accurately identified (Kaplan, 1965). Such work most notably investigates scholars’ citation habits and can inform our understanding of the present research context (Cano, 1989). For example, observations by Garfield (1974, 1990) in the early phases of bibliometrics found frequent and numerous citation errors in scholarly articles, introduced by authors during drafting. Misidentification of journals, misspelling of author names, and name mis-orderings were found to be common. Some errors have been found to be associated with a lack of citation verification (e.g. authors copying faulty citations from a faulty source) (Broadus, 1983), a phenomenon which persists and has been termed “referencing misbehaviour” (Liang, Zhong, & Rousseau, 2014). There are indications too of variation in the citing quality and behaviour of scholars across subjects, with patterns noted as being peculiar to specific subjects, compounding the problem (dos Santos, Peroni, & Mucheroni, 2022).

Early studies also found inaccuracies within 40% of all sampled citations (Key & Roland, 1977), with subsequent work by Asano et al. finding up to 48% of all articles containing one or more errors, of which author names and article titles were found to be the source of 70% of errors (Asano, Mikawa, Nishina, Maekawa, & Obara, 1995). Interestingly, a recent longitudinal study investigating citation accuracy in articles published at 10-year intervals in a well-known journal title (between 1991 and 2019) reported an overall citation error rate of 40% (Logan, 2022), with little fluctuation in this error rate between decade intervals. This body of work suggests that in many cases, scholars’ ability to accurately identify and cite scholarly sources is not as good as it should be, despite a perceived improvement in digital scholarship ability among scholars.

Citation is as old as scholarship itself – but fluency with PIDs includes an additional understanding of web technology and the notion that URIs can reference abstract entities on the web. The information science sub-domain of information literacy has for some time noted the general challenges arising from the growth in digital academia (Behrens, 1994; Bruce, 1995; Sorapure, Inglesby, & Yatchisin, 1998). Findings from studies produced in previous decades have alerted academia to information literacy deficiencies present within some academic disciplines (Webber, Boon, & Johnston, 2007; Secker, 2004). Influential information literacy conceptual models, most notably the “Framework for Information Literacy for Higher Education” proposed by The Association of College & Research Libraries (ACRL), have subsequently recognized potential deficiencies within the scholarly communities (ACRL, 2016). The ACRL framework presents “Scholarship as Conversation” as a core component of information literacy, within which scholars should develop “familiarity with the sources of evidence, methods, and modes of discourse” used to engage in “scholarly conversations.”

More recently, conceptions of information literacy have broadened to “digital literacy,” encompassing notions of web literacy and technical skills sets (Alexander, Becker, Cummins, & Giesinger, 2017), or the more expansive overarching concept of “metaliteracy” (Mackey & Jacobson, 2017). These broader digital competencies have been severely tested in recent years as scholars have been forced – some with great difficulty – to engage in online teaching and digital research as a consequence of the Covid-19 pandemic (Heriyanto, Christiani, & Rukiyah, 2022). Suffice to state that recent work studying digital literacy competencies suggest that such skill deficiencies are common among some academic groups (Ong, 2021), with deficiencies identified to be found across scholarship more generally. For example, a literature review by Basilotta-Gómez-Pablos et al. analysed 56 articles that studied the digital competencies of university teaching staff, finding that “low or medium–low” digital competence was dominant (Basilotta-Gómez-Pablos, Matarranz, Casado-Aranda, & Otto, 2022).

Though increasingly unreflective of current information literacy habits given its publication age, Wouters and de Vries (2004) found wide variability in the way in which web-hosted scholarly resources were referenced using hyperlinks by scholars, noting a lack of standardization in hyperlink conventions. Notwithstanding that these problems have a long history in academia, it could be speculated that more recent neglect among doctoral student cohorts is inherited from an assumption that younger scholars are “digital natives” and therefore more likely to be digitally competent or information literate, a notion that has been widely refuted by recent evidence (Judd, 2018). Indeed, information literacy experiments conducted by Greer and McCann (2018) involving the citation behaviour of final year undergraduate students revealed that supposedly “digitally literate” students “do not understand URLs,” with many unable to distinguish between official and unofficial URLs or redirects, or even correctly identify a digital source on the web (Greer & McCann, 2018). It is therefore reasonable to assume that these behaviours may be carried forward to doctoral level, and perhaps beyond; and an inability to even understand URLs does raise questions about the potential extent of PID competencies in academia more generally.

The notion of “PID literacy” is not a concept which has been defined in the literature. We introduce it here to help us conceptualize the expected competencies a typical scholar might need in order to interact with PIDs effectively. It is possible to identify some typical scenarios which help us understand how PID literacy might manifest itself in users. Indicative cases where PID literacy might be exercised could include one or more of the following scenarios:

• Correctly supplying an ORCID for all contributing authors when submitting an article via a manuscript submission system.

• Providing PIDs to related datasets within an article manuscript (e.g. a data availability statement); but could equally apply to providing PIDs to related software, research instruments, samples, etc.

• Correctly reproducing and referencing funders, grants, and other abstract entities using PIDs.

• Curating research-related PIDs (e.g. funders, organizations, collaborators, etc.) within a RAiD metadata “envelope” (a location for descriptive information about research activities to be organized within RAiD).

• Correctly identifying and citing (by PID) different expressions of the same scholarly work, as expressed through preprints, accepted author manuscripts, versions of records, etc.

• Understanding when creation, use, or reuse of a PID might be relevant and knowledge of suitable or applicable PID types.

The above noted examples are merely indicative and any number of alternative PID-related scenarios could be imagined. Nevertheless, on this basis, we could propose that a PID literate scholar might display the following competencies:

1. An understanding of persistent identification in scholarship, when it should be used, and its importance to the scholarly record and the wider PID graph.

2. An ability to accurately identify, reproduce, and cite PIDs in scholarship activities.

3. Cognizance of adjacent PID types relevant to scholars’ community of practice, such as those devised to identify scholarly “things” other than academic papers.

PID literacy, as conceived here, is not a holistic conception of information literacy but instead could be considered an extension or sub-literacy of existing frameworks, most notably ACRL’s “Scholarship as Conversation” frame (ACRL, 2016). This “frame” and its companion document on “Research Competencies in Writing and Literature” (ACRL, 2021) highlight the expected “knowledge practices” and “dispositions” of scholars if they are to meaningfully interact with research and scholarly communication processes. These practices and dispositions clarify the expected competencies of scholars but stop short of capturing PID literacy behaviours.

1.3 Motivation and Research Questions

We noted in earlier sections that, despite being largely ignored in the discussions surrounding PID development, scholarly users are frequently an important source of data to the PID graph, whether they are aware that they are the source of these data or not. Even perfunctory contributions greatly enrich the PID graph, furthering the relational and inferential potential of the graph – and are essential to the creation and interlinking of heterogeneous entities, including different “expressions” of textual objects, datasets, software, research instruments, equipment, and the related PIDs these items may in turn generate. Better understanding the nature of scholars’ understanding and perceptions of PIDs therefore motivates our research. It motivates our work, not only because of the contributions that scholars may or may not make to the PID graph because of their PID literacy but because scholars’ literacy with PIDs within wider scholarly infrastructures will increasingly be exercised.

The research questions that emerge from our motivation are therefore as follows:

• RQ1 – PID familiarity : To what extent are users familiar with the notion of the persistent identification of scholarly entities, their purpose, and understanding of the URIs that underpin them?

• RQ2 – Identifying identifiers : To what extent can users correctly identify and distinguish between common PID types and their purpose?

• RQ3 – PID perceptions : How are PIDs perceived by users and what levels of disciplinary or job role differences can be observed across academic groups, if any – and can any specific PID literacy gaps be identified?

• RQ4 – Habits : Current state of PID (re)use; what are the habits? Do they routinely contribute (knowingly or unknowingly) to the PID graph? Are they habitually (re)using them (e.g. citation/references, in the creation and/or linking of scholarly entities, etc.)?

2.1 Research Instrument Design

2.1.2 Instrument Sections

Since we are using quite a diverse population of scholarly participants, there is the possibility that computer efficacy in some of these sub groupings may differ significantly, thereby influencing their ability to complete the PID-related tests. In other words, a large determinant of their ability to complete the PID tests might be their general lack of CSE. Section 1 of the instrument therefore sought to benchmark participants’ computer efficacy prior to the PID recognition tests and perception measurement, thereby enabling cross-disciplinary analyses of PID literacy by efficacy metric. The instrument made use of Howard’s CSE measure (Howard, 2014). Howard’s CSE measure includes 12 items and was determined to be preferable because it demonstrated improvements over popular alternatives, such Murphy, Coover, and Owen (1989), including a reduction in the number of questions and the incorporation of post-2010 computer language. Howard’s measure also demonstrates improved internal reliability and psychometric properties (Howard, 2014, 2020). The 12-item measure includes 12 statements relating to computer efficacy, measured using a 7-point Likert scale (Strongly disagree (1) – Strongly agree (7)).

Section 2 was designed to elicit data about participants’ recognition of PIDs and test the extent to which PIDs are understood, thereby contributing to our answering of RQ1 (PID familiarity) and RQ2 (Identifying identifiers). Each of the four tasks included screenshots taken from prominent scholarly publishers, articles, and repositories, each displaying several PIDs in context (Figure 3). Each of the four tasks challenged participants to identify the PIDs in the screenshots and indicate to which entity they pertained (e.g. Publications on a publisher website or platform, Research data or open data, People, etc.). Each challenge had only two correct responses. Participants were challenged on DOIs, ORCIDs, and Handle.net, with the PIDs contextualized differently in each task.

Figure 3

Example of PID recognition test questions as used within Section 2 of the research instrument.

Additional six test challenges were included in this section. Each provided an example of a prominent PID type and requested participants to indicate the extent to which they were recognized and, if so, to which entity type they mostly associated them. For example, a DOI might be strongly recognized by the participant and mostly associated with Publications on a publisher website or platform and Research data or open data, and so forth. The creation of response options for the entities was informed by the “landscape analysis” published in the literature (Cousijn et al., 2021). Participants were challenged on their recognition of the following PID types: DOI, ORCID, Handle.net, ROR, ISNI, and Uniform Resource Name (URN).

Data to evaluate participants’ perceptions of PIDs were captured and operationalized using the semantic differential measurement technique (Osgood, 1957; Snider & Osgood, 1969). The semantic differential measurement technique is one that has long been deployed to measure attitudes and perceptions towards concepts (or in Osgood’s words, to “measure meaning”). Each semantic differential scale consists of a series of bipolar adjective scales on which a participant responds, in relation to the object or concept being examined. Each adjective also corresponds to one of three “semantic space dimensions”: evaluation, potency, and (oriented) activity. These semantic space dimensions, or “factors,” are hypothesized to be used by humans in their assessment of almost all phenomena and, when combined with appropriate factor analysis, can yield a reliable measure of a participant’s overall reaction to something (Stoutenborough, 2008). Though emerging from psychology, the semantic differential technique was quickly adopted for information science purposes (e.g. Allen & Matheson, 1977; Geus, Mulder, Zuurke, & Levine, 1982; Katzer, 1972) and continues to be deployed and refined to better understand a wide variety of information science research topics. Despite being a commonly used technique, it has been observed that few studies in information science use the approach optimally (Verhagen, Hooff, & Meents, 2015), with many erroneously conflating scale design and interpretation with Likert scales (Schrum, Johnson, Ghuy, & Gombolay, 2020).

In this instrument, PID perceptions were measured on the use of PIDs against the following four concepts: Scholarly Communications; and the use of PIDs to refer to People, Places, and Things (Huber, Diepenbroek, Brown, Demeranville, & Stocker, 2016; Meadows et al., 2019). A total of 13 bipolar adjective pairs were created to assist in the measurement of PID perception against these concepts, using 9 points ranging from –4 to +4 to construct the semantic differential scale. These values were not displayed to participants. Each of the adjective pairs addressed the aforementioned “semantic space” dimensions of Osgood’s approach: evaluation, potency, and (oriented) activity (Osgood, 1957) (Table 1). Osgood notes that a degree of relative importance exists within the semantic space dimensions, with evaluation representing a more powerful dimension in human thinking than either potency or activity (Osgood, 1957). The conception and design of this section of the research instrument were therefore optimized to ensure that both bipolar adjectives and semantic space dimensions were appropriately addressed. Data from this section contribute to our answering of our PID perceptions research question (RQ3).

Table 1

Summary of adjective pairs used in Section 3 of the research instrument and their mapping to Osgood’s semantic space dimensions

Semantic space dimension	Adjective pair
Evaluation	Bad–Good
Potency	Unimportant–Important
Activity	Complex–Simple
Evaluation	Unintuitive–Intuitive
Evaluation	Foolish–Wise
Potency	Unscientific–Scientific
Activity	Laborious–Effortless
Evaluation	Useless–Valuable
Evaluation	Unintelligible–Intelligible
Potency	Abstract–Concrete
Activity	Difficult–Easy
Evaluation	Negative–Positive
Potency	Unnecessary–Necessary

Section 4 simply comprised five questions exploring participant’s PID (re)use behaviour, as per the Habits research question (RQ4). This was a brief question section and attempted to elicit data on how PIDs were or were not being created, used, or reused by participants. The extent of PID creation among participants is a significant behaviour to measure; but reuse is arguably a more significant characteristic of PIDs (e.g. reusing PIDs that already exist within the PID graph).

2.2 Data Collection

Scholarly participants were recruited via social media (Twitter, Mastodon) and through established email lists. Email lists pertaining to open science, scholarly communications, and open repositories were targeted (e.g. code4lib, UKCORR). The memberships of these lists predominantly comprise individuals associated with the delivery of scholarly communications and research publishing support at scholarly organizations and who would have internal academic staff lists to whom the invitation to participate could be directed. The authors also circulated details of the participant invitation on local academic institutional lists. Only scholars who were outside the orbit of open science, scholarly communications, and open repositories were invited to participate, since participants from these groups would inevitably demonstrate abnormal PID literacy relative to other scholarly groupings. Participation was restricted to only those scholars who were active in the publication or dissemination of research.

The data collection approach generated a convenience sample, a limitation that is discussed in a later section. Data collection occurred during 26 May 2022–24 June 2022, with all data extracted shortly thereafter in .csv for cleaning, analysis, and visualization, all performed in standard spreadsheet software. A total of 106 academic users participated in the test, with the results of 27 removed during cleaning (i.e. invalid responses).

3.1 CSE

Recall that Section 1 of the test instrument benchmarked CSE across participants using the specified CSE 12-item 7-point measure (Howard, 2014). Internal consistency of the scales was tested using Cronbach’s alpha and demonstrated strong reliability (α = 0.94). Data from the measure are summarized in Table 4. CSE results across the group revealed a moderate level of efficacy (M = 4.98; Mdn = 5) with a high level of variation around the mean (SD = 1.52). The highest possible total score for each participant in the CSE measure was 84. Measures of central tendency on this dimension (M = 59.09; Mdn = 60; SD = 14.60) highlight the large spread of reported computer efficacy across the participant group.

Table 4 also segments CSE data by the ASJC discipline grouping of participants. Life Sciences (M = 4.84; Mdn = 5; SD = 1.77), Physical Sciences (M = 5.19; Mdn = 5.25; SD = 1.53), and Social Sciences (M = 4.85; Mdn = 5; SD = 1.41) all demonstrated considerable dispersion from the mean. Similarly, summarizing total CSE scores demonstrated dispersion but revealed that those in Physical Sciences (ps) demonstrated a slightly higher mean CSE score suggesting possible differences across discipline groupings (M _ps = 62.07; Mdn_ps = 61.50; SD_ps = 15.19; IQR_ps = 15.50; M _ls = 57.57; Mdn_ls = 63.50; SD_ls = 17.09; IQR_ls = 20.50; M _ss = 58.24; Mdn_ss = 60; SD_ss = 12.59; IQR_ss = 15.25). A one-way analysis of variance (ANOVA) (α = 0.05) of CSE scores across the discipline groups was performed and post-hoc comparisons were performed using the Games-Howell post-hoc procedures. The Games-Howell test is noted by recent statistical research to be most suitable when data do not satisfy homogeneity of variance assumptions, with unequal sample sizes and unequal variances. (Rusticus & Lovato, 2014; Sauder & DeMars, 2019). The ANOVA reported no statistically significant differences.

Participants were further segmented by job role to observe possible CSE differences arising from academic duties, experience, etc. Summary data are set out in Table 5. Here we can observe that participants belonging to specific job roles reported greater efficacy than others. Interestingly, Lecturers (M = 47.67; SD = 19.09; IQR = 27.25) and Professors/Readers (M = 54.30; SD = 13.28; IQR = 16.75) reported the lowest mean levels of efficacy but with greater levels of variation, while Research Assistants (M = 75; SD = 9.85; IQR = 9.5) and Postdocs (M = 67.80; SD = 5.81; IQR = 4) reported the inverse. Despite observable variances between groups, a one-way ANOVA (α = 0.05) of CSE scores across job role groups reported no statistically significant differences.

3.2 PID Recognition Challenges

Data arising from the first batch PID recognition challenges are set out in Table 6. These data relate to the four tasks within Section 2 of the instrument, which challenged participants to identify the PIDs in screenshots and indicate which entities they identified. Each of the four tasks had only two correct responses. The correct responses in each task are indicated in Table 6 by an asterisk; those without an asterisk denote erroneous responses to the task.

A significant proportion of participants were successful in the challenges and correctly interpreted PIDs, with a high and low rate of 86% and 61% for the two components of each specific challenge, respectively. However, we can note that some participants failed the task #1 challenge, with ∼15% of participants (n = 12) failing to correctly interpret the PIDs. The failure rate was higher for task #2, with ∼22% (n = 17) and ∼34% (n = 27) observed. Rates for tasks #3 and #4 noted similar failure rates, with ∼24% (n = 19) and ∼25% (n = 20) for task #3 and ∼14% (n = 11) and ∼39% (n = 31) for task #4.

These results can be segmented by discipline (Table 7) and job role (Table 8), within which we can observe the overall performance of participants in these challenges. Participants from Social Sciences performed least successfully, achieving a score of ∼66%, ergo 34% of responses were incorrect. The maximum score that any single participant could achieve across the four challenges was 8; yet those in Social Sciences demonstrated huge levels of dispersion (M = 5.24; SD = 2.89; IQR = 5), exposing a wide variation in the success of Social Science participants in the task. This contrasts with Life Sciences participants, whose overall performance yielded a higher score (∼85%), with individual participants demonstrating greater homogeneity (M = 6.83; SD = 1.40; IQR = 1.25). Those from Physical Sciences were found to sit between Life Sciences and Social Sciences, with an overall success score of 70%. The differences appeared notable but were nevertheless tested. Owing to the statistical nature of the data group, a Levene test was performed and found that the homogeneity of variance assumption was not satisfied (p ≥ 0.001). A one-way ANOVA (with Welch F test – α = 0.05) of participants’ scores by discipline was therefore performed, indicating a statistically significant difference (F(2, 39.52) = 3.86, p < 0.03) between the performances of discipline groups. Post-hoc comparisons using the Games-Howell procedure (described previously) confirmed a significant difference between the performance of Life Science and Social Science participants only (p < 0.04).

Summary results by job role are set out in Table 8, alongside the summary results by discipline and measures of central tendency provided for individual performance on recognition tests by discipline. Participants identifying as Research Assistant demonstrated the highest mean score in the PID recognition challenges (∼96%), while those identifying as Research Fellow scored the lowest (50%). A huge range in individual scores can be observed from some SD figures. For example, some of those identifying as Research Support (M = 64.06; SD = 3.31; IQR = 5.50) got none of the challenges correct. Following verification of data as demonstrating significant variance (Levene, p ≥ 0.02), a one-way ANOVA (with Welch F test – α = 0.05) of participants’ scores by job role suggested a significant difference (F(7, 19.65) = 3.11, p < 0.03); but post-hoc comparisons using the Games-Howell procedure failed to confirm this (p > 0.05). Despite many participants identifying the PIDs correctly in the challenges, a wide variety of additional erroneous responses were often provided (Table 6). Using the same statistical procedures as previously, a statistically significant difference was found between job roles when erroneous responses were provided (F(7, 21.53) = 5.98, p < 0.001), in which those designated as PhD Students (p < 0.02) and Others (p < 0.04) were more likely to offer erroneous responses, and in greater number.

Recall that additional six test challenges were included in this section of the research instrument. Each test provided an example of a prominent PID type and requested participants to indicate the extent to which they were recognized and, if so, to which entity type they most associated them. It was anticipated that some of these types would be better known to participants than others. Data are summarized in Figure 5.

Figure 5

Recognition levels of specific PID schemes by participants. Participant responses reported as percentage of total participants.

The proportion of participants strongly recognizing PIDs was unsurprisingly highest for DOIs and ORCIDs, at 68.35% and 63.29%, respectively, and lowest for URNs (5.06%) and ISNIs (5.06%). Test challenges relating to other PID types generated mixed responses. For example, recognition of Handle as a type of PID generated considerable spread across the response options (Do not recognize = 34.18%; Unsure = 12.66%; Somewhat recognize = 16.46%; Strongly recognize = 22.78%). In indicating to which entity type participants most associated the PIDs, those who recognized the PIDs provided a spread of responses, summarized in Table 9. Owing to the nonparametric nature of data in this section, Kruskal–Wallis test was performed between discipline and job role groupings. Statistically significant differences were observed by discipline in the recognition of Handles (H(2) = 8.14, p < 0.02) and URNs (H(2) = 6.08, p < 0.05) only, to varying levels of significance. Differences were also confirmed in the recognition of ORCID (H(2) = 12.77, p < 0.02), ROR (H(2) = 13.96, p < 0.03), and URNs (H(2) = 12.11, p < 0.03) by job role.

3.3 PID Perception Measurement

Combined with the PID recognition challenges, the PID perception measurement section represented the next significant portion of the research instrument. Table 10 presents the “factor scores” for the ratings of PID concepts across all participants. Factor scores are derived from averaging the results for each bipolar adjective pair by the number of semantic dimension subjects present. For example, potency features four times as a semantic dimension (Table 1) and therefore “4” is the denominator.

From Table 10, we can report that participants indicated their lowest collective response to the activity dimension, across all PID concepts (M = 0.40). Within this dimension, the use of PIDs to identify People – an allusion to PID types such as ORCID – was considered most positively (M = 0.72) albeit low in comparison to the scores attained in other dimensions. Conversely, use of PIDs within Scholarly Communications (M = 0.55) or to identify Places (M = 0.21) or Things (M = 0.13) was perceived more negatively in the activity dimension. The most positive perceptions were observed in the potency dimension, yielding a factor mean of 1.70, in which the scores for the tested concepts (excepting Places) were highest. Here the perceptions of PIDs in Scholarly Communications (M = 2.09), People (M = 1.87), and Things (M = 1.81) were perceived most positively and higher than the evaluation dimension in most cases.

Results for PID perception measures across discipline groupings by semantic dimension are provided in Table 11 and reveal some disciplinary differences. Participants from Life Sciences were generally found to perceive the use of PIDs more favourably, particularly in relation to the People concept, where participant rating across all dimensions was >3, considerably higher than other disciplines. The exception to this positive appraisal from Life Sciences was in relation to the Place concept, where some of the most negative perceptions across all concepts, dimensions, and disciplines were observed. Among all three disciplines, those from Physical Sciences displayed a generally more negative perception towards PIDs, with generally lower factor scores across all the tested concepts and their semantic dimensions. For example, the combined concept score (CCS) across all concepts – the overall perception measure – was lower than either Social Sciences or Life Sciences participants and was similarly lower based by dimension, with factor means of 1.27, 1.24, and 0.22 for evaluation, potency, and activity dimensions, respectively.

Osgood and others describe how the semantic distance can be charted using the distance notion (Osgood, 1957; Rosnow, 2000; Snider & Osgood, 1969; Stoutenborough, 2008). This can assist in identifying specific semantic dimensions or bi-polar adjectives within the semantic scales, which have triggered specific responses, and how those responses may relate to others. Figure 6 provides four semantic distance charts, one charting data across all participants (a) and three by discipline (b, c, and d). Data supplementing these charts are provided in Table 12. The semantic distance between concepts (D = distance) – derived using the generalized distance formula – is provided in Table 13.

Figure 6

PID perception measurements by all participants (a) and by discipline groupings (b = Physical Sciences; c = Social Sciences; d = Life Sciences), charted using the “semantic distance notion” approach. Charts plot the tested PID concepts according to their perception on the 13 bipolar adjective pairs comprising the semantic differential scale. Table 1 denotes how these adjective pairs were assigned to semantic dimensions (factors).

The significance of the chart profiles will be explored in more detail within the discussion section; suffice to state that visual inspection of the Life Sciences chart (D) demonstrates a greater propensity for extreme perception differences across semantic dimensions as well as on the tested PID concepts. By contrast, Social Sciences participants demonstrated generally consistent perceptions across the concepts, with fewer extremities noted. A tendency for the perception of different PID concepts to track each other can also be observed in Social Sciences. Within Physical Sciences, we note that People and Places track each other closely while Scholarly Communications and Things deviate from this chart profile, displaying comparatively irregular perceptions.

A Wilcoxon signed-rank test can be performed to examine D, as detailed in Table 13. This was used (α = 0.05) to determine whether D between specific discipline groupings were significant. Results are summarized in Table 14. Significant D differences were observed between Scholarly Communications and Things; Physical Sciences and Social Sciences demonstrated a significant distance to Life Sciences, with the former reporting significance at α = 0.05 (T = 3.58, z = 2.11, p = 0.02) and the latter at α = 0.01 (T = 3.74, z = 2.19, p = 0.01). Significant results were also observed for D between Places and Things, with the discipline groupings of Physical Sciences and Social Sciences notable (T = 2.52, z = 2.56, p = 0.01).

3.4 PID (re)use Habits

Recall that our research instrument concluded with five simple questions eliciting participant’s PID (re)use behaviour. On their (un)familiarity with using PIDs in scholarly communications, as measured on a 9-point bi-polar adjective scale (Unfamiliar – Familiar), participants reported themselves to be generally familiar, with median scores inferring considerable confidence (M = 7.68; Mdn = 9.00; SD = 1.81) (Table 15). On their understanding of the purpose of PIDs (Unknowledgeable – Knowledgeable), participants also considered themselves generally knowledgeable (M = 7.29; Mdn = 8.00; SD = 1.89), although with less confidence than their familiarity. This observation held across all discipline groupings and all job groups, except “Others” which reported equal familiarity and knowledge (Table 16). Results indicate less confidence from Physical Sciences relative to Social Sciences and Life Sciences (Table 15) but also within specific job groups, most notably Research Assistant (Table 16). Results indicate that Social Sciences participants considered themselves most familiar and knowledgeable about PIDs. Incrementally higher levels of dispersion from the mean can also be observed as job “seniority” declines. For example, (un)familiar and (un)knowledgeable for Professor/Reader (SD = 0.41; 0.41) and Others (SD = 3.25; 3.25).

Participants’ views on the purpose(s) of PIDs are detailed in Table 17, with >80% of participants noting that they exist to ensure the persistent and unambiguous citation of scholarly entities on the web. The use of PIDs as a way of mitigating “link rot” or “reference rot” in scholarly communications was also noted (>78%). The importance of PIDs in contributing to global scholarly graphs was, however, noted as a consideration for only 49% of participants.

Specific questions on the creation and (re)use of PIDS over the past 4 years indicated that close to 74% of participants reported creating a PID to identify a preprint or accepted author manuscript, with 54% reporting that it had been subsequently reused (Figure 7). Many participants indicated their creation and reuse of PIDs for people, at 57% and 49%, respectively. The creation and reuse of PIDs for identifying (open)research data were also noted, at 36% and 35%, respectively, with a longer tail of responses relating to PIDs for software, projects, and so forth. Few participants reported use of PIDs to identify research instruments (3%) or research equipment (1%).

Figure 7

Participant reporting on entities for which PIDs had been created and (re)used over the past 4 years.

4.1 PID Familiarity and “Identifying Identifiers”

First, although results from the PID recognition challenges were varied, it is possible to conclude that many participants across all discipline groups failed to satisfy the components of what we defined as PID literacy. The first four PID recognition tests challenged users to identify common PID types within context (e.g. within real-world published articles). A generous interpretation of data indicates that even in Life Sciences, where participants’ performance was best, circa 15% of responses were still incorrect. Rates were much lower in Physical Sciences (30%) and Social Sciences (34%), highlighting that almost one third of participants in these discipline groupings were unable to correctly identify PIDs, as they might commonly be presented within a scholarly journal article or repository. In the case of Social Sciences, we can also conclude that this result is statistically significant insofar as it stresses a distinct disciplinary divide between the aptitude of participants from Life Sciences and Social Sciences in this respect.

Closer examination of the results helps us to observe a level of uncertainty among scholarly participants when challenged in the four tasks. Despite many participants identifying the PIDs correctly in the challenges, a wide variety of accompanying erroneous responses were also often provided. For example, data for task #1 – the correct responses for which were “Publications (on a publisher website or platform)” (i.e. DOI) and “People (e.g. authors, editors, PIs, etc.)” (ORCID) – revealed that some of those who identified the two correct responses also included additional incorrect responses (equivalent to 21.6% of cases). It is worth adding that the responses to the challenge in task #1 included two of the most widely used PIDs (i.e. a DOI identifying a publication and ORCIDs identifying authors of that publication). That some participants considered these PIDS – that were contextualized within a real-world academic article – to also identify projects, research grants, a publication on a repository, etc. is a concerning indicator of PID literacy. Suffice to state that many participants who submitted correct responses were similarly uncertain about whether these PIDs identified other entities too. This is significant because such responses inadvertently reveal that these individuals did not appear to understand the notion of PIDs as unique identifiers.

One possible explanation is that social scientists typically publish less often (Hicks, 2013) and are more likely to view long-form publications as a vehicle for research dissemination (De Filippo & Sanz-Casado, 2018), resulting in reduced exposure to the emerging centrality of PIDs to scholarly communications. They are also less likely to be in receipt of research funding and are less likely to generate research data (Curty, 2016; Jarolimkova & Drobikova, 2019), both of which might ordinarily bring them into contact with adjacent PID concepts; for example, FAIR data (David et al., 2020), research funders (Lammey, 2020), software (Li, Lin, & Greenberg, 2016), research instruments (Stocker et al., 2020), and so forth. Such an explanation may prove unsafe in the long-term as recent evidence suggests that the publication behaviours of social scientists may be evolving in line with a global growth in national research evaluation exercises (Savage & Olejniczak, 2022). However, this does not explain the performance of participants from Physical Sciences. Though their performance was not found to be statistically significantly different to Life Sciences, visual inspection of data indicates that they were only marginally better than those in Social Sciences.

While disciplinary differences were clearly observable, disentangling job role effects from these is difficult without a larger sample to better represent individual job roles. No statistically significant differences between job role performance were detected. We can at least infer, perhaps unsurprisingly, that users’ CSE appears to have little connection to PID literacy. The CSE benchmarking and the analysis of the results found no significant differences between disciplinary or job role groups, suggesting that participants represented a relatively consistent level of computer efficacy. Their level of PID literacy, as measured in this research, therefore appears independent of this efficacy. However, we can note in the analysis of erroneous responses (as segmented by job role) that PhD students were particularly more likely to offer erroneous responses and that the number of these erroneous responses tended to be higher. The embryonic nature of a PhD student’s research career dictates a relative lack of experience with scholarly publishing and scholarly infrastructure, which may explain what was observed (Hatch & Skipper, 2016). It may also be reflective of the levels of digital illiteracy observed in the reviewed literature. The recent emergence of “researcher development” initiatives at research organizations has highlighted the need to better equip postgraduate researchers for their research career (Rospigliosi & Bourner, 2019), with useful examples emerging from research library contexts where training has coalesced with aspects of information literacy education (Fazal & Chakravarty, 2021). Though such initiatives are likely to assist researchers in navigating many aspects of the publication lifecycle and stress digital scholarship competencies, it is conceivable that the emerging centrality of PIDs to scholarly communications has been absent from 2020s teaching content.

Results from the additional six test challenges appeared to confirm our findings from the previous four challenges. Low recognition of esoteric PID types was expected (e.g. URNs and ISNIs). While DOIs and ORCIDs were the most strongly recognized PID types, it is somewhat surprising that PID types of such ubiquity and visual distinctiveness were recognized by only 68% and 63% of participants. This may signify that some researchers display a disconnection with two of the longest standing and most widely used PID types, perhaps because of their publication culture (e.g. publication venues do not support DOIs or ORCID) and, as statistically confirmed, their job role precludes them from experiencing them (e.g. their role is such that they lack exposure to these types). This, however, is an unsatisfactory explanation. At the time of writing, ORCID penetration within research organizations is high. More than 10 million ORCIDs have been added to the ORCID registry at the time of writing (Petro, 2020). Research funders and national research assessment exercises increasingly consider ORCIDs to be mandatory for research active staff (Choraś & Jaroszewska-Choraś, 2020). Most participants would be expected to therefore be ORCID literate, have an ORCID, or at least have recognition of them.

Uncertainty continues within this context too. For example, in indicating to which entity types the PIDs were most associated, DOIs – which can theoretically be coined for any web entity, but which tend to have specific applications within scholarship – were most noted as being associated with publications (77%), publications on a repository (37%), research data (19%), software (3%), and so forth. But ORCIDs were most noted for not only identifying people (68%), but also publications (13%), publications on a repository (14%), research data (5%), and other entities. This would indicate that even those individuals who recognize ORCIDs and perhaps even have an ORCID are unsure of their ultimate purpose. As we shall see in Section 4.2, PIDs as a mechanism to identify “people” were nevertheless perceived positively. Individuals therefore perceive ORCIDs to be a positive thing, irrespective of any confusion surrounding how they might work or cultural disciplinary differences.

The low recognition of Handles is notable, especially being statistically linked by discipline grouping (alongside URNs). Like DOIs, Handles (Handle.net) can be coined for virtually any web entity and forms the basis of a number of identifier types, such as RAiD (Janke et al., 2017). They are perhaps most commonly used in open repository infrastructure to ensure persistent identification of open research content (e.g. manuscripts of research articles, research data, etc.). The huge volume of open research content now being served by repositories (de Castro & CESAER, 2022) and their centrality to the wider open research ecosystem would suggest that users’ familiarity with Handle should be greater. Interpreting this lack of familiarity may be disciplinary, as statistically inferred. But it may also infer that some participants would tend not to use Handles when citing such content, instead rely on more transient URLs, and have therefore tended to ignore Handles when they are provided by repositories. This might be because they have not understood their purpose, although may also reflect the findings of the reviewed literature that some scholars’ struggle to correctly cite digital entities. The increased preference for DOIs to be used by repositories instead (cOAlition, 2022) is partly because they are considered more recognizable to users than Handles and therefore better contribute towards initiatives such as FAIR data (Dunning, De Smaele, & Böhmer, 2017) and Pubfair (Ross-Hellauer, Fecher, Shearer, & Rodrigues, 2019). Their data contribution to the PID graph is also currently superior (Cope, 2021).

4.2 PID Perceptions

Measuring participants’ perceptions of PIDs is important to our research motivation since such perceptions are likely to influence future PID user behaviour. Despite the findings associated with PID familiarity (RQ1) and Identifying identifiers (RQ2), it is significant to observe that participants perceived the use of PIDs to be generally a positive thing. This is especially clear in relation to the semantic dimensions of evaluation and potency across all the tested concepts, with Scholarly communications and People enjoying the highest perception ratings. Of the concepts tested, these are likely the most familiar to participants and therefore enjoy a level of demonstrable utility that Places and Things do not, owing to their association with the identification of scholarly publications and authors. However, it demonstrates that PID perceptions, in terms of participants’ evaluative attitudes and the potency with which these attitudes are held, are generally high. This observation is less true when we consider the activity semantic dimension, where factor scores were much lower. The activity dimension is an indicator that while positive perceptions exist on evaluative and potency terms, PIDs are perceived less favourably when action is required. For example, they are closer to being perceived by participants as “laborious” and “complex.” The significance of negative perceptions around activity become more obvious at a disciplinary level, particularly in Physical Sciences and Life Sciences, which held some of the most negative perceptions on this semantic dimension.

Of the PID concepts that were measured for perception, Places were universally perceived most negatively, irrespective of discipline. The persistent identification of places represents a typical PID application, for example, to identify research organizations such as universities or research funders. Interpreting this finding is difficult without additional qualitative data but we may speculate that the notion of PIDs for places was considered too abstract for some participants, whereas PIDs for People or Things was considered more tangible and more relatable to scholarly practice.

Results of PID perception measurements across disciplines revealed compelling differences. Overall, Life Sciences participants demonstrated the most favourable perceptions of PIDs, perceiving the use of PIDs to identify People, Things, and within Scholarly communications most positively, but they also demonstrated a greater inclination towards extreme perceptions on specific semantic dimensions or concepts. This was clearly observable from the corresponding semantic distance chart (D). For example, the People and Places concept yielded a perception rating of 3.75 and −1.00, respectively for the bi-polar pair, “Unintuitive–Intuitive” (belonging to the evaluation dimension). More extreme differences were observed across bi-polar scales belonging to the activity dimension (e.g. Complex – Simple, Laborious – Effortless, Difficult – Easy). The concepts of People and Places too were, in general perception terms, far wider apart than either Physical Sciences, Social Sciences, or all participants taken collectively. Also notable is the extent to which the profile of the semantic distance chart for People and Places covary, with the perception of one tracking the perception of the other but with large distances between them. In other words, participants in Life Sciences were the most likely to perceive PIDs for People positively but simultaneously the most likely to perceive PIDs for Places negatively. Conclusions about Life Sciences were corroborated with the measurement of D, with D calculated to be considerably higher across all but one concept when compared to other disciplines. That D was found to be statistically different to Physical Sciences and Social Sciences for specific concepts is noteworthy and highlights a potential perceptual distinction between the disciplines.

An explanation for this finding is difficult without additional qualitative data from Life Sciences participants, but we may suggest that it is fuelled by the rapid prominence of PIDs within Life Sciences. Whether through the recent but rapid growth in pre-printing (Johansson, Reich, Meyers, & Lipsitch, 2018; Sarabipour et al., 2019), a phenomenon accelerated by the COVID-19 pandemic (Fraser et al., 2021; Majumder & Mandl, 2020) and joined by an accompanying consciousness of FAIR data sharing approaches (Austin et al., 2021), or the recent proliferation of PID types for entities such as samples (Bandrowski & Martone, 2016; Lehnert & Klump, 2018), specimens (Hardisty et al., 2021), equipment (Haak, Meadows, & Brown, 2018), materials (Lehnert, Klump, Wyborn, & Ramdeen, 2019), instruments (Plomp, 2020), and so on, it is conceivable that the increased visibility of PIDs has exposed these researchers to their function. The increased exposure may also help to explain why stronger perceptions were found on specific concepts as well as specific semantic dimensions (i.e. activity).

By contrast, Social Sciences participants demonstrated generally consistent perceptions across the tested concepts, with fewer extremities noted. A cursory visual inspection of the corresponding semantic distance chart (C) shows this consistency, relative to both Life Sciences and Physical Sciences; confirmed in most of the calculations of D. But we should also acknowledge that activity was again the dimension attracting least positive perceptions. The differences of D between Physical Sciences and Social Sciences for Places and Things, though statistically significant, appear to be significant because of the generally lower perception ratings displayed by Physical Sciences to PIDs. Excepting an isolated outlier for Places on the activity dimension, those from Physical Sciences displayed a generally more negative perception towards PIDs across all the tested concepts and their semantic dimensions. Such a finding is counterintuitive since it might be expected that Physical Sciences participants would be more favourably disposed to PIDs than those in Social Sciences, particularly as the latter group demonstrated lower familiarity with PIDs in the recognition tests.

4.3 Habits

In interpreting results on PID (re)use habits, we can immediately observe that participants indicated high levels of familiarity and knowledge with PIDs (optimum levels if only median values are considered) but that this confidence was in no way reflected in the results of their corresponding PID recognition challenges, described in PID familiarity and Identifying the identifiers sections above. In other words, participants considered themselves to be generally familiar and knowledgeable about PIDs, despite simultaneously demonstrating that they were often both unfamiliar and unknowledgeable about them in practice. This is especially true of Social Science participants. But on these data, it appears that interpretation of the results by job role may be more meaningful, since far greater variation between roles than between disciplines is observable. Notwithstanding the reported high levels of familiarity and knowledge as per mean and median values for particular job role (e.g. Postdoc, Research Support, other), the high dispersion from the mean suggests high data spread – spread which appears to grow as job seniority declines. The incrementally higher standard deviation as job roles become less senior is again likely indicative of uncertainty among some participants. We should, however, acknowledge that it also suggests those at Professor/Reader or Lecturer level considered themselves to be more familiar and knowledgeable, and to be more consistent in this perception than, say, a participant from Research Support.

The more the PIDs are reused, the more the new nodes can be defined by existing PIDs, thereby enriching the graph. Our definition of PID literacy emphasizes the importance of users’ understanding of when PIDs should be used and when they should be reused. PID reuse is critical to the vitality of the resulting scholarly graph (Dappert et al., 2017). Typical enrichment might take place surrounding, say, an individual (via a person PID such as ORCID) or a research dataset (via explicit dataset linking from associated publications using a DOI). The recent rise in preprint publication (Lin, Yu, Zhou, Zhou, & Shi, 2020) presents a common point of PID creation for many researchers, since preprint deposit will often entail PID minting (i.e. DOI). It was therefore unsurprising to find that this was the most highly reported context for PID creation, even if reuse was some 20% lower. The creation of PIDs for research data may appear low at 36% but demonstrated high levels of subsequent reuse, most probably because research data are likely to form the basis of more than one associated research publication and/or be used repeatedly within a wider research project (Borgman & Wofford, 2019).

Accepting that only a significant minority of the study participants would have the need to create a PID for research data, software, or instrument, the proportion of participants who indicated reusing a PID for people was disappointing. Though several PID types exist for defining people, our research instrument was clearly alluding to ORCID, with which most participants would be familiar. That fewer than half of the participants indicated they had reused a PID for people was therefore a noteworthy finding, and one that should be the subject of further research. It is possible that the description of “PIDs for people” within the research instrument was too opaque for some participants to make the connection with ORCID, resulting in under reporting. However, it could also be a legitimate finding influenced by the disciplinary context of some participants. In the case of preprint PID reuse, is it that these PIDs became superseded by an accepted publication, rendering the PID defunct for reuse? Understanding participants’ PID reuse behaviour is something which requires an additional, separate piece of qualitative research, preferably through user interviews or protocol analysis approaches.

It is possible that these lower levels of creation and reuse are linked to responses surrounding the purpose of PIDs. PIDs perform a wide number of functions but no specific “PID purpose” was considered universally relevant by participants (Table 17), thereby demonstrating a degree of PID illiteracy among participants. The notion of PIDs as enabling “persistent and unambiguous citation of scholarly objects” was a purpose that only circa 80% considered core. The importance of PIDs in contributing to global scholarly graphs was therefore considered even less relevant, at 49% of participants. With no widely accepted understanding of why PIDs exist and why they are necessary, it seems apparent that unsatisfactory and erratic (re)use of PIDs will always occur among scholarly users.