Adding Perspective to the Bibliometric Mapping Using Bidirected Graph

1 Introduction

Bibliometrics is widely recognized as a tool for analyzing patterns in a bibliographical collection. With common techniques such as citation analysis, co-citation analysis, bibliographic coupling analysis, co-word analysis, and co-authorship analysis, bibliometric analysis is an emerging method for synthesizing research that focuses on the bibliographic information (Gan, Li, Robinson, & Liu, 2022). Several applications available provide the convenience of conducting bibliometric analysis that can be done independently or as a service by some libraries. VOSviewer, Bibliometrix, CiteSpace, SciMat, and Sci2 Tool, for example, are software that enable users to create maps based on network data and then visualize and explore those maps.

Mapping of networks visualizes multiple items (e.g., authors, journals, or keywords), and an edge represents a relationship between two items (Waaijer & Palmblad, 2015). This suggests that there is a unified relationship between the two keywords without regard to how keyword A perceives keyword B and vice versa. The study conducted by Li (2017) showed that keyword relationships can vary depending on the metrics used. This demonstrates that if any other measurement is used, another line that examines the relationship of each item must be considered.

2 Literature Review

3 Method

This study used 141 data entries from the Journal of Data and Information Science from 2018 to 2022 taken from Scopus. The data used in this study are in the form of author keywords, which were used to determine the relationship of keywords to one another. Keyword identification is made to find out similar keywords using lemmatization. By employing the lemmatization approach, the words used will be able to identify their basic form (Korenius, Laurikkala, Järvelin, & Juhola, 2004). Because some keywords may have the same meaning but in different forms, keyword cleaning is required (Gan et al., 2022; Santosa, 2023). The WordNet Lemmatizer from the Natural Language Toolkit was utilized in this study. Furthermore, because 12 entries lacked author keywords, only 129 were processed further.

In this study, FP-Growth is used to generate itemsets, which is known to be capable of generating frequent itemsets without generating candidates (Naresh & Suguna, 2019). Additionally, FP-Growth can perform better than pattern generation-based algorithms in terms of performance (Han, Pei, Yin, & Mao, 2004). This study established a Support value of 0.01 and a Confidence value of 0.21. The maximum number of itemsets that can be generated is set at 3, which allows seeing the relationship between numerous keywords. Furthermore, this research uses the Python library Mlxtend (machine learning extensions) to construct FP-Growth to extract frequent itemsets for AR on keywords. Visualizations are made using the Python library Streamlit Agraph, where each node represents a collection of itemsets (a single keyword or a group of keywords). Meanwhile, on the VOSviewer side, Full Counting and Fractional Counting were performed on author keywords using the co-occurrence analysis type.

4 Result and Discussion

As a result of the cleaning of the data, the original 533 keywords were reduced to 523 keywords. Some words were combined such as [Citation context] and [Citation contexts], [Research information system] and [Research information systems], and [Universities] and [University]. However, because this process failed to detect [scientometric] and [scientometrics], [performance-based funding] and [performance based funding], and [Performance-based Research Funding System] and [Performance-based Research Funding System (PRFSs)], changes had to be made manually. Apart from the instances where it encountered problems with synonyms and polysemies, this demonstrates the inadequacy of lemmatization. Keyword expansion is one method considered to address this issue (Ayetiran, 2020) or using other applications such as OpenRefine for maximum results. The top five keywords were bibliometrics, scientometrics, research evaluation, Norwegian model, and Twitter.

Both methods can display network visualizations with varying calculations. As shown in Figure 1, the network provided a single line of relationship between two points that VOSviewer produces. As a result of the AR-generated networks shown in Figure 2, two lines with a two-way relationship were produced. Different information is contained in each formed line. AR-generated networks can be chosen based on their Support or Confidence value. The two bibliometric maps also show that four of the five most popular keywords have a direct relationship and form a network group, whereas the Twitter keywords do not have a direct relationship and form their own network.

Figure 1

VOSviewer’s network visualization.

Figure 2

Bidirected graphs utilizing AR.

The difference occurs in the relationship between bibliometrics and the top three keywords (scientometrics, research evaluation, and Norwegian model). In VOSviewer, the resulting maps take the form of undirected graphs, with a connecting line for each keyword that has a relationship (Figure 1). While it can be seen in the bidirected graphs that these three keywords have lines that all point to bibliometrics, there is a missing link from bibliometrics to these three keywords (Figure 2). This occurs because the observed relationship does not meet the specified initial parameters because the resulting Confidence value is 0.18, which is less than the specified threshold of 0.21. It also appears that some nodes have two relationship lines, indicating that the relationship between nodes meets the value with the specified parameters.

With predefined parameters, AR generated as many as 33 relationships where 11 of them are relationships between single keywords and multiple keywords (Appendix). By adjusting the maximum number of itemsets generated, relationships between one and many keywords can also be created. While VOSviewer was unable to view one-to-many relationships, AR can give users a broad overview of the relationship values between different keywords. Using the resulting Antecedent and Consequent Support values, it appears that [bibliometrics, performance-based funding] and [performance-based funding, Norwegian model], with a value of 0.023, are the paired keywords that frequently appear. Both bibliometrics with performance-based funding and Norwegian model with performance-based funding have a Full Counting value of 3 and are the highest compared to other relationships when expressed through a one-to-one relationship. Apart from the fact that these two calculations depend on occurrences in data, this is what makes the relationship between the two keywords in VOSviewer comparable to the value of the Support keywords in pairs.

Full Counting was directly related to Support value of the 33 rule examples used. This is so because calculating the frequency is a fundamental component of both calculation models. The four relationships in numbers 14–17, as shown in Appendix, have the highest Full Counting and Support values. It can also be concluded that the Support value can be used to define link strength from one relationship to many keywords. Meanwhile, the value of Support does not differ between the relationships x → y and y → x, so using bidirected graphs to analyze the emergence of a keyword relationship is less precise.

At the same time, the lowest Confidence value is 0.250, and 11 relationships have a maximum value of 1. When Antecedents and Consequents switch positions, numbers 3–4 and 7–8 show the same perfect score, implying that [university, ranking] are interdependent and appear in the same document as [development of data and information services, higher education institution]. The Confidence value also has an impact on one-to-many relationships, such as [research information system] and Consequents [performance-based funding], [Norwegian model], and [performance-based funding, Norwegian model], where the one-to-one relationship in numbers 25–26 has the same value as 32.

The Confidence value, an useful metric to show the strength of the implications (Stit, Riffi, Yahyaouy, & Tairi, 2018), demonstrates what sets the relationship apart from x → y and y → x. For instance, [social medium] → [Twitter] has a value of 0.667, indicating that [social medium] is more frequently found in documents containing both keywords than [Twitter], which has a lower value of 0.40. Some keywords only have one line due to the difference in the relationship that Confidence displays because the resulting value does not meet the required threshold. Given that it will be challenging to distinguish different values from the relationship between the same two keywords using undirected graphs, it can be concluded that bidirected graphs are necessary to see this metric.

Lift refers to the potential influence of Antecedent keywords on the occurrence of Consequent keywords (Li & Chu, 2017). Additionally, the value of Lift is equal to one if there is no dependence between the items (Iqbal et al., 2022). There is no Lift value that is worth 1 of all the relationships that appear, and the value of 64.5 is the highest value that appears in numbers 4–5 and 8–9. This occurs because these keywords are always found in pairs in existing documents, allowing the Lift to be used to identify opportunities for topics that have not been exposed to other topics due to the fact that the keywords used are always found in pairs.

Furthermore, the relationship between x → y and y → x in Lift is constant. Because the component for calculating the Lift in the numerator is Support, which is always the same, and in the denominator it is the multiplication between Support (x) and Support (y), which is reversed, the result will still be the same, the position of the Lift value will remain constant for each keyword exchange. This similar relationship suggests that the use of bidirected graphs will be less advantageous because the lines formed have the same value, making the use of undirected graphs simpler.

The maximum Confidence value for 11 relationships results in an infinity value when using the Conviction metric. In interactions where the same values of Lift and Confidence are present, the value of Conviction will always be the same (Prajapati et al., 2017). The relationship between [scientometrics] toward [bibliometrics] appears to have the lowest value with a value of 1,22 where the probability of appearing randomly is greater when compared to [performance-based funding] toward [Norwegian model] with a value of 3,845 where it can be concluded that this relationship is more likely strong and not a mere coincidence.

Brin et al. (1997) stated that Conviction is a true measure of implication because it is directional. It is very appropriate to use bidirected graphs to visualize the relationship that is formed between the two items in Conviction because, like Confidence, the relationship between the two items in Conviction has the potential to produce different values. But interpreting infinity as a number brings a problem for Conviction. Contrarily, Confidence has a distinct range between 0 and 1, so it will be more appropriate to use Confidence rather than Conviction when attempting to adjust the style of a line, for instance by varying the line’s thickness.

5 Conclusions

This study demonstrated how a bidirected graph using AR could enhance keyword network analysis, particularly in terms of how keywords can support one another. The value of Support can indicate the frequency of occurrence of two keywords in a set of documents. In contrast, Confidence can help assess the presence of two keywords in documents that include one or more of these keywords. The Lift value can also be used as an input to determine the strength of the relationship from existing keywords; the higher the value, the stronger the relationship. On the other hand, there were numerous relationships between keywords that have the highest Lift value. Note that in this study, this can occur if two or more related keywords appear only in a few documents. This case affects the value of this high Lift, so more investigation into the maximum score on the Lift is required. Then, seeing Conviction can provide additional insight into the strength of keywords and how they may appear by coincidence.

Given that both Support value and Full Counting consider the presence of keywords in the data they have, they have a linear relation. The Support and Lift values between the two keywords are monotonous according to the four metrics used, making it simpler to use undirected graphs. However, the other two values, namely Confidence and Conviction, have variable values and view the relationship from two different angles, making the use of bidirected graphs unavoidable. In addition, Conviction has a problem with the infinity value, so variations in line thickness are more likely on Confidence values with definite range.

It is also possible to have one-to-many entity relationships, where the number of keywords is determined by how many itemsets will be produced. The disadvantage of a bidirected graph is that the more data there are, the more complicated the network becomes. In order to make it simpler to understand the relationship that occurs, it is, therefore, necessary to set the parameters used. Using new calculation techniques can add value to using this visualization, so this study is limited to using AR to see metrics that support bidirected graphs. Additionally, metrics that detect the presence of a keyword, like the value of Support, tend to have a smaller value, the larger the dataset used, requiring the derivation of the necessary parameters in order to detect a relationship.