Another Approach to Evaluating Scientific Collaboration

Adedayo: Another Approach to Evaluating Scientific Collaboration

Authors

INTRODUCTION

One of the top interests in science policy is the issue of research collaboration.[1] Collaboration is an arrangement between two or more people or organizations to work together to realize or achieve a goal. Research collaboration is a sophisticated cooperative arrangements among individuals, groups, departments, institutions, sectors and countries.[2,3] and it has become the norm in every field of scientific research.[4,5] Many studies have attempted to investigate various roles of collaborative research. Specifically, investigations have been made to ascertain whether scientific collaborations produce some of the best quality science.[6] Whether research collaborations have effects on publication productivity, i.e. do those who collaborate more tend to have more publications.[4] Also, if collaborations breed a transfer of knowledge among collaborators.[7] The effects of leadership styles on the impact collaborative work.[8] Etc. As a result, many services aimed at quantification of extent of collaboration are now available. These include the Collaboration Score of Nature Index.[6] Collaboration Metrics of Centre for Science and Technology Studies (CWTS), Leiden; etc. Measuring collaboration is now considered an indicator of research performance, as the Weighted Fractional Count (WFC) of Nature Index is widely applied, and can be used to identify the rising stars performers in the research world.[9,10]

Lee and Bozeman,[4] Bozeman et al.[11] Brew et al.,[12] and Katz and Martin.[3] Have all reported positive effects of collaboration on scientific productivity. The arguments are that: much collaboration is based on the joint use of expensive or unique equipment without which research would be, not only less productive but also impossible; some researches require collaboration to bring special expertise and knowledge not otherwise available but crucial to research outcomes. Often, tacit knowledge and knowledge of technique are best conveyed through collaboration; particularly for mentoring students and trainee researchers/scientists;[4] and to support innovation and address new and unmet needs, the value of cross-border collaboration and networking is important.[13] Sangam and Arali.[14] studied growth versus scientific collaboration in the field of genetics, using scientometric analysis. Their study found out that there is a relation between growth and scientific collaboration in the field of Genetics.

However, despite these good reasons to expect positive outcomes in scientific collaborations; equally, there are propositions as to why collaboration may undermine productivity. Landry and Amara.[15] Cautioned that transaction costs are usually an unavoidable consequence of working with others. Staying in touch by various media, engaging in social ingratiation, waiting for others to comment, respond, or do their part of the research - these are just some of the factors taking time and energy even in the best collaborative relationships4.

While it is researchers rather than institutions/countries who make arrangements to work together, these collaborations are aggregated and scaled-up with the focus of most collaboration evaluation on institutional/national levels.[16, 11] Some studies have attempted to establish links between collaboration and research productivity. Studies focusing on measuring and evaluating collaborative performance/strength of individual researchers are scarce. The prominent methods of evaluating individual researchers include: g-Index;[17] h-Index;[18]i10 – Index,[19] The individual scientist ranking scheme of Webometric, developed by CSIC - the Spanish National Research Council.[20] All these schemes measure research indices which are different from collaboration. The conceptual frameworks used by many of the available collaboration metric services are also grossly in error and unsuitable for evaluating individual researchers. For example, the scheme used by collaboration metrics of CWTS, Leiden is abstrusive. Information on operational principles of the methodology is not made public. The Nature Index/ Collaboration Score dwells on three measures to evaluate collaborations. These are: Article Count (AC), Fractional Count (FC); and Weighted Fractional Count (WFC). The flaw here is that, for AC, a publication is given a score of 1 unit irrespective of the number of authors listed on the publications [6]. The argument is that, if the number of authors listed does not affect the score attributed to a publication, then, of what value is collaboration? The proper conceptual thought line should be that quality of publication improves with the number of listed authors; this because the input of every contributing author should add to the quality of the publication. Where an author has not added to the quality of a publication, then what is he/her contribution to justify being listed? It is a widely known adage that two heads are better than one. This adage is widespread, cross-cultural, and equally supported by the Holy Bible (Ecclesiastes 4: 9-12). Definitely, the output is bound to be of higher quality where there is more quality inputs.

As regards FC, all contributing authors are attributed with a uniform count, which is determined by diving 1 unit with the number of contributing author. For instance, where a publication has 10 authors.[6,16] then it means that each author receives an FC of 0.1. This idea is equally flawed, because it is a fact, generally known in scientific publishing that the order of author listing is indicative of the extent of contribution/influence of the authors as per the published research. If order of author listing is irrelevant, author listing would rather follow an alphabetic order, which is not the case in real time publishing. Weighted Fractional Count (WFC) is a normalization of overrepresentation of paper, and it applies to the field of Astronomy. [6, 16]

In the light of foregoing observations, another approach to evaluating scientific collaborations of individual scientists is presented. In this present study, the collaborative strength of scientists at the Centre for Science and Technology Studies (CWTS), Leiden is investigated. The approach in the new scheme introduces a new feature which considers the position of scientists in the author list of their published works. With this perspective, the study is original, and has great potentials. Herein, the justification for the study is identified.

Methodology

Co-authorship pattern of academic staff at the CWTS, Leiden in Netherlands was studied. Only published academic staffs at the CWTS were investigated. The information on co-authorship pattern of these scientists was as obtained at the following link: https://www.cwts.nl/people as at 25th November, 2016. In all, there were about twenty eight academic staff at the CWTS, Leiden; however only twenty-one academic staffs have records of publication history.

The co-authorship of each publication as recorded against the scientist at the website is obtained. A simple count of the total number of authors listed on a particular publication is counted and recorded as n; the position of the scientist in the author list is recorded in ascending order as r; starting with the first author listed. A record of distribution of n and r was obtained for the academic staff of CWTS who had publication records. The collaborative strength of individual researchers is determined using the relationship expressed as follows.

(1)
Cs=i=1P(niri+1)4

Where CS is the collaborative strength, P is the total number of publications of the scientist and i is indicative of a particular publication of a scientist. The mean of the number of persons collaborating per paper (nmean) and mean of the positions of a specific research staff (rmean) were calculated using the following expressions

(2)
nmean=i=1PniP
(3)
rmean=i=1PriP

The adoption of co-authorship for measuring collaboration was informed from the premise laid by Katz and Martin,[3] which was similarly adopted by Gal et al.[13] Bozeman et al.[11] and Voutilainen and Kangasniem.[2] Count of number of listed authors and total number of publications have also been used in Nature Index.[6,9,10,16]

Table 1

Distribution of number of persons (n) collaborating per publication.

S/NCCMRCNJEJKTLIMENTRSRARCTRTVTMVLWIWEWJWPWAYZZ
1.4321043535543144222533
2.2334541032212344433333
3102410343251338102442324
4232 33332272332 22523
5235 3 452  335232 333
6432 5 311  325432 333
7432 1 511  425233 524
8452 1 2102  6 524  1034
9243 4 232  1 323  231
10324 4 422  10 433  232
11424 2 43   3 354  323
12432 2 23   3 523  333
131032 3 13   3 423  4 3
14232 3 33   3 524  3 3
153410 1 35   3 52   2 3
16 23 2 35   1 33   9  
17 23 2 25   1 44   2  
18 34 10 23   3 44   2  
19 23 3 33   2 101   2  
20 32 3 23   6 32   1  
21 33 3 25   1  2   1  
22 33 5 104   3  2   1  
23 33 5 31   1  10   2  
24 43 5 31   1  3   2  
25 25 2 51   2  3   2  
Table 2

Distribution of number of persons (n) collaborating per publication (Annex).

RCNJETLTRRTLWPW

S/NnS/NnS/NnS/NnS/NnS/NnS/NnS/NnS/NnS/NnS/NnS/NnS/NnS/NnS/Nn
2622655122635137631013261511763264514262512263
27327552227352277410242715227710275522272522273
282282533283532784103 281532783281534283534283
292292543293545792104 291544793291542292542293
302304552305554803105 301551804301553303552303
3123145610316562812  311561815313563313564314
322325573325575822  3215718243215710323572322
332333583334583833  33158183 335583335583333
342342593343593842  34559184 343593345593343
353353602351602854  35160285 352602355602353
3633626123616128610  36461186 36561236261103610
373372623376625873  37362187 372622374623371
383382633381633882  38563188 383633384632381
392392643393643893  39264189 39164539564239 
404402654401653903  40565290 40265340265240 
412412663411663912  41466291 41266 41366241 
423422674423674922  42567492 42367 42267442 
43243268 432682932  43368293 43268 43268343 
44444269 444693943  44169294 44269 44269344 
45345270 455702955  45170295 45470 45270445 
46346271 462712964  46171296 46171 4627146 
47347272 475723975  47172297 47272 4727247 
48 48273 484732984  48173198 48273 4827348 
49 49274 493742993  49174299 49174 4927449 
50 50275 5057541002  503752100 50475 5027550 
Table 3

Distribution of position (r) of authors listed in a publication.

S/NCCMRCNJEJKTLIMENTRSRARCTRTVTMVLWIWEWJWPWAYZZ
1.312321331332144221232
2.212221421212143132312
3222324224123392141323
4221 21232172131421211
5121 2 331  324222 331
6122 3 311  314322 322
7112 1 211  414222 322
8222 1 182  4 313  1032
9242 1 232  1 313  231
10222 3 222  5 321  231
11322 2 43   3 121  211
12322 2 23   1 212  321
13232 2 11   2 213  4 1
14212 2 21   3 411  3 1
15126 1 15   2 41   2 1
16 13 1 15   1 31   9  
17 12 1 15   1 41   1  
18 23 7 13   1 41   2  
19 13 2 13   2 91   2  
20 11 2 23   3 21   1  
21 13 2 21   1  1   1  
22 13 3 44   3  1   1  
23 12 3 21   1  1   1  
24 12 3 21   1  1   2  
25 12 2 51   1  1   1  
Table 4

Distribution of position (r) of authors listed in a publication (Annex).

RCNJETLTRRTLWPW

S/NrS/NRS/NrS/NrS/NrS/NrS/NrS/NrS/NrS/NrS/NrS/NrS/NrS/NrS/Nr
2612625122625137611012261511763261514262512263
2712725222725217731023271522778274521272521273
281282533282532783103 281532783281531282532283
291291543292542791104 291544793291541292541293
301301552302552802105 301551804301551301552303
311311566313561812  311561815311561311563314
321324573322571821  321571824321575321571322
331333581333581831  33158183 333582331581333
341342591342593842  34559184 341593341592342
352352601351601851  35160285 352602351601353
361361612361611867  36161186 3636113626113610
372371621374621872  37362187 371622372622371
383382632381632882  38563188 383633382631381
391392641392642892  39164189 39164439564239 
401402651401651902  40565290 40165140165140 
412411661411662911  41466291 41166 41166141 
422421672421672922  42567492 42167 42267242 
43143168 432682931  43368193 43168 43268243 
44244269 443692942  44169294 44169 44169144 
45145170 455702955  45170295 45270 45170345 
46146271 461712962  46171296 46171 4617146 
47147172 471723972  47172297 47272 47272 47 
48 48273 483732982  48173198 48273 4827348 
49 49274 492741993  49174299 49174 4917449 
50 50175 5027531001  503752100 50475 50175 50 
Table 5

Full names of CWTS Scientists Investigated.

S/NNamesInitialsNumber of PapersnmeanrmeanCs
1.Clara Calero-MedinaCCM154.001.932.60
2.Rodrigo CostasRC472.771.473.22
3.Nees Jan van EckNJE673.001.973.41
4.Joost KostenJK38.002.672.09
5.Thed van LeeuwenTL1023.022.043.86
6.Ingeborg MeijerIM43.501.751.82
7.Ed NoyonsEN263.542.192.79
8.Tong van Raan TR822.562.323.18
9.Sarah de RijckeSR102.401.702.03
10.Alex RushforthAR42.501.751.63
11.Clifford TatumCT43.753.251.57
12.Robert TijssenRT652.861.973.33
13.Vincent TraagVT73.141.432.09
14.Martijn VisserMV204.703.802.48
15.Ludo WaltmanLW702.941.563.59
16.Inge van der WeijdenIW143.432.002.41
17.Erik van WijkEW72.572.431.68
18.Jos WinninkJW42.251.251.68
19.Paul WoutersPW382.972.632.67
20.Alfredo YegrosAY122.672.172.06
21.Zohreh ZahediZZ153.671.472.48
Figure 1

Frequency distribution of n for the study.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g001.jpg
Figure 2

Cumulative frequency of n for the study.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g002.jpg

RESULTS AND DISCUSSION

Tables 1 and 2 present the distribution of number of persons (n); collaborating per publication. These Tables give the overview of n distribution for the study. Similarly, Tables 3 and 4 present the distribution of position (r) of authors listed in a publication. The Tables provide overview of distribution of r of listed authors in a publication. In Table 5, the full names of the initials of CWTS scientists investigated in this study are presented. In this Table, the total number of publications, nmean, rmean and CS for each staff is indicated.

The frequency distribution and cumulative frequency of number of persons (n) collaborating per publication for the study are presented in Figure 1 and 2 respectively. From these Figure, it can be seen that, CWTS scientists collaborate mostly in groups ranging from 1 to about 5 persons in a group. Specifically, the lower quartile from Figure 2 shows that about 25% of the papers are published by groups consisting of 1 or 2 persons; while the upper quartile from Figure 2 shows that about 75% of the papers are published by groups consisting of 1 to 4 persons.

Figure 3

Frequency distribution of r for the study.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g003.jpg
Figure 4

Cumulative frequency of r for the study.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g004.jpg
Figure 5

Distribution of collaborative strength.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g005.jpg
Figure 6

Distribution of average collaborations.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g006.jpg
Figure 7

Distribution of average position ranks.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g007.jpg
Figure 8

Distribution of total number of publications by each scientists.

https://s3-us-west-2.amazonaws.com/jourdata/jscires/JScientometRes_6_3_195_203-g008.jpg

Similarly, Figure 3 and 4 illustrate the frequency distribution and cumulative frequency of the positions of CWTS scientists in collaborative researches. From Figure 3, it is clear that these scientists are mostly listed between the first and third authors listed on the publications studied. It is in about 40% of the collaborations that CWTS staffs have been listed as first author.

Table 5 and Figure 5 to 8; present information on collaborative strength, average collaborations, average position ranks, and total number of publication per scientist. Overall, Thed van Leuween has the highest collaborative strength. He has published about 102 scientific articles, which translates to the fact that he has utilized about 102 opportunities for scientific collaborations. His average collaborations per publication (nmean) and average position rank (rmean) are 3.02 and 2.04 respectively. This implies that ordinarily, Thed van Leuween collaborates with about three persons per publication and is mostly either the first or the second author listed. Although in Figure 6 and 7; Joost Kosten has the highest average collaborations per publication while Jos Winnink has the strongest position rank, however, the total number of publications by these staffs is rather very small, and the reason for their observed weak collaborative strength.

Comparing Tong van Raan and Ludo Waltman, overall, Ludo Waltman has better collaborative strength despite that Tong van Raan has published more. Ludo Waltman published a total of 70 papers, while Tong van Raan published 82 papers. However, overall, Ludo Waltman has better collaborative strength because on the average, Ludo collaborates with about 3 persons per publication and is mostly the first listed author, where as Tong van Raan collaborates with about 2 persons per publication and mostly listed as the second author.

Ingeborg Meijer; Alex Rushforth; Clifford Tatum and Jos Winnink have all published the same number of publication which is 4. Ingeborg Meijer has the highest collaborative strength because he publishes with 3 or 4 persons per publication, and is mostly listed as the second author. Jos Winnink is next to Ingeborg Meijer. Although, Jos Winnink collaborates mostly with 2 persons, however, he is mostly listed as the first author in publications. Alex Rushforth is in third place. He collaborates mostly with 2 persons and is mostly listed as the second author. Clifford Tatum is last in this set. Although he collaborates mostly with 3 or 4 persons, however, his collaborative strength is weak because he is mostly the last or second last author listed.

CONCLUSION

A new scheme for evaluation of scientific collaboration has been introduced. The importance of total number of publications on the measure of collaboration as reported in earlier studies is upheld by the study. The role of number of persons collaborating in a particular research, and the relative positions of the collaborators in their published research is also confirmed relevant and important to overall measure of scientific collaborations.

ACKNOWLEDGEMENT

None.

Notes

[1] Conflicts of interest CONFLICT OF INTEREST None

REFERENCES

1. 

Basu A, Kumar BSV , authors. International Collaboration in Indian Scientific Papers; Scientometrics. 2000;48(3):381–402. DOI: 10.1023/A:1005692505687

2. 

Voutilainen A, Kangasniemi M , authors. Applying the ecological Shannon’s diversity index to measure research collaboration based on co-authorship: A pilot study. Journal of Scientometric Research. 2015;4(3):172–7

3. 

Katz JS, Martin BR , authors. What is research collaboration? Research Policy. 1997;26(1):1–18

4. 

Lee S, Bozeman B , authors. The Impact of Research Collaboration on Scientific Productivity. Social Studies of Science. 2005;35(5):673–702

5. 

Beaver D, Rosen R , authors. Studies in Scientific Collaboration: Part III - Professionalization and the Natural History of Modern Scientific Co-authorship. Scientometrics. 1979;1(3):231–45

6. 

Nature Index , author. A guide to the nature index. Nature Index. 2015;S83

7. 

Perez OH , author. Collaborative Information Behaviour in Completely Online Groups: Exploring the Social Dimensions of Information in Virtual Environments. Qualitative and Quantitative Methods in Libraries (QQML) Journal. 2015;4(5):775–87

8. 

Shah C , author. The blind leading the blind: Impromptu leaderships influenced by awareness in collaborative search. Aslib Journal of Information Management. 2016. 68(2):p. 212–26. DOI http://dx.doi.org/10.1108/AJIM-08-2015-0125.

9. 

Philips N , author. Nature Index 2016 – Rising Stars. Nature. 2016;535(7613):S49–S88

10. 

Nature Index. Nature Index 2016 – Rising Stars. 2016;S49

11. 

Bozeman B, Fay D, Slade CP , authors. Research collaboration in universities and academic entrepreneurship: The state of the art. Journal of Technology Transfer. 2013;38(1):1–67

12. 

Brew A, Boud D, Lucas L, Crawford K , authors. Reflexive deliberation in international research collaboration: Minimizing risk and maximizing opportunity. High Educ. 2013;66(1):93–104

13. 

Gal D, Glanzel W, Sipido KR , authors. Mapping cross-border collaboration and communication in cardiovascular research from 1992 to 2012. European Heart Journal (Special Article). 2016;(16):1249–58. DOI: 10.1093/eurheartj/ehw459

14. 

Sangam SL, Arali U , authors. Growth versus scientific collaboration in the field of genetics: A scientometrics analysis. COLLNET Journal of Scientometrics and Information Management. 2016. 10(1):p. 9–19. http://dx.doi.org/10.1080/09737766.2016.1177938.

15. 

Landry R, Amara N , authors. The Impact of Transaction Costs on the Institutional Structuration of Collaborative Academic Research’. Research Policy. 1998;27(9):901–13

16. 

Grayson M, Pincock S , authors. Nature Index 2015 – Collaborations. Nature. 2016;527(7577):49

17. 

Egghe L , author. Theory and practise of the g-index. Scientometrics. 2006;69(1):131–52. DOI: 10.1007/s11192-006-0144-7

18. 

Hirsch JE , author. “An index to quantify an individual’s scientific research output”. Proceedings of the National academy of Science. 2005;102(46):16569–72

19. 

Cornell University Library. Measuring your research impact: i10-Index. Retrieved August 18, 2016, from: http://guides.library.cornell.edu/c.php?g=32272andp=203393.

20. 

Webometric. 2016. Rankings based on Google Scholar Citations: Methodology. Retrieved January 23rd, 2017 from: http://www.webometrics.info/en/node/179.