1.Comparison of the precision efficiencies

S. no.


Search engines

Query set 1:
("RNA binding protein" OR "RNA binding proteins") AND "transcription initiation"

Query set 2:
("alternative promoter" OR "alternative promoters") AND (mouse OR mice OR mus)

Query set 3:
Liver AND toxicity AND "cell death"

IPS (on a 1 to 10 scale)

Score for search 1

Equivalence score

Score for search 2

Equivalence score

Score for search 3

Equivalence score

1

askMEDLINE

28

4

46.7

1

93.3

6

3.7

2

PubMed

42

8

126.7

9

93

6

7.7

3

Brij.in

36

6

86.7

5

26.7

1

4.0

4

Google Scholar

48

10

81.4

5

122.9

8

7.7

5

HighWire Press

48

10

105.7

7

123.6

8

8.3

6

PubMed Central

-

-

-

-

65.7

4

4.0

7

Scirus

18

1

36.4

1

64.3

4

2.0

8

BioAsk

26

3

115

8

117.1

8

6.3

9

CiteXplore

46

10

136.7

10

103

7

9.0

10

EBIMed

46

10

121.7

8

130

9

9.0

11

Scopus*

42

8

143.3

10

150.7

10

9.3

IPS – Indicative Precision Score

*A trial version of Scopus was used and the exercise was performed in 2007 and updated in 2009.

Please see Supplementary notes 4 for details of the scoring system.

NOTE: PubMed interfaces were not included in the analysis as their primary output is similar to that of PubMed. Instead of fine tuning the query set to make best use of the search engine, we chose a simple set that is uniformly applicable across the search programs selected for citation retrieval efficiency comparison. It is important to note that the results vary depending on the query set used.

2.Comparison of the relative recall efficiencies

S. no.

Search engines

Query set 1:
("RNA binding protein" OR "RNA binding proteins") AND "transcription initiation"

Query set 2:
("alternative promoter" OR "alternative promoters") AND (mouse OR mice OR mus)

Query set 3:
Liver AND toxicity AND "cell death"

RRS (on a 1 to 10 scale)

Score for search I

Equivalence score

Score for search II

Equivalence score

Score for search III

Equivalence score

1

askMEDLINE

1.3

1

2.3

1

0.4

1

1.0

2

PubMed

2

1

4.7

1

1.5

4

2.0

3

Brij.in

1.2

1

2.3

1

0.1

1

1.0

4

Google Scholar

8.3

4

15

5

1.6

4

4.3

5

HighWire Press

18.8

8

32.7

10

3.9

10

9.3

6

PubMed Central

-

-

-

-

1.2

3

3.0

7

Scirus

3.9

2

9.3

3

1.3

4

3.0

8

BioAsk

0.8

1

4.4

1

2.6

7

3.0

9

CiteXplore

25.9

10

16.1

5

1.3

4

6.3

10

EBIMed

0.5

1

4.3

1

1.6

4

2.0

11

Scopus*

1.7

1

4.8

1

1.9

5

2.3

RRS – Relative Recall Score

A trial version of Scopus was used and the exercise was performed in 2007 and updated in 2009.

Please see Supplementary notes 4 for details of the scoring system.

NOTE: PubMed interfaces were not included in the analysis as their primary output is similar to that of PubMed. Instead of fine tuning the query set to make best use of the search engine, we chose a simple set that is uniformly applicable across the search programs selected for citation retrieval efficiency comparison. It is important to note that the results vary depending on the query set used.

The results of the comparative analysis of tools, based on their citation retrieval efficiencies, showed that the following 6 tools could be more useful than the others:

Unlike, the citation retrieval efficiency studies done earlier (Supplementary notes 7, Tables 1 (p-2) and 2 (p-3)), the focus here was on obtaining information with specific objectives in mind.The best possible query set was derived by exploiting features of the query system available. Uniform restrictions were applied during all searches across tools to minimize the number of hits.For example, we looked for citations of articles published in a narrow time period and restricted the field of occurrence of key words within citations.

Details on the methods used for scoring is provided in the Supplementary notes 4.

3.Indicative Precision Scores

S. no.

Search tools

miRNA & breast cancer1

Quadruplex DNA in the context of AIDS

piRNA in non-testicular tissues

Transcriptional regulation in context of apoptosis and prostate cancer

Chikungunya

IPS

PPS

ES

PPS

ES

PPS

ES

PPS

ES

PPS

ES

1

PubMed

200

10

166.7

9

122

10

168.5

7

116.8

10

9.2

2

Google Scholar

58.42

3

34.6

1

42.5

1

189.8

10

43

1

3.2

3

HighWire Press

65.52

4

120

6

89.5

6

130.5

1

100.5

8

5

4

CiteXplore

0

1

200

10

127.9

10

ND

-----

107.5

9

7.5

5

EBIMed

200

10

200

10

126.8

10

ND

-----

122.7

10

10

6

Scopus

64.79

4

48.3

1

45.6

1

187

10

74

4

4

PPS- Preliminary Precision Score, ES- Equivalence Score, IPS- Indicative Precision Score (average ES)

No articles found for the particular query when searched in CiteXplore.

ND- No searches were done for this category in the particular tools.

4.Relative Recall Scores

S. no.

Search tools

miRNA & breast cancer1

Quadruplex DNA in the context of AIDS

piRNA in non-testicular tissues

Transcriptional regulation in context of apoptosis and prostate cancer

Chikungunya

RRS

PRRS

ES

PRRS

ES

PRRS

ES

PRRS

ES

PRRS

ES

1

PubMed

7.1

1

38.46

3

85.19

9

40.7

4

70.41

2

3.8

2

Google Scholar

30.4

4

53.85

5

12.96

1

11.1

1

100

10

4.2

3

HighWire Press

7.1

1

30.77

2

48.15

5

100

10

65.31

1

3.8

4

CiteXplore

0

1

15.38

1

100

10

ND

-----

72.45

3

3.75

5

EBIMed

1.8

1

15.38

1

81.48

8

ND

-----

67.35

1

2.75

6

Scopus

100

10

100

10

22.22

2

41.7

4

85.71

6

6.4

PRRS- Preliminary Relative Recall Score; ES- Equivalence Score; RRS- Relative Recall Score (average ES)

No articles found for the particular query when searched in CiteXplore.

ND- No searches were done for this category in the particular tools.

5.Indicative Exclusive Contribution Scores

S. no.

Search tools

miRNA & breast cancer

Quadruplex DNA in the context of AIDS

piRNA in non-testicular tissues

Chikungunya

IECS

PECS

ES

PECS

ES

PECS

ES

PECS

ES

1

PubMed

100

10

0

1

10

4

4

3

4.5

2

Google Scholar

40.6

5

23.4

6

24.6

9

14.1

8

7

3

HighWire Press

37.9

4

0

1

27.5

10

11.2

6

5.3

4

CiteXplore

0

1

0

1

0.9

1

0

1

1

5

EBIMed

0

1

0

1

5.6

2

19.3

10

3.5

6

Scopus

48.9

5

36.4

10

22.1

8

15

8

7.8

PECS-Preliminary Exclusive Contribution Score; ES-Equivalence Score; IECS-Indicative Exclusive Contribution Score (average ES)

Note: The search involving ‘Transcriptional regulation in context of apoptosis and prostate cancer’ was not considered for this exercise as
the citations were sampled and not all of them were analyzed.


6.Average percentage contribution by each tool to the relevant hits


Avg % relevant articles that could be missed,

S. no.

Search tools

while using the specific tool in isolation

by not using the specific tool (but using the other 3 top performers in combination)

1

Scopus

19.67

31.73

2

Google Scholar

36.73

14.67

3

HighWire Press

49.06

2.33

4

PubMed

48.74

2.66