[OAI-eprints] Re: Measuring cumulating research impact loss across
fields and time
Stevan Harnad
harnad at ecs.soton.ac.uk
Wed Nov 26 07:09:47 EST 2003
On Wed, 26 Nov 2003, David Spurrett & Subbiah Arunachalam wrote:
>ds> I look forward to the results of the empirical study you describe.
>ds> I would be curious to know... whether
>ds> there was a further pattern that related (a) the extent to which
>ds> publications by authors at particular institutions cited research
>ds> materials available through open access, with (b) their local
>ds> institutional budget for expenditure on journals.
>
>sa> Stevan Harnad talked about a study on the relative
>sa> citation rates of open-access and toll-access articles
>sa> he is conducting in collaboration with UQaM,
>sa> Southampton, Oldenburg and Loughborough. When will the
>sa> results become available? Will there be any interim
>sa> reports? I am curious to know.
The study is ongoing and we will report the results (as a pre-refereeing
preprint!) as soon as they are available. But meanwhile, much information
inheres in -- and many telling estimates can be made from -- the data
that are already available.
David Spurrett's & Subbiah Arunachalam's queries suggest the following
preliminary analysis, which can already be done by anyone on the basis
of the data already available. (I will ask our super-talented team at
Southampton if they can squeeze it in, along with all the other ongoing
studies!):
We know from the Lawrence study (below) that the citation enhancement factor
for open- vs. toll-access is about 4.5 in computer science (4.5 times as many
citations for open- vs. toll-access articles in the same venue).
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0006.gif
We know from the Eysenck and Smith RAE outcome study in Psychology
(and from the Oppenheim studies in other disciplines) that the correlation
between RAE outcome and citation impact is about .90 (in Psychology).
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0007.gif
We also know the 2001 RAE outcome, rank-ordering every department in every
university in the UK http://www.hero.ac.uk/rae/submissions/ and we also
know the size of the funding and the funding difference associate with
each rank.
Hence it is very easy to take those rank orders, for each discipline,
and calculate -- based on that discipline's correlation between its
RAE rank and its citation impact -- the estimated income increase that
would arise from the rank increase induced by the impact increase
caused by open access!
In particular, it would be possible to illustrate how the rank order
would change if, for example, the research output of the lowest-ranked
department in each discipline became open-access, and gained a 2-fold,
3-fold, 4-fold, or 4.5-fold increase in impact (depending on how
close it came to the Lawrence 4.5 estimate -- which might itself be
an underestimate in some disciplines!). The RAE/impact correlation
would predict what rank that department would get, and the RAE/funding
correlation would predict how much more money that would translate into.
Obviously if *all* the articles in all disciplines suddenly became
open-access overnight, there would not be such a dramatic change in
rankings (though it would give some research a better fighting chance),
because all impact would simply be scaled up. (*Simply scaled up*! But
that in itself would represent a huge benefit to research progress and
productivity.)
But never mind that. We must appeal to our lower instincts, in trying
to persuade individual researchers and their institutions that open
access is in their interests. So the above data should be taken in
a first-come, first-served competitive spirit: Right now, it is
definitely not the case that *all* articles are open access. Almost all
are not. Nor is the transition happening overnight (as it could have done,
already a decade ago).
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0024.gif
So the incentive to self-archive comes from the fact that those
who do it *now* stand the best chance of changing the relative
research impact-ranking (and hence the research funding) in their
favor: and the study I've sketched would estimate by just how much.
A dimensionless picture of the size of the increment is already
visible in:
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0025.gif
The RAE data are open-access, so anyone can do this study. But I will try
to persuade the Southampton team to do it, in order to provide ammunition
for those who are hard at working trying to inform university
administrators and research funders about the benefits to be expected
from mandating open-access provision for all their research output.
[A slight correction to David Spurrett's query about the correlation between
> "(a) the extent to which publications by authors at particular
> institutions cited research materials available through open access,
> with (b) their local institutional budget for expenditure on journals."
First, that's the wrong correlation. We've agreed it's not journal
budget expenditures that will persuade researchers to self-archive,
but research income. Second, we can already answer the question: That
correlation is zero, because the small existing volume of open-access
there is so far has not led to any toll-cancellations, in any discipline
(including Physics, where self-archiving and open-access are most advanced).
The correlation *might* change eventually, but that will not be a *cause* of
universal open access, but an *effect*:
http://www.nature.com/nature/debates/e-access/Articles/harnad.html#B1 ]
Lawrence, S. (2001) Free online availability substantially
increases a paper's impact. Nature Web Debates.
http://www.nature.com/nature/debates/e-access/Articles/lawrence.html
Kurtz, Michael J.; Eichhorn, Guenther; Accomazzi, Alberto;
Grant, Carolyn S.; Demleitner, Markus; Murray, Stephen S.;
Martimbeau, Nathalie; Elwell, Barbara. (submitted) The NASA
Astrophysics Data System: Sociology, Bibliometrics, and Impact.
http://cfa-www.harvard.edu/~kurtz/jasis-abstract.html
the forthcoming Schwartz et al. study
http://listserv.nd.edu/cgi-bin/wa?A2=ind0311&L=pamnet&D=1&O=D&P=1632
the work of Andrew Odlyzko:
http://www.dtc.umn.edu/~odlyzko/doc/complete.html
and Tim Brody's remarkable citebase usage and citation impact calculator
http://citebase.eprints.org/cgi-bin/search as well as his usage/citation
impact correlator
http://citebase.eprints.org/analysis/correlation.php
which can predict later citation impact from earlier usage (download)
impact using variable time-windows and ranges for the Physics ArXiv
(you need the latest java to be able to use it) at:
Smith, Andrew, & Eysenck, Michael (2002) "The correlation
between RAE ratings and citation counts in psychology," June 2002
http://psyserver.pc.rhbnc.ac.uk/citations.pdf
Oppenheim, Charles (1995) The correlation between citation counts and
the 1992 Research Assessment Exercises ratings for British library and
information science departments, Journal of Documentation, 51:18-27.
Oppenheim, Charles (1998) The correlation between
citation counts and the 1992 research assessment exercise
ratings for British research in genetics, anatomy
and archaeology, Journal of Documentation, 53:477-87.
http://dois.mimas.ac.uk/DoIS/data/Articles/julkokltny:1998:v:54:i:5:p:477-487.html
Holmes, Alison & Oppenheim, Charles (2001) Use of citation analysis
to predict the outcome of the 2001 Research Assessment Exercise for
Unit of Assessment (UoA) 61: Library and Information Management.
http://www.shef.ac.uk/~is/publications/infres/paper103.html
Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated online
RAE CVs Linked to University Eprint Archives: Improving the UK Research
Assessment Exercise whilst making it cheaper and easier.
Ariadne. http://www.ariadne.ac.uk/issue35/harnad/
Stevan Harnad
NOTE: A complete archive of the ongoing discussion of providing open
access to the peer-reviewed research literature online is available at
the American Scientist September Forum (98 & 99 & 00 & 01 & 02 & 03):
http://amsci-forum.amsci.org/archives/september98-forum.html
http://www.cogsci.soton.ac.uk/~harnad/Hypermail/Amsci/index.html
Post discussion to: september98-forum at amsci-forum.amsci.org
Dual Open-Access Strategy:
BOAI-2 ("gold"): Publish your article in a suitable open-access
journal whenever one exists.
BOAI-1 ("green"): Otherwise, publish your article in a suitable
toll-access journal and also self-archive it.
http://www.soros.org/openaccess/read.shtml
http://www.ecs.soton.ac.uk/~harnad/Temp/berlin.htm
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0026.gif
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0021.gif
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0024.gif
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving_files/Slide0028.gif
More information about the OAI-eprints
mailing list