Introduction

The so-called ‘serials crisis’ that was deeply experienced in Europe and the USA at the beginning of the last decade did not only have impacts on its own institutions, on publisher business models and on the redesign of the instruments to ensure access to scientific information resources. In Latin America, following the pioneering experience of Brazil's journals portal, CAPES, in 2000, countries such as Argentina, Uruguay and Colombia also began to develop interventions (arising from public agency initiatives) to ensure access to scientific information for their research communities.

The Chilean case is part of this trend, but with a few differences. Firstly, the public agency in charge, the National Commission of Scientific and Technological Research (CONICYT), was in favour of creating a separate institution – a non-profit corporation – to contract subscriptions without the constraints of public purchasing.

A second aspect in Chile was that CONICYT determined that the senior personnel of the corporation‘s founding organizations should participate in the strategic decision-making of this new body. This was to ensure that decisions were taken at the highest level.

A third aspect of the experience – for better or for worse – was the decision to avoid, as far as possible, the conflicts of interest among very heterogeneous institutions, making content selection procedures based on objective studies of the citation patterns of the Chilean scientific community.

Finally, a co-payment model was developed in an attempt to minimize cross-subsidies and reflect the potential use of each institution. This was to ensure the governance of the primary agreement that gave birth to the new entity, called CINCEL Corporation or Consortium for the Access to Electronic Scientific Information.

The starting point

The problem that we tried to address was the profoundly unequal distribution of access to scientific information, which also resulted in scientific output, measured as articles indexed in Web of Science (WoS), being highly concentrated in just a few institutions. For example, in 2002, when CINCEL was created, only CONICYT had online access to WoS. Thus, the consortium's first action was the acquisition of this product for all partners.

For journals, the overall picture was not very promising either. Of the 25 partner universities of CINCEL that in 2005 received public funding, only three had acceptable levels of access to serial publications. Those same three, for the same year, represented 69.8% of the scientific output (or articles indexed as proxy), which points to a correlation between access and productivity.

Both phenomena are illustrated in Figures 1 and 2 below.

Figure 1 

Chilean universities with public funding

Number of subscribed international journals (2005)

Source: Cincel Corporation Executive Secretary, based on the citation studies in the First Consultancy Mecesup Project UCV 0315, by the Exinde Consultancy, 2006

Figure 2 

CINCEL's universities

Articles indexed in WoS (1990–2004)

Source: CINCEL Corporation Executive Secretary, based on the citation studies in the Second Consultancy Mecesup Project UCV 0315, by Félix de Moya y Anegón, 2006

When access is so poorly distributed, the lack of information carries unquantified costs with a resulting loss of quality, relevance and timeliness of the research undertaken. Additionally, it is a weak basis for programmes aimed at improving ‘human capital’ (such as scholarships granted by Government for a variety of postgraduate programmes, training or postdoctoral courses), or for policies designed to decentralize scientific activity.

Public policies: design options

The significant asymmetry and inequity in access to information resources was the motivation for designing a public policy to reduce the gap.

CONICYT had several intervention options. In a state like Chile, that is still dominated by an ethos of entitlement from the so-called ‘Washington Consensus’, an obvious option was to subsidize the information demand by ‘voucher’ schemes. Although a very efficient measure to satisfy individual demands, this type of public policy does not generate economies of scale.

“The significant asymmetry and inequity in access to information resources was the motivation for designing a public policy …”

Another option was to create an institution to take charge of the problem. It is a commonly used approach, but a sterile one if it does not have leadership, competence and funding.

An alternative approach was to not undertake major institutional changes, but to make incremental improvements, for example, through performance agreements between the public agency and institutions with serious access problems. As the indicators of access, use and productivity improved, the institution would be ensured a guaranteed budget year after year.

Two further options, preferable in cases of budget scarcity, were to work with the most productive scientific groups – a striking meritocratic public policy because it would achieve early results (but rather than eliminating any gap, it would instead make it wider) – or the inverse approach, to focus resources on the less productive groups. The latter is the most complex option to manage, because there are longer-term results, where public policy is shielded from political cycles that alter the priorities of the authority in charge.

Whichever design option was chosen, there were at least three conditions to be met; public policy had to be:

  • technically impeccable (not produce negative consequences or unwanted effects)
  • politically viable and manageable (in most cases, the perfect technical solutions are not ‘politically correct’), which means willing to bear the transaction costs while the demand is aggregated and institutional heterogeneity is addressed
  • in the public good.

In other words, the final option needed to be the most efficient, effective, equitable and sustainable solution that could be achieved.

The response, in the Chilean case, was a mix between a partial subsidy (because there are co-financing resources available) and the creation of a body to take charge of negotiation, acquisition and management of the so-called National Infrastructure of Scientific Information Access. With a decentralized model of access, it provides a continuous service to electronic information resources.

The main resource is the Electronic Library of Scientific Information Program (BEIC), started in January 2008, and in which 27 universities participated, representing almost 85% of Chilean scientific publication. It is comprised of approximately 5,000 academic journals from eight publishers: AAAS, Nature Publishing Group, Annual Reviews, American Chemical Society, Oxford University Press, Wiley-Blackwell, Springer and Elsevier. Of the titles, 75% are indexed in WoS and 69% in Journal Citation Reports (JCR), 46% indexed in the first and second quartile of JCR. The accumulated cost to 2012 reached 35 million dollars.

However, state involvement in an initiative of this calibre meant that democratic access (with all its positive benefits) should also carry a key outcome: an increase in scientific productivity.

Evaluation limits

The evaluation of BEIC, undertaken by Nicholas Cop Consulting, and designed by the Executive Secretary of CINCEL and the Scientific Information Program of CONICYT, took place between January and November 2011. It included qualitative components such as satisfaction surveys of end users and contract analysis to protect the investment. The quantitative aspects were concentrated in a bibliometric study of citation usage and publication of new articles to determine if the existence of better access brings with it improvements in the productivity of the institutions.

“… state involvement in an initiative of this calibre meant that democratic access … should also carry a key outcome: an increase in scientific productivity.”

When evaluating a public policy instrument such as BEIC, it is important to note the limitations of a study that explains scientific productivity using only one variable – access to information – but ignores others, such as the existence or not of public policies that fund basic and applied research, researchers, infrastructure and equipment distribution, and variables whose impacts are even more difficult to measure, such as the social capital of scientists and their collaborative networks. There is also an important dimension in the measurement of impact related to an improved quality of life, which requires another approach.

In the case of BEIC, impact measurement is very conservative. The eventual use of information (articles) contained in BEIC journals for the publication of new knowledge (new articles and where they are published) is analysed from a bibliometric point of view.

For the analysis, it was considered that publications subject to evaluation would be those indexed in WoS and in JCR, including articles published in national open access journals (SciELO-Chile). This last decision allowed the estimation, in an indirect way, of the value of that collection, which CONICYT has managed since 1998, as part of our scientific communication cycle.

Some findings

Firstly, we have been able to confirm that the collection is well used, so it has not been a wasted investment: the usage of BEIC titles, measured as ‘downloads’, has significantly increased; before 2008, 40.5% of titles present in the collection were downloaded, increasing to 63.3% in the 2008–2010 period (see Table 1).

Table 1

Article downloads by publisher (2008–2010) showing percentage changes


ElsevierWiley-BlackwellSpringerNPGOUPACSAAASAnnual ReviewsTotal
20081,211,651268,653175,299100,05674,88056,51838,03629,4161,954,509
20091,469,978236,172201,729122,22989,70373,66340,00030,0952,263,539
20101,525,075232,148186,215136,94483,82681,93136,27728,3192,310,735
cumulative variation313,424−36,50510,91636,8888,94625,413−1,759−1,097356,226
cumulative percentage variation26%−14%6%37%12%45%−5%−4%

Source: Nicholas Cop Consulting, ‘Informe diagnóstico del uso de las colecciones BEIC’, June 2011

If usage is measured as citations, an average increase of 20% has been demonstrated compared with the period before BEIC.

Analysing disciplines, 60.4% of publications in the 2005–2010 period were concentrated in seven fields: Clinical Medicine, Space Science, Plant & Animal Science, Chemistry, Engineering, Social Sciences (general) and Physics. Clinical Medicine is the subject category with the highest concentration of publications (19.11% of all publications) and in the post- BEIC period, its share of publication in international journals increased to 26.5%, which is expected due to the publication rate in the discipline, but also because it is the discipline represented the most in national journals, along with Social Science and Humanities, because of the current specific incentives for publishing in SciELO-Chile. (See Table 2.)

Table 2

Percentage growth of Chilean articles published in the pre-BEIC period (2005–2007) and the post-BEIC period (2008–2010), in relation to BEIC and non-BEIC journals by subject category and BEIC publishers


Area (publications)BEIC publishersNoSciELO ChileTotal per Journal type
AAASACSAnnReuBlackweliElsevNPGOUPSpringerWileyBEICNoSciELO
Agricultural Sciences11,8%86,4%29,3%110,0%−36,4%4,3%76,9%414333504
Biology & Biochemistry−50,0%8,5%20,2%100,0%−11,1%47,4%−3,7%5,7%−34,7%611436195
Chemistry17,0%−8,9%20,4%32,3%−18,8%58,9%0,7%1.395422271
Clinical Medicine19,6%26,2%100,0%29,4%33,3%3,1%48,7%8,8%1.0831.1743.446
Compuier Science86,7%−33,3%−12,9%300,0%47,7%318436
Economics & Business23,8%124,0%166,7%19,0%50,0%83,8%195193
Engineering−30,8%−40,0%58,9%1,8%6,8%48,1%32,4%658970251
Environment/Ecology−66,7%64,3%24,0%0,0%−−9,1%57,1%42,2%159,3%65239097
Geosciences85,7%18,1%100,0%71,4%100,0%28,6%29,5%514352397
Human Sciences0,0%178,1%50,2%52421.251
Immunology200,0%23,8%−100,0%100,0%31,0%7167
Materials Science−100,0%−100,0%26,4%77,8%800,0%32,4%263237
Mathematics−40,0%26,6%−25,0%69,0%33,3%30,7%56252620
Microbiology=n52,9%42,9%−100,0%118,2%−22,2%58,6%147150
Molecular Biology & Genetics−100,0%100,0%−16,3%0,0%266,7%14,3%−1,7%45,2%269179
Multidisciplinary40,0%5,3%200,0%2,8%7373
Neuroscience & Behavior9,5%1,4%0,0%62,5%−15,0%24,2%250148
Pharmacology & Toxicology0,0%100,0%−18,0%400,0%100,0%117,9%145124
Physics75,0%−9,0%200,0%5,2%−14,3%24,5%5391.091
Plant & Animal Science166,7%34,4%41,2%−16,2%65,9%0,0%18,6%−8,4%897988228
Psychiatry/Psychology166,7%41,7%−100,0%75,0%57,0%−17,1%70221150
Social Sciences, general228,6%102,9%0,0%23,1%−50,0%210,1%36,7%1744881.077
Space Science0,0%32,2%−29,4%9,1%46,2%8,0%5632.105
All publications40,0%6,2%0,0%34,1%25,5%29,6%7,7%31,0%3,9%35,9%22,2%9.86811.3457.887

Note: AAAS: American Association for the Advancement of Science, ACS: American Chemical Society, AnnRev: Annual Reviews, Elsev: Elsevier, NPG: Nature Publishing Group, OUP: Oxford University Press.

The column ‘No’ is Non-BEIC (other international journals that are not part of the BEIC collection)

Source: Nicholas Cop Consulting, ‘El impacto de la BEIC sobre la producción científica chilena (enero 2008–abril 2010) y el patrón de uso de las publicaciones periódicas entre 2005 y 2010’, December 2011

Table 3 shows the percentage of titles in each collection of BEIC cited in articles by Chilean researchers. For example, ACS and Elsevier represent 46.9% of total titles cited in Chemistry.

The three subject categories in which the usage is better covered by BEIC journals are Immunology, Pharmacology/Toxicology and Multidisciplinary. But there are other subject categories such as Space Science or Social Science that use only 16.8% and 16.3% of BEIC titles, respectively, in its usage pattern.

Regarding the output levels of the 27 institutions of CINCEL analysed in this study, three categories were defined to compare the publication output and focus: high (five institutions with more than 5% of the total publication), medium (six institutions that publish between 2% and 5%) and low (16 institutions with less than 2%).

Figures 3 and 4 show the publication and usage distribution in these three categories for journals indexed in JCR, SciELO-Chile and non-indexed titles.

Table 3

BEIC collection representation in relation to the set of titles of Chilean usage, by subject category and publisher (2005–2010)


Area (usage)BEIC publishers (usage)NoTotal
AAASACSAnnRevBlackwellElsevNPGOUPSpringerWiley
Agricultural Sciences0.3%4.3%0.3%7.1%0.5%1.4%6.2%2.3%56.9%100%
Biology & Biochemistry14%2.2%0.8%6.7%18.2%3.1%2.3%5.7%3.6%55.9%100%
Chemistry0.6%19.0%0.1%1.6%27.9%0.6%0.3%3.7%7.2%38.9%100%
Clinical Medicine0.5%0.3%0.3%6.5%16.4%1.3%2.5%4.8%3.7%63.8%100%
Computer Science0.2%0.2%0.0%0.6%10.6%0.2%0.5%10.8%1.3%75.5%100%
Economics & Business0.1%0.0%0.1%9.4%12.7%0.1%1.2%3.0%1.5%72.0%100%
Engineering0.3%1.6%0.1%1.4%17.2%0.2%0.3%4.6%2.7%71.5%100%
Environment/Ecology1.5%1.2%0.3%10.7%15.6%1.4%1.3%8.1%1.0%58.9%100%
Geosciences1.7%0.4%0.3%3.2%19.4%1.6%1.1%4.6%1.1%66.8%100%
Human Sciences0.7%0.0%0.2%4.7%6.2%0.6%0.8%3.2%0.9%82.6%100%
Immunology1.2%0.3%1.6%7.7%17.6%3.9%1.7%2.6%3.8%59.6%100%
Materials Science1.0%4.5%0.3%2.6%27.8%0.9%0.5%5.3%3.7%53.4%100%
Mathematics0.1%0.0%0.0%1.6%16.6%0.1%1.5%12.5%2.4%65.2%100%
Microbiology1.0%1.0%0.8%8.1%16.1%2.1%2.8%6.7%1.7%59.8%100%
Molecular Biology & Genetics2.0%0.6%1.1%5.0%16.1%6.0%2.8%4.2%5.0%57.3%100%
Multidisciplinary3.6%0.5%1.2%7.4%10.7%8.0%2.2%4.0%1.7%60.7%100%
Neuroscience & Behavior2.1%0.3%0.9%7.1%25.4%5.1%1.3%4.0%5.6%48.2%100%
Pharmacology & Toxicology0.9%2.6%0.8%5.0%25.4%2.2%1.6%4.2%3.2%54.1%100%
Physics0.7%0.8%0.2%0.8%20.3%1.3%0.1%4.5%0.8%70.4%100%
Plant & Animal Science0.9%0.5%0.3%8.8%12.6%1.1%2.5%7.7%0.8%64.8%100%
Psychiatry/Psychology0.4%0.2%0.4%5.7%10.6%0.5%1.1%5.0%2.5%73.6%100%
Social Sciences, general0.3%0.0%0.2%4.4%7.0%0.2%1.0%2.3%1.0%83.7%100%
Space Science0.5%0.0%1.1%10.9%1.6%1.5%0.0%1.0%0.2%83.2%100%

Table colours: Orange identifies BEIC publishers with a greater proportion of Chilean usage, by subject category. Red represents the opposite, identifying the subject categories that cite the greater quantity of titles that are not in BEIC.

Source: Nicholas Cop Consulting, ‘El impacto de la BEIC sobre la producción científica chilena (enero 2008–abril 2010) y el patrón de uso de las publicaciones periódicas entre 2005 y 2010’, December 2011

Figure 3 

Chilean articles published between 2005 and 2010, by CINCEL institutions, by JCR quartiles, SciELO-Chile or without indexation

Figure 4 

Chilean article cites (usage) by category, publication level of CINCEL institutions and indexation in JCR by quartiles, in SciELO or non-indexed (2005–2010)

The institutions with a low publication level have a greater percentage of articles in SciELO-Chile. The other two categories concentrate their publication in the first two quartiles of JCR.

By usage and by JCR quartiles, the concentration is in the first quartile and in non-indexed titles. The institutions with a high publication level use fewer SciELO journals.

Conclusions: some lessons learned

The ‘big deal’ problem

The acquisition of journal packages is based on the economic logic of cost containment. But the global skewing of the scientific communication cycle is on the verge of destroying that model, and there seem to be no promising alternatives. At this stage, for countries like ours – marginal science producers – confronting the big oligopolies forces us to acknowledge that we are ‘piece takers’, as a classic economist would say (although in this market it is an imperfect extreme). At the same time comes the realization that there are few available tools to level the playing field. It is an inescapable fact that a publisher like Elsevier represents 60% of article downloads and 25% of citations in Chile. It is impossible not to include it in the design of any public policy on access to scientific information that aims to meet the needs of end users.

“… confronting the big oligopolies forces us to acknowledge that we are ‘piece takers’ …”

Understanding of the national cycle of scientific communication in order to optimize it

Chile has formed part of the SciELO network since 1998, and CONICYT has designed incentives to promote publication in these journals. The scene is then open to complementary policies, such as the mandate to deposit pre-prints in institutional repositories if the research has been funded with public resources or under a policy of research data access. Discussions in the USA and Europe regarding open access and its several routes (the ‘green’, which is the obvious, the ‘gold’, proposed by the Finch Report, or the ‘blue’ that is the preferred by the medical societies of the USA) are being closely followed, but we know that in our country part of the problem is already resolved because the government, through CONICYT, is the body that finances SciELO-Chile.

Evaluation for better decision-making

In our public system, evaluation is fashionable. If in the past our concern was the input (how much we spent, how many beneficiaries we reached, what forms of delivery were the most efficient), now we are more concerned by output (Is what we do useful? Can we improve? Is it a contribution to people's life quality? Is it the better solution for the problem or will it generate other unexpected consequential problems?).

Evaluation always involves a shake-up of the assumptions on which we base our actions and it is not easy to undertake in hierarchical, bureaucratic organizations which are subject to political control, such as public bodies. However, if we take the role of the public agency in the creation of value seriously, it is impossible to improve without an evaluation process. The result is a strengthening of the impact metrics, making them even more consistent and transparent, because we have to continue proving that access is a crucial element in the improvement of the scientific performance of individuals and institutions. This also includes strengthening the capacity of public agencies to design appropriate studies, including ensuring that staff have sufficient skills to interpret the data and translate them into public policy adjustments that can be implemented by the authority.

“Evaluation always involves a shake-up of the assumptions on which we base our actions …”

Finally, in Chile we still have a long way to go in defining an institutional framework capable of dealing with the current problems of scientific research and how to become even more collaborative and cross-disciplinary, and that has the merit of being conducive to strengthening the steps we are taking today, while ensuring the scientific basis of the future.