esss Berlin, 8 13 September 2013 Monday, 9 October 2013

Transcription

1 Journal-level level Classifications - Current State of the Art by Eric Archambault esss Berlin, 8 13 September 2013 Monday, 9 October 2013

2 Background The specific goal of a classification is to provide some insight into the organization of the data themselves. The classification scheme is the targeted end result of the agglomerative process, which reflects resemblance among items through either similarity or dissimilarity measures (Ruocco and Frieder, 1997) The classification of science into a disciplinary structure is at least as old as science itself Glänzell and Schubert t(2003) Though classification systems for monographs and serials exist, they are seldom used as journal classifications by editors and scientometricians 2

3 Background cont d Even if the overall situation looks somewhat like a free for all, there is currently a nearly closed to all situation with countless proprietary journal classifications, often difficult to access, and with unclear IP regime How useful is it to have a journal classification? What are alternative types of classifications? Is there a need for new ones? The Science Metrix journal classification 3

4 Why would scientometricians want a classification? To produce statistics on scientific knowledge production! Current types of classifications: Journal level l lclassification i Article level classification using clustering [Ad hoc, journal level themes or article level themes defined using keyword searches & hybrid approaches] 4

5 Journal level classifications In scientometrics, they are used to produce statistics that reflect the organisational structure of science, which is greatly influenced by the structure of universities as they are the main sites of scientific knowledge production Assumption that journals cover relatively coherent topics Similar journals can be grouped Have to deal with general types of journals, general and miscellaneous journals in each research field Advantage: Expediency & tradition relatively easy to group journals in groups that are fairly easy to name Journal level classifications typically have less granularity Not good at representing research topics, emerging areas of research and the scientific ifi flavour of the moment 5

6 Article level classifications Advocates often claim this is the "real McCoy" Assumption thatarticles articles cover relatively coherent topics Similar articles can be grouped Dealingwith general types of articles infrequently addressed from a theoretical perspectives simply left aside Potentiallystellar theoretical problems withmultidimensional definition of similarity (colour, sex, type of illness, living vs. nonliving) which are frequently set aside Advantage: granularity and (frequently) topic based (as opposed to discipline based) Naming clusters is one of the most serious challenges though automated techniques can be of help 6

7 Moving forward Both thjournal and article level l lclassifications are useful and desirable should not be a debate articulated on ideology! Focus of the presentation is on journal level classifications 7

8 The current situation journal editors Though there are a number of widely accepted general classification systems for books (such as the Dewey Decimal Classification, the Universal Decimal Classification and the Library of Congress Classification), there is no widely accepted standard for journal classifications; this includes journal editors Each editor has its own system 8

9 Each editor has its own system Elsevier 9

10 Each editor has its own system Wiley 10

11 Each editor has its own system Springer 11

12 The current situation journal editors Though there are a number of widely accepted general classification systems for books (such as the Dewey Decimal Classification, the Universal Decimal Classification and the Library of Congress Classification), there is no widely accepted standard for journal classifications; this includes journal editors Each journal editor has its own proprietary system Classifications are seldom made available to the public, and their construction is seldom made explicit or documented Of course, there is no standard dbody or working group that oversees the development of the classifications used by editors These are mostly used for convenience, and their principal i purpose is to help find articles 12

13 The current situation database publishers Each database publisher has its own proprietary system 13

14 Each database has its own system Inspec, Pascal 14

15 Each database has its own system Scopus 15

16 Each database has its own system WoS 16

17 The current situation database publishers Each database publisher has its own proprietary system Here again, classifications are not conveniently made available to the public and their construction is seldom explicit or well documented Here again, there is no standard body or working group that oversees the development of the classifications used by database publishers Here again, these are mostly used for convenience, and to easily find journals 17

18 Moving forward the sequel Two mainstream taxonomies used in scientometrics NSF (Narin et al.) for Science & Engineering gindicators and Thomson (aka ISI) for Current Contents, SCI, WoS, etc. PRESENT PROBLEMS NSF is neat, hierarchical and mutually exclusive, initial efforts were documented, but increasingly outdated Thomson Reuters lacks transparency and is not easy to download As part of a contract for the regular provision of scientometric indicators for the European Commission, Science Metrix designed an open system thatisfreely available to researchers 18

19 Design goals wish list for scientometrics As scientific knowledge is mostly produced by universities, classification should to some extent reflect the organisational structure of universities Incremental change: should use categories university people are familiar with Yet it should also be able to account for multidisciplinary and general journals Should reflect the fact that several mainstream fields have emerged, and will need to be changed to reflect new fields Journals should ldbe classified din a mutually exclusive manner (debatable) bl Should be hierarchical Should be discriminating enough to show the differences between publications from different disciplines or fields, but not so detailed as to submerge major trends within numerous minor categories (Narin, Pinski and Gee, 1976), i.e categories 19

20 The journal classifications used as seeds The US NSF: Tried and tested mutually exclusive classification used in the Science & Engineering Indicators since the 1970s. It was originally designed by CHI Research. The WoS classification: A widely used, constantly updated and authoritative classification, but with important overlaps between categories and possessing only one hierarchical level (although, internally, Thomson Reuters aggregates these classes). The Australian Research Council (ARC) Evaluation of Research Excellence (ERA) classification: A modern and newly designed scheme, hot off the press. Journals were assigned to fields by interested researchers. It has significant overlaps and, also, being so new, it has not yet been extensively tested. The following classifications were also briefly examined: The revised Field of Science and Technology (FOS) Classification in the Frascati Manual (OECD) The European Research Council (ERC) classification 20

21 Overall design approach Seed categories from existing journal classifications (NSF, Thomson, Excellence in Research for Australia ERA Journal List) Examine where each journal fits best use the junction of what categories journals make their references to and by which categories they are cited Help from semantic similarity as well as similarity of addresses (cosine similarity) HYBRID MODEL Many journals reclassified by hand (many in Scopus list of about 30,000 journals do not have enough references & citations to make it possible to effectively examine their similarity to journal categories) Morerecentfieldsreceived recent received help, e.g. nanotechnology Several rounds of computer suggestions, each followed by manual tuning Result is a logical taxonomy as opposed to a purely empirical one 21

22 Seeding the journals and their categories For example, seeds from the Excellence in Research for Australia ERA Journal List 22

23 Re classifying & creating a 3 level hierarchy Classifying heterogeneous subfields into a homogeneous 2 level structure (fields and subfields) A third level was later added to obtain a 3 level hierarchy 6 domains 22 fields 175 subfields 23

24 Assignation of journals to a single category Core classifying engine used the following equation: Journals assigned to the category with greatest score Attractiveness of the Image and Vision Computing journal, by subfield 24

25 Does the classification follow a power law? rnals per subfie eld Number of jou y = x R² = Subfields ranked by decreasing number of journals 25

26 Does the classification follow a power law? 26

27 A freely accessible resource for research (1) 27

28 A freely accessible resource for research (2) 28

29 Translation of the taxonomy 3 level taxonomy was translated by a crowd of bibliometricians & information scientists 29

30 Gadgets to explore fields, subfield connections 30

31 Gadgets in 20 languages 18) 31

32 More gadgets 32

33 And, for good measure, a map of science 33

34 Conclusion This is a simplified presentation of the work we did there were numerous iterations and a lot of head scratching and fine tuning We would appreciate feedback from practitioners what works, what needs to be improved, bugs in assignation Future steps Topic modelling (Latent Dirichlet Allocation) DONE! Bibliographic coupling, & article level classification Multi criteria vector citation and reference (and variations thereof), addresses, semantic similarity Find time to update it, or find an interested institution! 34

35 The end! Danke 35