How significant are the public dimensions of faculty work in review, promotion and tenure documents?

Abstract

Much of the work done by faculty at both public and private universities has significant public dimensions: it is often paid for by public funds; it is often aimed at serving the public good; and it is often subject to public evaluation. To understand how the public dimensions of faculty work are valued, we analyzed review, promotion, and tenure documents from a representative sample of 129 universities in the US and Canada. Terms and concepts related to public and community are mentioned in a large portion of documents, but mostly in ways that relate to service, which is an undervalued aspect of academic careers. Moreover, the documents make significant mention of traditional research outputs and citation-based metrics: however, such outputs and metrics reward faculty work targeted to academics, and often disregard the public dimensions. Institutions that seek to embody their public mission could therefore work towards changing how faculty work is assessed and incentivized.

Keywords: academic careers; higher education; institutional policy; metrics; none; open access; scholarly communications.

Figure 1.. Percentage of institutions mentioning public and community terms and concepts by type of institution.

Bars represent whether each term or concept (several terms and phrases) was identified within documents from doctoral/research-focused universities (R-type; blue), master’s colleges and universities (M-type; orange) and baccalaureate colleges (B-type; green). The terms "public" and "community," and the concept of "public and/or community engagement" appear less often in documents from B-type institutions than from M- and R-type. The conditions of the chi-square test were not met for the term "community," but the chi-square analysis reveals the difference in presence of term "public" and concept "public and/or community engagement" are significant. Chi-square tests: Term Public: χ² (2, N=129)=13.85, p<0.001; Concept Public and/or community engagement: χ² (2, N=129)=9.61, p<0.01.

Figure 2.. Percentage of institutions mentioning public and community terms and concepts by institution sub-type.

Bars represent whether each term or concept (several terms and phrases) was identified within documents of doctoral/research-focused universities, from the most research intensive (R1; blue), to those that are less so (R2; orange, and R3; green), as well as the Canadian research universities (RCan; red). The terms "public" and the concept of "public and/or community engagement" appear more at R1 and R2 institutions than R3, with RCan universities falling in the middle. However, sample sizes violate conditions for a chi-square test to measure the significance of these differences.

Figure 3.. Percentage of institutions mentioning public and community terms and concepts by discipline.

Bars represent whether each term or concept (several terms and phrases) was identified within documents of academic units from Social Sciences and Humanities (SSH; blue), Physical Sciences and Mathematics (PSM; orange), Life Sciences (LS; green) and multidisciplinary units (red). The terms and concepts appear more frequently in LS units than others. Sample size conditions for a chi-square test were only met for the concept of "public and/or community engagement," where it indicates that the difference in this category is significant. Chi-square test: Concept Public and/or community engagement: χ² (3, N=116)=12.45, p<0.05.

Figure 4.. Relative frequency of words surrounding the term "public".

Visual representation of the relative frequency of words near (within 15 words) the word "public" across all documents. The most frequent word near "public" is "service." Along with other frequent words, this suggests that in the context of RPT, "public" is most often associated with a service activity.

Figure 5.. Relative frequency of words surrounding the term "community".

Visual representation of the relative frequency of words near (within 15 words) the word "community" across all documents. The most frequent word near "community" is "university." Along with other frequent words, this suggests that the community most often referred to is that of other academics.

Figure 6.. Percentage of institutions mentioning terms and concepts related to research and metrics by institution type.

Bars represent whether each term or concept (several terms and phrases) was identified within documents from doctoral/research-focused universities (R-type; blue), master’s colleges and universities (M-type; orange), and baccalaureate colleges (B-type; green). Chi-square analysis suggests that the term "impact" and the concept of "metrics" is more present at R-type than at M-type, and more present at M-type than B-type. The concept of "traditional outputs" is present at over 90% of each type, although the conditions for a chi-square test were not met for this concept or for the term "open access." Chi-square tests: Term Impact: χ² (2, N=129)=24.13, p<0.001; Concept Metrics: χ² (2, N=129)=32.04, p<0.001.

Figure 7.. Percentage of institutions mentioning terms and concepts related to research and metrics by institution sub-type.

Bars represent whether each term or concept (several terms and phrases) was identified within documents of doctoral/research-focused universities, from the most research intensive (R1; blue), to those that are less so (R2; orange, and R3; green), as well as the Canadian research universities (RCan; red). The term "impact" appears less in R3 institutions, and the concept of "metrics" appears to decrease with research intensity (with RCan institutions at similar levels to the R2 institutions from the US) However, the conditions for a chi-square test were not met to measure the significance of these differences.

Figure 8.. Percentage of institutions mentioning terms and concepts related to research and metrics by discipline.

Bars represent whether each term or concept (several terms and phrases) was identified within documents of academic units from Social Sciences and Humanities (SSH; blue), Physical Sciences and Mathematics (PSM; orange), Life Sciences (LS; green) and multidisciplinary units (red). The concept of traditional outputs is present in the vast majority of units. The term impact is more present in LS, but a chi-square test suggests the difference is not significant. The chi-square analysis also indicates the difference in the presence of the concept of "metrics" (with PSM units mentioning it the most) is significant. The conditions for a chi-square test were not met for other terms and concepts. Chi-square tests: Term Impact: χ² (3, N=116)=5.75, p>0.05 (not significant). Concept: Metrics: χ² (3, N=116)=7.33, p<0.05.

Figure 9.. Relative frequency of words surrounding the term "impact".

Visual representation of the relative frequency of words near (within 15 words) the word "impact" across all documents. The most frequent word near "impact" is "research." Along with other frequent words, this suggests that the type of impact most valued is that which relates to research activities.