MU Chapter of the American Association of University Professors (AAUP)
April 4, 2021
To: President Choi, Provost Ramchand, Associate Provost Riley-Tillman, and Assistant Vice President Felts
Inquiry on the use of Academic Analytics data in promotion and tenure decisions
At the MU General Faculty Meeting on March 24, 2021, multiple faculty members raised substantive concerns about why and how data from Academic Analytics (AA) are being used in promotion and tenure decisions. Use of AA data for this purpose is a new practice at MU, and, as far as we are aware, is also new at the other UM System campuses. This new practice has been introduced without the approval of MU faculty, thereby bypassing faculty authority as outlined in the UM System Collected Rules and Regulations (CRRs). As noted in CRR 300.010 (the MU Faculty Bylaws), the faculty have “primary and direct authority,” that is, “essential decision-making authority in [the] . . . Formulation of criteria determining professional standing of faculty — including but not limited to such matters as tenure [and] promotion” (C.3 and a.(4)). This bypassing of faculty authority is especially concerning because MU has been under AAUP censure since 2016 and because serious mistakes were made at MU with the promotion process last year (as discussed in the October 15, 2020 Faculty Council meeting).
Furthermore, President Choi never consulted MU Faculty Council before making this decision, even though its members are the elected representatives of the faculty. Even after making this decision, the President never informed faculty about the new practice. But he did inform the Board of Curators at its February 4, 2021 meeting, where he presented a slideshow stating that “Chancellors [at the UM System campuses] are required to use Academic Analytics as a tool for promotion & tenure” (slides 10-11, emphasis added). If he believed that this new practice was sound policy, why did he not announce it to the faculty once it was formulated?
Ensuring fair and transparent promotion policies and practices is understandably a fundamental concern for faculty. Faculty tasked with reviewing their colleagues’ qualifications for promotion devote a lot of time in this effort and are expected to take this work seriously. Faculty seeking promotion work hard for many years to meet and preferably exceed standards and criteria that have been agreed to and recognized by faculty within their discipline here and at peer universities. Peer review, the standard by which research and creative works are judged worthy of funding and dissemination, is a core value of the academy. Peer review is implemented by carefully designed procedures that solicit outside evaluations from distinguished academics at comparable universities in the candidate’s field. The unilaterally imposed use of AA data poses a threat to this core value, and to the procedures designed to implement it.
Moreover, to ensure fairness and equity, standards and criteria for promotion should be communicated in writing to candidates at the point of hire or at the point of the previous promotion. This is sound and uncontroversial institutional practice. If MU administrators disagree and wish to propose a different practice, then they are obligated to present their views in writing to the faculty. To that end, we are requesting written responses to the following questions.
N.B. The term “AA data” as used in this document refers to any data, at a comparative (“benchmark”) or individual level, directly or indirectly derived from Academic Analytics, including but not limited to total honorific awards, grant dollars, average yearly journal articles, average years to publish a book, average yearly citations, or average conference proceedings, whether over a faculty member’s career or for a specified timeframe, and whether calculated per individual faculty member, department average per faculty member, national mean and/or median, or AAU public mean and/or median.
Origins: When was the decision made to use AA data to evaluate promotion and tenure
cases? Who was consulted? Why did the President inform the Board of Curators about
this new practice but not Faculty Council? Were AA data used in evaluating promotion
and tenure cases during the 2019-20 academic year? If so, how and by whom? Did
President Choi consult and/or use AA data last year when reviewing promotion and
tenure cases in his role as Interim Chancellor?
Access: Do faculty who are being considered for promotion have access to all the AA data being used to evaluate them? When describing how AA data are now being used for promotion and tenure cases, both at the March 18 Faculty Council meeting and the March 24 General Faculty Meeting, Provost Ramchand referred to “benchmarks,” presumably data from AA’s “Benchmarking Suite.” But according to a UM System website and comparison chart, only “academic administrators, such as deans and chairs” have access to that data. Similarly, the MU Campus Data Hub notes that AA “Comparative Benchmarking Data” are available only to “designated administrators.” Therefore, all other faculty do not have access to the AA data now being used to evaluate their promotion cases.
At the General Faculty Meeting, a faculty member mentioned MU guidelines for the use of AA developed under Vice Provost Mardy Eimers. That document is from 2017 and is apparently no longer available online at MU, and certainly not here. That 2017 document, which never mentions the use of AA data in evaluating faculty for promotion and tenure, states that:
In terms of providing access to the Academic Analytics data, the answer to the following question plays a significant role in determining this access: “does the individual have a managerial right to know?” In other words, does the individual maintain a level of authority over the unit that would enable him or her to initiate improvements within the program? For example, a dean has a managerial right over all of the departments and programs in his or her school/college, while a department chair has the managerial right over only the programs in his or her department.
According to this logic, the vast majority of faculty do not and will not have access to AA data for the “benchmarks” now being used in promotion and tenure cases. Is this restricted access because of the UM System’s contract with AA? Our understanding isthat administrators are not permitted to share the full AA data with faculty; the 2017 document cited above states that access to AA data “requires . . . user agreement tofollow AA identified restrictions for use.” How can the administration justify, in the context of individual promotion and tenure decisions, using information that faculty cannot access, have not had an opportunity to scrutinize, and never saw at the initial stages of their paths toward promotion?
Transparency: As detailed above, promotion criteria have now been introduced that have
never been communicated to candidates, a practice that violates the core principle of
transparency. How can faculty be expected to meet standards for promotion and tenure
when they do not and cannot know what those standards are?
At the March 18 Faculty Council meeting and the March 24 General Faculty
Meeting, administrators defended the introduction of additional information not in
promotion dossiers, presumably referring to the policy in CRR 320.035. That CRR states
that those reviewing dossiers may “solicit additional information” to determine if the
candidate is qualified for promotion (A.2.a.(1).(ii) etc.). However, what administrators
have so far failed to reference is a relevant sentence in the Provost Call Letters, the
campus documents that more fully outline the promotion process. Page 2 of the tenuretrack
and non-tenure-track versions of that document states that “If additional materials
are solicited at any level, those should be added to the dossier and the candidate should
be notified about the addition.” If AA benchmarking data are being used in any way to
evaluate a candidate’s promotion case, are candidates being notified? If such data are
used in the context of a negative decision that could lead toward denial of promotion, is
full access to that data being provided to candidates? That is, are candidates being given
the complete information so that they could present a fully informed appeal of a
negative evaluation or decision?
Equity: When promotion guidelines are not clear—and especially when they are not provided
in writing—they will likely be applied in inequitable ways. Historically, the application of
unclear or unwritten criteria has been used to discriminate against female faculty
and/or faculty of color. Given such concerns, has the administration adopted policies to ensure that the use of AA data in promotion will not be used in inequitable ways? If so,
what are those policies?
Not following AA’s own guidelines: In 2016, faculty at Rutgers University and elsewhere
raised concerns about how their administrations were using AA. The company
responded by clarifying that “it opposes using its data in faculty personnel decisions”
(Inside Higher Ed) and that “the company does not now believe that institutions should
use its information to make individual personnel decisions” (The Chronicle of Higher
Education). There is no evidence that AA has changed that position since 2016. Notably,
its current list of “How Universities Use Our Data” does not mention promotion or
tenure.
Other universities have publicly stated their opposition to using AA data for
promotion and tenure decisions. Purdue University, like MU a member of the
Association of American Universities (AAU), does not use AA data “to evaluate individual
faculty.” Northeastern University does not use AA data for “promotion and tenure
deliberations.” At the University of Illinois-Chicago, “The utilization or referencing of
Academic Analytics is entirely prohibited . . . [for] Promotion and tenure decisions.” Last
year, the Provost at the University of Massachusetts-Amherst stated categorically that
“the Academic Analytics company itself advises against the use of this tool in assessing
individual faculty . . . Academic Analytics data should not be considered by internal
evaluators . . . in academic personnel actions.”
Therefore by using AA data for promotion and tenure decisions at MU, the
administration is by definition misusing these data. On what basis does the MU
administration justify this misuse? Or does the administration have written guidelines
from AA about how its data ought to be used in promotion and tenure decisions? If so,
why have these guidelines not been shared with the faculty?
Costs and misallocation of funds: In 2017, the UM System signed a 3-year contract with AA
for $954,000. In 2019, that contract was extended for two more years and upgraded at a
cost of $1,384,000, which was “paid out of UM System strategic funding for Research &
Creative Works.” That source of System funding was announced as $50 million over 5
years, or an average of $10 million per year. Diverting almost $1.4 million from this
source of System funding to AA over a two-year period amounts to a nearly 7%
reduction. Therefore, money originally allocated to support faculty research was
redirected to an outside company that attempts to track faculty research. Why should
direct funding of faculty research be sacrificed in this way?
We look forward to the MU administration’s written responses to these questions, and
we encourage faculty to share this document and to raise other concerns.
From the following members of the MU AAUP chapter leadership team:
Theodore Koditschek, President (koditschekt@missouri.edu)
Srirupa Prasad, Co-Vice President (prasads@missouri.edu)
Francisco J. Sánchez, Co-Vice President (sanchezf@missouri.edu)
Martha Kelly, Secretary (kellymartha@missouri.edu)
Nathan Hofer, Treasurer (hofern@missouri.edu)
Members of the Executive Committee:
Elisa Glick
Rabia Gregory
Seth Howes
Stephen Karian
Charles Munter
Mary Jo Neitz