Proposed Principles for Policy Evaluation

Having spent a term on DIVCO, I believe that we ought to use a framework for policy analysis.

Background:

  • All policy proposals that have a material effect on faculty time compete with our terminal goals: research and teaching.
  • Senate leadership has emphasized that all new obligations ought to be offset by release of other burdens.
  • These non-binding principles are intended to motivate conversation and least-cost-necessary approaches to managing policy issues.

Questions with evaluative factors:

  1. What specific value is the new policy intended to create (what is the policy goal)?
    • Benefits should be construed broadly and we ought to give appropriate weight to norms such as fairness and equity.
  2. What is the evidence that a policy intervention is net useful? And how strong is that evidence?
    • Is the policy motivation driven by concern in/with just a few departments, or is the concern broad enough to justify campus-wide policy?
    • If the policy proponent invokes compliance or legal issues, has the proponent obtained a formal legal analysis and/or conclusion supporting the policy?
  3. What are the direct costs in faculty time of the proposal? How are they distributed across the faculty and staff?
    • The Berkeley campus is a diverse place. We have faculty whose work is desk research, while others run large labs with many different kinds of employees and obligations. Did the policy proponent consult with stakeholders? What input on the costs of the proposal was obtained?
    • Are there less time/labor intensive alternative means of capturing most of the benefits of the proposal, i.e. is their a “good enough” alternative approach?
  4. What are the indirect costs of the proposal?
    • Will the proposal cause unintended effects, such as shifting faculty incentives away from supervising more students, from taking on more grants, by creating disincentives for service, or by creating incentives to operate their activities through private means (shadow OPS, shadow IT, spending personal funds)?
  5. Does the proposal erode professional deference and judgment?
    • Does the proposal treat faculty as untrustworthy or withhold information from them?
  6. What is the evidence that the proposal will actually works?
    • If the evidence is that the policy “can’t hurt,” see 2. And 3. This is not enough evidence of efficacy.
  7. What machinery is proposed to evaluate the policy in its first few years to see if it should be abandoned?
    • Is an automatic “sunset” on the policy justified (a date by which the policy expires without action to renew it)?
  8. Who is accountable for the policy’s management and operation?