Our office was reviewed "externally" by a faculty/administrator committee as a
result of questions asked by our own administrators during our accreditation
process.  I'm sure many of you can guess the kind of review we received. (It
wasn't awful, but it wasn't particularly useful either) The description provide
by Bill of institutional evaluators is a tactful description what happens when
people outside a profession conduct an evaluation. I mention our review because
I think there were essential elements of this review that hold true for any
review or assessment.  Those that do the evaluating/review must have explicit
criteria for review and uniform standards for measurement.  Even if we are to
have comparisons across peer institutions, done by unbiased research
administrators, enough variability remains to suggest that evaluations have to
be customized to fit the unit being evaluated and evaluation criteria have to be
explicit.  Imagine giving an exam on a topic you didn't teach, minimally covered
or taught very differently from the exam questions.  What good would that exam
be to anyone?  Whenever we get involved in evalation we have to be very clear
about what we are evaluating and how we are doing it.
When we discuss evaluation we can cover a variety of areas including: best
practice, customer/client satisfaction, achievement of unit goals, and
allignment with institutional mission and strategic plans.  Since we serve many
masters (feds, univ. administrators, faculty) we also have to be aware of the
type of information we're collecting and who we'll be giving it to.  I always
advise anyone who is developing an evaluation plan to first evaluate what they
value, start small and simple, and then gradually develop a systematic ongoing
plan for evaluation.

The idea of an accrediting board or perhaps an audit committee of SRA/NCURA is
an interesting one. Isn't there a private firm that performs these services?

Sorry if this sound too much like a lecture.  I'm a former evaluator turned
research administrator and it's hard to resist the opportunity to say something
about evaluation.

Ann McGuigan

William New wrote:
>
> I think the opinion of the faculty is important, since they are one of our primary
> customers.  We have important external customers, (our sponsors and ensuing
> restrictions and regulations) and other important internal customers (Deans,
> Presidents, Trustees) to whom we must be responsive.  The interface of these
> responsibilities presents our conflicts and our challenges, and is also the setting
> for the "art" of effective research administration.
>
> Given that the faculty sees often only one side of the story, a true assessment must
> somehow weigh the other perspectives.  How can this be done?  Sometimes the executive
> and governance structures of our institutions are not well enough versed (and have
> little desire to be so educated) to balance faculty service with the encumbrances
> required by a very highly regulated environment (university science) and the
> restrictions of our sponsors.  They rely on us to keep them out of trouble.
>
> Faculty opinion of service is important, but what would probably be more valuable to
> the faculty is an independent assessment that the research administration policies
> and procedures at their institution (choose one depending whether its a bad or good
> day) (a) are no more restrictive or annoying than those that at comparable
> institutions housing their colleagues and competitors or (b) facilitate their
> activities at least as effectively (if not more so) than those practices at
> comparable institutions.
>
> What is called for, it seems to me, is some sort of peer review of the institution's
> research administration.   Peer review has served the science community very well for
> fifty years.  Peer review in animal facilities (AAALAC) and in hospitals (JCAHO) has
> driven a continually improving environment.  I am not proposing accreditation, rather
> wondering whether some ad hoc or voluntary system would be worthwhile.   If you have
> experienced some sort of external review, I'd be interested in hearing what it was
> like and whether it was of value.  I'd be interested in knowing whether SRA or NCURA
> have had presentations on this.
>
> Bill New  xxxxxx@tch.harvard.edu
>
> mjs wrote:
>
> > Bill:
> > Thanks very much for the reply. Yes, please send  a copy of  the "instrument" you
> > use. Unfortunately, no one on the list seemed to have anything. I had determined
> > that I would have to write one. So, that goes on the stack of  "things to do".
> >
> > Your interest and help are appreciated.
> >
> > Sincerely,
> >
> > Martha J. Spalding
> > Coordinator, Sponsored Programs
> > Marshall University Graduate College
> >
> > William Campbell wrote:
> >
> > > Martha, I just received your query this morning, been on vacation.  We've done
> > > 'quality evaluations' twice in the past few years, using a survey we wrote
> > > ourselves.  I think that's a wise strategy, since each university does research
> > > admin differently.  We asked specific questions about the specific services we
> > > offer requested suggestions for improvement.  Got some interesting feedback,
> > > too, resulting in our recent switch from paper-based communications with
> > > electronic backup to electronic communications with paper backup.
> > >
> > > I'll be happy to email you a copy of our surveys if you like.
> > >
> > > Regards, Bill
> > >
> > >
>
> ======================================================================
>  Instructions on how to use the RESADM-L Mailing List, including
>  subscription information and a web-searchable archive, are available
>  via our web site at http://www.hrinet.org (click on "Listserv Lists")
> ======================================================================

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Ann T. McGuigan, Ph.D.
University Research Coordinator
University Research Office, Illinois State University
Campus Box 3040, Normal, IL 61790-3040

tel: (309) 438-2528    fax: (309) 438-7912
xxxxxx@ilstu.edu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

======================================================================
 Instructions on how to use the RESADM-L Mailing List, including
 subscription information and a web-searchable archive, are available
 via our web site at http://www.hrinet.org (click on "Listserv Lists")
======================================================================