Mail*Link(r) SMTP RE>Hit Rate
This is a very interesting concept. In thinking about it I'm reminded how much
we must look at our own institutional culture and mission when making
decisions like these. I work at a small institution and our grants efforts
are firmly rooted in the concept of faculty development. We've been careful
not to measure our success by monetary gain.
I've been collecting data on the number of proposals submitted, the number
funded and $ amount. We look at the first two very closely since we feel they
measure faculty effort better than the $ figure. (It sometimes takes just as
much effort, and produces just as much benefit in terms of faculty
development, to write a proposal which yeilds $500 as one which brings in
$500,000.) I keep a database and keep track of about everything I can think
of so that I can, if I want to, track growth in a particular department or
school, look at productivity in the terms that Bill talks about etc. etc.
I also keep track of things like how many times I research funding for people,
how many consultations I have, how many times I provide technical help or help
in the actual proposal preparation and writing. All of this information is
very useful to me and gives me some data upon which to make decisions. (It
really doesn't take any time to collect this data once you have a system set
up -- I realize it sounds as though I do nothing but collect statistics!!) As
a bonus, it's also handy when people start wondering what it is that I
actually do!!!
I'm a one-person pre-award office and the information I'm looking for reflects
that focus.
We also keep track, post-award, of money actually received. I've never been
concerned with attempting to reconcile my preaward figures with those. At the
preaward level I'm measuring dollars promised which can vary greatly from
dollars actually received. (Awards may be multi-year; money can be rescinded,
projects may spend less that was anticipated etc.)
And, like Mary Watson, I count each time a proposal is sent to a different
sponsor as a new proposal.
------------------
Alexandra Thompson
Director, Grants Office
Armstrong Atlantic State University (formerly Armstrong State College)
Savannah GA.
--------------------------------------
Date: 7/14/96 2:25 PM
From: Research Administration Discus
It seems to me that year submitted v year rec'd issue shouldn't
matter too much provided you measure it consistently. You're really
looking for changes over time.
I've been thinking about this general issue for some time, and it
might be that perhaps a more meaningful indicator could be "yield".
Under this concept one would worry less about the numbers of proposals
and "hit rates", and would focus more on something like "cost per $
awarded", or "unit cost per successful grant", "cost as % of average
award amount". A hit rate tells me a narrow part of the "success"
story; a "yield" fills it in a little more. For example, we may both
have 50% hit rates (think big!) but you may be more "successful" if
your grants are bigger, or if it's costing you less to process each
proposal, or you're submitting a lot more proposals, etc. By using or
adding a "yield" indicator you can make more meaning of the hit rate.
I think both "hit rate" and "yield" have the weakness of being subject
to a lot of year to year random variation, especially when dealing
with small numbers of proposals.
Bill Kirby
NSF
703-306-1102
______________________________ Reply Separator
_________________________________
Subject: Hit Rate
Author: Research Administration Discussion Group
<xxxxxx@health.state.ny.us> at NOTE
Date: 7/12/96 10:52 AM
We are a small but growing research institution, and we are trying to
determine the best way to measure our success rate. We are interested in
hearing how other universities handle this in general. Do you measure
new proposals submitted in a given year against new proposals received in
that same year regardless of the year submitted? Do you do a one-to-one
correlation (e.g., if proposal X is submitted to NIH and is funded, that
counts as a hit regardless of the year submitted and received)?
A more specific issue that puzzles us is how should we count a proposal that
is submitted to several different sponsors. For example, if a researcher
submits the same proposal to three different sponsors, does that count as
3 proposals or just 1? If the submissions would be counted as just
one proposal, what if the same proposal is submitted to two different
sponsors, but the researcher modifies the name? Research administration
does not necessarily have the technical expertise to know that they are
the same.
Thanks in advance for any information you may be able to share.
Linda
(xxxxxx@umbc.edu)
------------------ RFC822 Header Follows ------------------
Received: by mailgate.armstrong.edu; 14 Jul 1996 14:20:34 U
Received: by gate1.health.state.ny.us; Sun, 14 Jul 1996 11:19:47 -0400
Message-Id: <xxxxxx@gate1.health.state.ny.us>
Received: by gate1.health.state.ny.us; Sun, 14 Jul 1996 11:19:47 -0400
Received: by gate1.health.state.ny.us; Sun, 14 Jul 1996 11:19:47 -0400
Date: Fri, 12 Jul 1996 14:25:33 EST
Reply-To: Research Administration Discussion Group
<xxxxxx@health.state.ny.us>
Sender: Research Administration Discussion Group
<xxxxxx@health.state.ny.us>
From: Research Administration Discussion Group
<xxxxxx@health.state.ny.us>
X-From: Bill Kirby <xxxxxx@NSF.GOV>
Subject: Re: Hit Rate
X-To: xxxxxx@albnydh2.health.state.ny.us
To: Multiple recipients of list RESADM-L <xxxxxx@health.state.ny.us>
