Stats #41: Writing a Research Grant

Content: There are four things you should think about when writing a research grant:

  1. Theme
  2. Blueprint
  3. Legibility
  4. Perspective

Objectives: In this class, you will learn how to:

I will also provide copies of two (2) successfully funded research proposals for class discussion.

Teaching strategies: Didactic lectures and small group exercises.

This class does not qualify for IRB Education Credits (IRBECs).

Contents

Overview of the STATS web pages (January 21, 2000)

What are the STATS web pages?

The STATS pages are a collection of handouts that I use in my job as a statistical consultant. The web provides a nice home for these handouts, because as I update my material, the newest version is immediately available to anyone who is interested.

Where can I find STATS?

If you have a web browser, like Internet Explorer or Netscape Navigator, you can surf on over to my site,

http://www.childrensmercy.org/stats

which is also found at http://internet1/stats, if you are attached to the Children's Mercy Hospital network. There are two obsolete sites: http://www.cmh.edu/stats and http://simon/stats. Do not use either of these sites.

Some of the fun stuff you can find on the STATS web pages.

Ask Professor Mean.  For the tough Statistics questions that Dear Abby won't touch.

Planning Your Research Study.  Things you need to plan for before you start collecting your data.

Selecting An Appropriate Sample Size.  How much data do you really need?

Managing Your Research Data.  Everything you want to know before you step to the keyboard.

Steps In a Typical Data Analysis.  I have my data on the computer. Now what?

How to Read a Medical Journal Article.  Reading a journal is hard work. Here's some help.

Professor Mean's Library.  Good books and good web sites about Statistics.

... and even more good stuff!!!

This webpage was written by Steve Simon, edited by Linda Foland, and was last modified on 07/08/2008. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page. Category: Website details

For CMH employees only: Statistical Consulting Services.

You can get free statistical consulting if you work for Children's Mercy Hospital. Steve Simon and Ashley Sherman provide a wide range of statistical consulting services to help you with your research projects. This help can start as early as the initial planning of your research. I also help with the analysis of your data, using SPSS or other statistical software. We can also provide assistance with the preparation of your presentations and publications.

Here area some examples of the services that we have provided:

Specific statistical advice has been outlined on a series of web pages which can be found at http://www.childrensmercy.org/stats/. The pages provide advice about planning your research, selecting an appropriate sample size, managing your research data, performing a variety of data analyses, presenting research data, and writing research papers.

How to get in touch with a statistician

If you would like to meet with Steve Simon or Ashley Sherman, you can set up an appointment by emailing or calling Judy Champion (jmchampion (at) cmh (dot) edu or 816-983-6784). If you have a very simple question, send an email directly to us (ssimon (at) cmh (dot) edu and aksherman (at) cmh (dot) edu).

This webpage was written by Steve Simon on 2003-04-30, edited by Steve Simon, and was last modified on 2008-07-08. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page. Category: Professional details

Directions to my new office (April 25, 2008).

I have moved to a new office. It is a modular building just north of Children's Mercy Hospital. It is between 23rd and 22nd street, just off of Kenwood Avenue (Kenwood is a small north/south street just west of Holmes). If you need to get from your office to mine, here are some directions written by my Administrative Assistant, Judy Champion.

This webpage was written by Steve Simon and was last modified on 2008-07-14. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page. Category: Professional details

Where do research ideas come from? by Ronan Conroy (September 20, 1999)

This is an HTML format version of an email by Ronan Conroy on April 9, 1999 to edstat-l, an Internet list and to sci.stat.edu, a USENET group. This email summarized a presentation he made about how to develop ideas for research. I have made some minor formatting changes (mostly the use of bolding, bulleting, and indenting to highlight the major themes), but all of the credit for writing up this summary belongs to Ronan Conroy. Part of this presentation represents a summary of discussions on edstat-l and sci.stat.edu. Here is the acknowledgement in Dr. Conroy's original email.

I'd like to thank the many people who took part in the discussion, or who wrote to me privately, and to stress that the quotes in it are often the person who made the point most memorably, rather than the only person who said it.

Many thanks to Chris Zorn, Gabriele Susinno, Giovanni S. Leonardi, the... inimitable dennis... roberts, Joe Ward, Michael Granaas, Roland Andersson, Jay Warner, Alex Heath, The Anthonys, Bob Frick, Jerry Dallal, Tjen-Sien Lim, Tim Cole, John F. Schnell, Robert C. Knodt, Paul Velleman and Joseph L. McCrary. Did I say Herman Rubin? I did now.

This material is reproduced with permission from Dr. Conroy. For what it's worth, I have included a copy of the original email. It's pretty clear that there was some formatting in Dr. Conroy's original document that got lost when it was translated to a text format for email.

Introduction

This paper tackles one of the questions that statisticians dread most: the most basic one of all. How do you start formulating a research project? It began life as a talk at a research seminar in the Rotunda Hospital, Dublin. Trying to write it up, I decided to mail the statistics lists that I subscribe to. This paper has been greatly enriched by the ideas and discussion generated on edstat-l, the statistics teaching list, as well as contributions from subscribers to the stata list and the UK statistics list allstat. Quotes are often attributed to the only person who made the point most memorably, but many of the ideas emerged repeatedly in different postings. I'd like to thank all those who took part in the discussion.

Exploring your environment

The first thing you need to do is identify your resources for research. This is often easier when you first arrive somewhere. After a while you begin seeing an environment as the place where you work or live or eat. You need to see it with a fresh eye to see it as a potential research environment.

Don't forget that your research environment includes not just your patients and your colleagues, but also includes any source of data, ideas or help that you have access to. Many of my own research projects have taken shape because my office is next door to the psychology department; a casual remark has often triggered a flurry of speculation, articles rooted out, contacts mentioned and so on.

The internet is also a valuable environment. Discussion lists abound,which can provide not just free advice but also an insight into current controversies and new directions in research. Simply subscribing to a list and reading the postings (the word for a person who does this is a lurker) without taking part in the discussions will often give you ideas.

General resources

How much time will you be able to devote to research? To what extent can you integrate it into your daily work?

Will colleagues help? For instance, if you need blood taken outside working hours, will the doctor on call oblige? Will nursing staff collaborate by collecting extra information?

Do you have access to a person, unit or department with a specific research interest? They can often be a useful source of ideas. Never underestimate the value of just going for coffee with someone who does a lot of research, or, better, a research team. The speed with which a bunch of researchers can take a vague idea and shape it into a research design is amazing. Most of these ideas go nowhere, but eavesdropping on the process can help you to do it yourself.

Giovanni Leonardi of the Environmental Epidemiology Unit at London School of Hygiene and Tropical Medicine put it like this: "There are many potential research ideas that never make it to becoming research projects, and the likelihood that a research idea will become a research project is heavily influenced by this idea having being selected and refined in an environment where potential ideas are routinely tested for their viability. Think of this as 'natural selection' of research ideas within the research environment."

Do you have access to a statistician, or someone who can advise you on study design and sample size?

What library facilities do you have access to? Skimming journals is a good ideas generator, which I will deal with in more detail later; but access to a good library, including literature searching and reprint ordering facilities, is a must. Add extra points for library staff who are willing to do literature searching with you looking over their shoulder to refine the search.

What computer facilities are available?

What are they funding this year? This sounds like a cynical point, but if there are funds available for research in specific areas, make use of them. What charities are there who might be interested in your research area? Talk to colleagues; there is often no single listing of available research sponsors, and you have to rely on the grapevine.

Specific resources

Do you have access to information already collected which could be the basis for a research project? This information could have been collected as routine clinical information. Although you probably cannot do a research project solely on the contents of patients' charts, routinely collected information may allow you to

Information may also be available as an offshoot of another research project. You may liaise with another research project and

It is a good idea to talk to people who are doing research in the setting in which you work. They will be able to spot potential difficulties in proposals, and may also have useful ideas as to what they would do if they had access to your facilities.

Potential projects

Now all you need to do is to get an idea for a project which will be realistic, given the resources available to you. This is often a stumbling block. I had one person come into the office to discuss a research project with me. 'I have 24 patients with rapid cycling mood disorder' he said. And stopped, waiting for me to say something. The trouble is that 24 patients with rapid cycling mood disorder is no more a research project than 24 trout in a shoebox. What you need to ask yourself is 'what do we not know about rapid cycling mood disorder'?

One very important piece of advice that recurred frequently in the edstat-l discussion was the need to develop many ideas simultaneously. Christie Brown, Assistant Professor of Marketing at University of Michigan Business School tells her students to imagine that inside them they have a large basket of research ideas, some better than others:

'I point students toward Donald Campbell's work on creativity. Campbell suggests one secret to generating better ideas lies in the QUANTITY of ideas generated. In other words you stand the best chance of pulling an idea from the "high" end of your good-idea basket if you make a lot of draws.' (Campbell, Donald T. "Blind variation and selective retentions in creative thought as in other knowledge processes." Psychological Review. 1960;67:380-400.)

Don't focus prematurely on a single idea. Develop a few together. It's like the process of conception: the chances of a child resulting from a single act of sexual intercourse are small. But the chances of a child not resulting from regular sexual intercourse are likewise small. Carry a notebook and write down every idea that you get, good or bad. You will learn from thinking about why the bad ones are bad as well as from why the good ones are good.

Christie Brown again: 'Write down everything. Do not self-censor. Keep a log of your baby-ideas in case they end up being worth pursuing. Get in the habit of generating at least one idea based on everything you read in your domain and even out of it.

Bob Frick, a cognitive scientist, actually forces students to develop a number of research ideas as a learning exercise. 'The assignment was to come up with three "kernels", and the students had about a month to do it. The notion was that they were supposed to find some original idea they had. It usually ended up being an original observation. Original to them -- it didn't have to be original to the field of psychology. Their original idea would then be a kernel that could be developed into an experiment. Most people have these, but they don't pay attention.'

Extending the ideas of others

Much of the discussion on edstat-l centred around where ideas for research projects come from. The sources of ideas divide into two:

I'll take the easy one first!

Repeating research that has been done by others doesn't sound like task, but there are several important reasons why it needs to be done, and there are some other benefits too. The reasons why research needs to be replicated include:

Local research is needed to make sure that findings from other countries apply locally. Indeed, basic research is constantly needed to monitor local health needs and to evaluate the services being delivered.

All research needs extension to new contexts and development along an obvious line - Clinical trials are often done on homogeneous, idealised patient groups; they need extension to realistic groups such as those with comorbidity, or beyond the age range of the original research. Think of

Factors which have been identified in a disease may be present in other similar diseases. Since its role in peptic ulcer disease was uncovered, H pylori has been investigated for many other unsolved crimes.

Yes, there is a feeling of a bandwagon rolling along, but someone has to check out these questions.

You may spot an explanation which the original study failed to identify and test. This is, of course, classic 'stroke-of-genius' research. Just remember, though, that the explanations that are most often overlooked are the commonest, most familiar things.

You may not believe a piece of research. Not all research is good research. I have, several times, replicated and extended research because I didn't believe it. Incredible research deserves to be replicated. If you confirm the original findings, you have helped to overcome the resistance that they will find in being accepted. If you fail to confirm the findings, this in itself is interesting. Though try to make sure that the original author isn't asked to review your paper!

Even doing a straightforward replication of a previous study can be a very worthwhile exercise. As a first project, it means that you already have a 'canned' methodology, and you will learn a lot about running, analysing and presenting research, But there are often surprises too.

Chris Zorn of Emory University wrote: 'As a social scientist responsible for training grad students in statistics, one thing that I've always found useful is replications While the main reason I use replications is to teach students statistics and/or software, these exercises often prompt them to extend the work they are replicating. These can range from the simple (e.g. testing for relationships in the data that the original investigators didn't look for) to the very involved. The result is often interesting, if a bit derivative, research projects, some of which have led to PhD theses, etc.'

Andersson Roland puts it simply: Dig where you stand. That is, make use of all the data that is already at hand and that nobody had time to analyse. Almost always there will be unexpected or unknown patterns in these data that can be detected if you analyse them with an open mind. You do not always need to have an research idea ready when you start. They will come up when you try to formulate an explanation for the patterns that you find in your data.

Alex Heath, an economist from Australia, wrote: A good way to get started thinking about research questions for me is to find things which have been done overseas (usually the US or the UK) and adapt them to Australian data. I find that once you start replicating things you find interesting twists and turns which allow you to say something completely new.

Although I have replicated several studies because I didn't believe them, this probably isn't the best spirit in which to replicate. But neither should you simply accept the original research as scripture. Paul Velleman, the person responsible for the DataDesk statistical package and ActivStats, a statistical teaching package, wrote in praise of an attitude of well-informed skepticism: This misses the most important part of the process -- an abiding skepticism. You must know your science before you can be intelligently skeptical about it, but just because you know what is common wisdom doesn't mean you should believe it. Indeed, if science is to progress, you must maintain a willingness to disbelieve. You don't do research by replicating previous results but by doubting them.

Dennis Roberts, responding to this, said: a good replication study does not have to be done BECAUSE one doubts them but rather, to bolster the case that the research findings made ...

I think that he and Paul really just differ in emphasis, with Dennis arguing that 'replication is very valuable ... we don't do enough of it ... ' while Paul cautions against literal-minded repetition. I think everyone would agree that the scientific idea of replication is doing something more intelligent than just looking for what the other guys already saw.

Paul makes the point, too, that it is hard to sit down and work carefully through a set of data without coming up with at least one pattern that needs further investigation. You may start by replicating a study, but this is almost guaranteed to act as a springboard to innovative questions of your own.

Getting a research idea by reading papers.

You can simply bury yourself in the library with a whole year's worth of your favourite journal and, starting from the most recent issue, use a series of filters to identify studies that you would be interested in and capable of extending. Even when I'm not in need of a research project, I often graze my way through a small stack of journals, picking up an interesting methodological approach here, or a useful measurement technique there. Many of my more prolific colleagues do this a lot. One, in particular, seems able to rummage out a half-a-dozen relevant journal articles from her shelves on any topic in about five minutes.

If I am looking for a potential project, I look at each article in turn and ask:

Getting your own ideas

This is an even harder subject to write about than extending and developing the ideas of others. (Did I say plagiarising? -- Never!). The secret seems to be keeping your eyes and ears open all the time. The observation doesn't have to be complicated. On the contrary, spotting an obvious question in an everyday event often has greater potential.

Jack Schnell of Department of Economics at the University of Alabama in Huntsville remembers simple advice he got as a student: 'look out of the window', meaning 'pay attention to what is happening out there in the world, look for issues that are ripe for investigation'. And since that time I have tried to do just that. For me, this has been more intellectually sustaining than, say, combing through some literature in the hopes of seeing a useful extension.

A simple observation can spark off a whole train of ideas. Roland Andersson, of the Department of Surgery in Joenkoeping, Sweden, said For me it started like this: I observed that we had had 12 patients with appendicitis during one week. The following weeks we had only one or two. I wondered: 'Had we had an epidemic of appendicitis?'. I happened to know about Knox space-time analysis and I started off from there and finally have written a thesis about 'Appendicitis - epidemiology and diagnosis'. Lots of new questions arise and I am now involved in a (as it seems) never ending project about aspects of appendicitis. (And please, don't worry if you have no idea what Knox space-time analysis is; the important point is that Roland brought together a specialised theoretical framework which he already knew and a common everyday observation. In other words, he applied the theory he knew to the world outside the window.)

But what frame of mind, what view of the world do you need in order to have productive research ideas? A lot of discussion focussed on this question. At one extreme was Robert Hamer, who very much doubted whether you could teach anyone how to look at the world in a questioning manner. I don't think that this is true, though. We are brought up in a way that does not encourage us to question the explanations we are given for things. Don't forget that all children are hungry to find things out, to know why things are so. This voice of hunger for knowledge and delight in figuring things out is much smaller and more timid by the time we have grown up, but with patience it can be called back. It takes time to rid ourselves of this learned uncuriuosity.

The trick is doing what children do: asking lots of questions and teasing out the logical consequences of the answers. Paul Velleman again: "Dennis is right that the problem is nudging the mind. We need to start that process in childhood. We must cultivate in our children and our students a broad-based skepticism coupled with a sense that there *is* order in the universe."

These are the sorts of questions that scientists and other children ask.

One must maintain an active and abiding skepticism about the explanations and models that have been proposed in science. Skepticism, which Paul Velleman identifies as a key attitude, doesn't involve simple disbelief, but rather being able to entertain a number of different explanations at once.

This struck a chord with Robert Knodt: After being involved with masters and doctoral students for over thirty-years and looking back for an answer to the original post, I find that the statement above applied to over 90% of those I helped... The first person I worked with was bothered by a statement in a 10th grade Biology book which said that trees were pruned in the fall in order to make them fill out areas and become more symmetrical. This still bothered him eight years later. He finally did is work on 'wound' hormones in plants.

Says who? Many pieces of medical knowledge are folkloric, and the evidence is slender. In particular

I don't believe that! Always trust your disbelief. Often a trip to the library will put your mind at rest, but think about

Why are we doing this? At every point in clinical practice there are decision forks. Some may be invisible (we always do X when Y happens) but these are the most interesting! For example

Why are they both right? Some disagreements in the literature are because no-one has yet spotted the reason why two different sets of investigators should have observed data that were seemingly contradictory.

Can we learn from the abnormal? We learn once from describing the normal--normal course of disease, normal range of variation etc. We learn a second time by examining cases that do not fit the general picture. Rare, pathological conditions can give us an insight into how more subtle, commonplace processes work.

Final thoughts

I don't know where ideas come from, but I do know that you get more ideas if you try to remember everything that happens that doesn't have a good explanation. I carry a little black notebook which can simply be used to note phone numbers and things I have to buy next time I go shopping, but it also means that I have a way of writing down an idea the instant I spot something interesting.

The last thing I want to say is based on my experiences teaching music to adolescents, as much as teaching research methods to medical students. The biggest obstacle you encounter is a feeling that you can't do this; that you aren't the sort of person who can sing, or make interesting observations or pose original questions. Just remember: this is what you did as a child, before you were taught any different. So you already know how to do this; just think of yourself as a little rusty.

The copyright for this page belongs to Ronan Conroy. This page was formatted by Steve Simon and was last modified on 2008-04-28. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page.

Developing a research hypothesis (August 18, 1999)

Dear Professor Mean, I want to do some research, but before the hospital won't approve anything until I have a protocol with a research hypothesis. I'm not sure why a research hypothesis is important. Can you help? -- Little Linda

Dear Little,

Think of it as job security for your local statistician.

Short answer

A research hypothesis provides clarity. A problem has to be stated clearly before it can be solved. The research hypothesis will also provide direction for writing the rest of your protocol.

There are several steps that you should follow:

  1. Identify the four components that most research hypotheses have.
  2. Select between a one sided and a two sided hypothesis.
  3. Use your hypothesis to guide the writing of your research protocol.

Stating a hypothesis

Ideally, your research hypothesis should be specified prior to the collection of any data. An exception would be an exploratory study. For example, if you are investigating the cause of poor morale among health care providers, you may not have enough information to specify anything more specific than a whole range of factors that might influence morale.

In general, a hypothesis will have four major components. Not every hypothesis can be fit into this framework, of course, but knowledge of these four components might help you if you have an incompletely formed hypothesis.

The first component is the subject group. In other words, who are you interested in studying? Subjects could be patients, their parents, or the health care providers.

The second component is the treatment or exposure. In other words, what is being done to part or all of your subject group. A treatment implies an action on your part, such as providing information or applying a new therapy. An exposure, on the other hand, implies some action that you do not control, such as lead poisoning or premature birth.

The third component is the outcome measure. In other words, how or in what manner is the treatment or exposure going to be assessed. It is very important that the outcome measure be defined precisely and unambiguously. For example, if your outcome is breast feeding rates, you should use standard definitions of breast feeding, such as those provided by the World Health Organization.

The fourth component is the control group. In other words, who are you comparing to. It is important for the control group to be as similar as possible to those who receive a treatment or exposure.

As mentioned earlier, not every research hypotheses will have all four components. For example, a cross-over design involves applying both a new treatment and a standard treatment using the same patients. For this study, the hypothesis would not involve a separate control group. Correlational studies look at relationships within a single group, such as a study of the factors that cause medication errors. This type of study would not have a treatment/exposure. The structure mentioned here, however, is still useful for developing most research hypotheses.

One sided versus two sided hypotheses

During the planning of your research, you need to specify whether you plan to use a one sided or two sided hypothesis. A two sided hypothesis states that there is a difference between the treatment/exposure group and the control group, but does not specify in advance what direction you think this difference will be. A one sided hypothesis states a specific direction (e.g., increase).

If you expect that a change in either direction is possible and that changes in either direction are interesting, then you should use a two sided hypothesis.

If changes in one direction are uninteresting and unpublishable, then use a one sided hypothesis. Also if a change in the unexpected direction is equivalent in practice to no change, then use a one sided hypothesis.

The best example of this is when you are comparing a new therapy to an existing therapy, where the new therapy is much more expensive, your only concern is to show that the new therapy is better. If it turns out that then new therapy is equal to or worse than the standard therapy, you will not adopt it.

Some important issues involving the control group

With a treatment, where you intervene, it is often possible to select those patients who receive the treatment through the use of randomization. Randomization ensures comparability, because the random selection ensures that, on average, subjects who receive the treatment will be comparable to subjects who do not receive the treatment.

When you have an exposure instead, it is often difficult to ensure that the subjects without the exposure are comparable to the the exposed subjects. Sometimes matching will help, but you should only use matching for very important prognostic variables. For example, birth weight plays a major role in infant mortality, so it is often helpful to match your exposure group to your control group on the basis of birth weight. Matching, however, will often present difficult logistics, especially when the pool of control subjects in not much larger than the pool of exposed subjects.

What are your next steps?

Other important issues to be considered in your protocol is

  1. determination of the sample size,
  2. identification of potential confounding variables, and
  3. what efforts at blinding will be used, if any.

Once you have a well defined research hypothesis, though, these details will fall into place. Hah, hah, did I really say that? The rest of the protocol is still pretty darn hard, but it would have been impossible if you didn't have that research hypothesis.

To determine an appropriate sample size, you need a research hypothesis, an estimate of the standard deviation of your outcome measure, and assessment of how much change is considered clinically relevant. Hey, you're already a third of the way there! Finding a standard deviation requires either reviewing previous research on that outcome measure or running a pilot study. The clinically relevant difference is a judgement that is made solely on medical knowledge. Your statistician cannot tell you what a clinically relevant difference would be.

Confounding variables are those variables which are related to your outcome measure and which may differ between your treatment/exposure group and your control group. Assessment of potential confounding variables is especially important when you cannot randomize.

Blinding means hiding information about the treatment/exposure from the patients, their parents, and any health care professional who interacts with the patients and their parents. Blinding is useful when it can be done, but blinding is not always possible. For example, in a comparison of a drug that is rectally administered to oral administration, the patient usually figures out quickly which group they are in. But even when the patients themselves know which group they are assigned to, you can sometimes still use blinding for laboratory personnel and for interviewers.

Summary

Little Linda needs to include a research hypothesis in her grant proposal, but doesn't know what it should say. Professor Mean explains that you should develop a hypothesis to giveyour research clarity. There are four components in most research hypotheses:

  1. a subject group,
  2. a treatment or exposure,
  3. an outcome measure, and
  4. a control or comparison group.

Other important issues to keep in mind while developing a research hypothesis:

  1. Use a one sided hypothesis when changes in the opposite direction are uninteresting.
  2. Randomization helps ensure that you have a comparable control group.
  3. Use the research hypothesis to guide the determination of sample size, the identification of confounding variables, and the efforts to blind information.

Annotated Bibliography

http://www.shef.ac.uk/~scharr/reswce/question.htm

This site provides information about evidence-based medicine, but much of the material is still relevant to developing research protocols. The four components to a research hypothesis come from this site.

Massey, V.H. (1995) Nursing Research, Second Edition, Springhouse PA: Springhouse Corporation

This book provides a "how to" framework for conducting research in any easy to skim outline format. The book includes topics on ethics, literature review, sampling techniques, data analysis, and presentation of research results. The sections that deal with planning are the best parts of this book.

Lang, T.A. and Secic, M. (1997) How to Report Statistics in Medicine. Annotated Guidelines for Authors, Editors, and Reviewers, Philadelphia, PA: American College of Physicians.

It seems ironic to recommend a book on writing the final results, but it helps to start out with your goal in mind. If you think about the information that belongs in your research paper, then you will have a good idea of what you need to specify during the planning stages of your research. This book also uses an easy to skim outline format, but it has significant narrative text under each outline element.

This webpage was written by Steve Simon on 2008-xx-xx, edited by Steve Simon, and was last modified on 2008-07-14. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page. Category: Ask Professor Mean, Category: Grant writing

Writing a research grant (September 13, 1999)

Dear Professor Mean, I'm writing a research grant to look at the impact of managed care on the care of children with chronic epilepsy. How do I structure the grant so I'm guaranteed to get funding? -- Ambitious Ann

Dear Ambitious,

You're writing a research grant, and you want Professor Mean to guarantee your funding? I can't do that, but if you mention my name at McDonald's you can get 10% off on your order of french fries.

Short answer

It's tricky to provide advice on how to write a grant, because each research problem is different and each granting agency has its own guidelines. I can outline a few of the things that I look for when I am asked to help write or revise a grant application.

First, you need to have a theme, which I also refer to as a primary objective. Your primary objective needs to be precise, prominent, and persistent. Your primary objective will serve to unify the writing of your proposal.

Second, you need to outline your research plan. You should specify what you plan to do, who you plan to do it to, how you plan to evaluate your work, and why you are doing this research.

Third, you need to make your proposal easy to read. An abstract on the first page and a summary at the end provides organization to your proposal. Make your proposal skimmable by highlighting important points with bullets, numbered lists, and tables. Keep your proposal simple, as much as is possible for innovative research, by using analogies and avoiding OMAs (Obscure Medical Acronyms).

Fourth, write from the perspective of the reviewer. Your reviewers want information about the benefits of your research. They want reassurance that you are qualified to perform this research and that you have plans to handle unexpected problems. They also want to see consistency between your work and the interests and priorities of the granting agency.

More details

There are four things you should think about when writing a research grant

  1. Theme
  2. Blueprint
  3. Legibility
  4. Perspective

Before I discuss each of these points, I need to outline some general categories of research.

Three general categories of research

There are many different types of research, so it is hard to describe what your plan should look like. I place research into three general categories:

. These categories aren't perfect, but help guide further discussion about writing grants.

Evaluating a treatment

Some research evaluates an active treatment or intervention for some or all of your research subjects. You will intervene throug the use of medication, surgery, or some other typoe of medical therapy. For this type of research, be sure to provide a detailed description of your treatment or intervention. Be specific. For example, we plan to hold offer a comprehensive weight loss program that includes education about diet, opportunities for exercise, and group counseling sessions.

Evaluating an exposure

Other research may involve an exposure rather than an intervention. The distinction is that an exposure is not under the direct control of the researcher. An example of exposure is breast feeding, which is a choice made by the individual mothers rather than the researcher.

Describing a population

Still other research studies are descriptive; they characterize individual variables and/or relationships among these variables. There are no treatments or exposures in this type of research. An example would be a description of pediatric patients who visit an emergency department. For this type of research, you would be interested in the average age, the proportion who have insurance, and the correlation between waiting time and staffing levels.

1. Theme: Develop a primary objective.

Your proposal should have a unifying theme. I will refer to this theme as a "primary objective" though some granting agencies may prefer to use nouns like aim, hypothesis, or goal in place of objective. It's important to understand the language that is commonly used by the researchers in your area. When they get bored, they invent criticisms like "this researcher is naive because he/she confused the aims with the objectives." These are the same people who argue about how toilet paper rolls should be hung in your bathroom.

Whatever you decide to call it, you must have a primary objective. It provides a unifying theme to your research proposal. Your primary objective should be:

Precise.

You should explain your primary objective briefly and succinctly. Use a single sentence, if possible. You don't want to ramble. Also be sure to use clear and unambiguous language in your primary objective.

Prominent.

You should highlight your primary objective so that it is obvious even to someone who just skims your proposal. This is not the time to be subtle. If the grant format allows it, mention your primary objective on the very first page of your proposal. Otherwise, draw attention to your primary objective by using a text box, shading, or bold formatting. Use language like "the primary objective of this study is" that identifies your theme unmistakably to your reviewers.

Persistent.

You should refer constantly to your primary objective throughout your proposal. Repeat your primary objective at every possible opportunity. You can do this implicitly by paraphrasing your primary objective or you can directly restate your primary objective in part or in whole. Repetition will reinforce your message and will force you to keep your proposal focussed.

Make sure that you have a primary objective and that you gauge all the material in your proposal against this. Revise or remove any material that is inconsistent with or unrelated to your primary objective.

2. Blueprint: Outline the plan for your research.

State very early in your proposal what your research plan is. If the format allows it, give a brief overview of your research plan on the first page of your proposal. The plan should address:

I will illustrate these points with an imaginary research study where we develop an expensive seminar to convince people that they would be happier if they gave us all their money.

What you plan to do.

In an intervention study, you should describe in detail what that intervention is. You should also describe how you will randomize, and what efforts will be made to blind the intervention. Not all interventions can be totally blinded, of course, but you should keep the allocation to the treatment or control group hidden until the patient has agreed to participate and the recruiting physician has verified that the patient is eligible.

For an exposure study, you should focus on efforts to make your control group as similar as possible to the exposure group. Describe any efforts to match, for example, or list potential confounding variables and explain how you will adjust for them.

For both an intervention study or an exposure study, you should also describe any outcome measures that you plan to collect. Outcome variables are those variables that you believe that the treatment or exposure might influence. These measures should be specific, reliable, and valid.

If your intervention is supposed to make your patients poor and happy, you need to provide an operational definition of poverty and use a survey instrument that has been shown by other researchers to be a valid and reliable indicator of happiness.

If you have multiple outcome measures, you should list all of them in a table. If some of your measures are more important than others, indicate this priority by designating primary and secondary outcome measures.

In a descriptive study, you should focus on measurement. List all the variables that you plan to measure. If any variables are difficult to measure, explain how you will measure them.

Who you plan to do it to.

Describe the subjects that you plan to study. Detail what type of subjects you will include in your research and what type of subjects you will exclude. For example, all our patients must be rich and gullible. We will exclude any reporters and law enforcement officers from our study.

Document how many of these subjects you will see in a year and estimate how many of them will agree to participate in your research. Also estimate the proportion of subjects who will drop out during the study. You may not have a very good idea about these estimates, but you still provide your best guess so the reviewers know that you are aware of the issues involving volunteers and dropouts.

How you plan to evaluate your work.

Evaluation usually implies some type of comparison, especially for intervention or exposure studies. This can be a "before and after" comparison (before we intervened, our patients were rich and miserable, now they are poor and happy). It can also be an internal comparison (those patients who attended only one session of our intervention were still rich and miserable, but those who attended all four are now poor and happy). Or it can be an external comparison (those patients who attended our sessions were poorer and happier than a control group who did not attend our sessions).

When your research involves comparison, you should specify how much of a difference you consider clinically relevant. This information will help you justify the sample size for your experiment.

Not all research implies comparison. In most descriptive studies, for example, the focus is on statistical estimates such as means and correlations. In this case, you should specify how precise you want those statistical estimates to be. That precision will help you justify the sample soze for your experiment.

Why you plan to do it.

You need to provide a rationale for your research plan. Explain what you will learn from this research and who will it help. This is an easy thing to ignore, because it is so obvious, at least to you. Some of the "why" will already be in your literature review, but repeat and expand on your rationale when you outline your research plan.

3. Legibility: Make your proposal easy to read.

Reviewers will spend more time reading a proposal that they can read quickly and easily. Less is more. It's better to have a 6,000 word proposal that is read twice than an 8,000 word proposal that is read once. So say less, but say it in a way that will invite the reviewers to read it a second time. You do this by making your proposal:

Organized.

An organized presentation is easier to read. If the grant guidelines allow for it, include an abstract at the start of your proposal and a summary at the end. The abstract and summary should discuss your theme and your blueprint. If the reviewer only reads the first page of your proposal or only the last page, make sure that the reviewer would have a good idea of what you want to do.

Your grant should follow a logical progression. When you are reviewing the literature, for example, start with the general and move to the specific. This sets up the "big picture" and places your proposal in the approprate context of the big picture.

When you are addressing a sequence of topics, use numbers to track the progress of your sequence. The "one, two, three" provides anchors of one, two, three, etc. that allow your reviewer to jump from point to point.

Don't write your grant proposal like a murder mystery. You don't want to keep your reviewers guessing about what you plan to do while you summarize the literature and all your previous research. Also use a logical and predictable sequence (such as general to specific) to make your proposal predictable. In short, you want to avoid any suspense or surprises.

Skimmable.

You want to make it easy for reviewers to skim your proposal. Most reviewers can't devote the amount of time to reviewing your proposal that you (or they ) would like to. When you make your grant skimmable, you encourage the reviewer to 'hit the high points".

Use a topic sentence at the start of each paragraph and make an obvious transition from one topic sentence to the next topic sentence. This allows the reviewer to review your proposal by reading the first sentence in each paragraph.

Space improves skimmability. Use a large readable font. Include a lot of white space.

Use section titles liberally. If the grant guidelines include specific titles, use those title verbatim and in the same order as the guideline suggests.

Use tables, numbered lists and/or bullets to highlight key information.

Don't cram. Cramming is the enemy of skimming. I understand that the temptation to cram is strong. You want to fit in a bit of extra information. So you lower the font size from 12 points to 11.5 and then 11. You fiddle with the margins. You move some required information into the appendix. You combine short paragraphs into long paragraphs. You take out a blank line here and a blank line there. You take out some of the subheadings. And you end up with a proposal that is dense. You're proud of yourself because you got an extra 2,000 words into your proposal. Don't be so proud. In order to squeeze in a few things, you've aggravated the people who will decide the fate of your proposal.

Simple.

Don't assume that the reviewer knows as much about your area of research as you do. Avoid jargon and minimize the number of acronyms that you use. If you have to use acronyms, be sure to define them at their first appearance. Also, keep the acronyms out of your abstract and out of your research hypotheses. If you can't avoid using a large number of acronyms, put in a table that lists all of them.

You should adopt a tutorial approach to the new and unusual aspects of your research. Take advantage of any analogies about your work. Although it may appear as oversimplification, you should remember that no one understands your research as well as you do. If your reviewers don't uderstand your research, they will have a hard time supporting it. We all have a tendency to mistrust those things that we don't understand. Make sure your reviewers understand your science.

Your writing style can also help simplify your proposal. Use short sentences. You should, for example, split any sentence into two or more sentences whenever you see the word "and" appearing more than once. Also use short paragraphs. Not only does this improve skimmability (see above), it also makes it easier for the reviewer to stay on track. Keep your nouns and verbs close to one another. A sentence with a lot intervening words between the subject and predicate unnecessarily causes confusion. With the subject and predicate close together, your reader will better understand your material.

4. Perspective: Put yourself in the shoes of the reviewer.

Your motivation for seeking a grant will rarely be the same as the motivation of the person who is giving the grant. Try to understand what your reviewers want and give it to them.

Reviewers are looking for four things:

Generalizability

One important motivator for reviewers is generalizability. If the work you plan has implications for a lot of patients beyond those that you plan to study, then be sure to brag about this. You are providing leverage; the money spent on your proposal will go a long way. For example, if you are examining the coping skills of children with chronic arthritis, mention how this work will provide direction for coping skills of children with other chronic diseases. If you are examining barriers to health care among Spanish speaking patients, emphasize how this will serve as a model for other language barriers.

Benefits

The reviewer wants to understand the benefits of your work. This is not the time to be dispassionate. Let your enthusiasm for your research shine through. Advocate the value of your work through the severity of the illness and/or the number of people affected. Explain how your work fits into "the big picture."

Although you need to be an advocate, you should avoid sounding close-minded. Too strong an emphasis can give the impression that you already know the answer and you won't let any data get in the way. Avoid any overtly partisan language that might lead the reviewers to believe you are biased.

Part of your advocacy will be stating how your approach is the best. If you can't explain why your approach is the best, you shouldn't be applying for the grant. That doesn't mean that you shouldn't acknowledge the shortcomings of your approach. You should, so the reviewers appreciate your realism. Also you need to acknowledge other approaches and compare their strengths and weaknesses to your approach.

Being an advocate without sounding partisan may sound contradictory, but you should simply avoid any extremes. Either a totally dispassionate presentation or a fanatical zealotry will turn off your reviewers.

Reassurance

The reviewers also want reassurance. They want to feel confident the money will be spent well and there will be a good return on the agency's investment. Some things that reassure your reviewers are:

  1. the training, experience, and resources you have to conduct this research
  2. an explicit time frame for completion of specific parts of your research
  3. a list possible problems that may occur during your research and how you plan to handle them.
  4. a plan for how you will disseminate the results of your work

You can also build confidence in your proposal by presenting a realistic plan and budget and demonstrating care and attention to details.

Consistency

If you want to be successful at writing grants, you need to understand the agency that you are applying to. What are their interests and priorities? Write your proposal to show consistency with these interests and priorities.

This is easy to do. Every agency will have guidelines for grant writers that will tell you what they are interested in. If you are like me, you don't want to bother reading these guidelines. It's like using software without reading the documentation. But reading the guidelines is very important.

Some things you want to look for in the guidelines are:

  1. the purpose of this particular granting program
  2. the eligibility criteria
  3. the grant deadlines
  4. the money limits
  5. start date and duration

Here are some excerpts from a grant guideline.

1. Purpose: invite applications for pilot studies to explore interventions for prevention of obesity in high risk individuals or populations. Applications incorporating unique cultural or social features specific for women or for special populations defined by race/ethnicity and/or socioeconomic status are encouraged.

2. Eligibility: Applications may be submitted by domestic and foreign, for-profit and non-profit organizaitons, public and private, such as universities, colleges, hospitals, laboratories, units of State and local governments, and eligible agencies of the Federal Government.

3. Deadlines: Letter of Intent Receipt Date: March 26, 1999 Application Receipt Date: April 27, 1999

4. Money: grants will be limited to $125,000 in direct costs in the first year and a maximum of $375,000 in direct costs over a three year period.

5. Start date and duration: The anticipated award date is September 30, 1999. It is intended that the three years of funding provided for these pilot studies will allow 18-24 months of treatment and/or follow-up to be included in the proposed study designs.

Also listed in this grant are three areas that you application should address: Applications should address: the content of the intervention (e.g., relative focus on aspects of diet, physical activity, both, or other factors), the setting of the intervention (e.g., in health care settings, community groups, social groups, home, school), and the method of intervention delivery (e.g., individual, group, computer-aided, internet-based, mass media). Novel or innovative aspects and the rationale for their use should be highlighted.

This is something you should tape to the side of your computer monitor as you write your grant. Address each one of these points in detail and be sure to address them in the order in which they are stated.

There are other things you can and should do beyond reading the grant guidelines. Many granting agencies have a program official who is available to discuss your proposal. Take advantage of this person's expertise.

You should also visit the granting agency's web site. Look for a mission statement and try to show how your grant will help that agency achieve its mission. For example the mission statement for NIOSH (National Institute for Occupational Safety and Health: www.cdc.gov/niosh) is: Delivering on the Nation's promise: Safety and health at work for all people through research and prevention. If you are applying for a NIOSH grant, be sure to adress any aspects of your work that relate to safety, health, work, and prevention.

The web site might also list other information that you can draw on. NIOSH has a page discussing NORA. National Occupational Research Agenda: 21 priorities for the 21st century. Be sure to show which of the 21 priorities your grant will address.

Example

It's difficult to provide a good example for two reasons. First, I do not have the room or the energy to reproduce a 25 page proposal. Second, it is easier for me to tell you how to write a good research grant than write one myself.

I'm hoping to get some examples of successful research grants that I can post online. If you have a research grant proposal that got funded and you are willing to share an electronic version of it for these web pages, contact me at

Summary

Here are some things you should be thinking about as you write your research grant.

  1. Theme: Develop a primary objective.
  2. Blueprint: Outline the plan for your research.
  3. Legibility: Make your proposal easy to read.
  4. Perspective: Put yourself in the shoes of the reviewer.

Further reading

Although it is targeted more towards research publications, the Mark Plonsky web page has a lot of useful and practical guidance. The article by Gotch and Stebbing (2002) is a brief list of useful tips.

  1. TIPS ON...: Securing a research grant. Frances Gotch and Justin Stebbing. BMJ 2002; 325: 103. http://bmj.com/cgi/content/full/325/7366/S103a
  2. PSYCHOLOGY WITH STYLE A Hypertext Writing Guide - Version 3.1. Mark Plonsky. http://www.uwsp.edu/psych/apa4b.htm

This webpage was written by Steve Simon on 2007-10-01, edited by Steve Simon, and was last modified on 2008-07-14. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page. Category: Ask Professor Mean, Category: Grant writing

Stats >> Training >> Stats #41: Practice Exercises

1. Identify a research idea that you have developed in the past or that you are planning to develop in the future. Note the source or sources for this idea. Relate them to the sources listed above. Are there any sources you used that are not listed in the above material?

2. Discuss the research idea from the previous exercise. Does this idea lend itself to one or more research hypotheses? Develop a draft statement of one or more research hypotheses, if appropriate.

3. Read one of the grants passed out to you in class and identify the research hypothesis. Does it include all of the elements suggested above?

4. Review one of the grants passed out in class.

Please fill out an evaluation form. Your input is important. These evaluation forms also ensure that we can offer Continuing Medical Education credits for this class.