Template for Applying the Time Series Designs to Evaluate Practice Effectiveness

1. Describe your client or clients, indicating age, race, marital status, educational status, health status, sources of income and in-kind benefits, their strengths, and their needs.

2. Describe the behavior that you have targeted for change, this behavior should lend itself to repeated measurement. How do you plan to count this behavior? Will you take frequency, duration or intensity measurements?

How long is the baseline phase? How long is the treatment phase? How long will you do follow-up?

How often will you measure the dependent variable during the baseline, treatment, and follow-up phases?

How will you make sure to continue to take reliable and valid measurements of the client target behavior during and after the intervention phase?

Make a graph and a table in which you depict the client target behavior throughout the experiment.

4. What treatment or intervention(s) will you use? Why do you believe this intervention(s) will be effective? Present empirical (scientific studies) and/or logical support for the intervention.

Describe the intervention in detail: How frequently will you meet with the client(s)? How long will these meetings last? What format will these meetings take? Will others besides you and the client be present? What content will be discussed or presented in these sessions? What homework or real life practice will take place between sessions? Over what period of time will the interventions take place?

5. How will you describe the findings of this experiment/intervention to your client(s) and to the class?

Make sure to respect client anonymity.

Your presentation should compare the average measurement (of your client or clients) during the baseline phase with the average measurement during the treatment, and the average post-treatment measurement to determine whether the client(s) has (have) sufficiently changed in the intended direction. (Tests for statistical significance are commonly used to see whether the amount of change is great enough to exceed the rules of chance. Don't hesitate to check with a research instructor and/or consult an introductory statistics textbook such as R.W. Weinbach & R. M. Grinnell's Statistics for Social Workers or D.M. Pilcher's Data Analysis for the Helping Professions.)

If you feel uncertain about statistical tests, you can often notice meaningful changes by graphing the behavior over time. The horizontal axis shows time, i.e. March 1^st, 4^th, 7^th and the vertical axis shows the frequency (or duration or intensity) of the dependent variable on each of those days.

______________________________________________________________________________________

You will recall that a time series design involves repeated measurements of the client's or client's behavior during a baseline phase before providing any interventions and then continuing to measure the same client'(s)' behavior during intervention and after intervention has stopped. The thinking is that one should notice improvement in client behavior as a consequence of the intervention. There are, however, other possible explanations for changes in client behavior. These other explanations are sometimes called extraneous variation or threats to internal validity; internal validity, being the confidence that is the manipulation of the independent variable/treatment/intervention, and not other factors that cause the intended changes. Extraneous variation can derive from events outside the experiment that occur in everyday life during the duration of the experiment; these are known as historical threats to internal validity. It is also possible that the client or clients have matured into positive behavior. As time passes, individuals learn to cope with all sorts of emotions and problems; this threat to internal validity is known as maturation. The strength of the time series design is that by taking frequent measurements of the dependent variable (the target behavior) over time during the baseline phase, one can notice whether the client is improving just due to maturation. However, if the client improves when the intervention begins, you still can't be sure that there wasn't some outside event that wasn't responsible for the change in the client(s)' behavior even though you'd like to believe it was the intervention (your manipulation of the independent variable). Some clinical researchers recommend using a multiple baseline design or a withdrawal design to increase their certainty that it was the manipulation of the independent variable (the treatment/intervention) that was responsible for changes in client behavior and not outside influences (the history threat). To refresh your memory of these designs, please consult a research book in which time series or single subject designs are discussed.

Briefly this paper will take you through a simple time series design that can be used with single subjects or with multiple subjects. The first step is to describe the behavior that you have targeted for change. Perhaps the client needs to reduce some excessive behavior, e.g. smoking, over-eating, hyperactivity, substance abuse or increase a limited behavior, e.g. initiating social activities, self-disclosure, assertiveness. The targeted behavior needs to be defined carefully such that it can be counted and measured reliably and validly. Once the behavior is defined, you can then decide how you will measure it. Will you count the number of times the behavior takes place during a specific time frame, i.e. frequency per hour or day? You can also count the onset and duration of a particular behavior sequence, e.g. the length of time that a temper tantrum lasts. But neither frequency nor duration can be measured in a reliable way unless the behavior has been clearly defined. For example, what is a temper tantrum? If you define a tantrum as a raising of the voice to a level comparable to a shout and a phrase that underscores the speaker's frustration and unwillingness to go along with someone else's agenda, then some political speak could be counted as a tantrum. In general, a duration measurement is more difficult to take and will be less reliable than a frequency measurement. Besides frequency and duration, you can measure the intensity of a behavior, attitude or feeling by applying a scale. You can observe or question someone about their behavior, attitude or feeling. Their answers can be computed into a score or their answers can be analyzed using a scale which has been anchored by descriptions of each point along the scale. It is difficult to observe feelings in a precise way so measurement of feelings is usually done through self-report. However, client behavior can be observed by significant others or trained observers; this method can be very reliably and valid especially if the client doesn't know when s/he is being observed. Again, the behavior can be counted in terms of frequency, duration and/or intensity.

After deciding on the behavior targeted for change and determining the way this behavior will be measured (frequency, duration or intensity), your next decision is the number of times you plan to measure the behavior during the baseline phase, during the treatment phase, and after treatment has been terminated. In general, you should take at least 5 measurements of the behavior targeted for change (dependent variable) during the baseline phase. The number of measurements during the treatment phase depends on the length of treatment. If, for example, treatment is to last 4 weeks, 2-3 measurements each week would result in from 8-12 measurements during the treatment phase. A 4 week follow-up after terminating with the client with 2 measurements per week would result in 8 measurements during the post-treatment phase. Whether you have one or more clients, you can compare the average measurement during the baseline with the average measurement during the treatment, and the average post-treatment measurement to determine whether the client has sufficiently changed in the intended direction

In time series designs, the worker/researcher decides when to introduce the treatment/intervention; this allows him/her to notice changes in the client before and after being exposed to the intervention. As a worker, you should choose interventions that have been shown to be effective in the past and/or that have a logical basis for being useful to the client given the client's problem, strengths, and current situation. The next part of the time series design involves your description of the intervention and your reasons for believing that this intervention or series of interventions should be helpful. For example, if your client wants to increase his ability to initiate conversations with acquaintances, you need to think about the skills your client needs to learn and practice. You also need to consider whether s/he would be better off learning these skills in a group setting or one-on-one. Reading journals in which social skills and shyness are discussed will help you to locate interventions that have been shown to be empirically effective. Using your own preliminary empathy, you can also identify attitudes that might interfere with social skills. You can then develop behavioral and cognitive interventions in which teach new behaviors are demonstrated and practiced and dysfunctional attitudes are recognized and replaced. You will need to decide upon the number of sessions and the length of each session, as well as issues related to content and process. Once the interventions have been planned and agreed upon with your client(s), you implement them as described, and you continue to measure the frequency of the client target behavior. If you notice that one-on-one sessions have had little effect, you can talk with your client about adding or replacing this intervention with a group model. The clinical research model allows for flexibility because this is needed to be most helpful to your client(s).

You continue to measure the client behavior even after intervention has terminated because you want to see if the positive effects have been maintained.