Testing, testing

ALMOST EVERYTHING FROM THE PAST COUPLE OF DAYS HAS BEEN TAKEN FROM THE BOOK Research in Social Work BY: WILLIAM J REID AND AUDREY D SMITH

Those of you who do add a quantitative piece to your research will be picking and choosing between statistical tests.

The simplest test to measure differences between two groups is the t test.

If we randomly assigned people in an agency to a treatment and a control group to test a marital therapy intervention, we would use the t test in several different ways.

First we might decide to see if the two groups differ much. Just because we've assigned randomly doesn't mean that the characteristics of the two groups will be very well matched. So we could compare age, number of years married, and number of children. If the means tests on any of these variables found one of them to be significantly different, we might want to redo the assignment, perhaps use matching.

Then we might do a pretest of all of the subjects of the experiment to see how both groups fared on the dependent variable. Let's make that marital satisfaction.

Then after the intervention was performed we would see if the pre-test/post-test scores for each group made any significant changes. Then we could compare the changes between the groups to see if there was a significant difference between them.

Very elegant, no?

If we're testing nominal data then we frequently use a contingency table and test for significance with the chi-square test. Instead of comparing means, as the t test does, the chi-square compares frequencies. The greater the difference in observed frequencies from an expected chance value, the greater the chi-square value for each cell.




Above is a table and results of a significant chi-square test
If you tested three groups against a control group, used three different types of marital therapy and tried to determine which was the best, then the t test wouldn't work for you. You would need to use an f test, perform Anova (analysis of variance) or regression.

We'll discuss this in a different post, the one below.

The Meaning Behind the Measures of Association

The strength of an association is expressed in numbers between -1 and +1.
And the minus sign makes a difference, tells us that the direction of the data is negative, that there’s a negative association between the variables. More of one variable indicates less of another.

The Pearson r is the statistic we use for interval and ratio data, so you will see it in much of the literature you read.

The r, a measure between -1 and 1, shows the direction the variables are taking when they present together in a sample, and the correlation. The r predicts, too, meaning once you know the r, if you know the value of one variable, you can figure out the value of another.



Any association close to -1 or +1 is a strong association. So a negative correlation, like a -.95 is a strong correlation, stronger than a positive correlation of only .13.

95% of the variance is associated with the two variables, as opposed to thirteen percent. _.95 tells us that when you see one of the variables, you'll probably find the other variable. The direction doesn't matter, they're still strongly associated.

Whereas an r of .13 means that if you find one variable, there's only a thirteen percent probability that you'll find the other.

To find out what portion of variation is explained by another variable, you square the correlation. So a Pearson r of .80, a strong relationship between variables, when squared, indicates to us that .64 or 64 percent of the variation is explained by that relationship.

Something else, some other variables are explaining the rest of the variation.

Now you're into multivariate data analysis. Adding a third variable, we can partial out which variables affected the others by taking them two at a time. The computations are complex, but ultimately give us information about how much each variable contributed to the results of our statistical enterprises.

We ask our computers to spit out what are called correlation matrices. Below is a correlation matrix. The study in question looked for reasons parents had for visiting their foster children. Very low correlations between the reasons on the left and the amount of parent contact indicate that those variables didn't contribute very much to a parent's reasons for visiting.

In this table only three of the reasons on the left showed any contribution at all, Years in Care r = .256
Reason placed (child's behavior) r = .265,
and the parent's discharge objective, (wanting to take the child home), r = .387.

The little note under the table about dichotomous variables tells us that these were nominal variables "dummy" coded with 0's and 1's.

It is the magnitude, not the significance that matters to us in regression, one way to analyze multivariate data. Regression helps us parse out the contributions of independent variables to the dependent variable(s).



The "beta" is called a beta weight, a standard partial regression coefficient, sometimes called a standardized regression coefficient.

The b weights are standardized (this discussion isn't finished)

The one versus the two-tailed test

Obviously, what we're shooting for is to be able to reject a null hypothesis so we can say our research hypothesis is correct, or might be correct, at least is potentially correct.

So the bigger the rejection area (for we're testing to reject the null hypothesis) the better. That's why social workers tend to set their alphas at .05, not .01.

Another way to do this is to use a one-tailed test. In the two-tailed, non-directional tests, the rejection area, that five percent, is divided into .025 and .025. We're testing whether or not the intervention helped, for example, or perhaps if it hurt.

There isn't always a purpose to see it something hurt, so we can use a directional, as opposed to a non-directional test. This way we set the alpha at .05 and the rejection area seems larger because it's all at one side of the curve, 1.65 SD from the mean, as opposed to 1.96 SD in a 2-tailed test.

That increased the power, or the likelihood the test might reject the null, by increasing the size of the rejection region.

Type I and Type II Errors

This goes along with hypothesis testing.

If you've rejected the null, found reason to believe that your research hypothesis might have some truth to it, but in reality you've made a mistake, then you made a Type I error.

If our test statistic falls into the rejection region (p < or = .05) then we are saying that 19 times out of 20 we're correct in our decision to reject the null. But we're going to wrong 1 time in 20, 5 percent of the time.

To make sure we don't do that, we can decrease the rejection region, make it .01, only 1 chance in a hundred that we reject the null by mistake.

We call the probability of making a Type I error alpha. The probability of making a Type II error is beta.

A Type II error occurs when we accept a false null hypothesis.

The ability of a statistical test to reject the null hypothesis when it is false and the research hypothesis is true is called the POWER of the test. The best way to increase this power is to increase the size of the sample.

Hypothesis Testing

Why does this work, testing variables to see if they come from separate populations?

We get probabilities that enable us to say that our findings are significant because we believe that all things fall out, for the most part, normally. If I take an infinite number of samples from a population and test for any one thing, then make a frequency distribution of all of the findings, that distribution will look like a normal curve.

This is called a sampling distribution, and it is usually of the parameters, any parameter (mean, median, mode, SD, variance).

Test results of infinite samples will show that half of the samples will fall out on one side of the average, that center line that bisects the top of the normal curve and the horizontal line, (see why you once had to learn geometry, so you could know the word bisect) and the other half will be on the other side of that center line.

So every time you test a sample, you can compare your results by comparing them to what would be the theoretical average, the norm, the normal curve. You're comparing two means, or two averages, to see if the differences are so great that they probably don't belong in the same sampling distribution.

You either took a sample and divided it up into two groups to intervene in one and not the other and compare, or you are merely testing one group to see if variables within that group are associated with one another).

Your null hypothesis is that the difference between these means is minimal, they are so alike that they must come from the same population. What you did, or what you found, is no big deal, 95-99% of the population is this way.

If you find huge differences, however, that perhaps only 5% of the sampling distribution would have the trait you are testing (attitude, eye color, anger) associated with another, then this five percent is from another distribution, one in which 95-99% of the people in it are just like this.

You've established a new population, basically. What you did mattered, it is significant that two things are associated. Either what you did made the difference, or these things really do hang together in some normal distribution. Either way, the likelihood is greater, that under certain conditions, your findings are not due to chance.

On the other hand, assuming there is a normal distribution for what you found, in every population there is still a 1-5% chance that chance is the reason for your findings.

Now how confusing is that? Very confusing. Do your best to let it sink in.

Thus we test the null, that what is likely to be found in the universe, based upon infinite samples that theoretically fall into normal distributions, to see if we can reject it, the idea that the averages, the means of the variables we're testing are really the same in two groups.

But if they are different, then we can reject that hypothesis, the null hypothesis, which supports our research hypothesis.

Our research hypothesis is that the difference, actually, between the groups will be great because what we did, our intervention, made a big difference. There is only a 5% (or less) likelihood that there would be such a big difference in the parameter we've tested (usually a mean) were it not for the intervention. You've ruled out chance with a significance test.

The significance test compared your results with what should happen in the universe if there were no differences between the two groups, if they came from the same sampling distribution. You've tested the independence of the means.

The normal sampling distribution has one statistic, an average statistic. Your sample is compared to that. If your results vary significantly, then there must be something you did that increased the likelihood that this would happen.

You knew that, which is why you theorized there would be some association between your intervention, your independent variable, and the dependent variable. You had a theory about why this would happen when you compared groups or variables in the first place.

Univariate, Bivariate, and Multivariate Data

In other words, data analysis all depends upon how many variables you're analyzing

Univariate analysis is about analysis of one variable. Usually we'll find descriptive statistics (measures of central tendency), or measures of variability, i.e. the range, SD and variance. We'll often make frequency distributions to see how often a certain variable popped up in the data.

Bivariate and multivariate analysis attempt to do more than describe. They explain relationships between one or more variables: which variables are related, and how.

Bivariate implies an analysis of only two.

Multivariate analysis helps us to specify the conditions under which relationships hold. We simultaneously analyze relationships between variables.

Often we use cross-tabulations to tabulate the joint occurrence of 2 or more variables. The result is called a contingency table.

Employed Not employed Total
Men 75 25 100
Women 25 75 100

Total 100 100 200


Here you can see that with univariate analysis, you would have found that there are 100 men, or 100 women, or 100 are employed, and 100 are not employed.

A bivariate analysis allows us to say that three-fourths of the men are employed, and only one-quarter of the women are employed.

A contingency table like the one above allows you to “see” data. Another way, some might consider a more refined way to look at of looking at is to create a scatterplot. You should be able to identify one of these.

Read the post below to refresh your memories

Scatter Plots

Courtesy of UIUC, a direct copy of scatterplots">this link

Scatter plots are similar to line graphs in that they use horizontal and vertical axes to plot data points. However, they have a very specific purpose. Scatter plots show how much one variable is affected by another. The relationship between two variables is called their correlation .

Scatter plots usually consist of a large body of data. The closer the data points come when plotted to making a straight line, the higher the correlation between the two variables, or the stronger the relationship.
If the data points make a straight line going from the origin out to high x- and y-values, then the variables are said to have a positive correlation . If the line goes from a high-value on the y-axis down to a high-value on the x-axis, the variables have a negative correlation.




A perfect positive correlation is given the value of 1. A perfect negative correlation is given the value of -1. If there is absolutely no correlation present the value given is 0. The closer the number is to 1 or -1, the stronger the correlation, or the stronger the relationship between the variables. The closer the number is to 0, the weaker the correlation. So something that seems to kind of correlate in a positive direction might have a value of 0.67, whereas something with an extremely weak negative correlation might have the value -.21.

An example of a situation where you might find a perfect positive correlation, as we have in the graph on the left above, would be when you compare the total amount of money spent on tickets at the movie theater with the number of people who go. This means that every time that "x" number of people go, "y" amount of money is spent on tickets without variation.

An example of a situation where you might find a perfect negative correlation, as in the graph on the right above, would be if you were comparing the speed at which a car is going to the amount of time it takes to reach a destination. As the speed increases, the amount of time decreases.

On the other hand, a situation where you might find a strong but not perfect positive correlation would be if you examined the number of hours students spent studying for an exam versus the grade received. This won't be a perfect correlation because two people could spend the same amount of time studying and get different grades. But in general the rule will hold true that as the amount of time studying increases so does the grade received.

Go to UIUC (link at top) to see some examples. Most scatter plots will not show perfect correlations.

Measures of Association

Aside from describing variables and relationships, and noting central tendencies, or the degree of variation in samples, we also use numbers to describe the magnitude, or strength of relationships.

We use different statistics to express how associated variables might be, depending upon the type of data we’ve used in our sample, nominal, ordinal, interval, or ratio.

Measures of association for nominal data include lambda, tau, the phi coefficient, the contingency coefficient, Yule’s Q, and Cramer’s V.

For ordinal data we use gamma, Kendall’s coefficient of concordance (W), Somer’s D, and Spearman’s rank-order correlation coefficient.

Linear relationships between 2 variables on interval or ratio scales use Pearson product moment correlation coefficient ® or the coefficient of determination (r squared).
If data is not linear, we use eta.

Inferential Statistics

Now let’s continue with the logic of inferential statistics, what we started in class on June 7. It is always possible that chance is responsible for our findings. You might remember that if we take random samples, then they are going to differ because of chance. But if we’re taking them from the same population, the differences will be small.

So if I take random samples off of a list of social workers who are members of NASW, then compare them for certain attitudes, it is likely the attitudes will be different, but if we’re taking large samples, the differences will be small. These people are really from the same population.

But if my results are really different, if the attitudes vary greatly, then we might wonder if the samples came from different populations. The differences in the samples might be attributed to training, for example.

In the same way, if we find ANY large differences in samples that we’re comparing, the differences are less likely to be by chance. They are therefore significant. If we intervened in a sample in some way, perhaps by education or some other intervention, and the outcomes differ largely, we have reason to believe that what we did caused the change in the sample that got the intervention. Now we say that THAT sample no represents a different population, the population of people who have had the intervention.


That’s what most tests of significance are seeking to find. Is the sample from an entirely different population? And why. We hope it’s because of something we did.
The question tends to be, how big a difference does it take to say that the outcome isn’t by chance, rather is because of what we did? Or what we suspected?

If you’re testing a group to see if they went to work within the first six months of their child’s life, you might also check out what kinds of jobs they’re hoping to get, or how much education they have. A hypothesis might be that women who go back to work in the first six months have prospects of better jobs because they had more education.

If this is true, if your sample shows that there is this strong likelihood, then we say that these women come from a different population from the ones who did not go back to work, a better educated population.

If there’s only a small difference in the two groups, the one that went back to work and the one that didn’t, then we say that small difference was due to chance. The groups are too alike.

The test will tell us that there is only a .05 or less chance that the relationship we found is due to chance. Then we’ll say, must be a significant relationship, here.

The null hypothesis and the research hypothesis

We don’t know, unfortunately, the population parameters of variables we wish to study. If we did, there would be no need to test samples. What we’re doing, when we measure samples, is trying to infer to the population from our sample’s results.
Tests of significance are simply called statistical tests.

Commonly used test statistics include the t, the F, and the chi-square. The distributions of these tests aren’t exactly normal, but there are tables, neverltheless, for determining the critical areas under their curves and the probablilities that are associated with them.

There are four components to a statistical test: a null hypothesis, a research hypothesis, a test statistic (decision maker) and a rejection region.
The null hypothesis is a hypothesis about the population (from which we’ve drawn our sample) that asserts that sampling error explains the results. Our results are from sampling error, not the intervention.

We have two groups, one with an intervention, and one without. We test both of the groups and then compare the results. If the differences are small then it is possible that the two groups are now representing different populations. Our intervention caused the difference, and it’s such a large, significant difference, that we have to reject the null hypothesis that the groups are from the same population. We’re testing to see if they’re independent of one another.

These tests are often called tests of independence.

You don’t need two groups. You can be testing one sample to see if variables are associated. In this case, we’re asking, are these two variables from the same population? If they are not associated, if there’s an insignificant association, a low r, for example, then we say that the variables are not independent, they are from the same sample. We don’t reject the null.

The null hypothesis is saying that the two variables are from the same population.
We do that test and that test is how we determine if we can accept the alternative hypothesis, that which we call the research hypothesis.

The null is that two variables are from the same population. If we accept this we’re saying the difference in the two v’s is due to chance.

The research hypothesis is that the two variables are from different populations.
When we reject a null we automatically accept the research hypothesis. The difference is not due to chance. Our theory about why the two variables are associated significantly is supported by the research.

The null hypothesis is that the mean of the first group is the same as the mean of the second group. So if the means differ significantly, we reject the mean and accept the research hypothesis that says that the two samples are from different populations.

What this distills down to is you doing a test, getting a result, and checking a table to see if that result is significant or not. Sometimes the result is the difference between two averages, you might be comparing the means of two different groups, using a t test for two variables, an F for three or more. Sometimes the result indicates how much two variables are associated with one another, how high or low the correlation, or r.

The table will say that the test statistic is < than or = to a particular number, and you will interpret the statistic based upon what you set your level of significance to be. So if you say that you want the odds of your results to be over 95% that you will have results that are significant, then the rejection statistic will be < or = to .05. that < or = finding is the likelihood that your results were due to chance.

So if the finding on the table is .11, then you would not reject the null. .11 is not in the rejection region of your curve.

But if it is .03, then you do reject the null. There is little likelihood that chance is responsible for your findings, only 3 times out of 100 will you get your result. It’s probably due to your intervention, or if you’re testing to see if there’s a relationship between variables, there’s reason to believe that they are associated, that it is not by chance, it is a real phenomenon in the world.

The Normal Curve

You've seen that diagram before, when we were studying standard deviations, although mine was something very crude on a blackboard.

We tend to think that most things fall out normally, that if we could count up a huge population, that all of the variables associated with that population would have a mean, median and mode at the very center. That vertical center line that passes through the top of the curve to the horizontal line at the bottom represents all three measures of central tendency. The other vertical lines represent one, two, and three deviations, standard deviations from the mean.

We can find probabilities using this curve. We assume that the sum of the area under the curve equals one. So any other area (other than all of the space under the line) will represent some proportion less than one.

The area between an SD of 1 and an SD of -1 represents .6826 of the area under the curve, or almost 68%. The area between an SD of 1.96 and an SD of -1.96 represents .9544, or 95% or the area. The area between an SD of 3 and an SD of -3 represents .9974 of the probability.



It is the very same type of curve, the normal curve, that helps you to see the rejection areas, the critical regions that we're looking for to be able to reject a null hypothesis.

Figure 10.6 (the top one)

The Research Proposal

THE RESEARCH PROCESS

L. Freedman, PhD

Spring 2008 ICSW, Chicago,

IL Distance Learning Program

THE RESEARCH PROPOSAL

I. Statement of the problem

a. Background, rationale, significance of the study

b. Literature review on research about the problem

c. Theory about the problem and a solution

d. Development of a research question,hypotheses, and variable

II. Method

a. Describe the design of the study and methods of inquiry

b. Describe the data collection procedures
1. Define instruments for measurement and how that is done
2. Define the sampling plan
3. Describe the data collection method

c. Describe how will the data be analyzed
1. Statistical analysis (which statistics will you use, why)
2. How will the results look when the findings are in (i.e., frequency tables, crosstabs)

d. Ethical considerations and protections for human subjects

III. Implications of the study, value to social science

a. Implications for theory
1. Will it change?
2. Does the study validate the theory?

b. Implications for practice
1. How might social workers incorporate finding in practice?

c. Implications for education
1. Might your study affect the way social work is taught?

d. Implications for future research

e. Implications about other studies

IV. Limitations of the study

V. Dissemination Plan

a. Who will be interested in your findings?

b. How will you disseminate your findings?

The Video Idea

I know I said I would leave it up, but there are other youtube videos that somehow embedded together with mine. You might have noticed.

There were dozens of them.

Since I don't have time to check them for content, I took down the posts.

Sorry about that. You're now stuck with whatever notes you have and your book.

Fourth Class Feb 22, at ICSW

Read Chaps 7 and 8.

No Quiz

For the Third Class

Homework is to read about culturally competent research (Chap 5) and work really hard at Chapter 6, Problem Formulation.

I'm going to quiz you on the NASW ethics as they apply to researchers and the Institutional Review Board (IRB) process, what we want from you and how you should go about convincing us that your research is ethical.

I might give you a few hints in the next couple of weeks on the blog. Come on over once in awhile to see if there's anything new.

Ethics

There are ethics to everything we do, in everything we see. As social workers we have an obligation to uphold the ethics of our profession.

When you studied for your Masters degree, you learned the social work code of ethics and applied it to practice. Now you'll be using similar principles for research.

Let's ask this question first. If social workers had produced the original Mash television series, would NASW have permitted the use of this song?



Watch these videos about research gone awry.

First we see Vanessa Northington Gamble, M.D., Ph.D., Associate Professor at Johns Hopkins University, and is an internationally recognized expert on the history of race and racism in American medicine,cultural competence, and diversity. She discusses the enduring causes and consequences of the Tuskegee Syphilis Study Series: LeNoir - NMA Pediatric Lecture Series [Health and Medicine] [Professional Medical Education] [Show ID: 7025] (less)




And this one is on the Milgram Studies. Can you see how people push that button? Why do you think people rely so much on authority? If you were to design a study about authority, how could you contribute to the literature and social science without using deception?



Our code of ethics is profound. It is required reading for this course. Not only does it illustrate the heart of ethical practice and research, but it might help you to see where you want to go with your doctoral research proposal. Keep in mind that your research should be important to our field, and that the primary mission of our field is to enhance human well-being.

Paradigms, theory, deduction and induction

The lecture is in 7 parts, short audio clips.
Each one has a title, like "DW_B0069.wav".
The seven clips are DW_B0068, 69, 70, 71, 72, 73, up to DW_B0074. All files

Click HERE to begin Part 1 68

Click Here to begin Part 2 69

Click HERE to begin Part 3 70

Click Here to begin Part 4 71

Click Here to begin Part 5 72

Click Here to begin Part 6 73

Click Here to begin Part 7 74




You're going to go to the web site that hosts these mini lectures in a minute. They're embedded in boxes that look like record albums or CD's. Each box holds one snip of my lecture and has a name like DW_B0068. You'll then click on them one at a time to hear me teach.

ESnips put the clips up on two different pages, but backwards, last to first, not first to last. So the first and second clips are on the second page. Wouldn't you just know it.

When you click here, you'll scroll down on page two to two boxes with the first two files.,

Do that now, and click on DW_B0068.wav, the one farthest down the page. After you finish that, click on DW_B0069.wav.

When you have finished listening to those two, you'll click on "Previous" or the little square with the "1" to get to the other snips on page 1.
It looks like this: Page: « Previous 1 2 You click "1" or "Previous".

Again, scroll down to almost the bottom, begin with DW_B0070.wav, and make yourself up to the top. You'll listen to DW_B0071.wav, then DW_B0072.wav, then DW_B0073.wav,end with DW_0074, in that order.

We'll give it a try, okay?