Communication Research Methods: Quantitative (C & J 607) Fall 2008
Office: CJ 129e, 204 Office Hours: T R 1-2
Course Description/Objectives:
This course is an advanced survey of quantitative research methods and data analysis. In the course, we discuss five different research methods that have specific relevance to communication study: content analysis, interaction analysis, network analysis, questionnaire construction, and experimental design. We place special emphasis on the validity and reliability of measurement in each of these methods, as well as sampling. We also discuss 4 advanced procedures for analyzing statistical data: multivariate analysis of (co)variance, confirmatory factor analysis, model testing (structural equation models and possibly hierarchical regression analysis), and a topic to be chosen by the class (e.g., logistic regression, hierarchical data analysis).
There are three factors of which you should be aware for this course. First, I assume that you have a working knowledge of statistics equivalent to having C & J 507 or another intro to statistics course (analysis of variance, correlation, t-test, and regression). Second, I assume that you have a working knowledge of research methods equivalent to having C & J 501 or another introduction to research methods course (conceptualization, measurement, reliability, validity, and basic research design). Third, the course is a survey course so it will not be the definitive coverage of any one topic. You will receive a taste of each topic and enough to carry out a preliminary research project. However, to become proficient at the method/data analysis you will need to do some further reading on the topic. I will provide some leads for further reading to get you started.
Required Texts:
Reading Packet available from copy center in the Department of Education.
Mertler, C. A., & Vannatta, R. A. (2001). Advanced and multivariate statistical methods:
Practical application and interpretation. Los Angeles: Pyrczak.
Course Procedures/Policies:
1) I will accept late papers for one week after a deadline with a 10% deduction.
2) Please do not procrastinate in this class. You need to keep up with what we are doing so that you donÕt fall behind.
3) Qualified students with disabilities needing appropriate academic adjustments should contact me as soon as possible to ensure your needs are met in a timely manner. Handouts are available in alternative accessible formats upon request.
4) This course encourages different perspectives related to such factors as gender, race, nationality, ethnicity, sexual orientation, religion, and other relevant cultural identities. This course seeks to foster understanding and inclusiveness related to such diverse perspectives and ways of communicating.
5) The course emphasizes ethical practices and perspectives. Above all, students and instructors should strive to communicate and act, both in class interactions and in assigned coursework, in a manner directed by personal integrity, honesty, and respect for self and others. Included in this focus is the need for academic honesty by students as stated by the UNM Pathfinder. Students need to do original work and properly cite sources. For example, be aware of plagiarismÑdirectly copying more than 3 or 4 words from another author without quoting (not just citing) the author is plagiarism. Further, course content will encourage the ethical practices and analysis of communication research.
6) You will receive an F on a plagiarized paper and reported to appropriate parties.
Course Assignments:
Assignment % of final grade
Mini Assignments 45%
2 of 3 Methods Assignments (Content Analysis, Interaction Analysis, Network Analysis)
3 of 4 Statistics Assignments (Multivariate Analysis of Covariance, Model Testing, Confirmatory Factor Analysis, Data Cleaning/Sample Size/Power Analysis)
Large Assignment (Research Project) 50%
Participation 5%
Note 1: If you want to do all three methods assignments or all four stats assignments (or both), IÕll take the best 2 (or 3) grades on the assignments.
Note 2: I am somewhat flexible with my assignments (the exception is the research project, which you have to do). If you want to suggest an alternative assignment (or set of assignments) that better fits your learning style, let me know. WeÕll talk and negotiate an alternative.
Grading Scale:
93-100% A 87-89% B+ 70-79% C
90-92% A- 83-86% B 60-69% D
80-82% B- below 60% F
Grading Criteria:
1) OriginalityÑThis point focuses on the degree to which you are moving beyond basic classroom discussion and prior research. The research project should make an original contribution to the literature.
2) OrganizationÑThe essay should flow well. I can see the progression of ideas and understanding why they are laid out as they are. There is an introduction, body, and conclusion for small assignments (at least) and an intro, lit review, methods, results, and discussion for the article-length assignment.
3) Grammar, spelling
4) Accuracy/appropriateness of data analysisÑquality analysis and conclusions.
5) Depth of critical thinkingÑquality of arguments (i.e., using evidence to support opinions).
6) Following directions--a necessary but not sufficient criterion for a good paper. I will provide instructions for the assignment. However, they are not a laundry list of what you need to do get an A. Doing excellent work is not simply about following directions; itÕs about challenging yourself, thinking in original ways, and writing well.
Description of Assignments:
Methods: Choose Two of the Following Three
#1 Content Analysis
For this assignment, I want you to analyze the content found in some sort of mass media (e.g., TV shows, newspapers, magazines, etc.). Your task is to select the content and a research question associated with the content. Then, youÕll need to carry out the study. The purpose is to give you some practice utilizing content analysis. Specific steps include:
1) Select the media (e.g., compare two TV shows or the coverage of a topic in two different newspapers)
2) Develop a research question to frame the study.
3) Develop a suitable set of categories to analyze the media (i.e., the content scheme). Determine the units of analysis.
4) Categorize and tabulate the data. Determine the intercoder reliability and statistically analyze the data to determine the answer to your research question.
5) Interpret and write a short (3-4 pp.) report. The report should include an introduction and research question, method, findings, and brief conclusion (noting any shortcomings or limitations of your study).
#2 Interaction Analysis
This assignment will be similar to that of content analysis except that now you will be examining the structure and content of patterns of communicative interaction. You can choose to analyze any sort of communicative dialogue (e.g., a play or I have some videotapes that may work for you). Best if there is a transcript available, but it is not an outright necessity. Here are the steps:
1) Select the interaction
2) Develop a research question to frame the study.
3) Develop a suitable set of categories to analyze the interaction. It should include both content and structure at some level (such as who initiates and responds). You may use an existing coding scheme (such as one-up, one-down, one-across). The scheme should fit the research questions. Determine the unit of analysis.
4) Categorize and tabulate the data. Determine the intercoder reliability and statistically analyze the data to determine the answer to your research question (if you canÑif you have a sequencing question, you probably wonÕt be able to do this. WeÕll discuss in class further).
5) Interpret and write a short (3-4 pp.) report. The report should include an introduction and research question, method, findings, and brief conclusion.
#3 Network Analysis
For this assignment, I want you to conduct a network analysis of the C & J dept or another organization. Select only 8-10 people. Your task will be to create a sociogram of the C & J network (or your org.) and identify key roles in the network. Here are the specific steps:
1) Select an aspect of the communication (e.g., task-oriented/relationally oriented) of C & J you want to examine and choose a research question or two.
2) Choose a method for determining the network. For example, you can ask every person to complete a questionnaire about how many times they speak to everyone else in the department per week.
3) Determine what constitutes a link and justify your way of defining a link.
4) Tabulate data and create the sociogram. Statistically analyze the data if appropriate.
5) Interpret and write a short (3-4 pp.) report. The report should include an introduction and research question, methods, findings, and brief conclusion.
Statistics: Choose Three of the Following Four Assignments
IÕll make available a data set on SPSS to complete these assignments. The data set includes participants from the China, Japan, Germany, and the US about a recall interpersonal conflict. IÕve provided you with a copy of the questionnaire so that you can see how particular variables were measured. When you do all of these assignments, make sure to start by selecting cases and including the condition ÒIf country <=4.Ó
#4 Data Cleaning and Sample Size Determination
This assignment will focus on several key aspects of data analysis and research design. LetÕs start with the design. I want you to calculate the sample size I should have used with this data set given alpha = .05, power = .80, and small, medium, and large effect sizes. Tell me if I had an adequate sample size. Second, I want you to do some pre-analysis data screening. Pick 3 variables. Check normality, linearity, and homoscedasticity. If there are problems, discuss what you should do about them (if anything). Do not actually clean up the data
#5 Measurement and Confirmatory Factor Analysis
The first 20 items of the questionnaires measure self-construals (independent and interdependent), while the next 34 items measure face concerns (self, other, and mutual). I want you to analyze the measurement of one of these sets of scales (either self-construal or face concerns). (If you are feeling up to it, you can do all 5 subscales at the same time and try to confirm as many of the factors as you possibly can; Hint: I could only do 4 of the 5). The first step will be to standardize the items by culture (IÕve done this for you to save time). Second, confirm the existing scales and eliminate poor items (remember that you will need to replace items with missing values before doing the confirmatory analysis). Complete only two iterations of this analysis in order to confirm the scales. Write up your results in a short 1-page report (or so) with appropriate tables (i.e., factor loadings).
#6 Model Testing
I want you to create a model of the factors that influence face behaviors. Some of the face behaviors in the data set include:
rlaggcÑAggression
rlpsusÑProblem-solving
rldefc--Defending
rlrespus--Respect
rlpretusÑPretend
I can provide more detail about these variables. I want you to develop a model for at least 2 of these behaviors. WeÕll talk about the potential models, but I donÕt want to direct you too much. Then, I want you to these the fit of your model with path analysis. Once you have a model, you can tests its fit. Modify the model using modification indexes. Complete only 2-3 iterations of the model testing. Write up the results in a short 1-2 pp. report: Include the hypothesized model, the findings with the path model, and a table of correlations. Make sure to discuss why the modifications are appropriate (or why there were not).
#7 Multivariate Analysis of Covariance
I have already done some analysis on the data and have created the following variables (there are others, but this is sufficient for our task here):
relindusÑIndependent self-construal (self as independent)
reldepcÑInterdependent self-construal (self as connected to others)
relpowusÑPower distance
relothcÑOther-face concern
rlselfusÑSelf-face concern
rlmutusÑMutual-face concern
Other variables of interest are
StatusÑParticipants were asked to recall someone of equal status (1) or higher status (2)
IntimacyÑParticipants were asked to recall someone close (1) or distant (2) from them
CountryÑUS (1), Japan (2), Germany (3), China (4)
I want you to investigate the affect of national culture on face concerns. There are several potential intervening variables including self-construals and power distance. Utilize multivariate analysis of covariance to investigate this topic. Write up your findings in a short 1-2-page report. List your hypotheses and statistics to test the hypotheses. Include a table of means and regression coefficients if appropriate.
Participation (5%)
Participation will focus on your attendance and, more importantly, level of engagement during class periods. You can have one absence as a freebie, everyone after that hurts your participation grade (10% reduction per absence unless there is a legitimate excuse for missing more than one classÑe.g., family emergency, attending professional conferences). Level of engagement will be judged on several criteria: a) preparation for class (did you do the reading), b) participation in discussions, c) attentiveness to the discussion/demonstrations, and d) asking/answering questions during demonstrations. The class will be a mixture of seminar discussions and Òdemonstrations.Ó
Major Project (50%)
The major project for this course is a research project. You may focus on any topic in communication you like, so long as you quantify the variables under study. Obviously, there are topics that are not appropriate for quantitative study. You are required to plan, research, collect data, analyze data, and write up the findings. This could be a pilot study for your thesis/dissertation or simply a project in which you are interested. The research must be original and make a contribution to the field (or at least moving in that direction in the case of the pilot study). Additionally, it must use at least one method/statistical procedure we discuss in the class. I will encourage you to send your papers to a conference so that you get experience presenting research, as well as submit them for publication.
It is very important that you choose a topic that is manageable and can be completed in one semester. You will need to get data from a minimum of 60 participants or units (30 per each experimental group if there were two groups) and I encourage you to get 100-200 (or more) if it is manageable (if not during this semester, in the immediate future--larger sample sizes are attractive to reviewers because of greater powerÑof course, you can do a power analysis to identify the precise sample you need). I will not give incompletes for this project except in extreme circumstances. It is critical that you get started now. To assist you in completing this project on time, I have set up the following deadlines for turning in parts of the overall paper:
a) Aug. 29--Propose a general idea/topic (1 p.)
b) Sept. 26--Proposal (Literature review/methods) (12-15 pp.)
c) Dec. 5ÑPresentation of the findings and final paper due (20-25 pp.);
I will be available to help you on this project and I will provide feedback on each of your rough drafts (a and b above). You are welcome to turn in assignments early. My concern on the rough drafts is that the project is manageable, original, and well conceived. I will not grade the rough drafts per se other than noting if it was turned in on time.
*Please note that will also need to get IRB Human Subjects Board Approval for your project if relevant (i.e., you are collecting data from humans; most content analyses do not require approval, but check with me or the web site if you are unsure). I suggest submitting this early in the semester. The forms and directions are on the UNM web site (type IRB in the search line on main page).
Course Outline
Note: Readings and assignments are due on the day they are listed. I list a number of suggested readings. I do not expect you to read these for this class. I list them as references in case you want an in-depth coverage of a particular topic.
Week Topic/Readings/Assignments
1 (8/26-28) Introduction to the Course. Discussion of fears, hopes, and dreams about quantitative research methods. Also, weÕll talk about your interests and how to set up the major assignment. Difficulties in conducting intercultural or cross-cultural research. WeÕll discuss when to use which data analysis procedure. Further, given the focus of our program on doing intercultural communication research, weÕll discuss some of the problems with reliability and validity of quantitative research in this context.
Readings: Gudykunst in the packet; Johnson & Tuttle in the packet
Suggested Reading: A basic text of statistics for review
1-page research topic is due (8/28)
2 (9/2-4) Content Analysis. WeÕll discuss the basics of what is content analysis and how to do it.
Reading: Krippendorff in the packet
Suggested Readings: Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Newbury Park, CA: Sage. Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage
3 (9/9-11) No ClassÑJohn out of town. Take this opportunity to review basic stats (Ch. 1 and 2 in the book or in a basic stats book. WeÕll take some class time to review relevant stats later in the semester as we discuss advanced stats).
4 (9/16-18) Intercoder reliability. WeÕll discuss how to calculate intercoder reliability. WeÕll also begin to discuss interaction analysis. We talk about the basics this week.
Readings: Hirokawa in the packet; Lukasko Emmert in the packet
5 (9/23-25) Interaction analysis. WeÕll look at one of the more famous interaction analysis coding schemes as well as how to analyze sequential data.
Readings: Rogers & Farace in the packet; Bakeman & Gottman in the packet
Suggested Reading: Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis (2nd ed.). Cambridge, UK: Cambridge University Press.
Assignment #1 is due (9/25)
6 (9/30-10/2) Network analysis. WeÕll look at this procedure describing the relationships among a group of people/organizations.
Readings: Rogers & Agarwala-Rogers in the packet; Knoke & Kuklinski in the packet
Suggested Readings: Rogers, E. M., & Kincaid, D. L. (1981). Communication networks: Toward a new paradigm for research. New York: Free Press. Degenne, A., & Forse, M. (1999). Introducing social networks. London: Sage. Knoke, D., & Kuklinski, J. H. (1982). Network analysis. Beverly Hills, CA: Sage.
Proposal is due (9/30)
7 (10/7-10/9) Pre-analysis screening and sample size determination. WeÕll discuss techniques for checking the assumptions for statistical analysis and then identify procedures for determining how large your sample should be.
Readings: Ch. 3 in multivariate book and Cohen in the packet
Assignment #2 is due (10/7)
Assignment #3 is due (10/9)
8 (10/14) Constructing Questionnaires and Scales. WeÕll go through the steps to construct an effective questionnaire.
Readings: Larkey in the packet; Oetzel in the packet
Suggested Reading: Fink, A., & Kosecoff, J. (1998). How to conduct surveys: A step-by-step guide. Thousand Oaks, CA: Sage. Peterson, R. A. (1999). Constructing effective questionnaires. Thousand Oaks, CA: Sage.
Assignment #4 is due (10/14)
NO CLASS ON TH 10/16 (FALL BREAK)
9 (10/21-23) Cross-cultural comparisons using surveys. WeÕll discuss the steps for creating equivalence of concepts across cultures including using anchoring vignettes.
Readings: King et al; King & Wand in the packet.
10 (10/28-30) Confirmatory and Exploratory Factor Analysis. WeÕll discuss measurement issues and how data analysis can help improve measurement.
Readings: Ch. 9 in multivariate book AMOS printouts
Suggested Reading: Kline, R.B. (2004). Principles and practice of structural equation modeling (2nd ed). New York: Guilford Press.
11 (11/4-6) Model testing. WeÕll discuss how to test models with regression analysis. Then, we introduce structural equation modeling.
Readings: Maruyama in the packet, Ch. 8 multivariate, AMOS printouts
Suggested Readings: Maruyama, G. M. (1998). Basics of structural equation modeling. Thousand Oaks, CA: Sage.
Assignment #5 is due (11/4)
12 (11/11-13) Experimental Design. WeÕll examine issues of doing field research, as well as looking at complex designs involving intervening variables. WeÕll include a discussion of message effects research.
Readings: Rogers in the packet, Jackson in the packet, Frey et al. in the packet
Suggested Readings: Brown, S. R., & Melamed, L.E. (1990). Experimental design and analysis. Newbury Park, CA: Sage. Jackson, S. (1992). Message effects research: Principles of design and analysis. New York: Guilford.
Assignment #6 is due (11/11)
13 (11/18-20) Multivariate analysis of (co)variance. WeÕll examine how to statistically control for intervening variables as well as analyze multiple dependent variables.
Readings: Ch 6-7 in multivariate book
14 (11/25) TuesdayÑOpen day for catching up
Thursday (11/27) ThanksgivingÑNO CLASS
15 (12/2-4) Open day for topic of class choice (e.g., hierarchical linear modeling, logistic regression)
Readings: to be determined
Assignment #7 is due (12/2)
16 (12/9-11) Presentations of research project. (12/11)
Research Project is due (12/11)
17 (12/16) Final Exam Week. If we need Week 16 to catch up or explore other topics, weÕll move the presentation to the final exam period. (Tuesday, Dec. 16 from 10-12).
Week 1
Gudykunst, W. B. (2000). Methodological issues in conducting theory-based cross-
cultural research. In H. Spencer-Oatley (Ed.), Culturally speaking: Managing relations in talk across cultures. London: Cassell.
Johnson, J. D., & Tuttle, F. (1989). Problems in intercultural research. In M.
Asante & W. Gudykunst (Eds.), Handbook of international and intercultural communication. Newbury Park, CA: Sage.
Week 2
Krippendorff, K. (1980). Content analysis: An introduction to its methodology.
Newbury Park, CA: Sage. Ch. 2 & 4
Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage.
Ch. 4
Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability
in content analysis. Journal of Applied Communication Research, 27, 258-284.
Instructions on running intraclass correlation on SPSS
Week 4
Hirokawa, R. Y. (1987). Group communication research: Considerations for the
use of interaction analysis. In C. Tardy (Ed.), A handbook for the study of human communication. Ablex.
Lukasko Emmert, V. J. (199?). Interaction analysis. In P. Emmert & L. Barker,
(Eds.), Measurement of communication behavior. New York: Longman.
Week 5
Rogers, L. E., & Farace, R. V. (1975). Analysis of relational communication in
dyads: New measurement procedures. Human Communication Research, 1, 222-239.
Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to
sequential analysis (2nd ed.). Cambridge, UK: Cambridge University Press.
Chapter 6
Interaction analysis handoutÑÒeasyÓ how to
Week 6
Knoke, D., & Kuklinski, J. H. (1982). Network analysis. Beverly Hills, CA: Sage.
Ch. 3
Rogers, E.M., & Agarwala-Rogers, R. (1976). Communication in organizations.
New York: Free Press. Ch. 5.
Network analysis handouts
Week 7
Cohen, J. (1977). Statistical power analysis for the behavioral sciences. Orlando,
FL: Academic Press. Ch. 1
Power analysis handouts
Week 8
Larkey, L. K. (1996). The development and validation of the workforce diversity
questionnaire: An instrument to assess interactions in diverse workgroups. Management Communication Quarterly, 9, 296-337.
Developing a cultural sensitive measure and structural equation modeling--handouts
Week 9
King, G., Murray, C.J.L., Salomon, J.A. & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98, 191-207.
King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15, 46-66.
Week 10
AMOS Printouts of Confirmatory Factor Analysis.
Week 11
Maruyama G. M. (1998). Basics of structural equation modeling. Thousand Oaks,
CA: Sage.
AMOS printouts of Structural Equation Modeling.
Week 12
Rogers, E. M. (1998). Problems with conducting field experiments. Lecture to
Dept. of C & J, University of New Mexico.
Jackson, S. (1992). Message effects research: Principles of design and analysis.
New York: Guilford. Chapter 1
Frey, L. R., Botan, C. H., & Kreps, G. L. (2000). Investigating communication: An
introduction to research methods (2nd ed.). Boston: Allyn & Bacon.
Week 13
MANOVA handouts
Week 15
Logistic regression handouts
Other handouts
á Codifying academic writing: Quantitative Research Studies
á Directions for using SPSS
o Factor analysis
o Reliability
o Compute New Variable
o Reverse Scoring
o T-tests
o Single Factor ANOVA
o Multiple Factor ANOVA
o MANOVA
o Correlations
o Linear Regression
o Nonparametic
á Overview of statistical procedures
o Factor analysis
o T-test
o ANOVA
o MANOVA
o Correlation
o Regression
á Statistical printouts
o T-test
o One-way ANOVA
o Univariate analysis of variance
o Correlations
o Regression
o Factor analysis
o More regression
á Instrument for data set (Face concerns in 4 cultures)