Communication
Research Methods: Quantitative (C & J 607)
Fall 2010
Professor: John Oetzel
Office:
CJ 204
Office Hours: T 3-4, W 9:30-11 or by appointment
Email:
Course Description/Objectives:
This course is an advanced
survey of quantitative research methods and data analysis. In the course,
we discuss four different research methods that have specific relevance
to communication study: content analysis, interaction analysis, questionnaire
construction, and experimental design. We place special emphasis on
the validity and reliability of measurement in each of these methods,
as well as sampling. We also discuss 5 advanced procedures for analyzing
statistical data: multivariate analysis of (co)variance, confirmatory
factor analysis, model testing (structural equation models), and hierarchical
linear modeling. We may also have time for a topic or two to be chosen
by the class (e.g., logistic regression and network analysis).
There are three factors
of which you should be aware for this course. First, I assume that you
have a working knowledge of statistics equivalent to having C &
J 507 or another intro to statistics course (analysis of variance, correlation,
t-test, and regression). Second, I assume that you have a working knowledge
of research methods equivalent to having C & J 501 or another introduction
to research methods course (conceptualization, measurement, reliability,
validity, and basic research design). Third, the course is a survey
course so it will not be the definitive coverage of any one topic. You
will receive a taste of each topic and enough to carry out a preliminary
research project. However, to become proficient at the method/data analysis
you will need to do some further reading on the topic. I will provide
some leads for further reading to get you started. Please note that
I have place some review handouts and instructions in the ÒStatistics
ReviewÓ folder in WebCT. This includes a handout on how to write quantitative
research papers.
Required Texts:
Mertler, C. A., &
Vannatta, R. A. (2001). Advanced and multivariate statistical methods:
Practical
application and interpretation. Los
Angeles: Pyrczak.
Readings available on
WebCT
Course Procedures/Policies:
1) I will accept late
papers for one week after a deadline with a 10% deduction.
2) Please do not procrastinate
in this class. You need to keep up with what we are doing so that you
donÕt fall behind.
3) Qualified students
with disabilities needing appropriate academic adjustments should contact
me as soon as possible to ensure your needs are met in a timely manner.
Handouts are available in alternative accessible formats upon request.
4) This course encourages
different perspectives related to such factors as gender, race, nationality,
ethnicity, sexual orientation, religion, and other relevant cultural
identities. This course seeks to foster understanding and inclusiveness
related to such diverse perspectives and ways of communicating.
5) The course emphasizes ethical practices and perspectives.
Above all, students and instructors should strive to communicate and
act, both in class interactions and in assigned coursework, in a manner
directed by personal integrity, honesty, and respect for self and others.
Included in this focus is the need for academic honesty by students
as stated by the UNM Pathfinder. Students need to do original work and
properly cite sources. For example, be aware of plagiarismÑdirectly
copying more than 3 or 4 words from another author without quoting (not
just citing) the author is plagiarism. Further, course content will
encourage the ethical practices and analysis of communication research.
6) You will receive an F on a plagiarized paper and
reported to appropriate parties.
Course Assignments:
Assignment% of final grade
Mini Assignments
40%
1 of 2 Methods Assignments (Content Analysis, Interaction Analysis)
3
of 5 Statistics Assignments (Multivariate Analysis of Covariance, Model
Testing, Confirmatory Factor Analysis, Data Cleaning/Sample Size/Power
Analysis, Hierarchical Modeling)
Large Assignment (Research
Project)
55%
Participation 5%
Note
1: If you want to both methods assignments or all five stats assignments
(or both), IÕll take the best 1 (or 3) grades on the assignments.
Note
2: I am somewhat flexible with my assignments (the exception is the
research project, which you have to do). If you want to suggest an alternative
assignment (or set of assignments) that better fits your learning style,
let me know. WeÕll talk and negotiate an alternative.
Grading Scale:
93-100%
A 87-89%
B+ 70-79%
C
90-92%
A- 83-86%
B 60-69%
D
80-82% B-
below 60% F
Grading Criteria:
1)OriginalityÑThis point focuses on the degree to which you are
moving beyond basic classroom discussion and prior research. The research
project should make an original contribution to the literature.
2)OrganizationÑThe essay should flow well. I can see the progression
of ideas and understanding why they are laid out as they are. There
is an introduction, body, and conclusion for small assignments (at least)
and an intro, lit review, methods, results, and discussion for the article-length
assignment.
3)Grammar, spelling, and proper APA format
4)Accuracy/appropriateness of data analysisÑquality analysis and
conclusions.
5)Depth of critical thinkingÑquality of arguments (i.e., using
evidence to support opinions).
6)Following directions--a necessary but not sufficient criterion
for a good paper. I will provide instructions for the assignment. However,
they are not a laundry list of what you need to do get an A. Doing excellent
work is not simply about following directions; itÕs about challenging
yourself, thinking in original ways, and writing well.
Description of
Assignments:
Methods: Choose
One of the Following Two
#1 Content Analysis
For this assignment,
I want you to analyze the content found in some sort of mass media (e.g.,
TV shows, newspapers, magazines, etc.). Your task is to select the content
and a research question associated with the content. Then, youÕll need
to carry out the study. The purpose is to give you some practice utilizing
content analysis. Specific steps include:
1)Select the media (e.g., compare two TV shows or the coverage
of a topic in two different newspapers)
2)Develop a research question to frame the study.
3)Develop a suitable set of categories to analyze the media (i.e.,
the content scheme). Determine the units of analysis.
4)Categorize and tabulate the data. Determine the intercoder reliability
and statistically analyze the data to determine the answer to your research
question.
5)Interpret and write a short (3-4 pp.) report. The report should
include an introduction and research question (no need to cite literature),
method, findings, and brief conclusion (noting any shortcomings or limitations
of your studyÑdonÕt worry about discussing findings since this is a
mini-assignment. I want to see the focus on what you learned about what
you did).
#2 Interaction
Analysis
This assignment will
be similar to that of content analysis except that now you will be examining
the structure and content of patterns of communicative interaction.
You can choose to analyze any sort of communicative dialogue (e.g.,
a play or I have some videotapes that may work for you). Best if there
is a transcript available, but it is not an outright necessity. Here
are the steps:
1)Select the interaction
2)Develop a research question to frame the study.
3)Develop a suitable set of categories to analyze the interaction.
It should include both content and structure at some level (such as
who initiates and responds). You may use an existing coding scheme (such
as one-up, one-down, one-across). The scheme should fit the research
questions. Determine the unit of analysis.
4)Categorize and tabulate the data. Determine the intercoder reliability
and statistically analyze the data to determine the answer to your research
question (if you canÑif you have a sequencing question, you probably
wonÕt be able to do this. WeÕll discuss in class further).
5)Interpret and write a short (3-4 pp.) report. The report should
include an introduction and research question (no need to cite literature),
method, findings, and brief conclusion (see #1 for what to put in conclusion).
Statistics: Choose
Three of the Following Five Assignments
IÕll make available
a data set on SPSS (on WebCTÑSPSS Files folder) to complete these assignments.
The data set includes participants from the China, Japan, Germany, and
the US about a recall interpersonal conflict. IÕve provided you with
a copy of the questionnaire and original scoring of variables (on WebCTÑSPSS
Files folder) so that you can see how particular variables were measured.
NOTE: You may only do #4 OR #5 not both. If you do both, it will be
consider an extra one and IÕll take highest grade of the two.
Additionally, I have
included three articles that I have published based on this data set
in the Conflict Research Articles folder. In addition, I have placed
additional articles I have written on HLM and CFA in the Other Analysis
Articles folder. These articles will provide some templates of how to
write up these types of statistics and give you and idea of how to interpret
these statistics.
#3 Data Cleaning
and Sample Size Determination
This assignment will
focus on several key aspects of data analysis and research design. LetÕs
start with the design. I want you to calculate the sample size I should
have used with this data set given alpha = .05, power = .80, and small,
medium, and large effect sizes. Tell me if I had an adequate sample
size. Second, I want you to do some pre-analysis data screening. Pick
3 variables. Check normality, linearity, and homoscedasticity. If there
are problems, discuss what you should do about them (if anything). Do
not actually clean up the data
#4 Measurement
and Confirmatory Factor Analysis
The first 20 items
of the questionnaires measure self-construals (independent and interdependent),
while the next 34 items measure face concerns (self, other, and mutual).
I want you to analyze the measurement of the face concern items (weÕll
do self-construal in class). (If you are feeling up to it, you can do
all 5 subscales at the same time and try to confirm as many of the factors
as you possibly can; Hint: I could only do 4 of the 5). The first step
will be to standardize the items by culture (IÕve done this for you
to save time; IÕve also replaced the missing values, which is needed
for AMOS). Second, confirm the existing scales and eliminate poor items
using rules (see handout on CFA in Confirmatory Factor Analysis folder).
Complete only two iterations of this analysis in order to confirm the
scales. Write up your results in a short 1-page report (or so) with
appropriate tables (i.e., factor loadings).
#5 Model Testing
I want you to create
a model of the factors that influence face behaviors. Some of the face
behaviors in the data set include:
rlaggcÑAggression
rlpsusÑProblem-solving
rldefc--Defending
rlrespus--Respect
rlpretusÑPretend
I can provide more
detail about these variables. I want you to develop a model for at least
3 of these behaviors. WeÕll talk about the potential models, but I donÕt
want to direct you too much. Then, I want you to these the fit of your
model with a structural equation model. Once you have a model, you can
test its fit. Modify the model using modification indexes. Complete
only 2-3 iterations of the model testing (see handout in the Structural
Equation Modeling folder). Write up the results in a short 1-2 pp. report:
Include the hypothesized model, the findings with the final model, and
a table of correlations. Make sure to discuss why the modifications
are appropriate (or why there were not).
#6 Multivariate
Analysis of Covariance
I have already done
some analysis on the data and have created the following variables (there
are others, but this is sufficient for our task here):
relindusÑIndependent
self-construal (self as independent)
reldepcÑInterdependent
self-construal (self as connected to others)
relpowusÑPower distance
relothcÑOther-face
concern
rlselfusÑSelf-face
concern
rlmutusÑMutual-face
concern
Other variables of
interest are
StatusÑParticipants
were asked to recall someone of equal status (1) or higher status (2)
IntimacyÑParticipants
were asked to recall someone close (1) or distant (2) from them
CountryÑUS (1), Japan
(2), Germany (3), China (4)
I want you to investigate
the affect of national culture on face concerns. There are several potential
intervening variables including self-construals. Utilize multivariate
analysis of covariance to investigate this topic (see handouts in the
MANOVA folder). Write up your findings in a short 1-2-page report. List
your hypotheses and statistics to test the hypotheses. Include a table
of means and regression coefficients if appropriate.
#7 Hierarchical
Linear Modeling
I have a data set
involved 561 workers organized in 41 work groups at a manufacturing
plant (Level 1 and Level 2 folder for the SPSS files). I have variables
created that describe the groups (size of group and diversity of group
in terms of sex and ethnicity) as well as variables that describe the
individual members (self-construals, face concerns, perceived quality
of the groupÕs process, perceived quality of the groupÕs work).
For this assignment,
the DV will be the perceived quality of the groupÕs work and the IVs
at the individual level are self-construals (independent and interdependent),
self-face, other-face, and perceived quality of the groupÕs process.
The IVs at level two are the size of the group and the diversity of
the group in terms of sex and diversity of group in terms of ethnicity.
These should be used to predict intercepts and slopes removing. Build
the model in three steps: a) the unconditional model (no DVs and only
IV), b) the level one IVs, c) level one and level two IVs (see handout
in the HLM folder). Write up your findings in a short 1-2-page report
and include a table of regression coefficients. If you have cross-level
interaction effects, include a figure.
Participation
Participation will
focus on your attendance and, more importantly, level of engagement
during class periods. You can have one absence as a freebie, everyone
after that hurts your participation grade (10% reduction per absence
unless there is a legitimate excuse for missing more than one classÑe.g.,
family emergency, attending professional conferences). Level of engagement
will be judged on several criteria: a) preparation for class (did you
do the reading), b) participation in discussions, c) attentiveness to
the discussion/demonstrations, and d) asking/answering questions during
demonstrations. The class will be a mixture of seminar discussions and
Òdemonstrations.Ó
Major Project
The major project for
this course is a research project. You may focus on any topic in communication
you like, so long as you quantify the variables under study (if you
major is not communication you may choose a different type of variable
after discussing that with me). You are required to plan, research,
collect data, analyze data, and write up the findings. This could be
a pilot study for your thesis/dissertation or simply a project in which
you are interested. The research must be original and make a contribution
to the field (or at least moving in that direction in the case of the
pilot study). Additionally, it must use at least one method/statistical
procedure we discuss in the class. I will encourage you to send your
papers to a conference so that you get experience presenting research,
as well as submit them for publication.
It is very important
that you choose a topic that is manageable and can be completed in one
semester. You will need to get data from a minimum of 60 participants
or units (30 per each experimental group if there were two groups) and
I encourage you to collect enough to have the appropriate power (if
not during this semester, in the immediate future). I will not give
incompletes for this project except in extreme circumstances. It
is critical that you get started now. To assist you in completing this
project on time, I have set up the following deadlines for turning in
parts of the overall paper:
a) Aug. 31--Propose a general idea/topic (1 p.)
b) Sept. 21--Proposal (Intro/literature review/methods) (12-15 pp.)
c) Dec. 9ÑPresentation of the findings and final paper due (20-25 pp.);
I will be available
to help you on this project and I will provide feedback on each of your
rough drafts (a and b above). You are welcome to turn in assignments
early. My concern on the rough drafts is that the project is manageable,
original, and well conceived. I will not grade the rough drafts per
se other than noting if it was turned in on time.
*Please note that will also need to get IRB Human
Subjects Board Approval for your project if relevant (i.e., you are
collecting data from humans; most content analyses do not require approval,
but check with me or the web site if you are unsure). I suggest submitting
this early in the semester. The forms and directions are on the UNM
web site.
Course Outline
Note: Readings
and assignments are due on the day they are listed. I list a number
of suggested readings. I do not expect you to read these for this class.
I list them as references in case you want an in-depth coverage of a
particular topic.
Week
Topic/Readings/Assignments
1
(8/24) Introduction
to the Course. Discussion of fears, hopes, and dreams about quantitative
research methods. Also, weÕll talk about your interests and how to set
up the major assignment. Difficulties in conducting intercultural or
cross-cultural research. WeÕll discuss when to use which data analysis
procedure. Further, given the focus of our program on doing intercultural
communication research, weÕll discuss some of the problems with reliability
and validity of quantitative research in this context.
Readings: Gudykunst; Johnson & Tuttle in Intercultural Research
Issues folder
Suggested Reading: A basic text of statistics for review
2
(8/31) Content
Analysis. WeÕll discuss the basics of what is content analysis and how
to do it.
Reading: Krippendorff in the Content Analysis folder
Suggested
Readings: Krippendorff, K. (1980). Content analysis: An introduction
to its methodology. Newbury
Park, CA: Sage. Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage
1-page research
topic is due (8/31)
3
(9/7)
Intercoder reliability. WeÕll discuss how to calculate intercoder reliability.
Readings: Potter & Levine-Donnerstein; Intraclass correlation instructions
in the Content Analysis folder
4
(9/14) Interaction
analysis. WeÕll discuss the basic concepts and how to do interaction
analysis.
Readings: Hirokawa;
Lukasko Emmert in the Interaction Analysis folder
Assignment #1
is due (9/14)
5
(9/21) Interaction
analysis. WeÕll look at one of the more famous interaction analysis
coding schemes as well as how to analyze sequential data.
Readings: Rogers & Farace; Bakeman & Gottman; ÒInteraction Analysis
Basic how toÓ in the Interaction Analysis folder
Suggested Reading: Bakeman, R., & Gottman, J. M. (1997). Observing
interaction: An introduction to sequential analysis
(2nd ed.). Cambridge, UK: Cambridge University Press.
Proposal is due (9/21)
6
(9/28) Pre-analysis
screening and sample size determination. WeÕll discuss techniques for
checking the assumptions for statistical analysis and then identify
procedures for determining how large your sample should be.
Readings: Ch. 3 in multivariate book; Cohen & power table in the
Power Analysis folder
Assignment #2
is due (9/28)
7
(10/5) Constructing
Questionnaires and Scales. WeÕll go through the steps to construct an
effective questionnaire.
Readings: Larkey; Oetzel handout in the Survey-Scale Construction folder
Suggested Reading: Fink, A., & Kosecoff, J. (1998). How to conduct
surveys: A step-by-step guide. Thousand
Oaks, CA: Sage. Peterson, R. A. (1999). Constructing effective
questionnaires. Thousand Oaks, CA: Sage. King et al & King &
Wand in the Interaction Analysis folder (these last two articles focus
on using anchoring vignettes for cross-cultural validity)
Assignment #3 is due (10/5)
8
(10/12) Confirmatory
and Exploratory Factor Analysis. WeÕll discuss measurement issues and
how data analysis can help improve measurement.
Readings: Ch. 9 in multivariate book; CFAselfconstrualoutput in Confirmatory
Factor Analysis folder
Suggested Reading: Kline, R.B. (2004). Principles and practice of
structural equation modeling (2nd
ed). New York: Guilford Press.
9
(10/19) Model testing.
WeÕll discuss how to test models using structural equation modeling.
Readings: Maruyama & SEMfacenegotiationoutput in Structural Equation
folder; Ch. 8 in multivariate
Suggested Readings: Maruyama, G. M. (1998). Basics of structural
equation modeling. Thousand Oaks, CA: Sage.
Assignment #4 is due (10/22)
10
(10/26) Experimental Design for
Message Effects. WeÕll discuss the complexity of designing research
to test message effects.
Readings: Jackson & Frey et al. in the Experimental Design folder
(Frey et al as a review of experimental design)
Suggested
Readings: Jackson, S. (1992). Message effects research: Principles
of design and analysis. New
York: Guilford.
Assignment #5 is due (10/29)
11
(11/2) Multivariate
analysis of (co)variance. WeÕll examine how to statistically control
for intervening variables as well as analyze multiple dependent variables.
Readings: Ch 6-7 in multivariate book; MANOVAoutput and instructions
in the MANOVA folder
12
(11/9) Hierarchical
linear modeling. WeÕll examine the assumptions and processes for conducting
analyses that involve multiple levels (we will focus only on two levels)
Readings: Snijders & Bosker; HLMoutput in the HLM folder
Suggested Reading: Snijders, T. A., & Bosker, R. J. (1999). Multilevel
analysis: An introduction to basic and advanced multilevel modeling. London: Sage.
Assignment #6
is due (11/12)
13
(11/16) MondayÑFinish up HLM
Wednesday--Open day for topic of class choice (e.g., network analysis,
logistic regression, cross-cultural surveys)
14 (11/23)
NO CLASS (Thanksgiving)
Assignment #7
is due (11/23)
15
(11/30) Open Week for class choice(s)
16 (12/7)
MondayÑopen day for finalizing projects/asking questions
Presentations
of research project. (12/7)
Research Project
is due
(12/9)
17
(12/14) Final Exam Week. If we need
Week 16 to catch up, weÕll move the presentation to the final exam period.
Citations for Articles
Intercultural
Research Issues
Gudykunst, W. B. (2000). Methodological issues in conducting theory-based
cross-
cultural research.
In H. Spencer-Oatley (Ed.), Culturally speaking: Managing relations
in talk across cultures. London:
Cassell.
Johnson, J. D., & Tuttle, F. (1989). Problems in intercultural research.
In M.
Asante &
W. Gudykunst (Eds.), Handbook of international and intercultural
communication. Newbury Park, CA: Sage.
Content Analysis
Krippendorff, K. (1980).
Content analysis: An introduction to its methodology.
Newbury
Park, CA: Sage. Ch. 2 & 4
Weber, R. P. (1990).
Basic content analysis
(2nd ed.). Newbury Park, CA: Sage.
Ch.
4
Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity
and reliability
in content
analysis. Journal of Applied Communication Research, 27, 258-284.
Interaction Analysis
Hirokawa, R. Y. (1987). Group communication research: Considerations for
the
use of interaction
analysis. In C. Tardy (Ed.), A handbook for the study of human communication. Ablex.
Lukasko Emmert, V. J. (199?). Interaction analysis. In P. Emmert &
L. Barker,
(Eds.), Measurement of communication behavior. New York: Longman.
Rogers, L. E., & Farace, R. V. (1975). Analysis of relational communication
in
dyads: New
measurement procedures. Human Communication Research, 1, 222-239.
Bakeman, R., & Gottman, J. M. (1997). Observing
interaction: An introduction to
sequential analysis (2nd ed.). Cambridge, UK: Cambridge University
Press.
Chapter 6
Power Analysis
Cohen, J. (1977). Statistical power analysis for the behavioral sciences.
Orlando,
FL: Academic Press. Ch. 1
Survey-Scale Construction
Larkey, L. K. (1996). The development and validation of the workforce
diversity
questionnaire:
An instrument to assess interactions in diverse workgroups. Management
Communication Quarterly, 9, 296-337.
Oetzel--Developing a cultural sensitive measure and structural equation
modeling--handouts
King, G., Murray, C.J.L., Salomon, J.A. & Tandon, A. (2004). Enhancing
the validity and cross-cultural comparability of measurement in survey
research. American Political Science Review, 98, 191-207.
King, G., & Wand, J. (2007). Comparing incomparable survey responses:
Evaluating and selecting anchoring vignettes. Political Analysis,
15, 46-66.
Structural Equation Modeling
Maruyama G. M. (1998). Basics of structural equation modeling. Thousand Oaks,
CA: Sage.
Experimental Design
Jackson, S. (1992).
Message effects research: Principles of design and analysis.
New
York: Guilford. Chapter 1
Frey, L. R., Botan,
C. H., & Kreps, G. L. (2000). Investigating communication: An
introduction
to research methods (2nd
ed.). Boston: Allyn & Bacon.
HLM
Snijders, T. A., & Bosker, R. J. (1999). Multilevel analysis: An
introduction to basic and advanced multilevel modeling. London: Sage.
Other handouts in Logistics Regression folder or Statistics Review
