Anth 473L/573L: Archaeological Measurement and Lab Analysis

Spring 2003

 

Exercise 2: Validity and Reliability

 

Due Friday February 14 at the beginning of lab.

 

This exercise is intended to help you integrate the concepts of validity and reliability into your thinking as evaluative tools through a detailed consideration of the research of other archaeologists.  Specifically, you will apply the concepts in an evaluation of the measurements, units and inferences regarding the number of ceramic vessels deposited in the trash mound at Pueblo Alto as well as “explanations” of the causes of that deposition.  Pueblo Alto is a large, 133 -room masonry pueblo and is considered one of the “Great Houses” in the Chacoan system.  It is located on the mesa top north of Chaco canyon, approximately 50 m above the valley floor where Pueblo Bonito and several other Great Houses are located. 

 

 

GENERAL BACKGROUND

 

The archaeological record at Chaco Canyon, especially in a central area along approximately 16km of Chaco Wash around Pueblo Bonito, has played a seminal role in the history of Southwestern archaeology.  The 14 large ruins along the canyon bottom and on the mesa tops are impressive (go visit, if you haven’t already!).  Because of the dramatic architecture and fine construction of the ruins, and especially of Pueblo Bonito, Chaco was one of the first locations of major (semi-) systematic excavation in the Southwest.  Pueblo Bonito was excavated by George Pepper and others beginning in the 19th century.  Neil Judd continued excavations there in the early decades of the 20th century and Florence Hawley Ellis conducted field schools for UNM at other sites in Chaco Canyon in the 1930s and 40s.  The National Park Service, in association with UNM Department of Anthropology continued that work in the 1970s and 80s, examining roads, Great Houses away from the central canyon (“outliers”), and smaller settlements.  Some of that material is still being processed, analyzed and published. 

 

The archaeological sequence in Chaco Canyon is very long, but the apex of development occurred over a 200+ year period, from roughly AD 900 to 1120. Archaeologists refer to the cultural expressions during this time as the “Chaco Phenomenon”, or the Chaco regional system, and it is this period that figures so prominently in considerations of pre-Hispanic cultural complexity in the Southwest.  Depending on how it is measured, the regional system encompassed at least 65,000 km2 of the San Juan Basin and adjacent uplands.  A number of traits define this expression, including:

Ø       Shared architectural styles and similar material culture inventories.

Ø       Two settlement types – large, architecturally impressive Great Houses and contemporaneously occupied small sites, usually located near each other.

Ø       Outliner communities throughout the San Juan Basin and beyond.

Ø       A “road system” connecting (at least symbolically) many of the outliers and some smaller sites to each other and to the Great Houses in the canyon.

Ø       Extensive exchange, long distance transport of raw materials for construction and, perhaps, specialized production of craft items.

Ø       High quality and exotic materials in the Great Houses in the canyon.

 

The Great Houses, especially those within the canyon, appear to have been planned, and most were multi-storied.  The sandstone core-veneer masonry is extremely well made, and the rooms, by comparison with others of the same period, are spacious with high ceilings.  The material from these Great Houses includes habitation debris, high quality “art objects” of turquoise and other materials and exotics, including Macaw skeletons and copper bells from Mexico.

 

The meaning, structure, and functioning of the Chacoan system are the subjects of ongoing debates.  Some archaeologists view the Chacoan system in economic, subsistence, risk management terms.  In this interpretation, the Great Houses of the canyon functioned as both economic redistribution centers (for trade, etc.), and as storage facilities.  The Great Houses existed to buffer the risks of agricultural subsistence in the marginal, spatially and temporally variable Southwest climate.  More recent explanations of Chaco phenomenon have emphasized religion and power, with the Great Houses in the canyon symbolizing real and assumed power (e.g. Renfrew 2001).  Rather than villages or towns, the Great Houses are interpreted as ceremonial centers and pilgrimage destinations with a small resident population who controlled sacred knowledge and a ceremonial calendar. The number of Great Houses in the canyon is sometimes explained as the consequence of competing or sequential power centers (e.g. Sebastian 1992).  Steve Lekson (e.g. Lekson 1984; Stein and Lekson 1992), among others, has argued that much of the Great House architecture in the canyon is built for display and/ or ritual enactment.  Trash mounds, including the large trash mound at Pueblo Alto, are suggested to have been “platforms” for staging rituals that included the purposeful destruction of ceramic vessels (Toll 2001; but see Wills 2001 for a well-reasoned alternative interpretation that was part of the inspiration for this exercise).

 

 

PUEBLO ALTO ARCHAEOLOGY AND THE WESTERN TRASH MOUND

 

As mentioned above, Pueblo Alto is considered a Great House. The architectural portion of the site covers approximately 8 hectares.  The trash mound is located at the southern end of the central plaza and, although badly eroded, is approximately 60 meters long and 26 m wide. Consistent with the pre-1990’s models of the Chacoan system, the western trash mound was initially seen as a village dump for household debris that accumulated during the occupational span of the pueblo, primarily AD 950 – 1200.  Parts of pueblo, including some of the trash mound, were excavated by the Chaco Project (NPS-UNM) in 1976 – 1978.  The goals of the excavation included determining whether Pueblo Alto was an administrative center involved in redistribution of agricultural products to small communities.  The convergence of several road segments at Pueblo Alto made this hypothesis seem plausible. Unfortunately, evidence to support the hypothesis of redistribution was not uncovered. 

 

Currently, based on the analyses following the excavation, Pueblo Alto is interpreted as a ritual center and, in part because it is the only Great House excavated using modern methods, this interpretation is extended to other Chacoan Great Houses.  The trash mound is suggested to have been a constructed platform where, at least during one interval, imported ceramics were deliberately broken and deposited.  As Toll (1985: 196) states: “Translating the estimates into identifiable imports during the Gallup portion of the Trash mound, at least 49,270 trachyte tempered pots were brought in and deposited [in addition to] 31,310 gray ware jars… 3,834 chalcedonic sandstone graywares… [and] 1,738 San Juan igneous tempered whitewares.“  The estimated number of vessels was considered too great to be accounted for by simple domestic use, leading to the inference that the ceramics were deposited in a ritual context.  This interpretation was first developed and discussed in two sources:

 

            Toll, H. Wolcott, 1985, Pottery Production, Public Architecture, and the Chaco Anasazi System.  Ph. D. Dissertation.  University of Colorado

and

            Toll, H. Wolcott and McKenna, Peter J., 1987, The Ceramography of Pueblo Alto, in Investigations at the Pueblo Alto Complex, Chaco Canyon, New Mexico 1976-1979. Vol III. edited by Mathien, F. M. and T. C. Windes, National Park Service, Department of Interior.  Santa Fe.

 

 

TRASH MOUND EXCAVATION

 

The trash mound excavation had three components.  First, the investigators dug a long, narrow trench running from northwest to southeast, separated into 11 grid units of 2 x 2 meters each. A total of 20 levels are listed on their map, and each level was 20 cm deep. Because of slumping in the mound, exact provenience became a problem.  To resolve this problem, the investigators excavated a series of short stratigraphic columns (termed “booths”) from the southwest wall of the test trench.  62 of the layers in the booths duplicated those in the mound.  Third, there were three backhoe tests, although these were not analyzed in detail and they are hardly mentioned in the reports.

 

Decorated ceramics, cross-dated to tree-ring dates, were used to identify temporal periods in the mounds, but it is important to note that these periods (deposits) were not really discrete and stratigraphically bounded.  The mound was not a simple layer cake. There was evidence of slumping and various types of mixing (especially bioturbation) and identifiable fingerlings of later strata projected into the lower strata.  Later sherd types were present in the earlier unit, albeit in lower frequencies, and each excavated level was assigned to one or the other temporal/ depositional unit on the basis of the majority of the ceramics present.  The earlier stratigraphic unit was identified as the “Red Mesa Mound” after the Red Mesa ceramic series.  The date range of this unit is AD 970-1020.  18.9 cubic meters of this deposit were excavated.  The second depositional unit is called the “Gallup Trash Mound” after the Gallup black on white ceramic type.  The date range of this unit is A D 1020-1120, and 35.5 cubic meters were excavated.  Based on Tom Windes’ estimates of the total volume of the mound and on the assumption that the proportions of each depositional unit in the trench were equivalent to their proportions in the entire mound, the investigators estimated that 2.2% of each depositional unit was excavated.  Therefore, the sample size of each stratigraphic/ cultural unit was 2.2%.

 

 

TABLED DATA

 

Tables 1 and 2 present the data that led the excavators to the interpretation that ceramic consumption at Pueblo Alto was extraordinary. Table 1 presents the counts of ceramics, volumetric measurements and estimations and projected consumption of ceramic vessels at Pueblo Alto.  In order to arrive at vessel counts, the excavators assumed (without clearly stating their assumption) that each rim sherd that could not be refitted with another in the sample represented one vessel.  Table 2 shows estimates of per annum pottery consumption at Pueblo Alto and other settlements.  Estimates of number of families (and therefore consumption rate of ceramics) are often based on extrapolation from ethnography and archaeological data including room sizes, types of rooms and the presence of certain features, especially hearths, in some rooms.  In this case, the investigators simply counted the number of rooms facing the plaza.  Note that a smaller proportion of the midden was excavated at Pueblo Alto than at other sites and that the projection of total number of ceramic vessels consumed is much larger, especially in the Gallup Trash Mound.  Again, the estimated consumption rates are the basis of the inference of ritual breaking and deposition of ceramic vessels.

 

As Toll recognizes, the final inference of ritual destruction of ceramic vessels at Pueblo Alto required a series of assumptions, inferences, calculations and estimates.  He states, “Estimating the ceramic population at a site and arriving at per annum and per family use rates is a procedure fraught with guessing, assuming, fudging, and leaping at any site; at Alto the degree of these gerunds is larger than usual.  Two major reasons lie at the root of this exaggeration: the small relative amount of the site dug, and the greater than usual uncertainty as to the use and resident population of the site through time.”(1985: 176 – 177)  The (simplified) outlines of their argument can be restated as follows:

 

1.       Assume that each rim sherd represents 1 vessel.

2.       Assume that the vessels in the mound were used and broken at Pueblo Alto.

3.       Estimate the total size of the mound.

4.       Estimate the total size of each temporal-stratigraphic unit based on the assumption that the proportion of each in the excavated area is the same as the proportion in the unexcavated portions of the mound.

5.       Assume that the density of sherds (#/ cubic meter) is constant throughout the mound.

6.       Extrapolate from the sample of rim sherds to the number of vessels in each temporal-stratigraphic unit.

7.       Estimate the number of years represented by each temporal-stratigraphic unit based on the dominant ceramic types and calculate number of vessels deposited per year.

8.       Create an estimate of the number of families who lived at Pueblo Alto.

9.       Estimate the consumption rate of ceramic vessels per family per year.

10.   Infer that the excess of ceramic consumption above what would be expected is due to ritual.

 

 

ACTIVITIES

 

Before leaving the lab, study the tables and the information we have provided until you understand how Toll moved from potsherds in a trash mound to ritual killing of vessels.  If anything is unclear, ask questions.  Chances are it is unclear to others as well.

 

 

WRITE UP

 

This exercise is not particularly well suited to the standard lab report format.  Instead, the assignment consists of two parts.  The first is a series of questions and brief exercises intended to get you thinking about the reliability, empirical validity and abstract validity of the unit structures created and described by the Chaco Project participants.  Write a concise answer (not more than half of a page, with the possible exception of question 11) to each question.  For the second part of the assignment, write a short paper (~5 pages) about the validity and reliability of the measurements and units used in the investigation at Pueblo Alto.  Try to create a logical flow of ideas in your paper and incorporate and synthesize the issues raised by the questions.  (There will naturally be some redundancy between the two parts of the assignment.)  Ultimately, the point is not to trash the Chaco Project but to carefully evaluate the research of others so that, when you build your own research projects, you will be able to think clearly about the reliability and validity of the unit structures you create and the measurements you perform. 

 


QUESTIONS:

 

There are three variables that form the basis of estimates of rates of ceramic consumption: pots, people and time.  The estimates of all three are necessarily based on assumptions regarding the archaeological record, sampling and the people (“cultural system”) that created the record.  In answering the following questions, you will identify some of these assumptions.  Then you will substitute other plausible assumptions and use them to derive different estimates for each variable.  This is one way of evaluating the reliability and validity of the unit structures employed at Alto and it should clarify the logic behind the inferences and conclusions that were published by the excavators. 

 

sherds and vessels

1.       What synthetic unit does Toll use when he is building his arguments about rates of consumption?  What does he actually measure to create his synthetic unit (what is his observational unit)?  Evaluate the linkage connecting the two in terms of precision, accuracy, empirical validity, and/or conceptual validity. 

2.       Toll and McKenna (1987: 37 – 39, tables p.43, 44) note that there is a difference in the proportions of graywares to decorated wares between sites and (at Alto) when measured by total sherd counts versus estimated numbers of vessels represented.  They attribute between- site differences to differential post-occupational disturbance.  At Alto, they note the confounding effects of the greater durability of trachyte tempered vessels (52% of the graywares and 12% of the decorated wares) causing the graywares to persist as larger sherds (fewer sherds/ vessel) coupled with the larger average size of grayware vessels (more sherds/ vessel).  It is a convoluted argument.  The graywares in the Alto sample are 99.8% jars and the decorated wares are 63% bowls, 16% jars and 21% other forms.  How could vessel form affect their estimates of number of vessels present, given their methods?  Given the different distributions of forms, how would this affect the ratio of graywares to decorated wares in bulk sherd counts vs. vessel estimates?  If the “target” is an estimate of the number of vessels, are their methods precise and accurate, given the effects of vessel shape?

3.       Toll and McKenna calculate that their numbers imply an average of 16.8 sherds/ vessel (1987: 206).  How could you empirically evaluate whether this is a reasonable number of sherds per vessel for this sample?  In his excavations at Mug House (a roughly contemporary site in the Chaco area with a similar estimated population size) Rohn estimated 138 sherds per corrugated jar and 63 sherds per whiteware (decorated) vessel (cited in Toll 1985: 187 - 188).  Although there is no particular reason to believe Rohn’s estimates have any bearing on Pueblo Alto, use the tabled data to calculate the number of grayware and whiteware vessels present using his estimates.

4.       Looking at the problems you identified in questions 1 - 3, are counts of potsherds a reliable (accurate and precise) way to estimate the number of vessels in an archaeological deposit? Is there another unit of measurement that would avoid some of the problems of comparability between sites, differential breakage of ceramic types, different shapes of ceramic vessels etc?  What is the unit, and how might you use it to approach the “target” of estimating number of ceramic vessels?

 

samples and populations

5.       In order to extrapolate from the excavated sample to the total number of vessels deposited in the mound, the investigators performed calculations that used an estimate that 2.2% of the mound was excavated.  Toll (1985: 187 – 189) and Toll and McKenna (1987: 206-207) assert that this estimate is problematic, and throw out a guess that as much as 10% of the mound might have been excavated.  Using the information in the tables (and accepting for now all of the other assumptions) how many vessels were deposited if 10% of the mound was excavated?  What is the difference between the estimates of vessels present based on different estimates of how much of the mound was excavated? 

6.       In passing, Toll and McKenna note “the low ceramic content of the backhoe cuts” (1987: 39) in the trash mound.  The sample of sherds recovered from the trench (and not from the backhoe cuts) was used to estimate the total number of vessels in the mound.  What does Toll implicitly assume regarding the representativeness of the two different samples?  What does this suggest about the reliability (both precision and accuracy) of his estimates? 

7.       Toll and McKenna state that, “the numbers in the estimates generated in this way are frankly alarming – it seems almost inconceivable that so many pots were disposed of at this one site.” (1987: 206)  Given your answers to previous questions, should we be “alarmed” by the number of pots or is there something else to be concerned with?  Briefly discuss the reliability and validity of their methods for estimating the number of vessels present in the mound.

 

derived estimates

8.       To estimate consumption rates, the investigators had to use estimates of the length of time represented by each deposit.  They recognize (Toll and McKenna, 1987: 117) that these estimates may be flawed.  Returning to the data in the tables, how do the estimated consumption rates change if each of the two time periods is longer by a decade?  By fifty years?  By a century? 

9.       The third estimated value (in addition to number of vessels and time) used to derive rates of consumption (vessels per year per family) is the number of people that inhabited Pueblo Alto.  Based on the number of rooms facing the plaza, the investigators estimated 20 family units.  A common pattern in ancestral Puebloan sites is for each family to occupy suites of 2 to 4 rooms (one for living, and another one to three for storage).  Given 133 rooms at Alto, calculate the number of family units present if each occupied 4 rooms and if each occupied 2 rooms.  Another approach used by some archaeologists to estimate population size is to apply an ethnographically derived mean value of approximately 10 square meters of roofed area per person (Narroll 1962).  Using the map of Alto, measure (don’t be too concerned with the accuracy of your measurement, but do tell me what it is) the roofed area (excluding kivas) and, assuming 5 people per family, estimate the number of families that lived there.  What new estimates of number of vessels per family per year can you derive with the new estimates of number of families? 

10.   In this question, you will combine the effects of beginning from different assumptions than those of the excavators.  Basically, you will derive a new estimate of ceramic consumption that represents a “lowest plausible value” for comparison with the estimates derived by Toll and McKenna.  Take the number of vessels you calculated in question 3, then assume that 10% (instead of 2.2%) of the mound had been excavated like you did in question 5.  What is the estimated number of vessels in the mound using these assumptions?  Now assume that each time period was 50 years longer than stated by the investigators, and add in your highest population estimate for the site.  What is the use rate of vessels per family per year that you estimate using these assumptions?  How does it compare to ceramic consumption rates at other sites shown on the table?

 

ceramics and ritual

11.   The ultimate conclusion of the Pueblo Alto excavation was that there was a very high rate of ceramic deposition there due to ritual gatherings at the site.  Given your previous answers, is this a secure inference?  Were the excavations of the trash mound at Alto and the calculations of ceramic use rates a valid means of measuring ritual?  Were they designed to be?  Briefly trace the line of reasoning from potsherds to interpreting Chaco as a “Location of High Devotional Expression” (Renfrew 2001).  Using the concept of abstract validity, suggest where there are major weak points in the argument. 

 

extra credit

12.   You are (miraculously) given permission to conduct excavations at a Great House in Chaco Canyon to investigate whether it was, in fact, a pilgrimage destination and ritual center.  What would you measure, how would you quantify it, and how is it related to what you want to know?

 

 

 

 

REFERENCES:

 

Narroll, R.

1962          Floor Area and Settlement Population.  American Antiquity 27: 587 – 589.

 

Lekson, Stephen

1984          The Standing Architecture of Chaco Canyon and the Interpretation of Regional Organization.  In Recent Research on Chaco Prehistory, edited by W.J. Judge and J.D. Schelberg, pp. 55 – 73.  Reports of the Chaco Center No. 8, National Park Service, Albuquerque.

 

Renfrew, Colin

2001          Production and Consumption in a Sacred Economy: The Material Correlates of High Devotional Expression at Chaco Canyon.  American Antiquity 66 (1): 14 – 25.

 

Sebastian, Lynne

1992          The Chaco Anasazi: Sociopolitical Evolution in the Prehistoric Southwest.  Cambridge University Press, Cambridge.

 

Stein, J. and S. Lekson

1992          Anasazi Ritual Landscapes, in Anasazi Regional Organization, edited by D. Doyel.  Maxwell Museum of Anthropology Papers No. 5, Albuquerque.  p. 87 – 100.

 

Toll, H. Wolcott

(see references in text above)

2001          Making and Breaking Pots in the Chaco World.  American Antiquity 66(1): 56 – 78.

 

Wills, W.H.

2001          Ritual and Mound Formation during the Bonito Phase in Chaco Canyon.  American Antiquity 66(3): 433 – 452.