SlideShare a Scribd company logo

Chapter 4 Clustering I02142018 Intro.docx

•Download as DOCX, PDF•
•
K
keturahhazelhurst

Chapter 4: Clustering I 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is Cluster Analysis? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups Inter-cluster distances are maximized Intra-cluster distances are minimized 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Applications of Cluster Analysis Understanding Group related documents for browsing, group genes and proteins that have similar functionality, or group stocks with similar price fluctuations Summarization Reduce the size of large data sets Clustering precipitation in Australia 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is not Cluster Analysis? Simple segmentation Dividing students into different registration groups alphabetically, by last name Results of a query Groupings are a result of an external specification Clustering is a grouping of objects based on the data Supervised classification Have class label information Association Analysis Local vs. global connections 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Notion of a Cluster can be Ambiguous How many clusters? Four Clusters Two Clusters Six Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusterings A clustering is a set of clusters Important distinction between hierarchical and partitional sets of clusters Partitional Clustering A division of data objects into non-overlapping subsets (clusters) such that each data object is in exactly one subset Hierarchical clustering A set of nested clusters organized as a hierarchical tree 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Partitional Clustering Original Points A Partitional Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Traditional Hierarchical Clustering Non-traditional Hierarchical Clustering Non-traditional Dendrogram Traditional Dendrogram 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters Well-separated clusters Center-based clusters Contiguous clusters Density-based clusters Property or Conceptual Described by an Objective Function 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Well-Separated Well-Separated Clusters: A cluster is a set of points such that any point in a cluster is closer (or more similar) to every other point in the cluster than to any point not ...

Education
1 of 195
Chapter 4: Clustering I 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is Cluster Analysis? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups
Inter-cluster distances are maximized Intra-cluster distances are minimized 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Applications of Cluster Analysis Understanding Group related documents for browsing, group genes and proteins that have similar functionality, or group stocks with similar price fluctuations Summarization Reduce the size of large data sets
Clustering precipitation in Australia 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is not Cluster Analysis? Simple segmentation Dividing students into different registration groups alphabetically, by last name Results of a query Groupings are a result of an external specification Clustering is a grouping of objects based on the data Supervised classification Have class label information Association Analysis Local vs. global connections 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Notion of a Cluster can be Ambiguous
How many clusters? Four Clusters
Two Clusters
Six Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusterings A clustering is a set of clusters Important distinction between hierarchical and partitional sets of clusters Partitional Clustering A division of data objects into non-overlapping subsets (clusters) such that each data object is in exactly one subset Hierarchical clustering A set of nested clusters organized as a hierarchical tree 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Partitional Clustering
Original Points A Partitional Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Traditional Hierarchical Clustering Non-traditional Hierarchical Clustering Non-traditional Dendrogram Traditional Dendrogram 02/14/2018 Introduction to Data Mining, 2nd Edition
‹#› Types of Clusters Well-separated clusters Center-based clusters Contiguous clusters Density-based clusters Property or Conceptual Described by an Objective Function 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Well-Separated Well-Separated Clusters: A cluster is a set of points such that any point in a cluster is closer (or more similar) to every other point in the cluster than to any point not in the cluster. 3 well-separated clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Types of Clusters: Center-Based Center-based A cluster is a set of objects such that an object in a cluster is closer to the “center” of a cluster, than to the center of any other cluster The center of a cluster is a centroid. The average of all the points in the cluster is a medoid 4 center-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Contiguity-Based Contiguous Cluster (Nearest neighbor or Transitive) A point in a cluster is closer (or more similar) to one or more other points in the cluster than to any point not in the cluster. 8 contiguous clusters
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Density-Based Density-based A cluster is a dense region of points, which is separated by low- density regions, from other regions of high density. Used when the clusters are irregular or intertwined, and when noise and outliers are present. 6 density-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Conceptual Clusters Shared Property or Conceptual Clusters Finds clusters that share some common property or represent a particular concept. . 2 Overlapping Circles
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Objective Function Clusters Defined by an Objective Function Finds clusters that minimize or maximize an objective function. Enumerate all possible ways of dividing the points into clusters and evaluate the `goodness' of each potential set of clusters by using the given objective function. (NP Hard) Can have global or local objectives. Hierarchical clustering algorithms typically have local objectives Partitional algorithms typically have global objectives A variation of the global objective function approach is to fit the data to a parameterized model. Parameters for the model are determined from the data. Mixture models assume that the data is a ‘mixture' of a number of statistical distributions. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Characteristics of the Input Data Are Important Type of proximity or density measure Central to clustering Depends on data and application Data characteristics that affect proximity and/or density are Dimensionality Sparseness Attribute type Special relationships in the data
For example, autocorrelation Distribution of the data Noise and Outliers Often interfere with the operation of the clustering algorithm 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Clustering Algorithms K-means and its variants Hierarchical clustering Density-based clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering Partitional clustering approach Number of clusters, K, must be specified Each cluster is associated with a centroid (center point) Each point is assigned to the cluster with the closest centroid The basic algorithm is very simple
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering – Details Initial centroids are often chosen randomly.
Clusters produced vary from one run to another. The centroid is (typically) the mean of the points in the cluster. ‘Closeness’ is measured by Euclidean distance, cosine similarity, correlation, etc. K-means will converge for common similarity measures mentioned above. Most of the convergence happens in the first few iterations. Often the stopping condition is changed to ‘Until relatively few points change clusters’ Complexity is O( n * K * I * d ) n = number of points, K = number of clusters, I = number of iterations, d = number of attributes 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Evaluating K-means Clusters Most common measure is Sum of Squared Error (SSE) For each point, the error is the distance to the nearest cluster To get SSE, we square these errors and sum them. x is a data point in cluster Ci and mi is the representative point for cluster Ci can show that mi corresponds to the center (mean) of the cluster Given two sets of clusters, we prefer the one with the smallest error One easy way to reduce SSE is to increase K, the number of clusters A good clustering with smaller K can have a lower SSE than a poor clustering with higher K
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Two different K-means Clusterings Sub-optimal Clustering Optimal Clustering Original Points 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means K-means has problems when clusters are of differing Sizes Densities Non-globular shapes K-means has problems when the data contains outliers. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Sizes
Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Density Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Non-globular Shapes Original Points K-means (2 Clusters)
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations Original PointsK-means Clusters One solution is to use many clusters. Find parts of clusters, but need to put together. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations
Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids 02/14/2018 Introduction to Data Mining, 2nd Edition
‹#› Importance of Choosing Initial Centroids … 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids … 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Problems with Selecting Initial Points If there are K ‘real’ clusters then the chance of selecting one centroid from each cluster is small. Chance is relatively small when K is large If clusters are the same size, n, then For example, if K = 10, then probability = 10!/1010 = 0.00036 Sometimes the initial centroids will readjust themselves in
‘right’ way, and sometimes they don’t Consider an example of five pairs of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Solution s to Initial Centroids Problem Multiple runs Helps, but probability is not on your side Sample and use hierarchical clustering to determine initial centroids Select more than k initial centroids and then select among these initial centroids Select most widely separated Generate a larger number of clusters and then perform a hierarchical clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram A tree like diagram that records the sequences of merges or splits
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Strengths of Hierarchical Clustering Do not have to assume any particular number of clusters Any desired number of clusters can be obtained by ‘cutting’ the dendrogram at the proper level They may correspond to meaningful taxonomies Example in biological sciences (e.g., animal kingdom) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Two main types of hierarchical clustering Agglomerative:
Start with the points as individual clusters At each step, merge the closest pair of clusters until only one cluster (or k clusters) left Divisive: Start with one, all-inclusive cluster At each step, split a cluster until each cluster contains an individual point (or there are k clusters) Traditional hierarchical algorithms use a similarity or distance matrix Merge or split one cluster at a time 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Agglomerative Clustering Algorithm Most popular hierarchical clustering technique Basic algorithm is straightforward Compute the proximity matrix Let each data point be a cluster
Repeat Merge the two closest clusters Update the proximity matrix Until only a single cluster remains Key operation is the computation of the proximity of two clusters Different approaches to defining the distance between clusters distinguish the different algorithms 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Starting Situation
Start with clusters of individual points and a proximity matrix
p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . . Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Intermediate Situation After some merging steps, we have some clusters C1 C4 C2 C5 C3 C2 C1 C1 C3 C5 C4 C2 C3
C4 C5 Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Intermediate Situation We want to merge the two closest clusters (C2 and C5) and update the proximity matrix.
C1 C4 C2 C5 C3 C2 C1 C1 C3 C5 C4 C2 C3 C4 C5
Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› After Merging The question is “How do we update the proximity matrix?”
C1 C4 C2 U C5 C3 ? ? ? ? ? ? ? C2 U C5 C1 C1 C3 C4 C2 U C5 C3 C4
Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Distance
p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . . Similarity? MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error
Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . .
Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . .
Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . .
Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . .
Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› MIN or Single Link Proximity of two clusters is based on the two closest points in the different clusters Determined by one pair of points, i.e., by one link in the proximity graph Example:
Distance Matrix: 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: MIN Nested Clusters Dendrogram 1 2 3 4 5 6 1
2 3 4 5 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Strength of MIN Original Points Six Clusters Can handle non-elliptical shapes 02/14/2018 Introduction to Data Mining, 2nd Edition
‹#› Limitations of MIN Original Points Two Clusters Sensitive to noise and outliers Three Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› MAX or Complete Linkage Proximity of two clusters is based on the two most distant points in the different clusters Determined by all pairs of points in the two clusters Distance Matrix:
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: MAX Nested Clusters Dendrogram 1 2 3 4 5 6 1 2
5 3 4 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Strength of MAX Original Points Two Clusters Less susceptible to noise and outliers 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of MAX
Original Points Two Clusters Tends to break large clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Group Average Proximity of two clusters is the average of pairwise proximity between points in the two clusters. Need to use average connectivity for scalability since total proximity favors large clusters Distance Matrix:
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Group Average Nested Clusters Dendrogram 1 2 3 4 5 6 1
2 5 3 4 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Group Average Compromise between Single and Complete Link Strengths Less susceptible to noise and outliers Limitations Biased towards globular clusters 02/14/2018 Introduction to Data Mining, 2nd Edition
‹#› Cluster Similarity: Ward’s Method Similarity of two clusters is based on the increase in squared error when two clusters are merged Similar to group average if distance between points is distance squared Less susceptible to noise and outliers Biased towards globular clusters Hierarchical analogue of K-means Can be used to initialize K-means 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Comparison Group Average Ward’s Method
1 2 3 4 5 6 1 2 5 3 4 MIN MAX
1 2 3 4 5 6 1 2 5 3 4
1 2 3 4 5 6 1 2 5 3 4 1 2 3
4 5 6 1 2 3 4 5 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Time and Space requirements O(N2) space since it uses the proximity matrix. N is the number of points. O(N3) time in many cases There are N steps and at each step the size, N2, proximity
matrix must be updated and searched Complexity can be reduced to O(N2 log(N) ) time with some cleverness 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Problems and Limitations Once a decision is made to combine two clusters, it cannot be undone No global objective function is directly minimized Different schemes have problems with one or more of the following: Sensitivity to noise and outliers Difficulty handling clusters of different sizes and non-globular shapes Breaking large clusters
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Cluster Validity For supervised classification we have a variety of measures to evaluate how good our model is Accuracy, precision, recall For cluster analysis, the analogous question is how to evaluate the “goodness” of the resulting clusters? But “clusters are in the eye of the beholder”! Then why do we want to evaluate them? To avoid finding patterns in noise To compare clustering algorithms To compare two sets of clusters To compare two clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Clusters found in Random Data Random Points K-means Complete Link 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Determining the clustering tendency of a set of data, i.e., distinguishing whether non-random structure actually exists in the data. Comparing the results of a cluster analysis to externally known results, e.g., to externally given class labels. Evaluating how well the results of a cluster analysis fit the data without reference to external information. - Use only the data Comparing the results of two different sets of cluster analyses to determine which is better. Determining the ‘correct’ number of clusters.
For 2, 3, and 4, we can further distinguish whether we want to evaluate the entire clustering or just individual clusters. Different Aspects of Cluster Validation 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Numerical measures that are applied to judge various aspects of cluster validity, are classified into the following three types. External Index: Used to measure the extent to which cluster labels match externally supplied class labels. Entropy Internal Index: Used to measure the goodness of a clustering structure without respect to external information. Sum of Squared Error (SSE) Relative Index: Used to compare two different clusterings or clusters. Often an external or internal index is used for this function, e.g., SSE or entropy Sometimes these are referred to as criteria instead of indices However, sometimes criterion is the general strategy and index
is the numerical measure that implements the criterion. Measures of Cluster Validity 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Two matrices Proximity Matrix Ideal Similarity Matrix One row and one column for each data point An entry is 1 if the associated pair of points belong to the same cluster An entry is 0 if the associated pair of points belongs to different clusters Compute the correlation between the two matrices Since the matrices are symmetric, only the correlation between n(n-1) / 2 entries needs to be calculated. High correlation indicates that points that belong to the same cluster are close to each other. Not a good measure for some density or contiguity based clusters. Measuring Cluster Validity Via Correlation
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Measuring Cluster Validity Via Correlation Correlation of ideal similarity and proximity matrices for the K- means clusterings of the following two data sets. Corr = -0.9235 Corr = -0.5810 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Order the similarity matrix with respect to cluster labels and inspect visually. Using Similarity Matrix for Cluster Validation
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Using Similarity Matrix for Cluster Validation Clusters in random data are not so crisp K-means 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Using Similarity Matrix for Cluster Validation Clusters in random data are not so crisp Complete Link
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Clusters in more complicated figures aren’t well separated Internal Index: Used to measure the goodness of a clustering structure without respect to external information SSE SSE is good for comparing two clusterings or two clusters (average SSE). Can also be used to estimate the number of clusters Internal Measures: SSE 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Internal Measures: SSE SSE curve for a more complicated data set
SSE of clusters found using K-means 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Correlation of ideal similarity and proximity matrices for the K- means clusterings of the following two data sets. Statistical Framework for Correlation Corr = -0.9235 Corr = -0.5810 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Cluster Cohesion: Measures how closely related are objects in a
cluster Example: SSE Cluster Separation: Measure how distinct or well-separated a cluster is from other clusters Example: Squared Error Cohesion is measured by the within cluster sum of squares (SSE) Separation is measured by the between cluster sum of squares Where |Ci| is the size of cluster i Internal Measures: Cohesion and Separation 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Internal Measures: Cohesion and Separation Example: SSE BSS + WSS = constant 1 2 3 4 5 m1 m2 m
K=2 clusters: K=1 cluster: 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› A proximity graph based approach can also be used for cohesion and separation. Cluster cohesion is the sum of the weight of all links within a cluster. Cluster separation is the sum of the weights between nodes in the cluster and nodes outside the cluster. Internal Measures: Cohesion and Separation
cohesion separation 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› “The validation of clustering structures is the most difficult and frustrating part of cluster analysis. Without a strong effort in this direction, cluster analysis will remain a black art accessible only to those true believers who have experience and great courage.” Algorithms for Clustering Data, Jain and Dubes Final Comment on Cluster Validity
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Discovered Clusters Industry Group 1 Applied-Matl-DOWN,Bay-Network-Down,3-COM-DOWN, Cabletron-Sys-DOWN,CISCO-DOWN,HP-DOWN, DSC-Comm-DOWN,INTEL-DOWN,LSI-Logic-DOWN, Micron-Tech-DOWN,Texas-Inst-Down,Tellabs-Inc-Down, Natl-Semiconduct-DOWN,Oracl-DOWN,SGI-DOWN, Sun-DOWN Technology1-DOWN 2 Apple-Comp-DOWN,Autodesk-DOWN,DEC-DOWN, ADV-Micro-Device-DOWN,Andrew-Corp-DOWN, Computer-Assoc-DOWN,Circuit-City-DOWN, Compaq-DOWN, EMC-Corp-DOWN, Gen-Inst-DOWN, Motorola-DOWN,Microsoft-DOWN,Scientific-Atl-DOWN Technology2-DOWN 3 Fannie-Mae-DOWN,Fed-Home-Loan-DOWN,
MBNA-Corp-DOWN,Morgan-Stanley-DOWN Financial-DOWN 4 Baker-Hughes-UP,Dresser-Inds-UP,Halliburton-HLD-UP, Louisiana-Land-UP,Phillips-Petro-UP,Unocal-UP, Schlumberger-UP Oil-UP Discovered Clusters Industry Group 1 Applied-Matl-DOWN,Bay-Network-Down,3-COM-DOWN, Cabletron-Sys-DOWN,CISCO-DOWN,HP-DOWN, DSC-Comm-DOWN,INTEL-DOWN,LSI-Logic-DOWN, Micron-Tech-DOWN,Texas-Inst-Down,Tellabs-Inc-Down, Natl-Semiconduct-DOWN,Oracl-DOWN,SGI-DOWN, Sun-DOWNTechnology1-DOWN
2 Apple-Comp-DOWN,Autodesk-DOWN,DEC-DOWN, ADV-Micro-Device-DOWN,Andrew-Corp-DOWN, Computer-Assoc-DOWN,Circuit-City-DOWN, Compaq-DOWN, EMC-Corp-DOWN, Gen-Inst-DOWN, Motorola-DOWN,Microsoft-DOWN,Scientific-Atl-DOWN Technology2-DOWN 3 Fannie-Mae-DOWN,Fed-Home-Loan-DOWN, MBNA-Corp-DOWN,Morgan-Stanley-DOWN Financial-DOWN 4 Baker-Hughes-UP,Dresser-Inds-UP,Halliburton-HLD-UP, Louisiana-Land-UP,Phillips-Petro-UP,Unocal-UP, Schlumberger-UP Oil-UP p4 p1
p3 p 2 p4 p1 p3 p 2 p4 p1 p2 p3 p4 p1 p2 p3 -2 -1.5 -1 -0.5
0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3 x y Iteration 1 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0
0.5 1 1.5 2 2.5 3 x y Iteration 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3
x y Iteration 3 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3 x y Iteration 4 -2 -1.5 -1
-0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3 x y Iteration 5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
0 0.5 1 1.5 2 2.5 3 x y Iteration 6 å å = Î = K i C x i i x m dist SSE
1 2 ) , ( -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3 x y -2 -1.5
-1 -0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3 x y -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
0 0.5 1 1.5 2 2.5 3 x y -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3
x y Iteration 1 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3 x y Iteration 2 -2 -1.5 -1
-0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5 3 x y Iteration 3 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
0 0.5 1 1.5 2 2.5 3 x y Iteration 4 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 0.5 1 1.5 2 2.5
3 x y Iteration 5 0 5 10 15 20 -6 -4 -2 0 2 4 6 8 x y Iteration 1 0 5 10 15 20
-6 -4 -2 0 2 4 6 8 x y Iteration 2 0 5 10 15 20 -6 -4 -2 0 2 4 6 8 x
y Iteration 3 0 5 10 15 20 -6 -4 -2 0 2 4 6 8 x y Iteration 4 1 2 3 4 5 6 1
2 3 4 5 1 3 2 5 4 6 0 0.05 0.1 0.15 0.2 ... p1p2p3p4p9p10p11p12 ...� p1� p2� p3� p4� p9� p10� p11�
p12� ... p1p2p3p4p9p10p11p12 ...� p1� p2� p3� p4� p9� p10� p11� p12� ... p1p2p3p4p9p10p11p12 ...� p1� p2� p3� p4� p9� p10� p11� p12� ... p1p2p3p4p9p10p11p12
...� p1� p2� p3� p4� p9� p10� p11� p12� 3 6 2 5 4 1 0 0.05 0.1 0.15 0.2 3 6 4 1 2
5 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 | |Cluster | |Cluster ) p , p proximity( ) Cluster , Cluster proximity( j
i Cluster p Cluster p j i j i j j i i ´ = å Î Î 3 6 4 1 2 5 0
0.05 0.1 0.15 0.2 0.25 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x y 0
0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x y 0 0.2 0.4 0.6 0.8 1 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x y 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6
0.7 0.8 0.9 1 x y Points Points 20 40 60 80 100 10 20 30 40 50 60 70 80 90 100 Similarity 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Points Points 20 40 60 80 100 10 20 30 40 50 60 70 80
90 100 Similarity 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2 5 10 15 20 25 30 0 1 2 3
4 5 6 7 8 9 10 K SSE 5 10 15 -6 -4 -2 0 2 4 6 1 2 3 5 6 4
7 å å Î - = = i C x i i m x WSS SSE 2 ) ( å - = i i i
m m C BSS 2 ) ( 10 9 1 9 ) 3 5 . 4 ( 2 ) 5 . 1 3 ( 2
1 ) 5 . 4 5 ( ) 5 . 4 4 ( ) 5 . 1 2 ( ) 5 . 1 1 (
2 2 2 2 2 2 = + = = - ´ + - ´ = = - + - + - + - =
= Total BSS WSS SSE 10 0 10 0 ) 3 3 ( 4 10 ) 3 5 ( ) 3 4 ( ) 3
2 ( ) 3 1 ( 2 2 2 2 2 = + = = - ´ = = - + - + - +
- = = Total BSS WSS SSE JOURNAL OF APPLIED BEHAVIOR ANALYSIS WHAT MAKES EXTINCTION WORK: AN ANALYSIS OF PROCEDURAL FORM AND FUNCTION BRIAN A. IWATA THE UNIVERSITY OF FLORIDA GARY M. PACE THE KENNEDY INSTITUTE, BALTIMORE, MARYLAND GLYNNIS EDWARDS COWDERY WE CARE, INC., BERKELEY, CALIFORNIA
AND RAYMOND G. MILTENBERGER NORTH DAKOTA STATE UNIVERSITY We examined methods for determining how extinction should be applied to different functions of self-injurious behavior (SIB). Assessment data indicated that the head banging of 3 children with developmental disabilities was maintained by different reinforcement contingencies: One subject's SIB was positively reinforced by attention from adults, the 2nd subject's SIB was negatively reinforced by escape from educational tasks, and the 3rd subject's SIB appeared to be automatically reinforced or "self-stimulatory" in nature. Three functional variations of extinction-EXT (attention), EXT (escape), and EXT (sensory)-were evaluated, and each subject was exposed to at least two of these variations in reversal or multiple baseline designs. Reductions in SIB were observed only when implementation of "extinction" involved the discontinuation of reinforcement previously shown to be responsible for maintaining the behavior. These results highlight important differences among
treatment techniques based on the same behavioral principle (extinction) when applied to topo- graphically similar but functionally dissimilar responses, and further illustrate the practical impli- cations of a functional analysis of behavior disorders for designing, selecting, and classifying ther- apeutic interventions. DESCRIPTORS: extinction, functional analysis, self-injurious behavior Research on the functional analysis of severe behavior disorders has produced a variety of inter- ventions based on the modification of antecedent events that occasion behavior problems as well as the consequences that maintain them (see Iwata, Vollmer, & Zarcone, 1990, and Mace, Lalli, & Lalli, 1991, for recent reviews). Much of the em- phasis in treatment has been on strengthening new stimulus-response-consequence relationships. By contrast, relatively little attention has been paid to eliminating reinforcement that maintained the be- This research was supported in part by a grant from the Developmental Disabilities Planning Council and the Florida Department of Health and Rehabilitative Services.
Reprints may be obtained from Brian Iwata, Psychology Department, University of Florida, Gainesville, Florida 3261 1. havior problem in the first place, even though many of the interventions described in the literature im- plicitly contained such a provision. That is, pun- ishment of Response A is almost always combined with the cessation of reinforcement; likewise, treat- ment involving reinforcement of Response B as a replacement for Response A usually coincides with the termination of reinforcement for Response A. Moreover, data from several recent studies suggest that the effects of reinforcement-based interventions may be limited unless extinction (withholding the behavior's maintaining reinforcers) is included as part of the treatment program. It has been shown, for example, that interventions based on the de- velopment of communicative responses (Carr & Durand, 1985) and on behavioral momentum (Mace & Belfiore, 1990) produced either mixed 131 1994127,131-144 NUMBER I (SPRING 1994)
BRIAN A. IWATA et al. results (Wacker et al., 1990) or no treatment effect (Zarcone, Iwata, Hughes, & Vollmer, 1993) when implemented without extinction, and that the crit- ical feature of procedures such as differential rein- forcement of other behavior (DRO) contingencies appears to be the extinction component rather than the reinforcement component (Mazaleski, Iwata, Vollmer, Zarcone, & Smith, 1993). The assessment and treatment of self-injurious behavior (SIB) provide an important context for the examination of extinction, because it has been demonstrated that SIB can be maintained through a variety of operant mechanisms. Nevertheless, be- cause the "discontinuation of reinforcement" for behavior problems in general often involves ces- sation of ongoing events, this procedure has become so common that some textbooks define extinction solely through reference to "ignoring," time-out, and other examples of stimulus termination (e.g.,
LaVigna & Donnellan, 1986). However, research has shown that procedural time-out can serve dif- ferent functions, not all of which are behavior re- ducing (Plummer, Baer, & LeBlanc, 1977; Solnick, Rincover, & Peterson, 1977), and the same is true of procedural approaches to defining extinction. Ignoring misbehavior may represent extinction in some cases but not in others; conversely, the correct application of extinction may require termination of events in some cases but continuation in others. The procedures that define extinction in a given situation are determined by the specific nature of the reinforcement to be "discontinued." In cases in which SIB was maintained by social- positive reinforcement in the form of adult atten- tion, extinction consisted of withholding attention or terminating it contingent on the occurrence of SIB (e.g., Day, Rea, Schussler, Larsen, & Johnson, 1988; Lovaas & Simmons, 1969). By contrast, SIB maintained through social-negative reinforcement in the form of escape from task demands has been extinguished by preventing escape; in other words, continuing and not terminating the ongoing sit- uation (e.g., Iwata, Pace, Kalsher, Cowdery, &
Cataldo, 1990; Repp, Felce, & Barton, 1988). Finally, SIB apparently maintained by nonsocial, automatic reinforcement (e.g., sensory stimulation) has shown extinction-like decreases when the in- dividuals wore equipment that allowed the behav- ior to occur, but attenuated its consequences (e.g., Rincover & Devany, 1982). Thus, at least three functional variations of extinction have been used as treatment for SIB, each designed to terminate a different source of reinforcement, and each ame- nable to a number of procedural modifications and descriptive labels. Although extinction procedures are clearly unique from the standpoint of both form (specific therapist actions) and function (the maintaining contingen- cies to which they apply), these variations are not well differentiated through existing terminology. Planned ignoring, the label perhaps most often applied to extinction of attention-maintained be- havior (Nelson & Rutherford, 1983), provides an adequate description of the therapist's response, but it does not describe the underlying behavioral pro- cess (extinction), nor does it identify the source of
reinforcement being withheld (attention). Escape extinction, a term used to describe extinction of escape-maintained behavior (Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990), specifies the relevant process as well as the reinforcer, but not the pro- cedure. Finally, sensory extinction, which refers to a variety of techniques designed to attenuate stim- ulation directly produced by a response (Rincover, 1978; Rincover, Cook, Peoples, & Packard, 1979), has the same descriptive characteristics as escape extinction (it specifies process and reinforcer, but not procedure); in addition, the wording is awk- ward because nothing "sensory" is extinguished. In an attempt to promote brevity as well as con- sistency of terminology, we have adopted through- out this paper the convention of referring to dif- ferentfunctional variations of extinction by using the abbreviation "EXT" followed in parentheses by the source of reinforcement withheld, as in EXT (attention), EXT (escape), and EXT (sensory). We chose "EXT" (attention) over "attention EXT" because the former places emphasis on behavioral process, with the reinforcer incidental, whereas the latter connotes that attention is extinguished.
This terminology is still incomplete because it does not differentiate procedural variations within 132 VARIATIONS OF EXTINCTION a given functional class of extinction techniques. For example, EXT (attention) may take different forms depending on the situational context in which it is applied. If the target behavior occurs in the absence of ongoing social interaction, EXT (atten- tion) requires no response from the therapist and simply consists of withholding attention that pre- viously followed the behavior. A more complicated situation exists if the target behavior occurs while the therapist is interacting with the client. In this case, withholding reinforcement [EXT (attention)] would consist of terminating the interaction.' Be- cause similar variations are also characteristic of both EXT (escape) and EXT (sensory), we did not attempt to specify any particular procedure by way of descriptive label; this practice is consistent with
terms such as DRO, time-out, punishment, and so forth, all of which require further elaboration. A reasonable conclusion based on descriptions of behavior, procedure, and outcome reported in the above studies is that extinction designed for SIB serving one function may have little or no therapeutic effect on SIB serving a different func- tion. Given the variations of extinction described previously and the maintaining contingencies for which they might be used, it is possible to construct a contingency matrix such as that found in Figure 1, which allows predictions about behavioral out- come. For example, elimination of social reinforce- ment for SIB using EXT (attention) or EXT (es- cape) will not interfere with the delivery of sensory (automatic) reinforcement, and EXT (sensory) would have no effect on social sources of reinforce- ment. Most seriously, EXT (attention) applied to escape-maintained SIB (i.e., stimulus removal con- tingent on escape behavior) and, conversely, EXT (escape) applied to attention-maintained SIB (i.e., stimulus continuation contingent on attention-get- ting behavior) may be countertherapeutic and se- riously exacerbate the behavior problem. At the
In this example, it is unclear if the subsequent change in behavior reflects withholding of contingent reinforcement that maintained SIB (extinction) or removal of ongoing re- inforcement (time-out). In either case, and from a practical standpoint, the only means of not reinforcing the behavior consists of terminating interaction. FUNCTIONAL VARIATIONS OF EXTINCTION 0 LL 0 z LU 0 z 1 z 0 0 z Ir.2 f ifis_---_..Treatment
Figure 1. Expected outcomes when functional variations of extinction are applied to different maintaining contingen- cies for SIB. present time, evidence directly supporting these pre- dictions can be found in only a few studies. Research on the treatment of self-injurious escape behavior, for example, sometimes has included a baseline condition in which time-out from educational tasks was made contingent on SIB (Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990; Repp et al., 1988; Steege et al., 1990). In effect, this condition pro- cedurally amounted to EXT (attention) but was associated with increases in escape-maintained SIB. In the present study, we provide an empirical evaluation of the key predictions found in Figure 1 by conducting a comparative analysis of extinc- tion techniques. The relative effects of three func- tional variations of extinction were examined when applied to the same topography of SIB maintained by three different contingencies of reinforcement. METHOD
Subjects and Setting Subjects were selected for participation based on (a) their having similar topographies of SIB, and (b) the outcomes of functional analysis baselines indicating that their SIB was maintained by dif- ferent sources of reinforcement. Three children with developmental disabilities who met these criteria were included as participants. All of them exhibited head banging that produced contusions or lacera- tions on the head and face. Donnie, a 7-year-old boy with severe mental retardation, was ambulatory 133 BRIAN A. IWATA et al. and could feed himself. He did not follow vocal instructions, although he had a fairly good imitative repertoire, and he communicated his needs by pointing. Donnie's parents could not identify any situations that were predictive of either high or low
levels of SIB. His previous treatments consisted of response interruption, redirection to another activ- ity, time-out, and differential reinforcement. Jack, a 12-year-old boy with severe mental retardation, was nonambulatory but could move his wheelchair over short distances, feed himself, and manipulate small objects. He followed a few instructions and communicated primarily through idiosyncratic ges- ture (i.e., reaching and making a high-pitched vocal noise). Jack's parents and teachers reported that his SIB seemed to occur as part of a tantrum, although they could not reliably describe the types of situ- ations in which tantrums occurred. His previous treatments consisted of time-out and differential reinforcement. Millie, an 8-year-old girl with mod- erate mental retardation, was ambulatory, could feed herself, and occasionally followed simple in- structions. Although she did not exhibit any ex- pressive language, she appeared to be socially re- sponsive to adults (e.g., she approached adults when they were nearby, smiled at them, etc.). Millie's parents indicated that her SIB occurred "most of the time," but became worse when she did not "get her way." Her previous treatments consisted of verbal reprimands, response interruption, man-
ual restraint, redirection, and differential reinforce- ment. She was also placed in a hard seizure helmet as a means of protection when her SIB was deemed to be "uncontrollable." Sessions were usually conducted individually in therapy rooms containing one-way observation win- dows, but occasionally (due to scheduling neces- sities) were run in a large group area where two or three other clients and one or two other staff mem- bers were present but located in a different part of the room (i.e., at least 6 m away). This periodic change in location seemed to have no effect on the subjects' behavior. Sessions were 15 min in duration and usually were conducted 5 days per week, with four to eight sessions daily and at least 15-min breaks between sessions. Response Measurement and Reliability SIB was defined as any audible contact between the head or face and either a fist or an object (e.g., furniture or walls). Data were recorded using one of two methods: (a) on paper during continuous 10-s intervals, which were cued by cassette tape;
or (b) on a hand-held computer (Panasonic Model RL-H1800). All data were converted into per- centage of 10-s intervals during which one or more instances of SIB occurred. Data were also collected on a variety of experimenter behaviors (delivery of instructions or attention, withdrawal of instructions or attention, placement and removal of apparatus) as a means of monitoring procedural consistency; these data indicated that experimenter compliance exceeded 90% across all categories. Interobserver agreement was assessed by having a second observer simultaneously but independently record data during 23% of all sessions. Agreement percentages were calculated based on interval-by- interval comparison of observers' records by divid- ing the number of agreements by the number of agreements plus disagreements and multiplying by 100%. Mean agreement was 92% across subjects, with all values exceeding 86%. Experimental Designs and Sequence During the initial baseline, subjects were exposed to a series of assessment conditions in a multiele-
ment design (Sidman, 1960; Ulman & Sulzer-Aza- roff, 1975) to identify the maintaining variables for their SIB. Subsequently, each subject was ex- posed to two or more functional variations of ex- tinction by way of reversal or multiple baseline designs. Because the specific conditions and their order of presentation were determined by subjects' baseline performances, more complete details are provided in the Results section for each subject. Functional Analysis Baseline Subjects were exposed to four conditions based on Iwata, Dorsey, Slifer, Bauman, and Richman (1982). A brief description of each condition is provided here. Attention. This condition was designed to test 134 VARIATIONS OF EXTINCTION
for positive reinforcement (attention) as a main- taining variable for SIB. An experimenter initially placed the subject in contact with toys that were in the room, then proceeded to ignore the subject while seeming to do paperwork. Contingent on the occurrence of SIB, the experimenter expressed con- cern (e.g., "Stop. Don't do that; you'll hurt your- self.") and briefly interrupted the response. Demand. This condition was a test for negative reinforcement (escape from demands) as a main- taining variable. An experimenter presented aca- demic tasks (e.g., object identification, sorting, drawing, and other tasks requiring simple motor responses) comparable to those found in the sub- ject's individual educational plan. Tasks were pre- sented in a discrete-trial format approximately once every 30 s. The experimenter used modeling and physical guidance as supplementary prompts to produce compliance, delivered praise contingent on correct responses, and implemented a 30-s time- out contingent on the occurrence of SIB. The time- out consisted of removing the materials from the table and turning away from the subject.
Alone. By restricting access to social and material sources of stimulation, this condition provided a test for SIB maintained by nonsocial (automatic) reinforcement. The subject was observed while alone in the room (which was empty except for a chair or couch). Play. This condition served as a control. The experimenter provided continuous access to toys, delivered praise approximately every 30 s contin- gent on the absence of SIB, and ignored occurrences of SIB. Extinction Conditions Three variations of extinction were evaluated, each designed to discontinue a different source of reinforcement for SIB. An attempt was made to implement each type of extinction with each sub- ject, but this was not always possible if (a) little or no SIB occurred in a given baseline condition, or (b) the baseline condition itself was not amenable to implementation of the procedure as described. For example, it was not possible to prevent escape from tasks (i.e., to withhold negative reinforce-
ment) during the alone baseline, because no tasks were presented initially. EXT (attention). This variation of extinction was designed to discontinue positive reinforcement in the form of attention delivered by the experi- menter. EXT (attention) already was a component of three of the baseline conditions: during the de- mand condition, attention was withdrawn contin- gent on SIB; during the alone condition, no adult was present to deliver any attention; and during the play condition, attention was withheld contin- gent on SIB. Thus, the attention condition was the only baseline on which EXT (attention) could be introduced as a new manipulation. This extinction contingency consisted of having the experimenter ignore (not attend to) SIB, and was combined with experimenter attention for the absence of SIB, which was delivered on a 30-s DRO schedule. Occur- rences of SIB reset the DRO interval. EXT (escape). This variation of extinction dis- continued negative reinforcement in the form of escape from tasks. It was implemented during the
demand condition, which was the only condition containing a task requirement. During EXT (es- cape), learning trials continued as during baseline but were not terminated contingent on SIB. Instead, the experimenter physically guided the subject to comply with the instruction and continued the ses- sion according to schedule. It has been noted pre- viously (Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990) that the physical guidance component of this intervention may represent a punishment con- tingency. However, (a) guidance was the inevitable consequence of noncompliance even during base- line, and (b) behavior change observed following implementation of this procedure in the above study as well as in others (e.g., Goh & Iwata, 1994; Repp et al., 1988) showed either temporary burst- ing or gradual decreases consistent with extinction processes. EXT (sensory). This variation of extinction was designed to attenuate the sensory consequences pro- duced by subjects' head banging, and could be implemented across any of the baseline conditions. Each subject was fitted with an oversized helmet (either a seizure or a lacrosse helmet), which was
135 BRIAN A. IWATA et al. reinforced with additional padding. During EXT (sensory), the helmet was placed on the subject's head at the beginning of a session and remained in place throughout (subjects never attempted to remove the helmets). An extension of this "non- contingent" helmet condition was implemented for Donnie and is described below. RESULTS Donnie Figure 2 shows the results obtained for Donnie. He exhibited a considerable amount of SIB across all of the baseline conditions, thus permitting a series of manipulations on each baseline. Moderate levels of SIB were observed during the demand, attention, and play conditions (lower three panels),
whereas SIB occurred almost continuously during the alone condition (upper panel) containing no access to social stimulation, play materials, or tasks. These data suggested that Donnie's SIB was main- tained by factors independent of the social and physical environment and were consistent with be- havior reinforced by directly produced, automatic consequences. On the alone baseline (top panel), EXT (sensory) was implemented in a reversal (ABAB) design, and results showed decreases in SIB associated with the helmet intervention. The final condition on the alone baseline involved two modifications of the EXT (sensory) procedure. First, play materials that Donnie was observed to manipulate during the original play baseline were introduced. Second, the protective helmet was applied only contingent on the occurrence of SIB. As long as SIB did not occur, Donnie was free to play with the toys and move about without wearing the helmet. When SIB oc- curred, an experimenter, who was in the room but did not interact with Donnie during the session, placed the helmet on Donnie for a 2-min time- out, during which the toys were removed from
sight. These procedural modifications were based on results reported by Dorsey, Iwata, Reid, and Davis (1982), who found that it was possible to transfer control of SIB from a noncontingent to a contingent helmet condition. SIB increased initially when this change was made on the alone baseline, but eventually decreased to a level similar to that observed during the EXT (sensory) conditions. On the demand baseline (second panel from the top), the introduction of EXT (sensory) was asso- ciated with a rapid and large decrease in Donnie's SIB. This reduction occurred in spite of the fact that the original baseline contingency for SIB- EXT (attention) or, alternatively, escape from the task contingent on SIB-remained in effect. Next, EXT (sensory) was removed and EXT (escape) was implemented; this resulted in an increase in SIB to its baseline level. Finally, the contingent helmet procedure was combined with EXT (escape). The experimenter presented learning trials as before and, contingent on SIB, guided Donnie through the task and then placed the helmet on him. The helmet remained on for approximately 2 min (four learning
trials). This condition was associated with a decrease in SIB similar to that seen during the previous EXT (sensory) condition. On the attention baseline (third panel from the top), the introduction of EXT (attention) had no effect on Donnie's SIB. During the next condition, EXT (sensory), SIB decreased even though it pro- duced attention from the experimenter. The final procedure implemented on this baseline consisted of EXT (attention) combined with a 2-min time- out in the helmet contingent on occurrences of SIB, and was associated with a reduction in SIB similar to that seen in the EXT (sensory) condition. On the play baseline (bottom panel), which con- tained an EXT (attention) component, no decrease in SIB was observed until EXT (sensory) was in- troduced. Subsequently, SIB increased temporarily but decreased again when the procedure was changed to a 2-min time-out in the helmet contingent on SIB. Jack The initial baseline of Figure 3 shows the results
of Jack's assessment. He exhibited little or no SIB except during the demand condition, which con- tained two distinctive features: the presentation of task demands and a brief time-out from the task contingent on SIB. The time-out component pro- 136 DONNIE 100- Baseline 800 n:n -a 40 - 20 - VARIATIONS OF EXTINCTION
r EXT(sensory) 1 BL I Toys + Conting( -a- Ati -a-- -D -.*-- Al4 -a- Pli ent Helmet ttention emand lone lay 1 ---- -i- 40 EXT Contingent Helmet Baseline EXT(sensory) (escape) + EXT(escape) 20 o r , . . 40 Contingent Helmet
Baseline EXT(attn) EXT(sensory) + EXT(attn) 20 40 Contingent Baseline EXT(sensory) Helmet 20 0 20 40 60 80 100 137 co) -J cn w H z uL
0 Lu w z w 0L 20 40 60 80 1 00 SESSIONS Figure 2. Percentage of intervals of SIB exhibited by Donnie across experimental conditions. ou -1 A II
BRIAN A. IWATA et al. EXT(sensory) 100 co C,) -J Er w z LL 0 LLJ z LU cc
LLJw- 80 60 40 20 0 -H0. SESSIONS Figure 3. Percentage of intervals of SIB exhibited by Jack across experimental conditions. cedurally resembled EXT (attention) but appeared to function as (negative) reinforcement for escape behavior. Because little or no SIB was observed during other conditions, all subsequent manipu- lations were made on the demand baseline. EXT (sensory) and EXT (escape) were implemented con-
currently and resulted in a large decrease in Jack's SIB. A partial reversal was then conducted in which EXT (escape) was removed while EXT (sensory) remained in effect; this change resulted in an in- crease in Jack's SIB to its original baseline level. SIB decreased again when EXT (escape) was re- introduced, and it remained low following the re- moval of EXT (sensory). Millie Figure 4 shows the results obtained for Millie. During the initial baseline assessment, her SIB oc- curred almost exclusively during the attention con- dition, suggesting that the behavior was maintained by social-positive reinforcement. Very little SIB was observed during the play and demand conditions, both of which contained a brief time-out from attention contingent on SIB, and SIB decreased throughout the alone condition, which contained no attention whatsoever. Thus, Millie's assessment data revealed not only a reinforcement effect during the attention condition but also an apparent ex- tinction effect during others.
In light of these results, subsequent interventions were implemented only on the attention baseline. EXT (attention) with attention delivered a 30-s DRO schedule was introduced concurrent with EXT (sensory) and resulted in a large decrease in Millie's SIB. Next, while EXT (sensory) remained in effect, a reversal was conducted in which EXT (attention) + DRO was removed; this partial return to base- line was associated with an immediate and large increase in SIB. When EXT (attention) + DRO was reinstated, SIB decreased to near-zero levels. EXT (sensory) was then removed while EXT (at- tention) + DRO remained in effect, and no increase was observed in SIB. During this final condition, the length of the DRO interval was gradually in- creased to 1, 2, and finally 5 min, and SIB remained low across all schedule changes. 138 VARIATIONS OF EXTINCTION
EXT(sensory) _,4,0MILLIE EXT(attn) + DRO Baseline BL -a Attention *- Demand -0- Alone 0 Play 20 40 60 80 SESSIONS Figure 4. Percentage of intervals of SIB exhibited by Millie across experimental conditions. Summary of Key Comparisons Table 1 contains a summary of the results from selected baseline and extinction conditions for Don- nie, Jack, and Millie. The data reflect reductions or increases in SIB expressed as mean percentages of baseline responding (EXT mean/baseline mean) for relevant and irrelevant extinction conditions
compared to their appropriate baselines. Because SIB during relevant extinction conditions usually showed decreasing trends, comparisons based on overall condition means would not accurately depict end-of-treatment responding. Therefore, all per- centages are based on the last five sessions of a condition. For example, under "Relevant EXT," Donnie's mean percentage of SIB during the last five sessions of EXT (sensory) occurred at 0.5% of the mean SIB during the last five sessions of his alone baseline, whereas under "Irrelevant EXT," his five-session mean percentage of SIB during EXT (escape) occurred at 81.9% of his demand baseline mean. These comparisons show that SIB for all subjects was reduced to below 15% of its baseline Table 1 Mean Change (Reduction or Increase) in SIB Expressed as a Percentage of Baseline Responding for Relevant and Irrelevant Extinction Conditions Compared to Their Relevant Baselines. Percentages Are Based on the Last Five Sessions of a Condition
Relevant EXT / baseline Irrelevant EXT / baseline Subject Conditions compared % of BL Conditions compared % of BL Donnie EXT (sensory) / alone BL 0.5 EXT (escape) / demand BL 81.9 EXT (sensory) / demand BL 6.9 EXT (attn) / attn BLa 81.7 EXT (sensory) / attn BLa 14.3 EXT (sensory) / play BL 8.2 Jack EXT (escape) / demand BL 5.4 EXT (sensory) / demand BL 75.8 Millie EXT (attn) / attn BL 3.6 EXT (sensory) / attn BL 126.3 Mean change as % of baseline SIB 6.5 91.4 a Donnie's attention baseline consisted of only four sessions. 139 co CO C,) -J
cc z LL 0 w C!, z uLJ co wa. wJ 80 - 60 - 40 20 0- 1 00
BRIAN A. IWATA et al. mean during relevant extinction conditions (mean reduction = 6.5% of baseline), but was never re- duced to lower than 75% of baseline during irrel- evant extinction conditions (mean reduction = 91.4% of baseline). Follow-Up At the completion of the study, a treatment program designed for each subject was imple- mented throughout the day in the appropriate sit- uational contexts. Donnie's program consisted of several components. First, free access to toys was provided whenever possible (except during training sessions, self-care routines, etc.). Second, occur- rences of SIB were followed by wearing his padded helmet for 2 min. Finally, and as a preventive measure, the contingent helmet procedure was com- bined with either EXT (attention) or EXT (escape) depending on the situational context. If Donnie
exhibited SIB while participating in training, the session was continued while he wore the helmet. If SIB occurred at other times, the helmet was com- bined with time-out from social interaction. Jack's treatment consisted of EXT (escape) identical to that described previously, and it was implemented during all training activities. Millie's treatment con- sisted of attention contingent on appropriate social initiations, defined as any approach behavior while not engaged in SIB. A 5-min DRO contingency also was implemented with attention as the rein- forcing consequence. If Millie exhibited SIB while an adult was interacting with her, the adult would turn away and ignore her until (a) Millie ap- proached while not engaging in SIB, or (b) she successfully completed a 5-min DRO interval. Thus, Millie's program contained an explicit EXT (at- tention) component. Subjects' parents and teachers were instructed in the use of the above procedures until they dem- onstrated proficiency in implementation. Follow- up contacts were faded over a 6-month period, during which SIB occurred at well below baseline levels for all subjects.
DISCUSSION Results of this study showed that therapeutic techniques previously defined as extinction in the applied literature are different not only procedurally but also in their behavioral effects when applied to the same response topography. Each of three vari- ations of extinction-EXT (attention), EXT (es- cape), and EXT (sensory)-reduced SIB only when it resulted in discontinuation of the specific source of reinforcement maintaining the behavior. Thus, the process of extinction is not tied to any particular form of intervention (e.g., ignoring undesirable be- havior); instead, it is determined by the behavior's maintaining contingency. Moreover, the data pre- sented here indicate that if the source of reinforce- ment for a behavior disorder such as SIB can be identified on a pretreatment basis, much of the guesswork is removed from the task of selecting an appropriate extinction technique. Results obtained for Donnie provide the most complete analysis in the present study because he
was exposed to all three functional variations of extinction. Neither EXT (attention) nor EXT (es- cape) was effective in reducing Donnie's SIB. His head banging, which apparently was maintained by sensory consequences, was reduced across all baseline conditions when EXT (sensory) was ap- plied, even though SIB still produced attention and escape during the attention and demand baselines, respectively. Jack, for whom EXT (escape) and EXT (sensory) were evaluated explicitly as treat- ments, was also exposed implicitly to EXT (atten- tion). The brief time-out contingent on SIB during his initial demand baseline closely resembled EXT (attention), yet it actually amounted to negative reinforcement and resulted in the highest levels of SIB when compared to other assessment conditions. EXT (sensory) also had little suppressive effect on Jack's SIB, which decreased only when it no longer produced termination of learning trials during the EXT (escape) condition. For Millie, whose baseline data indicated that her SIB was maintained by positive reinforcement in the form of attention, effective treatment consisted of EXT (attention) combined with DRO (attending to her when she did not exhibit SIB), whereas EXT (sensory) had
no apparent effect on her behavior. Millie's results are limited in two respects. First, a stronger dem- onstration of extinction effects would have been provided had EXT (attention) not been combined 140 VARIATIONS OF EXTINCTION with the DRO procedure, although recent data (Mazaleski et al., 1993) suggest that extinction is the major therapeutic component of DRO contin- gencies. Second, because little or no SIB was ob- served during her demand baseline, it was not pos- sible to examine the effects of EXT (escape) on her behavior. It is important to note that the procedures used in this study, although labeled "EXT" for each subject, differed in two respects. First, because only one variation of extinction was effective in reducing SIB (a different one for each subject), the other two variations did not functionally amount to extinc-
tion. We labeled all procedures similarly to reduce confusion and to emphasize the facts that (a) each procedure has been described as an extinction tech- nique in the literature, and (b) procedures so labeled might not produce reductions in behavior if they do not terminate the behavior's maintaining con- tingency. Based on the results obtained during as- sessment and treatment, a more accurate description of the procedures used with each subject might be as follows. Donnie was exposed to EXT (sensory), ignoring SIB or terminating interaction contingent on SIB [labeled EXT (attention)], and task con- tinuation and prompting contingent on SIB [la- beled EXT (escape)). Jack was exposed to EXT (escape), a helmet condition [labeled EXT (sen- sory)1, and terminating interaction contingent on SIB (his demand baseline). Millie was exposed to EXT (attention) and a helmet condition [labeled EXT (sensory)}. A second difference can be seen within one of the extinction components. EXT (es- cape) and EXT (sensory) were applied in a consis- tent manner during treatment sessions. EXT (at- tention), however, took two different forms: ignoring SIB that occurred in the absence of interaction, and
terminating ongoing interaction contingent on the occurrence of SIB. These differences illustrate a point already noted: Extinction procedures can be defined with respect to both function (the rein- forcement contingency being discontinued) and, within function, form (specific therapist actions). The lack of therapeutic effects observed when a given variation of extinction was applied to SIB maintained by a different (irrelevant) source of re- inforcement provides new data on some of the limiting conditions of extinction. On the other hand, the positive treatment effects observed during rel- evant extinction conditions were not particularly novel. Previous studies have reported reductions in SIB when the reinforcer withheld during extinction was relevant to behavioral maintenance: EXT (at- tention) for attention-maintained SIB (e.g., Lovaas & Simmons, 1969), EXT (escape) for escape-main- tained SIB (e.g., Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990), and EXT (sensory) for auto- matically reinforced or self-stimulatory SIB (e.g., Rincover & Devaney, 1982). Nevertheless, these findings have not given rise to a consistent termi-
nology that distinguishes one type of extinction from another, they have not been adopted by au- thors of most textbooks on applied behavior anal- ysis (see Cooper, Heron, & Heward, 1987, for a notable exception), and they have not been incor- porated into guidelines regulating the use of be- havioral interventions (e.g., Florida HRS Manual 160-4, 1989). Extinction is similar to other behavioral pro- cesses, such as reinforcement or punishment, from which a number of therapeutic techniques can be derived. Although this fact applies to all behavioral interventions, the case of punishment is of greatest concern because some of its procedural variations are considered to be highly intrusive or otherwise unacceptable (O'Brien & Karsh, 1990). As a result, administrative and treatment manuals place a great deal of emphasis on the topographical aspects of intervention in order to promote procedural con- sistency by specifying the acceptable limits of ther- apist behavior when implementing a given tech- nique. Although this approach to treatment classification results in a well-defined system of pro- cedures, behavioral effects may be difficult to pre-
dict because therapist behavior, although clearly defined, may enter into multiple contingencies with respect to client behavior. Thus, "planned ignor- ing" functions as EXT (attention) for attention- seeking behavior but as negative reinforcement for escape behavior, whereas procedures such as "re- direction" or other attempts to prompt compliance with task demands function as EXT (escape) for escape behavior but as positive reinforcement for attention-seeking behavior. Functions other than 141 BRIAN A. IWATA et al. EXT (sensory) that might be attributed to the hel- met intervention used in this study are not entirely clear at the present time, but they might include punishment (wearing the apparatus as an aversive event), social reinforcement (attention paired with equipment placement), or time-out (overall re- duced access to reinforcement while wearing the equipment).
Problems may arise when any of these procedures serves unintended functions such as those just de- scribed. At best, these discrepancies may result in the use of ineffective treatments described as ex- tinction (e.g., as in ignoring automatically rein- forced behavior); at worst, they may result in the use of highly inappropriate treatments described as extinction (as in ignoring or time-out for escape behavior). In either case, failure to distinguish be- tween the principle of extinction and the ways in which it may be applied can lead to the erroneous conclusion that "extinction" is a relatively ineffec- tive intervention. When these errors are formalized by way of regulation, an unfortunate state of affairs may result: Procedural manuals designed to protect persons from intrusive interventions (e.g., punish- ment) to reduce behavior may require the use of nonintrusive interventions (e.g., misapplied ignor- ing or attention) that strengthen dangerous behav- ior to the point that the intrusive intervention is required. Although the results of this study are limited to consideration of extinction effects, they have im-
plications for the design of interventions based on other learning mechanisms. For example, behav- ioral function is highly relevant to the selection of "reinforcers" delivered and withheld in differential reinforcement contingencies (Carr & Durand, 1985), and although social reprimands often have been found to function as punishment, they can also serve as positive reinforcement (Van Houten & Doleys, 1983). Thus, data from a number of stud- ies on the functional analysis of behavior disorders indicate that current methods for classifying inter- ventions are inadequate. There is a critical need to develop classification systems and treatment man- uals that specify not only procedures, topographies for which they may be used, and preferred hierarchy based on consideration of factors such as "intru- siveness," but also how the same principle of learn- ing may be translated into different procedures based on differences in behavioral function (see Iwata, Vollmer, Zarcone, & Rodgers, 1993, and Repp & Karsh, 1990, as examples). Because the present results indicated that iden- tification of a behavior's maintaining reinforcers was
a prerequisite to the appropriate use of extinction (unknown reinforcers cannot be withheld), it ap- pears that an answer to the question "Is a functional analysis of a behavior problem useful in developing an effective treatment program?" can be provided by way of data. This represents a significant im- provement over other answers based on theory, logic, or humane philosophy without direct em- pirical evidence. But extinction represents only one approach to treatment. There is ample evidence in both the basic and applied literature that punishment can override the effects of reinforcement. However, almost all studies involving the use of punishment to reduce behavior problems contain a limitation noted pre- viously: Administration of punishment is con- founded with the termination of reinforcement. Ex- tinction is relevant to a consideration of punishment effects because, from the standpoint of methodol- ogy, the suppressive properties of a punishing stim- ulus should be evaluated while behavior is con- currently reinforced, unless it can be shown first that extinction alone does not reduce behavior. Such an arrangement increases the likelihood that ob-
served reductions in behavior can be attributed to punishment rather than to extinction, which seems to be important as a justification for using punish- ment over extinction. At the present time, it is unclear that punishment so applied would have the same suppressive effects as those typically reported in the literature. This fact, in addition to the pos- sibility that stimuli delivered as punishment could function as reinforcement, underscores the relevance of both functional analysis procedures and extinc- tion controls to research on punishment. With respect to reinforcement-based interven- tions, the finding that arbitrary or irrelevant rein- forcers-those that do not maintain the target be- 142 VARIATIONS OF EXTINCTION 143 havior-delivered in a DRO or DRA schedule successfully competed with an intact baseline of reinforcement for the behavior problem (i.e., the
target behavior still produced the maintaining re- inforcer) argues against the necessity of identifying the maintaining reinforcer. That is, if reinforcers could be found that are different than but substi- tutable for those maintaining the behavior problem, the task involves merely finding powerful reinforc- ers. Several studies on DRO as treatment for SIB have shown that reinforcer substitutability can be achieved (e.g., Corte, Wolf, & Locke, 1971; Cow- dery, Iwata, & Pace, 1990), but results from other studies indicate that reinforcers irrelevant to be- havioral function do not always compete success- fully with relevant ones (e.g., Carr & Durand, 1985). Thus, it appears that reinforcer substitut- ability is possible under conditions that remain to be identified at the present time (see Green & Freed, 1993, for a general review of the topic). Never- theless, it seems that a search for those conditions should include specification of the maintaining re- inforcer for which another stimulus would serve as substitute, and that the evaluation of substitution effects should include an extinction control (see Mazaleski et al., 1993, for an example of this methodology).
Extinction alone is rarely recommended and per- haps accounts for the fact that little research has been conducted on extinction effects per se. In light of the present results indicating that properly de- signed extinction procedures can produce powerful clinical effects, future research on the treatment of severe behavior disorders should include more care- ful consideration of extinction as an integral com- ponent of intervention as well as a significant source of confounding effects when evaluating both re- inforcement- and punishment-based approaches to treatment. REFERENCES Carr, E. G., & Durand, V. M. (1985). Reducing behavior problems through functional communication training. Journal of Applied Behavior Analysis, 18, 111-126. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus, OH: Merrill. Corte, H. E., Wolf, M. M., & Locke, B. J. (1971). A comparison of procedures for eliminating self-injurious
behavior of retarded adolescents. Journal of Applied Behavior Analysis, 4, 201-2 13. Cowdery, G. E., Iwata, B. A., & Pace, G. M. (1990). Effects and side effects of DRO as treatment for self-injurious behavior. Journal of Applied Behavior Analysis, 23, 497-506. Day, R. M., Rea, J. A., Schussler, N. G., Larsen, S. E., & Johnson, W. L. (1988). A functionally based approach to the treatment of self-injurious behavior. Behavior Modification, 12, 565-589. Dorsey, M. F., Iwata, B. A., Reid, D. H., & Davis, P. A. (1982). Protective equipment: Continuous and contin- gent application in the treatment of self-injurious be- havior.Journal ofApplied Behavior Analysis, 15, 217- 230. Goh, H., & Iwata, B. A. (1994). Behavioral persistence and variability during extinction of self-injury maintained by escape. Journal of Applied Behavior Analysis, 27, 175-176. Green, L., & Freed, D. E. (1993). The substitutability of
reinforcers. Journal of the Experimental Analysis of Behavior, 60, 141-158. Florida HRS manual 160-4. (1989, April). Tallahassee, FL: Florida Department of Health and Rehabilitative Services, Developmental Services Program Office. Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1982). Toward a functional analysis of self-injury. Analysis and Intervention in Develop- mental Disabilities, 2, 3-20. Iwata, B. A., Pace, G. M., Kalsher, M. J., Cowdery, G. E., & Cataldo, M. F. (1990). Experimental analysis and extinction of self-injurious escape behavior. Journal of Applied Behavior Analysis, 23, 11-27. Iwata, B. A., Vollmer, T. R., & Zarcone, J. R. (1990). The experimental (functional) analysis of behavior dis- orders: Methodology, applications, and limitations. In A. C. Repp & N. N. Singh (Eds.), Perspectives on the use of nonaversive and aversive interventions for persons with developmental disabilities (pp. 301-331). Syca- more, IL: Sycamore Publishing Co.
Iwata, B. A., Vollmer, T. R., Zarcone, J. R., & Rodgers, T. A. (1993). Treatment classification and selection based on behavioral function. In R. Van Houten & S. Axelrod (Eds.), Behavior analysis and treatment (pp. 102-12 5). New York: Plenum. LaVigna, G. W., & Donnellan, A. M. (1986). Alternatives to punishment: Solving behavior problems with non- aversive strategies. New York: Irvington. Lovaas, 0. I., & Simmons, J. Q. (1969). Manipulation of self-destruction in three retarded children. Journal of Applied Behavior Analysis, 2, 143-157. Mace, F. C., & Belfiore, P. (1990). Behavioral momentum in the treatment of escape-motivated stereotypy. Journal of Applied Behavior Analysis, 23, 507-514. Mace, F. C., Lalli, J. S., & Lalli, E. P. (1991). Functional analysis and treatment of aberrant behavior. Research in Developmental Disabilities, 12, 15-180. Mazaleski, J. L., Iwata, B. A., Vollmer, T. R., Zarcone, J. R., & Smith, R. G. (1993). Analysis of the reinforce-
144 BRIAN A. IWATA et al. ment and extinction components of DRO contingencies with self-injury. Journal of Applied Behavior Analysis, 26, 143-156. Nelson, C. M., & Rutherford, R. B. (1983). Timeout revisited: Guidelines for its use in special education. Ex- ceptional Education Quarterly, 3, 56-67. O'Brien, S., Karsh, K. G. (1990). Treatment acceptability: Consumer, therapist, and society. In A. C. Repp & N. N. Singh (Eds.), Perspectives on the use of nonaversive and aversive interventions for persons with develop- mental disabilities (pp. 504-516). Sycamore, IL: Syc- amore Publishing Co. Plummer, S., Baer, D. M., & LeBlanc,J. M. (1977). Func- tional considerations in the use of time out and an ef- fective alternative. Journal of Applied Behavior Anal- ysis, 10, 689-705. Repp, A. C., Felce, D., & Barton, L. E. (1988). Basing
the treatment of stereotypic and self-injurious behaviors on hypotheses of their causes. Journal of Applied Be- havior Analysis, 21, 281-289. Repp, A. C., & Karsh, K. G. (1990). A taxonomic ap- proach to the nonaversive treatment of maladaptive be- havior of persons with developmental disabilities. In A. C. Repp & N. N. Singh (Eds.), Perspectives on the use of nonaversive and aversive interventions for persons with developmental disabilities (pp. 331-347). Syca- more, IL: Sycamore Publishing Co. Rincover, A. (1978). Sensory extinction: A procedure for eliminating self-stimulatory behavior in developmentally disabled children. Journal of Abnormal Child Psychol- ogy, 6, 299-310. Rincover, A., Cook, R., Peoples, A., & Packard, D. (1979). Sensory extinction and sensory reinforcement principles for programming multiple adaptive behavior change. Journal of Applied Behavior Analysis, 12, 221-233. Rincover, A., & Devany, J. (1982). The application of sensory extinction procedures to self-injury. Analysis and Intervention in Developmental Disabilities, 2, 67-81.
Sidman, M. (1960). Tactics of scientific research. New York: Basic Books. Solnick, J. V., Rincover, A., & Peterson, C. R. (1977). Some determinants of the reinforcing and punishing ef- fects of time out. Journal ofApplied Behavior Analysis, 10, 415-424. Steege, M. W., Wacker, D. P., Cigrand, K. C., Berg, W. K., Novak, C. G., Reimers, T. M., Sasso, G. M., & DeRaad, A. (1990). Use of negative reinforcement in the treatment of self-injurious behavior. Journal of Ap- plied Behavior Analysis, 23, 459-467. Ulman, J. D., & Sulzer-Azaroff, B. (1975). Multielement baseline design in educational research. In E. Ramp & G. Semb (Eds.), Behavior analysis: Areas of research and application (pp. 359-376). Englewood Cliffs, NJ: Prentice-Hall. Van Houten, R., & Doleys, D. M. (1983). Are social reprimands effective? In S. Axelrod & J. Apsche (Eds.), The effects ofpunishment on human behavior (pp. 45- 70). New York: Academic Press.
Wacker, D. P., Steege, M. W., Northup, J., Sasso, G., Berg, W., Reimers, T., Cooper, L., Cigrand, K., & Donn, L. (1990). A component analysis of functional commu- nication training across three topographies of severe be- havior problems.Journal ofApplied Behavior Analysis, 23, 417-429. Zarcone, J. R., Iwata, B. A., Hughes, C. E., & Vollmer, T. R. (1993). Momentum versus extinction effects in the treatment of self-injurious escape behavior. Journal of Applied Behavior Analysis, 26, 135-136. Received March 12, 1993 Initial editorial decision May 4, 1993 Revisions received July 23, 1993; August 6, 1993 Final acceptance August 26, 1993 Action Editor, F. Charles Mace Chapter 4: Clustering I
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is Cluster Analysis? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups
Inter-cluster distances are maximized Intra-cluster distances are minimized
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Applications of Cluster Analysis Understanding Group related documents for browsing, group genes and proteins that have similar functionality, or group stocks with similar price fluctuations Summarization Reduce the size of large data sets Clustering precipitation in Australia 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is not Cluster Analysis? Simple segmentation
Dividing students into different registration groups alphabetically, by last name Results of a query Groupings are a result of an external specification Clustering is a grouping of objects based on the data Supervised classification Have class label information 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Notion of a Cluster can be Ambiguous
How many clusters?
Four Clusters
Two Clusters
Six Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusterings A clustering is a set of clusters Important distinction between hierarchical and partitional sets of clusters Partitional Clustering A division of data objects into non-overlapping subsets (clusters) such that each data object is in exactly one subset Hierarchical clustering A set of nested clusters organized as a hierarchical tree
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Partitional Clustering Original Points A Partitional Clustering
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Traditional Hierarchical Clustering Non-traditional Hierarchical Clustering Non-traditional Dendrogram Traditional Dendrogram 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Other Distinctions Between Sets of Clusters Exclusive versus non-exclusive In non-exclusive clusterings, points may belong to multiple clusters. Can represent multiple classes or ‘border’ points Fuzzy versus non-fuzzy In fuzzy clustering, a point belongs to every cluster with some weight between 0 and 1 Weights must sum to 1 Probabilistic clustering has similar characteristics Partial versus complete In some cases, we only want to cluster some of the data Heterogeneous versus homogeneous Clusters of widely different sizes, shapes, and densities 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters Well-separated clusters Center-based clusters
Contiguous clusters Density-based clusters Property or Conceptual Described by an Objective Function 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Well-Separated Well-Separated Clusters: A cluster is a set of points such that any point in a cluster is closer (or more similar) to every other point in the cluster than to any point not in the cluster. 3 well-separated clusters
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Center-Based Center-based A cluster is a set of objects such that an object in a cluster is closer to the “center” of a cluster, than to the center of any other cluster The center of a cluster is a centroid. The average of all the points in the cluster is a medoid 4 center-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Contiguity-Based
Contiguous Cluster (Nearest neighbor or Transitive) A point in a cluster is closer (or more similar) to one or more other points in the cluster than to any point not in the cluster. 8 contiguous clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Density-Based Density-based A cluster is a dense region of points, which is separated by low-
density regions, from other regions of high density. Used when the clusters are irregular or intertwined, and when noise and outliers are present. 6 density-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Conceptual Clusters Shared Property or Conceptual Clusters Finds clusters that share some common property or represent a particular concept. . 2 Overlapping Circles
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Objective Function Clusters Defined by an Objective Function Finds clusters that minimize or maximize an objective function. Enumerate all possible ways of dividing the points into clusters and evaluate the `goodness' of each potential set of clusters by using the given objective function. (NP Hard) Can have global or local objectives. Hierarchical clustering algorithms typically have local objectives Partitional algorithms typically have global objectives A variation of the global objective function approach is to fit the data to a parameterized model. Parameters for the model are determined from the data. Mixture models assume that the data is a ‘mixture' of a number of statistical distributions.
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Characteristics of the Input Data Are Important Type of proximity or density measure Central to clustering Depends on data and application Data characteristics that affect proximity and/or density are Dimensionality Sparseness Attribute type Special relationships in the data For example, autocorrelation Distribution of the data Noise and Outliers Often interfere with the operation of the clustering algorithm 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Clustering Algorithms K-means and its variants Hierarchical clustering Density-based clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering Partitional clustering approach Number of clusters, K, must be specified Each cluster is associated with a centroid (center point) Each point is assigned to the cluster with the closest centroid The basic algorithm is very simple
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering – Details Initial centroids are often chosen randomly. Clusters produced vary from one run to another. The centroid is (typically) the mean of the points in the cluster. ‘Closeness’ is measured by Euclidean distance, cosine similarity, correlation, etc. K-means will converge for common similarity measures mentioned above. Most of the convergence happens in the first few iterations. Often the stopping condition is changed to ‘Until relatively few points change clusters’ Complexity is O( n * K * I * d ) n = number of points, K = number of clusters, I = number of iterations, d = number of attributes
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Evaluating K-means Clusters Most common measure is Sum of Squared Error (SSE) For each point, the error is the distance to the nearest cluster To get SSE, we square these errors and sum them. x is a data point in cluster Ci and mi is the representative point for cluster Ci can show that mi corresponds to the center (mean) of the cluster Given two sets of clusters, we prefer the one with the smallest error One easy way to reduce SSE is to increase K, the number of clusters A good clustering with smaller K can have a lower SSE than a poor clustering with higher K
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Two different K-means Clusterings Sub-optimal Clustering Optimal Clustering Original Points 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means K-means has problems when clusters are of differing Sizes Densities Non-globular Shapes K-means has problems when the data contains outliers/noise.
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Sizes Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Density
Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Non-globular Shapes Original Points K-means (2 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations
Original PointsK-means Clusters One solution is to use many clusters. Find parts of clusters, but need to put together. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
Overcoming K-means Limitations Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids …
02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids … 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Problems with Selecting Initial Points If there are K ‘real’ clusters then the chance of selecting one centroid from each cluster is small. Chance is relatively small when K is large If clusters are the same size, n, then
For example, if K = 10, then probability = 10!/1010 = 0.00036 Sometimes the initial centroids will readjust themselves in ‘right’ way, and sometimes they don’t Consider an example of five pairs of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›

Recommended

chap8_basic_cluster_analysis.ppt
chap8_basic_cluster_analysis.pptchap8_basic_cluster_analysis.ppt
chap8_basic_cluster_analysis.ppt
Darkkali1
 
Clustering
ClusteringClustering
Clustering
Kiran Bhowmick
 
Cluster analysis
Cluster analysisCluster analysis
Cluster analysis
Kamalakshi Deshmukh-Samag
 
chap7_basic_cluster_analysis.pptx
chap7_basic_cluster_analysis.pptxchap7_basic_cluster_analysis.pptx
chap7_basic_cluster_analysis.pptx
DiaaMustafa2
 
Chap8 basic cluster_analysis
Chap8 basic cluster_analysisChap8 basic cluster_analysis
Chap8 basic cluster_analysis
guru_prasadg
 
Clustering.pptx
Clustering.pptxClustering.pptx
Clustering.pptx
Sivam Chinna
 
clustering.pptx
clustering.pptxclustering.pptx
clustering.pptx
rithika858339
 
Data mining concepts and techniques Chapter 10
Data mining concepts and techniques Chapter 10Data mining concepts and techniques Chapter 10
Data mining concepts and techniques Chapter 10
mqasimsheikh5
 

Recommended

chap8_basic_cluster_analysis.ppt
chap8_basic_cluster_analysis.pptchap8_basic_cluster_analysis.ppt
chap8_basic_cluster_analysis.ppt
Darkkali1
 
Clustering
ClusteringClustering
Clustering
Kiran Bhowmick
 
Cluster analysis
Cluster analysisCluster analysis
Cluster analysis
Kamalakshi Deshmukh-Samag
 
chap7_basic_cluster_analysis.pptx
chap7_basic_cluster_analysis.pptxchap7_basic_cluster_analysis.pptx
chap7_basic_cluster_analysis.pptx
DiaaMustafa2
 
Chap8 basic cluster_analysis
Chap8 basic cluster_analysisChap8 basic cluster_analysis
Chap8 basic cluster_analysis
guru_prasadg
 
Clustering.pptx
Clustering.pptxClustering.pptx
Clustering.pptx
Sivam Chinna
 
clustering.pptx
clustering.pptxclustering.pptx
clustering.pptx
rithika858339
 
Data mining concepts and techniques Chapter 10
Data mining concepts and techniques Chapter 10Data mining concepts and techniques Chapter 10
Data mining concepts and techniques Chapter 10
mqasimsheikh5
 
Chapter 5.pdf
Chapter 5.pdfChapter 5.pdf
Chapter 5.pdf
DrGnaneswariG
 
XL-MINER: Data Exploration
XL-MINER: Data ExplorationXL-MINER: Data Exploration
XL-MINER: Data Exploration
DataminingTools Inc
 
XL-MINER:Data Exploration
XL-MINER:Data ExplorationXL-MINER:Data Exploration
XL-MINER:Data Exploration
xlminer content
 
Lx3520322036
Lx3520322036Lx3520322036
Lx3520322036
IJERA Editor
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
Datamining Tools
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
DataminingTools Inc
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
guest0edcaf
 
Unsupervised Learning.pptx
Unsupervised Learning.pptxUnsupervised Learning.pptx
Unsupervised Learning.pptx
GandhiMathy6
 
Data Mining Primitives, Languages & Systems
Data Mining Primitives, Languages & SystemsData Mining Primitives, Languages & Systems
Data Mining Primitives, Languages & Systems
Niloy Sikder
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
ImXaib
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
tarungupta276841
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
RajeshT305412
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
aneeshsperumprath
 
Chapter 10.1,2,3.pptx
Chapter 10.1,2,3.pptxChapter 10.1,2,3.pptx
Chapter 10.1,2,3.pptx
Amy Aung
 
15857 cse422 unsupervised-learning
15857 cse422 unsupervised-learning15857 cse422 unsupervised-learning
15857 cse422 unsupervised-learning
Anil Yadav
 
BIM Data Mining Unit5 by Tekendra Nath Yogi
BIM Data Mining Unit5 by Tekendra Nath Yogi BIM Data Mining Unit5 by Tekendra Nath Yogi
BIM Data Mining Unit5 by Tekendra Nath Yogi
Tekendra Nath Yogi
 
CLUSTERING IN DATA MINING.pdf
CLUSTERING IN DATA MINING.pdfCLUSTERING IN DATA MINING.pdf
CLUSTERING IN DATA MINING.pdf
SowmyaJyothi3
 
84cc04ff77007e457df6aa2b814d2346bf1b
84cc04ff77007e457df6aa2b814d2346bf1b84cc04ff77007e457df6aa2b814d2346bf1b
84cc04ff77007e457df6aa2b814d2346bf1b
PRAWEEN KUMAR
 
ClusetrigBasic.ppt
ClusetrigBasic.pptClusetrigBasic.ppt
ClusetrigBasic.ppt
ChaitanyaKulkarni451137
 
Chapter 10.1,2,3 pdf.pdf
Chapter 10.1,2,3 pdf.pdfChapter 10.1,2,3 pdf.pdf
Chapter 10.1,2,3 pdf.pdf
Amy Aung
 
1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
keturahhazelhurst
 
1. Some potentially pathogenic bacteria and fungi, including strains.docx
1. Some potentially pathogenic bacteria and fungi, including strains.docx1. Some potentially pathogenic bacteria and fungi, including strains.docx
1. Some potentially pathogenic bacteria and fungi, including strains.docx
keturahhazelhurst
 

More Related Content

Similar to Chapter 4 Clustering I02142018 Intro.docx

Chapter 5.pdf
Chapter 5.pdfChapter 5.pdf
Chapter 5.pdf
DrGnaneswariG
 
XL-MINER: Data Exploration
XL-MINER: Data ExplorationXL-MINER: Data Exploration
XL-MINER: Data Exploration
DataminingTools Inc
 
XL-MINER:Data Exploration
XL-MINER:Data ExplorationXL-MINER:Data Exploration
XL-MINER:Data Exploration
xlminer content
 
Lx3520322036
Lx3520322036Lx3520322036
Lx3520322036
IJERA Editor
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
Datamining Tools
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
DataminingTools Inc
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
guest0edcaf
 
Unsupervised Learning.pptx
Unsupervised Learning.pptxUnsupervised Learning.pptx
Unsupervised Learning.pptx
GandhiMathy6
 
Data Mining Primitives, Languages & Systems
Data Mining Primitives, Languages & SystemsData Mining Primitives, Languages & Systems
Data Mining Primitives, Languages & Systems
Niloy Sikder
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
ImXaib
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
tarungupta276841
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
RajeshT305412
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
aneeshsperumprath
 
Chapter 10.1,2,3.pptx
Chapter 10.1,2,3.pptxChapter 10.1,2,3.pptx
Chapter 10.1,2,3.pptx
Amy Aung
 
15857 cse422 unsupervised-learning
15857 cse422 unsupervised-learning15857 cse422 unsupervised-learning
15857 cse422 unsupervised-learning
Anil Yadav
 
BIM Data Mining Unit5 by Tekendra Nath Yogi
BIM Data Mining Unit5 by Tekendra Nath Yogi BIM Data Mining Unit5 by Tekendra Nath Yogi
BIM Data Mining Unit5 by Tekendra Nath Yogi
Tekendra Nath Yogi
 
CLUSTERING IN DATA MINING.pdf
CLUSTERING IN DATA MINING.pdfCLUSTERING IN DATA MINING.pdf
CLUSTERING IN DATA MINING.pdf
SowmyaJyothi3
 
84cc04ff77007e457df6aa2b814d2346bf1b
84cc04ff77007e457df6aa2b814d2346bf1b84cc04ff77007e457df6aa2b814d2346bf1b
84cc04ff77007e457df6aa2b814d2346bf1b
PRAWEEN KUMAR
 
ClusetrigBasic.ppt
ClusetrigBasic.pptClusetrigBasic.ppt
ClusetrigBasic.ppt
ChaitanyaKulkarni451137
 
Chapter 10.1,2,3 pdf.pdf
Chapter 10.1,2,3 pdf.pdfChapter 10.1,2,3 pdf.pdf
Chapter 10.1,2,3 pdf.pdf
Amy Aung
 

Similar to Chapter 4 Clustering I02142018 Intro.docx (20)

Chapter 5.pdf
Chapter 5.pdfChapter 5.pdf
Chapter 5.pdf
 
XL-MINER: Data Exploration
XL-MINER: Data ExplorationXL-MINER: Data Exploration
XL-MINER: Data Exploration
 
XL-MINER:Data Exploration
XL-MINER:Data ExplorationXL-MINER:Data Exploration
XL-MINER:Data Exploration
 
Lx3520322036
Lx3520322036Lx3520322036
Lx3520322036
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
 
Cluster Analysis
Cluster AnalysisCluster Analysis
Cluster Analysis
 
Unsupervised Learning.pptx
Unsupervised Learning.pptxUnsupervised Learning.pptx
Unsupervised Learning.pptx
 
Data Mining Primitives, Languages & Systems
Data Mining Primitives, Languages & SystemsData Mining Primitives, Languages & Systems
Data Mining Primitives, Languages & Systems
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
 
lecture12-clustering.ppt
lecture12-clustering.pptlecture12-clustering.ppt
lecture12-clustering.ppt
 
Chapter 10.1,2,3.pptx
Chapter 10.1,2,3.pptxChapter 10.1,2,3.pptx
Chapter 10.1,2,3.pptx
 
15857 cse422 unsupervised-learning
15857 cse422 unsupervised-learning15857 cse422 unsupervised-learning
15857 cse422 unsupervised-learning
 
BIM Data Mining Unit5 by Tekendra Nath Yogi
BIM Data Mining Unit5 by Tekendra Nath Yogi BIM Data Mining Unit5 by Tekendra Nath Yogi
BIM Data Mining Unit5 by Tekendra Nath Yogi
 
CLUSTERING IN DATA MINING.pdf
CLUSTERING IN DATA MINING.pdfCLUSTERING IN DATA MINING.pdf
CLUSTERING IN DATA MINING.pdf
 
84cc04ff77007e457df6aa2b814d2346bf1b
84cc04ff77007e457df6aa2b814d2346bf1b84cc04ff77007e457df6aa2b814d2346bf1b
84cc04ff77007e457df6aa2b814d2346bf1b
 
ClusetrigBasic.ppt
ClusetrigBasic.pptClusetrigBasic.ppt
ClusetrigBasic.ppt
 
Chapter 10.1,2,3 pdf.pdf
Chapter 10.1,2,3 pdf.pdfChapter 10.1,2,3 pdf.pdf
Chapter 10.1,2,3 pdf.pdf
 

More from keturahhazelhurst

1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
keturahhazelhurst
 
1. Some potentially pathogenic bacteria and fungi, including strains.docx
1. Some potentially pathogenic bacteria and fungi, including strains.docx1. Some potentially pathogenic bacteria and fungi, including strains.docx
1. Some potentially pathogenic bacteria and fungi, including strains.docx
keturahhazelhurst
 
1. Terrestrial Planets                           2. Astronomical.docx
1. Terrestrial Planets                           2. Astronomical.docx1. Terrestrial Planets                           2. Astronomical.docx
1. Terrestrial Planets                           2. Astronomical.docx
keturahhazelhurst
 
1. Taking turns to listen to other students is not always easy f.docx
1. Taking turns to listen to other students is not always easy f.docx1. Taking turns to listen to other students is not always easy f.docx
1. Taking turns to listen to other students is not always easy f.docx
keturahhazelhurst
 
1. The main characters names in The Shape of Things are Adam and E.docx
1. The main characters names in The Shape of Things are Adam and E.docx1. The main characters names in The Shape of Things are Adam and E.docx
1. The main characters names in The Shape of Things are Adam and E.docx
keturahhazelhurst
 
1. Select one movie from the list belowShutter Island (2010; My.docx
1. Select one movie from the list belowShutter Island (2010; My.docx1. Select one movie from the list belowShutter Island (2010; My.docx
1. Select one movie from the list belowShutter Island (2010; My.docx
keturahhazelhurst
 
1. Select a system of your choice and describe the system life-cycle.docx
1. Select a system of your choice and describe the system life-cycle.docx1. Select a system of your choice and describe the system life-cycle.docx
1. Select a system of your choice and describe the system life-cycle.docx
keturahhazelhurst
 
1. Sensation refers to an actual event; perception refers to how we .docx
1. Sensation refers to an actual event; perception refers to how we .docx1. Sensation refers to an actual event; perception refers to how we .docx
1. Sensation refers to an actual event; perception refers to how we .docx
keturahhazelhurst
 
1. The Institute of Medicine (now a renamed as a part of the N.docx
1. The Institute of Medicine (now a renamed as a part of the N.docx1. The Institute of Medicine (now a renamed as a part of the N.docx
1. The Institute of Medicine (now a renamed as a part of the N.docx
keturahhazelhurst
 
1. The Documentary Hypothesis holds that the Pentateuch has a number.docx
1. The Documentary Hypothesis holds that the Pentateuch has a number.docx1. The Documentary Hypothesis holds that the Pentateuch has a number.docx
1. The Documentary Hypothesis holds that the Pentateuch has a number.docx
keturahhazelhurst
 
1. Search the internet and learn about the cases of nurses Julie.docx
1. Search the internet and learn about the cases of nurses Julie.docx1. Search the internet and learn about the cases of nurses Julie.docx
1. Search the internet and learn about the cases of nurses Julie.docx
keturahhazelhurst
 
1. Search the internet and learn about the cases of nurses Julie Tha.docx
1. Search the internet and learn about the cases of nurses Julie Tha.docx1. Search the internet and learn about the cases of nurses Julie Tha.docx
1. Search the internet and learn about the cases of nurses Julie Tha.docx
keturahhazelhurst
 
1. Review the three articles about Inflation that are found below th.docx
1. Review the three articles about Inflation that are found below th.docx1. Review the three articles about Inflation that are found below th.docx
1. Review the three articles about Inflation that are found below th.docx
keturahhazelhurst
 
1. Review the following request from a customerWe have a ne.docx
1. Review the following request from a customerWe have a ne.docx1. Review the following request from a customerWe have a ne.docx
1. Review the following request from a customerWe have a ne.docx
keturahhazelhurst
 
1. Research risk assessment approaches.2. Create an outline .docx
1. Research risk assessment approaches.2. Create an outline .docx1. Research risk assessment approaches.2. Create an outline .docx
1. Research risk assessment approaches.2. Create an outline .docx
keturahhazelhurst
 
1. Research has narrowed the thousands of leadership behaviors into .docx
1. Research has narrowed the thousands of leadership behaviors into .docx1. Research has narrowed the thousands of leadership behaviors into .docx
1. Research has narrowed the thousands of leadership behaviors into .docx
keturahhazelhurst
 
1. Research Topic Super Computer Data MiningThe aim of this.docx
1. Research Topic Super Computer Data MiningThe aim of this.docx1. Research Topic Super Computer Data MiningThe aim of this.docx
1. Research Topic Super Computer Data MiningThe aim of this.docx
keturahhazelhurst
 
1. Research and then describe about The Coca-Cola Company primary bu.docx
1. Research and then describe about The Coca-Cola Company primary bu.docx1. Research and then describe about The Coca-Cola Company primary bu.docx
1. Research and then describe about The Coca-Cola Company primary bu.docx
keturahhazelhurst
 
1. Prepare a risk management plan for the project of finding a job a.docx
1. Prepare a risk management plan for the project of finding a job a.docx1. Prepare a risk management plan for the project of finding a job a.docx
1. Prepare a risk management plan for the project of finding a job a.docx
keturahhazelhurst
 
1. Please define the term social class. How is it usually measured .docx
1. Please define the term social class. How is it usually measured .docx1. Please define the term social class. How is it usually measured .docx
1. Please define the term social class. How is it usually measured .docx
keturahhazelhurst
 

More from keturahhazelhurst (20)

1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
1. The ALIVE status of each SEX. (SEX needs to be integrated into th.docx
 
1. Some potentially pathogenic bacteria and fungi, including strains.docx
1. Some potentially pathogenic bacteria and fungi, including strains.docx1. Some potentially pathogenic bacteria and fungi, including strains.docx
1. Some potentially pathogenic bacteria and fungi, including strains.docx
 
1. Terrestrial Planets                           2. Astronomical.docx
1. Terrestrial Planets                           2. Astronomical.docx1. Terrestrial Planets                           2. Astronomical.docx
1. Terrestrial Planets                           2. Astronomical.docx
 
1. Taking turns to listen to other students is not always easy f.docx
1. Taking turns to listen to other students is not always easy f.docx1. Taking turns to listen to other students is not always easy f.docx
1. Taking turns to listen to other students is not always easy f.docx
 
1. The main characters names in The Shape of Things are Adam and E.docx
1. The main characters names in The Shape of Things are Adam and E.docx1. The main characters names in The Shape of Things are Adam and E.docx
1. The main characters names in The Shape of Things are Adam and E.docx
 
1. Select one movie from the list belowShutter Island (2010; My.docx
1. Select one movie from the list belowShutter Island (2010; My.docx1. Select one movie from the list belowShutter Island (2010; My.docx
1. Select one movie from the list belowShutter Island (2010; My.docx
 
1. Select a system of your choice and describe the system life-cycle.docx
1. Select a system of your choice and describe the system life-cycle.docx1. Select a system of your choice and describe the system life-cycle.docx
1. Select a system of your choice and describe the system life-cycle.docx
 
1. Sensation refers to an actual event; perception refers to how we .docx
1. Sensation refers to an actual event; perception refers to how we .docx1. Sensation refers to an actual event; perception refers to how we .docx
1. Sensation refers to an actual event; perception refers to how we .docx
 
1. The Institute of Medicine (now a renamed as a part of the N.docx
1. The Institute of Medicine (now a renamed as a part of the N.docx1. The Institute of Medicine (now a renamed as a part of the N.docx
1. The Institute of Medicine (now a renamed as a part of the N.docx
 
1. The Documentary Hypothesis holds that the Pentateuch has a number.docx
1. The Documentary Hypothesis holds that the Pentateuch has a number.docx1. The Documentary Hypothesis holds that the Pentateuch has a number.docx
1. The Documentary Hypothesis holds that the Pentateuch has a number.docx
 
1. Search the internet and learn about the cases of nurses Julie.docx
1. Search the internet and learn about the cases of nurses Julie.docx1. Search the internet and learn about the cases of nurses Julie.docx
1. Search the internet and learn about the cases of nurses Julie.docx
 
1. Search the internet and learn about the cases of nurses Julie Tha.docx
1. Search the internet and learn about the cases of nurses Julie Tha.docx1. Search the internet and learn about the cases of nurses Julie Tha.docx
1. Search the internet and learn about the cases of nurses Julie Tha.docx
 
1. Review the three articles about Inflation that are found below th.docx
1. Review the three articles about Inflation that are found below th.docx1. Review the three articles about Inflation that are found below th.docx
1. Review the three articles about Inflation that are found below th.docx
 
1. Review the following request from a customerWe have a ne.docx
1. Review the following request from a customerWe have a ne.docx1. Review the following request from a customerWe have a ne.docx
1. Review the following request from a customerWe have a ne.docx
 
1. Research risk assessment approaches.2. Create an outline .docx
1. Research risk assessment approaches.2. Create an outline .docx1. Research risk assessment approaches.2. Create an outline .docx
1. Research risk assessment approaches.2. Create an outline .docx
 
1. Research has narrowed the thousands of leadership behaviors into .docx
1. Research has narrowed the thousands of leadership behaviors into .docx1. Research has narrowed the thousands of leadership behaviors into .docx
1. Research has narrowed the thousands of leadership behaviors into .docx
 
1. Research Topic Super Computer Data MiningThe aim of this.docx
1. Research Topic Super Computer Data MiningThe aim of this.docx1. Research Topic Super Computer Data MiningThe aim of this.docx
1. Research Topic Super Computer Data MiningThe aim of this.docx
 
1. Research and then describe about The Coca-Cola Company primary bu.docx
1. Research and then describe about The Coca-Cola Company primary bu.docx1. Research and then describe about The Coca-Cola Company primary bu.docx
1. Research and then describe about The Coca-Cola Company primary bu.docx
 
1. Prepare a risk management plan for the project of finding a job a.docx
1. Prepare a risk management plan for the project of finding a job a.docx1. Prepare a risk management plan for the project of finding a job a.docx
1. Prepare a risk management plan for the project of finding a job a.docx
 
1. Please define the term social class. How is it usually measured .docx
1. Please define the term social class. How is it usually measured .docx1. Please define the term social class. How is it usually measured .docx
1. Please define the term social class. How is it usually measured .docx
 

Recently uploaded

ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
PECB
 
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama UniversityNatural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Akanksha trivedi rama nursing college kanpur.
 
PIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf IslamabadPIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf Islamabad
AyyanKhan40
 
The basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptxThe basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptx
heathfieldcps1
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
National Information Standards Organization (NISO)
 
How to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRMHow to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRM
Celine George
 
DRUGS AND ITS classification slide share
DRUGS AND ITS classification slide shareDRUGS AND ITS classification slide share
DRUGS AND ITS classification slide share
taiba qazi
 
World environment day ppt For 5 June 2024
World environment day ppt For 5 June 2024World environment day ppt For 5 June 2024
World environment day ppt For 5 June 2024
ak6969907
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
Peter Windle
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
writing about opinions about Australia the movie
writing about opinions about Australia the moviewriting about opinions about Australia the movie
writing about opinions about Australia the movie
Nicholas Montgomery
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
TechSoup
 
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
eBook.com.bd (প্রয়োজনীয় বাংলা বই)
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
Nicholas Montgomery
 
The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
History of Stoke Newington
 
The basics of sentences session 5pptx.pptx
The basics of sentences session 5pptx.pptxThe basics of sentences session 5pptx.pptx
The basics of sentences session 5pptx.pptx
heathfieldcps1
 
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Dr. Vinod Kumar Kanvaria
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
Nguyen Thanh Tu Collection
 
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdfANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
Priyankaranawat4
 
CACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdfCACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdf
camakaiclarkmusic
 

Recently uploaded (20)

ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
 
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama UniversityNatural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
 
PIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf IslamabadPIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf Islamabad
 
The basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptxThe basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptx
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
 
How to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRMHow to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRM
 
DRUGS AND ITS classification slide share
DRUGS AND ITS classification slide shareDRUGS AND ITS classification slide share
DRUGS AND ITS classification slide share
 
World environment day ppt For 5 June 2024
World environment day ppt For 5 June 2024World environment day ppt For 5 June 2024
World environment day ppt For 5 June 2024
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
writing about opinions about Australia the movie
writing about opinions about Australia the moviewriting about opinions about Australia the movie
writing about opinions about Australia the movie
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
 
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
 
The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
 
The basics of sentences session 5pptx.pptx
The basics of sentences session 5pptx.pptxThe basics of sentences session 5pptx.pptx
The basics of sentences session 5pptx.pptx
 
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
 
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdfANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
 
CACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdfCACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdf
 

Chapter 4 Clustering I02142018 Intro.docx

  • 1. Chapter 4: Clustering I 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is Cluster Analysis? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups
  • 2. Inter-cluster distances are maximized Intra-cluster distances are minimized 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Applications of Cluster Analysis Understanding Group related documents for browsing, group genes and proteins that have similar functionality, or group stocks with similar price fluctuations Summarization Reduce the size of large data sets
  • 3. Clustering precipitation in Australia 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is not Cluster Analysis? Simple segmentation Dividing students into different registration groups alphabetically, by last name Results of a query Groupings are a result of an external specification Clustering is a grouping of objects based on the data Supervised classification Have class label information Association Analysis Local vs. global connections 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Notion of a Cluster can be Ambiguous
  • 6. Six Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusterings A clustering is a set of clusters Important distinction between hierarchical and partitional sets of clusters Partitional Clustering A division of data objects into non-overlapping subsets (clusters) such that each data object is in exactly one subset Hierarchical clustering A set of nested clusters organized as a hierarchical tree 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Partitional Clustering
  • 7. Original Points A Partitional Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Traditional Hierarchical Clustering Non-traditional Hierarchical Clustering Non-traditional Dendrogram Traditional Dendrogram 02/14/2018 Introduction to Data Mining, 2nd Edition
  • 8. ‹#› Types of Clusters Well-separated clusters Center-based clusters Contiguous clusters Density-based clusters Property or Conceptual Described by an Objective Function 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Well-Separated Well-Separated Clusters: A cluster is a set of points such that any point in a cluster is closer (or more similar) to every other point in the cluster than to any point not in the cluster. 3 well-separated clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 9. Types of Clusters: Center-Based Center-based A cluster is a set of objects such that an object in a cluster is closer to the “center” of a cluster, than to the center of any other cluster The center of a cluster is a centroid. The average of all the points in the cluster is a medoid 4 center-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Contiguity-Based Contiguous Cluster (Nearest neighbor or Transitive) A point in a cluster is closer (or more similar) to one or more other points in the cluster than to any point not in the cluster. 8 contiguous clusters
  • 10. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Density-Based Density-based A cluster is a dense region of points, which is separated by low- density regions, from other regions of high density. Used when the clusters are irregular or intertwined, and when noise and outliers are present. 6 density-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Conceptual Clusters Shared Property or Conceptual Clusters Finds clusters that share some common property or represent a particular concept. . 2 Overlapping Circles
  • 11. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Objective Function Clusters Defined by an Objective Function Finds clusters that minimize or maximize an objective function. Enumerate all possible ways of dividing the points into clusters and evaluate the `goodness' of each potential set of clusters by using the given objective function. (NP Hard) Can have global or local objectives. Hierarchical clustering algorithms typically have local objectives Partitional algorithms typically have global objectives A variation of the global objective function approach is to fit the data to a parameterized model. Parameters for the model are determined from the data. Mixture models assume that the data is a ‘mixture' of a number of statistical distributions. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Characteristics of the Input Data Are Important Type of proximity or density measure Central to clustering Depends on data and application Data characteristics that affect proximity and/or density are Dimensionality Sparseness Attribute type Special relationships in the data
  • 12. For example, autocorrelation Distribution of the data Noise and Outliers Often interfere with the operation of the clustering algorithm 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Clustering Algorithms K-means and its variants Hierarchical clustering Density-based clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering Partitional clustering approach Number of clusters, K, must be specified Each cluster is associated with a centroid (center point) Each point is assigned to the cluster with the closest centroid The basic algorithm is very simple
  • 13. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering – Details Initial centroids are often chosen randomly.
  • 14. Clusters produced vary from one run to another. The centroid is (typically) the mean of the points in the cluster. ‘Closeness’ is measured by Euclidean distance, cosine similarity, correlation, etc. K-means will converge for common similarity measures mentioned above. Most of the convergence happens in the first few iterations. Often the stopping condition is changed to ‘Until relatively few points change clusters’ Complexity is O( n * K * I * d ) n = number of points, K = number of clusters, I = number of iterations, d = number of attributes 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Evaluating K-means Clusters Most common measure is Sum of Squared Error (SSE) For each point, the error is the distance to the nearest cluster To get SSE, we square these errors and sum them. x is a data point in cluster Ci and mi is the representative point for cluster Ci can show that mi corresponds to the center (mean) of the cluster Given two sets of clusters, we prefer the one with the smallest error One easy way to reduce SSE is to increase K, the number of clusters A good clustering with smaller K can have a lower SSE than a poor clustering with higher K
  • 15. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Two different K-means Clusterings Sub-optimal Clustering Optimal Clustering Original Points 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means K-means has problems when clusters are of differing Sizes Densities Non-globular shapes K-means has problems when the data contains outliers. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Sizes
  • 16. Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Density Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Non-globular Shapes Original Points K-means (2 Clusters)
  • 17. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations Original PointsK-means Clusters One solution is to use many clusters. Find parts of clusters, but need to put together. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations
  • 18. Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids 02/14/2018 Introduction to Data Mining, 2nd Edition
  • 19. ‹#› Importance of Choosing Initial Centroids … 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids … 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Problems with Selecting Initial Points If there are K ‘real’ clusters then the chance of selecting one centroid from each cluster is small. Chance is relatively small when K is large If clusters are the same size, n, then For example, if K = 10, then probability = 10!/1010 = 0.00036 Sometimes the initial centroids will readjust themselves in
  • 20. ‘right’ way, and sometimes they don’t Consider an example of five pairs of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 21. Solution s to Initial Centroids Problem Multiple runs Helps, but probability is not on your side Sample and use hierarchical clustering to determine initial centroids Select more than k initial centroids and then select among these initial centroids Select most widely separated Generate a larger number of clusters and then perform a hierarchical clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram A tree like diagram that records the sequences of merges or splits
  • 22. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Strengths of Hierarchical Clustering Do not have to assume any particular number of clusters Any desired number of clusters can be obtained by ‘cutting’ the dendrogram at the proper level They may correspond to meaningful taxonomies Example in biological sciences (e.g., animal kingdom) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Two main types of hierarchical clustering Agglomerative:
  • 23. Start with the points as individual clusters At each step, merge the closest pair of clusters until only one cluster (or k clusters) left Divisive: Start with one, all-inclusive cluster At each step, split a cluster until each cluster contains an individual point (or there are k clusters) Traditional hierarchical algorithms use a similarity or distance matrix Merge or split one cluster at a time 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Agglomerative Clustering Algorithm Most popular hierarchical clustering technique Basic algorithm is straightforward Compute the proximity matrix Let each data point be a cluster
  • 24. Repeat Merge the two closest clusters Update the proximity matrix Until only a single cluster remains Key operation is the computation of the proximity of two clusters Different approaches to defining the distance between clusters distinguish the different algorithms 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Starting Situation
  • 25. Start with clusters of individual points and a proximity matrix
  • 26. p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . . Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 27. Intermediate Situation After some merging steps, we have some clusters C1 C4 C2 C5 C3 C2 C1 C1 C3 C5 C4 C2 C3
  • 28. C4 C5 Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Intermediate Situation We want to merge the two closest clusters (C2 and C5) and update the proximity matrix.
  • 30. Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› After Merging The question is “How do we update the proximity matrix?”
  • 31. C1 C4 C2 U C5 C3 ? ? ? ? ? ? ? C2 U C5 C1 C1 C3 C4 C2 U C5 C3 C4
  • 32. Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Distance
  • 33. p1 p3 p5 p4 p2 p1 p2 p3 p4 p5 . . . . . . Similarity? MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error
  • 34. Proximity Matrix 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
  • 36. Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
  • 38. Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
  • 40.
  • 41. Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› How to Define Inter-Cluster Similarity
  • 43. Proximity Matrix MIN MAX Group Average Distance Between Centroids Other methods driven by an objective function Ward’s Method uses squared error 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› MIN or Single Link Proximity of two clusters is based on the two closest points in the different clusters Determined by one pair of points, i.e., by one link in the proximity graph Example:
  • 44. Distance Matrix: 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: MIN Nested Clusters Dendrogram 1 2 3 4 5 6 1
  • 45. 2 3 4 5 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Strength of MIN Original Points Six Clusters Can handle non-elliptical shapes 02/14/2018 Introduction to Data Mining, 2nd Edition
  • 46. ‹#› Limitations of MIN Original Points Two Clusters Sensitive to noise and outliers Three Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› MAX or Complete Linkage Proximity of two clusters is based on the two most distant points in the different clusters Determined by all pairs of points in the two clusters Distance Matrix:
  • 47. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: MAX Nested Clusters Dendrogram 1 2 3 4 5 6 1 2
  • 48. 5 3 4 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Strength of MAX Original Points Two Clusters Less susceptible to noise and outliers 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of MAX
  • 49. Original Points Two Clusters Tends to break large clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Group Average Proximity of two clusters is the average of pairwise proximity between points in the two clusters. Need to use average connectivity for scalability since total proximity favors large clusters Distance Matrix:
  • 50. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Group Average Nested Clusters Dendrogram 1 2 3 4 5 6 1
  • 51. 2 5 3 4 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Group Average Compromise between Single and Complete Link Strengths Less susceptible to noise and outliers Limitations Biased towards globular clusters 02/14/2018 Introduction to Data Mining, 2nd Edition
  • 52. ‹#› Cluster Similarity: Ward’s Method Similarity of two clusters is based on the increase in squared error when two clusters are merged Similar to group average if distance between points is distance squared Less susceptible to noise and outliers Biased towards globular clusters Hierarchical analogue of K-means Can be used to initialize K-means 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Comparison Group Average Ward’s Method
  • 56. 4 5 6 1 2 3 4 5 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Time and Space requirements O(N2) space since it uses the proximity matrix. N is the number of points. O(N3) time in many cases There are N steps and at each step the size, N2, proximity
  • 57. matrix must be updated and searched Complexity can be reduced to O(N2 log(N) ) time with some cleverness 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering: Problems and Limitations Once a decision is made to combine two clusters, it cannot be undone No global objective function is directly minimized Different schemes have problems with one or more of the following: Sensitivity to noise and outliers Difficulty handling clusters of different sizes and non-globular shapes Breaking large clusters
  • 58. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Cluster Validity For supervised classification we have a variety of measures to evaluate how good our model is Accuracy, precision, recall For cluster analysis, the analogous question is how to evaluate the “goodness” of the resulting clusters? But “clusters are in the eye of the beholder”! Then why do we want to evaluate them? To avoid finding patterns in noise To compare clustering algorithms To compare two sets of clusters To compare two clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 59. Clusters found in Random Data Random Points K-means Complete Link 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Determining the clustering tendency of a set of data, i.e., distinguishing whether non-random structure actually exists in the data. Comparing the results of a cluster analysis to externally known results, e.g., to externally given class labels. Evaluating how well the results of a cluster analysis fit the data without reference to external information. - Use only the data Comparing the results of two different sets of cluster analyses to determine which is better. Determining the ‘correct’ number of clusters.
  • 60. For 2, 3, and 4, we can further distinguish whether we want to evaluate the entire clustering or just individual clusters. Different Aspects of Cluster Validation 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Numerical measures that are applied to judge various aspects of cluster validity, are classified into the following three types. External Index: Used to measure the extent to which cluster labels match externally supplied class labels. Entropy Internal Index: Used to measure the goodness of a clustering structure without respect to external information. Sum of Squared Error (SSE) Relative Index: Used to compare two different clusterings or clusters. Often an external or internal index is used for this function, e.g., SSE or entropy Sometimes these are referred to as criteria instead of indices However, sometimes criterion is the general strategy and index
  • 61. is the numerical measure that implements the criterion. Measures of Cluster Validity 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Two matrices Proximity Matrix Ideal Similarity Matrix One row and one column for each data point An entry is 1 if the associated pair of points belong to the same cluster An entry is 0 if the associated pair of points belongs to different clusters Compute the correlation between the two matrices Since the matrices are symmetric, only the correlation between n(n-1) / 2 entries needs to be calculated. High correlation indicates that points that belong to the same cluster are close to each other. Not a good measure for some density or contiguity based clusters. Measuring Cluster Validity Via Correlation
  • 62. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Measuring Cluster Validity Via Correlation Correlation of ideal similarity and proximity matrices for the K- means clusterings of the following two data sets. Corr = -0.9235 Corr = -0.5810 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Order the similarity matrix with respect to cluster labels and inspect visually. Using Similarity Matrix for Cluster Validation
  • 63. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Using Similarity Matrix for Cluster Validation Clusters in random data are not so crisp K-means 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Using Similarity Matrix for Cluster Validation Clusters in random data are not so crisp Complete Link
  • 64. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Clusters in more complicated figures aren’t well separated Internal Index: Used to measure the goodness of a clustering structure without respect to external information SSE SSE is good for comparing two clusterings or two clusters (average SSE). Can also be used to estimate the number of clusters Internal Measures: SSE 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Internal Measures: SSE SSE curve for a more complicated data set
  • 65. SSE of clusters found using K-means 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Correlation of ideal similarity and proximity matrices for the K- means clusterings of the following two data sets. Statistical Framework for Correlation Corr = -0.9235 Corr = -0.5810 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Cluster Cohesion: Measures how closely related are objects in a
  • 66. cluster Example: SSE Cluster Separation: Measure how distinct or well-separated a cluster is from other clusters Example: Squared Error Cohesion is measured by the within cluster sum of squares (SSE) Separation is measured by the between cluster sum of squares Where |Ci| is the size of cluster i Internal Measures: Cohesion and Separation 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 67. Internal Measures: Cohesion and Separation Example: SSE BSS + WSS = constant 1 2 3 4 5 m1 m2 m
  • 68. K=2 clusters: K=1 cluster: 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› A proximity graph based approach can also be used for cohesion and separation. Cluster cohesion is the sum of the weight of all links within a cluster. Cluster separation is the sum of the weights between nodes in the cluster and nodes outside the cluster. Internal Measures: Cohesion and Separation
  • 69.
  • 70. cohesion separation 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› “The validation of clustering structures is the most difficult and frustrating part of cluster analysis. Without a strong effort in this direction, cluster analysis will remain a black art accessible only to those true believers who have experience and great courage.” Algorithms for Clustering Data, Jain and Dubes Final Comment on Cluster Validity
  • 71. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Discovered Clusters Industry Group 1 Applied-Matl-DOWN,Bay-Network-Down,3-COM-DOWN, Cabletron-Sys-DOWN,CISCO-DOWN,HP-DOWN, DSC-Comm-DOWN,INTEL-DOWN,LSI-Logic-DOWN, Micron-Tech-DOWN,Texas-Inst-Down,Tellabs-Inc-Down, Natl-Semiconduct-DOWN,Oracl-DOWN,SGI-DOWN, Sun-DOWN Technology1-DOWN 2 Apple-Comp-DOWN,Autodesk-DOWN,DEC-DOWN, ADV-Micro-Device-DOWN,Andrew-Corp-DOWN, Computer-Assoc-DOWN,Circuit-City-DOWN, Compaq-DOWN, EMC-Corp-DOWN, Gen-Inst-DOWN, Motorola-DOWN,Microsoft-DOWN,Scientific-Atl-DOWN Technology2-DOWN 3 Fannie-Mae-DOWN,Fed-Home-Loan-DOWN,
  • 107. - = = Total BSS WSS SSE JOURNAL OF APPLIED BEHAVIOR ANALYSIS WHAT MAKES EXTINCTION WORK: AN ANALYSIS OF PROCEDURAL FORM AND FUNCTION BRIAN A. IWATA THE UNIVERSITY OF FLORIDA GARY M. PACE THE KENNEDY INSTITUTE, BALTIMORE, MARYLAND GLYNNIS EDWARDS COWDERY WE CARE, INC., BERKELEY, CALIFORNIA
  • 108. AND RAYMOND G. MILTENBERGER NORTH DAKOTA STATE UNIVERSITY We examined methods for determining how extinction should be applied to different functions of self-injurious behavior (SIB). Assessment data indicated that the head banging of 3 children with developmental disabilities was maintained by different reinforcement contingencies: One subject's SIB was positively reinforced by attention from adults, the 2nd subject's SIB was negatively reinforced by escape from educational tasks, and the 3rd subject's SIB appeared to be automatically reinforced or "self-stimulatory" in nature. Three functional variations of extinction-EXT (attention), EXT (escape), and EXT (sensory)-were evaluated, and each subject was exposed to at least two of these variations in reversal or multiple baseline designs. Reductions in SIB were observed only when implementation of "extinction" involved the discontinuation of reinforcement previously shown to be responsible for maintaining the behavior. These results highlight important differences among
  • 109. treatment techniques based on the same behavioral principle (extinction) when applied to topo- graphically similar but functionally dissimilar responses, and further illustrate the practical impli- cations of a functional analysis of behavior disorders for designing, selecting, and classifying ther- apeutic interventions. DESCRIPTORS: extinction, functional analysis, self-injurious behavior Research on the functional analysis of severe behavior disorders has produced a variety of inter- ventions based on the modification of antecedent events that occasion behavior problems as well as the consequences that maintain them (see Iwata, Vollmer, & Zarcone, 1990, and Mace, Lalli, & Lalli, 1991, for recent reviews). Much of the em- phasis in treatment has been on strengthening new stimulus-response-consequence relationships. By contrast, relatively little attention has been paid to eliminating reinforcement that maintained the be- This research was supported in part by a grant from the Developmental Disabilities Planning Council and the Florida Department of Health and Rehabilitative Services.
  • 110. Reprints may be obtained from Brian Iwata, Psychology Department, University of Florida, Gainesville, Florida 3261 1. havior problem in the first place, even though many of the interventions described in the literature im- plicitly contained such a provision. That is, pun- ishment of Response A is almost always combined with the cessation of reinforcement; likewise, treat- ment involving reinforcement of Response B as a replacement for Response A usually coincides with the termination of reinforcement for Response A. Moreover, data from several recent studies suggest that the effects of reinforcement-based interventions may be limited unless extinction (withholding the behavior's maintaining reinforcers) is included as part of the treatment program. It has been shown, for example, that interventions based on the de- velopment of communicative responses (Carr & Durand, 1985) and on behavioral momentum (Mace & Belfiore, 1990) produced either mixed 131 1994127,131-144 NUMBER I (SPRING 1994)
  • 111. BRIAN A. IWATA et al. results (Wacker et al., 1990) or no treatment effect (Zarcone, Iwata, Hughes, & Vollmer, 1993) when implemented without extinction, and that the crit- ical feature of procedures such as differential rein- forcement of other behavior (DRO) contingencies appears to be the extinction component rather than the reinforcement component (Mazaleski, Iwata, Vollmer, Zarcone, & Smith, 1993). The assessment and treatment of self-injurious behavior (SIB) provide an important context for the examination of extinction, because it has been demonstrated that SIB can be maintained through a variety of operant mechanisms. Nevertheless, be- cause the "discontinuation of reinforcement" for behavior problems in general often involves ces- sation of ongoing events, this procedure has become so common that some textbooks define extinction solely through reference to "ignoring," time-out, and other examples of stimulus termination (e.g.,
  • 112. LaVigna & Donnellan, 1986). However, research has shown that procedural time-out can serve dif- ferent functions, not all of which are behavior re- ducing (Plummer, Baer, & LeBlanc, 1977; Solnick, Rincover, & Peterson, 1977), and the same is true of procedural approaches to defining extinction. Ignoring misbehavior may represent extinction in some cases but not in others; conversely, the correct application of extinction may require termination of events in some cases but continuation in others. The procedures that define extinction in a given situation are determined by the specific nature of the reinforcement to be "discontinued." In cases in which SIB was maintained by social- positive reinforcement in the form of adult atten- tion, extinction consisted of withholding attention or terminating it contingent on the occurrence of SIB (e.g., Day, Rea, Schussler, Larsen, & Johnson, 1988; Lovaas & Simmons, 1969). By contrast, SIB maintained through social-negative reinforcement in the form of escape from task demands has been extinguished by preventing escape; in other words, continuing and not terminating the ongoing sit- uation (e.g., Iwata, Pace, Kalsher, Cowdery, &
  • 113. Cataldo, 1990; Repp, Felce, & Barton, 1988). Finally, SIB apparently maintained by nonsocial, automatic reinforcement (e.g., sensory stimulation) has shown extinction-like decreases when the in- dividuals wore equipment that allowed the behav- ior to occur, but attenuated its consequences (e.g., Rincover & Devany, 1982). Thus, at least three functional variations of extinction have been used as treatment for SIB, each designed to terminate a different source of reinforcement, and each ame- nable to a number of procedural modifications and descriptive labels. Although extinction procedures are clearly unique from the standpoint of both form (specific therapist actions) and function (the maintaining contingen- cies to which they apply), these variations are not well differentiated through existing terminology. Planned ignoring, the label perhaps most often applied to extinction of attention-maintained be- havior (Nelson & Rutherford, 1983), provides an adequate description of the therapist's response, but it does not describe the underlying behavioral pro- cess (extinction), nor does it identify the source of
  • 114. reinforcement being withheld (attention). Escape extinction, a term used to describe extinction of escape-maintained behavior (Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990), specifies the relevant process as well as the reinforcer, but not the pro- cedure. Finally, sensory extinction, which refers to a variety of techniques designed to attenuate stim- ulation directly produced by a response (Rincover, 1978; Rincover, Cook, Peoples, & Packard, 1979), has the same descriptive characteristics as escape extinction (it specifies process and reinforcer, but not procedure); in addition, the wording is awk- ward because nothing "sensory" is extinguished. In an attempt to promote brevity as well as con- sistency of terminology, we have adopted through- out this paper the convention of referring to dif- ferentfunctional variations of extinction by using the abbreviation "EXT" followed in parentheses by the source of reinforcement withheld, as in EXT (attention), EXT (escape), and EXT (sensory). We chose "EXT" (attention) over "attention EXT" because the former places emphasis on behavioral process, with the reinforcer incidental, whereas the latter connotes that attention is extinguished.
  • 115. This terminology is still incomplete because it does not differentiate procedural variations within 132 VARIATIONS OF EXTINCTION a given functional class of extinction techniques. For example, EXT (attention) may take different forms depending on the situational context in which it is applied. If the target behavior occurs in the absence of ongoing social interaction, EXT (atten- tion) requires no response from the therapist and simply consists of withholding attention that pre- viously followed the behavior. A more complicated situation exists if the target behavior occurs while the therapist is interacting with the client. In this case, withholding reinforcement [EXT (attention)] would consist of terminating the interaction.' Be- cause similar variations are also characteristic of both EXT (escape) and EXT (sensory), we did not attempt to specify any particular procedure by way of descriptive label; this practice is consistent with
  • 116. terms such as DRO, time-out, punishment, and so forth, all of which require further elaboration. A reasonable conclusion based on descriptions of behavior, procedure, and outcome reported in the above studies is that extinction designed for SIB serving one function may have little or no therapeutic effect on SIB serving a different func- tion. Given the variations of extinction described previously and the maintaining contingencies for which they might be used, it is possible to construct a contingency matrix such as that found in Figure 1, which allows predictions about behavioral out- come. For example, elimination of social reinforce- ment for SIB using EXT (attention) or EXT (es- cape) will not interfere with the delivery of sensory (automatic) reinforcement, and EXT (sensory) would have no effect on social sources of reinforce- ment. Most seriously, EXT (attention) applied to escape-maintained SIB (i.e., stimulus removal con- tingent on escape behavior) and, conversely, EXT (escape) applied to attention-maintained SIB (i.e., stimulus continuation contingent on attention-get- ting behavior) may be countertherapeutic and se- riously exacerbate the behavior problem. At the
  • 117. In this example, it is unclear if the subsequent change in behavior reflects withholding of contingent reinforcement that maintained SIB (extinction) or removal of ongoing re- inforcement (time-out). In either case, and from a practical standpoint, the only means of not reinforcing the behavior consists of terminating interaction. FUNCTIONAL VARIATIONS OF EXTINCTION 0 LL 0 z LU 0 z 1 z 0 0 z Ir.2 f ifis_---_..Treatment
  • 118. Figure 1. Expected outcomes when functional variations of extinction are applied to different maintaining contingen- cies for SIB. present time, evidence directly supporting these pre- dictions can be found in only a few studies. Research on the treatment of self-injurious escape behavior, for example, sometimes has included a baseline condition in which time-out from educational tasks was made contingent on SIB (Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990; Repp et al., 1988; Steege et al., 1990). In effect, this condition pro- cedurally amounted to EXT (attention) but was associated with increases in escape-maintained SIB. In the present study, we provide an empirical evaluation of the key predictions found in Figure 1 by conducting a comparative analysis of extinc- tion techniques. The relative effects of three func- tional variations of extinction were examined when applied to the same topography of SIB maintained by three different contingencies of reinforcement. METHOD
  • 119. Subjects and Setting Subjects were selected for participation based on (a) their having similar topographies of SIB, and (b) the outcomes of functional analysis baselines indicating that their SIB was maintained by dif- ferent sources of reinforcement. Three children with developmental disabilities who met these criteria were included as participants. All of them exhibited head banging that produced contusions or lacera- tions on the head and face. Donnie, a 7-year-old boy with severe mental retardation, was ambulatory 133 BRIAN A. IWATA et al. and could feed himself. He did not follow vocal instructions, although he had a fairly good imitative repertoire, and he communicated his needs by pointing. Donnie's parents could not identify any situations that were predictive of either high or low
  • 120. levels of SIB. His previous treatments consisted of response interruption, redirection to another activ- ity, time-out, and differential reinforcement. Jack, a 12-year-old boy with severe mental retardation, was nonambulatory but could move his wheelchair over short distances, feed himself, and manipulate small objects. He followed a few instructions and communicated primarily through idiosyncratic ges- ture (i.e., reaching and making a high-pitched vocal noise). Jack's parents and teachers reported that his SIB seemed to occur as part of a tantrum, although they could not reliably describe the types of situ- ations in which tantrums occurred. His previous treatments consisted of time-out and differential reinforcement. Millie, an 8-year-old girl with mod- erate mental retardation, was ambulatory, could feed herself, and occasionally followed simple in- structions. Although she did not exhibit any ex- pressive language, she appeared to be socially re- sponsive to adults (e.g., she approached adults when they were nearby, smiled at them, etc.). Millie's parents indicated that her SIB occurred "most of the time," but became worse when she did not "get her way." Her previous treatments consisted of verbal reprimands, response interruption, man-
  • 121. ual restraint, redirection, and differential reinforce- ment. She was also placed in a hard seizure helmet as a means of protection when her SIB was deemed to be "uncontrollable." Sessions were usually conducted individually in therapy rooms containing one-way observation win- dows, but occasionally (due to scheduling neces- sities) were run in a large group area where two or three other clients and one or two other staff mem- bers were present but located in a different part of the room (i.e., at least 6 m away). This periodic change in location seemed to have no effect on the subjects' behavior. Sessions were 15 min in duration and usually were conducted 5 days per week, with four to eight sessions daily and at least 15-min breaks between sessions. Response Measurement and Reliability SIB was defined as any audible contact between the head or face and either a fist or an object (e.g., furniture or walls). Data were recorded using one of two methods: (a) on paper during continuous 10-s intervals, which were cued by cassette tape;
  • 122. or (b) on a hand-held computer (Panasonic Model RL-H1800). All data were converted into per- centage of 10-s intervals during which one or more instances of SIB occurred. Data were also collected on a variety of experimenter behaviors (delivery of instructions or attention, withdrawal of instructions or attention, placement and removal of apparatus) as a means of monitoring procedural consistency; these data indicated that experimenter compliance exceeded 90% across all categories. Interobserver agreement was assessed by having a second observer simultaneously but independently record data during 23% of all sessions. Agreement percentages were calculated based on interval-by- interval comparison of observers' records by divid- ing the number of agreements by the number of agreements plus disagreements and multiplying by 100%. Mean agreement was 92% across subjects, with all values exceeding 86%. Experimental Designs and Sequence During the initial baseline, subjects were exposed to a series of assessment conditions in a multiele-
  • 123. ment design (Sidman, 1960; Ulman & Sulzer-Aza- roff, 1975) to identify the maintaining variables for their SIB. Subsequently, each subject was ex- posed to two or more functional variations of ex- tinction by way of reversal or multiple baseline designs. Because the specific conditions and their order of presentation were determined by subjects' baseline performances, more complete details are provided in the Results section for each subject. Functional Analysis Baseline Subjects were exposed to four conditions based on Iwata, Dorsey, Slifer, Bauman, and Richman (1982). A brief description of each condition is provided here. Attention. This condition was designed to test 134 VARIATIONS OF EXTINCTION
  • 124. for positive reinforcement (attention) as a main- taining variable for SIB. An experimenter initially placed the subject in contact with toys that were in the room, then proceeded to ignore the subject while seeming to do paperwork. Contingent on the occurrence of SIB, the experimenter expressed con- cern (e.g., "Stop. Don't do that; you'll hurt your- self.") and briefly interrupted the response. Demand. This condition was a test for negative reinforcement (escape from demands) as a main- taining variable. An experimenter presented aca- demic tasks (e.g., object identification, sorting, drawing, and other tasks requiring simple motor responses) comparable to those found in the sub- ject's individual educational plan. Tasks were pre- sented in a discrete-trial format approximately once every 30 s. The experimenter used modeling and physical guidance as supplementary prompts to produce compliance, delivered praise contingent on correct responses, and implemented a 30-s time- out contingent on the occurrence of SIB. The time- out consisted of removing the materials from the table and turning away from the subject.
  • 125. Alone. By restricting access to social and material sources of stimulation, this condition provided a test for SIB maintained by nonsocial (automatic) reinforcement. The subject was observed while alone in the room (which was empty except for a chair or couch). Play. This condition served as a control. The experimenter provided continuous access to toys, delivered praise approximately every 30 s contin- gent on the absence of SIB, and ignored occurrences of SIB. Extinction Conditions Three variations of extinction were evaluated, each designed to discontinue a different source of reinforcement for SIB. An attempt was made to implement each type of extinction with each sub- ject, but this was not always possible if (a) little or no SIB occurred in a given baseline condition, or (b) the baseline condition itself was not amenable to implementation of the procedure as described. For example, it was not possible to prevent escape from tasks (i.e., to withhold negative reinforce-
  • 126. ment) during the alone baseline, because no tasks were presented initially. EXT (attention). This variation of extinction was designed to discontinue positive reinforcement in the form of attention delivered by the experi- menter. EXT (attention) already was a component of three of the baseline conditions: during the de- mand condition, attention was withdrawn contin- gent on SIB; during the alone condition, no adult was present to deliver any attention; and during the play condition, attention was withheld contin- gent on SIB. Thus, the attention condition was the only baseline on which EXT (attention) could be introduced as a new manipulation. This extinction contingency consisted of having the experimenter ignore (not attend to) SIB, and was combined with experimenter attention for the absence of SIB, which was delivered on a 30-s DRO schedule. Occur- rences of SIB reset the DRO interval. EXT (escape). This variation of extinction dis- continued negative reinforcement in the form of escape from tasks. It was implemented during the
  • 127. demand condition, which was the only condition containing a task requirement. During EXT (es- cape), learning trials continued as during baseline but were not terminated contingent on SIB. Instead, the experimenter physically guided the subject to comply with the instruction and continued the ses- sion according to schedule. It has been noted pre- viously (Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990) that the physical guidance component of this intervention may represent a punishment con- tingency. However, (a) guidance was the inevitable consequence of noncompliance even during base- line, and (b) behavior change observed following implementation of this procedure in the above study as well as in others (e.g., Goh & Iwata, 1994; Repp et al., 1988) showed either temporary burst- ing or gradual decreases consistent with extinction processes. EXT (sensory). This variation of extinction was designed to attenuate the sensory consequences pro- duced by subjects' head banging, and could be implemented across any of the baseline conditions. Each subject was fitted with an oversized helmet (either a seizure or a lacrosse helmet), which was
  • 128. 135 BRIAN A. IWATA et al. reinforced with additional padding. During EXT (sensory), the helmet was placed on the subject's head at the beginning of a session and remained in place throughout (subjects never attempted to remove the helmets). An extension of this "non- contingent" helmet condition was implemented for Donnie and is described below. RESULTS Donnie Figure 2 shows the results obtained for Donnie. He exhibited a considerable amount of SIB across all of the baseline conditions, thus permitting a series of manipulations on each baseline. Moderate levels of SIB were observed during the demand, attention, and play conditions (lower three panels),
  • 129. whereas SIB occurred almost continuously during the alone condition (upper panel) containing no access to social stimulation, play materials, or tasks. These data suggested that Donnie's SIB was main- tained by factors independent of the social and physical environment and were consistent with be- havior reinforced by directly produced, automatic consequences. On the alone baseline (top panel), EXT (sensory) was implemented in a reversal (ABAB) design, and results showed decreases in SIB associated with the helmet intervention. The final condition on the alone baseline involved two modifications of the EXT (sensory) procedure. First, play materials that Donnie was observed to manipulate during the original play baseline were introduced. Second, the protective helmet was applied only contingent on the occurrence of SIB. As long as SIB did not occur, Donnie was free to play with the toys and move about without wearing the helmet. When SIB oc- curred, an experimenter, who was in the room but did not interact with Donnie during the session, placed the helmet on Donnie for a 2-min time- out, during which the toys were removed from
  • 130. sight. These procedural modifications were based on results reported by Dorsey, Iwata, Reid, and Davis (1982), who found that it was possible to transfer control of SIB from a noncontingent to a contingent helmet condition. SIB increased initially when this change was made on the alone baseline, but eventually decreased to a level similar to that observed during the EXT (sensory) conditions. On the demand baseline (second panel from the top), the introduction of EXT (sensory) was asso- ciated with a rapid and large decrease in Donnie's SIB. This reduction occurred in spite of the fact that the original baseline contingency for SIB- EXT (attention) or, alternatively, escape from the task contingent on SIB-remained in effect. Next, EXT (sensory) was removed and EXT (escape) was implemented; this resulted in an increase in SIB to its baseline level. Finally, the contingent helmet procedure was combined with EXT (escape). The experimenter presented learning trials as before and, contingent on SIB, guided Donnie through the task and then placed the helmet on him. The helmet remained on for approximately 2 min (four learning
  • 131. trials). This condition was associated with a decrease in SIB similar to that seen during the previous EXT (sensory) condition. On the attention baseline (third panel from the top), the introduction of EXT (attention) had no effect on Donnie's SIB. During the next condition, EXT (sensory), SIB decreased even though it pro- duced attention from the experimenter. The final procedure implemented on this baseline consisted of EXT (attention) combined with a 2-min time- out in the helmet contingent on occurrences of SIB, and was associated with a reduction in SIB similar to that seen in the EXT (sensory) condition. On the play baseline (bottom panel), which con- tained an EXT (attention) component, no decrease in SIB was observed until EXT (sensory) was in- troduced. Subsequently, SIB increased temporarily but decreased again when the procedure was changed to a 2-min time-out in the helmet contingent on SIB. Jack The initial baseline of Figure 3 shows the results
  • 132. of Jack's assessment. He exhibited little or no SIB except during the demand condition, which con- tained two distinctive features: the presentation of task demands and a brief time-out from the task contingent on SIB. The time-out component pro- 136 DONNIE 100- Baseline 800 n:n -a 40 - 20 - VARIATIONS OF EXTINCTION
  • 133. r EXT(sensory) 1 BL I Toys + Conting( -a- Ati -a-- -D -.*-- Al4 -a- Pli ent Helmet ttention emand lone lay 1 ---- -i- 40 EXT Contingent Helmet Baseline EXT(sensory) (escape) + EXT(escape) 20 o r , . . 40 Contingent Helmet
  • 134. Baseline EXT(attn) EXT(sensory) + EXT(attn) 20 40 Contingent Baseline EXT(sensory) Helmet 20 0 20 40 60 80 100 137 co) -J cn w H z uL
  • 135. 0 Lu w z w 0L 20 40 60 80 1 00 SESSIONS Figure 2. Percentage of intervals of SIB exhibited by Donnie across experimental conditions. ou -1 A II
  • 136. BRIAN A. IWATA et al. EXT(sensory) 100 co C,) -J Er w z LL 0 LLJ z LU cc
  • 137. LLJw- 80 60 40 20 0 -H0. SESSIONS Figure 3. Percentage of intervals of SIB exhibited by Jack across experimental conditions. cedurally resembled EXT (attention) but appeared to function as (negative) reinforcement for escape behavior. Because little or no SIB was observed during other conditions, all subsequent manipu- lations were made on the demand baseline. EXT (sensory) and EXT (escape) were implemented con-
  • 138. currently and resulted in a large decrease in Jack's SIB. A partial reversal was then conducted in which EXT (escape) was removed while EXT (sensory) remained in effect; this change resulted in an in- crease in Jack's SIB to its original baseline level. SIB decreased again when EXT (escape) was re- introduced, and it remained low following the re- moval of EXT (sensory). Millie Figure 4 shows the results obtained for Millie. During the initial baseline assessment, her SIB oc- curred almost exclusively during the attention con- dition, suggesting that the behavior was maintained by social-positive reinforcement. Very little SIB was observed during the play and demand conditions, both of which contained a brief time-out from attention contingent on SIB, and SIB decreased throughout the alone condition, which contained no attention whatsoever. Thus, Millie's assessment data revealed not only a reinforcement effect during the attention condition but also an apparent ex- tinction effect during others.
  • 139. In light of these results, subsequent interventions were implemented only on the attention baseline. EXT (attention) with attention delivered a 30-s DRO schedule was introduced concurrent with EXT (sensory) and resulted in a large decrease in Millie's SIB. Next, while EXT (sensory) remained in effect, a reversal was conducted in which EXT (attention) + DRO was removed; this partial return to base- line was associated with an immediate and large increase in SIB. When EXT (attention) + DRO was reinstated, SIB decreased to near-zero levels. EXT (sensory) was then removed while EXT (at- tention) + DRO remained in effect, and no increase was observed in SIB. During this final condition, the length of the DRO interval was gradually in- creased to 1, 2, and finally 5 min, and SIB remained low across all schedule changes. 138 VARIATIONS OF EXTINCTION
  • 140. EXT(sensory) _,4,0MILLIE EXT(attn) + DRO Baseline BL -a Attention *- Demand -0- Alone 0 Play 20 40 60 80 SESSIONS Figure 4. Percentage of intervals of SIB exhibited by Millie across experimental conditions. Summary of Key Comparisons Table 1 contains a summary of the results from selected baseline and extinction conditions for Don- nie, Jack, and Millie. The data reflect reductions or increases in SIB expressed as mean percentages of baseline responding (EXT mean/baseline mean) for relevant and irrelevant extinction conditions
  • 141. compared to their appropriate baselines. Because SIB during relevant extinction conditions usually showed decreasing trends, comparisons based on overall condition means would not accurately depict end-of-treatment responding. Therefore, all per- centages are based on the last five sessions of a condition. For example, under "Relevant EXT," Donnie's mean percentage of SIB during the last five sessions of EXT (sensory) occurred at 0.5% of the mean SIB during the last five sessions of his alone baseline, whereas under "Irrelevant EXT," his five-session mean percentage of SIB during EXT (escape) occurred at 81.9% of his demand baseline mean. These comparisons show that SIB for all subjects was reduced to below 15% of its baseline Table 1 Mean Change (Reduction or Increase) in SIB Expressed as a Percentage of Baseline Responding for Relevant and Irrelevant Extinction Conditions Compared to Their Relevant Baselines. Percentages Are Based on the Last Five Sessions of a Condition
  • 142. Relevant EXT / baseline Irrelevant EXT / baseline Subject Conditions compared % of BL Conditions compared % of BL Donnie EXT (sensory) / alone BL 0.5 EXT (escape) / demand BL 81.9 EXT (sensory) / demand BL 6.9 EXT (attn) / attn BLa 81.7 EXT (sensory) / attn BLa 14.3 EXT (sensory) / play BL 8.2 Jack EXT (escape) / demand BL 5.4 EXT (sensory) / demand BL 75.8 Millie EXT (attn) / attn BL 3.6 EXT (sensory) / attn BL 126.3 Mean change as % of baseline SIB 6.5 91.4 a Donnie's attention baseline consisted of only four sessions. 139 co CO C,) -J
  • 144. BRIAN A. IWATA et al. mean during relevant extinction conditions (mean reduction = 6.5% of baseline), but was never re- duced to lower than 75% of baseline during irrel- evant extinction conditions (mean reduction = 91.4% of baseline). Follow-Up At the completion of the study, a treatment program designed for each subject was imple- mented throughout the day in the appropriate sit- uational contexts. Donnie's program consisted of several components. First, free access to toys was provided whenever possible (except during training sessions, self-care routines, etc.). Second, occur- rences of SIB were followed by wearing his padded helmet for 2 min. Finally, and as a preventive measure, the contingent helmet procedure was com- bined with either EXT (attention) or EXT (escape) depending on the situational context. If Donnie
  • 145. exhibited SIB while participating in training, the session was continued while he wore the helmet. If SIB occurred at other times, the helmet was com- bined with time-out from social interaction. Jack's treatment consisted of EXT (escape) identical to that described previously, and it was implemented during all training activities. Millie's treatment con- sisted of attention contingent on appropriate social initiations, defined as any approach behavior while not engaged in SIB. A 5-min DRO contingency also was implemented with attention as the rein- forcing consequence. If Millie exhibited SIB while an adult was interacting with her, the adult would turn away and ignore her until (a) Millie ap- proached while not engaging in SIB, or (b) she successfully completed a 5-min DRO interval. Thus, Millie's program contained an explicit EXT (at- tention) component. Subjects' parents and teachers were instructed in the use of the above procedures until they dem- onstrated proficiency in implementation. Follow- up contacts were faded over a 6-month period, during which SIB occurred at well below baseline levels for all subjects.
  • 146. DISCUSSION Results of this study showed that therapeutic techniques previously defined as extinction in the applied literature are different not only procedurally but also in their behavioral effects when applied to the same response topography. Each of three vari- ations of extinction-EXT (attention), EXT (es- cape), and EXT (sensory)-reduced SIB only when it resulted in discontinuation of the specific source of reinforcement maintaining the behavior. Thus, the process of extinction is not tied to any particular form of intervention (e.g., ignoring undesirable be- havior); instead, it is determined by the behavior's maintaining contingency. Moreover, the data pre- sented here indicate that if the source of reinforce- ment for a behavior disorder such as SIB can be identified on a pretreatment basis, much of the guesswork is removed from the task of selecting an appropriate extinction technique. Results obtained for Donnie provide the most complete analysis in the present study because he
  • 147. was exposed to all three functional variations of extinction. Neither EXT (attention) nor EXT (es- cape) was effective in reducing Donnie's SIB. His head banging, which apparently was maintained by sensory consequences, was reduced across all baseline conditions when EXT (sensory) was ap- plied, even though SIB still produced attention and escape during the attention and demand baselines, respectively. Jack, for whom EXT (escape) and EXT (sensory) were evaluated explicitly as treat- ments, was also exposed implicitly to EXT (atten- tion). The brief time-out contingent on SIB during his initial demand baseline closely resembled EXT (attention), yet it actually amounted to negative reinforcement and resulted in the highest levels of SIB when compared to other assessment conditions. EXT (sensory) also had little suppressive effect on Jack's SIB, which decreased only when it no longer produced termination of learning trials during the EXT (escape) condition. For Millie, whose baseline data indicated that her SIB was maintained by positive reinforcement in the form of attention, effective treatment consisted of EXT (attention) combined with DRO (attending to her when she did not exhibit SIB), whereas EXT (sensory) had
  • 148. no apparent effect on her behavior. Millie's results are limited in two respects. First, a stronger dem- onstration of extinction effects would have been provided had EXT (attention) not been combined 140 VARIATIONS OF EXTINCTION with the DRO procedure, although recent data (Mazaleski et al., 1993) suggest that extinction is the major therapeutic component of DRO contin- gencies. Second, because little or no SIB was ob- served during her demand baseline, it was not pos- sible to examine the effects of EXT (escape) on her behavior. It is important to note that the procedures used in this study, although labeled "EXT" for each subject, differed in two respects. First, because only one variation of extinction was effective in reducing SIB (a different one for each subject), the other two variations did not functionally amount to extinc-
  • 149. tion. We labeled all procedures similarly to reduce confusion and to emphasize the facts that (a) each procedure has been described as an extinction tech- nique in the literature, and (b) procedures so labeled might not produce reductions in behavior if they do not terminate the behavior's maintaining con- tingency. Based on the results obtained during as- sessment and treatment, a more accurate description of the procedures used with each subject might be as follows. Donnie was exposed to EXT (sensory), ignoring SIB or terminating interaction contingent on SIB [labeled EXT (attention)], and task con- tinuation and prompting contingent on SIB [la- beled EXT (escape)). Jack was exposed to EXT (escape), a helmet condition [labeled EXT (sen- sory)1, and terminating interaction contingent on SIB (his demand baseline). Millie was exposed to EXT (attention) and a helmet condition [labeled EXT (sensory)}. A second difference can be seen within one of the extinction components. EXT (es- cape) and EXT (sensory) were applied in a consis- tent manner during treatment sessions. EXT (at- tention), however, took two different forms: ignoring SIB that occurred in the absence of interaction, and
  • 150. terminating ongoing interaction contingent on the occurrence of SIB. These differences illustrate a point already noted: Extinction procedures can be defined with respect to both function (the rein- forcement contingency being discontinued) and, within function, form (specific therapist actions). The lack of therapeutic effects observed when a given variation of extinction was applied to SIB maintained by a different (irrelevant) source of re- inforcement provides new data on some of the limiting conditions of extinction. On the other hand, the positive treatment effects observed during rel- evant extinction conditions were not particularly novel. Previous studies have reported reductions in SIB when the reinforcer withheld during extinction was relevant to behavioral maintenance: EXT (at- tention) for attention-maintained SIB (e.g., Lovaas & Simmons, 1969), EXT (escape) for escape-main- tained SIB (e.g., Iwata, Pace, Kalsher, Cowdery, & Cataldo, 1990), and EXT (sensory) for auto- matically reinforced or self-stimulatory SIB (e.g., Rincover & Devaney, 1982). Nevertheless, these findings have not given rise to a consistent termi-
  • 151. nology that distinguishes one type of extinction from another, they have not been adopted by au- thors of most textbooks on applied behavior anal- ysis (see Cooper, Heron, & Heward, 1987, for a notable exception), and they have not been incor- porated into guidelines regulating the use of be- havioral interventions (e.g., Florida HRS Manual 160-4, 1989). Extinction is similar to other behavioral pro- cesses, such as reinforcement or punishment, from which a number of therapeutic techniques can be derived. Although this fact applies to all behavioral interventions, the case of punishment is of greatest concern because some of its procedural variations are considered to be highly intrusive or otherwise unacceptable (O'Brien & Karsh, 1990). As a result, administrative and treatment manuals place a great deal of emphasis on the topographical aspects of intervention in order to promote procedural con- sistency by specifying the acceptable limits of ther- apist behavior when implementing a given tech- nique. Although this approach to treatment classification results in a well-defined system of pro- cedures, behavioral effects may be difficult to pre-
  • 152. dict because therapist behavior, although clearly defined, may enter into multiple contingencies with respect to client behavior. Thus, "planned ignor- ing" functions as EXT (attention) for attention- seeking behavior but as negative reinforcement for escape behavior, whereas procedures such as "re- direction" or other attempts to prompt compliance with task demands function as EXT (escape) for escape behavior but as positive reinforcement for attention-seeking behavior. Functions other than 141 BRIAN A. IWATA et al. EXT (sensory) that might be attributed to the hel- met intervention used in this study are not entirely clear at the present time, but they might include punishment (wearing the apparatus as an aversive event), social reinforcement (attention paired with equipment placement), or time-out (overall re- duced access to reinforcement while wearing the equipment).
  • 153. Problems may arise when any of these procedures serves unintended functions such as those just de- scribed. At best, these discrepancies may result in the use of ineffective treatments described as ex- tinction (e.g., as in ignoring automatically rein- forced behavior); at worst, they may result in the use of highly inappropriate treatments described as extinction (as in ignoring or time-out for escape behavior). In either case, failure to distinguish be- tween the principle of extinction and the ways in which it may be applied can lead to the erroneous conclusion that "extinction" is a relatively ineffec- tive intervention. When these errors are formalized by way of regulation, an unfortunate state of affairs may result: Procedural manuals designed to protect persons from intrusive interventions (e.g., punish- ment) to reduce behavior may require the use of nonintrusive interventions (e.g., misapplied ignor- ing or attention) that strengthen dangerous behav- ior to the point that the intrusive intervention is required. Although the results of this study are limited to consideration of extinction effects, they have im-
  • 154. plications for the design of interventions based on other learning mechanisms. For example, behav- ioral function is highly relevant to the selection of "reinforcers" delivered and withheld in differential reinforcement contingencies (Carr & Durand, 1985), and although social reprimands often have been found to function as punishment, they can also serve as positive reinforcement (Van Houten & Doleys, 1983). Thus, data from a number of stud- ies on the functional analysis of behavior disorders indicate that current methods for classifying inter- ventions are inadequate. There is a critical need to develop classification systems and treatment man- uals that specify not only procedures, topographies for which they may be used, and preferred hierarchy based on consideration of factors such as "intru- siveness," but also how the same principle of learn- ing may be translated into different procedures based on differences in behavioral function (see Iwata, Vollmer, Zarcone, & Rodgers, 1993, and Repp & Karsh, 1990, as examples). Because the present results indicated that iden- tification of a behavior's maintaining reinforcers was
  • 155. a prerequisite to the appropriate use of extinction (unknown reinforcers cannot be withheld), it ap- pears that an answer to the question "Is a functional analysis of a behavior problem useful in developing an effective treatment program?" can be provided by way of data. This represents a significant im- provement over other answers based on theory, logic, or humane philosophy without direct em- pirical evidence. But extinction represents only one approach to treatment. There is ample evidence in both the basic and applied literature that punishment can override the effects of reinforcement. However, almost all studies involving the use of punishment to reduce behavior problems contain a limitation noted pre- viously: Administration of punishment is con- founded with the termination of reinforcement. Ex- tinction is relevant to a consideration of punishment effects because, from the standpoint of methodol- ogy, the suppressive properties of a punishing stim- ulus should be evaluated while behavior is con- currently reinforced, unless it can be shown first that extinction alone does not reduce behavior. Such an arrangement increases the likelihood that ob-
  • 156. served reductions in behavior can be attributed to punishment rather than to extinction, which seems to be important as a justification for using punish- ment over extinction. At the present time, it is unclear that punishment so applied would have the same suppressive effects as those typically reported in the literature. This fact, in addition to the pos- sibility that stimuli delivered as punishment could function as reinforcement, underscores the relevance of both functional analysis procedures and extinc- tion controls to research on punishment. With respect to reinforcement-based interven- tions, the finding that arbitrary or irrelevant rein- forcers-those that do not maintain the target be- 142 VARIATIONS OF EXTINCTION 143 havior-delivered in a DRO or DRA schedule successfully competed with an intact baseline of reinforcement for the behavior problem (i.e., the
  • 157. target behavior still produced the maintaining re- inforcer) argues against the necessity of identifying the maintaining reinforcer. That is, if reinforcers could be found that are different than but substi- tutable for those maintaining the behavior problem, the task involves merely finding powerful reinforc- ers. Several studies on DRO as treatment for SIB have shown that reinforcer substitutability can be achieved (e.g., Corte, Wolf, & Locke, 1971; Cow- dery, Iwata, & Pace, 1990), but results from other studies indicate that reinforcers irrelevant to be- havioral function do not always compete success- fully with relevant ones (e.g., Carr & Durand, 1985). Thus, it appears that reinforcer substitut- ability is possible under conditions that remain to be identified at the present time (see Green & Freed, 1993, for a general review of the topic). Never- theless, it seems that a search for those conditions should include specification of the maintaining re- inforcer for which another stimulus would serve as substitute, and that the evaluation of substitution effects should include an extinction control (see Mazaleski et al., 1993, for an example of this methodology).
  • 158. Extinction alone is rarely recommended and per- haps accounts for the fact that little research has been conducted on extinction effects per se. In light of the present results indicating that properly de- signed extinction procedures can produce powerful clinical effects, future research on the treatment of severe behavior disorders should include more care- ful consideration of extinction as an integral com- ponent of intervention as well as a significant source of confounding effects when evaluating both re- inforcement- and punishment-based approaches to treatment. REFERENCES Carr, E. G., & Durand, V. M. (1985). Reducing behavior problems through functional communication training. Journal of Applied Behavior Analysis, 18, 111-126. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus, OH: Merrill. Corte, H. E., Wolf, M. M., & Locke, B. J. (1971). A comparison of procedures for eliminating self-injurious
  • 159. behavior of retarded adolescents. Journal of Applied Behavior Analysis, 4, 201-2 13. Cowdery, G. E., Iwata, B. A., & Pace, G. M. (1990). Effects and side effects of DRO as treatment for self-injurious behavior. Journal of Applied Behavior Analysis, 23, 497-506. Day, R. M., Rea, J. A., Schussler, N. G., Larsen, S. E., & Johnson, W. L. (1988). A functionally based approach to the treatment of self-injurious behavior. Behavior Modification, 12, 565-589. Dorsey, M. F., Iwata, B. A., Reid, D. H., & Davis, P. A. (1982). Protective equipment: Continuous and contin- gent application in the treatment of self-injurious be- havior.Journal ofApplied Behavior Analysis, 15, 217- 230. Goh, H., & Iwata, B. A. (1994). Behavioral persistence and variability during extinction of self-injury maintained by escape. Journal of Applied Behavior Analysis, 27, 175-176. Green, L., & Freed, D. E. (1993). The substitutability of
  • 160. reinforcers. Journal of the Experimental Analysis of Behavior, 60, 141-158. Florida HRS manual 160-4. (1989, April). Tallahassee, FL: Florida Department of Health and Rehabilitative Services, Developmental Services Program Office. Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1982). Toward a functional analysis of self-injury. Analysis and Intervention in Develop- mental Disabilities, 2, 3-20. Iwata, B. A., Pace, G. M., Kalsher, M. J., Cowdery, G. E., & Cataldo, M. F. (1990). Experimental analysis and extinction of self-injurious escape behavior. Journal of Applied Behavior Analysis, 23, 11-27. Iwata, B. A., Vollmer, T. R., & Zarcone, J. R. (1990). The experimental (functional) analysis of behavior dis- orders: Methodology, applications, and limitations. In A. C. Repp & N. N. Singh (Eds.), Perspectives on the use of nonaversive and aversive interventions for persons with developmental disabilities (pp. 301-331). Syca- more, IL: Sycamore Publishing Co.
  • 161. Iwata, B. A., Vollmer, T. R., Zarcone, J. R., & Rodgers, T. A. (1993). Treatment classification and selection based on behavioral function. In R. Van Houten & S. Axelrod (Eds.), Behavior analysis and treatment (pp. 102-12 5). New York: Plenum. LaVigna, G. W., & Donnellan, A. M. (1986). Alternatives to punishment: Solving behavior problems with non- aversive strategies. New York: Irvington. Lovaas, 0. I., & Simmons, J. Q. (1969). Manipulation of self-destruction in three retarded children. Journal of Applied Behavior Analysis, 2, 143-157. Mace, F. C., & Belfiore, P. (1990). Behavioral momentum in the treatment of escape-motivated stereotypy. Journal of Applied Behavior Analysis, 23, 507-514. Mace, F. C., Lalli, J. S., & Lalli, E. P. (1991). Functional analysis and treatment of aberrant behavior. Research in Developmental Disabilities, 12, 15-180. Mazaleski, J. L., Iwata, B. A., Vollmer, T. R., Zarcone, J. R., & Smith, R. G. (1993). Analysis of the reinforce-
  • 162. 144 BRIAN A. IWATA et al. ment and extinction components of DRO contingencies with self-injury. Journal of Applied Behavior Analysis, 26, 143-156. Nelson, C. M., & Rutherford, R. B. (1983). Timeout revisited: Guidelines for its use in special education. Ex- ceptional Education Quarterly, 3, 56-67. O'Brien, S., Karsh, K. G. (1990). Treatment acceptability: Consumer, therapist, and society. In A. C. Repp & N. N. Singh (Eds.), Perspectives on the use of nonaversive and aversive interventions for persons with develop- mental disabilities (pp. 504-516). Sycamore, IL: Syc- amore Publishing Co. Plummer, S., Baer, D. M., & LeBlanc,J. M. (1977). Func- tional considerations in the use of time out and an ef- fective alternative. Journal of Applied Behavior Anal- ysis, 10, 689-705. Repp, A. C., Felce, D., & Barton, L. E. (1988). Basing
  • 163. the treatment of stereotypic and self-injurious behaviors on hypotheses of their causes. Journal of Applied Be- havior Analysis, 21, 281-289. Repp, A. C., & Karsh, K. G. (1990). A taxonomic ap- proach to the nonaversive treatment of maladaptive be- havior of persons with developmental disabilities. In A. C. Repp & N. N. Singh (Eds.), Perspectives on the use of nonaversive and aversive interventions for persons with developmental disabilities (pp. 331-347). Syca- more, IL: Sycamore Publishing Co. Rincover, A. (1978). Sensory extinction: A procedure for eliminating self-stimulatory behavior in developmentally disabled children. Journal of Abnormal Child Psychol- ogy, 6, 299-310. Rincover, A., Cook, R., Peoples, A., & Packard, D. (1979). Sensory extinction and sensory reinforcement principles for programming multiple adaptive behavior change. Journal of Applied Behavior Analysis, 12, 221-233. Rincover, A., & Devany, J. (1982). The application of sensory extinction procedures to self-injury. Analysis and Intervention in Developmental Disabilities, 2, 67-81.
  • 164. Sidman, M. (1960). Tactics of scientific research. New York: Basic Books. Solnick, J. V., Rincover, A., & Peterson, C. R. (1977). Some determinants of the reinforcing and punishing ef- fects of time out. Journal ofApplied Behavior Analysis, 10, 415-424. Steege, M. W., Wacker, D. P., Cigrand, K. C., Berg, W. K., Novak, C. G., Reimers, T. M., Sasso, G. M., & DeRaad, A. (1990). Use of negative reinforcement in the treatment of self-injurious behavior. Journal of Ap- plied Behavior Analysis, 23, 459-467. Ulman, J. D., & Sulzer-Azaroff, B. (1975). Multielement baseline design in educational research. In E. Ramp & G. Semb (Eds.), Behavior analysis: Areas of research and application (pp. 359-376). Englewood Cliffs, NJ: Prentice-Hall. Van Houten, R., & Doleys, D. M. (1983). Are social reprimands effective? In S. Axelrod & J. Apsche (Eds.), The effects ofpunishment on human behavior (pp. 45- 70). New York: Academic Press.
  • 165. Wacker, D. P., Steege, M. W., Northup, J., Sasso, G., Berg, W., Reimers, T., Cooper, L., Cigrand, K., & Donn, L. (1990). A component analysis of functional commu- nication training across three topographies of severe be- havior problems.Journal ofApplied Behavior Analysis, 23, 417-429. Zarcone, J. R., Iwata, B. A., Hughes, C. E., & Vollmer, T. R. (1993). Momentum versus extinction effects in the treatment of self-injurious escape behavior. Journal of Applied Behavior Analysis, 26, 135-136. Received March 12, 1993 Initial editorial decision May 4, 1993 Revisions received July 23, 1993; August 6, 1993 Final acceptance August 26, 1993 Action Editor, F. Charles Mace Chapter 4: Clustering I
  • 166. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is Cluster Analysis? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups
  • 167. Inter-cluster distances are maximized Intra-cluster distances are minimized
  • 168. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Applications of Cluster Analysis Understanding Group related documents for browsing, group genes and proteins that have similar functionality, or group stocks with similar price fluctuations Summarization Reduce the size of large data sets Clustering precipitation in Australia 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› What is not Cluster Analysis? Simple segmentation
  • 169. Dividing students into different registration groups alphabetically, by last name Results of a query Groupings are a result of an external specification Clustering is a grouping of objects based on the data Supervised classification Have class label information 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Notion of a Cluster can be Ambiguous
  • 173. Six Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusterings A clustering is a set of clusters Important distinction between hierarchical and partitional sets of clusters Partitional Clustering A division of data objects into non-overlapping subsets (clusters) such that each data object is in exactly one subset Hierarchical clustering A set of nested clusters organized as a hierarchical tree
  • 174. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Partitional Clustering Original Points A Partitional Clustering
  • 175. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Hierarchical Clustering Traditional Hierarchical Clustering Non-traditional Hierarchical Clustering Non-traditional Dendrogram Traditional Dendrogram 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 176. Other Distinctions Between Sets of Clusters Exclusive versus non-exclusive In non-exclusive clusterings, points may belong to multiple clusters. Can represent multiple classes or ‘border’ points Fuzzy versus non-fuzzy In fuzzy clustering, a point belongs to every cluster with some weight between 0 and 1 Weights must sum to 1 Probabilistic clustering has similar characteristics Partial versus complete In some cases, we only want to cluster some of the data Heterogeneous versus homogeneous Clusters of widely different sizes, shapes, and densities 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters Well-separated clusters Center-based clusters
  • 177. Contiguous clusters Density-based clusters Property or Conceptual Described by an Objective Function 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Well-Separated Well-Separated Clusters: A cluster is a set of points such that any point in a cluster is closer (or more similar) to every other point in the cluster than to any point not in the cluster. 3 well-separated clusters
  • 178. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Center-Based Center-based A cluster is a set of objects such that an object in a cluster is closer to the “center” of a cluster, than to the center of any other cluster The center of a cluster is a centroid. The average of all the points in the cluster is a medoid 4 center-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Contiguity-Based
  • 179. Contiguous Cluster (Nearest neighbor or Transitive) A point in a cluster is closer (or more similar) to one or more other points in the cluster than to any point not in the cluster. 8 contiguous clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Density-Based Density-based A cluster is a dense region of points, which is separated by low-
  • 180. density regions, from other regions of high density. Used when the clusters are irregular or intertwined, and when noise and outliers are present. 6 density-based clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Conceptual Clusters Shared Property or Conceptual Clusters Finds clusters that share some common property or represent a particular concept. . 2 Overlapping Circles
  • 181. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Types of Clusters: Objective Function Clusters Defined by an Objective Function Finds clusters that minimize or maximize an objective function. Enumerate all possible ways of dividing the points into clusters and evaluate the `goodness' of each potential set of clusters by using the given objective function. (NP Hard) Can have global or local objectives. Hierarchical clustering algorithms typically have local objectives Partitional algorithms typically have global objectives A variation of the global objective function approach is to fit the data to a parameterized model. Parameters for the model are determined from the data. Mixture models assume that the data is a ‘mixture' of a number of statistical distributions.
  • 182. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Characteristics of the Input Data Are Important Type of proximity or density measure Central to clustering Depends on data and application Data characteristics that affect proximity and/or density are Dimensionality Sparseness Attribute type Special relationships in the data For example, autocorrelation Distribution of the data Noise and Outliers Often interfere with the operation of the clustering algorithm 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 183. Clustering Algorithms K-means and its variants Hierarchical clustering Density-based clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering Partitional clustering approach Number of clusters, K, must be specified Each cluster is associated with a centroid (center point) Each point is assigned to the cluster with the closest centroid The basic algorithm is very simple
  • 184. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Example of K-means Clustering
  • 185. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› K-means Clustering – Details Initial centroids are often chosen randomly. Clusters produced vary from one run to another. The centroid is (typically) the mean of the points in the cluster. ‘Closeness’ is measured by Euclidean distance, cosine similarity, correlation, etc. K-means will converge for common similarity measures mentioned above. Most of the convergence happens in the first few iterations. Often the stopping condition is changed to ‘Until relatively few points change clusters’ Complexity is O( n * K * I * d ) n = number of points, K = number of clusters, I = number of iterations, d = number of attributes
  • 186. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Evaluating K-means Clusters Most common measure is Sum of Squared Error (SSE) For each point, the error is the distance to the nearest cluster To get SSE, we square these errors and sum them. x is a data point in cluster Ci and mi is the representative point for cluster Ci can show that mi corresponds to the center (mean) of the cluster Given two sets of clusters, we prefer the one with the smallest error One easy way to reduce SSE is to increase K, the number of clusters A good clustering with smaller K can have a lower SSE than a poor clustering with higher K
  • 187. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Two different K-means Clusterings Sub-optimal Clustering Optimal Clustering Original Points 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means K-means has problems when clusters are of differing Sizes Densities Non-globular Shapes K-means has problems when the data contains outliers/noise.
  • 188. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Sizes Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Differing Density
  • 189. Original Points K-means (3 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Limitations of K-means: Non-globular Shapes Original Points K-means (2 Clusters) 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations
  • 190. Original PointsK-means Clusters One solution is to use many clusters. Find parts of clusters, but need to put together. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Overcoming K-means Limitations Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 191. Overcoming K-means Limitations Original PointsK-means Clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids
  • 192. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids …
  • 193. 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Importance of Choosing Initial Centroids … 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› Problems with Selecting Initial Points If there are K ‘real’ clusters then the chance of selecting one centroid from each cluster is small. Chance is relatively small when K is large If clusters are the same size, n, then
  • 194. For example, if K = 10, then probability = 10!/1010 = 0.00036 Sometimes the initial centroids will readjust themselves in ‘right’ way, and sometimes they don’t Consider an example of five pairs of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#› 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›
  • 195. 10 Clusters Example Starting with two initial centroids in one cluster of each pair of clusters 02/14/2018 Introduction to Data Mining, 2nd Edition ‹#›