《判別與分類》PPT課件

《《判別與分類》PPT課件》由會員分享，可在線閱讀，更多相關《《判別與分類》PPT課件（115頁珍藏版）》請在裝配圖網上搜索。

1、111 Discriminant Analysis判別分析判別分析ec.3 兩正態總體分類ec.4 評估判別函數Sec.5 多總體分類2 The ideas associated with discriminant analysis can be traced back to the 1920s and work completed by the English statistician Karl Pearson,and others,on intergroup distances,e.g.,coefficient of racial likeness(CRL),(Huberty,1994).

2、In the 1930s R.A.Fisher translated multivariate intergroup distance into a linear combination of variables to aid in intergroup discrimination.Methodologists from Harvard University contributed much to the interest in application of discriminant analysis in education and psychology in the 1950s and

3、1960s(Huberty,1994).Klecka(1980)provided several historical references that deal mostly with early applications of DA.歷史2022-12-21中國人民大學六西格瑪質量管理研究中心3 目錄 上頁 下頁 返回 結束 第四章第四章 判別分析判別分析 回歸模型普及性的基礎在于用它去預測和解釋度量(metric)變量。但是對于非度量(nonmetric)變量，多元回歸不適合解決此類問題。本章介紹的判別分析來解決被解釋變量是非度量變量的情形。在這種情況下，人們對于預測和解釋影響一個對象所屬

4、類別的關系感興趣，比如為什么某人是或者不是消費者，一家公司成功還是破產等。判別分析在主要目的是識別一個個體所屬類別的情況下有著廣泛的應用。潛在的應用包括預測新產品的成功或失敗、決定一個學生是否被錄取、按職業興趣對學生分組、確定某人信用風險的種類、或者預測一個公司是否成功。在每種情況下，將對象進行分組，并且要求使用這兩種方法中的一種可以通過人們選擇的解釋變量來預測或者解釋每個對象的所屬類別。2022-12-21中國人民大學六西格瑪質量管理研究中心44.1 4.1 判別分析的基本理判別分析的基本理論論 有時會遇到包含屬性被解釋變量和幾個度量解釋變量的問題，這時需要選擇一種合適的分析方法。比如，我們

5、希望區分好和差的信用風險。如果有信用風險的度量指標，就可以使用多元回歸。但我們可能僅能判斷某人是在好的或者差的一類，這就不是多元回歸分析所要求的度量類型。當被解釋變量是屬性變量而解釋變量是度量變量時，判別分析是合適的統計分析方法。判別分析能夠解決兩組或者更多組的情況。當包含兩組時，稱作兩組判別分析。當包含三組或者三組以上時，稱作多組判別分析（Multiple discriminant analysis）。判別分析的假設條件 判別分析最基本的要求是,分組類型在兩組以上；在第一階段工作是每組案例的規模必須至少在一個以上。解釋變量必須是可測量的，才能夠計算其平均值和方差，使其能合理地應用于統計函數。

6、2022-12-21中國人民大學六西格瑪質量管理研究中心5 目錄 上頁 下頁 返回 結束 4.1 4.1 判別分析的基本理判別分析的基本理論論判別分析的假設之一，是每一個判別變量（解釋變量）不能是其他判別變量的線性組合。即不存在多重共線性問題。判別分析的假設之二，是各組變量的協方差矩陣相等。判別分析最簡單和最常用的形式是采用線性判別函數，它們是判別變量的簡單線性組合。在各組協方差矩陣相等的假設條件下，可以使用很簡單的公式來計算判別函數和進行顯著性檢驗。判別分析的假設之三，是各判別變量之間具有多元正態分布，即每個變量對于所有其他變量的固定值有正態分布。在這種條件下可以精確計算顯著性檢驗值和分組歸

7、屬的概率。當違背該假設時，計算的概率將非常不準確。6Overview Discriminant function analysis,a.k.a.discriminant analysis or DA,主要用于分類.好的判別函數,應該正確判斷率比較高.Discriminant function analysis is found in SPSS under Analyze,Classify,Discriminant.One gets DA or MDA from this same menu selection,depending on whether the specified groupin

8、g variable has two or more categories.7There are several purposes for DA and/or MDA:To classify cases into groups using a discriminant prediction equation.To test theory by observing whether cases are classified as predicted.To investigate differences between or among groups.To determine the most pa

9、rsimonious way to distinguish among groups.To determine the percent of variance in the dependent variable explained by the independents.To determine the percent of variance in the dependent variable explained by the independents over and above the variance accounted for by control variables,using se

10、quential discriminant analysis.To assess the relative importance of the independent variables in classifying the dependent variable.To discard variables which are little related to group distinctions.To infer the meaning of MDA dimensions which distinguish groups,based on discriminant loadings.8 Dis

11、criminant analysis has two steps:(1)F檢驗(Wilks lambda)可以用于檢驗判別模型是否顯著,(2)如F檢驗顯著,然后考察獨立變量在類別之間的差異，以便對依賴變量進行分類。Suppose an anesthesiologist needs to determine whether an anesthetic is safe for a person who is having a heart operation.Based on these kinds of criteria,the anesthesiologist would like to kno

12、w the following:can this knowledge be used to construct a rule that will classify new patients as to whether they are going to be safe or unsafe recipients of the anesthetic?what is the rule and can the rule be used to classify new patients?what are the chances of making mistakes when using the rule

13、?麻劑10Discriminant analysis 為用來建立規則一種多元技術，該技術能幫助樣本進行適當分類。Discriminant analysis 類似于回歸分析，但是其依賴變量或者被解釋變量為定性變量，而不是連續的。.Discriminant analysis is 也稱為分類分析.目的:從不同總體（或類別）中刻畫個體的特征。盡量從不同類別使用判別器或分類器分離開來.Goal of classification:把不同個體分類到不同類別中.問題是找到一個好的規則，能最優的對新個體進行分類！1211.2 兩總體分類兩總體分類主要問題(1)分類兩類個體 or(2)把新個體指派到其中一個類

14、別。記兩個類別為 1 and 2.The objects are separated or classified on the basis of measurements on p associated random variables X=X1,X2,Xp.The observed values of X differ to some extent from one class to the other.我們把第一類的個體看成一個總體 1 and 第二類的個體看成一個總體 2.這兩個總體對應的概率密度函數為f1(X)and f2(X),and consequently,這樣可以就可以討論如何

15、指定個體屬于那個類.Example 11.1 考慮某城鎮中兩類人群:1,割草機擁有者,and 2,those 不擁有者.In order to identify the best prospect for an intensive sales campaign,生產商 is interested in classifying families as prospective owners or nonowners on the basis of x 1=income and x 2=lot size.Random samples of n 1=12 current owners and n 2=

16、12 current nonowners are selected.The sample observations yield the scatter plot(Figure 11.1).Remark 1.一個好的判別方法應該產生少數錯誤分類.2.要考慮先驗概率.3.考慮誤判的成本或代價.(e.g.diagnose disease)基本思想 令 f1(X)and f2(X)分別為兩總體 1 and 2 對應的密度函數.我們的目的是要把X指定給其中一個總體中.令 為全空間.令R1 為x的一個集合，的一個集合，當x屬于R1時，我們把對象x分配給總體 1，反之如果屬于 R2=-R1 則分配給總體 2

17、.假定 集合 R1 和 R2 互斥，構成全空間.令 p1 為 1的先驗概率 and p2 為 2的先驗概率,其中 p1+p2 =1.那么 P(觀測對象被正確地劃入 1)=P(X R1|1)P(1)=P(1|1)p1 P(觀測對象被錯誤劃入 1)=P(X R1|2)P(2)=P(1|2)p2 P(觀測對象被正確劃入2)=P(X R2|2)P(2)=P(2|2)p2 P(觀測對象被錯誤劃入 2)=P(X R2|1)P(1)=P(2|1)p1 （11-3）錯分代價可以代價矩陣來表示:其中 c(2|1)為屬于 1 被錯誤劃入 2 的代價，and c(1|2)為屬于 2 被錯誤劃入 1 的代價.那么平均

18、的或期望的錯分代價為(ECM)ECM=c(2|1)P(2|1)p1+c(1|2)P(1|2)p2 (11-5)一個合理的分類法則應該有最小或盡可能小的ECM.結論結論 11.1.是ECM達到最小的區域R1 and R2 由下列不等式確定：We need to show that the regions R1 and R2 that minimize the ECM are defined by the vlues x for which the following inequalities hold:Substituting the expressions for P(2|1)and P(1|

19、2)into(11-5)givesWe get the result 11.1.11-7 假設有一個新觀測點 x 0，其中 f 1(x 0)=.3 and f 2(x 0)=.4.問該點應該劃入那個總體?Then 我們發現 x 0 R1，因此應該將其分入1 Other criteria 總錯誤概率（總錯誤概率（TPM）最小化原則）最小化原則。TPM=P(錯分 1 的觀測值或錯分 2 的觀測值)=p1 R1 f1(x)dx+p2 R2 f2(x)dx (11-8)數學上這個問題等價于在錯分代價相同情況下師期望錯分代價最小化。因此，這種情況下的最優區域由(11-7)中的(b)給出.最大后驗概率原則

20、 當 P(1|x0)P(2|x0)時,x0 劃入總體 1.注釋:相當于采用(11-7)中的總錯分概率的法則(b)，因為上式中分母相同，因為上式中分母相同.但是，在觀測到x0 后再計算總體 1 和 2 的概率，這對識別不很明確的分配來說常常有用。11.3 兩正態總體的分類 正態總體分類方法簡單高效。假定 f1(X)and f2(X)為多元正態密度函數，,分別有均值 1 and 協方差矩陣 1 and 均值向量 2 and 協方差矩陣 2.（二）兩個總體距離判別法 先考慮兩個總體的情況，設有兩個協差陣相同的p維正態總體，對給定的樣本Y Y，判別一個樣本Y Y到底是來自哪一個總體，一個最直觀的想法是

21、計算Y Y到兩個總體的距離。故我們用馬氏距離來給定判別規則，有：),(),(22121222222121GydGydGdGdGGdGdG如待判，如，如，yyyyyy1、方差相等)()()()(),(),(1112121222 yyyyyyGdGd22211yyy12 )(2211y)()(21211)(2)(221121y221令),()(21paaa211)2(1111 11yyy則前面的判別法則表示為0)(0021YWWGWG如待判，。）（如，）（如，yyyy 當 和已知時，是一個已知的p維向量，W（y）是y的線性函數，稱為線性判別函數。稱為判別系數。用線性判別函數進行判別分析非常直觀，使

22、用起來最方便，在實際中的應用也最廣泛。21,)(211）yyy()()(W)()(111pppyayay 假定兩總體 1 and 2 具有(11-10)的密度函數.這時使 ECM 最小化的分配法則如下：最小化的分配法則如下：把 x0 分配給 1 如 Allocate x0 to 2 otherwise.上式中判別函數現在變成了一個線性函數了！Proof.Since the quantities in(11-11)are nonnegative for all x,we can take their natural logarithms and preserve the order of the

23、 inequalities.Moreover Consequently,combine with(11-11),we get the results.當總體參數 1,2,and 未知.Wald and Anderson suggest 建議將總體參數用樣本對應量來代替.1:正常人群 n1=30 2:A型血友病犯者 n2=22 調查信息 因此代價相同，先驗概率相同情況下 得到,分配規則 如果 x0 =.210,.044,then y0 =6.62 4.61.我們把其分給 2.假設先驗概率已知:p 1=.75,p 2 =.25.并假定 c(1|2)=c(2|1).利用判別統計量 有 w =6.62

24、 (4.61)=2.01,Applying(11-18),we see that 這樣我們可以分配給 2,an obligatory carrier.協方差矩陣 12的分類 如果協方差矩陣不等，分配規則如下.11.4 評估分類函數 判斷分類方法優劣的一個重要方法就是計算其誤判率或錯分率?？傚e分率為 通過適當選擇 R 1 and R 2得到該量的最小值,稱為 最優失誤率(OER).其中R1和R2有（11-7）中的（b）確定。樣本分類函數的效果可以用真實失誤率來評估(AER),一般來說AER不能計算，因為它依賴未知的密度函數，但是用表現失誤率（APER）來替代，定義為訓練樣本中被錯分的比率。11.

25、5 多總體分類 1.最小期望錯分代價法。Let f i(X)be the density associated with population i,i=1,2,g.Let p i=the prior probability of population i,i=1,2,g.c (k|i)=the cost of allocating an item to k when it belongs to i,for k,i=1,2,g.Rk=the set of xs classified as k.Result 11.5.能使 ECM(11-37)達到最小的分類域，可以通過將 x 分配給 k,k=1,

26、2,。,g,如果下式最?。翰恢挂粋€最小，則將 x 分配給其中任意滿足要求的總體.證明見張堯庭等（209）正態總體分類 (1)協方差不等時 二次判別函數，分配給第i個總體(11.46)(2)協方差矩陣相等時 相等時，判別得分為 因此可以定義線性判別得分11.6 Fishers 判別函數 Fishers idea-把多元變量 x 變成一元變量 y，使得 ys 能盡量分類總體 1 and 2 A fixed linear combination of the xs takes the values y11,y12,y1n,for the observations from 1 and the val

27、ues y21,y22,y2n for the observations from 2 The separation of these two sets of univariate ys is assessed in terms of the difference between y1 and y2 expressed in standard deviation units.That is 其平方后，分子相當于組間差組內差67典型判別函數典型判別函數典型判別函數的思想由 Fisher首次提出。典型判別分析通過對原始變量做線性變換來構建新變量。構建的典型變量使得它們包含原始變量集中有用的信息。換

28、句話說，它們類似主成分和因子分析方法，當然計算方法有所不同。68不考慮典型函數是否可以解釋，其優點是它們可以簡化實際數據的維數，從而使得數據可以可視化.典型函數允許研究人員開發簡單的判別規則。http:/69典型分析的思想:70假設研究人員獲得來自總體Gi的 ni 個樣本,假設該總體服從分布為 Np(i,),for i=1,2,k.并假設這些總體具有相同的協方差矩陣)()(2)(121)1()1(2)1(11,:,:1knkkkknkxxxGnnnnxxxG71Let xaxuRaTp)(,)()(2)(1)1()1(2)1(11,:,:1knTkTkTknTTTkxaxaxaGxaxaxaG

29、那么組間的離差為 B:BaaaxxxxnaxaxanSSGTTikiiiTTiTkii)()()(1)(2)(172組內變差為:EaaaxxxxaxaxaSSETTiijnjiijkiTiTijTnjkiii)()()()(1)()(12)()(11EaaBaakknFTT1 經典判別分析的思想是，對原始數據進行投影使得變化后經典判別分析的思想是，對原始數據進行投影使得變化后的樣本組間差別最大，組內差別最小，即使得比值最大。的樣本組間差別最大，組內差別最小，即使得比值最大。73可以證明EaaBaakknFTTaa1maxmax00E-1B的最大特征值.a1 為 E-1B 對應的最大特征向量.線

30、性組合y1=a1Tx 就是單個線性判別函數就是單個線性判別函數，其提供了總體之間的最大差異.這里F可以用于檢驗兩組之間的均值是否相同！74 a1 is the largest eigenvalue,Proof(here we change some symbol)為什么V-1/2AV-1/2和V1A的特征根一樣，因為AB和BA的非0特征根相同！75V-1A的特征向量就是要找的系數a7677X的有效判別可以基于 a1Tx,a1T1,a1Tk,令 di=|a1Tx-a1Ti|如果 di.最小，則x應該分配給第i個總體y2=a2Txdi2=(a1Tx-a1Ti)2+(a2Tx-a2Ti)2Assig

31、n x to the population that gives the minimum value for di2.80818283Determining the dimensionality of the cannonical spaceThe dimensionality of the cannonical space s is bounded above y the minimum of p and m-1.We can construct SCREE plots of the eigenvalues or consider what proportion of the total v

32、ariability is being accounted for by each cannonical function and select enough to account for a large proportion of the total variability.84Let data in iris become 1,2,3,then we use discriminant analysis Iris is grouping variable 85data gpa;infile T11-6.dat;input gpa gmat admit;proc discrim data=gp

33、a pool=yes manova wcov pcov listerr crosslisterr;class admit;var gpa gmat;run;去掉先驗概率Proc CanDisc Data=Iris All Out=OIris;Class Species;Var y1 y2;Run;8687888990919293This shows a test for homogeneity of the variance-covariance matrices for the three test is significant and the hypothesis of equal wou

34、ld be rejected.The linear discriminant functions often work quite well even though the vaiance-c are the prob of correct classification are high enough to satisfy the user,then the user should not be too concerned that he is using a linear discriminant rule rather than a quadritic rule.SPSS cannot.9

35、495We see the eigenvalues of W-1B,as well as statistical tests for determining the dimensionality of the cannonical this example,both eigenvalues are significant(p=0.0000).The first accounts for 99.1%of total variability,so the second is not,the means for these three varieties come close to lying on

36、 a straight line within the four-dim sample space.96Define standardized cannonical functions-these could be used on data that has been standardized to determine the projections of data points onto the cannonical space.97The first lists vectors that define unstandardized cannonical example,we could c

37、ompute undstandardized cnnonical scores viaThe location of the three variety means in the unstandardized cannonical space are shown at the is plot on the territorial map,their locations are given by the(*)on the plot.9899100Spss also locates the perpendicular bisectors between the variety means on t

38、his calls this plot a territorial bisectors divide the cannonical space into three distinct the projection of a new data point falls into one of the regions,then the new points is closets to the means(*)in that region which determines the variety to which the observation would be classified.101102Th

39、is table provide a listing of how each iris plant in the data set would be classified by the discriminant rule column labeled ACTUAL GROUP identifies the variety from which the observation came;the Highest shows the variety to which the observation would be assigned by the discriminatnt rule.103The

40、first column labeled P(G/D)is the posterior probability for the group to which the observation is assigned.This posterior prob is 0.885 for case 3.SPSS gives posterior prob for only the best group and the second best Second P(D/G)can be ignored.The last column,labeled DISCRIM SCORES,give the locatio

41、ns of the projections of the observations to the cannonical plot of these projections for all observations in the data set is shown on page 271.104105This table give a summary of the classification results by the resubstitution method.Examination of this summary shows that all 50 of the variety 1 pl

42、ants are reclassified into variety 1.48of the variety 2 are reclassified into varitey 2,49 of the variety 3 are reclassified into variety 3.Overall,98%of the observation are correctly classifictions are made by applying the rule to the data used to build the rule,and so they may overestimate the act

43、ual prob of correct classifiction.SPSS does not have an option for cross-vaidation of the data as was available in SAS.SPSS does allow us to create a holdout data set by using its select option.106Change as We get107108109110Page 271111The projections of the means of the three iris groups are also l

44、ocated by the asterisks on this plot is interesting in that we can see that variety 1 is quite distinct from varieties 2 and 3.Also there is very little overlap between varieties 2 and 3.So discriminantion should be quite good for this iris data.2022-12-21中國人民大學六西格瑪質量管理研究中心113判別分析方法步驟及框圖判別分析方法步驟及框圖 判別分析的邏輯框圖如下：2022-12-21中國人民大學六西格瑪質量管理研究中心114判別分析方法步驟及框圖判別分析方法步驟及框圖 圖圖4.1 4.1 判別分析步驟框圖判別分析步驟框圖 判別分析步驟 一，收集數據 二，計算先驗概率 三，檢驗協方差矩陣是否相同以確定是使用線性判別函數還是二次判別函數 四，估計條件概率f(X|i)下的參數.五，計算判別函數 六，使用交叉驗證方法來估計錯分率 七，進行分配。