Using image recognition, images of 20 peanut varieties are collected through scanner. For each variety, the images of 100 peanut seeds’ positive and two sides are collected; from each image, 50 characteristic features related to shape, color, and texture are obtained. A cluster analysis model based on these features and another cluster analysis model with principal component analysis (PCA) data optimizing are built. In the next step, we have got a pedigree clustering tree for 20 peanut varieties. The first 17 principal components of PCA features have been able to fully simulate the 50 statistical features. There is almost no difference between the classification categories of peanut when the cumulative contribution rate is 85% or more.