Young People Survey: Cluster Analysis

Intro to Programming in Python: Final Project

Completed: 03/12/2026

1. Overview

This project applies cluster analysis to data from the Young People Survey (YPS) to identify groups of young people with similar patterns of interests and hobbies. Two clustering methods are used: K-means and hierarchical clustering. Their performance is compared using silhouette-based evaluation and visual inspection. The resulting clusters are interpreted in relation to respondents’ age and gender.

2. Repository Structure

YPS project.ipynb contains the complete analysis, including data preprocessing, exploratory analysis, clustering, model comparison, visualizations, and interpretation. YPS report.docx contains the written report with the research question, methodology, results, model comparison, and discussion of limitations.

3. Research Question

Which groups of respondents with similar patterns of hobbies and interests can be identified in the Young People Survey data, and how are these groups related to socio-demographic characteristics (age and gender)?

4. Dataset

The analysis is based on the Young People Survey, containing 1,010 respondents and 150 variables covering interests, hobbies, attitudes, personality traits, and socio-demographics.

Dataset source: https://www.kaggle.com/datasets/miroslavsabo/young-people-survey

A subset of 34 variables was selected:

• 32 hobby and interest variables (5-point Likert scale)
• age
• gender

5. Data Preprocessing

Missing values (132 in total) were handled as follows:

• Ordinal variables: imputed with the median (to preserve ordinal scale)
• Age and gender: observations with missing values were removed (14 cases)

Final sample size: N = 998

Gender was recoded:

• 0 = male
• 1 = female

All variables were standardized using StandardScaler before clustering.

6. Methods

The analysis includes:

• Exploratory data analysis
• Distribution and normality checks
• K-means clustering
• Hierarchical clustering
• Silhouette analysis
• Elbow method (inertia)
• Dendrogram analysis
• PCA visualization
• Cluster interpretation and comparison

Clustering models

• K-means (KMeans, scikit-learn)
• Hierarchical clustering (AgglomerativeClustering, scipy dendrograms)

Evaluation was based on the silhouette coefficient.

7. Results

K-means

The elbow method suggests 4 clusters, but the improvement in inertia decreases gradually, indicating weak natural separation.

Silhouette analysis shows low clustering quality:

• average silhouette ≈ 0.07
• several negative values
• strong overlap between clusters

PCA visualization confirms that clusters are not clearly separable, with substantial overlap and similar centroid distances.

Hierarchical clustering

The dendrogram also suggests a 4-cluster solution, but cluster structure is even more uneven and diffuse.

Compared to K-means:

• clusters are more imbalanced
• separation is weaker
• silhouette score is lower (≈ 0.05)

Comparison

• K-means performs slightly better than hierarchical clustering
• both methods show weak and unstable cluster structure
• Adjusted Rand Index ≈ 0.36, indicating only partial agreement between solutions

Overall, the data does not contain strongly defined cluster boundaries, which is typical for behavioral survey data.

Cluster interpretation (K-means)

Despite low quality, clusters show interpretable tendencies:

• Cluster 0: higher interest in PCs and cars, lower interest in humanities and arts
• Cluster 1: generally low engagement across most domains, especially sciences
• Cluster 2: strong focus on biology, chemistry, and medicine (science-oriented profile)
• Cluster 3: broad interest profile, especially humanities (history, politics, law, art, religion)

Socio-demographics

• Age differences between clusters are small (around 19–21 years on average)
• Gender differences are more visible: some clusters are male- or female-dominated

8. Limitations

The study has several limitations:

• Median imputation and removal of missing values may slightly distort variable distributions
• All variables are ordinal, but are treated as continuous after standardization, which affects interpretability
• Clusters are weakly separated (low silhouette scores), so results should be interpreted as soft patterns rather than strict groups
• The choice of 4 clusters is not fully stable; a simpler 2-cluster solution could also be reasonable given the weak structure in the data
• Gender imbalance in the sample may influence cluster composition

9. Conclusion

The analysis shows that while both clustering methods identify some interpretable patterns of interests, the overall cluster structure in the YPS data is weak. The results should be considered exploratory and descriptive rather than definitive segmentation of respondents.

10. Author

Maria Tikhonova

Project completed as part of the Intro to Programming in Python course.

11. Reference

Sabo, M. Young People Survey. Kaggle.

Dataset: https://www.kaggle.com/datasets/miroslavsabo/young-people-survey