GenAI Usage Clustering

Clustering analysis on ~50k students to check whether how someone uses GenAI tools (hours, tool diversity, paid vs free, dependency, prompting skill) says more about their academic/wellbeing outcomes than their major does.

Short answer: yes, by a decent margin, though the effect sizes are modest and one clustering method disagrees with the other on how many groups there actually are. Details below.

Why

Most "AI and students" takes online are either "it's ruining education" or "it's a superpower," with not much in between backed by numbers. I wanted to see if there's an actual, measurable behavioral pattern that predicts things like burnout or skill retention better than the obvious demographic split (major).

Data

Data_Ai.csv — per-student records with GenAI usage habits (weekly hours, number of tools used, subscription status, primary use case, self-reported dependency, prompting skill level) plus academic outcomes (GPA before/after, study hours, exam anxiety, skill retention, burnout risk) and basic demographics (major, year, institutional policy).

What I did

1. Built behavioral features from the raw columns — mapped Primary_Use_Case to an ordinal activity score (copy-pasting an answer vs. using AI as a debugging partner), turned skill level into an ordinal, kept use-case one-hots aside for later profiling instead of feeding them into clustering (they'd just duplicate the activity score).
2. Standard-scaled everything, ran PCA and UMAP purely for 2D visualization — actual clustering runs on the scaled features, not on the reduced coordinates.
3. Clustered with both K-means (elbow + silhouette to pick k=5) and HDBSCAN (grid-searched min_cluster_size to avoid the default splitting everything into dozens of tiny groups).
4. Validated: silhouette / Davies-Bouldin for both methods, 10-run bootstrap stability check, Adjusted Rand Index between the two methods.
5. Named and profiled the 5 k-means clusters based on feature means/medians and use-case distribution per cluster.
6. Tested whether cluster membership actually explains anything: Kruskal-Wallis across clusters for each outcome, and eta² comparing cluster vs. major as predictors.

The clusters

Cluster	Size	Profile
Casual Free-Tier Beginners	13,840	low hours, one tool, no subscription, low prompting skill
Invested Power Users	8,484	multiple tools, paying, high skill
Paying but Unskilled	8,809	pays for a subscription but sticks to one tool, low skill
Resourceful Free-Tier Explorers	12,882	multiple tools, high skill, doesn't pay
Heavy / Dependent Users	5,985	~25.6h/week (vs. ~5-7h everywhere else), dependency score more than double the rest, use case skews toward debugging

That last cluster is the interesting one — it's not really about skill or payment, it's a much heavier, more dependent usage pattern, and it's a fairly small slice (~12%) of the sample.

Does it actually predict anything

Cluster beats major on all five outcome variables (eta²):

Outcome	cluster	major
GPA change	0.016	0.003
Study hours	0.015	0.001
Exam anxiety	0.054	0.003
Skill retention	0.075	0.003
Burnout risk	0.132	0.007

Kruskal-Wallis comes out significant everywhere, but with n≈50k that's not saying much on its own — eta² is the number that matters here, and it's telling a more honest story than the p-values.

Worth being upfront about the size of these effects: 0.13 for burnout is real but not huge — cluster explains about 13% of the variance there, and less than 2% for GPA and study hours. So the headline is "usage pattern beats major as a predictor, especially for burnout/anxiety/retention," not "usage pattern explains burnout."

Where it gets messy

• Silhouette scores sit around 0.20–0.23 for both methods — the clusters are statistically real (stable under bootstrap, ARI ~0.998 for k-means across resamples) but not sharply separated. Overlapping blobs, not neat islands.
• K-means and HDBSCAN disagree with each other — ARI between the two is 0.39. HDBSCAN, across every min_cluster_size I tried and every bootstrap run, consistently lands on just 2 clusters, not 5. That's a real signal, not noise (it's perfectly stable, ARI=1.0 across resamples) — it suggests the underlying structure might be closer to "high-dependency heavy users vs. everyone else" than the 5-way segmentation k-means gives you. I kept the 5-cluster k-means version because it's more interpretable and the profiles are more actionable, but this disagreement is a legitimate limitation, not something to gloss over.

Running it

pip install -r requirements.txt

Open notebooks/project_en.ipynb and run all cells. You'll need to point DATA_PATH at wherever you put the CSV.

Repo layout

data/           the CSV
notebooks/      the analysis
requirements.txt