The problem
A modern cancer dataset measures several kinds of features per patient: RNA-seq expression for thousands of genes, DNA methylation at hundreds of thousands of CpG sites. A clinical label like ER status is one number. Feature selection means picking the small subset of features that predicts the label and means something biologically.
This is hard for three reasons. First, features outnumber patients by a wide margin, so almost any method can find a combination that fits the training data. Second, features come in correlated groups (genes in the same pathway move together), and selecting one gene from a group at random gives a different answer on every data split. Third, modalities differ: expression and methylation have different scales, different noise, and different group structures.
The data: TCGA-BRCA
The thesis simulations are the controlled case study. The real-data case study is breast cancer from The Cancer Genome Atlas (TCGA-BRCA):
- RNA-seq (RSEM, via cBioPortal): primary tumors, one sample per patient, filtered to the 1,500 most variable genes in the RNA-only phase.
- Methylation (Illumina 450K, probe level for selection; probes map to genes only for interpretation).
- Endpoint: ER status by immunohistochemistry (IHC), 806 ER-positive and 237 ER-negative. This is an independent clinical label, not derived from the RNA, so it can honestly validate the selection.
- Combined cohort: n = 737 patients with both modalities.
- Design: 70/30 stratified train/test splits, all learned operations (variance filtering, scaling, grouping, weights, selection, lambda) inside the training set, held-out AUC on the test set.
What counts as ground truth
Prediction accuracy alone cannot tell you the selection is right: many wrong gene sets predict ER status well. So I used two independent biological yardsticks:
- Positive controls: ESR1, PGR, FOXA1, and GATA3 are the canonical ER-pathway transcription factors. A method that claims to find ER-related genes should find these.
- Signature enrichment: hypergeometric overlap tests against the PAM50 and Oncotype DX gene signatures. If the selected set is enriched for these independently defined signatures, it is finding real breast cancer biology, not noise.
A secondary endpoint checks consistency, not validity: PAM50 Basal-like vs rest, where the labels themselves are derived from the same RNA, so that endpoint can only confirm the method behaves sensibly.