preprocess.bmiq
preprocess.bmiq
BMIQ: put type II probes on the type I scale.
WHY THE TWO PROBE TYPES NEED RECONCILING. An Infinium type I probe reads both of its beads in one colour; a type II probe reads methylated in green and unmethylated in red. The chemistries differ, and the consequence is not subtle: type II betas are compressed toward the middle, so the same underlying methylation reads lower at the top of the range and higher at the bottom. On the corpus’s EPIC v1 arrays the median type I beta is 0.095 and the median type II beta is 0.646 – most of that is which probes are where on the genome, and some of it is the chemistry, and nothing downstream can separate them.
It matters here because clock coefficients were fitted on corrected data. A clock trained on BMIQ-normalised betas and applied to raw ones is being asked about a slightly different quantity at every type II probe, which is most of them: 723,760 of 865,918 on EPIC v1.
THE METHOD (Teschendorff et al., Bioinformatics 2013;29:189-196)
- Fit a three-state beta mixture – unmethylated, hemimethylated, methylated – to the type I betas, and another to the type II betas.
- Assign every probe to the state it most likely belongs to.
- For the unmethylated and methylated states, map each type II probe through its own fitted distribution and back out through the corresponding type I one, so a probe at the 30th percentile of type II’s unmethylated state comes out at the 30th percentile of type I’s.
- Rescale the hemimethylated state linearly into the gap the other two leave.
Step 3 is the whole idea: a quantile map within a state, not across the whole distribution. Mapping the two distributions wholesale would also erase the real difference in which genomic regions each chemistry covers.
FITTED BY EM WITH METHOD-OF-MOMENTS M-STEPS. The original uses RPMM’s beta-mixture fitter. There is no RPMM here and porting one is not the point: a three-component beta mixture is a short EM, and a beta distribution’s two parameters follow from a weighted mean and variance in closed form. That is what this does, and the fit is checked against a mixture drawn with known parameters.
Classes
| Name | Description |
|---|---|
| BetaMixture | A fitted three-state beta mixture: weights, and (a, b) per state. |
BetaMixture
preprocess.bmiq.BetaMixture(weights, a, b, n, iterations, converged)A fitted three-state beta mixture: weights, and (a, b) per state.
Methods
| Name | Description |
|---|---|
| state | Hard assignment: 0 unmethylated, 1 hemimethylated, 2 methylated. |
state
preprocess.bmiq.BetaMixture.state(x)Hard assignment: 0 unmethylated, 1 hemimethylated, 2 methylated.
Functions
| Name | Description |
|---|---|
| bmiq | Normalise type II probes onto the type I scale, sample by sample. |
| fit_beta_mixture | Three-state beta mixture by expectation-maximisation. |
bmiq
preprocess.bmiq.bmiq(data, *, probe_type=None, platform=None, max_iter=200)Normalise type II probes onto the type I scale, sample by sample.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| probe_type | pd.Series | None | "I"/"II" per feature. Taken from the array manifest when not given, which needs the manifest fetch. |
None |
| platform | str | None | Overrides the dataset’s declared platform for the manifest lookup. | None |
Notes
Fitted per sample, because the type I/type II offset is a property of a chip and a run rather than of a cohort. Fitting once across samples would be faster and would smear one array’s dye batch onto every other.
The correction is applied to type II probes only. Type I probes are the reference and come out unchanged, which is worth stating because it means BMIQ never moves a clock built purely on type I probes.
fit_beta_mixture
preprocess.bmiq.fit_beta_mixture(x, *, max_iter=200, tol=1e-05, seed=0)Three-state beta mixture by expectation-maximisation.
Initialised by splitting at 1/3 and 2/3 rather than at random. The states are not exchangeable – they mean unmethylated, hemimethylated and methylated – so a random start can converge to a relabelled solution that then maps the wrong state onto the wrong one, and the output looks plausible. A fixed, ordered start makes the fit reproducible and keeps the labels meaning what they say.