Converge each gene of a SpaNorm fit to its own optimum (R polishSpaNorm).
:func:~spanormpy.spanorm fits every gene in one shared IRLS loop, with a gene-averaged cell weight vector and an aggregate convergence criterion, so a gene can be left short of its own optimum (bright, spatially restricted genes most of all). This takes the stored fit, converges each gene separately under SpaNorm’s own model, stores the polished fit and rewrites layers["logcounts"] from it.
SpaNorm’s mean for gene :math:g in spot :math:i is :math:\log\mu_{gi} = \bar\mu_g + a_1 w_{i1} + \sum_{j\ge2} W_{ij}\alpha_{gj}, where :math:w_{i1} is the log library size and :math:a_1 is shared by every gene. The polish maximises each gene’s penalised NB log-likelihood by damped Newton (see :func:~spanormpy.polish.polish_nb), with :math:a_1 held at the fit’s value (ls="fixed"), :math:\bar\mu_g an unpenalised per-gene intercept, and the other coefficients ridge-penalised as SpaNorm penalises them. The polished coefficients are the optimum of the unwinsorised penalised likelihood; the normalisation still applies SpaNorm’s usual winsorisation, as for an unpolished fit.
A gene with no counts in the polished spots, or none in every polished spot of one batch level, has no finite optimum and is not polished: it keeps its input coefficients and dispersion, and its diagnostics row reads polished=False with held_out naming the reason ("all-zero" or "zero-batch-level"). Genes held out for a batch level are counted in a warning. A gene the Newton engine cannot converge also keeps its input fit, and a warning counts these. A design whose unpenalised columns are collinear is detected up front and no gene is passed to the engine; the fit is marked polished either way.
With ls="joint" the shared coefficient moves to the optimum of the total penalised log-likelihood over the polished genes: Newton steps on its profile log-likelihood alternate with a warm re-polish of every gene at the candidate value. It stops when the standardised score :math:|U|/\sqrt{I} is below 1e-3 or after 10 steps; maxit and tol are the per-gene Newton’s. The joint estimate weights genes by their information, so bright genes dominate it. Genes that are not polished take the new shared :math:a_1 but keep their other coefficients and dispersion.
The polished fit replaces spanorm and the input fit is kept as spanorm_unpolished (see :func:~spanormpy._fit.store_fit for the layout). With null=True a stored spanorm_null (from the SVG test) is polished with the same settings and the input kept as spanorm_null_unpolished. SVG results in var were computed from the unpolished fit, so they are removed with a warning; rerun spanorm_svg. A fit stored for a different number of spots (a spot subset of the object) is an error, as in R; a gene subset reuses the stored fit (see docs/r-discrepancies.md).
The NB likelihood is defined on counts only, so a non-integer counts matrix is refused.
Parameters
Name
Type
Description
Default
data
An AnnData or SpatialData that :func:~spanormpy.spanorm has been run on, with the raw counts in X or layer.
required
adj_method
str
"auto" (= "logpac"), "logpac", "pearson", "medbio" or "meanbio": the adjustment used to rewrite the normalised data.
'auto'
scale_factor
float
Scaling factor for the adjusted counts.
1.0
psi_method
str
"fixed" keeps the fit’s dispersion and converges the mean under it; "profile" re-estimates it by profile ML at the converged mean and re-polishes.
'fixed'
ls
str
The library-size coefficient shared by every gene: "fixed" holds it at the fit’s value, "joint" re-estimates it jointly with the per-gene coefficients.
'fixed'
cells
str
The spots each gene is converged over: "all", or "fit" for the ones :func:~spanormpy.spanorm sampled to fit the model.
'all'
null
bool
Also polish the null model stored by spanorm_svg, when there is one. The SVG test compares the two fits, so they must be estimated the same way.
True
maxit
int
The Newton iteration cap and relative log-likelihood tolerance, per gene.
50
tol
int
The Newton iteration cap and relative log-likelihood tolerance, per gene.
50
engine
str
As in :func:~spanormpy.polish.polish_nb; backend and n_jobs also drive the normalisation.
'batch'
batch_size
str
As in :func:~spanormpy.polish.polish_nb; backend and n_jobs also drive the normalisation.
'batch'
backend
str
As in :func:~spanormpy.polish.polish_nb; backend and n_jobs also drive the normalisation.
'batch'
n_jobs
str
As in :func:~spanormpy.polish.polish_nb; backend and n_jobs also drive the normalisation.
'batch'
overwrite
bool
Polish an already polished fit again, starting from the unpolished fit kept as spanorm_unpolished (and spanorm_null_unpolished), never from the polished one.
False
verbose
bool
Print progress messages (to stderr, like R’s message).
True
layer
str | None
Counts layer (default X) and, for SpatialData, the table.
None
table_key
str | None
Counts layer (default X) and, for SpatialData, the table.
None
copy
bool
Return a modified copy instead of modifying data in place.
False
**kwargs
R’s ...: block_size and psi_range are passed to polish_nb; any other key is ignored with a UserWarning.
{}
Returns
Name
Type
Description
None, or with copy=True a modified copy of the same container type.
Examples
Note
The code below runs when the site is built, on a small pre-fitted subset of HumanDLPFC (301 genes x 4015 spots; spanorm(sample_p=0.05, df_tps=2), spanorm_svg() and spanorm_pca() already applied) that is fitted once per build and loaded as adata. The outputs are those of the code shown.
import spanormpy as sn# adata: the HumanDLPFC AnnData (raw counts in X, coordinates in obsm["spatial"]),# already fitted with sn.spanorm(adata, sample_p=0.05, df_tps=2)sn.polish_spanorm(adata, verbose=False)adata.var.filter(like="polish").head()