We begin with what stays the same. spidepy is a port of spiDE 0.99.37: it runs the same method, the slopes engine and the sandwich engine, the same pooled and condition-specific tests and the same gene-set and abundance tests. A test suite compares spidepy with reference values computed by the R package, and the numbers agree to numerical closeness. The method is defined by the R package, so that is the one to cite.
What changes is the way you call it. Names are snake_case in Python with R’s defaults (min.cells is min_cells), and R’s BPPARAM is n_jobs. Inputs and outputs follow Python conventions:
| R (spiDE) | Python (spidepy) | |
|---|---|---|
| Container | SpatialExperiment |
AnnData |
| Counts | assay counts |
layers["counts"] |
| Coordinates | spatialCoords() |
obsm["spatial"] |
| Niche matrices | reducedDim(spe, "Niche30") |
obsm["Niche30"] |
| Updates the object | returns a modified copy | in place (copy=True to return a copy) |
| Plots | ggplot2 objects |
matplotlib figures |
| Parallelism | BPPARAM (BiocParallel) |
n_jobs |
| Result tables | data.frame |
pandas.DataFrame, with R’s column names |
The two pages below are the package’s own documentation of the correspondence. The first lists every exported R function with its Python counterpart; the second lists the places where the two behave differently, with the reason for each.
R to Python mapping
Every export of R spiDE 0.99.37 (commit c8db60a, NAMESPACE) and its spidepy counterpart. The NAMESPACE also exports the $ method of the result classes; in Python that is plain attribute access (res.contrast, fit.engine).
Input is an AnnData instead of a SpatialExperiment: counts live in layers["counts"] (assay= names another layer, "X" uses adata.X), cell coordinates in obsm["spatial"], and niche matrices in obsm["Niche<sigma>"] instead of reducedDim. The first argument spe is called adata. Functions that R runs on a copy (buildNiches, mergeNiches) work in place and return the object; pass copy=True to leave the input alone.
Argument names
Arguments are snake_case with R’s defaults.
| R | Python |
|---|---|
spe |
adata |
object |
res or fit (the result or fit object) |
min.cells, min.patients, min.detect, min.tiles |
min_cells, min_patients, min_detect, min_tiles |
min.size, max.size |
min_size, max_size |
prior.count, max.leverage |
prior_count, max_leverage |
min.calls, colour.by, n.pcs, point.size, scale.bar |
min_calls, colour_by, n_pcs, point_size, scale_bar |
BPPARAM |
n_jobs (an integer; results are identical for any value) |
verbose |
verbose (messages go to stderr, only when True) |
Column names of the result tables keep R’s spelling (p.heterogeneity, q.global), so a table from Python reads like one from R. The differences between the two packages, with reasons, are listed in r-discrepancies and summarised in the README.
Exports
| R export | Python | Notes |
|---|---|---|
buildNiches |
spidepy.build_niches |
writes obsm["Niche<sigma>"]; in place unless copy=True |
fitSpiDE |
spidepy.fit_spide |
returns a SpiDEFit; engine="slopes" or "sandwich" |
mergeNiches |
spidepy.merge_niches |
groups is a dict of name to list of cell types; group map in uns["spiDE_niche_groups"] |
patientIntercepts |
spidepy.patient_intercepts |
returns a DataFrame |
patientSlopes |
spidepy.patient_slopes |
returns a DataFrame; slopes engine only |
plotCallMap |
spidepy.pl.plot_call_map |
matplotlib; returns a Figure |
plotNicheMap |
spidepy.pl.plot_niche_map |
matplotlib; returns a Figure |
plotNicheResponse |
spidepy.pl.plot_niche_response |
matplotlib; returns a Figure |
plotPValues |
spidepy.pl.plot_p_values |
matplotlib; returns a Figure |
plotPatientEffects |
spidepy.pl.plot_patient_effects |
matplotlib; returns a Figure |
plotPatientSlopes |
spidepy.pl.plot_patient_slopes |
matplotlib; returns a Figure |
plotSpillover |
spidepy.pl.plot_spillover |
matplotlib; returns a Figure |
plotTopTriplets |
spidepy.pl.plot_top_triplets |
matplotlib; returns a Figure |
plotTripletHeatmap |
spidepy.pl.plot_triplet_heatmap |
matplotlib; returns a Figure |
results |
spidepy.results |
returns a DataFrame |
spiDE |
spidepy.spide |
extra fit options pass through **fit_kwargs; name is the niche key prefix; the niche matrix it builds is removed again |
spiDEColours |
spidepy.pl.spide_colours |
the same colour-blind-safe palettes (palette V, with the "spatial" role) |
spiGSEA |
spidepy.spi_gsea |
genesets is a dict of set name to gene list; exact argument values only |
testNicheAbundance |
spidepy.test_niche_abundance |
returns a DataFrame |
testSpiDE |
spidepy.test_spide |
returns a SpiDEResults |
theme_spiDE |
spidepy.pl.style |
a context manager (with spidepy.pl.style(): ...) instead of a ggplot theme |
SpiDEFit |
spidepy.SpiDEFit |
exported class (a dataclass); $ is attribute access |
SpiDEResults |
spidepy.SpiDEResults |
exported class (a dataclass); $ is attribute access |
toySpiDE |
spidepy.toy_spide |
R’s own example data as an AnnData; spidepy.datasets.simulate_toy, simulate_niche and simulate_clustered are numpy-seeded simulators |
Differences from R spiDE
One bullet per item: what R does, what spidepy does, and why. Reference: spiDE 0.99.37 (c8db60a). Where R has a bug or an R-specific accident, spidepy matches R and logs it here. Items are grouped by area.
Interface
- AnnData instead of SpatialExperiment. Counts are read from
layers[assay](orX), coordinates fromobsm["spatial"], niche matrices live inobsm["Niche<sigma>"]instead ofreducedDim, and only_io.pytransposes to R’s genes x cells.build_nichesandmerge_nicheswork in place unlesscopy=True(R returns a modified copy). n_jobsreplacesBPPARAM. An integer; the result is identical for any value.- Result tables keep R’s column names.
p.heterogeneityandq.globalare not renamed to snake_case, so tables from the two packages line up column for column. spidedoes not keep the niches it builds. R’sspiDE()returns the results and leaves the caller’s object untouched;spidebuilds the niche matrix in place (to avoid copying the counts) and removes it fromadata.obsmonce the fit is done. A niche matrix already present is used and kept.spiGSEAargument matching. R’smatch.argaccepts unambiguous prefixes (type = "self",test = "pool").spi_gseatakes exact values only, astest_spidedoes; an unknown value raisesValueError. Alsospi_gsearefuses anything but aSpiDEResultswith aTypeError(R dispatches and errors on the missing method).spiGSEAdirection of a missing statistic. R writesNA_character_;spi_gseausesNonein thedirectioncolumn (object dtype).- Result and fit objects. R stores
character(0)where Python hasNone:SpiDEFit.conditionwhen no condition was given;SpiDEResults.conditionisNonewhen no condition was tested, andcontrastis""then. - Legacy-object guard. R’s
checkResultsalso rejects pre-0.99.23 mixed-model objects (.assertCurrent). Not ported: spidepy has no such objects. - Strata that vary within a patient. R’s
fitSpiDEreadsas.character(patients[[strata]]); a strata column that is not constant within patients is absent from the patient table, so R fails with the obscure “‘names’ attribute [n] must be the same length as the vector [0]”.fit_spideraisesValueError("strata column '...' is not a patient-level column")(same for a condition, as R’s.conditionCoding). Same inputs fail in both; only the message differs.
Ordering and sorting
- Sorting of character labels. R sorts character labels (patients, cell types, conditions) in the session locale (
en_US.UTF-8, which interleavesa A b Band ignores punctuation at first pass).spidepy._rcompat.r_sort_str(and Pythonsorted) uses code-point order (R’smethod = "radix"/LC_COLLATE=C), so mixed-case or punctuated labels can order differently (for exampleb, Cin R butC, bhere). Labels that share case and punctuation style, such asS1, S10, S2orNon-responder, Responder, order identically. Code-point order is deterministic and independent of the user’s locale, and it matches the reference fixtures; it is the documented choice rather than emulating glibc collation. Likewise the rows ofpatient_slopes/patient_intercepts: R sorts rows withorder()on character keys (locale collation); Python sorts in code-point order (see “Sorting of character labels”). Identical for labels such asG1, G10, G2,S1, S10, S2. - Niche bandwidth lookup. R finds niche reducedDims with the regex
^Niche[0-9.]+$, which misses a bandwidth thatpaste0writes in exponent form (Niche1e+05)._io.niche_keysuses the same pattern, so such a niche is not found either. Matched to R because bandwidths of 1e5 or more are not meaningful in spatial units;niche_keystill writes the R string.
Niches
- Niche kernel density. R
densityfunreturns the values of in-window queries only (drop = TRUE) and hasdiggle = FALSEas its own default._kde.density_atkeeps the query order, returnsnanfor a query outside the window and defaults todiggle = True(the valuebuildNichespasses). The numbers for in-window queries are identical: spatstat uses the exactpnormedge weight for a rectangular window, not a pixel image. merge_nichesgroup named like a leftover column. If a group name equals a column that is not in any group (for example{"B": ["A", "C"]}with a columnB), R produces two columns namedBand its[[lookup of the group map sees the first. We keep both columns and record the first inuns["spiDE_niche_groups"](a dict cannot hold duplicate keys).merge_nicheskey prefix. Withsigma = None, R greps^NicheoverreducedDimNames, so any key beginning withname(for exampleNicheFoo) is merged and given a group map, not onlyNiche<number>keys.merge_nichesdoes the same overobsmkeys.
Fit: design and slopes
- Constant-column test in
_niche_columns. R’svaraccumulates in long double, so a niche column of identical values has variance exactly 0 and is dropped._design._colvardoes the same (long double, two-pass mean); a plain double mean can leave about 1e-33 and keep the column. _partial_blocksingularity. R’ssolvefails when the reciprocal condition number ofZ'WZ(LAPACKdgeconestimate) is below machine epsilon, and the block is then skipped (NULL). We compute the exact 1-norm reciprocal condition number from the inverse and apply the same threshold; the two can disagree only for blocks within a few percent of that threshold.r2of the depth blocks. R useslm.fit(pivoted QR, tolerance 1e-7) for the R^2 of a niche column on a patient’s block; we usenumpy.linalg.lstsq. They agree whenever the block has full column rank (it does for the sections whose df rule applies); a numerically rank-deficient block could drop different directions.
limma and empirical-Bayes moderation
fitFDistUnequalDF1with exactly two informative genes. R setsprior.weights <- NULLafter having set itsPriorWeightsflag, so when NA or tiny-df genes are also present the weighted branch runs on NULL weights: the prior scale is0/0 = NaNand the objective is identically 0, sooptimizereturns the value of a flat search. Matched (scale = nan, samedf2); the posterior variances are then allnan, as in R.- limma
weightedLowess._limma._weighted_lowessis a port ofweighted_lowess.cfrom limma 3.66.0 (Bioconductor RELEASE_3_22). The window search is vectorised (other points sorted by distance, right side first on ties, then a cumulative sum in the same order of additions) instead of growing the window point by point; results agree with R to rounding, including on tied x.rsort_with_indexties among equal residuals are broken stably (only affects the robustness-iteration median when residuals tie exactly). The equal-weights branch ofloessFit(plainlowess) is not ported: it cannot occur fromfitFDistUnequalDF1. eBayesoutputs not computed.var.prior,lodsand theFstatistics (tmixture.matrix,classifyTestsF) are not read by spiDE and are not returned;lmFit’scat("Coefficients not estimable")message is not printed (aliased coefficients arenan).- Unequal-df prior to ~1e-9.
fitFDistUnequalDF1findsdf2by Brent’s method (Roptimize) on a sum that R accumulates in long double and numpy in double, sodf.prioragrees with R to about 1e-9 relative, not to machine precision. - Robust
fitFDistUnequalDF1dropsspanin its refit. R refits with prior weights throughRecall(x, df1, covariate, robust = FALSE, prior.weights = FDR), which omitsspan, so the refit trend always uses the defaultchooseLowessSpan(n, small.n = 500). We do the same. - Robust fits with tied F statistics.
fitFDistRobustlyranks the F statistics forprob.outlier. When a trend interpolates several genes exactly their statistics tie up to rounding noise (about 1e-16) and the ranks, henceprob.outlieranddf2.shrunkof those genes, depend on the last bits (R and numpy can differ). Not matched beyond rounding. qfwithlog.p.fitFDistRobustlywith unequaldf1callsqf(..., log.p = TRUE); scipy has no log-scale quantile, so we useexp(logp)first. Identical unless the tail probability underflows (about 1e-308).fitFDistRobustlywith two variances returnsfitFDist’s result withoutdf2.shrunk, as R does; reaching that after set-aside genes fails in R (replacement has length zero) and raisesValueErrorhere..pooledColumnTestrobust-to-non-robust fallback. R wrapseBayes(robust = TRUE)intryCatchand, on failure, retries withrobust = FALSE. The helpereb()reads the free variablefit, andfit <- eb(TRUE)has just overwritten it withNULL, so the retry runseBayes(NULL), fails, and the whole column is NA._pooled_column_testreproduces this: a failed robust prior leaves every row of the column NA (no retry). Unit-tested with a failinge_bayes; no reference fixture reaches it.- Tie noise in limma’s robust prior on small gene families. With a handful of genes whose residuals from the expression trend are exactly zero, their F statistics tie up to rounding (1e-16) and
fitFDistRobustlyranks, hence the shrunk prior df, depend on that noise: a 1e-15 relative perturbation of the residual variances movesdf.priorof the tied genes by up to 1e-2 in Python, and R’s own value is one such draw. On the end-to-end slopes fixture (toyspe_s16_n150_g12_seed7, index A, niche B, 12 genes) the pooled se/t/df/p differ from R by up to 1e-5 / 1e-3 / 6e-4 although every input to the test is identical to 1e-14. Not fixable (R’s draw depends on its BLAS rounding); tests compare those rows at 2e-2 (0.53% was seen between numpy / BLAS builds).
Toy data
- Toy-data simulators. R’s
.toySPE(),.toyNiche()and.toyClustered()seed R’s Mersenne-Twister;simulate_toy(),simulate_niche()andsimulate_clustered()use the same generative model withnumpy.random.default_rng(seed), so a givenseedgives a different data set from R’s (R’s draws cannot be reproduced). The planted effect is the same but its statistical power is draw-dependent (R’s seed 7 is a favourable draw; here seed 4 is for the same design). Neither touches the global RNG. They return AnnData (the layoutload_spebuilds from R’s SpatialExperiment) with plain-stringsample_id,conditionandcell_typeand float counts.toy_spide()is R’s owntoySpiDEobject, not a simulation; its text columns are returned as strings rather than the categories stored in the bundled file. .toyNiche()and.toyClustered()are gone from R. R dropped the originals of these two internal simulators in c8db60a (the simplification pass). spidepy keeps their ports,simulate_niche()andsimulate_clustered(), as public toy-data helpers; they no longer have an R original to be compared with, andsimulate_toy()(R’s.toySPE()) is still the one with a counterpart.
Plots
- Plots return matplotlib figures, not ggplot objects.
plot_niche_mapandplot_niche_responsereturn aFigure; R’sp$dataand layer data are available from the private builders_niche_map_dataand_niche_response_data(a dict:data, andsummaryandlinesorcentroids), which keep R’s column names, row order and plotmathgene_labstrings. Panels are boxed titles (not full-width strips), legends are two stacked legends placed by hand, andscale_y_log10labels use a comma for thousands via a formatter; pixel identity is not a goal. - R’s
message()inplotNicheResponse. “G2 not fitted in A, left out” is an unconditionalmessage()in R;plot_niche_responseemits it as aUserWarningso callers can catch it. .fmtqpadsq = 1.formatC(1, digits = 2, format = "g")returns" 1"(padded to three characters);_fmt_qreproduces it.- Plotted counts are dense.
_index_cell_datareturns the plotted genes’ counts as a dense genes x cells array (R keeps a sparse subset); only the requested genes are ever extracted. - Italic gene labels. R plotmath (
italic("G1")) becomes mathtext ($\mathit{G1}$) with each non-alphanumeric character set in\text{}(a bare-would render as a spaced minus sign), so names with_ - " \ $ { } % # & ^print as written. plotPValuesrow order followsmerge()’s unstable tie order. R’smerge(tab, bands, by = c("gene", "index"))sorts the matches of each key with an unstable shell sort (isort_with_index), so the rows of one (gene, index) are not in table order._isort_with_indexreproduces that sort, sop$datahas R’s row order; the histograms do not depend on it.- Clustered heatmap rows.
hclust(dist(m))$orderis reproduced from scipy’s complete-linkage merges (singleton before cluster, lower-numbered cluster first, leaves read left to right); only exact distance ties could differ. - Spillover labels. R places them with
ggrepel(seed = 1); matplotlib has no equivalent andggrepelis optional in R. Labels are right-aligned atlog2(fold) - 0.4, pushed apart vertically, with a segment to the point. Which markers are labelled (largest|z|) is R’s. markerwith a missing fold. R’smarkerisNAwhere the fold isNA; here it isFalse(the fold of a gene is onlyNAwhen the gene is missing from the object, which is an error).- Principal-component signs.
prcompand numpy’s SVD may flip a component; only magnitudes (and adjusted R^2, which is sign invariant) are compared with R. - The call graph’s edges.
geom_curve(curvature = 0.22)becomes a matplotlibarc3arrow; the drawn curvature and arrow heads are approximate, the edge data are R’s. - Patient-slope x breaks. R’s
scales::breaks_extended(n = 3)(Wilkinson’s extended algorithm) becomes matplotlib’sMaxNLocator(nbins = 3); the tick positions can differ, the plotted values do not. - Spillover threshold label. R nudges it with
hjust = 1.05; here it is plainly right-aligned at the fold threshold, so it sits a few points closer to the line.