The Replica Set Method is a Robust, Accurate, and High-Throughput Approach for Assessing and Comparing Lifespan in C. elegans Experiments

Abstract

The advent of feeding based RNAi in Caenorhabditis elegans led to an era of gene discovery in aging research. Hundreds of gerogenes were discovered, and many are evolutionarily conserved, raising the exciting possibility that the underlying genetic basis for healthy aging in higher vertebrates could be quickly deciphered. Yet, the majority of putative gerogenes have still only been cursorily characterized, highlighting the need for high-throughput, quantitative assessments of changes in aging. A widely used surrogate measure of aging is lifespan. The traditional way to measure mortality in C. elegans tracks the deaths of individual animals over time within a relatively small population. This traditional method provides straightforward, direct measurements of median and maximum lifespan for the sampled population. However, this method is time consuming, often underpowered, and involves repeated handling of a set of animals over time, which in turn can introduce contamination or possibly damage increasingly fragile, aged animals. We have previously developed an alternative "Replica Set" methodology, which minimizes handling and increases throughput by at least an order of magnitude. The Replica Set method allows changes in lifespan to be measured for over one hundred feeding-based RNAi clones by one investigator in a single experiment- facilitating the generation of large quantitative phenotypic datasets, a prerequisite for development of biological models at a systems level. Here, we demonstrate through analysis of lifespan experiments simulated in silico that the Replica Set method is at least as precise and accurate as the traditional method in evaluating and estimating lifespan, and requires many fewer total animal observations across the course of an experiment. Furthermore, we show that the traditional approach to lifespan experiments is more vulnerable than the Replica Set method to experimental and measurement error. We find no compromise in statistical power for Replica Set experiments, even for moderate effect sizes, or when simulated experimental errors are introduced. We compare and contrast the statistical analysis of data generated by the two approaches, and highlight pitfalls common with the traditional methodology. Collectively, our analysis provides a standard of measure for each method across comparable parameters, which will be invaluable in both experimental design and evaluation of published data for lifespan studies.

Keywords: Caenorhabditis elegans; biostatistics; high throughput; lifespan; survival modeling.

FIGURE 1

RSM and TLM assays employ different strategies for population sampling. (A) Handling in traditional lifespan assays (TLM). A dish of animals is maintained for the duration of the experiment. Vital status observation may be facilitated by prodding animals to stimulate movement. When more than one experimental condition is included, such as different strains or RNAi treatments, a separate plate is maintained throughout the experiment for each condition. (B) Handling in replica set lifespan assays (RSM). Each observation uses a different plate derived from a synchronized population, with all the plates for expected observations of the experiment having been set up at the beginning of the experiment. M9 solution is typically added to wells to separate animals from the bacterial lawn and to help assess vital status, in conjunction with prodding as necessary. While replica set assays can be done with single-well dishes as well, the approach is particularly well-suited to use of multi-well plates, in which wells of the plate can be different RNAi conditions. Every plate in the replicate group is set up with an identical layout. (C–F) Example survival curves from simulated lifespan experiments, assuming daily scoring. The black dashed curve represents the generating logistic distribution with mean/median of 20 days and shape parameter s = 2. The vertical dashed lines indicate the median for the respective curve. (C–E) A simulated TLM experiment was run with 80 animals and fit with Kaplan-Meier (KM) using right-censoring (blue curve) (C), interval-censoring (purple line) (D), or with parametric fitting to a logistic distribution (fit logistic curve is dark blue) (E). The grey shaded region is the confidence interval for KM curves. (F) shows an example simulated replica set experiment using 25 animals per observation with the proportions of live animals for each observation shown as points, and the fit logistic curve, both in dark red. The median survival for the simulated TLM experiment is 21 days with right censoring (C), 20.5 days with interval censoring (D), and 20.4 days with parametric fitting (E). The median survival for the simulated replica set experiment (F) is 19.3 days.

FIGURE 2

A Literature survey for lifespan studies using C. elegans highlights dramatic variation in sample sizes and levels of detail reported. (A) Histogram of average number of animals per trial for each study considered in a survey of literature reporting lifespan experiments for N2 (wild-type). Histogram bar width represents 10 animals. The dashed line is the smoothed density curve for the same data. Median: 90, mean: 99.46. This excludes two outlying data points, each reported once, that were unlikely representative of single-trial results (360 and 721 animals). (B) A summary of the level of detail reported on censored observations in the surveyed publications. We divided results into categories as follows: [A]- censoring was not mentioned in the manuscript and no censor information was reported in the data, [B]- Censoring was mentioned in the manuscript (e.g. “animals that crawled off plates were censored”), but no censor information was reported in the accompanying data, [C]- Censoring was mentioned in the manuscript and a summary of censored animal information (e.g. “12 animals were censored for trial 1”) was reported in the included data. [D]- Censoring was mentioned in the manuscript, and data for censored animals was provided for each time point of each trial. Note that we did not find an example of this last level among the literature surveyed. (C) Histogram of the reported scoring intervals from across the studies. Studies where a variable scoring interval was specified are indicated by a range of values. The most common scoring schedule was every-other-day scoring, indicated by an interval of two.

FIGURE 3

Comparable accuracy and precision can be obtained for RSM and TLM, but with many fewer total animal observations for RSM. 10,000 simulated RSM and TLM experiment trials were run for each increment in sample sizes across a range of 5–50 for RSM and 5—150 for TLM, both in increments of 5. For RSM, “sample size” refers to the number of animals per timepoint scored, whereas for TLM this is the number of animals at the start of the trial. The columns correspond to scoring intervals from 1 day (every day) or 2 days (every other day). We find fewer total animal observations are necessary to obtain a given level of precision (standard error of median lifespans and accuracy) (B), (mean-squared error of median lifespans) (C) for RSM (median of 620 observations) compared to TLM (median of 2366 observations), with corresponding population sizes of 20 and 110 animals respectively. Increasing the interval between scoring timepoints negatively impacts both metrics for both assay types, particularly at small sample sizes. The same set of simulated TLM experiment data was used for all TLM analysis treatments. (A) The total number of animal observations (i.e. the number of times any animal is scored during the course of a trial, including live and dead animals, and those that have been scored on previous observations for TLM) scales differently with sample size between RSM (dark red) and TLM (blue) assays. “Sample size” is the number of animals per observation for RSM, and the number of animals per trial for TLM. The size of the circle at a given position indicates the number of trials which had that result. (B) SE (standard error) across the median lifespans for the 10,000 simulated lifespan trials for each sample size, for the given assay type and analysis method, indicating how precision improves with increasing sample size between the assay types. Right-censoring vs. interval-censoring shifts the medians but does not alter the dispersion, so only one set of results is shown for Kaplan-Meier analyses (denoted “KM”). For the left-most column (daily scoring) the horizontal black dashed line indicates a sample size for which SE is similarly low (0.003) across RSM and TLM (with parametric fitting to a logistic distribution, denoted “logistic”), 20 animals per observation for RSM and 110 animals per trial for TLM. (C) MSE (mean-squared error) across the median lifespans for the 10,000 simulated lifespan trials for each sample size, for the given assay type and analysis method, indicating how accuracy improves with increasing sample size between the assay types. For the left-most column (daily scoring), the horizontal black dashed line indicates a sample size for which MSE is similarly low (0.1) across RSM and TLM (with parametric fitting to a logistic distribution, denoted “logistic”), 20 animals per observation for RSM and 110 animals per trial for TLM.

FIGURE 4

Non-parametric right-censored treatment of TLM data leads to right-edge bias, which is not observed with interval censoring, nor with parametric (logistic) fitting of TLM or RSM data. Across 10,000 simulated experiments for a sample size of 20 for RSM (# animals per observation) and 110 for TLM (# animals per trial) we find that non-parametric analysis of TLM data with right-censoring (denoted “Right-censor”) leads to a positive bias in estimated median (A) and mean (B) lifespan equivalent to approximately half the scoring interval. The dashed orange line indicates the mean/median of the generating logistic distribution (20 days). This bias is not observed in the TLM experiments analyzed with either a non-parametric treatment and interval censoring (denoted “Interval-censor”), or parametric treatment (logistic curve fit, denoted “Logistic”), nor with simulated RSM experiments and associated parametric analysis.

FIGURE 5

RSM shows reasonable performance even with populations generated from a distribution other than logistic. As expected, simulation of systemic errors in an experiment similarly impacts all assay and analysis types. (A–C) While C. elegans survival data has been shown to fit better to a logistic distribution than to Gompertz (Vanfleteren et al., 1998; Stroustrup et al., 2016), the possibility of treatments or genetic perturbations that can alter the shape of the curve cannot be dismissed. As such, we performed an experiment to ascertain how such a change in the generating distribution would affect the performance of the two assay types, while maintaining fitting to the logistic distribution for parametric treatments. (A) Animals were drawn from a logistic (orange) or Gompertz (yellow) distribution, both having median lifespan of 20 days. Parameters were chosen for the Gompertz curve such that the slopes were similar (see Materials and Methods). Note that unlike with logistic, the median and mean are not equal for Gompertz. (B) Median lifespan across 10,000 simulated lifespan trials for the indicated assay type and analysis treatment, for simulated animals with lifespan drawn from a logistic distribution (left) or Gompertz distribution (right). 20 animals per observation were used for simulated RSM experiments, and 110 animals per trial were used for the simulated TLM experiments. The input TLM experiment data was the same across the different analysis types. The “true median” of the generating distribution is indicated by the dashed orange line. As expected, non-parametric analysis of TLM experiments is not affected by the change in generating distribution, while the mismatch between the generating and fitting distributions results in a slight underestimate of lifespan for the RSM and parametric treatments of TLM- 19.48 and 19.71 days, respectively (median of the median lifespan across the 10,000 simulated trials, Supplementary Table S3). Precision is minimally affected by the change in distribution, as indicated by standard error (SE) of the median lifespans ((C), left side), while mean-squared error (MSE) ((C), right side) reflects the shift observed in (B) when compared to the similar plots of SE and MSE for the case of a non-mismatched distribution in Figure 3. (D,E) Systemic experimental errors, exemplified by a simulated case of mixing two C. elegans strains with different lifespans, affect all assay and analysis treatments similarly. (D) Schematic representation of a simulation of an experiment in which two strains with different lifespans are mixed. “Strain A” (dark gray) has median lifespan of 20 days, and represents the strain for which the assay might have been intended, but it becomes contaminated with “Strain B” (blue) which has substantially shorter lifespan of 16 days. The corresponding logistic curves show the generating distributions for these two populations. The result is a mixed population with 67% of animals having median lifespan of 20 days, and 33% of animals having median lifespan of 16 days, which yields a lifespan experiment result that is not the 20 days our experimenter expected (two example RSM experiment results from the mixed simulation are shown in red and light red). (E) Both assay types, and all TLM analysis approaches similarly reflect the result of the mixed experiment, yielding median lifespan estimates in between 16 and 20 days. 20 animals per observation were used for simulated RSM experiments, and 110 animals per trial were used for the simulated TLM experiments.

FIGURE 6

The RSM is more resilient to incorrect scoring of survival, as well as premature animal death caused by accidentally rough handling. As animals age, they demonstrate locomotory decline and increased fragility, making them more difficult to score for viability, and increasing the possibility of scoring errors to which RSM is more robust. (A) An illustration of the phases of movement and the decline thereof during C. elegans lifespan (Huang, C., Xiong, C., and Kornfeld, K. (2004), Newell Stamper, Breanne L., et al. (2018)). Young animals move spontaneously and frequently, with a consistent sinusoidal motion (indicated by blue bar). For scoring lifespan assays, it is usually quickly determined if animals in this stage are alive, as any that have temporarily stopped moving can be stimulated by gently tapping the plate. In “middle aged” animals, movement becomes less coordinated and frequent, but they still may be moving without any external stimulus (indicated by yellow bar). A stronger plate tap, or occasional prod with a pick or fiber may occasionally be necessary to stimulate movement. In aged animals, spontaneous translational body movement has largely ceased, with only occasional posture shifts of the head or tail (indicated by red bar). In this stage, it is usually not immediately evident if an animal is alive or dead, and touch stimulation is usually necessary to determine their vital state. As this can be laborious, the possibility of making an error in determining the state of an animal may increase, as does the chance of accidentally killing animals that were actually alive in the process of poking them to stimulate movement. Even in isogenic C. elegans populations in identical conditions housed on a single plate, there may be substantial variability between the movement states of different animals, represented as the color gradients of the bars. (B) Simulated error probability curves shown in relation to the generating logistic distribution (black solid curve) for single-sample experiments (mean/median = 20 and s = 2). Error rates were simulated at a fixed probability or as a function of time. Constant error rates were set at either 2, 4, or 10% (dotted lines). Modeling error as a function of time more closely mimics the increased likelihood that an experimenter will make a mistake as animals become paralyzed or fragile with age. The hazard function from the logistic distribution is used to increase the error probability as animals age, starting from either mid-life (dashed lines) or late-life (solid colored lines) reaching the maximum (2, 4, or 10%) near the end of life. The constant error and 10% error scenarios are not intended as a simulation of likely error rates during actual experiments, but serve to demonstrate the degree to which RSM improves robustness to accidental scoring and handling errors under extreme conditions. (C–H) Median lifespan across 10,000 simulated RSM and TLM trials for the indicated error models and analysis treatments, with a sample size of 20 animals for RSM (per observation) and 110 for TLM (per trial), and assuming daily scoring. The orange dashed line indicates the mean/median of the generating logistic distribution (20 days). (C) No simulated error. (D) Constant probability of mis-scoring. (E,F) Mis-scoring with probability modeled as function of time, rising from 0% to the indicated maximum rate starting in mid-life (E) or late-life (F). (G,H) Simulated accidental death, in which an animal which was actually alive is accidentally killed by rough handling and subsequently called as dead, with probability modeled as function of time, rising from 0% to the indicated maximum rate starting in mid-life (G) or late-life (H).

FIGURE 7

RSM provides similar statistical detection power to TLM, with many fewer necessary animal observations. 100 lifespan experiment trials were simulated for RSM and TLM each for every combination of sample size (5–50 per observation for RSM, and 2–150 per trial for TLM, both in increments of 5) and generating distribution mean/medians from 16—24 days (in increments of 0.1 days), all assuming daily scoring. For a given sample size and assay/analysis type, each trial was compared using an appropriate testing strategy against a reference population generated from a distribution with mean/median of 20 days. p-values determined from these tests were corrected for multiple testing (FDR) within the set of likewise trials and used to calculate power at alpha level of 0.01. (A) Statistical detection power is comparable across RSM and all analysis treatments of TLM for a broad range of effect sizes when considering 20 animals per observation for RSM and 110 animals per trial for TLM. (B,C) For an effect size of 10% (+/- 2 days compared to the reference population) RSM saturates power at a number of animals per observation (B) that corresponds to a far smaller number of total animal observations per trial compared to TLM (C). The analysis treatments of TLM experiments all perform similarly when the scoring interval is held constant.