Science is the accumulation of knowledge by repeated observation or experimentation. To be persuasive in an article in science, one has to prove that the results can be replicated. This could be through performing the entire experiment in isolation multiple times or by investigating that the results of independent studies are collected, and a formal process of statistical inference is used, usually using confidence intervals (CIs) or statistical significance tests. In the last few years, several journals have improved their guidelines to authors and editorial policies to ensure that errors are outlined in the figure legends if they appear in the images–and also to establish guidelines for using image processing software. This has improved image quality and decreased the number of papers featuring figures that display error bars but don’t explain the error bars. But, there are still issues with how replicated results are presented and understood. Since biological experiments are complex, the results of replicates are typically used to evaluate the effectiveness of the investigation. However, these replicates aren’t independently tested tests for the hypotheses, which means they can’t be used to prove the validity of the primary results. In this paper, we present our argument to show why replicate data cannot be used to make conclusions regarding the validity of a theory and, consequently, shouldn’t be used to determine CIs and the P values or be included in figures.
…replicates cannot be considered independent tests to test the hypothesis. Therefore they are not able to prove the reliability of the principal findings.
Let’s say we’re testing the idea that the BDL protein Biddelonin (BDL) encoded in the Bdl gene is necessary for bone marrow cells to expand in response to the cytokine HH CSF. Fortunately, we can have wild-type (WT) and homozygous Bdl mice that have been deleted from their available genes and a vial of recombinant HHCSF. We make the bone marrow suspension cells derived from one WT and one Bdl mouse (same littermates of the same sex from Bdl+/- heterozygous crossover) and then count the cell suspensions with a hemocytometer. We adjust them to ensure that there are 1 x 10 105 cells per milliliter within the soft agar medium solution. We add 1 ml of aliquots of the suspension into ten 35×10 millimeter Petri dishes, each containing 10 milliliters of either Saline or recombinant purified mouse HHCSF.
Therefore, we place four plates of 10 soft agar cultures in the incubator. One container contains Bone marrow cell lines that are WT that have Saline; the other includes Bdl -+/– cells that have Saline; the third one has WT cells with HH CSF, and the fourth Bdl -+/– cells that have HH-CSF. After one week, we take these plates from the incubator and measure each colony (groups of more than 50 cells) within each plate using dissecting microscopes. Several colonies counted are listed. Bone Marrow cells were placed in soft agar containing 1 ml cells with or without 1mM HH-CSF. The number of colonies per plate was measured after one week. WT wild type, or wild type.
We could plot the count of each plate on graphs. Suppose we planned only the colony counts on one scale for each type ( Fig 1A illustrates the data available for the plate). In that case, It’s evident that HH-CSF is required to create many colonies, but it’s not immediately apparent if the response of Bdl -or- cells is different in comparison to WT cells. In addition, the graph appears to need to be’scientifiable’ enough; there aren’t any errors or P-values. In addition, by displaying the data only for one plate, we’re violating the basic rule of science. All relevant data must be documented and subjected to an investigation unless justifications can be provided for why data should be omitted.
The size of the bars of error in ( B), ( C), and ( D) show the variance in sampling for the replicates. In each case, the SDs from the replicas could be expected to be roughly equivalent to the square roots of the average quantity of colonies. Additionally, axes should begin at zero, except in the case of exceptional situations, like log scales. SD, standard deviation; SE, standard error.
To enhance the look of it to make it look better, we can make it easier to add the mean numbers of colonies from the initial three plates from each species onto the diagram ( Fig 1B) along with error bars that indicate what is the average error (SE) for the three values for each type. It now looks more like an image from a top-of-the-line journal. Still, using the data of the three plates that replicated each, we can evaluate the significance of the statistical differences in the responses of Bdl -/- and WT cells Bdl -cells to HH-CSF; we find that the difference is not significant. Cells to HH CSF, we observe P greater than 0.05, and this indicates that there is no significant difference.
But, even though the differences are statistically significant, the widths of the columns aren’t drastically different, and it’s hard to discern those error bars. To fix this, one could set the y-axis at 40 instead of the zero value to highlight the different responses to the HH-CSF. Even though this means removing the control of Saline, they’re less crucial than the visually impacting journals with prominent names.
With just a tiny amount of effort and without further studies, we’ve transformed mediocre results ) into one that provides the most substantial evidence that BDL is essential for the response to HH CSF which has a high P-value as well as a diagram which looks like it could be to one of the most prestigious journals.
What is the problem? The first issue is that our results do not support the notion that BDL is necessary for bone marrow cells to expand in response to HH-CSF. They contradict the idea. Clearly, bone marrow cells are growing even in absent BDL; however, they aren’t as high as when Bdl genes are present. Terms like required essential’,’ required,’ and ‘obligatory’ aren’t related, yet they are commonly used when partial effects are observed. In the simplest sense, we need to rethink our idea, maybe changing it to “BDL is needed for a full response of bone marrow colony-forming cells to the cytokine HH-CSF.”
…by giving the data only for one plate, we’re violating the basic rule of science, which states that every relevant data must be recorded and subjected to an analysis…
Another issue concerns how calculations on statistical significance and statistical importance depend on the average of replicates. However, the ten replicates tested in the four conditions were all made of one bone marrow cell taken from a single mouse. Therefore, we can determine a statistically significant distinction in the number of colony-forming cells in bone marrow cell suspensions from this particular WT mouse as well as the bone marrow suspension of the mouse with that specific gene deleted. We’ve only made one study. Therefore, the number of plates = one regardless of how many plates we count as replicates. We have to repeat our experiment several times to determine if the results can be applied to all mice with WT and Bdl -and- mice. We will also conduct independent comparisons using a variety of mice of each kind.
Leave a Reply