Read 0%PDF
Calibration fundamentalsModel selection6 min read

Linear vs. quadratic calibration curves: how to choose, and how to defend the choice.

A quadratic calibration curve models a detector whose response is genuinely curved. The regulations allow it only with more standards and tighter conditions, and a better fit alone does not justify it.

The short version
  1. EPA 8000D, 40 CFR 136.6 and ICH Q2(R2) all allow a quadratic calibration curve, with conditions attached.
  2. Under EPA rules it needs at least six levels, a monotonic curve, no saturation, and no extrapolation.
  3. Choose the model by its residuals: a systematic arch in the linear residuals is your evidence of real curvature.

Why a quadratic calibration curve needs a reason.

A linear ICAL fails at the low end at 4:40 on a Friday. The data system offers a quadratic fit, one click away, and the curve passes.

Every initial calibration (ICAL) rests on a model of how the detector responds to concentration. For most chromatographic methods the first choice is a straight line: an average response factor or a linear regression. The quadratic is the usual fallback when the line does not fit.

The click is easy. The justification takes more work. A second-order model adds a parameter, bends the curve to follow the data, and almost always improves the fit statistics.

That is exactly why reviewers and auditors look at it closely. A better fit does not show that the detector is curved. It may be a sign that something else is wrong.

What the regulations allow.

The major frameworks all permit second-order calibration. Each one attaches conditions to it.

SourceMinimum levels for a quadraticConditions
EPA SW-846 Method 8000D, Sec. 11.5.36 (7 for cubic)r² ≥ 0.99; no more than four parameters; curve continuous and monotonic over the range; not to compensate for detector saturation or poor maintenance
40 CFR 136.6(b)(4)(x)(A)6Acceptable when calibration data routinely display quadratic character; no extrapolation above the most concentrated calibrator
ICH Q2(R2), Sec. 3.2.2.2Not specifiedSuitability of a non-linear model assessed by non-linear regression; linearity of response is not required

Method 8000D is the most specific. It asks analysts to confirm that every tangent to the curve within the calibration range has the same sign and that none is zero. In plain terms, the curve must keep rising across the whole range [1].

Method 8000D also states that non-linear calibration is not meant to compensate for detector saturation or to avoid proper instrument maintenance. The wastewater rule in 40 CFR 136.6 frames the same idea from the other side. A quadratic fit may be acceptable when the data routinely display quadratic character [2].

One curved calibration is not a pattern. Before you select a quadratic, check whether your earlier calibrations curve the same way.

Linear vs. quadratic calibration on one data set.

The table below is an eight-level calibration from 1 to 200 ppb. It was generated from a detector whose response bends slightly downward at the top of the range, as electron capture and some mass spectrometric detectors can. Each standard was back-calculated against three fits.

True conc. (ppb)Response (area)Linear, unweightedLinear, 1/x²Quadratic, 1/x²
1999−313.1%−3.5%−0.6%
22,006−143.3%+3.4%+1.0%
54,947−43.4%+5.7%+0.5%
109,727−11.6%+5.2%−0.5%
2524,083+7.5%+4.9%+0.3%
5046,284+9.5%+1.1%−1.1%
10088,596+7.9%−3.1%+0.0%
200157,715−2.6%−13.7%+0.4%
RSE141.9%7.2%0.8%
r² of fitted curve0.995880.970380.99998

Illustrative data set constructed for this article. Values are the percent error of each back-calculated standard. Every derived value is computed from the responses shown.

7.2%
RSE of the linear 1/x² fit, inside the 20% default limit
−13.7%
Error at the top standard, linear 1/x²
1.1%
Largest error at any level, quadratic 1/x²

The unweighted line clears the familiar r² of 0.99 and still fails badly at the low end. That is the pattern described in Insights 01. Weighting the line by 1/x² fixes the low end.

By Method 8000D’s model-independent checks, the weighted line now passes. Every standard is within 30%, and the RSE of 7.2% is under 20% [1]. The quadratic fit passes as well, holding every standard within 1.1%.

So both models are acceptable on paper. The acceptance criteria cannot choose between them.

Read the residuals, not just the verdict.

Figure 1 Percent error at each level, linear vs. quadratic Linear, 1/x² Quadratic, 1/x²
−20%−15%−10%−5%0%+5%+10%125102550100200Calibration level (ppb)

The weighted linear errors are not random. They rise through the middle of the range and fall away at both ends, reaching −13.7% at 200 ppb. That arch is the signature of a straight line laid across a curved response.

The quadratic errors show no structure. They scatter around zero at every level.

The acceptance criteria tell you a model is acceptable. The residuals tell you whether it is right.

ICH Q2(R2) makes the same point for pharmaceutical procedures. It asks that the impact of any non-random pattern in the residuals be assessed [3]. A residual plot is your evidence that a curved model describes the detector and is not absorbing an error.

The reverse case matters just as much. Suppose a quadratic is needed only because the top standard has flattened. The detector may be saturating, the standard may be mis-made, or the source may need cleaning.

Method 8000D says plainly that a non-linear model is not the remedy for those problems [1]. If only the top standard pulls away, check the instrument and the standard before you change the model.

Where a quadratic calibration curve turns over.

A downward-curving quadratic has a maximum. Beyond it, two concentrations produce the same response, and the curve can no longer report a unique result.

In this example the fitted quadratic peaks at about 496 ppb, roughly 2.5 times the top standard. The maximum sits well above the top standard, so the curve is monotonic from 1 to 200 ppb, as Method 8000D requires.

The maximum is also why no framework allows a quadratic curve to be extrapolated. 40 CFR 136.6 states that concentrations should never be extrapolated for responses above the most concentrated calibrator [2]. Method 8000D requires samples above the range to be diluted back into it [1].

How to defend the choice.

  1. Start with the simplest model. Average response factor, then linear, then weighted linear, as EPA’s calibration guidance describes [4].
  2. Count the levels. A quadratic needs at least six under Method 8000D and 40 CFR 136.6. A cubic needs seven.
  3. Show the residual pattern. A systematic arch in the linear residuals is the evidence for curvature. Keep the plot with the ICAL record.
  4. Show it is routine. Curvature that appears in calibration after calibration is detector behavior. Curvature that appears once is a problem to investigate.
  5. Rule out saturation and maintenance. Record that the top standard is within the detector’s working range and that the system was in control.
  6. Confirm the curve is monotonic. Locate the vertex and confirm it lies above the top standard.
  7. Never extrapolate. Dilute any sample whose response exceeds the top calibrator.

Where WLTR fits.

Most labs evaluate the one model the CDS applies by default. They look at residuals only when a curve fails. Comparing linear and quadratic fits at every weighting, for every analyte in a multi-analyte method, is where manual review stops.

WLTR is calibration data integrity infrastructure for regulated analytical laboratories. For each analyte, it derives nine calibration regression models: average response factor, plus linear and quadratic fits across four weightings each. It scores every model against fifteen ICAL parameters, including RSE. All of it is computed orthogonally to the CDS, under U.S. Patent 12,038,448 B2 [P].

Calibration verification

Detect calibration failures before audit exposure.

See linear and quadratic fits side by side for every analyte in your ICAL, with residuals and RSE for each.

Get Compliance Gap Assessment Download the white paper
White paper · PDF
Linear vs. quadratic calibration · Letter size · October 2026
Sources
  1. U.S. EPA, SW-846 Test Method 8000D, Determinative Chromatographic Separations, Revision 5, March 2018, Sec. 11.5.3. www.epa.gov/hw-sw846/sw-846-test-method-8000d-determinative-chromatographic-separations
  2. 40 CFR 136.6(b)(4)(x), Methods modifications and analytical requirements, as amended May 19, 2021. www.ecfr.gov/current/title-40/chapter-I/subchapter-D/part-136/section-136.6
  3. ICH Q2(R2), Validation of Analytical Procedures, adopted November 2023, Sec. 3.2.2. database.ich.org/sites/default/files/ICH_Q2%28R2%29_Guideline_2023_1130.pdf
  4. U.S. EPA Forum on Environmental Measurements, Calibration Curves: Program Use/Needs, Final, October 2010. 19january2021snapshot.epa.gov/sites/static/files/2014-05/documents/calibration-guide-ref-final-oct2010.pdf
  5. U.S. Patent 12,038,448 B2, inventor Walter “Chip” Hinchman Sr. patents.google.com/patent/US12038448B2
WLTRWLTR, Inc. · Nevada C-Corp · 2026