Class DataReconciliationEngine

java.lang.Object
neqsim.process.util.reconciliation.DataReconciliationEngine
All Implemented Interfaces:
Serializable
public class DataReconciliationEngine extends Object implements Serializable
Data reconciliation engine using weighted least squares (WLS) with linear constraints.

Adjusts measured process variables so that mass (and optionally energy) balance constraints are exactly satisfied, while minimizing the weighted sum of squared adjustments. The weights are inversely proportional to the measurement variance (1/sigma^2).

The engine uses the classic linear WLS data reconciliation formula:

Given measurements y, covariance V = diag(sigma_i^2), and constraints A*x = 0:

$$x_{adj} = y - V \cdot A^T \cdot (A \cdot V \cdot A^T)^{-1} \cdot A \cdot y$$

After reconciliation, a global chi-square test and per-variable normalized residual test are applied to detect gross measurement errors at a configurable confidence level.

Typical usage from Python (via jneqsim): 

engine = DataReconciliationEngine()

# Add measured variables with name, measured value, uncertainty (sigma)
engine.addVariable(ReconciliationVariable("feed", 1000.0, 20.0))
engine.addVariable(ReconciliationVariable("product", 600.0, 15.0))
engine.addVariable(ReconciliationVariable("waste", 380.0, 10.0))

# Add constraint: feed - product - waste = 0 (mass balance)
engine.addConstraint([1.0, -1.0, -1.0])

# Reconcile and check results
result = engine.reconcile()
print(result.toReport())
Version:
1.0
Author:
Process Optimization Team
See Also:

  • Field Details

    • serialVersionUID

      private static final long serialVersionUID
      See Also:

    • logger

      private static final org.apache.logging.log4j.Logger logger
      Logger for this class.

    • variables

      private final List<ReconciliationVariable> variables
      Registered reconciliation variables.

    • constraintRows

      private final List<double[]> constraintRows
      Constraint rows (each double[] has length = number of variables).

    • constraintNames

      private final List<String> constraintNames
      Optional constraint names for reporting.

    • grossErrorThreshold

      private double grossErrorThreshold
      Gross error detection threshold (z-value). Default 1.96 for 95% confidence. Set to 2.576 for 99%.

  • Constructor Details

    • DataReconciliationEngine

      public DataReconciliationEngine()
      Creates an empty data reconciliation engine.

  • Method Details

    • addVariable

      public DataReconciliationEngine addVariable(ReconciliationVariable variable)
      Adds a variable to the reconciliation problem.
      Parameters:
      variable - the reconciliation variable with measured value and uncertainty
      Returns:
      this engine for chaining

    • addConstraint

      public DataReconciliationEngine addConstraint(double[] coefficients)
      Adds a linear constraint row: sum(coefficients[i] * variable[i]) = 0.

      For a mass balance around a mixer with feed1, feed2 going in and product going out: addConstraint(new double[]{1.0, 1.0, -1.0}) means feed1 + feed2 - product = 0.

      Parameters:
      coefficients - array of coefficients, one per variable in order added. Use +1 for inlets, -1 for outlets.
      Returns:
      this engine for chaining
      Throws:
      IllegalArgumentException - if coefficients length does not match variable count

    • addConstraint

      public DataReconciliationEngine addConstraint(double[] coefficients, String name)
      Adds a named linear constraint row: sum(coefficients[i] * variable[i]) = 0.
      Parameters:
      coefficients - array of coefficients, one per variable
      name - descriptive name for this constraint (e.g., "Separator mass balance")
      Returns:
      this engine for chaining
      Throws:
      IllegalArgumentException - if coefficients length does not match variable count

    • getVariables

      public List<ReconciliationVariable> getVariables()
      Returns the current list of variables.
      Returns:
      unmodifiable list of registered variables

    • getVariableCount

      public int getVariableCount()
      Returns the number of variables.
      Returns:
      variable count

    • getConstraintCount

      public int getConstraintCount()
      Returns the number of constraints.
      Returns:
      constraint count

    • getGrossErrorThreshold

      public double getGrossErrorThreshold()
      Returns the gross error detection threshold (z-value).
      Returns:
      threshold value (default 1.96 for 95% confidence)

    • setGrossErrorThreshold

      public DataReconciliationEngine setGrossErrorThreshold(double threshold)
      Sets the gross error detection threshold (z-value).
      Parameters:
      threshold - z-value, e.g., 1.96 (95%), 2.576 (99%), 3.291 (99.9%)
      Returns:
      this engine for chaining

    • clear

      public void clear()
      Clears all variables and constraints.

    • clearConstraints

      public void clearConstraints()
      Clears only the constraints, keeping variables.

    • reconcile

      public ReconciliationResult reconcile()
      Performs the data reconciliation using weighted least squares with linear constraints.

      The closed-form solution is: 

      x_adj = y - V * A ^ T * (A * V * A ^ T) ^ (-1) * A * y

      where y = measurement vector, V = diag(sigma^2), A = constraint matrix.

      After solving, computes normalized residuals and flags gross errors.

      Returns:
      reconciliation result with adjusted values and statistical tests

    • solveWLS

      private ReconciliationResult solveWLS(int n, int m, long startTime)
      Solves the weighted least squares reconciliation problem.

      Builds the measurement vector y, covariance matrix V, and constraint matrix A, then applies the closed-form WLS formula. Computes constraint residuals before and after reconciliation, objective value, global chi-square test, and per-variable gross error detection.

      Parameters:
      n - number of variables
      m - number of constraints
      startTime - system time when reconciliation started, for timing measurement
      Returns:
      the reconciliation result

    • detectGrossErrors

      private void detectGrossErrors(int n, org.ejml.simple.SimpleMatrix vAdj)
      Detects gross errors using normalized residual tests.

      For each variable i, the normalized residual is: 

      r_i = (x_adj_i - y_i) / sqrt(V_ii - V_adj_ii)

      If |r_i| exceeds the gross error threshold (z-value), the variable is flagged.

      Parameters:
      n - number of variables
      vAdj - covariance matrix of reconciled adjustments (V - V*A^T*(AVA^T)^-1*A*V)

    • getChiSquareCritical

      private double getChiSquareCritical(int dof)
      Returns an approximate chi-square critical value at 95% confidence.

      Uses the Wilson-Hilferty approximation for the chi-square distribution inverse CDF at the 95th percentile.

      Parameters:
      dof - degrees of freedom (must be positive)
      Returns:
      approximate chi-square critical value at 95% confidence

    • reconcileWithGrossErrorElimination

      public ReconciliationResult reconcileWithGrossErrorElimination(int maxEliminations)
      Performs reconciliation with iterative gross error elimination.

      After each reconciliation, the variable with the largest normalized residual exceeding the threshold is removed (its uncertainty is set to a very large value, effectively ignoring it). The reconciliation is repeated until no gross errors remain or the maximum elimination count is reached.

      Parameters:
      maxEliminations - maximum number of variables to eliminate
      Returns:
      reconciliation result with the final set of gross errors identified

    • addMassBalanceConstraint

      public DataReconciliationEngine addMassBalanceConstraint(String name, List<String> inletNames, List<String> outletNames)
      Convenience method to add a mass balance constraint around a node.

      Specifies which variables are inlets (+1) and which are outlets (-1) by their names. All other variables get coefficient 0.

      Parameters:
      name - constraint name (e.g., "Separator mass balance")
      inletNames - names of inlet variables
      outletNames - names of outlet variables
      Returns:
      this engine for chaining
      Throws:
      IllegalArgumentException - if a variable name is not found

    • addMassBalanceConstraint

      public DataReconciliationEngine addMassBalanceConstraint(String name, String[] inletNames, String[] outletNames)
      Convenience method to add a mass balance constraint using arrays of names.
      Parameters:
      name - constraint name
      inletNames - inlet variable names
      outletNames - outlet variable names
      Returns:
      this engine for chaining
      Throws:
      IllegalArgumentException - if a variable name is not found

    • findVariableIndex

      private int findVariableIndex(String name)
      Finds the index of a variable by name.
      Parameters:
      name - variable name to search for
      Returns:
      index in the variables list, or -1 if not found

    • getVariable

      public ReconciliationVariable getVariable(String name)
      Returns a variable by name.
      Parameters:
      name - the variable name
      Returns:
      the variable, or null if not found