C ＃ use laida criterion (3) σ Criteria) reject abnormal data (.Net reject singular values in a group of data)

1、 Problem presentation ：

Battery production , The measurement result data of a batch of batteries ：

among , The voltage of some batteries is too low and too high , Does not conform to the normal distribution .

Now we need to eliminate these abnormal battery data .

2、 Methods the principle ：

3σ The criterion is also known as the laida criterion , It assumes that a set of test data contains only random errors , The standard deviation is calculated and processed , Determine an interval according to a certain probability , If the error exceeds this range , It's not a random error, it's a gross error , Data containing this error should be eliminated .
In the normal distribution σ Represents the standard deviation ,μ For mean .x=μ That is, the axis of symmetry of the image
3σ principle ：
The values are distributed in （μ-σ,μ+σ) The probability of 0.6827
The values are distributed in （μ-2σ,μ+2σ) The probability of 0.9544
The values are distributed in （μ-3σ,μ+3σ) The probability of 0.9974
It can be said that ,Y The values of are almost all concentrated in （μ-3σ,μ+3σ) Within the interval , There is no possibility of going beyond that 0.3%.

3、C# The concrete realization of ：

// Define voltage - Class of quantitative relationship
    public class VoltageCount
    {
        public Double Voltage { get; set; }
        public int CountV { get; set; }
        public VoltageCount()
        {
        }

        public VoltageCount(Double voltage, int countV)
        {
            this.Voltage = voltage;
            this.CountV = countV;
        }
    }

// Key classes use the laida criterion （3σ Rules ） Eliminate data exceptions

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Photo.QQAA.Net.Helper
{
    /// <summary>
    /// Use the laida rule （3σ Rules ） Eliminate data exceptions
    /// </summary>
    public class ExceptionVoltageHelper
    {
        List<VoltageCount> listVoltageCount;
        double average = 0.0;
        int _badDataCount = -1;// Number of singular values

        /// <summary>
        /// Get the number of singular values
        /// </summary>
        public int BadDataCount
        {
            get { return _badDataCount; }
        }

        public ExceptionVoltageHelper(List<VoltageCount> list)
        {
            this.listVoltageCount = list;
            SetAverage();
        }

        /// <summary>
        /// Obtain the average voltage value
        /// </summary>
        /// <returns></returns>
        protected double GetAvgVoltage()
        {
            double avg = 0;
            double total = 0;
            int allCount = 0;
            foreach (VoltageCount vc in listVoltageCount)
            {
                double v = vc.Voltage;
                int c = vc.CountV;
                total += v * c;
                allCount += c;
            }
            avg = total / (allCount * 1.0);

            return Math.Round(avg, 3, MidpointRounding.AwayFromZero);
        }

        /// <summary>
        /// Average
        /// </summary>
        /// <returns></returns>
        void SetAverage()
        {
            this.average = GetAvgVoltage();
        }

        /// <summary>
        /// Standard deviation
        /// </summary>
        /// <returns></returns>
        double StandardDeviation()
        {
            List<double> listDataV = new List<double>();
            foreach (VoltageCount vc in this.listVoltageCount)
            {
                double v = vc.Voltage;
                int countV = vc.CountV;
                for (int i = 0; i < countV; i++ )
                {
                    listDataV.Add((v - this.average) * (v - this.average));
                }
            }
            double sumDataV = listDataV.Sum();
            double std = Math.Sqrt(sumDataV / (listDataV.Count - 1));

            return std;
        }

        public List<VoltageCount> GetGoodList()
        {
            _badDataCount = 0;
            double sd3 = StandardDeviation() * 3;//3 Times the standard deviation
            List<VoltageCount> listVC = new List<VoltageCount>();
            foreach (VoltageCount vc in this.listVoltageCount)
            {
                if (Math.Abs(vc.Voltage - this.average) <= sd3)
                {
                    listVC.Add(vc);
                }
                else
                {
                    _badDataCount += vc.CountV;
                }
            }

            return listVC;
        }

    }
}

4、 Limitations and precautions ：

Ben 3σ The rule is limited to the processing of sample data with normal or approximate normal distribution , It is also applicable when there are many groups of data .
The principle and method of this discrimination processing are based on the premise that the number of measurements is sufficiently large , When the number of measurements is small, it is not reliable to eliminate gross errors with criteria . therefore , In the case of fewer measurements , It's best not to use the guidelines , And use other criteria .