|
Data Integrity During Product Development
This series will cover three interrelated topics as they affect the integrity of functional response data during development of technology and product performance capability. They are:
- Measurement Systems
- Sample Size
- Adjustable Prototypes
This article discusses data integrity resulting from the best practice known as Measurement Systems Analysis (MSA). When practicing Critical Parameter Development & Management (CPD&M), every new, unique or difficult (NUD) functional response must pass the first of the Big 7 checks for criticality - Measurability. If you cannot measure a candidate critical functional response then you are dead in the water. This issue MUST be resolved in order to proceed through the remaining Big 7 metric tests to prove the NUD functional response is or is not critical.
Measurement Systems: Definition and Characteristics
A Measurement System can be defined as:
Fixtures, procedures, transducers (gauges), instruments, software, lab environment and personnel that, together, make it possible to assign a number to a measured characteristic or response
Measurement systems have the following characteristics:
Accuracy - ability to calculate the mean from a sample of data
Precision - ability to calculate the variability from a sample of data
Measurement errors can manifest in the following ways:
1) Bias - the mean of measured values differ from a true standard value
2) Non-Linearity - Bias and variability across a range of readings from low mean values to high mean values (in light of a range of true standard values)
3) Instability - the effect on the accuracy and/or precision change over time or environmental conditions
4) Variability - the amount of variability being recorded around one or more averages
- Reproducibility - the effect of the user of the measurement system on the variance
- Repeatability - the effect of repeated sampling on the measurement system on the variance
5) Poor Resolution - the smallest amount of discrimination a measurement system can detect between two different samples of data (caution: can be noticeably affected by sample size!)
Other sources of error can include:
- Poor experimental measurement technique
- Broken equipment / fixtures
- Mis-read / mis-recorded data
- Improperly set control factors
- Poor communication
Implications of Errors from the Measurement System
The integrity of data acquired from a physical experiment or test scenario must be underwritten by accounting for all the aforementioned areas where errors can occur within the reported samples as they are then computed into mean and variance information. We can be wrong about the mean; we can be wrong about the standard deviation - which means we can be wrong about the Cp & Cpk indices that depend on them!
Recognize that the mean value based upon a sample of data can be lower than or higher than the actual mean that truly exists. This is what Bias can do to your perception of the response. This will impact the value of Cpk, the mean shifted capability index. This is very important because the Cpk value shows the effect of noise factors as they shift the mean and also inflate the variance around the mean.
Recognize that the mean and the variance value based upon a range of sample of data can be under or over stated in terms of Linearity. In addition, if you are specifically studying the variance of a sample of data, the reported value, s2, will typically be inflated due to the contribution to the variance coming from the repeatability and reproducibility within the measurement system.
Recognize that measurement systems are susceptible to the effects of "noise factors" that affect the robustness of the data acquisition system. These include sensitivities to Unit-to-Unit variation from the manufacturing, assembly and supply chain of the parts and materials that go into the hardware, External/Environmental noise factors that affect the functions within the hardware and finally, the deteriorative factors that can degrade the performance of the data acquisition hardware. It is important that you understand the robustness of your measurement systems. This is often addressed in the light of Stability and Gage R&R Studies.
Measurement Systems: Buy or Create?
Do not be penny-wise and pound-foolish when it comes to investment in measurement equipment. Be sure it meets your requirements and buy the best! If you cannot buy a data acquisition system for your new, unique and difficult (NUD) measurement needs, then you are going to have to invent one! If your development organization is practicing a sound strategy for product and technology portfolio development, then projects of this type will be no surprise. You will need to forecast when and how likely new measurement system development projects are going to occur. There should be a budget and resourcing plan that can be activated as part of your strategic contingency plan when such demands emerge as part of your innovation work flow.
Inventing a new measurement system requires the following steps, which are no different than any other development project:
1. Identify the Needs, Constraints & Conflicts for measuring a Functional Response
2. Translate, Rank & Prioritize into "measurable" measurement system Requirements
3. Generate & Evaluate System Level Measurement System Concepts for a new measurement system including its application process
4. Develop Base-line, Sub-level Design Elements for the new measurement system for nominal use conditions
5. Develop Robustness & Optimize the Design Elements under stressful or noisy conditions
6. Integrate Robust & Tunable Sub-level Designs into the complete measurement system; establish tolerances & balance the integrated system to meet requirements under nominal & stressful conditions
7. Verify the Integrated System Meets all Requirements & is under control for use in its nominal & stressful environments; verify and document an application/use/maintenance guide, continuous improvement plan & transfer to new measurement system owners
Gage R&R and Responsibilities
Some functions may need measurement systems that require people to interact with the hardware that acquires the data samples. These cases will require the technical team to assess the need for a Gage Repeatability & Reproducibility Study (GR&R). If the operator of the instrument interacts with the hardware such that it can physically affect the reading, then a GR&R should be conducted. Otherwise normal calibration and maintenance procedures should be followed.
The Lead Systems Engineer is accountable for ALL NUD functional measures and the capability of the measurement systems used across the development project. The teams at each design sub-level are responsible for the benchmarking, purchase, calibration, maintenance, procedural use and GR&R studies of their measurement systems.
Conclusion
All of these issues should encourage the development of a proactive plan to assure you will not be deceived by the data coming from your measurement systems. Your data is the strongest link you have to the truth. After capable measurement system performance, there are two additional issues to assure high integrity data: proper sample sizes and adjustable prototypes. We will address these in future articles.
| 
