AFP — validation is still needed

13. A study of automated function point counting software has identified variation when compared to manual counts as high as 800 percent. Over 90 percent of the counts over-represented the actual count by 40 percent or more. The remaining count under-represented the actual count by 400 percent.3 While the proposed model should help to reduce these variations, validation is still needed.
Comment, not a defect or issue with the OMG proposal: Ironically, only manual function point counting can validate any assertion that a product conforms as “close as possible” to the IFPUG ISO standard.

Since no citation was provided for the study referenced here, we cannot evaluate its validity. However, manual Function Point counters have repeated been shown to disagree by 10% or more with each other. Our experience with automated counting is that it often uncovers aspects of the application that had been overlooked by manual counters and that is one source of higher counts from automation.
Although this point will be controversial, validation with manual counts will not be the critical issue in the long term. Ultimately methods for manual estimating from requirements specs will seek to be validated against consistent, accurate counts of function points from the resulting source code. Since the IFPUG-based ISO standard contains the ambiguities requiring judgment that result in differences among manual counters, it has a weaker basis in measurement theory than a clearly defined standard that supports automation.
Revised Text:
>none<
Disposition: Closed, no change