Definition of External Input:
Consider to merge (ISO). (ISO/IEC 20926:2009) into just the last one: "(ISO/IEC 20926:2009)"
10. One of the primary benefits of using FPA is the ability to functionally size a product prior to its development based on business needs. As a result, cost comparisons can be performed against multiple proposals or COTS solutions. The OMG document omits this usage, and understandably so since its focus is on the subsequent source code. As written, the document does nothing to inform the reader of this “real limitation.”
Recommendation: State specifically that the proposed model addresses functional sizing only after code has been developed, and, that this approach is different than the IFPUG function point method.
The specification is clear that it only addresses counting from the source code. Differences with the IFPUG counting guidelines are stated and discussed in section 2.3.
Revised Text:
>none<
Disposition: Closed, no change
9. Software engineers are often incented to write more lines of code rather than fewer. The propensity to write more “code” may be represented in productivity measures (though not good ones), driven by ego (a desire to write more than their peers), or poor engineering skills (poor design or implementation2). Research suggests that size is not related to a developer only, but also to the familiarity of the developer with the problem being solved. Again, the proclivity to “write more code” is associated with poor designs, more defects, and higher support costs. So, an automated function point counting tool that utilizes source code may encourage sloppiness over elegance. Worse, the foreknowledge that the code will be measured may inspire the developer to add functionality that increases the generated size. Unfortunately, this reaction will increase product volatility and further remove the product from what was contracted to what looks “bigger” (e.g., more valuable from the developer’s (versus the customer’s) perspective).
This is a problem in how Function Point counts are used in productivity analysis rather than in the specification. This and many other points must be considered to valid data and guide improvements.
Revised Text:
>none<
Disposition: Closed, no change
13. A study of automated function point counting software has identified variation when compared to manual counts as high as 800 percent. Over 90 percent of the counts over-represented the actual count by 40 percent or more. The remaining count under-represented the actual count by 400 percent.3 While the proposed model should help to reduce these variations, validation is still needed.
Comment, not a defect or issue with the OMG proposal: Ironically, only manual function point counting can validate any assertion that a product conforms as “close as possible” to the IFPUG ISO standard.
Since no citation was provided for the study referenced here, we cannot evaluate its validity. However, manual Function Point counters have repeated been shown to disagree by 10% or more with each other. Our experience with automated counting is that it often uncovers aspects of the application that had been overlooked by manual counters and that is one source of higher counts from automation.
Although this point will be controversial, validation with manual counts will not be the critical issue in the long term. Ultimately methods for manual estimating from requirements specs will seek to be validated against consistent, accurate counts of function points from the resulting source code. Since the IFPUG-based ISO standard contains the ambiguities requiring judgment that result in differences among manual counters, it has a weaker basis in measurement theory than a clearly defined standard that supports automation.
Revised Text:
>none<
Disposition: Closed, no change
12. Function point analysis provides value and insight after the application has been delivered. When support requests and upgrades are being assessed, FPA helps to determine the value, and the size and effort associated with those change requests. Again, these values are determined from requirements, vague or specific as they may be. Code does not exist to assist with this determination, nor does an automated count using code as the driver.
This is not an issue with the specification. However, having a repository of accurate, up-to-date counts to calibrate estimates from requirements allows an organization to consistently improve its estimated methods, and this adds great value.
Revised Text:
>none<
Disposition: Closed, no change
8. Another characteristic of delivered solutions is to use middleware or “vendor supplied code.” In both of these cases, access to source code may violate intellectual property rights rendering access and then counts, impossible. This situation may further compound the effects of the “multiple languages” when present
This is a problem for any automated analysis of application source code whether for sizing, quality, or productivity. In this case code is analyzed up to the interface, since this is the part of the system on which effort will be spent. This is a problem to be handled in the contracts, not the specs.
Revised Text:
>none<
Disposition: Closed, no change
7. Today many applications are written with more than one “application language,” scripts, XML, and DML. When the same “operation” is being performed in different languages how does the “model” address reconciliation? An automated counting procedure would need to eliminate duplicate functions IF the algorithm could identify them. Identification is not particularly difficult, but determining if those similar (or duplicate) code segments are indeed duplicative and not merely slightly different could be a challenge within a language and more so across a potpourri of languages.
This is a vendor implementation issue and only vendors that can reintegrate the metadata from the parses of several languages in developing an abstract representation of the application can identify and reconcile multiple instances of the same function. Each vendor will have to make their own claim about how they achieve reconciliation and provide a demonstration to clients who ask this important question.
Revised Text:
>none<
Disposition: Closed, no change
5. As a member of the CISQ and a recognized expert in software measurement, Capers Jones was not involved with the drafting of this model. I share this point not as a “defect” but rather as an opportunity to increase the integrity of the model content.
Not sure of what your point is. Is the integrity increased because Capers was not involved? Or are you suggesting it would be increased if he became involved. The drafting team was led by an individual with even greater expertise in Function Point counting techniques, David Herron, who co-authored Function Point Analysis.
Revised Text:
>none<
Disposition: Closed, no change
4. Press releases indicate that the model will lead to “real function point” counts. The CISQ and OMG communities need to be careful about the use of speculation in their enthusiasm to foretell their success. Phrases like “real function points” do little to advance the discussion of automated standards-based measurement improvement and are not based on a defined standard.
Recommendation: For the sake of the measurement community, phrases like “ISO-based function points” or “IFPUG-based function points” or “OMG automated function point counts” might render a less ambiguous (or less divisive) interpretation
We do not use the phrase ‘real Function Points’ in the specification. Nor in fact does the word ‘real’ appear in the specification. We will try to discourage the media from using it.
Revised Text:
>none<
Disposition: Closed, no change
6. The list of model contributors does not appear to have any current Certified Function Point Specialists. This oversight may be due to the lack of the IFPUG community’s own involvement or an oversight on the part of the OMG.
David Herron who led the team is the co-founder and technical leader of one of the world’s largest collections of certified Function Point counters, the David Consulting Group. He kept IFPUG’s aware of the CISQ effort throughout the entire process and they did not choose to join CISQ. The team included some very capable PhD computer scientists who diligently to mirror the IFPUG counting rules as closely as possible while still resolving ambiguities in the counting guidelines that were necessary to automate the counting.
Revised Text:
>none<
Disposition: Closed, no change
11. Similar to the last point, function point analysis is used today to evaluate proposals from providers. At the time of the request for information, based on business requirements, source code does not exist. When requests for proposals are solicited outside the organization, “a” source code may exist but access to that code is unlikely. So, an automated function point count from code ignores one of the primary purposes of function point analysisto assist in the evaluation of proposed solutions.
This is not an issue with the specification which focuses solely on situations in which the source code exists and is available for analysis. Access to source code is an issue that must be solved in relationships between clients and vendors, not in the specification. However, the existence of this specification may bring greater focus on the issue of access to source code for analysis.
Revised Text:
>none<
Disposition: Closed, no change
3. I applaud that the model stays far from “backfiring” to ascertain a function point count.
Recommendation: Continue to avoid any “backfiring” as a means of deriving function point counts.
We promise to avoid ‘backfiring’.
Revised Text:
>none<
Disposition: Closed, no change
2. Implementations of the proposed model will need to demonstrate “consistent” counts for the same solution developed in different languages, as well as the same software solution written in the same language but authored by different persons. Statistical data published in CrossTalk has shown the same product to be as large as 22 times larger depending on the author, not the language. 1 Manual function point counting overcomes these differences because source code is never the basis of the count.
Recommendation: Using an objective independent organization, conduct such analyses and publish as it becomes available
Feasibility comparisons of automated Function Points with manual counts were conducted in a large telecommunications company prior to developing the AFP spec, and the comparison was good provided application boundaries were well-efined and all source code was available.
The vendor in the feasibility study whose technology was able to parse multiple languages and then reintegrate the metadata to create an abstract representation of the entire application demonstrated the capability to avoid the problems described with multi-language applications.
The recommendation to have an independent organization ‘validate’ the consistency of the AFP spec with manual counts will be considered after the spec has been implemented, entered use, and is producing the data needed for consistency checks.
Revised Text:
>none<
Disposition: Closed, no change
1. The OMG proposal suggests that consistency in counting, that is, a repeatable count given the same “source code,” is more useful that being consistent with the ISO standard. As such, any variation may make comparing and benchmarking existing data with automated counts less useful.
Recommendation 1: Advise (“warning label” sounds right but too strong) the measurement community that the OMG generated function points may not be useful for comparison with historic function point benchmarks.
Recommendation 2: Before this standard is fully accepted by the OMG and before it is submitted to ISO as a PAS submission, align it as much as possible to ISO/IEC 20926 IFPUG function points, or remove the reference to IFPUG function points from this OMG proposal.
Recommendation 1 is a recommendation for communication and not an FTF issue. CISQ is undertaking this effort in its outreach. Recommendation 2 is both accepted and rejected because during its development, the AFP spec was aligned as closely as possible with the current IFPUG counting guidelines. However, the ambiguities in the IFPUG spec were eliminated in order to support automation and this is the source of any differences. The IFPUG reference will not be removed.
Revised Text:
>none<
Disposition: Closed, no change