Stage 2: Project Attributes Sizing
| Introduction | Methodology Overview | Best Practices | Project Attributes | Market Capabilities | Market Readiness | |
Objective
Where Stage 1 focus is on non measurable information such as the project organisation, and process, the objective of stage two is to focus on quantitative dimensions of the project. We use quantitative data in combination with best practice results from Stage 1 to compute what we call Project Attributes.
Scope
There can be hundreds of quantitative data available for a software project. However in the OW2 MRL methodology information specifically collected in stage two is limited to data common to all projects under review. This results in 13 control points as providing information of users, bugs, licenses and general software quality.
Data Collection
Stage 2 data is automatically collected from software engineering tools such as the project's issue tracker and source code manager and the application of analysis tools such as SonarQube and ScanCode. The table below provides the details of the variables that are collected and their sources.
For bug tracker stats, a maximum of 200 bugs are analyzed (100 most recent open + 100 most recent closed). If less bugs are available, all are taken into account.
| Source | Variables | Definitions | |
| Bug tracker | ActiveUsers | Number of users with bug(s) assigned | |
| Bug tracker | AverageResponseTime | Average number of minutes between bug creation and update | |
| SonarQube | BlockerIssues | SonarQube "blocker_violations" metric | |
| SonarQube | CriticalIssues | SonarQube "critical_violations" metric | |
| Bug tracker | BugOpentime | Average number of days between bug creation and resolution | |
| Bug tracker | UnansweredBugs | Unanswered bugs are open bugs with no comment | |
| ScanCode | ScancodeRatioNoLic | Total unlicensed files / Total files count | |
| ScanCode | ScancodeTotalFiles | Total files count | |
| ScanCode | ScancodeTotalLicensed | Total files with license count | |
| ScanCode | ScancodeUniqueLic | Count of all distinct licenses found (same license with N versions counts for N, and 1 license max. is counted per file) | |
| SonarQube | QualitySonarQTestCoverage | SonarQube "coverage" metric | |
| SonarQube | QualitySonarQTestSuccess | SonarQube "test_success_density" metric |
Computation
First of all, all quantitative data is normalized so as to help compare projects and compute averages. The distribution of values along the standard OW2 MRL normalization for each variable is determined by the amplitude of the values as measured across all OW2 projects. In this sense, data normalisation is specific to the OW2 code base (i.e. it might not be relevant for the apache Foundation or the Eclipse Foundation code bases). Outliers, if any, are handled by hand on a case by case basis after consultation with project leaders.
Project attributes are complex characteristics resulting from the combination of results from the best practices verification (stage one data collection) and data automatically collected from the project's development environment (stage two data collection).
To compute attributes, we use an intermediary step we call aggregates. Aggregates are themselves the result of raws values. Aggregates help add meaning to raw values. Some raw values may contribute to more than one aggregate or more than one attribute. Some aggregates may be comprised on only one raw value when such raw value is sufficiently explicit. Attributes are defined by the data from which they result. Raw values are all normalised along the same value scale so as to enable their combinations.
The first table below brings out the structure of the aggregates and the second one shows how the attributes are computed.
| The Aggregates below are computed... | ...from the average of the following Best Practices... | ...and the following Project Values. | |
| Active Users (cty_activeusers) | ActiveUsers | ||
| Bug Handling (act_bugs) | Bug Open Time (days) Unanswered Bugs | ||
| Bugs Open Time (sus_bugopen) | Bug Open Time (days) | ||
| Community Practices (cty_omm) | Project Community Project Documentation Development Infrastructure Project Management | ||
| License Practices (lic_omm) | Project License | ||
| Liveliness (act_liveliness) | Project Communication Project Community | ActiveUsers | |
| Quality Practices (qua_omm) | Project Documentation Development Infrastructure Development Process Testing Process Release Management Security and Vulnerability Mgt | ||
| Ratio (lic_ratio) | Ratio Files_No_Lic / Number_Files | ||
| Reactivity (act-reactivity) | Bug Average Response Time (Cumulative) (s) | ||
| Serious Issues (qua_issues) | Number of Blocker Issues Number of Critical Issues | ||
| Sustainability Practices (sus_omm) | Project Communication Project Community Project Documentation Project Management Release Management Security and Vulnerability Mgt | ||
| Tests (qua_test) | Testing Process | Unit Test Coverage Percentage Unit Test Success Percentage | |
| Unanswered Bugs (qua_unbugs) | UnansweredBugs | ||
| Unique Licenses (lic_uniquelicense) | Number of Unique Licenses |
The attributes are computed by combining aggregates as explicited in the table below.
| The Attributes below are computed... | ...from the average of the following Aggregates | |
| Activity | Liveliness Bug Handling Reactivity | |
| Community | Community Practices ActiveUsers UnansweredBugs | |
| Compliance | License Practices Unique Licenses Ratio | |
| Quality | Quality Practices Tests UnansweredBugs Serious Issues | |
| Sustainability | Sustainability Practices Active Users Bugs Open Time |
Results
The values, between 0 and 5, of the five attributes are published in the shape of a radar graph. These graphs suggest project profiles in a simple and efficient way.
Stage 1: Best practices verification < Previous | Next > Stage 3: Market capabilities