Assessing OW2 Project Market Readiness Levels


Introduction

Open source software is a powerful enabler for collaborative innovation, however, making the source available does not automatically attract contributors or grant immediate market access. OW2 is developing a market readiness program to address what we call the delivery challenge of open source software development. Our aim is to facilitate the adoption of OW2 software by mainstream decision makers, i.e. those that are not necessarily open source software supporters.

Please check out the MRL introduction slide deck. The OW2 Market Readiness Levels (OW2 MRL) approach has also been presented at several events: OW2con'17, Cloud Expo Europe, FOSDEM, FOSS Backstage, OW2con'18, OW2con'19, SFScon 2020.

Through the OW2 OSCAR quality program we have been able to collect dozens of data points that we already report in the project dashboards. The information contained in the dashboards is dense and may be complex to read. With the OW2 Market Readiness Levels our objective is to enhance the meaningfulness of this information. We do this by integrating the different data into a composite index: the OW2 Market Readiness Levels (OW2 MRL). Obviously the OW2 Market Readiness Levels are inspired by the well-known Technology Readiness Levels developed and made popular by the NASA. 

Methodology

Our approach is both top-down and bottom-up. Top-down because we use our un-scientific business wisdom to pre-define the OW2 Market Readiness Levels and bottom-up because we use objective data collected from the projects to match them with the pre-defined levels. Given this approach we will gradually update the definition of the OW2 Market Readiness Levels, with the help of OW2 project leaders feedback and based on the data collection and analysis we will be able to produce.

Defining the OW2 Market Readiness Levels

The OW2 Market Readiness Levels (OW2 MRL) are a series of business-related situations that reflect the evolution of a project from initial software development to market leadership. The sequence must take into consideration the changes that must occur for a project to transform itself from a purely technology endeavour to a market powerhouse. We define the OW2 MRL by looking at the market hierarchy of open source projects - not a very scientific approach here, only our own modestly expert evaluation. And because we replicate the NASA TRL scale, the OW2 MRL scale also has nine levels emoticon_smile.

The exhibit below is the current status of the pre-defined OW2 MRL. Please note the examples are purely indicative and do not reflect any results from the data analysis part of the methodology, they are here only to indicate what kind of projects were used to help define the OW2 MRL.

OW2 MRLBusiness ScaleTechnology ScaleRationaleExamples(NASA TRL)
9Established PlayerPlatform, Sub-Project GenerationEstablished product with properly financed and organized Business support  Customer base, Repeat sales and Market recognitionKnowAge,
XWiki
“Flight Proven”
8Actively CompetitiveBuilding Range (Products/Features)Early customer base, appropriate financing or active community support, recognized softwareOW2 Proactive“Flight Qualified”
7Business Build-upFull Q&AEarly customer base and fledgeling financing or active community support, Start-upRocket.Chat“Prototype in Space”
6Market BroadeningGrowing InteroperabilityEarly customer base and fledgeling financing or active community support, Start-upLutece“Proven Demo”
5Market OpeningUser Interface MaturingSome customers, recent market opening, un-proven governanceReachOut“Relevant Envt Validation”
4Usefulness DemonstratedMVP DefinedSeveral users, project leadership well establishedCLIF, SeedStack“Lab Validation”
3Fledgeling UsefulnessGolden ScenarioOne declared user (can be internal) with declared project leaderPItest“Proof of Concept”
2DevelopmentPOCBasic R&D code developed with one demonstrated use case, some documentationCHOReVOLUTION“Application Formulation”
1ResearchMethodology & AlgorithmsBasic R&D code developedFASTEN“Basic Principles”

Collecting and processing project data

This part will evolve from the OSCAR quality program previously implemented by OW2. Here we leverage OW2's participation in three EU-funded collaborative projects: RISCOSS, CROSSMINER and STAMP.

Project information is collected in three stages:

  • the first stage is checklist of 50 best practices completed by project leaders,
  • the second one is a set of 15 data points automatically collected from the project's development environment,
  • the third stage is an expert assessment of the project market capabilities based on a structured expert interview with the project leader.

The list of best practices also serves as a blueprint to help project leaders implement better management of their projects. The list covers the following ten topic ares: Communication, Community, Documentation, Infrastructure, Management, Development, Testing, Release and Security.
You are welcome to check out this example of the MRL Best Practice form

Data of different nature are normalisazed so as to be compared. They are also combined into aggregates and models that help derive meaning from raw data.

Data automatically collected is updated daily and manual data when justified by a significant event and at least two or three times a year.

As a result we have a combined graphic representation of the different scores as illustrated below from our beta version: a bar chart for the Best Practices score, a radar for the Project Attributes and a cursor for the Market Readiness Scale.

Screenshot 2020-11-18 at 11.09.30.png

References

Countless different works have already been done on maturity and readiness models. As an introduction, here is a short list of useful links. Feel free to recommend more resources.

Request for Comments

At OW2 we are convinced this tool must be collaboratively designed, built and maintained by those who will use it, the open source market participants and, in the first place, the leaders and the users of OW2 projects. We are not entirely sure how to organize it but we know this must be a transparent and open process.

Please feel free to comment! We encourage everyone to use the comment feature on this page to provide us with feedback, suggestions, references, advice, etc.

Thank you for your contributions.