Assessing OW2 Project Market Readiness Levels


OverviewBest Practices Project AttributesMarket Readiness 
      

Introduction

Open source software is a powerful enabler for collaborative innovation, however, making the source available does not automatically attract contributors or grant immediate market access. OW2 is developing a market readiness program to address what we call the delivery challenge of open source software development. Our aim is to facilitate the adoption of OW2 software by mainstream decision makers, i.e. those that are not necessarily open source software supporters.

Please check out the MRL introduction slide deck. The OW2 Market Readiness Levels (OW2 MRL) approach has also been presented at several events: OW2con'17, Cloud Expo Europe, FOSDEM, FOSS Backstage, OW2con'18, OW2con'19, SFScon 2020.

Through the OW2 OSCAR quality program we have been able to collect dozens of data points that we already report in the project dashboards. The information contained in the dashboards is dense and may be complex to read. With the OW2 Market Readiness Levels our objective is to enhance the meaningfulness of this information. We do this by integrating the different data into a composite index: the OW2 Market Readiness Levels (OW2 MRL). Obviously the OW2 Market Readiness Levels are inspired by the well-known Technology Readiness Levels developed and made popular by the NASA. 

Methodology

Our approach is both top-down and bottom-up. Top-down because we use our un-scientific business wisdom to pre-define the OW2 Market Readiness Levels and bottom-up because we use objective data collected from the projects to match them with the pre-defined levels. Given this approach we will gradually update the definition of the OW2 Market Readiness Levels, with the help of OW2 project leaders feedback and based on the data collection and analysis we will be able to produce.

Defining the OW2 Market Readiness Levels

The OW2 Market Readiness Levels (OW2 MRL) are a series of business-related situations that reflect the evolution of a project from initial software development to market leadership. The sequence must take into consideration the changes that must occur for a project to transform itself from a purely technology endeavour to a market powerhouse. We define the OW2 MRL by looking at the market hierarchy of open source projects - not a very scientific approach here, only our own modestly expert evaluation. And because we replicate the NASA TRL scale, the OW2 MRL scale also has nine levels emoticon_smile.

The exhibit below is the current status of the pre-defined OW2 MRL. Please note the examples are purely indicative and do not reflect any results from the data analysis part of the methodology, they are here only to indicate what kind of projects were used to help define the OW2 MRL.

OW2 MRLMRL RationaleMarket RationaleExamplesTechnology Rationale(NASA TRL)
9Market LeaderSignificant market share and global customer base. Properly financed and organized business support. Global active community. Platform, Sub-Project Generation“Flight Proven”
8Established OutsiderCustomer base of mainstream users. Appropriate financing. Active community support and contributions. Recognized softwareFusionDirectory, XWiki, KnowAge, OW2 Proactive,Building Range (Products/Features)“Flight Qualified”
7Established BusinessEstablished product. Customer base of early and mainstream users. Stable financing. Open to community support and contributions.DocDokuPLM, Imixs Workflow, LemonLDAP::NG, Rocket.ChatFull Q&A“Prototype in Space”
6Sizable Adoption or MarketProven product. Customer base of early users. Project fit for third party contributions. Implicit community governanceASM,Authzforce, Joram, Sat4J, Spoon, Sympa Growing Interoperability“Proven Demo”
5Fair Adoption or MarketSome customers, recent market opening, Core team of developers, untested open source governanceLuteceUser Interface Maturing“Relevant Envt Validation”
4Usefulness VerifiedSeveral users, project leadership well establishedCLIF, SeedStack, ReachOutMVP Defined“Lab Validation”
3Fledgeling UsefulnessOne declared user (can be internal) with declared project leaderPItestGolden Scenario“Proof of Concept”
2DevelopmentBasic R&D code developed with one demonstrated use case, some documentationCHOReVOLUTIONPOC“Application Formulation”
1ResearchBasic R&D code developedDSpotMethodology & Algorithms“Basic Principles”

Collecting and processing project data

This part will evolve from the OSCAR quality program previously implemented by OW2. Here we leverage OW2's participation in three EU-funded collaborative projects: RISCOSS, CROSSMINER and STAMP.

Project information is collected in three stages:

  • the first stage is a checklist of 50 best practices completed by project leaders,
  • the second one is a set of 15 data points automatically collected from the project's development environment,
  • the third stage is an expert assessment of the project market capabilities based on a structured expert interview with the project leader.

The list of best practices also serves as a blueprint to help project leaders implement better management of their projects. The list covers the following ten topic areas: Communication, Community, Documentation, Infrastructure, Management, Development, Testing, Release and Security.
You are welcome to check out this example of the MRL Best Practice form

Data of different nature are normalised so as to be compared. They are also combined into aggregates and models that help derive meaning from raw data.

Data automatically collected is updated daily and manual data when justified by a significant event and at least two or three times a year.

As a result we have a combined graphic representation of the different scores as illustrated below from our beta version: a bar chart for the Best Practices score, a radar for the Project Attributes and a cursor for the Market Readiness Scale.

Screenshot 2020-11-18 at 11.09.30.png

References

Countless different works have already been done on maturity and readiness models. As an introduction, here is a short list of useful links. Feel free to recommend more resources.

Request for Comments

At OW2 we are convinced this tool must be collaboratively designed, built and maintained by those who will use it, the open source market participants and, in the first place, the leaders and the users of OW2 projects. We are not entirely sure how to organize it but we know this must be a transparent and open process.

Please feel free to comment! We encourage everyone to use the comment feature on this page to provide us with feedback, suggestions, references, advice, etc.

Thank you for your contributions.

..