What is the ROI of my Community?

Louis-Pierre Guillaume, 15 mars 2019

1. Why is ROI so difficult to measure?

Sarah leads the Quality Community, with about a hundred members scattered over different sites. The community is bustling with informal exchanges on the company’s Enterprise Social Network (ESN) and its webinars regularly attract around thirty participants. The Community of Practice (CoP) was created two years ago and its sponsor is the Director of Quality. Sarah spends about 20% of her time leading her community, a mix of organizing webinars and workshops, monitoring ENS exchanges and communication.

During her quarterly evaluation, her manager pointed out that 20% is a little high and asked her to demonstrate the value of the CoP for the business, to justify her time spent. Sarah then provides him with statistics on community activity on the ENS (number of messages, comments, likes, participants, readers, etc.) and figures on webinars (number of webinars, participants, etc.). The figures show an increase from the beginning, with a decline in the last 6 months, as the CoP has reached its cruising speed. Her manager is not convinced by the value, neither is the sponsor. They believe that since the community works, it has less need for Sarah. Six months later, members are less and less active and the statistics are getting worse.

Alice leads her company’s knowledge management program. In particular, she supervises the leaders of the twenty communities of practice and organizes quarterly meetings to make them share their experiences. She compiles and publishes statistics for all the communities. Challenged on his budget, her manager asked her to demonstrate the value of the communities of practice program. Alice then sends a survey to the community leaders, with some questions about the value of their community. Her manager is not convinced by the survey, because he claims the answers are biased (the community leaders are judges and parties) and there is no tangible evidence. Alice feels that her position is weakening.

2. What lessons can be learned from these two stories

• The statistics of a tool do not predict the value of the communities that use it.
• A director becomes a sponsor of a community of practice (CoP) if he or she perceives a business interest.
• The leader of a community of practice must ensure that the community’s objectives and expected benefits are achieved.
• A manager will give time to lead a CoP to someone on his team only if he or she perceives the value of the community.
• Measuring the value of a community program requires interviewing community members.

3. How to measure value? Three levels of measurement

3.1. Adoption & participation

All ESN platforms provide usage statistics. For example, the number of documents available, read, liked and shared; the number of messages, responses, like, participants and focus group visits.

This measure is not an indicator of value. In some communities, members simply post articles, as they would on their LinkedIn profile, without any added value or reflection on the content.

However, these statistics are useful during the launch phase to confirm the growth.

To summarize, a discussion group in a enterprise social network with a flat encephalogram is an alarm signal, but regular activity does not provide sufficient information on value. Another measure is necessary.

3.2. Engagement & satisfaction

Community members are the most likely to know the tangible value their community brings to their business, their customers or themselves. A survey allows to know the opinion of the members, the voice of the customer (VoC) in a way.

In a company with several communities, such as Alice’s, it is useful to be able to compare the perceived value from one year to the next and it is essential to avoid saturating members with a different survey per community. Such a survey must provide anonymized results to each community facilitator and aggregate results for the rest of the company, while ensuring a minimum response rate to make the results credible.

Schneider Electric has been measuring the value of its communities of practice through surveys since 2013, a measure I designed. The main question is “I consider that my community brings tangible value to my business, my customers or me”. The survey is sent to 12,000 people, with a participation rate of around 25%. From the answers, a tangible value score is calculated, per community, as well as an overall score. The community with enough voters and a score above a predefined threshold receives the label “Active Community”. Other questions allow anyone to specify the type of benefit, suggest improvements, and especially to propose a personal experience or an example.

The results make it possible to measure the tangible value of each community, as perceived by members. But this is not enough, hence the need to collect experiences and examples to feed the third level of measurement, efficiency.

3.3. Efficiency

The effective way for a community leader to convince management that his/her community brings tangible value is to provide a success story with numbers and testimonials. The success story should illustrate how the community has brought tangible value (e. g. reduce costs, save time, or bring more business), and how this value could not have existed without the community.

The problem is often to identify the first steps that will lead to a success story. For example, it could be a request for help for a customer problem on the quality community discussion forum, which ends, following exchanges between experts located in different remote sites, with a solution that will reduce a maintenance cost. The community leader will then interview the protagonists and write the story, including the avoided costs and verbatims. The survey described above makes it easier to identify these potential stories because members can directly suggest them.

4. The benefits of this measurement of value

A manager will only give Community of Practice (CoP) animation time to a member of his team (20% is the recommended minimum for a community leader) if the perceived value of the community is at least qualitative. A quantified success story convinces the most sceptical managers. The sponsor can help to convince the manager.

A secondary advantage of survey value measurement is that it asks members for their opinions and wishes about their community. The community leader takes this into account and proposes an action plan.

Measuring the overall tangible value and success stories of all communities allows the knowledge management program manager to demonstrate that the community program is working. This ensures the sustainability of the knowledge management program.

5. For more information on this topic

APQC Next-generation Communities of Practice: Schneider Electric SE Case Study