Data Disciple
20 July 2015
Monday 20 July 2015
This month, our Chief Executive Rosie Ferguson was interviewed by CYP Now on the importance of impact measurement in the youth sector. In case you missed it, here is the full intervew:
Rosie Ferguson describes herself as naturally "instinctive and passionate", but it is her newfound enthusiasm for data collection and analysis that consumes much of the discussion when we meet.
Since being appointed chief executive of London Youth three years ago, Ferguson has led the youth work network to develop more of a focus on collecting information about the beneficial outcomes for vulnerable children and young people that its 400 member organisations across the capital deliver.
"My main task as chief executive has been to develop a robust framework around measuring what we do," says Ferguson, before going on to explain how this was produced.
"Measurement isn't the first thing: that is understanding the outcomes you're trying to achieve and that all the people delivering them can clearly articulate what they can achieve for young people.
"We worked with all our members and delivery staff to find out what they thought was the difference they made to young people and then we pulled these together. Three things were consistent across everything: confidence, resilience and relationship skills."
London Youth's framework aimed to capture the impact that organisations and interventions have on these three outcomes by using a measurement tool, the Life Effectiveness Questionnaire, to collect information from young people.
"We now have data on how we're doing on confidence, resilience and relationship skills," says Ferguson. This was seen in London Youth's recent Good Youth Work Works? report, which showed that 70 per cent of the young people that had taken part in a sports programme reported an average 18 per cent increase in their social and emotional capabilities, with two-thirds saying their determination and resilience rose 19 per cent on average.
Ferguson's "total conversion" to using data since taking the helm at London Youth reflects a growing recognition sector wide of the need to show politicians and funders the beneficial impact of youth work. Some are wary that collecting outcome data is just a mechanism to justify investment and funding bids. But Ferguson is quick to point out that the exercise is not to "prove stuff" but "to learn and do the best we can for young people".
She adds: "The more data we share with our staff and members about the impact they're having, once they get that and understand it, the hungrier they are for more.
"We've found our staff are more engaged, our senior team decisions are more informed and our strategy is more coherent.
"It's not my strength (data analysis), but it is a skill set that the sector will now need. That doesn't have to be at senior level – but it has to have buy-in at senior level. It's something that could be shared and used across the sector."
Limited resources is the key challenge, according to Ferguson, for developing greater recognition and adoption of outcomes monitoring and data collection across the youth sector is to ensure it is accessible for smaller community and youth groups. These organisations, she says, are doing some of the most "impactful work on very limited resources", but will only be able to gather the evidence of that if outcomes monitoring is done in a way that is proportional.
"It's about the approach: if it's all based around how do we do better for young people and not on how do we tick the boxes for funders it can work.
"What we don't want is for all the bigger organisations with the most rigorous evaluation teams to produce the best evidence. That doesn't mean they are doing the most effective work.
"For us the focus for the next three years is how we provide some resource to our members to be able to articulate and evaluate their own outcomes."
Some also see data collection as a means to hold leaders to account, making a direct link between the outcomes achieved by an organisation and the performance of decision makers. Ferguson views it differently, and argues that data collection offers the chance for organisations to refine and become more intelligent in what they do and take a longer view in measuring success.
She says: "We are often incentivised in the voluntary sector to over-claim on impact, which often means our institutions compete – we put the institutions first and the mission second. "
As chief executives, we are often judged on things not necessarily conducive to social impact, such as growth and brand profile, and I would rather be judged on honesty, collaboration, improvement and fun.
"I do want to be held to account on the finances but on the long-term view of the finances and not very short term performance measures, which can sometimes incentivise taking money that doesn't fit with the values, mission or outcomes we're trying to achieve."
Ferguson's view of what effective leadership looks like has gradually evolved as she has felt her way into the job, her first as a chief executive. She admits there was an initial period of conforming to what is expected of a chief executive – "I thought 'oh, now I'm the boss I need to wear an M&S suit'" – before realising that "being yourself is one of the strongest ways to help other people to lead".
She has recently returned from a week-long study trip to India for social impact leaders that combined visiting some of that country's new social enterprises with coaching on leadership skills.
"It was a very inspiring, immersive experience," says Ferguson. "It gave space to reflect on personal leadership style and values.
"Having been at London Youth for 10 years, what I've learned is the values that I believe in are my personal and the organisation's values. "The Indian experience was part of a journey of deepening my understanding around that."
Ferguson's values of collaboration and commitment to improvement are very much at the heart of the Centre for Youth Impact, which was launched last September as a place where practitioners, funders and policymakers could meet to discuss and share evidence on the impact that good youth work has on the lives of young people. The centre, funded by the Cabinet Office until the end of June, worked with London Youth and two other early adopters to build their capacity, skills and infrastructure around evidence. Although funding for the centre to continue is still to be secured, Ferguson is a fan of its work and hopes investment can be found.
"It is about facilitating the development of organisations to be able to do that stuff themselves and providing an avenue for them to be able to share learning as much as it is a hub of data," she says. Ferguson believes the centre is a key component in embedding the cultural shift needed for youth work to really grasp the opportunities around using data to improve what it does.
"The private sector is used to having metrics and designing programmes based on data – whether about their customers or products – and we're not really that used to that as a sector," she says.
"We're now starting to make decisions and allocate resources based on what outcomes have been delivered. That is quite a cultural shift. One of the things we're not used to doing is innovating based on that data – how do we as a sector become much better at processing, analysing, responding to the data that we have on the programmes we're delivering?"
While the cultural shift may have started, it is still in its early days. This is reflected by the lack of recognition given to evidencing impact by youth work funders, explains Ferguson. "I'm yet to be convinced money is actually following this stuff," she says. "People are talking about it and want to see you have impact data, but actually 99 per cent never come back (to ask about it).
"Funding still follows charisma and the skills of a person filling in a form."
So if organisations may not see a return on the time invested in gathering and analysing outcomes data, is there a strong incentive to actually do it? Ferguson thinks rather than financial imperative, the agenda must be driven by improving the support offered to young people to tackle the challenges they face.
She says: "You see a lot of people doing things because of a personal interest, not because there is necessarily a need. Gut instincts can be right and wrong. In a world of decreasing resources we have a responsibility to young people to make sure the resources spent on them are done so in the best way."
Interview by Derren Hayes
