Evidence based policy making was promoted by the Blair Government as a supposedly more objective, rational way of making and implementing social, health and economic policies. The 1997 New Labour manifesto proclaimed that ‘what counts is what works’. However, what counted as the ‘evidence’ remained contested (Wells, 2007) and Blair himself was reportedly disappointed by the research offerings of academics (Glasby, 2011). Perhaps it isn’t coincidence then, that the non-academic evidence of ‘think tanks’ has become increasingly influential for policy making. The term ‘think tankery’ has even been invented to describe their combined research, policy and lobbying activities (Tchilingirian, 2014).

The think tank Demos was particularly influential in supplying ‘evidence’ for social care personalisation policy reforms in the form of a report presenting an account of self directed support and personal budgets as a ‘simple but transformational approach to public services’ (Leadbeater et al, 2008). This report appeared to be more influential on policy implementation than the large scale research study of individual budgets (as they were then constituted and called), which presented nuanced and complex findings on the intervention that were perhaps more challenging than the government policy makers were hoping (Glendinning et al, 2008).

So if an influential think tank report isn’t generally a piece of robust research with transparent methodology and reliable findings that would stand up to the peer-review journal process, what is it? This blog is a critical account of a new report by the Institute for Public Policy Research (IPPR) that discusses ‘e-marketplaces’ and ‘next-generation social care’ and offers a few thoughts about its quality as ‘evidence’.

E-marketplaces are defined as local authority ‘digital platforms’ that allow people to find out about and organise their services online, they may have purchasing functions or just allow access to an online service directory. As new research by Birmingham University shows, micro-enterprises can offer social care and support on a scale that’s ‘just right’, so effective ways of identifying and organising that type of support are required (Needham et al, 2015). We’ll be blogging on that research soon!

Method

Although the report doesn’t have a methodology section, there is a short paragraph in the introductory section that says a literature review on e-marketplaces was conducted along with inteviews with twelve experts and stakeholders (from local and central government, service and IT provider organisations and other ‘think tank’-type organisations) and an analysis of a group of e-marketing case studies. There is no further methodological detail, other than this brief descriptive paragraph.

Findings

This research does not look at the effectiveness of e-markets for enabling service users and carers to choose and control their services and support. Instead, the focus is on exploring and making the case for the use of local authority online, digital platforms for personalisation and improving individual commissioning in social care. Based on the investigation, the report outlines three main ‘opportunities to improve personalised care that e-markets present’:

• ‘improving access to the market for new and small providers
• enabling user commissioning
• integrating networks of informal and formal care’ (p.19)

In addition, the following recurring themes were identified, which are presented as conclusions:

• ‘digital services must be designed around the user’s experience and journey
• proactive offline activity is necessary for an e-market to succeed
• cultural changes, particularly around trusting users and adopting appropriate attitudes to risk, are prerequistes for success’ (p.29)

Local authority motivation for implementing e-market places were found to be cost savings; responding to new responsibilities to provide information and advice under the Care Act 2014; giving service users and carers greater choice and control; and responding to increasing digitisation.

Conclusion

The report concludes by making some recommendations for the effective development of e-markets for social care, based on their research:

• ‘Work across local authority boundaries
• Work in close partnership with the Government Digital Service (GDS) to embed local digital government
• Developing the workforce for next-generation social care
• Care coordinators, based in the community, to teach digital skills’ (p.32-33)

Strengths and limitations

It’s an interesting task to critically appraise a think tank report which presents itself as research, using an approach usually reserved for peer-reviewed journal articles. And perhaps it’s unfair! However, such reports are regularly used by policy makers as research ‘evidence’ for designing and implementing policies and interventions and so perhaps need to be scrutinised once in a while.

The report sets out the case for e-marketplaces as solutions for local authorities needing to find ways to address the challenges of personalisation, the increasing market of micro-providers, obligations under the Care Act 2014 and of course diminishing budgets. It does so by using a selection of the literature, interviews with a variety of stakeholders and though case studies. However, because there’s no detailed methodology it’s very difficult to discern how the literature was identified, selected and analysed; how the interviews were conducted and analysed or how the case studies were identified and selected. What is notable, is that a glance at the reference list of 29 articles shows that only one study published in a peer-reviewed journal was included, along with two academic research study reports. Five citations are for IPPR reports.

One shortcoming of the report is that it doesn’t fully address the case against e-marketplaces in the course of the argument. E-marketplaces are presented as the sensible thing to do, without any reference to reliable research studies on their effectiveness or acceptability. Research that was cited in the report and blogged on here suggested that information from friends or family is more likely to be trusted than many professional sources of information. How can online e-markets address this?

More concerning perhaps is the lack of engagement with people who will ultimately be expected to use e-marketplaces to identify and organise care and support – service users and carers. Nor were social workers and care coordinators interviewed about their perspective, yet the report recommends that they should have a role in implementing e-marketplaces for social care.

Summing up

In a blog a little while ago, Mike Clark looked at a study on how policy makers use research evidence. The research suggested that policy makers lack time to read relevant studies and that universities were not often consulted when evidence was sought.

This raises questions about what is being used as ‘evidence’ in policy making, and reports like that produced by IPPR seem to be accepted as policy evidence. However, if it is still true that ‘what counts is what works’, think tanks need to engage more fully with research studies on effectiveness and outcomes, as well as with all stakeholders, to make sufficiently robust cases for social care change.

Link

Roberts, C. (2015) Next-generation social care: The role of e-marketplaces in empowering care users and transforming services London: IPPR [Full Text]

References

Glasby, J. (ed.) (2011) Evidence, policy and practice: Critical perspective in health and social care Bristol: Policy Press

Glendinning, C., Challis, D., Fernandez, J., Jacobs, S., Jones, K., Knapp, M., Manthorpe, J., Moran, N., Netten, A., Stevens, M. and Wilberforce, M. (2008) Evaluation of the Individual Budgets Pilot Programme: Final Report, Social Policy Research Unit, University of York, York. [Full Text]

Leadbeater, C., Bartlett, J. & Gallagher, N. (2008) Making it personal London: Demos [Full Text]

Needham, C. et al (2015) Micro-enterprises: Small enough to care? Birmingham: University of Birmingham/ESRC [Full Text]

Tchilingirian, J. (2014) Think-tankery, Policy Research and Contemporary Intellectuals: A Personal Network Perspective British Sociological Association (BSA). Annual Conference 2014: ‘Changing Society’.

Wells, P. (2007) New Labour and evidence based policy making: 1997-2007 People, Place & Policy Online 1 (1) pp. 22-29. [Full Text]