Measuring the impact of the book-gifting programme Bookbug

Ahead of the 2017 Bookbug conference research fellow Emma Davidson shares a few findings from CRFR's Bookbug research.

---

Bookbug is the Scottish Book Trust’s Early Years programme, encouraging parents and children to share stories, songs and rhymes from birth. The Scottish Book Trust’s Early Years programme has gifted free bags of books and resources to children in Scotland for many years, and there are Bookbug Sessions taking place every day in Scotland’s libraries, nurseries, and early years centres.

Over the last two years CRFR has been investigating the impact of the Bookbug programme on parents, carers, children and early years professionals. Amongst other questions, we have been seeking to understand the impact Bookbug has had on attitudes to reading, and whether there is a link between the way Bookbug Bags are gifted and the impact it has on families.

One of the key principles of Bookbug is that is a universal programme. This means that Bookbug Bags are available for all families in Scotland, and that Bookbug Sessions are – without exception - free to attend.

The evidence from our study shows the importance of universal delivery. Bookbug, we found, can have a positive impact on all families: not just those who might be judged ‘in need’ should the programme be means tested. For many of the parents we spoke to – including those who already described themselves as placing a high value on reading – Bookbug played a key part in reminding them of the importance of reading and singing to their children, and prompting them to dedicate time to do this:

“I definitely do think that it’s made a difference, it’s made me make sure I set, you know, 10-15 minutes aside every day, to do that [reading] with him.” 

Moreover, parents described Bookbug as encouraging them to do this at an earlier age than they initially intended:

“I would have thought “oh, she’s too young”, so it prompted me to think “oh well if they’re giving it to me, she can’t be too young…These things are good to remind you to do it…like in theory I should know what I’m doing because this is number 2, but getting books does make you think ‘oh yeah, I should actually be reading to her and not just to her sister.” 

What is clear from the data is that we cannot make assumptions about who might benefit from Bookbug based on the number of children they have, their income, gender, ethnicity and so on. Regardless of these factors families spoke about the positive impact that book-sharing has had on bonding and attachment between themselves and their children and their child’s enthusiasm for books. At the same time, Bookbug Sessions enabled many families to form new social and support networks.

We look forward to sharing more findings from our Bookbug research with you over the forthcoming months - keep an eye on the Bookbug webpages for more information, or join the CRFR research network mailing list for updates about this and other ongoing projects.

Learning about evidence and policy at the Scottish Government

CRFR PhD student Rebecca Black reports back from her internship at the Scottish Government 

This summer I had the fantastic opportunity to work as a PhD intern at the Scottish Government. I worked with the Children and Families Analytical Unit, there for three months - a departure from my normal role as a CRFR associate PhD student.

The internship, facilitated by the Scottish Graduate School of Social Science, was designed to provide the opportunity to work outside of my PhD topic while developing valuable experience and skills. Throughout the internship I took on a variety of projects across a range of topics: From early learning and childcare to children’s rights to Safeguarders’ in the Children’s Hearing System.

I was quickly thrown into the action with a consultation analysis. The Scottish Government recently produced guidance on Children's Rights and Children's Services Planning sections of the Children and Young People (Scotland) Act 2014 (Part 1, section 2 and Part 3 respectively). This guidance was put out for consultation and I found analysing the responses received really interesting. I had to quickly become familiar with the core components of the guidance and the Act and enjoyed distilling the responses into feedback for the policy team[1]. I also learnt a lot during this process, which illustrated the importance of the advocating and lobbying role that institutions such as CRFR play in ensuring that all voices are heard in the government’s work.

I also had the opportunity to see how policy and analyst teams work together to ensure evidence-based policy is crafted and implemented. The increased provision of early learning and childcare was one such policy initiative that I was involved in, producing two preliminary reviews of evidence for the policy team. This involved consolidating vast amounts of research to create a report that succinctly summarised the current state of evidence. These reviews were then incorporated in to briefing packs that were sent to the Ministers and policy teams to aid their decision-making.

One of the key highlights of my time however was working on the development and tendering of a piece of research. It was a novel experience and provided insight in to how research bids are put together and the selection process. It was great to see how analysts and policy-makers collaborate to ensure quality research is conducted and valuable outputs are produced. Having spent quite a while in the academic environment, being part of this process gave me real insight into a different facet of research and the practical processes surrounding this.

The internship, while short in duration, provided me with a wealth of experiences and developed a range of transferable skills (e.g. writing succinctly, writing in lay terms and working with different expectations). The experience has also offered me plenty of ideas on how I can improve my own PhD study’s relevance to policy, hopefully widening its future dissemination and expanding engagement with policy makers.


[1] The analytical report has now been published and available to view online at: http://www.gov.scot/Publications/2016/09/3725/downloads.

It’s complicated…

CRFR Co-Director, Dr Pam Warner offers a review of a recently published paper to ask what is the role of interpretive quantitative methods in the social sciences?

A recently published paper has made a plea for:

“more meaningful, more understandable, more applicable practice of quantitative methods in social science”. (Babones 2015[1])

As you read this, you are probably thinking “Hear, hear!” Certainly this paper caught my attention because it articulates a number of points that have been bubbling away in the back of my mind…

These days many published research articles report on social research studies that have applied statistical analysis methods to multivariable data sets. For many active researchers the meaning and implications of the reported findings can be difficult to fathom, even more so probably for practitioners or policy-makers, especially if a very specialised technique has been used.

Undoubtedly it would be helpful to practitioners and policy makers if research gave clarity as to cause and effect – because if factor F causes some happy outcome H, then finding some way to introduce/ intensify F would produce/ enhance H… (Or conversely, for adverse outcomes.) However, where findings of multivariate observational social research studies are reported in a ‘causal’ way, unjustifiably, there is real risk that the research ‘consumer’ will be misled.

First thing to acknowledge is that life is complicated, of course - so it would be optimistic to expect a simple account of interrelationships among the available variables, particularly in family and early years studies. Indeed, can we even expect to discover causes from an observational research study? By way of introduction, the Babones paper reminds us that there are two main types of social science research data – qualitative (eg interview transcripts) or quantitative (measurements or survey question responses) - and also two main research strategies - interpretive and positivist:

• Positivist research should be based on strong prior theory, with the data being used to challenge (test) that theory. This means that positivist research can often make causal claims, if the data confirm the theory.
• In contrast, interpretive research seeks to draw out theory (or meaning) from the data. It thus follows that interpretive approaches need minimal amounts of prior theorising, but cannot make strong claims about causality.

A general if simplistic presumption prevails, that quantitative research is ‘positivist’, while qualitative research is ‘interpretive’. However Babones points out that this is not entirely the case, and argues that much quantitative research reported in the literature is not truly positivist. This is because the prior theorisation is inadequate, or insufficiently explicit, or the model used in the analysis was tweaked a number of times en route, so that the findings are actually more ‘emergent’ than giving support to prior theory. This is a crucial point, and undermines confidence in the published causal claims. It also partly explains why it so often happens that different studies on the same topic give conflicting understandings of associations between an outcome of interest, and the many potential explanatory variables.

Regarding the role that statistical techniques can play, Babones is very positive, stating that they can lead to findings that are “meaningful, understandable and applicable” and that they can offer “the incredible scope and power that can only be achieved through the application of multivariate statistical models to data derived from sample surveys”. He believes that most often the problem is that quantitative sociology “unnecessarily retains the philosophical, rhetorical and methodological baggage” of a positivist approach to research. As a remedy he urges that it would be possible, and useful, to develop an interpretive approach to analysis of multi-variable quantitative data sets, and he characterises this by deploying an evocative analogy –

“…relationships among these observed variables are the measurement tip of the causal iceberg. The task of interpretive social science is to surmise what lies unobserved beneath”.

Babones concludes that more realistic policy implications could be formulated from research providing understandings of causality derived from interpretive quantitative sociology, and involving triangulation of findings from different sources and approaches, and thorough reflexivity. Let’s do it!

If this blog echoes your experiences, or suggests counter-arguments, I would be interested to hear from you.Dr Pam Warner. 

 

 

[1] http://soc.sagepub.com/content/early/2015/05/20/0038038515583637.full ‘online first’ 

 

 

Developing an evidence service for the children and families workforce

Karen Seditas shares learning from the Evidence Request Bank - a one year pilot development project which aimed to open up the evidence base for delivering public and third sector services to children and families in Scotland.

The Evidence Request Bank (ERB) aimed to help services to access and use social research evidence by supporting them to identify specific gaps in knowledge and consider how evidence could help address them. The ERB produced reviews of relevant evidence, and supported services to plan how to use evidence in practice. An evidence-to-action cycle was created to support project partners we worked with.

The project was led by the Centre for Research on Families and Relationships (CRFR) in collaboration with Parenting Across Scotland, West Lothian Council, Children in Scotland and the Scottish Government, and based on previous development work by CRFR.

How project partners are using the ERB evidence reviews:

•  West Lothian Council: used an evidence review on transitions to primary school to create a driver diagram to inform tests of change as part of its Early Years Collaborative work. The review enabled them to direct resources to areas that are likely to be effective, and help practitioners be confident that they are doing the right things in a complex area.
• Parenting Across Scotland: held a seminar to discuss a review of data on family households, used the data to inform its online information resources and policy work, and will discuss specific areas, gaps in data, and implications for policy and practice further.
• Parenting Across Scotland and the Scottish Government are developing accessible materials (e.g. infographics) from the report on family households for further sharing.
• Children in Scotland: is planning forum events to discuss implications of the ERB evidence reports relating to children and families, and sharing evidence through its training programme, magazine and website.

What we learned about enabling the children and families workforce to use evidence includes:

• Even when people want to use evidence to inform their work, they find it difficult to be clear about what they want to know and to make decisions about what should be prioritised.
• Approaches to using evidence may differ depending on the context and purpose of why it is needed, and on the previous experience of staff in using evidence in their work.
• Involving whole teams in discussions on how to use evidence helped to explore different understanding and expectations of using evidence and demonstrated how evidence can inform service delivery. It also helped to direct resources where they were most likely to be effective.
• A range of issues may affect how organisations react to evidence. Even where organisations want to use evidence, services may not have the resources to change in response.

You can read more here: http://www.crfr.ac.uk/assets/briefing-73web.pdf

Read the evidence reviews here: http://evidencerequestbank.org/evidence-responses/

What next for the ERB?
In the immediate future, the Evidence Request Bank will be integrated with What Works Scotland (WWS) (www.whatworksscotland.ac.uk), a new initiative to improve the way local areas in Scotland use evidence to make decisions about public service development and reform.

Keep up to date with Evidence Bank developments and What Works Scotland by joining the mailing list here: http://whatworksscotland.ac.uk/contact-us/.

The ERB development project was funded by the Economic and Social Research Council (ESRC), with additional funding and in-kind contributions from the partners.
 
What Works Scotland is a collaboration between the Universities of Edinburgh and Glasgow and funded by ESRC and the Scottish Government for 3 years until 2017.

 

  

 

  

 

 

Understanding K* and what it means for knowledge exchange in Scotland

The CRFR KE reading group focus in August 2014 was to explore ideas of knowledge exchange through reading Expanding our understanding of K* - a concept paper emerging from the K* conference held in Hamilton, Ontario, Canada, April 2012, that CRFR Co-Director Sarah Morton had attended. We wanted to explore how K* concepts relate to our roles as knowledge brokers here in Scotland.

What is K*? There are many terms used to describe the processes of getting research and evidence into policy, practice, there public or other action. These include knowledge management, knowledge mobilisation, knowledge exchange, knowledge transfer, knowledge to action and others. The idea of K* or KSTAR is that it embraces all of these terms through the use of the wildcard star or *

Understanding the K* spectrum
 The K* discussion document sets out what might be involved in different roles to increase these processes. It considers knowledge brokerage from informational, through relational to system functions. At the information end tasks might include improving access to research and knowledge, whilst at the system end roles would be more embedded in change in complex systems.

While our members are involved in slightly different aspects of knowledge exchange work - from strategic planning, knowledge brokering, dissemination to supporting evidence to action, we found the discussion of what K* is to be a helpful peg on which to hang our, sometimes many, hats. The concept of K*, as described in the paper, is the collective term to describe what needs to happen to share knowledge and to enable change to happen. From our round-the-table discussion, it seems that good knowledge exchange requires involvement in the full range of activities, a sentiment supported by the authors. Trying to rank the importance of activities, or to see activities in a linear way, really misses the point. At CRFR, we are involved on a daily basis in dissemination activities: writing research briefings, posting on social media (@CRFRTweets), creating visualisations of research findings. From this we create opportunities to bring people together to share and talk about research and what the findings mean to different sectors and different organisations. Within specific projects we are then able to build partnerships to raise questions about what research topics are most needed, who can contribute to the research and how the resulting evidence can be used. Any one one of these functions can not happen on its own - we invest time at all levels.

The relationship between supply and demand for knowledge
The section exploring the relationship between the supply and demand for knowledge had the greatest resonance with our group members. They appreciated an opportunity to reflect on the nature of the many tasks they do, and sometimes the complexity of what is asked of them. We within social science research, we can be faced with gaps in the evidence - this may be because the nature of qualitative data makes it hard to generalise across wider populations, or because the specific question being asked has not been the focus of existing research or service evaluations. As the paper suggests, the boundaries between what knowledge is wanted and what knowledge we have is fuzzy. We discussed the ways in which we tailored our approaches, to either link people with similar interests but potentially different backgrounds, to support organisations to apply evidence to their own work or to find creative ways of sharing complex ideas.

The difference between knowledge exchange and communications
It seems there have been various conversations between our own colleagues, and within the global K* group, of the difference between knowledge exchange and communications. Within our setting we agreed that the roles are different, with reasonably different specialist skill sets, yet there is a large degree of overlap between the two. My own title includes both - as a way, when it was created, to recognise the increasing value of knowledge exchange within academia but also acknowledging that it was a term still relatively unknown by the wider public, where this type of function may have typically been carried out by a communications or policy professional. What did our discussion add to the debate? Perhaps just to record that within our University setting, the breadth of knowledge exchange activities, particularly the brokering, impact analysis and research synthesis seems to extend beyond typical a communications role. Similarly, our communications colleagues would have greater specialist skills in journalism, marketing and public relations.

Creating knowledge exchange roles within a University setting
Another key point made in the paper is that K* needs to focus on the function, rather than the organisation or individual person who performs them. Our members are all employed by the University of Edinburgh, yet each of us have a different job title and different remit - and we know from our membership of other networks and research by Knight and Lightowler, that the variation in knowledge exchange roles is becoming ever wider. Yet, there are overlaps in many of the tasks and activities we do. We wondered of the feasibility for an organisation such as the University to take a proactive role in creating standardised knowledge exchange roles and descriptions as a way of professionalising what we do, and capturing the breadth of activities involved within knowledge exchange. This is something we as a group will be exploring over the coming year.

Kirsten Thomlinson
Knowledge Exchange and Communications Co-ordinator
CRFR


Refs:
Knight, C., & Lightowler, C. (2010). ‘Reflections of “Knowledge Exchange Professionals” in the Social Sciences: Emerging Opportunities and Challenges for University-Based Knowledge Brokers.’. Evidence & Policy, 6(4), 543-556. 10.1332/174426410X535891

Journal Club: Is that clear?

Relevance, access, collaboration, timeliness – magic words in the world of knowledge exchange. They were also key factors, helpfully identified by Oliver et al in their systematic review of studies of barriers and facilitators in using evidence. Taking the lead from Research Impact’s Journal Club, the CRFR Knowledge Exchange reading group ‘Practice what you preach’ met to discuss the review and reflect on our own experience of working in the field of evidence-use. We have captured our thoughts here, with links to some examples.

 

Oliver, K., Innvar, S., Lorenc, T., Woodman, J and Thomas, T. (2014). A systematic review of barriers to and facilitators of the use of evidence by policymakers, BMC Health Services Research, 14:2

Stop talking about dissemination

• Is it time to move beyond the concept of dissemination[i]? Dissemination was a key term in many of the papers in this systematic review. Research needs to get out there, but this one-way concept isn’t going to achieve results in getting policy makers engaged with research. We, along with the rest of the academic community, talk more about engagement, impact, and collaboration. It is important to plan, from the outset, how different audiences will be able to access and use research.
• It is a shame that the report authors grouped ‘access to research’ with ‘dissemination’ in their factor analysis. They are two very distinct issues – does it mean access to primary research, and/or research being made accessible through translation? A breakdown of these terms would have provided a clearer understanding of the barriers and facilitators for policy-makers.

How clear is clear?

• It would have been helpful if the authors had unpacked what was meant by 'clarity of research findings' within the studies they analysed. Were they referring to research which was written clearly and easy to understand, or were they referring to research which had clear recommendations and proposals? Research rarely provides clear recommendations of potential actions even though that is often what policy-makers and practitioners want.
• Knowledge exchange and mobilisation work means working with the available evidence to help shape what might happen next. We explored this in our CRFR briefing ‘Claiming an evidence base for services and interventions’. We are constantly working in this terrain as knowledge brokers for example:
• Growing Up in Scotland (GUS) – practitioners often ask for more information about how to use study research findings in practice. 
• About Families project and Evidence Request Bank development project– we often explain that research doesn’t provide direct answers, it poses questions and helps service professionals reflect on practice and share understandings with colleagues
• So we need to separate out ideas of knowledge exchange and implementation, as Melanie Barwick, who is running a KE training course here in Edinburgh in May says:

  “Knowledge translation and implementation are complimentary but different constructs. Knowledge translation involves helping others to understand the evidence; implementation involves supporting them to make the changes needed to apply the evidence. Impact means capturing that people knew what to do with the knowledge you shared.”

What are people doing?

• We noted that most studies included in the review were based on researcher’s or policy-makers perceptions of evidence use. There is a big difference between what people think they or others do, and what those people actually do. Sarah Morton’s forthcoming publication in Evidence and Policy will explore exactly what research-users do to apply research to policy or practice.
• The research-policy gap still appears as a barrier to evidence use within this review. Lack of policymaker research skills was also highlighted as a barrier, and we wondered, given the sheer amount of policy-relevant research that is produced, whether these two factors were linked. The issue of a research skills gap in local policy-making has been highlighted elsewhere[ii].
• So does more need to be done to present research in formats that can be easily digested into the policy environment? We have been working on this…
• GUS briefing in partnership with the Scottish Childcare and Protection Network, ‘What can research do for you?’ 
• GUS Briefing from Scottish Government Community Regeneration and Tackling Poverty Learning Network Learning Point 50 ‘GUS: Using evidence in practice’
• GUS Briefing for the Improvement Service ‘Key Messages from Growing Up in Scotland – Briefing for elected members no.13’
• Evidence Request Bank and About Families project - produces and shares appraised summaries of evidence in direct response to practice needs.
• We facilitated a meeting with Scottish Government policy staff to help inform the development of the Parenting Strategy, highlighting research from several of our Directors.

Improving collaboration

• In our experience collaboration and access are closely linked. Close collaborations between policy-makers and researchers increases access to research surely?
• There is general recognition of the importance of relationships between policy and research sectors, and issues are now more about nurturing a collaborative approach where both parties are active participants.
• We were also pleased to see that comparing the previous review with this one, mistrust is less of an issue. It seems a success of those working within the broad knowledge exchange field that the various audiences are at least recognising the role each other plays.

Tactical evidence

• We noted that the article didn’t refer to tactical or political use of evidence; that is when evidence is used or presented in a certain way to provide a desired answer. We felt that it must have been mentioned in some of the research included in the review.

[i] http://www.jrf.org.uk/publications/promoting-change-through-research-impact-research-local-government  http://www.alliance4usefulevidence.org/publication/squaring-the-circle/
[ii] http://blogs.lse.ac.uk/impactofsocialsciences/2013/08/19/is-it-time-to-ban-the-term-dissemination/