We are testing a new beta website for gov.scot go to new site

Identifying the Priorities of Tenants of Social Landlords

Listen

ANNEX B: METHODOLOGY

This research was designed to inform an understanding of the priorities of social tenants on services provided by their landlord and to explore awareness and views on the jurisdiction and powers of the SHR. Specifically, the research comprised five components:

  • a literature review of the current evidence base on tenants' satisfaction with services delivered by social landlords
  • a telephone survey of a representative sample of social housing tenants
  • a postal survey of all Registered Tenant Organisations ( RTOs) in Scotland, who answered on behalf of their tenant members
  • qualitative research among mainstream tenants
  • qualitative research among elected tenant members from each of the Regional Networks in Scotland, who were representing tenants in their area

Sampling

Survey of tenants

The survey sample of tenants of social landlords was drawn from a database of people who had taken part in the Scottish Household Survey ( SHS) 45 in the first three quarters of 2008 and had agreed to take part in further research. This was a highly effective way of drawing a sample for this research for four main reasons:

  • Given that SHS collects information on tenure type, we were able to target tenants living in social rented accommodation
  • The database includes information about households collected during the SHS interview. This allowed the analysis to include a number of additional variables without the need to collect fresh data in the survey and thus helped to minimise questionnaire length
  • The selected individuals were more likely to be willing to take part in the research having given their informed consent to be re-contacted
  • The SHS database includes full contact details for all individuals included in it, making it possible to send a personalised advance letter about the survey, explaining the purpose and aims of the research, thereby encouraging participation in the research.

Achieved sample

The sample comprised a total of 1,149 eligible households from which 500 interviews were achieved. Table B.1 outlines the response to the survey.

Table B.1: Response to the survey of tenants

Outcome

N

Total dialled sample

1,359

Ineligible leads

210

Valid sample

1,149

Refusals

199

Unable to contact (engaged, no answer)

450

Response rate for contacted sample

72%

To ensure that the achieved sample was in line with the profile of tenants of social landlords in Scotland, the data was weighted by dwelling type, landlord type 46, urban/rural indicator and whether or not the respondent was in receipt of Housing Benefit. Data was weighted using 2007 Scottish Household Survey data.

Table B.2 shows the weighted and the non-weighted profile of the achieved sample on a number of key variables and compares these with the characteristics of the population of tenants of social landlords from the Scottish Household Survey.

Table B.2: Sample profile

Tenants' Priorities Survey
Unweighted profile

Tenants' Priorities Survey
Weighted profile

SHS 2007 data

%

%

%

Dwelling type

House or bungalow

55

44

44

Flat, maisonette or apartment

43

55

55

Other type of dwelling

2

1

1

Landlord type

Council

57

66

66

Housing association or other RSLs

43

34

34

Receipt of housing benefit

Yes

56

51

51

No

44

49

49

Urban/rural

Large urban

36

44

44

Other urban

30

33

33

Accessible small towns

10

9

9

Remote small towns

9

4

4

Accessible rural

8

6

6

Remote rural

7

4

4

Table B.3 shows the weighted 47 and the non-weighted profile of the achieved sample based on sex, age and working status and compares these with the characteristics of the population of tenants of social landlords from the Scottish Household Survey. When interpreting this table, it is important to bear in mind that the comparison is not like-for-like. In the SHS, the respondent is either the householder or the spouse of the householder. By comparison, while the sample for the survey was comprised of named respondents to the SHS, any adult member of the named household was asked to participate in the survey, who, in most cases, would be the householder or spouse of the householder. In general, the profile of respondents is broadly in line with the profile of respondents in the SHS.

Table B.3: Weighted versus unweighted profile of the sample

Tenants' Priorities Survey
Unweighted profile

Tenants' Priorities Survey
Weighted profile

SHS 2007 data
All respondents 48

Sex

Male

31

31

37

Female

69

69

63

Age

16-34

16

17

22

35-54

38

39

37

55+

46

45

41

Working status

Working

32

35

30

Not working (excludes retired and studying)

32

30

38

Retired

32

31

30

Studying

2

2

2

Other

2

2

1

Table B.4 shows the profile of tenants across Scotland by selected local authority and compares this to the population in Scotland of tenants of social landlords from the Scottish Household Survey 49.

Table B.4: Profile of tenants across Scotland by selected local authority

Tenants' Priorities Survey
Weighted profile

SHS 2007 data

*column percentages

Local Authority

RSL

All

Local Authority

RSL

All

Local Authority

%

%

%

%

%

%

Aberdeen City

8

2

6

7

2

5

Dundee City

5

2

4

4

3

4

Edinburgh City

9

3

7

6

6

6

Glasgow City

0

42

14

0

49

17

Scotland

100

100

100

100

100

100

Survey of RTOs

The sample of RTOs was provided by the Scottish Government. In total the sample comprised 661 RTOs and 211 50 returned questionnaires, of which 193 were valid, representing a valid response rate of 29%.

Survey method

Survey of tenants

Fieldwork for the telephone survey of tenants took place from 26th January to 15th February 2009. The interviews were conducted by Ipsos MORI Telephone, using Computer Assisted Telephone Interviewing ( CATI). CATI offered three important advantages for this survey.

  • First, routing is automated using CATI and eliminates potential interviewer error. Given that the questionnaire included questions to be asked of specific groups depending on answers to previous questions, CATI automatically directed interviewers to the correct questions, ensuring no questions were accidentally missed.
  • Second, CATI also facilitated the rotation of questions items thus minimising ordering effects. This was particularly significant in questions relating to rating the importance of particular services.
  • Further, CATI facilitated the inclusion of a paired choice exercise in the survey 51 by allowing for different modules of questions or categories to be asked of different respondents in the sample. Indeed, nine separate versions of blocks of eight pairs of services were developed, with each version being randomly asked of 55 respondents. This would not have been possible using traditional paper and pen techniques.

Survey of RTOs

Fieldwork for the survey of tenants took place from 26th January to 6th March 2009. A questionnaire was sent to all 661 RTOs. In advance of receiving their questionnaire, each RTO was sent a letter by the Scottish Government, outlining the main aims of the research. All completed questionnaires were returned to Ipsos MORI for scanning and data processing. During the fieldwork period, one reminder letter was sent to RTOs to try and encourage those RTOs who had not already responded to participate in the survey.

Questionnaire design

Both questionnaires were designed by Ipsos MORI in close consultation with the Scottish Government and other national stakeholders. To inform the objectives of the research both questionnaires covered a number of topics, including:

  • priorities for tenants and RTOs relating to overall services, as well as specific aspects of customer and day-to-day repairs services
  • perceptions of services provided by social landlords
  • attitudes relating to tenant consultation and participation
  • awareness and perceptions of service quality and value for money
  • awareness and views on inspection and regulation.

Both questionnaires are included in Annex C.

Piloting the questionnaires

To ensure that the tenants' questionnaire was clear and easy to understand, it was piloted among 46 respondents via telephone between 12th and 13th January. More specifically, one of the main reasons for undertaking the pilot was to test different approaches 52 to measuring the level of importance placed on core services provided by social landlords.

The RTO questionnaire was piloted among representatives of the Scottish Regional Tenants Network ( SRTN).

Analysis

Computer tables were prepared to facilitate reporting. Each question in the survey of tenants was analysed by a number of key variables 53, namely:

  • gender
  • age (16-34, 35-54 and 55+)
  • employment status (working, not working and retired)
  • Type of landlord (Council and Registered Social Landlord ( RSL))
  • Dwelling type (house or bungalow and flat, maisonette or apartment)
  • Length of residence (Up to 2 years, 3 to 10 years, more than 10 years)
  • Disability or health problem (yes or no)
  • SIMD (20% most deprived areas and other area (excluding 20% least deprived areas 54)
  • Receipt of Housing Benefit (Yes - full, Yes - partial, Yes and No)
  • Household income (£10k or less, more than 10k to 15k, more than 15k to 20k and more than 20k)
  • SHS Household type (Single adult, small adult, single parent, single pensioner, small/large family, older smaller household and large adult household)
  • Urban/rural indicator (urban and rural)
  • Perception of value for money (Good value for money and poor value for money).

Similarly, each question in the survey of RTOs was analysed by a number of key variables 55, including:

  • Type of landlord represented (Local authority, an RSL area and a mixture of both)
  • Number of tenants represented (1-399 and 400 or more)
  • Number of landlords represented (One or two or more)
  • Time in existence (less than 10 years and more than 10 years)
  • Perceived level of influence of RTO (at least some influence and a little or no influence).

Presentation and interpretation of the survey data

Quantitative research allows us to infer assumptions about the general population. However, when interpreting the findings from the survey of tenants it is important to remember that the results are only based on a sample of tenants of social landlords, and not on the entire social tenants' population. Consequently, results are subject to sampling tolerances, and not all differences between subgroups are therefore statistically significant. Throughout the report, we have only commented upon differences which are statistically significant at the 95 per cent level.

Similarly, when interpreting the finding from the survey of RTOs, it is important to bear in mind that the results are based on a sample of RTOs and not all RTOs in Scotland. Given the limited information that is known about non-responding RTOs, the findings of the RTO survey should be considered indicative, rather than representative, of the total population of RTOs in Scotland. Further, due to small base sizes and incomplete data 56, subgroup analysis of RTO data was not always possible. Where differences have been highlighted in the RTO survey, they should be treated with some caution.

Where percentages do not add up to 100 per cent, this may be due to computer rounding, the exclusion of don't know/not stated categories or multiple answers.

Qualitative research

Six focus groups were carried out with tenants of social landlords: five with, what might be considered, mainstream tenants; and one with tenants' representatives of Regional Networks, representing RTOs.

Focus groups with tenants

Five focus groups were conducted between 10th and 19th March 2009 across Scotland. These were designed to provide an in-depth understanding of tenants' priorities and explore awareness and perceptions of inspection and regulation.

Composition and recruitment of focus groups

The composition of focus groups was designed to capture the views of tenants from a broad range of socio-economic backgrounds and a mix of social landlords. In addition, within specific areas, the composition of focus groups was designed to include tenants living in sheltered accommodation, from BME groups and with disabilities.

A recruitment questionnaire was designed so that only people who satisfied the criteria set to inform the objectives of the research were invited to take part in the focus groups. In addition to the project specific criteria, the questionnaire screened out people who work in market research, media, advertising, PR and/or had attended a group discussion or workshop in the previous six months.

Participants were recruited face to face in their homes between 2nd March and 16th March 2009 by experienced recruiters from Magnetic Field. Ten participants were recruited for each group on the assumption that eight will turn up at the scheduled time. Recruiters telephoned participants prior to the groups to check that they would be attending. Table B.5 sets out the compositions of each group and provides details on attendance. All attendees were paid £20 to cover any costs they might have incurred as a result of attending the focus groups.

Table B.5: Focus Group composition

Group No

Date

Location

Landlord type

Dwelling type

Demographics

Disability

Ethnicity

No. attended

1

Tues 10th March, 7pm

Galashiels, Scottish Borders

Mixed RSL

At least 2 people living in Sheltered Housing

Mixed length of stay in area; mixed gender; mixed age (16 and over); mixed working status; mixed receipt of Housing Benefit

At least 2 people with a disability

Any

5

2

Wed 11th March, 7pm

North Glasgow

Mixed RSL

At least 2 people living in Sheltered Housing

Mixed length of stay in area; mixed gender; mixed age (16 and over); mixed working status; mixed receipt of Housing Benefit

Any

At least 2 BMEs

10

3

Thur 12th March, 7pm

Dundee

Mixed Council and RSL

Mixed dwelling type

Mixed length of stay in area; mixed gender; mixed age (16 and over); mixed working status; mixed receipt of Housing Benefit

At least 2 people with a disability

At least 2 BMEs

10

4

Wed 18th March, 7pm

Fraserburgh, Aberdeenshire

Mixed Council and RSL

At least 2 people living in Sheltered Housing

Mixed length of stay in area; mixed gender; mixed age (16 and over); mixed working status; mixed receipt of Housing Benefit

Any

Any

7

5

Thurs 19th March, 7pm

Kilmarnock

Mixed Council and RSL

Mixed dwelling type

Mixed length of stay in area; mixed gender; mixed age (16 and over); mixed working status; mixed receipt of Housing Benefit

At least 2 people with a disability

At least 2 BMEs

8

Focus group with members of RTOs

In addition to the focus group with tenants, one focus group was conducted with elected tenant members from each of the Regional Networks 57 in Scotland. These tenant members were invited to speak on behalf of tenants in their area about aspects of their landlord's services which were most important. However, given the greater knowledge of this group vis a vis mainstream tenants with regards the regulation of social housing, it was possible to explore perceptions of the SHR in more detail than was possible in groups with tenants. In order to recruit participants, the chairperson of each Regional Network was sent a letter asking them to nominate a tenant member to attend the focus group; the nominees were subsequently phoned by a member of the Ipsos MORI research team and a time and date suitable for all representatives was arranged. Following this, an invitation letter was sent to all participants. The focus group was held on 1st April at the Ipsos MORI office in Edinburgh. All but one Regional Network committee - Region 9 Glasgow and Eilean Siar 58 - was represented in the focus group. All attendees were reimbursed travel costs incurred as a result of attending the group.

Topic guide

A topic guide was designed by Ipsos MORI with input from the Scottish Government. The guide was used to facilitate the focus groups discussions with tenants and RTOs, although for the discussion with RTOs, the guide was tailored to accommodate their greater knowledge of regulation and wider housing policy issues. The main themes covered in the guide were:

  • service priorities and examples of good and bad landlord practice
  • perceptions of service quality and value for money and willingness to pay extra for additional or improved services
  • in the tenants focus groups, general awareness and perceptions of inspection, regulation and the SHR
  • in the RTO focus group, perceptions of inspection, regulation and the SHR and the extent of the powers of the SHR

Interpretation of qualitative findings

In contrast to the quantitative elements of the project, the aim of qualitative research is not to generalise about the wider population in terms of the prevalence of attitudes or behaviour, but to identify and explore the different issues and themes relating to the subject being researched. The assumption is that issues and themes affecting participants are a reflection of issues and themes in the wider population. Although the extent to which they apply to the wider population or specific sub-groups cannot be quantified, the value of qualitative research is in identifying the range of different issues involved and the way in which they affect people.

Literature review

Heriot-Watt University undertook a review of academic and government research on tenants' satisfaction with social landlord services. As well as setting the current research within a broader context, the review also helped to inform the survey element of the work 59. Professor Pawson also contributed to the design of research instruments.