This content has been written by Faye Eldridge, director of FYAMI. You must not copy this information without permission. You can share for insight purposes with references to this post. This content forms part of doctorate (Doctor of business, Phd equivalent) studies by the director of FYAMI.COM
It is to be used for educational purposes.
COPYRIGHT (C) FAYE ELDRIDGE, JANUARY 2020.
October 2020 ALL RIGHTS RESERVED.
Write a brief critical reflection on the strengths and limitations of survey research. Discuss the
potential role of survey data and analysis in your own research and how you would ensure
you collected and analysed quantitative data effectively.
- THIS IS A BRIEF VERSION/ DRAFT and DOES NOT INCLUDE ALL THE STRENGTHS AND LIMITATIONS OF SURVEY RESEARCH NOR IS IT A FULL INDEPTH EXPLAINATION RELATING TO QUANTITATIVE DATA.
Surveys are known to be a good way to gather information from many people. Surveys can
be used to find out what people think and feel. Surveys can be used for descriptive,
explanatory or exploratory types of research. My research is mostly exploratory; however this
may change as my methodology is not yet finalised.
Survey research has several key strengths compared to other research methods. Surveys help
us to measure unobservable data, like peoples attitudes and beliefs or factual information like
‘how much do you earn year?’
In my research I aim to conduct targeted surveys with stakeholders and non stakeholders in
SME businesses about their opinions on collaboration and leadership more specific to the
chief marketing officer and the chief technology officer.
Whilst I haven’t finalised the methodology of this yet including the questions, I am aware
that overall survey research ‘tends to be reliable if conducted in a valid and well thought out
way. Although that doesn’t mean surveys are always reliable.
Surveys that ask respondents questions, can be carried out via different ways including
online, via email or via the phone and these ways are seen often as convenient both to the
respondents and researchers. However, there are risks with phone call surveys as the
respondent could miss hear you. In my research I can ask participants to answer questions via
email. Email surveys are cost effective. Surveys can also be carried our in a speedy manner
and they are considered to be a cost-effective way of gathering data. It’s also useful as
researchers are able to collect data from people anywhere in the world that they may need to,
providing they have an internet connection, as they are able to send an email link out to
engage respondents to participate. Researchers can use the power of social media to
encourage responses to the survey. An example of a less reliable survey could be down to a
poorly worded question. This could cause those who respond to either misinterpret it wrongly
or different to others. If the questions are well thought out and well-constructed with a
thorough survey design, then we would expect this type of survey methodology to produce
reliable results. There is also a term called, ‘the art of question phrasing.’ Gideon (2012).
This is about the way questions are phrased and how this can also have bearing on whether
someone is non responsive to a specific question. How the words are phrased is also a part
the questionnaire design, something I need to take seriously as part of my research.
There are suggestions that survey errors come from non-sampling errors. (Assael, Keon
1982) This includes things like data entry errors, non responses on some items and biased
questions.. There is a big importance on questionnaire design and specific question phrasing
and how they affect response rate and the quality of the responses.
In 2007 I worked on market research projects conducting surveys over the phone gathering
responses from finance directors. What I noticed from doing this is that some of them asked
me over the phone, “what do you mean by that question?” They wanted further clarification
of the question. I was informed not to prompt, help or explain anything about the question
and to tell them to answer it based on their own interpretation of the question. Some
respondents then decided not to answer the specific question as they didn’t understand what
was fully meant by it. The fact that some respondents asked me to provide an explanation of
the question, implies that some of the survey questions were up for mis interpretation. This is
something I need to be aware of in my survey research.
To further emphasise on this, scholars agree that ‘the way that a question is phrased can have
a large impact on how a research participant will answer the question. (Zechmeister, Jeanne,
(2011). In some cases they won’t answer the question. I understand that it’s important that I
am conscious of my wording when writing survey questions especially as words can be
misinterpreted.
The major issue with this is that surveys are in some way inflexible. So, if you realise after
surveying 50 people that the phrasing on a particular question is confusing some respondents,
it’s a bit too late to change it. I plan to test the questionnaire with some neutral/friendly
participants before launch and refine based on feedback.
One of the questions I may ask the CMO and CTO’s is, ‘how often do you communicate on
matters relating to the business – daily, weekly, fortnightly, monthly, less than monthly. This
however will be grouped, as I will find both the CMO and CTO in a business and see how
they answer this. If the CMO says less than monthly and the CTO in the same firm says daily,
this highlights some further investigation. This requires me to carry out a type of a cronbach
test to identify how consistent the responses are (basically the reliability of the responses) and
it requires me to do a number of t-tests to see if different conditions affect the variable. The
businesses I will be surveying are all in the professional service sector but I also want to
understand if there are different results between certain industries/sectors.
4
Also, some responses may be, ‘sensitive to the types of questions asked.’ (Tourangeau , Yan
2007). Poorly framed or ambiguous questions will likely result in meaningless responses.
Dillman (1978) recommends several rules for creating good survey questions and that every
question should carefully scrutinised.
As noted above one of the key considerations to consider when I carry ou my survey research
is to consider the wording of the questions and perhaps the order of the questions. I could
consider asking a similar question and using both free response questions and closed
questions. Free response questions are open ended and closed based questions aren’t and are
usually multiple choice like, a ‘yes or no,’ option, however using ‘yes or no,’ isn’t the best
form of multiple choice despite it being simple and direct because a person may not see their
answer as simple as either yes or no, it could be yes… if this or no… if this. For example a
question like, ‘ who are you voting for in the next elections, pick person A or B, may not be
the best question as this may be based on whether person A or B takes action on something
within a certain time frame. These questions always have a series of why or other related
conditions in the survey to test the reliability of the data and to address certain points.
Furthermore, what if someone’s opinion was more complex than a simple yes or no or a
rating scale of 1-5? Also you could skew a result by having a question that doesn’t mean
anything like ‘maybe, maybe not.’ Meaningless responses won’t provide value in my
research.
Also to note is that the people you may be surveying may not have a lot of time. Surveys
though I would argue a god way to save time instead of asking people to travel for face to
face surveys. I have done phone interviews before where we got towards the end and they
said they’ve not good enough time as they are about to go into a meeting and to just put
“anything down for the last few questions.” I couldn’t do this, but it highlights that time may
affect responses – lack of time towards the end of the survey could also affect responses.
Also I could go deeper with this and say someones mood and level of stress on the day could
have an affect on how they answer the question and not just their lack of time.
Another draw back of surveys is that some respondents may be unwilling or unable to provide
information. You also get biased responses and people providing inaccurate information as
well as lack of understanding of the question.
To ensure my research is carried out successfully, it’s also important that I survey the relevant
people as part of my research. There are guidelines and tools researchers can use, which I plan
to look into as well which will help me complete, ‘a valid and reliable questionnaire,’ that,
‘will guide researchers through the complex and daunting task from single item phrasing to
the presentation of an entire questionnaire.’ (Gideon 2012)
If a low number of people reply to a survey then this isn’t good. Good research needs a good
data sample. To encourage survey responses, I will inform the prospect respondents on how I
will deal with their data and the levels of confidentiality I will use. I will need to gather data
that can be used to show (prove my hypothesis) that certain management practice delivers
better or best results and for this to happen I need to have historic information about
performance that some people may be less willing to share. I will also let the businesses
/respondents that I am surveying know how I will report and record the survey data and that I
will provide them with a copy of the results if they’d like this. For a respondent to answer a
survey they may want something knowledgeable out of it – like the findings.
Some researches in their survey analyses aim to find, ‘associations and explanations, less
towards description and enumeration and more towards prediction, less likely to ask, ‘how
many’ or ‘how often’ than ‘why’ and ‘what goes with what’. (Oppenheim 1992) I am aware that
my research study is likely to have some different variables and this is something I need to
consider when forming the methodology and the way I then look to analyse the data. Survey
methodology is the study of looking at the survey methods. The most commonly used
statistical measures for analysing surveys are the mean and median averages and the standard
deviations of variables. These not only tell us about the individual data items but are also key
factors needed to conduct more advanced statistical analysis.
Interestingly, a scholar states that, ‘research surveys are neither good nor bad nor are they
right or wrong. ‘It is only in relation to how they are used that they take on the mantle of
being good bad right or wrong.’ (Denscombe 2014) The key thing here and the focus then is
the method of the survey undertaking and then how the data is looked at. There are
suggestions from a scholar that some researchers don’t take full advantage of the information
available in their statistical [survey] results. (King, Tomz, Wittenberg 2000). I don’t want
miss opportunities to see knowledge and findings in the data and this requires me to
understand more about survey methodology. I would also like to make sure my research is
useful to those at management levels and that it is presented in a clear and visual way.
To summarise, surveys are a valuable tools to help with research providing that they are
conducted well. I could argue that survey research may only be as good as the researcher! As
It’s important that I ensure my survey questions are fit for purpose and that I don’t miss out
on anything the data can show me.
