If I want to say find people’s preference for let’s say Chipotle vs. Taco Bell (as an example) what would be the best way to do this? I want to justify or be proven wrong on an idea but cannot find any data in the market on it currently.
I’m currently thinking along the lines of:
Suveymonkey
Creating poll topics in active forums (ex: reddit)… whether or not they are conducive to polling is a different story.
Calling 1000+ people for a sample size doesn’t really seem reasonable to me…
for starters, 1000+ seems fairly large sample size for population proportion. I don’t know what confidence intervals you have in mind, but you will probably do fine with 200-400ish sample participants for a rough verification.
The real challenge is ensuring that your sample is representative of the population and also that there is no response bias (i.e. one set of preferences or demographics being more likely to respond to your survey than others). This usually means you have to track down nonresponders and pester them to respond in a way that doesn’t influence their responses. In terms of sample bias, the problem is that just because people may be a random sample *to you* doesn’t mean that they are random in a statistical sense.
For a split up/down decision, a sample of 2000 generally gets you to a 95% confidence interval of +/- 3% of the mean, unless the responses are highly lopsided, in which you’ll get less. Add more categories, and you need more.
For my example, my priority is to answer the question “Will Cantina Bell eat at Chipotle’s market share and how much?” with a more educated guess than “Given Taco Bell’s store base, similarity in offerings and demographics, and competitive price point I believe this will be a substantial threat and eventually reach X% share of the market in Y years”.
I don’t know how to answer this question without a survey of statistical significance (which it seems like Greenlight did). As bchad points out, even if I did conduct this survey it would most likely be full of statistical biases.
If it’s a market share question, then you might survey people who are leaving both venues and try to gauge their willingness to switch. Is one group more fiercely loyal than the other? This doesn’t addres all the biases, but it focuses the question on something that is easier to sample than “the population of the United States.”
If you phrase the question like this, I don’t even see why the population of the US is relevant. You are trying to gauge whether Cantina Bell will pull away market share from Chipotle, so your relevant sampling population is the existing Chipotle customers. Best sampled at the door of the establishment rather than through online surveys.
Remember, though that Chipotle can also steal from Catnina Bell, so you would want to sample those establishments as well.
That leaves the uncomitted taco eater out of the population, which means that you either have to assume that the uncommitted taco eater population who will actually commit is small (possible) or fairly similar to the existing population of taco eaters (possible). To get those people, you would have to sample the broader population.
Survey work is hard if you are trying to do it well.
This thread has been really helpful so far. My takeaway are:
Shape your question into a more actionable idea. I’m obvoiusly not looking at burritos but looking at why someone would switch product from incumbent seems to matter much more than general preferences (if there is a specific reasoning it should be discovered pretty easily anyways).
Hitting your target audience matters much more than the sample size. I probably need at most 40 data points from people who are at a Chipotle rather than questioning the population to see whether or not they ate at either or both and their preferences between price, quality, and ingredients.
Cantina Bell doesn’t have an established customer base since it was launched less than a year ago, so trying to control your survey for potential outflow of customers towards Chipotle is not a reasonable use of time and effort. Cantina Bell’s initial customer base is most likely a subsection of Taco Bell’s existing customers. Would they suddenly decide to switch to Chipotle based on the trigger that their existing establishment expanded its service offering by introducing Cantina Bell? Not a very logical scenario. On the contrary, Chipotle’s customers might be attracted to this new option, so clearly that’s where you need to sample.
Sampling uncommitted taco eaters doesn’t seem very relevant to me either - even if you are able to find that larger proportion of the uncommitted taco eaters would consider trying Cantina Bell as opposed to Chipotle, this doesn’t necessarily tell you that Cantina is eroding Chipotle’s growth prospects. Maybe the overall market for taco eaters have expanded because of the introduction of this new offering, and it doesn’t need to happen at Chipotle’s expense. You have no way of comparing Chipotle’s growth opportunities conditioned on Cantina Bell’s launch vs. Chipotle’s growth opportunities conditioned on Cantina Bell’s not existing - better leave uncommitted taco eaters aside then.
I’m just thinking the method through. If there’s a good reason to ignore Cantina Bell and the rest of the public, that’s great. Personally, I’ve never heard of Cantina Bell and I avoid Chipotle.
I’m not actually looking at Cantina Bell vs. Chipotle. It was just an example that I put out which is somewhat similar to what I am interested in looking at. I thought it would be a good one because of the consumer visibility and the fact that Einhorn did a presentation on it less than a year ago.
It’s not just food. I’m thinking about product preferences in general. It would most likely be too time and capital consuming to ask the population about whether or not they preferred iPhone and Samsung and why, but if I find the core Apple fanboys are now switching to Samsung, there’s clearly trouble. I think we all know at least 10 people whom fit into the Apple fanboy bracket. I can show all the statistics related to processing speed, graphics, memory, etc., but at the end of the day I really doubt my boss will care unless people are switching and the shares move. If they don’t he’ll say to me so why do these specifications matter??
Once again, this is just another example and not what I’m actually looking at but pretty relevant.