Rival Trade-off Approaches

Versions of ‘conjoint’ methods are the frequently used alternatives to SIMALTO by the market research community. The main suppliers of the software used to analyse the conjoint data, Sawtooth Software Inc. have questioned its suitability. Quotes from their 2007 website:-
“In recent years marketing researchers have become aware of potential problems with CBC questionnaires and the way respondents answer CBC questions. It seems overwhelmingly likely that respondents accomplish this by simplifying their procedures for making choices, possibly in a way that is not typical of how they would behave if buying a real product.

“Answering several (CBC) choice tasks  …  is often seen to be repetitive and boring and  …  respondents are less engaged in the process than the researcher might wish.

(CBC conjoint is where alternative product or service profiles/specifications are shown to respondents and they score them according to their preference, or simply pick out the winner – the ‘discrete choice’ version. This method is usually recommended only when there are less than 8 attributes. With 8 or more attributes the ‘original’ ACA conjoint version is often recommended where respondents directly ‘score’ each alternative option of each attribute and then an iterative process of refining these scores is undertaken.)
Richard Johnson of Sawtooth Software, the originator of ACA conjoint method, said in 1974  “The greatest strength of the procedure seems to be its ability to generate rather refined predictions from quite primitive data the cost of this strength is the rather heroic assumption of no interaction among attributes.” We at RFT still feel this to be true today, and it also has the problem of how consistent respondents are when they allocate scores, and then how they react to the repetitive trade off questions used to refine them.

A paper appeared in the January 2007 issue of the International Market Research Journal (IMJR) by Destadli and Lines. Quoting from this paper discussing CBC :-

“The majority of the respondents reported to have used a judgement rule different from the one assumed by the conjoint methodology.

“As expected, but shown empirically for the first time, the majority of conjoint respondents that participated in this study reported to use simplified judgement rules when evaluating the profiles.

The above comments from Sawtooth and the IMJR are obvious from anyone who has tried to seriously answer a CBC questionnaire, but it is nice to see these data collection problems have been ‘officially’ recognized at last. A recent quote from Sawtooth said :-

“Past research has shown that respondents enjoy ‘Buy Your Own’ questionnaires and answer them rapidly, and that the resulting choices have lower error levels than repetitive choices from CBC questionnaires.

SIMALTO is a form of ‘buy your own’ questionnaire with cause and effect logic in its preference-modelling analysis. Because of its robust data collection, depending on the homogeneity of the respondents, it can deliver reliable analysis with relatively small sample sizes.