Friday, March 22, 2013

Morgan's multiple modalities

Morgan has described four of its polls to date as “multi-mode”.

  • The first multi-mode poll was published on 5 March. It had a whopping sample of 9101 respondents. It comprised “web-based surveying combined with face-to-face surveying.
  • The second multi-mode poll, published on 12 March, also comprised “face-to-face interviewing and … online surveying”.
  • The third multi-modepoll, published on 20 March, saw a different methodology. It was "conducted via face-to-face interviewing, online surveying and via SMS polling". 
  • The fourth multi-mode poll, published on 22 March, gave rise to a third methodology. It was solely conducted by SMS polling.

Analytically, this flexibility (some would say inconsistency) in methodology makes it difficult to benchmark the house effects that might arise in Morgan's multi-mode polls. In plain-English, with each Morgan multi-mode poll we may well be comparing apples and oranges. 

Because these flexibilities affect the underpinning logic for the Bayesian aggregation, I am seriously thinking of removing all Morgan multi-mode polls from the aggregation.


2 comments:

  1. Hi Mark,

    thanks for your notes here. We appreciate the interest you have taken in our (Morgan) polls recently. With the rise of different technologies, our polling techniques need to evolve as well to make sure we continue to capture a broad cross-section of the Australian population.

    With the long-lead time to this year's announced election date we have decided to release a weekly Morgan Poll to track closely the voting intentions of Australians.

    Our first multi-mode Morgan Poll was a large sample as we stepped into this method of polling for the first time.

    We have since added SMS Polling to the mix and I can assure you now that every Federal Voting Intention Morgan Poll released from this point forward will consist of face-to-face interviewing, online surveying and SMS polling. As such - you will be able to directly compare each Multi-mode Morgan Poll released on Federal Voting Intention from March 20 onwards.

    Yesterday's poll was conceived at the time the Leadership spill was announced as a Multi-mode Morgan Poll, however, there was simply not enough time between that announcement and the time of the purported leadership spill - 4.30pm - to properly conduct the online component and get the results out in a timely manner.

    I have now corrected the record to state that this was purely an SMS Poll - not strictly a Multi-mode Morgan Poll.

    Our experience with our SMS Polling in recent years has found it is a very accurate method of polling - particularly for sudden and immediate events. Our SMS Poll conducted in line with the 2010 Federal Election was very very accurate in predicting the final result.

    http://www.roymorgan.com/news/polls/2010/4573

    Kind Regards,
    Julian McCrann
    Morgan Poll Manager

    ReplyDelete
  2. Julian - thanks for the feedback - based on your promise of consistency, I will continue with the Morgan multi-mode poll.

    ReplyDelete