The Surprising Difficulty of Inclusive Surveys
What tool do you use to run a survey? In a crowd of product hunters and serial entrepreneurs SurveyMonkey and Google Forms are often top of mind. And those kinds of tools can work… if the folks you want to survey are consistently online.
So what happens when they are not?
Or … can’t read?
Or see?
That’s what we’ll do in this article: have a brief look at other ways to survey lots of folks.
There is no sales pitch here, no affiliate links. This article is for the curious. For those who‘d like to try on a new pair of thoughts. It’s like changing your brand of tea for the day. Or wearing your socks inside out (under the pants of course, we’re not being crazy).
So let’s push the 📦 (box) over and peak outside.
So, no internet. Hm. Who has no internet? Turns out lots of people. Like 3.7 billion lots. Almost half. (Isn’t that crazy?) Becomes way more when you think about who doesn’t have regular and stable internet access.
So how do you survey someone without internet?
SMS?
Yup, SMS surveys are a thing. They work without mobile data, and an old feature phone will do fine.
Does anybody still remember those, Nokia 3210 and the ilk?
Haha, those days … and yeah, feature phones are still around. A lot.
So, SMS is awesome. You send someone a question, and they reply, and back and forth it goes.
Wait a second. Reply. Uhm, roadblocks ahead!
⚠️ To reply, you need a phonenumber to send too.
Okay, that we can solve. There are two solutions:
#1 Use a phone as “SMS relay“
What the?
No magic there. It’s a normal iPhone or Android app that you install from the store. With that app installed, you connect to your survey platform — and it will use that phone to send out SMS and receive incoming SMS.
Downsides: That phone needs to be in the country. You got to keep the phone charged. You got to make sure it doesn’t run out of mobile credits and mobile data. And sometimes mobile providers (telcos) get a little bit suspicious when a normal phone suddenly sends hundreds of SMS.
SMS relays are a bit of a hassle.
Here is a better option:
#2 Get a virtual phonenumber
That’s like a Skype phonenumber: a number that doesn’t need a SIM. It’s always “on“, and you can send and receive at your pleasure.
Which brings us to the second roadblock:
⚠️ Replying with SMS costs money, money, money!
Not much, but … that’s relative, right? If you’re living on a dollar and a half, you’re not going to be spend 20 cents on a survey. So how do you question someone who can’t afford to reply?
The solution:
Toll-free numbers
Toll-free means they’re reverse-billed: someone needs to pay for those SMS, and if it’s not them, it’s you.
The problem with toll-free numbers (and virtual numbers in general) is that you can’t get them in every country. Or they’re insanely expensive. Not change-your-tea-brand mad but positively insane to the tune of 1,000 bucks to setup. That’s more than 350 espressi at Starbucks in Thailand and that’s a fact.
Don’t believe me? I quickly googled: this company offers a toll-free, all-network long code (normal-length phonenumber) for Nigeria: N550,000 (~$1,300) one-time setup fee, N195,000 (~$470) monthly.
Ouch. No one is going to do a customer satisfaction survey for their tiny herbal tea shop when that kind of dough is involved.
Side-note: SMS leave traces
Other considerations can exclude SMS as an option, too: An SMS can be dangerous to the one receiving them.
No, this isn’t the SMS version of Ringu.
For example, let’s say an NGO operates in a country where contracting HIV isn’t seen as a medical but as a moral issue: as a sin. In such a world you put people at risk by asking them questions about HIV symptoms or by sending them information about where to get tested. Because what if they forget to delete the SMS and someone sees it?
All this aside, you only need to bother with SMS if everyone you want to talk to can read and write at least a bit. What if they can’t?
Let’s talk
Normal phone calls, anyone does those still? In many countries they seem to be a dying breed, but they can be a great tool to survey people anywhere.
Folks don’t need to be literate, and they can reply without spending one dollar, lira, peso, or gulden.
How does such a voice survey work? Fully automated or assisted.
If you got your survey fully automated, you play pre-recorded questions, and they reply by talking or typing on their keypad.
In an assisted setup (CATI — computer assisted telephone interview) the selecting and dialing is automated, and the interviewer guides the participant through the questions, entering answers as they go.
There are obvious difference in cost, scale, and what you can do between the two scenarios that I won’t go into.
AI & Voice recordings
Back to talking in the fully automated case. They speak, you record these responses … and then you spend twenty hours and endless cups of that new tea brand (it was a bad, bad idea!) listening to all those recordings, most of which will be empty or garbage.
That isn’t nearly as fun as… it doesn’t sound fun at all! But, no worries, happy sailor … nowadays, we throw AI at those problems, right? Voice-recognition to the rescue!
Sounds like a great idea.
Until you leave the realm of mainstream languages. Google’s Speech-to-text has a pretty decent selection of languages as far as these tools go, but even they got only 125. That might be a bit of a problem if you’re switching to voice because people are illiterate. Chances are they don’t speak Oxford English.
Right. So, let’s say surveys with both SMS and phone calls are not an option? Is that it then? You have to walk around with an iPad and talk to people? Actually, you’ve got one more.
WhatsApp, Messenger & Audio clips
WhatsApp?? But dude, you just said no internet!
Dang, did I? Fine, I did. Readers with good memory, no fun! Kidding aside, there is such thing as a bit of internet.
If you can’t afford mobile data, Facebook with its insane desire to connect people and better the world haha I mean sell ads is making Facebook services available in many countries. They can be totally free, or included in very cheap prepaid packages.
And that means you can sometimes run a survey by people with Facebook or WhatsApp even when they can’t access your website. And here are fun reasons to do that:
- cost: it’s free for them, and very cheap for you
- convenience: many people are on FB/ WhatsApp anyway
- multimedia
Multimedia means pictures: Hey, we’re doing an anti-corruption campaign. Here is a picture of a road near you. We were told it was finished half a year ago. Do you recognize that road?
Pictures can go both ways: we’re sorry your Mangosteen peeler arrived broken. Send us a picture of the damaged box and we’ll make it right.
Multimedia also means voice clips. And there is an exciting application to use those: surveying illiterate or blind people.
By sending questions as (again pre-recorded) audio clips, you get around the text problem. People then reply by recording a voice clip of their own, or by sending single words, characters, or digits.
The UN used this approach in Lebanon. While the article is bland, the underlying use of WhatsApp voice messages for a survey is very cool.
(Yes, you still got to solve the “listen to the recordings“ problem.)
And then came that stupid pandemic
Exploring survey tech was first about saving costs and time. Sending people around with clipboards is expensive and relatively slow.
When the pandemic came along, people realized that sending someone to a remote village community for a few interviews is a fairly ghastly thing to do with covid about. Being able to run a survey from afar then suddenly had health benefits, too.
And we’re leaving the rabbit hole — thank you for joining!