07.09.2020 change 07.09.2020
Ludwika Tomala
Ludwika Tomala

i-Spy! Tech giants may not spy on us, but others do, say experts

Credit: Fotolia Credit: Fotolia

You talked to a friend about a product yesterday, and today you see an ad for it on the Internet. Does this mean that the devices we surround ourselves with eavesdrop on our conversations in order to 'push' more products? According to experts, no. In an interview with PAP they explain that although the technology exists, for the time being there is no conclusive evidence that tech giants are spying on us for marketing purposes.

Professor Adam Wójtowicz from the University of Economics and Business in Poznań says that in order to determine whether an application is listening or not, you need not anecdotes, but repeated experiments or an analysis of the application code.

He says: “So far, there is no hard evidence that known applications from large vendors obtain data for marketing purposes in this way without the consent of users.” He points out, however, that some smaller applications have already been caught sending, for example, data from cams to their manufacturers. “But a similar activity has not been proven in the case of applications from leading vendors,” he says.

Dr. Karolina Małagocka from Kozminski University says: “There was an attempt to prove that the giants were eavesdropping on us, but it turned out that the devices exposed to voice recordings did not activate any processes not found in devices that were not exposed to sound recordings."

For people who suspect that their smartphones may be listening, the researcher advises to call the number *#21#. “The provider will then send a message informing whether your audio and video are transmitted anywhere,” she explains.

But Dr. Małagocka points out, that even if tech giants do not eavesdrop on our conversations, they do obtain our data from other, cheaper sources. “They use digital traces we leave online. It is easier to analyse clicks, texts, user activity than voice samples,” she says.

Experts agree that social media giants already have users quite well profiled, for example thanks to the analysis of visited pages, hours and model of media use, and financial activity. On this basis, it is possible to determine someone's interests, age, personality type, health, political views.

Dr. Piotr Kaczmarek-Kurczak from Kozminski University explains that in order to automatically analyse what the conversation is about, you need relatively high computing power. The voice analyser would have to convert sound into text, send it to the central system, extract keywords. And that, in his opinion, would be an exaggeration in terms of the cost-benefit ratio. In his opinion, you can learn much more about a given person by following their activity, for example in the search engine. Or by getting their data when they do a quiz.

Professor Adam Wójtowicz sees the matter differently. In his opinion, voice analysis would allow companies to additionally supplement the knowledge about users in order to increase the accuracy of micro-targeting, and thus increase profits. In his opinion, voice analysis is not a technical problem. “A smartphone is a computer that can process what we say locally, for example search for specific keywords,” he says, pointing out that if a voice assistant is installed, it listens for commands anyway. If such extra words are detected in the conversation, they could be transcribed, sent as text data, and then used for marketing purposes. And this would happen without a significant loss to the computing power or bandwidth available to the user.

He adds that voice acquisition could take place at the level of the application, program libraries or the operating system. He notes that many apps have access to the microphone. And he urges you to check on your devices which applications you have granted permissions and to what resources. And whether it is necessary in each case.

Dr. Małagocka also believes that companies could have an interest in analysing voice data. She says:“Big companies need a lot of data to train their algorithms,” and is then interrupted by her smartphone's Google Assistant, which has apparently recognized a command in this sentence and says: 'I could not find the results'. A sign that Google's listening algorithms - if we want them to - can hear conversations, but they are still far from perfect.

Researchers note that algorithms are able to track not only the content of conversations, but also other variables contained in the voice: the person's age, health and mental state. Dr. Kaczmarek-Kurczak reports that some call centres already analyse the voices of employees (and customers). If signs of nervousness or fatigue are detected in the employee's voice, the program prompts the employee to take a break to prevent customers from being put off by a bad-tempered employee. And if irritation quickly appears in the voice of customers, it may be a signal that conversation procedures should be revised.

Why we sometimes see ads for the products we have talked about may not be just a coincidence.

Dr. Małagocka explains that online advertisers know very well what interests us. If you talk to a friend about a cosmetic procedure, you might have read about it before, made print screens of advertisements or products from this area. This activity can be analysed and sooner or later companies will send you an ad on this subject.

Another effect, this time related to psychology, is that if you talked about something with a friend, you start to pay attention to displayed ads on the topic, ads you probably ignored before.

Professor Adam Wójtowicz adds that ads may include products related to people with whom we are in contact. So it may be that a person you talk to is openly interested in a topic or product and often searches for it online. And the advertising network - through probability analysis - sends advertisements for these products to people, with whom that person has close contact, detected in physical space (thanks to geolocation) or in cyberspace (thanks to the analysis of communication or social networks). The appearance of the advertisement does not have to be related to the content of the conversation, but to the fact that the conversation took place at all.

Dr. Kaczmarek-Kurczak explains that apart from specific groups of people, maybe except specific journalists or politicians, large businesses are not really interested in determining the preferences of an individual, predetermined person. These companies are more interested in aggregates, customer groups. For example, childless women aged 20-25. The offer is tailored to such groups.

When asked if he is concerned about how much companies know about him, Dr. Kaczmarek-Kurczak replies: “No, because I understand their business goal. And I know that I am not a target for them, but just a single grain of sand. I am much more concerned about the fact that governments and political institutions can have access to these data. And that it gives them the opportunity to shape views on a mass scale."

Professor Adam Wójtowicz agrees. In his opinion, the ability to eavesdrop on conversations for marketing purposes is less of a problem than the fact that this technology can be used for more detailed user profiling and further erosion of privacy, for surveillance by hackers, malware, and governments. And that these methods can be used not only to match advertisements to users, but also, for example, to manipulate users' preferences or moods with precision tailored to each individual, but on a mass scale.

Since 2013 and the Edward Snowden scandal, we know that state agencies can spy on us on a massive scale. Dr. Małagocka says: “Any device that has a microphone, a camera and the ability to transmit audio and video, can potentially be used as a spy tool.

"Unfortunately, the choice we are facing is privacy or convenience. We either choose security or user-friendly solutions. These spheres clash with each other.”

PAP - Science in Poland, Ludwika Tomala

lt/ ekr/ kap/

tr. RL

Copyright © Foundation PAP 2021