What are recommender systems? | CryptoNewsHerald

What are recommender systems?

Recommender systems are algorithms that select relevant products and services based on user data.

Technology is one of the subsections of machine learning.

When did recommender systems appear?

Recommender systems have appeared relatively recently. In 1990 technology first mentioned Swedish scientist Jussi Karlgren, describing it as like a “digital bookshelf”. This work formed the basis of his future research.

In the 2000s, recommendation algorithms began to make their way into the e-commerce space. One of the pioneers in this area is the online retailer Amazon.

In 2006, Netflix, then a subscription DVD rental company, launched competition for the best recommendation algorithm with a prize pool of $1 million. To obtain it, independent developers had to improve the accuracy of the recommendation algorithm by 10%. In 2009, the prize was presented to BellKor’s Pragmatic Chaos team.

In 2010, recommender systems appeared on social networks. Today, most popular platforms have abandoned the use of a chronological feed in favor of an algorithmic one.

How do recommender systems work?

There are two main approaches currently used in recommender systems: collaborative filtering and a content-based model.

The basic principle of collaborative filtering is to generate recommendations based on data about other users with similar interests. Filtration happens user-based And item-based.

The main task user-based algorithm – to find users whose interests are as similar as possible based on the products they consumed and the ratings they gave. Let’s say Anna and Vadim bought juice, a bun and yogurt. It is also known that Maxim often purchases juice and buns. That means he needs to be recommended to buy yogurt.

item-based recommendations consider the problem from the opposite side: to find similar objects and see how they were evaluated before. Let’s try to find out if Maxim likes yogurt. We know that he likes juice and buns. Yogurt, as a food product, has similar characteristics. So we can assume that Maxim will like this product.

The purpose of collaborative filtering is to find a user who has rated a particular object and calculate the correlation coefficient of the vectors of his ratings of all objects in the database. For this, the method is often used k-nearest neighbors.

in the center of the model, content based, the object is located. User ratings are not required for the algorithm to work. It is important for a model to know any properties that characterize an object: author, genre, country of origin, manufacturer, etc. At the same time, it is necessary to understand that not all of them are relevant to the consumer, so you should limit yourself to only the main attributes.

Recently, content-based models have become very popular. They do not need to be trained for a long time, developers can immediately start recommending products for users.

However, this method also has disadvantages. Many users noticed that after searching for a certain product on Google, they were “chased” by an advertisement offering to purchase this product in some online store. To reduce the number of negative reviews about the irrelevance of such ads, developers complement the algorithms with knowledge-based models. They also do not rely on ratings, but only take into account user and product profiles.

How do recommender systems collect data?

Data for recommender algorithms can be collected in explicit and implicit ways.

Explicit methods include asking the user to rate items on a differentiated scale, rank them from best to worst, compare two similar products, or make a list of favorite items. The key point is that the user understands that his data is used by algorithms and agrees to their processing.

During the covert method, website visitors are not always aware that their actions can be used by recommender systems. This includes cookies, Google or Facebook ad trackers, detailed analysis of video interactions, and more.

Typically, many governments require websites to notify visitors when such data is being collected. However, users do not always have the option to opt out of this.

Where are recommender systems used?

As already mentioned, recommender systems are widely used in e-commerce. With their help, online stores can advise customers on relevant products in the “You may also like” block or offer complementary products directly in the basket. Also, if the product is not in stock, the algorithms can find analogues.

Mailing lists also often use personalized recommendations.

Similar algorithms are used by retailers like Amazon, Ozon or Wildberries.

Major streaming services also use recommender systems. Among them are Netflix, Spotify, Apple Music, Yandex.Music, YouTube, Megogo and others.

Recommendation algorithms are also widely used in social networks. Facebook, Twitter, Instagram, VKontakte and others have been showing users content collected by algorithms for many years. Only a few of them allow you to switch to the timeline.

What are the problems with recommender systems?

Recommender systems have a number of limitations. One of them is the problem of a cold start – when enough data has not yet been accumulated for the algorithm to work. This is a typical situation for a new or unpopular property that has been appreciated by a small number of users, or for an extraordinary consumer whose preferences are very different from the average user.

In such cases, ratings are adjusted artificially. For example, the score is calculated not as a position average, but as a smoothed average. With a small number of reviews, the rating of an object will gravitate towards a certain “safe average”, and when a sufficient number of real ratings are collected, then artificial averaging is turned off.

Another problem with recommender algorithms is bias. Inaccurately tuned algorithms, stereotypes embedded in them, as well as user actions can affect the ranking of information.

In 2021 Facebook Ads Algorithms disproportionately showed different job postings for men and women. The automatic photo cropping tool for the home Twitter feed focused on young and slim girls in most cases.

In both cases, the developers quickly fixed the errors, but this is not always possible. Google is constantly facing criticism of the work of recommender algorithms.

For example, search results for “athletes” and “athletes” are very different. In the case of men, the algorithms show articles with the professional achievements of athletes. However, in relation to women, the system gives different ratings of “attractiveness” and “sexiness”.

Not only users, but also bots can influence search results. In 2018, Reddit users held intentional manipulation with Google’s algorithms so that the search for “idiot” displays a photo of former US President Donald Trump.

During congressional hearings on the incident, Corporation CEO Sundar Pichai informedthat the company’s employees do not interfere in the ranking of information. The algorithms do it themselves, he says, by scanning millions of search strings and ranking them according to more than 200 parameters.

Algorithm bias can also be exploited by recommender system developers. In October 2021, a former Facebook employee published documents proving the intentional use of “harmful” tools on the site. According to her, top management knew that the algorithms were intolerant towards the unprotected segments of the population. But the company was in no hurry to fix the bugs, as such content engaged users more and increased the company’s revenue through displaying ads.