Nowadays with the help of Internet we all are using so much of applications as like Youtube,Amazon,Flipcart,Myntra etc..And to personalised the user experience this companies using the recommendation systems
Lets understand How actually recommendation system works…
Every major e-commerse or entertainment website recomments products to you based on various factors as like how Youtube,Netflix ,Amazon Prime recommends movies and shows to you and how Amazon shows you products that it thinks you might be interested in …
Based on your search history and how you have interacted with the services provided by them or how similar peoples have interacted with the service they try to make experience more personalized.
Let us take an example of Flipcart. Let’s say you want to buy a headphones from Flipcart.when you type Boat Headphone you will get something like this as shown in the image
Now let’s say you liked one of them and you want to buy the product then went inside to see the price ,specifications reviews of the product
Flipcart is collectiong all your information , every action of yours,flipcart has started to know your likes and dislikes
Why flipcart doing this? Reason behind this is Flipcart wants you to recommend a product based on what you may like or what you may buy from its website, it’s a very beautiful idea if you think about it.
Here flipcart showing other similar products as like you selected or wish to buy.The main reason behind this is that
“If you like this Boat Rockers Headphones then there are more chances that you will like similar Headphones as well”
Now see Youtube recommendation system….
Network Structure-
YouTube wants you to stick around for as long as possible. The more videos you watch, and the longer you watch for, the more ads it can serve you, and the more revenue it generates
That is precisely why the YouTube recommendation function exists. It is designed to guide users toward content they love and actively want to watch, based on a range of factors
How do youtube recommendation systems work?
Content based recommendation system-
It is a very simple concept, in which the service suggests you on the basis of what you are iterating with. Like if you are watching videos on a particular topic, youtube would recommend you videos on similar or same topics.
The system internally has a set of different features and a score related to that for all items. Now the system checks from the scores of the content or item you are interacting with and based on that it compares with feature scores of different items and the closest it finds it suggests. These kinds of models also rank feature scores based on the user details like their age, s or location, and other personal information…
Collaborative filtering method-
This works by recommending items that similar users have liked, therefore it groups similar users together and shares their interests within their group. This is not limited to a particular item but also what kind of item it was and what properties it holds. It can also be bidirectional in nature, for example, it recommends content that similar users have liked.
Also when a user likes some content, it helps in grouping them with similar users, thereby improving everyone’s suggestions. It is based on a simple assumption that if person A has similar taste for most things as person B, his taste will match user B’s in future interactions as well.
There are two methods of implementing collaborative filtering:
1. Memory based
In this, past histories of customers are plotted in a matrix. It uses statistical methods to group together members with similar user-item interaction matrices, and recommend products based on that. Simply said the algorithm tries to match similar profiles and based on this it recommends to the user.
2. Model based
In this method, we try to reduce to the user-item interaction matrix. As the number of users and the number of products grow, having such a large dataset and using statistical methods for calculating relevance turns out to be an intensive task. So we try different methods of dimensionality reduction and matrix factorization with traditional machine learning algorithms or with some new deep learning algorithm, the one we are going to use today.
For this task we are going to use a deep learning method for building the recommendation system.
Understanding Youtube Video Recommendation System-
Researchers from Google released this paper demonstrating how deep learning can be used for such a task efficiently. Although the real recommendation system is probably a hybrid one combining properties of content based, collaborative filtering and popularity matrix methods for recommending in production.
The way this paper treats the problem of recommending stuff is extreme multi class classification, where the model outputs or classifies which video is suitable for watching with context to an input user. The main problem is the extremely huge corpus of videos that youtube has, so the idea of this paper is to pass the data into two networks, reducing the number of videos at every stage. The model consists of two network each with their specific tasks
1. Candidate generator network-
This network shrinks down the large corpus of videos, possibly billions of videos to some hundreds of videos. It takes in the huge number of videos as well as user log history as input. Then based on several factors such as search query, user history, demographic details and other user information it narrows down the list of relatable video to that user from billions down to hundreds. The authors of the paper tell us that this model aims for accuracy and relevance and may remove videos with higher views, but may not be relevant.
2. Ranking network-
Based on the output of a list of hundreds of videos it gets from the first network and other features like user engagement behaviours like watch time, clicks, likes, dislikes. It also takes in consideration factors like user information, video information, and then scores each video according to the possibility it may be liked by the user. After taking all of this into consideration it ranks the videos and then selects top n videos according to the requirements. All this seems easy but taking into context all the features it has to deal with, it gets complicated pretty quick.
