Recommender System

A Recommender System is a good example of AI and Machine Learning specifically used to increase revenue and customer engagement on e-commerce sites. They generate customer prompts like:  “Frequently bought together” or “You may also like” which we see often. 

They  are used because they deliver:

A Recommender System consists of three components:

  • Customer profile/preferences
  • Product description
  • Software algorithm that can match the two in aN optimal way.

Discovering what a customer likes:

To discover which product a customer may like we must observe their interaction with our e-commerce site. Computers learn this by tracking each user individually. For example, a simple way to do this is to track the pages a user visits and assign points. One point for each visit and minus one to pages they click away right away because this means they did not like it. Pages a user stays on for a length of time are give additional points. Even more points are assigned if the buy a products.

Customer profiling is used primarily on sites that are frequently visited by users.  Social media, e-commerce sites like Amazon, subscription sites like Netflix or Spotify use very advanced recommender systems. There is not much value in tracking a one time visitor to an e-commerce site. It is better to capture their email and cultivate their engagement via traditional Internet marketing methods. Let’s look at couple examples to see how user tracking works.

Facebook Example of AI in user targeting: 

Facebook uses Machine Learning to increase revenue and user engagement by serving content users like. If a user likes the content then they are likely to click on a related ad Facebook displays. Every time a user clicks on an ad Facebook gets few dollars of revenue.  In order to increase the probability that this happens Facebook wants to discover a user’s interest so that the right ad appears on their screen when they visit. To do this Facebook collects 98 personal data points about each user such as age, gender, marital status, job type, interests, credit card type, preferred TV shows, product types they buy, etc…. In addition to this somewhat static information Machine Learning is used to track users’ dynamic behavior. The following table illustrates in principle (Facebook does not disclose its algorithms) how Facebook uses Machine Learning to accomplish this.

Facebook use of Machine Learning

The first column represents typical user actions for each content they see on a FB page. The second shows the percentage of time this user spent on each action. Third column is a fraction of time each percentage represents. The fourth column shows the value Facebook may assign to each action.

The points reflect what Facebook thinks is valuable in determining user engagement. Someone who watches a long cat video and shares it with friends will score high on this content. If they spend no time on a cat video or “hide” it means they don’t like cats.  No point serving a cat related advertisement to this person. It is important to figure out which cat video is most engaging to them. To do that total score for each content is calculated. Simply multiplying the Fraction and Points entries in each row and adding them up yields a score for each content viewed by a user. This way Facebook “knows” that serving a cat related ad along a high score cat content maximizes the probability that a user will click on the ad.

Recommender System Challenges:

A big challenge for Machine Learning recommendation systems is labeling each product with attributes so that a computer can pick and show the products that best fit the user preferences.

A good example is Netflix. They have over 4,000 different shows. It is impractical for a user to browse these, read descriptions and pick one to watch. It would be too frustrating and lower their user engagement. Netflix has to make recommendations. To do this they label each movie with over 100 parameters. Netflix employs approximately 40 specialists who watch each show and assign labels. The specialists assign genre, directors name, actors, ending type (happy, sad, mysterious), level of violence or sexual content, etc…. Given these labels Netflix can determine which movies to recommend that are similar to the ones a user liked. After a user watches many movies, the pattern of what they like and what to recommend emerges. More on this topic at 180find.

The music streaming service Spotify has even a bigger challenge which is a result of the volume of content. They have over 35 million songs.  It is impractical to manually label them and keep up with the countless songs produced daily. Spotify deploys a number of techniques to label its songs. They use artist name, song popularity, text and what other users listened to when listening to this song. Spotify searches the Internet to find any text, opinions and ratings of each song and artist. Spotify also analyzes the audio signature of each song and is able to classify it in terms of about 16 parameters such as beats, tempo, key, pitches, danceability, energy, loudness, etc…

An e-commerce store must also label its products so the recommender system can select them for recommendation.  Simple product codes (SKU’s) that are assigned when setting up an e-commerce site are most common and sufficient to use in an off the shelf recommender systems such as Shopify or Magento. More advanced product labeling such as color combinations ( eg. what goes with a green blouse) require extensive manual labeling.

Recommender System:

Majority of e-commerce sites use simple recommendations methods of the type: most popular, frequently bought together, new or similar products and are available from the major e-commerce platform providers. The challenge is to decide what to recommend in order to maximize the probability that a customer will select one or more of the recommendations. Making too many suggestions may overwhelm a customer and they may ignore the recommendations altogether? A category of “you may also like” represents another challenge. Recommending colors is hard given that there are so many of them ( there are 445 of red, 260 blue and 130 yellow colors) Recommending styles gets lot more complicated and error prone. This type of recommendation system requires more advance deep learning algorithms specialized for fashion in this case.

Key considerations before proceeding:

  1. Off the shelf recommender systems for e-commerce such as Shopify or Magento provide a good starting point.
  2. Over 200 companies listed in the Crunchbase database offer advanced recommendation solutions ranging from consumer products to travel, insurance, pets and food preparations.
  3. Frame the project in terms of a business objective and benefits you expect.
  4. Ask vendors to explain how they can meet them. Use a site like 180find to help you get answers.