Amazon Personalize to zarządzana usługa AWS, która generuje spersonalizowane rekomendacje na podstawie danych o interakcjach użytkowników z produktami. Integruje się z pozostałymi usługami AWS: S3 służy do przechowywania danych, AWS Glue i Athena do ich przygotowania, a DynamoDB oraz Lambda/ECS do udostępniania wyników w aplikacji.
A Recommendation System in a Mill (Part 2): Recommendations in the AWS Cloud—Amazon Personalize
Amazon Personalize is a managed service that generates personalized recommendations from data about users’ interactions with products. It integrates with other AWS services—for example, S3 for data storage, AWS Glue/Athena for data preparation, and DynamoDB and Lambda/ECS for making results available in an application.
Amazon Personalize automates the training of machine-learning models on data supplied by the customer and makes the results available through a simple API. Its advantage is that we do not have to implement the algorithms ourselves. We need only prepare the data properly—interaction history, the product catalog and, optionally, user profiles—and the service handles model training and the delivery of recommendations.
Amazon Personalize grew out of the experience gained while developing the recommendation system for the Amazon.com store. It provides, among others, the following practical examples: “Recommended for you” (general personalization), “Customers who bought X also bought Y” (recommendations of complementary products), “More like this” (items similar to the one being viewed), as well as trend lists and user segmentation.
How does Amazon Personalize work?
Amazon Personalize primarily uses data on users’ interactions with products—for example, viewing, adding to the cart, purchasing and rating. This is known as item-interaction data. The data may come from historical logs in batches or be sent in real time, for example from an application through an API.
Product metadata may also be supplied in the form of a catalog, including categories and prices, as may user metadata such as age, sex and loyalty segment. Some Personalize algorithms can also use these data in modeling. Unlike the Azure approach, we do not select an ML algorithm directly. Instead, we select a recipe or use case—a predefined modeling scheme:
- User-Personalization, in the default Retail use case, trains a model that accounts for sequences of user interactions and is based on recurrent neural networks (RNNs).
- Similar-Items creates a product-similarity model based on co-occurrence.
- Personalized-Ranking reranks a list of items for a particular user.
A custom solution with an individual configuration can also be created, although ready-made recipes suited to a business scenario are normally used.
Example: an online flour store
Step one: creating the data
To use Amazon Personalize, we first define data schemas and create a Dataset Group in the service. Three datasets will be required.
User–product interaction history (Interactions dataset)
In the simplest form, this is a CSV or JSON table containing the columns user_id, item_id and timestamp, and optionally event_type and event_value. For a flour store, events may include Purchase, View and AddToCart. Each row represents one user action involving a particular product at a specified time.
A sufficient quantity of such data is required. AWS recommends more than 1,000 users and more than 50,000 interactions for meaningful results. Our sales-transaction database will supply some of these interactions—purchases—but other behavior should also be included if available. In a food wholesaler, purchases will nevertheless be the principal signal.
We import these interactions into Personalize, most frequently by uploading a file to an S3 data bucket and then pointing the dataset-import service to it.
Product catalog (Items dataset)
This optional dataset contains product information. For our flour store, attributes might include grain type, country of origin and whether the product is gluten-free. The Amazon Personalize item schema requires a unique ITEM_ID and may contain metadata fields—strings, numbers or categories.
This information may help the model understand relationships among products, such as products in the same category, and handle the cold-start problem more effectively when a new product has few interactions.
Static user data (Users dataset)
This dataset is also optional. It might contain a customer’s age and sex, type of business or country of residence. Amazon Personalize allows several such user attributes to be defined and taken into account by an algorithm—for example, in segmentation. Not every recipe uses these data, but they may be useful when creating user segments.
In our case, the static data are held in the customer database, so they can be imported. It should be noted, however, that advanced custom attributes such as a “propensity to click banners” have no immediately obvious place in the schema. They could be encoded as a numeric user attribute on a specified scale and imported into the Users dataset. Amazon Personalize would probably treat the field like any other categorical or numeric feature and potentially use it during training.
Note: AWS recipes primarily emphasize user–product interactions. These can of course be read from application logs.
Step two: selecting the appropriate AWS recipe and starting training
Once the data have been prepared and loaded from S3 into Personalize, we create a Solution and select an algorithm, or recipe. For general personalized recommendations, we might choose aws-user-personalization.
We then initiate model training. Personalize automatically calibrates the model using the supplied data. Hyperparameters can be provided, but this is not required; default settings can be used. Training takes from several minutes to several hours. On completion, we receive a Solution Version—a trained model that can be used in two ways.
Real time: a Campaign
By launching a Campaign, we reserve AWS resources that host the model and make real-time queries possible. We can then call the Personalize GetRecommendations API, providing a userId for a “for you” recommendation list or an itemId for items similar to a particular item if the SIMS recipe was used. The response is a list of recommended item_id values with probability scores.
Our scenario does not require recommendations to be generated live every time a user visits, because updates are to take place once a week. A Campaign is therefore unnecessary; we will instead use batch mode.
Batch files: a Batch inference job
Amazon Personalize can run a job that generates recommendations for all users or a selected group at once and saves the results to a file in S3. This is ideal for personalized mailings or periodic refreshing of recommendations in a database.
We can schedule such a job once a week after updating the model. The service might generate a CSV file containing a list of N recommended products for each user. We can then import the file into the store database and serve recommendations from there. This preserves offline operation: users see a fixed set of recommendations for a week, refreshed weekly from the latest transaction data.
AWS offers ready-made solutions that simplify automation. One example is Maintaining Personalized Experiences with ML, which uses AWS Step Functions to orchestrate the entire pipeline: regular import of new data, training a new model version, updating a Campaign and running batch jobs. It can be adapted to a weekly schedule.
For integrating results with an application, AWS often presents an architecture in which AppSync (GraphQL) or Lambda retrieves recommendations from Personalize and combines them with database data so that complete product information is returned. Under our weekly batch approach, it is enough to import the finished results into the store database or display them from the file.
Summary and differences compared with Azure
Amazon Personalize is a black-box solution. It greatly simplifies deployment but limits our insight into the model. We do not have to write ML code, but neither do we know precisely how strongly our custom store-user attributes were used. The set of algorithms available to us is limited.
In relation to customers’ behavioral features, Personalize allows us to import them as additional data, but provides no guarantee that the selected recipe will use them. It probably will, but in an unknown manner. Azure provides full control; here we hand control to Amazon.
On the other hand, Personalize integrates easily with AWS infrastructure—S3, Lambda, DynamoDB and Pinpoint for email—and is designed for scale and updating as new data arrive. Events can be sent as a stream, for example, and the AWS model will take them into account almost immediately through incremental mechanisms. Our flour-store project does not require streaming, but Personalize offers the option if traffic grows and more dynamic personalization becomes necessary.
Personalize charges for model-training time and queries. With 20,000 customers and weekly updates, the cost should be reasonably predictable. A free tier can also be used for the first two months.
An example of the result is a “Recommended for you” section on the store’s home page, containing perhaps five products—different types of flour—that Personalize regards as most interesting to the logged-in customer on the basis of purchase history and similarity to other customers. Individual product pages could contain a “Customers who bought this product also bought” section using a separate SIMS (Similar Items) model.
The whole system would be updated once a week, for example on Monday. Personalize would retrain the model using new data from the entire preceding week, and customers would see refreshed recommendations from Tuesday. This approach satisfies the requirement that the system should not operate strictly in real time while still using AWS’s advanced ML infrastructure.
Wojciech Moszczyński
Wojciech Moszczyński—graduate of the Department of Econometrics and Statistics of Nicolaus Copernicus University in Toruń; specialist in econometrics, finance, data science and management accounting. He specializes in optimizing production and logistics processes. He conducts research into the development and application of artificial intelligence. For years, he has been engaged in popularizing machine learning and data science in business environments.