Blog

What is Product Data Transformation?

Data transformation is, simply, the change of data from one format, structure, or value to another.

Product information or product data, is the information you will see displayed on product detail pages on eCommerce websites, such as images, descriptions, price, weight, dimensions and SKU numbers.

So together, it is the process of changing product data or product details from one state to another. Data transformation is essential to the activities of product data integration and product data management.

Product data transformation is applied to create structured and normalized product data to feed your eCommerce website. To do this would likely require several different transformation activities. Such as validating data to remove duplicates, converting data types, enriching the data, or performing aggregations, depending on your data goals.

Why is product data transformation necessary?

When collecting product data from a range of suppliers (vendors) to populate the digital shelves of your online store, there is a good chance you are receiving that data in a range of formats and quality levels.

Raw data from vendors is often,

Poor quality:

Containing both relevant and irrelevant data
With duplicates, missing fields, or inaccurate data

Disparate:

Using a range of different naming conventions
Images in various sizes, resolutions, and styles
Different units of measure

Not customer friendly or optimized for SEO:

Filled with manufacturer acronyms, codes, and jargon

Unstructured:

Not in a classification or taxonomy structure
Not matching your product classification structure

With a broad product catalog in the thousands and with many custom attributes and option sets, it increases the challenge of preparing that data to be in the right state for your online store and other channels.

Data transformation is essential to translate all of this data into a standardized format, classified into the taxonomy needed for your website to deliver visitors a consistent shopping experience and a site that is easy to navigate to find the products they are looking to find.

How is data transformed?

There are four main options for how to transform the product data.

Manually in spreadsheets, which is likely only possible if you have a small and straightforward product range of a few hundred products.
SQL or Python scripts to extract and transform data. This method is also a manual process requiring technical knowledge.
On-premise ETL tools (Extract, Transform, Load). These tools make the process of scripting the transformation far more efficient through automation of the process. These tools are hosted on your company sites and often require extensive expertise to run and maintain and come with a high cost.
Cloud-based transformation tools – hosted in the cloud, allows you to use the vendor’s infrastructure and expertise. These tools also allow you to automate the process to run transformations efficiently without the knowledge and a lower price tag.

What are some examples of product data transformations?

There are many different ways that data can be changed to meet your goals. Below is a fairly comprehensive list of some of the types of transformations that can be performed –

Adding: Adding in a word or numbers – for example, you may wish to add a prefix to product codes coming from suppliers to ensure all product codes are unique.

Deleting: Removing a word or space or unnecessary punctuation. For example, supplier data may include their internal product code in the product name, not relevant for your website; you could delete this.

Replacing: Replace keywords with other words. There may be a range of descriptions used for colors, such as aqua, pastel blue, cerulean, sky blue – that you would like replaced with the color descriptor – “blue” to allow for easier on-site filtering and search.

Recalculating: Completing a calculation on a number. For example, you may have a supplier price and, through transformation, wish to add a % margin to reach the RRP price to list on your website. Similarly, calculations can manage price breaks and rebates.

Moving: Moving a word, phrase, or number to a new location. For example, for a particular supplier’s data, you may wish to move the product code from the beginning of the product name to the end of the product name to align the naming structure across your range.

Sorting: Putting data in a specific order. For example, options for size may be unordered, but you want them to be in sequential order xs, s, m, l, xl, xxl on your website drop-down selector.

Expanding: Expanding acronyms and abbreviations. For example, supplier data may use the abbreviation “BLK” in the supplied product data. Via transformation, you can expand this to the consumer-friendly full word “Black.”

Replicating: You may want the same data in two sections. For example, you may want to have the dimensions included in the product name and in a product specifications section of a product detail page.

Joining: Putting two or more attributes together. For example, you might want to combine the supplier product name + color + size or weight dimensions to create the product name displayed on your website.

Filtering: You might want to filter to get a select group. For example, you could filter a group of products to identify those with a price or stock level of 0 to exclude from your site.

Photo Editing: Changing the size, resolution, or cropping of images. Helpful to automate the updating of supplier provided photos to fit the specifications of your website theme.

The data transformation process

There are four main phases in the process of data transformation. The first step is essential only for the initial set-up. Once established, the transformations can run automatically on-going with little input needed –

1. Data interpretationThe initial analysis is a critical step in understanding the data you have and its current state, and exactly how you wish it to be after transformation.
2. Pre- translation data quality checkCheck for errors in the data, missing attributes, etc. that could lead to mistakes during the transformation process.
3. Data translationTransforming the data from one state to another.
4. Data quality checkA check that the transformation has been successful.

Steps 2, 3, and 4 are completed automatically by the transformation tools with cloud-based transformation tools like Vesta eCommerce.

How much technical knowledge do I need to run product data transformations?

To code SQL scripts yourself or use on-premise ETL tools, a high level of technical expertise is needed. However, cloud-based platforms like Vesta eCommerce make data transformation easy for mid-sized businesses and no technical knowledge or coding is needed. You provide the expertise in your products and industry, and the Vesta team provides the data expertise to get the automated systems up and running for you.

Get Vesta to do it for you

Automated product transformation tools are needed for eCommerce businesses to scale their operations to manage the sourcing and cleaning of large product catalogs.

Using data transformation tools offers the benefits of organizing many diverse supplier data feeds into a normalized, consistent format with the right form and product classification to integrate with your PIM or eCommerce platform.

If you are ready to take the next step in scaling your eCommerce, broadening your range and improving your product content, then book a call with Vesta for a free consultation.