Retail in real-time: Getting ahead with data analytics

The most successful retailers generally have three core attributes:

1. an entrepreneurial drive to understand cutting edge trends;
2. the ability to pivot quickly in times of cultural and economic change;
3. and the power to position themselves as the go-to brand in their marketplace.

Historically, achieving these fundamentals for success was down to taking calculated risks and the gutfeel of experienced retail executives. However, as the world increasingly shares and connects digitally, data analytics is giving retailers the competitive edge and has resulted in the explosive success of mega-brands like Amazon and, more recently, Shein.

Changing the way we shop

Mandatory lockdowns during the pandemic caused UK consumers to rapidly shift away from shopping in-store to online. Online as a proportion of all retail spend peaked at 36.2% in March 2021. Many in the industry speculated if the way we shop had changed forever?

Recent data from the Office of National Statistics indicates this is not the case, the increases we saw during the pandemic have dropped back. Online sales as a proportion of all retail spend has now stabilised at around 26% at the end of 2022. However, despite this fall, the upward trend we were seeing pre-pandemic continues with online sales up 5% since 2019.

As the long-term trajectory for internet shopping continues, so too do the increasing digital opportunities for retailers.

Consumers continue to document their behaviour online

In 2021, there were 1.21 billion monthly active users of Meta's Instagram, making up over 28% of the world's internet users and 15% of the global population. By 2025, it has been forecast that there will be 1.44 billion monthly active users of the social media platform, which would account for 31.2% of global internet users.

With these staggering user figures and the mass amount of personal data platforms like Instagram and TikTok generate, the insight that can be gleaned from consumers sharing their habits and interests online are vast. It’s never been easier to understand what customers want, how to reach them and how to adapt product lines to service customers better.

It’s no wonder then, that retailers are increasingly turning to social media as a valuable source of information about their consumers.

Social media - unstructured data

The traditional view of data analysis involves a table of pre-defined data fields; what’s known as structured data. Social media by comparison is what we call unstructured data – information that is undefined and can seem intimidating in both its variety and volume. How do you consistently analyse blocks of free text, turn photos into data visualisation or harness videos into insight that can inform business decisions?

Yet unstructured data is a valuable source of information and should not be ignored. In fact, most of the data created today is unstructured. Hundreds of hours of video are uploaded to YouTube every minute; tens of millions of tweets are sent every hour; hundreds of millions of photos are uploaded to Facebook every day. And all this data allows retailers and other brands to dive into the current narrative of their target audience. What TV programs are they watching? Where do they want to go on holiday? When do they want to go on holiday? What do they need for their holiday? Which celebrity is the current zeitgeist and what and who are they wearing? Is gardening back in-vogue? And what are the latest wellness trends being searched?

Harnessing the power of social media through the effective use of data analytics can inform every decision from buying and design, and inventory and stock allocation, right through to marketing ideation and timing for campaigns. Obviously, the quantity of data available on social media is far too large to be viewed using traditional methods, but retailers investing in utilising this valuable source will have an edge on competitors.

Retailers are already using social media to:

1. Learn in real-time what audiences want to purchase and identify trends;
2. Understand how to reach target audiences; and
3. Gain feedback on products and an understanding of how to adapt and improve products.

So, what tactics might retailers leverage to pull insight from unstructured data? The options are wide-ranging, so below we have detailed some areas where retailers can make an effective impact quickly.

Understanding sentiment

What trends might be worth investing in or abandoning? 

How do consumers feel about competitors vs you?

How are existing product lines being perceived and should they be invested in further? 

The term "sentiment analysis" might sound familiar, but what does it really mean? Sentiment analysis is digging into text to better understand the attitude of the writer. If a product is reviewed online, retailers can easily glean insight from a few comments and use their judgement to understand the general feelings of their customers. But imagine the power of applying this logic to millions of potential customers on social media platforms to gain real-time feedback from posts, comments, reviews and hashtags. Now think about all the growing digital platforms where target audiences are freely offering up their opinions: TikTok, Twitter, Instagram, Reddit, Facebook and reviews on competitor's sites to name a few.

Companies like the fashion retailer Shein, use this as one of their techniques to analyse at a rapid speed the sentiment around product line launches; understand how customers are feeling, and then make rapid decisions on whether to expand, adjust or eliminate products.

Text mining as a tool to understand sentiment

Text mining is no longer the preserve of social media platform holders or international conglomerates. Most social media platforms offer an interface where retailers and other brands can access much of their user data, including the contents of comments and tweets. Where this is unavailable, web-scraping (data extraction software) allows comments to be pulled from user reviews and harnessed by businesses. Low-code data analytics software products such as Alteryx or Power BI have sentiment analysis built in which then allows for a regular stream of reporting and data visualisation. For those with a more technical background, well established code-libraries are available for programming languages, such as Python, that enable in-depth tweaks to all stages of the data science workflow, from data extraction to dashboard creation.

In practice, this means that when using the right tools, retailers can significantly move the dial on customer insight going beyond fixed data collection from websites and stores.

Search Engine Optimisation (SEO): is TikTok the new Google?

Another form of unstructured data retailers are utilizing to gain a competitive edge revolves around search trends. Traditionally retailers have used platforms like google to understand what consumers are searching for. They then use this information to incorporate these search trends into their marketing strategy. This could be by incorporating specific terms into their website content to drive organic traffic, or by purchasing online ad space associated with these search trends.

While Google is still a great platform to analyse search data, the Millennial and especially Gen Z demographics are increasingly looking to platforms like TikTok as the new form of "Googling". Marketing teams are now leveraging spaces like TikTok to drive search engine optimisation efforts to better understand how to reach their target audiences. This data is currently freely available for retailers to access and leverage as an analytic opportunity.

Ultimately blended data is the recipe for success

Text mining for sentiment analysis or efficient use of search terms on TikTok does not replace already established forms of data analysis. Instead, harnessing unstructured data alongside traditional forms of customer data will allow for more meaningful insight than was previously available. Retailers who take advantage of unstructured data alongside existing customer data will be better able to capitalise on trends and offer targeted promotions down to an individual level. This will help drive brand engagement, improve customer experience, and increase overall sales.

The key to doing this successfully is to have a suitable structure for obtaining and storing data that comes from multiple sources. Stock levels and sales data, sentiment analysis of product reviews, lists of social media followers and text mining of hashtags – all this data and more can be captured by automated data extraction and processing technology. Once the information is stored in a central data repository it can be made accessible to data analysts and surfaced in a single, standard Business Intelligence tool for review by key stakeholders in real-time.

Cloud-based technology solutions have alleviated many of the problems that were once associated with analysis of unstructured data, and advances in machine learning, artificial intelligence and voice recognition will continue to improve these processes.

What’s next for data analytics in retail?

Computer Vision - the next generation of AI allowing computers to interpret visuals - will, in the future, allow additional context alongside hashtags and emojis on social media posts. This may be as simple as listing the products being worn in a series of un-boxing snaps on Instagram or summarising in text a product review from a YouTube video. Whilst this technology isn’t available today, it’s worth thinking that 10 years ago completing a Captcha online to prove you were human involved reading and entering text. Today, you’re more likely to be matching images. What you might not be aware of, is that these response tests, whilst validating data entry and protecting both consumers and businesses from fraud, also help train the next generation of AI. Computer Vision will allow computers to understand and interpret images and videos or as IBM put it: “If AI enables computers to think, computer vision enables them to see, observe and understand”.

For more information on harnessingdata analytics to give your business the competitive edge, contact Sarah Belsham.