Benefits and Applications of Automated Sentiment Analysis in Customer Feedback

Abstract:

With the explosion of social media, businesses have started to pay more attention to consumer feedback posted on the Internet. However, due to the volume and diversity of information available, analysing this feedback manually is a major challenge for businesses and therefore restricts their analytical capabilities.

This paper describes how businesses, particularly restaurants, can adopt a more scientific approach by using entity extraction and sentiment analysis to quantify unstructured consumer feedback in order to derive valuable insights with speed and precision.

Introduction:

The Internet comprises of millions of reviews and Tweets across many social media platforms. Trip Advisor has an estimated five hundred million reviews, and there are approximately six thousand Tweets posted every second.

Technorati estimates twelve thousand blogs are created daily, along with nearly three million new posts each day discussing consumer opinion on products and services.

These sources of consumer feedback have become instrumental in the success of businesses as they have the ability to influence potential customers. Positive reviews and Tweets can increase interest or sales in your product or services and also improve your search engines rankings (a). Equally, negative reviews can be put off customers and damage reputations.

Service industries rely heavily on online reviews to provide feedback to their business as a whole. This is mainly used to gauge consumer satisfaction, identify strengths and weaknesses, and areas of concern.

Most restaurants have a marketing team which analyse this information manually. Due to the volume and diversity of information available online, the process can be time-consuming and the analysis is often basic which only reveals obvious areas of concern.

Automating this manual process can help businesses make sense of consumer sentiment quickly and accurately. Implements solutions using entity extraction and sentiment analysis can create platforms that can be used by businesses to generate a range of valuable insights. More importantly, as these solutions are fast, accurate, and scalable, businesses can use them to generate insights for their competition in order to benchmark their performance.

Social media platforms like Trip Advisor and Yelp usually provide each restaurant with an overall score based on their reviews which may be useful to consumers to convey the overall experience. However, without reading each and every review, customers and business owners are not able to appreciate the context that led users to give that particular rating.

This project aims to deliberate how entity extraction and sentient analysis can be used to overcome the challenges of analysing social media feedback manually. There are two concepts fundamental to this project; entity extraction and sentiment analysis. A case-study will be used to illustrate the implementation and results of adopting such technology.

Several ideas, methods, and conclusions expressed in the project are derived from my own experience. I founded a start-up called Sentiment Search in 2012 that specialises in consumer insights for the restaurant and hospitality industry based on consumer feedback.

We work with several restaurants in London, and I have used these experiences heavily during the course of this project to illustrate the implementation and benefits of using entity extraction and sentiment analysis.

Sentiment Search was developed and validated with the assistance of the Old World Hospitality Group.

The Old World Hospitality Group own a range of fine-dining restaurants in the United States, United Kingdom, and India. They have a public relations department that is used to track thousands of reviews from customers on multiple social media platforms including Trip Advisor, Open Table, Facebook, and Twitter.

It was vital to get industry experts involved in the development of this start-up in order to understand the requirements and challenges faced by restaurants.

Sentiment Search aggregates reviews, Tweets, and blogs for thousands of restaurants across multiple cities in order to build a complete and comprehensive picture of consumer sentiment and trends across the industry.

The platform can be used to identify strengths and weaknesses of a restaurant, identify relationships between the various aspects of a restaurant (e.g. food, service, cost, ambiance), identify frequently mentioned comments, benchmark performance in comparison to competitors, and search for keywords or themes across all reviews across all major social media platforms.

Background:

There are three major challenges associated with the analysis of social media. This section describes the challenges and how they are governed by human limitations and capabilities. 

Volume of Information

The abundance of information available online is an extremely valuable source of consumer feedback for businesses, but at the same time this sheer volume of information is also a problem. For businesses, particularly restaurants, collecting, fileting and analysing this information is predominantly a manual process. Therefore, performing this process within a within a cost-effective time frame is very challenging.

For example, a popular coffee shop chain may have hundreds of outlets all over the United Kingdom. Manually monitoring and analysing social media feedback for every outlet on every social media platform would be tedious, time-consuming, and bound to lead to human error. The business may also wish to track social media feedback of its competitors in order to benchmark its performance, however due to the manual intensity of the process, it is highly impractical, if not unachievable.

The average human takes one minute to read two hundred words. Assuming there is an average of fifty words in a review (a conservative assumption), it would take a human approximately two hundred and forty years to read all the reviews on Trip Advisor alone!

Due to practical reasons, businesses typically focus on analysing reviews and Tweets relating solely to their own business. This is only a mere fraction of the all the available information. This limits the analytical potential of a business and deprives them of a potential competitive edge.

Other drawbacks of manually analysing review and Tweets include inconsistencies owing to human error. This may arise due to fatigue or variations in interpretation from person to person. It is worth mentioning that the average attention span for a human is approximately twenty minutes.

Given the human limitations and volume of information available, this highlights the need for faster, more efficient processed such as automated quantification.

Fragmentation and Diversity of Information

To make this process more complex, there are many platforms that businesses need to track in order to get a “complete picture” of their online reputation.

Restaurants will typically track consumer feedback on Trip Advisor, Open Table, Yelp, Yell, Time Out, Twitter, Instagram, and Facebook.

Fragmentation creates problems wherein data has to be collected and grouped together from different sources before it may effectively be analysed by a business. Hence, the decision-making process may be delayed depending on how fast the unstructured data can be aggregated and analysed.

Another problem that arises due to the diversity of information is the challenge of dealing with social media feedback in multiple languages. Most businesses have teams that are dedicated to monitoring and analysing social media feedback manually. These teams have to translate each review in a foreign language into a universal language such as English before making any decisions. This adds yet another step to the overall process. 

Relevance and Quality of Feedback

Social media feedback comprises of a lot of information that is not relevant to a business, otherwise known as “noise”.

Gundecha and Liu (2012) mention that social media discussions tend to wander off topic and data collected may contain other content and not necessarily linked or critical to the business.

For example, if a restaurant review states “I visited the restaurant for dinner last week. The food was horrible.” The business will primarily be concerned with the comment “The food was horrible”. The comment “I visited the restaurant for dinner last week” does not offer much value. This is a classic example of how businesses have to sift through reviews to extract relevant comments in order to make objective conclusions about their service. This is a very time-consuming and inefficient process.

Zeng et al. (2010) provide the example of a business expecting feedback relating to a new product it has recently launched. However, such feedback is usually embedded amongst information on other products and services offered by the company. Therefore, to analyse the data for their new launch, manual tracking and locating has to be undertaken.

The quality of feedback is also essential as it helps businesses to make informed decisions regarding the strategies to implement. However social media feedback can often be false or bias. Mikhail (2017) argues that reviews and Tweets tend to come from consumers who have already formed an opinion about the product or service. Therefore, feedback generated tends to be polarised. Therefore, the analysis of such data is complex as the business has to find a way to appropriately interpret the data in order to make well-informed decisions. This can only be achieved using automated approaches. 

Current Analytical Capabilities

The manual dependency of the process severely limits the potential of analysis that could be performed. As a result, the level of analysis exercised by businesses is very primitive and potentially critical insights are not known to the business.

For example, in order for a restaurant to plot the average monthly consumer sentiment over the last year, the restaurant will have to read every review on every platform, score individual comments as positive or negative and finally group them on a monthly basis. This process can take a few days if not weeks to perform manually. If a restaurant wants to plot the same data for every category (food, service, ambiance, cost), the workload will increase exponentially.

After engaging with several restaurants, it is clear that reviews and Tweets are mainly used for daily feedback to the restaurant managers. The aim of these reports is simply to highlight obvious strengths and weaknesses. For example, “service was excellent”, “waiter was rude”, “air conditioning was too high”. The information is not collected and analysed over several weeks in order to identify developing patterns. Businesses are usually reluctant to analyse retrospective data due to staff costs restraints.

These limitations of can deprive businesses of insights that could potentially be critical to their success. 

The Need for New Analytical Methods

From the foregoing discussion, it may be inferred that there is a need for new analytics methods to be developed in order to help business organisations deal effectively with the challenges and limitations of unstructured social media feedback.

For organisations to carry out evidence-based decision-making it is critical for information to be sorted and presented meaningfully and comprehensibly. However, the correct tools and processed are needed to facilitate insight extraction from unstructured data. Hobbs (2014) notes that over 85% of large organisations are creating unstructured data at a faster rate than ever seen before, and it is therefore becoming critically important to mine the content in an efficient manner for the value it possesses.

The following section describes a new framework for quantifying unstructured consumer data which will involves the adoption of entity extraction and sentiment analysis.

Method and Technology

There are two concepts fundamental to this project; entity extraction and sentiment analysis. There are several algorithms and artificial intelligent concepts that can be implemented to achieve entity extraction and sentiment analysis, however the algorithm selection criteria for this project was based on ease of use and configuration complexity.

Entity Extraction is a discipline in artificial intelligence that aims to classify named entities into pre-defined categories. These categories could be the names of people, organisations, locations etc. In other words, it refers to identifying the subject of a piece of text. See the following example:

An entity extraction algorithm will be able to identify service and food as the mentioned entities. This automated mechanism of identifying subjects of sentences can be extremely valuable to businesses attempting to process large volumes of social media feedback. Using this method, data can be grouped into various categories which can be used to gain insight about particular aspects of a business.

There are several entity extraction algorithms that can be used to solve this problem. For the purpose of this project we have used a simple dictionary based entity recognition approach. In this approach, the user creates a list of categories and subcategories that the algorithm will use to compare each word against in order to tag is a recognised entity.

The following example illustrates how the process works:

https://nlp.stanford.edu/IR-book/html/htmledition/tokenization-1.html

Stage 1 –  Tokenization – This is the process of splitting a piece of raw text into individual words or “tokens”. These tokens serve as the building blocks for entity extraction and sentiment analysis. A simple strategy is to split the sentence at every whitespace character as illustrated below:

Once the text is tokenised, the individual tokens are compared to a predefined list of words in order to label the token as entity or non-entity. This process has been described in further detail in the next stage.

There are more “intelligent” strategies that split a piece of text at punctuations and whitespaces in order to add a layer of accuracy however that too has its limitations.

Stage 2: Categorisation – This step involves tagging each token with a label that will identify it as an entity or non-entity. A dictionary based categorisation approach has been adopted for this project. In this approach, each “token” is compared to a list of predefined categories in order to determine if the “token” is a recognised entity.

The following table states the predefined categories used for entity extraction:

Parent Word	Related Words
Food	Dish, Meal, Breakfast, Lunch, Dinner
Service	Waiter, Waitress, Staff, Manager, Team
Ambiance	Atmosphere, Environment, Place
Cost	Price, Money
Menu	Cuisine, A La Carte
Beverages	Alcohol, Drinks, Coffee
Music	Songs
Cleanliness	Hygiene

This is a quick approach to entity extraction; however, it is also a very binary approach.

Consider the following example:

The token “servcie” actually refers to “service”. However, the token in its current form will not be recognised as an entity due to the spelling error. Considering the language used on social media tends to contain grammatical errors, such approaches may be considered rigid. In order to compensate for this “rigidity”, regular expressions can be used.

https://www.princeton.edu/~mlovett/reference/Regular-Expressions.pdf

Regular Expressions – Princeton defines a regular expression (regex for short) as a special text string for describing a search pattern. This concept can play a very powerful role in entity extraction.

Consider the following examples:

Each of the above sentences follows a very specific pattern;

A simple regular expression was implemented that functions as a filtering mechanism to capture tokens that satisfy this rule. For example, the following regular expression captures the entity mentioned in each sentence in the figure above.

This approach can be used in addition to dictionaries in order to identify unknown entities or misspelled entities. in order to overcome the obvious limitations of dictionary based entity extraction.

Stop words were also used in order to eliminate irrelevant words. Examples of stop words include, “the”, “it”, “to” “is”. These words are irrelevant and can be ignored when it comes to entity extraction.

Sentiment Analysis,

Sentiment analysis, also called opinion mining, is the field of study that analyses people’s opinions, sentiment, attitudes and emotions towards particular entities. See the following example:

A sentiment analysis algorithm will be able to identify “excellent” as a positive sentiment and “cold” as a negative sentiment. This automated mechanism of identifying emotion can be extremely valuable for businesses attempting to evaluate consumer satisfaction from social media feedback.

There are several concepts and algorithms that have been developed in order to determine the sentiment of a piece of text. For the purpose of this project we have used the Naïve Bayes classification algorithm.

Naïve Bayes Classification

This algorithm is one of the most popular approaches used to identify the sentiment of a piece of text. It implements simple yet very effective statistical concepts. This aim of the algorithm is to determine the probability of word being positive or negative.

The Naïve Bayes Classification algorithm processes a piece of text as a bag-of-words. As the name suggests, this model is one where text is represented as an unordered set of words. Only the frequency of words is considered, not the order.

https://web.stanford.edu/~jurafsky/slp3/6.pdf

Consider the following example.

The bag-of-words representation of this piece of text would simply be the unique list of words with its corresponding frequency as follows:

This model simply represents the multiplicity of the words present in the given piece of text. This concept is central to the Naïve Bayes classifier.

A typical Naïve Bayes classifier must first be trained using tagged datasets in order for the algorithm to calculate the probability of a word being positive or negative.

A standard dataset would contain thousands of individual sentences, each tagged “positive” or negative by a human. See the following example:

In simple terms, once the algorithm has been trained using such data, it creates a list of probabilities for every word appearing in a positive or negative sentence. Let’s assume the training data consists of 100 sentences that consist of the word “excellent”. Let’s assume that 90 sentences have been marked positive and 10 sentences have been marked negative either due to human error or perhaps because the sentence contained a strong negative connotation defined by another word.

The probability of “excellent” being a positive word, in this case is 90%. Therefore, when new text containing the word “excellent” is processed, the algorithm can confidently determine that the probability of the text being positive is 90%.

The Naïve Bayes classifier was trained using 500,000 Trip Advisor reviews during the course of this project. 250,000 one-star and two-star reviews were used as negative training data. 250,000 four-star and five-star reviews were used as positive training data. Three-star reviews were not used as they are highly polarised reviews.

The following example illustrates the results produced after processing reviews on Trip Advisor:

This table clearly illustrates the Naïve Bayes is an effective classification algorithm that can predict sentiment with a high degree of accuracy.

It is important to state that the bag-of-words approach does not take into account the sequence of words. It determines the probability of each token in isolation.

More advanced versions of the algorithm can be used calculate the probability of a sequence of words, also known as ngrams, being positive or negative. This adds a degree of accuracy as the order of words is considered unlike in the conventional Naïve Bayes classifier. However, this version was not implemented for this project.

One problem that was observed on multiple occasions is worth mentioning. For example, the word “good” is commonly written in many variations such as “goooood”. In this case, the Naïve Bayes classifier will rightly classify the “amazing” as being positive. However, it is unable to classify “goooood” as it is technically a different word that does not appear as frequently as “amazing”.

In the above table, the different variations of “good” are not mentioned enough times for the algorithm to predict the probability with confidence. The last three variations were only mentioned once, however the first two variations were mentioned thousands of times.

It is possible to put in rules in place to handle such problems, however these were not implemented for the course of this project.

Storage

Elastic Search was chosen as the storage and indexing solution for this project. Elastic Search is known for its full-text search capabilities and is an obvious choice for unstructured data such as social media feedback. In addition, it offers a range of tools that can be used to visualise the results of advanced queries.

Elastic Search is highly customizable and offers users several options for storing and indexing data. There are two fundamental types in Elastic Search; keyword, and text. The Keyword type refers to whole values. This type is intended for structured fields such as postcodes, email addresses and other tags that consist of single words. The Text type implies that the field will be tokenized and indexed. For example, if the value of a field is “the food was good”, each token will be stored separately in the index; “the”, “food”, “was”, “good”. Elastic Search has a built-in stop word list that contains a list of tokens that are ignored when implementing the Text type. Tokens like “the”, “was”, “is” etc. are usually ignored. The Text type is suited for fields that require full-text search capabilities. The advantages of full-text search have been illustrated in the Analysis section.

A custom Elastic Search index had to be created in order to optimize the storage and search speeds of the reviews. This custom index is instrumental as it enables full-text capabilities that set the stage for performing complex queries. This has been described in more detail in the “Analysis” section.

 

Review Crawl (Pre-Requisite)

A major pre-requisite to this process was downloading 1 million Trop Advisor reviews. Web crawlers were developed using Microsoft C Sharp that downloaded all reviews for the top 5000 restaurants in London. This was a technically challenging task as there was no API available to download Trip Advisor data. The relevant HTML tags had to be identified in order to download the relevant content. The data downloaded included the review title, review rating, review text, review date, review author, author location, and page URL. The following diagram illustrates a downloaded review.

Unit Tests

Given the complexity of the process and number of moving parts (review download, entity extraction, sentiment analysis, storage and index), basic unit tests were developed in Microsoft C Sharp. Despite the time and effort required to write unit tests, this step was instrumental in ensuring the functionality of the algorithms was intact after each iteration of development. The following image illustrates a simple unit test:

Project Work

The Old World Hospitality Group played an instrumental role in developing use-cases for the automated quantification of social media feedback using entity extraction and sentiment analysis. It was vital for this study to have experts from the industry in order to develop realistic, and useful use-cases.

The following section describes six major use cases that illustrate the potential of adopting such technology.

“Comment” is mentioned several times as part of the explanation for the use cases. A “comment” is a snippet of text, usually a sentence, from a review. For example, if the review mentioned “The food was excellent, but the service was poor.” This review comprises of two comments; “the food was excellent”, “but the service was poor”.

“Category” refers to an “Entity”. Entities in this case are “Food”, “Service”, ‘Ambiance”, and “Cost”.

1. Comment Counts per Category

Description – This use case involves determining the number of comments mentioned per category. For example, determining how many times a comment related to the service of the restaurant was mentioned across all the reviews.

Purpose – This capability would allow restaurants to gauge how the various categories compare against each other. The idea is that, this will reveal the dominating categories and in turn the categories that need more attention.

2. Comment Counts per Category per Month

Description – This use case involves determining the number of comments mentioned per category per month over a year. For example, determining how many comments related to the service are mentioned every month.

Purpose – This capability would allow restaurants to gauge how the various categories perform on a monthly basis in order to identify the degree of traction. For example, if a restaurant has a bad reputation in terms of service, they might change their staff. This insight could be used to identify how many customers mentioned comments related to the service before and after the change. This could also indicate whether changing the restaurant staff improved customer by comparing service by comparing the overall sentiment in reviews mentioning service, before and after the intervention.

3. Average Sentiment per Category

Description – Most social media platforms offer an overall rating for the major categories such as “Food”, “Service”, “Ambiance”, and “Cost”. However, the overall rating for other categories such as “Menu”, “Cleanliness”, “Music” is not stated.

This use case involves determining the average sentiment for a range of categories. For example, determining the average sentiment of the “menu” across all reviews.

Purpose – Restaurants may want specific customer feedback on particular aspects of their business, for example, live music or private dining. This insight would restaurants the freedom to define their own categories. This will also allow business to gauge how the various categories compare against each other. The idea is that this will reveal the category in which the restaurant is performing poorly and till need more attention.

4. Average Sentiment per Category per Month

Description – This use case involves determining the average sentiment for a range of categories per month over the year.

Purpose – Similarly, restaurants can gauge how the various categories compare against each other over a period time. The idea is that, this will reveal the dominating categories and in turn the categories that need more attention. Restaurants are able to implement changes and compare feedback before and after the changes in order to see whether they were successful in improving the customer experience.

5. Most Frequently Mentioned Comments

Description – This use case involves clustering comments that are mentioned frequently for each category across all reviews.

Purpose – Restaurants can identify comments that appear frequently for a given category. For example, “the service is poor” may be mentioned only a few times, and therefore not raise any concerns. However, with automated analysis, it may appear that “the service was poor” was only mentioned on weekends. This is actionable insight that would likely be missed by manual analysis.

 

6. Insight Comparison with Competitors

Description – This use case involves comparing and benchmarking the performance of a restaurant against its competitor. For example, determining the quantity and sentiment of comments of a particular dish served at the restaurant versus their competitor.

Purpose – This use case is the most interesting and perhaps the most significant one as it puts the other insights in context. The idea here is to allow a restaurant to compare itself to its competitors, category by category. Restaurants may discover different insights, for instance, their cocktails are more expensive, or their vegetarian dishes are not as extensive, or perhaps their waiting times for tables are significantly longer than that of their competitors. These insights can produce actionable outcomes that can help restaurants gain a competitive edge.

The entity extraction and sentiment analysis algorithms described in the previous section were used to process over 1 million reviews and Tweets for approximately five thousand restaurants in London.  The following diagram illustrates how the raw data was tagged, processed, stored.

The following image illustrates the results or “processed” comments that were stored in Elastic Search:

The following table provides a summary of the output created after processing 1 million Trip Advisor reviews.

Entity or Category	Number of Comments
Food	3.4 Million
Service	1.4 Million
Ambiance	400 Thousand
Cost	300 Thousand
Menu	200 Thousand
Beverages	500 Thousand
Music	50 Thousand
Cleanliness	40 Thousand

Total Comments	~ 6.3 Million
Total Time Taken	~ 2 Hours

As a result, a fully categorised, quantified, structured, and searchable data set consisting of 6.3 million comments was created in a span of two hours. This data set, in essence created a platform for performing advanced queries in order to generate potentially critical and previously unknown insights.

Analysis

This section illustrates the effectiveness of using the created data set to run advanced queries in order to solve the defined use cases.

Interactions with the business owners of the Old World Hospitality group were extremely valuable to validate this project. Business owners such as the Old World Hospitality Group are often surprised to learn how quickly machines can process these reviews in order to give grouped data. The processed data gave insights which were useful to every member of the team.

Most mentioned

Within the processed data, it is easy to derive the most mentioned words across all reviews for  a given category. Although, sentiment is not taken into account, discovering the most mentioned words is still important. For Chor Bizarre, under ambience, ‘décor’ was the most mentioned word. Overall, it had positive sentiment with people stating it was ‘quirky, ethnic and authentic’. The most mentioned word in the beverage category was found to be ‘wine’. After reading the extracted statements mentioning ‘wine’, we deduced that the wine overall had negative reviews. Complaints said the wine was expensive and of poor quality. This generated an action point for the restaurant. The manager was in discussions with the owner to find ways to improve their wine selection and perhaps reduce the menu prices.

Co-relation between topics

Once the data has been processed through sentiment analysis, it becomes very malleable. We were able to take the sentiment of multiple categories and plot those against time. You could compare large dips in sentiment, for instance, between ‘food’ and ‘ambience’ and discover that when food had a dip in sentiment, ambience had an more positive sentiment. By reading the highlighted reviews which dipped the sentiment, remarkably, we discovered that that the ambient but dim lighting was made it more difficult for customers to see their food or cutlery well.

Sentiment analysis was also useful when we used the software to evaluate a single category. During a period in November 2016, Chor Bizarre’s  (a restaurant owned by the Old World Hospitality Group) service category dipped to a lower than average sentiment of 7.3. It was later shown to peak in January 2017. After discussing this with the management team, we understood that there had been some problems and complaints with the general manager of the restaurant. In December 2016, the general manager had been replaced with a new member of staff. From the decrease in mentions of service and increase in sentiment, we could deduce that this ‘intervention’ to improve service at Chor Bizarre was a positive one. This shows that sentiment analysis can be used to look at data retrospectively too in order to understand previous problems and to hopefully avoid these issues in the future.

Moving forward, we could explore the implementation of a search engine so that the user can multiple enter keywords which can be plotted against time. For example, if the chef has changed the tasting menu by adding some palate cleanser dishes, you could plot ‘tasting menu’ to see how the sentiment has changed over time. Furthermore, you could additionally plot ‘amuse bouche’ or ‘palate cleanser’ and see whether these were related to the positive or negative sentiment changes in the tasting menu.

Competitors

Knowing who your competitor is talking to is important, but knowing what people are saying about your competitor is crucial. Using sentiment analysis, we were able to gather and sift through the online presence of a rival restaurant of Chor Bizarre. The team commented how easy it would be to watch your competitor’s business strategies and learn from their failures when you compare the data side by side to their own restaurant.

For instance, staff saw how easy it was to generate new customers from their competitor’s unhappy customers by commenting below the negative tweet or comment. Tracking your competitors is very important for restaurants who want to be one step ahead in the business. There is huge potential for restaurants to generate new leads and innovate new ideas and campaigns.

Consumers and sentiment analysis

Sentiment analysis software widely available on the market has largely been targeted at businesses and brands looking to gain feedback on their online image. One outcome of this project was that we need to start thinking about using sentiment analysis within an internet application or website which the average person could use. The information processed online regarding various restaurants generated outcomes could be both interesting and useful to consumers. For example, customers could search for the cheapest ‘crispy duck’ dish or overall highest rates ‘vegetable fried rice’ dish across multiple restaurants and get a list of options organised by sentiment. This could even extend to features apart from food, such as overall best ‘Bring Your Own Bottle’ restaurants or simply a list of bars with a ‘beer garden’. This not be difficult to implement when you have user friendly visualisations and infographics.

The insights that can be extracted from the processed data are infinite. The advantage is that these insights can be presented in different ways to appeal to business owners, consumers and marketing firms. Feedback from staff

Chor Bizarre’s manager was checking multiple restaurant review websites every 2 days for new reviews prior to implementation of the sentiment analysis software. He found the sentiment analysis software to be quick and more thorough than he could be. He had used the insights gained in 2 different ways. Firstly – on a day-day basis, where he was able to keep up to date with any swings in sentiment and in turn, new reviews from consumers. Barring any major positive or negative reviews which needed to be dealt with immediately, he was able to use insights about the quality of food, cleanliness of the bathrooms, speed of service to feedback to his staff on a weekly basis. Positive sentiment such as ‘friendly and attentive staff’ was used to praise staff and help boost morale. Similarly, very negative reviews emphasized some key issues. One negative insight highlighted that consumers felt that the wait staff were not equipped well with dealing with unsatisfied customers. If the food was cold or the wrong order was given to the customer, the customer felt that there was a lack of apology and compensation to them by the wait staff. Reading through the original reviews, we discovered that often complaints weren’t fielded straight to the manager. In turn, wait staff were retrained and a restructure is being considered in the future where one of the more experienced staff members will be promoted and will be responsible for fielding all in house customer complaints.

The business owner had little to do with the day to day running of the restaurant. His interest lay with the information the sentiment analysis software could provide about the restaurant’s competitors. We used data on one of Chor Bizarre’s competitors, Gymkhana, situated across the road, to discover some insights. We found that Gymkhana had the most mentions of its ‘lamb’ dish within the food category and these were largely positive with multiple reviews commenting that ‘the lamb was tender and excellent’. We also discovered that it’s beverage category had very positive sentiment with comments such as ‘great gin cocktails’ and ‘reasonably priced wine’. The owner will be using these insights to brainstorm new menu ideas with the chef perhaps involving a lamb dish to rival Gymkhana’s and more innovative cocktails. As discussed earlier in this paper, owners can also use sentiment analysis to make important staffing decisions.

Furthering sentiment analysis

Sentiment analysis creates a platform for real-time parallel technologies to develop. One example is social listening, where one can monitor online conversations in order to gather information about a product or brand. This is important because it can help businesses monitor for potential crises. If a stream of restaurant –related tweets occur, we can filter tweets by sentiment and accordingly respond to negative tweets quicker. This helps communication move more smoothly which is relevant when businesses are working other third parties such as PR and marketing firms.

The information processed by the software can also be presented in many different ways. Using graphs and other visual data, the information can be displayed to business owners in a more user-friendly way. This can make it more appealing for people from non-technical backgrounds to engage with sentiment analysis.

Advantages

ASA has accuracy levels of 70-80%, comparable to what you would expect from a human analyst (c). Automation is fast and close to real-time. This is a particular advantage to restaurants which deal with customer service as they can be alerted when there is an increase in chatter online about a particular dish or  large changes in sentiment occur when a complain goes viral.

ASA can also be more cost-effective in processing the same amount of data when compared with human analysts. Focussing on restaurant reviews, ASA can also allows quicker insight into consumer data and therefore with the ability to also act on this insight faster too. Most ASA software is also easy to implement and requires little training. General Limitations

In order to maintain statistically accurate data, it is important to mine a large sample. If someone having a bad day has written a particularly negative review, a large sample can ensure that such extreme comments are diluted so that the overall sentiment is preserved.

Limitations

Developing Discipline

If we look at the database industry, there are a range of established, off the shelf solutions that can be implemented to satisfy a list of given requirements; RedShift (a scalable cloud-based RDBMS), MySql (a standard RDBMS), Elastic Search (a comprehensive full-text search solution).

Unstructured data analysis is an evolving discipline that is not yet fully established. There are several approaches, concepts, and solutions available that offer various levels of functionality, however none of them can be considered a “go-to” solution.

There are a number of sentiment analysis algorithms available; Naïve Bayes, Support Vector Machines, Neural Networks, Maximum Entropy. Each approach has its strengths and weaknesses. Each algorithm requires specific skills and thorough algorithm-specific knowledge in order to perform a successful implementation. In addition, the data required to train each algorithm also varies depending on the level accuracy required. Therefore, implementing sentiment analysis can be overwhelming for a business with little experience in this domain.

The same problem applies to entity extraction; there are a number of algorithms that can be used to identify the entity of a sentence. For example, rule-base systems developed by linguistic experts, statistical models, and machine learning algorithms. Again, algorithm-specific knowledge is required to develop a working solution.

Therefore, it is clear that there is no one-size-fits-all solution. This lack of establishment and specialised skill set requirement can deter businesses from investing time and money into building an entity extraction and sentiment analysis solution.

As suggested by Barbier and Liu (2011), the reason why successful management of unstructured data is proving difficult is due to a lack of appropriate tools, difficulties in integrating unstructured data with existing information systems, and shortage of skills.

Training Data

The quality of training data is instrumental to sentiment analysis algorithms. ‘Noisy’ data can have a major negative bearing on the accuracy of these algorithms. As mentioned previously, for the course of this project, four-star and five-star reviews were used as positive training data. One-star and two-star reviews were used a s negative training data. Ideally businesses should train their algorithms with individual sentences rather than whole reviews. Using individual sentences enables the algorithm to determine with better granularity which words are statistically positive or negative.

Using a training data set of one million sentences labelled as positive or negative will produce better results than using one million reviews labelled as positive or negative. However, creating such a data set is significantly more expensive and time consuming as it involves humans manually labelling one million sentences as positive or negative.

Labelled training data is available for businesses to buy on the Internet, however to achieve the most accurate results, there is no substitute for creating training data sets that are tailored to the business’s industry.

Language Specific Limitations

Human language is complex and words can often have different meanings. For example, the word ‘wicked’ which originally painted a picture of something bad or terrible is now used to describe something good or excellent. When words undergo semantic changes such as these, it can be difficult for sentiment analysis algorithms to determine the sentiment accurately. Similarly, sarcastic and ironic comments are difficult for a machine to process in isolation; for example, ‘Dear restaurant, you have the friendliest staff and the lowest prices. You suck!’ This sentence is another example of where sentiment analysis may fail to extract the correct sentiment of the review as it is unable to identify the co-relationship between sentences.

With comparative and complex statements, algorithms find it difficult to process, whether the piece of text on the whole is neutral, positive or negative.

Machines are also limited by their ability to process shorthand and abbreviations ‘’lol’ shorthand for ‘laugh out loud’, ‘atm’ is shorthand ‘at the moment’. New abbreviations are frequently created on social media. As a result, sentiment analysis algorithms would have to be trained regularly in order to achieve the best possible results.

Another example of language complexity involves identifying the context the of a word. For example, the word “heavy” may be considered negative in a restaurant review; “The waiter was thick”. However, the same word may be considered positive in the following sentence; “the gravy was nice and heavy”. These are domain-specific nuances that must be incorporated into the selected sentiment analysis algorithm.

Language complexities possibly pose the biggest technical challenge for entity extraction and sentiment analysis algorithms.

Other Technical Limitations

Sentiment analysis algorithms have ‘acceptable’ accuracy when it comes to analysing individual sentences. However, determining the overall sentiment of paragraph or a large piece of text is not as dependable. Therefore, such algorithms cannot make accurate judgments as to weather a business should or should not be concerned with a customer’s feedback. For example, consider the following review; “the food was excellent, the cocktails were fantastic, and the waiters were very warm and attentive. However, we discovered a dead rat opposite us.” A sentiment analysis may tag this piece of text as positive overall. However, a human reader would be able to judge that “a dead rat” is unacceptable regardless of the positive feedback. Therefore, this review should actually be labelled as “negative”.

The inability to determine the overall sentiment of a piece of text is another major limitation of entity extraction and sentiment analysis algorithms.

Foreign Language Translation

A major drawback of using automated approaches to analyse social media feedback is the inability of computers to accurately translate text from a foreign language to English without losing the cultural nuances and details. There are several APIs available for language translation, however the accuracy of these services is not always acceptable. Consider the following sentence written in Hindi, “हाथों के तोते उड़ जाना”. Google translates this sentence into its literal meaning; “The parrots of my hands flew away”. However, the metaphorical meaning of this piece of sentence means “to get worried”. This “irrelevant” translation would result in the sentiment analysis algorithm determining the sentiment incorrectly.

Furthermore, if the API being used does not support a particular language, there is no choice but to discard that piece of text.

Technology Infrastructure

A comprehensive infrastructure is required to build and run a social media sentiment analysis solution. A typical solution would require the development of web crawlers, configuration of databases and indexing engines, and finally a dashboard to analyse the data. Such solutions are expensive and time-consuming to develop. There are range of off-the-shelf solutions available on the market, however these vary in capabilities, accuracy, and often cannot be used to answer the complex questions restaurants ask.  From the conversations with Old World Hospitality Group, it is clear that the restaurants would prefer to have access

Human Replacement

As mentioned earlier, restaurants usually have dedicated teams responsible for social media analysis. A significant chunk of the work performed by these teams can be replaced by automated software solutions. Even though this is beneficial for businesses in the long term, replacing human labour with automated alternatives has always been a controversial subject.

Conclusion

Automated sentiment analysis is the future. Some industries may still require human analysts to process the data which has been filtered by machines. However, this project demonstrates that sentiment analysis alone can be used successfully within the restaurant industry to produce valuable insights which are actionable.The “success” of adopting entity extraction and sentiment analysis can only be determined once it has been implemented. This is because the success primarily depends on the feedback what insights are hidden. It is possible the businesses discover insights that change their understanding of their consumer perception. It is also possible the businesses do not discover any new insights. Therefore, it is difficult to say whether the adoption will guarantee business changing insights. However, the adoption will certainly, future proof your consumer analysis infrastructure and enable the business to identify insights in the future that could…

Serves as a platform for a host of new tools like real-time monitoring and alerts.