What is semantic analysis? In short, it is the Machine Learning techniques for text mining.
Humans are able to understand the implied or practical meaning of the text content efficiently and almost unconsciously. We can filter the context surrounding a word, phase, object or scenario and pull out relevant pieces. By comparing these content components with our past experiences, we can deepen our understanding of the content at hand.
Machines were not very good at understanding content in the past. They lack of the filter to determine relevant. As the advancement of Machine Intelligence and Natural Language Processing (NLP), machines are getting so much better at this task through advanced semantic analysis algorithms, more powerful computers, and a lot of practice.
In a report published in December 2015, Gartner analysts distinguish four categories of stakeholders:
- Generalist analytics vendors – these large companies offer data mining tool including textual analysis; analysis of the text is only one part of their offering and their solutions; the solutions are highly technical, requiring significant cost deployment and integration. Example: IBM, Oracle …
- Text analytics workbench providers – these players offer technical components dedicated to language analysis. They do not sell software, but algorithms that can be integrated into third-party solutions. Example: Lexalytics, Monkeylearn …
- Component providers – these vendors have developed turnkey plugins to add features of textual analysis to existing software. Example: Bitext plugin for Salesforce.
- Specialist solution – The actors of this last category addressed a need specific trade; analysis of the text is not the heart of their offer, but a tool to get a result. Example: Dictanova.
How to choose semantic analysis software?
Semantic analysis solutions can meet the needs for many use cases. It is important to think carefully about your needs before starting to evaluate solutions.
For example, semantic analysis can be used for many purposes:
- Email classification for automatic routing to the appropriate service;
- Agent conversation;
- “Smart” FAQ with automatic proposal responses;
- Interactions between customers and agents analysis;
- Predictive analysis; and
- Reputation analysis of a brand in social networks (social analytics).
Some generic solutions can potentially meet all of these use cases, but at a very high cost. In fact, if you choose to use this type of player, you should ensure that you have in internal resources and adequate skills. A specialized solution dedicated to a use case, will often be easier to implement and integrate.
Key questions to ask before choosing a solution:
- What is my budget?
- What human resources and skills do I have?
- What period of implementation of the project?
- What is the use case that fits my needs?
- What are the sources of data that are important to me?
- What languages do I need to analyze?
Seth Grimes, founder of Alta Plana Corporation, and an international specialist in language processing, summarized that:
“Go with your business objectives. Identify what indicators, what teachings, what support you need. Nobody needs a reliability rate of 98.7% on the analysis of opinion, in 48 languages, in a dozen industries. Be reasonable: the drop-UTRA detailed checklists that evaluate the features you will never use. Differentiate most of the optional, and set aside what you do not need. Then Evaluate the solutions that meet your needs – including by testing (proof of concept), if possible – to confirm, in your short list, what solution can transform your data into meaningful lessons for your business, in agreement with your budget and your requirements in terms of performance ” .
In summary, it is better to respond to a type of need, and do it well, rather than do everything poorly.