The Smart Data Blog

Riding The Competitive Intelligence Automation Wave - Part 2

Posted by Partha Sarathi Bhattacharjee on Mar 25, 2017 1:44:00 PM

Find me on:

"The legal and ethical collection and analysis of information regarding the capabilities, vulnerabilities, and intentions of business competitors" - Strategic and Competitive Intelligence Professionals

This is how Strategic and Competitive Intelligence Professionals (SCIP), the most well-known global body on Competitive Intelligence (CI), defines the concept of CI. For the uninitiated, the page on the organization's Code of Ethics for CI professionals sheds lights on the innards of the practice. One does not have to belabor the importance of the practice in the context of successfully operating a business. It is in the operational details, as is the case with most details, that the devil lies.

hawaii-181082_640.jpgFrom an operational standpoint, CI professionals can be classified as either employees of a company or that of a specialized service provider. In case of the former, the practice team can either be housed within a larger department or, infrequently, have an independent existence. The service sector has burgeoned over the last decade or so with the rising tide of Knowledge Process Outsourcing (KPO). Both these sets of professionals are increasingly being impacted by automation, to an extent, in profoundly different ways.

The in-house CI practitioners are increasingly interfacing with IT departments as well as analytical tools given the augmentation of data and the need for infusion of technology for attendant analysis. Such interaction, through facilitating pronounced value generation, fosters increased time to value. On the flip side, the limitations of the traditional approach of manual data curation render it difficult for the small-ish CI teams to effectively meet time-sensitive business needs. The gamut of challenges that the specialist offshoring industry faces is of a different hue altogether.

The recent statement of Capgemini India's CEO that a large portion of India's IT industry is not equipped to deal with technology evolution made it to the headlines. Whether or not the prediction manifests itself remains to be seen. What is undeniable, though, is that the CI offshoring space as well is in the eye of the perfect storm of technological disruption. Conversation after conversation with senior management of pharmaceutical and technology companies that I have been privy to recently have centered on the themes of process automation, accelerating time to value, and of course, limiting costs. The advent of solutions anchored to advances in a suite of technologies ranging from semantic enrichment and natural language processing to new age storage and databases pose formidable contest to the traditional CI approaches. The competition can be explained by the symmetry between technology offering and the nature of the practice of CI.

An absorbing piece in The New Yorker reported studies by MIT economists that characterized jobs as functions of cognition and routine. The simplistic illustration below categorizes occupations in a matrix using the parameters.

The framework is anchored to what is known as the "task approach" to labor markets. For those with an academic inclination, a deeper understanding of the subject can be gleaned from this paper. Given that academicians are exceedingly polite folks in general, the term 'Routine' is not to be perceived as a derogatory remark. As explained here, routine tasks mean those in which "computer capital substitutes for workers in carrying out a limited and well-defined set of cognitive and manual activities, those that can be accomplished by following explicit rules". On the face of it, CI does not appear to be such a profession. A CI analyst needs subject matter understanding and people skills, aspects that computers are not particularly good at presently. In a nutshell, CI appears to be a profession that requires human intelligence and ingenuity for one to excel. I would go out on a limb and suggest that such a perception may not be wholly accurate.

My mild skepticism is borne out of mapping individual CI tasks to the aforementioned matrix as shown below.

The conclusion that jumped out on me from the mapping is that most of the standard CI tasks can be broken down into rules that computers can perform as well as, if not better than, humans. Advances in natural language processing, for instance, when coupled with semantic technologies can do a much better job than a person in performing secondary research on large document sets. Several analytical tools have achieved near-automation of data analysis with click-and-view options. The abstraction of data science concepts such that business users primarily interact with analytical output instead of the mathematical underbelly will continue to be a reinforcing trend. Similarly, large sections of reports can be automated in an age where algorithms write news articles. The technological progress is acting as a forcing function for proactive CI practitioners and service providers to reassess their value proposition. The others, unfortunately, will either not be around or serve as pale shadows of themselves in a few years from now. So how should a CI practitioner adapt to the changing scenario? Some would say the answer lies in a notion from centuries ago.

As lucidly described in this article, present day professionals need to revert to the notion of artisans, "people who made things or provided services with a distinctive touch in which they took personal pride" before mass production became the norm. Application of mental faculties by a well-informed CI professional with social and interdisciplinary skills across subject matter, technology, and data is invaluable. A "new artisan" CI practitioner will be an expert who is meaningfully engaging a suite of technologies in pursuit of answers to complex business questions. She will not invest three-quarters of her time in curating data and information from a plethora of sources. Instead, she will be equipped to configure and apply a combination of tools to drill down to information of interest in an expedited manner. She will apply her subject matter expertise to discover knowledge and extract insights by meaningfully interacting with large swathes of interconnected data. Also, she will utilize the time freed from routine tasks for primary research and feeding the new data into said interconnected data ecosystem. She will ride the automation wave, quite high and dry.

To learn more about Semantic Technology's impact on CI, watch our on-demand webinar "CI Informatics: a Deeper, More Comprehensive Approach to Competitive Intelligence Using Semantic Technology".

Watch the Webinar

This blogpost was originally posted at LinkedIn.

Topics: Data Management, Big Data, Analytics, Competitive Intelligence