Text Analysis Systems Mine Workplace Emails to Measure Staff Sentiments

Image from Pixabay.com

Have you ever been employed in a genuinely cooperative and productive environment where you looked forward each day to making your contribution to the enterprise and assisting your colleagues? Conversely, have you ever worked in a highly stressful and unsupportive atmosphere where you dreading going back there nearly every day?  Or perhaps you have found in your career that your jobs and employers were somewhere in the mid-range of this spectrum of office cultures.

For all of these good, bad or indifferent workplaces, a key question is whether any of the actions of management to engage the staff and listen to their concerns ever resulted in improved working conditions and higher levels of job satisfaction?

The answer is most often “yes”. Just having a say in, and some sense of control over, our jobs and workflows can indeed have a demonstrable impact on morale, camaraderie and the bottom line. As posited in the Hawthorne Effect, also termed the “Observer Effect”, this was first discovered during studies in the 1920’s and 1930’s when the management of a factory made improvements to the lighting and work schedules. In turn, worker satisfaction and productivity temporarily increased. This was not so much because there was more light, but rather, that the workers sensed that management was paying attention to, and then acting upon, their concerns. The workers perceived they were no longer just cogs in a machine.

Perhaps, too, the Hawthorne Effect is in some ways the workplace equivalent of the Heisenberg’s Uncertainty Principle in physics. To vastly oversimplify this slippery concept, the mere act of observing a subatomic particle can change its position.¹

Giving the processes of observation, analysis and change at the enterprise level a modern (but non-quantum) spin, is a fascinating new article in the September 2018 issue of The Atlantic entitled What Your Boss Could Learn by Reading the Whole Company’s Emails, by Frank Partnoy.  I highly recommend a click-through and full read if you have an opportunity. I will summarize and annotate it, and then, considering my own thorough lack of understanding of the basics of y=f(x), pose some of my own physics-free questions.

“Engagement” of Enron’s Emails

By Enron [Public domain], via Wikimedia Commons

Andrew Fastow was the Chief Financial Officer of Enron when the company infamously collapsed into bankruptcy in December 2001. Criminal charges were brought against some of the corporate officers, including Fastow, who went to prison for six years as a result.

After he had served his sentence he became a public speaker about his experience. At one of his presentations in Amsterdam in 2016, two men from the audience approached him. They were from KeenCorp, whose business is data analytics. Specifically, their clients hire them to analyze the email “word patterns and their context” of their employees. This is done in an effort to quantify and measure the degree of the staff’s “engagement”. The resulting numerical rating is higher when they feel more “positive and engaged”, while lower when they are unhappier and less “engaged”.

The KeenCorp representatives explained to Fastow that they had applied their software to the email archives of 150 Enron executives in an effort to determine “how key moments in the company’s tumultuous collapse” would be assessed and a rated by their software. (See also the February 26, 2016 Subway Fold post entitled The Predictive Benefits of Analyzing Employees’ Communications Networks, covering, among other things, a similar analysis of Enron’s emails.)

KeenCorp’s software found the lowest engagement score when Enron filed for bankruptcy. However, the index also took a steep dive two years earlier. This was puzzling since the news about the Enron scandal was not yet public. So, they asked Fastow if he could recall “anything unusual happening at Enron on June 28, 1999”.

Sentimental Journal

Milky Way in Mauritius, Image by Jarkko J

Today the text analytics business, like the work done by KeenCorp, is thriving. It has been long-established as the processing behind email spam filters. Now it is finding other applications including monitoring corporate reputations on social media and other sites.²

The finance industry is another growth sector, as investment banks and hedge funds scan a wide variety of information sources to locate “slight changes in language” that may point towards pending increases or decreases in share prices. Financial research providers are using artificial intelligence to mine “insights” from their own selections of news and analytical sources.

But is this technology effective?

In a paper entitled Lazy Prices, by Lauren Cohen (Harvard Business School and NBER), Christopher Malloy (Harvard Business School and NBER), and Quoc Nguyen (University of Illinois at Chicago), in a draft dated February 22, 2018, these researchers found that the share price of company, in this case NetApp in their 2010 annual report, measurably went down after the firm “subtly changes” its reporting “descriptions of certain risks”. Algorithms can detect such changes more quickly and effectively than humans. The company subsequently clarified in its 2011 annual report their “failure to comply” with reporting requirements in 2010. A highly skilled stock analyst “might have missed that phrase”, but once again its was captured by “researcher’s algorithms”.

In the hands of a “skeptical investor”, this information might well have resulted in them questioning the differences in the 2010 and 2011 annual reports and, in turn, saved him or her a great deal of money. This detection was an early signal of a looming decline in NetApp’s stock. Half a year after the 2011 report’s publication, it was reported that the Syrian government has bought the company and “used that equipment to spy on its citizen”, causing further declines.

Now text analytics is being deployed at a new target: The composition of employees’ communications. Although it has been found that workers have no expectations of privacy in their workplaces, some companies remain reluctant to do so because of privacy concerns. Thus, companies are finding it more challenging to resist the “urge to mine employee information”, especially as text analysis systems continue to improve.

Among the evolving enterprise applications are the human resources departments in assessing overall employee morale. For example, Vibe is such an app that scans through communications on Slack, a widely used enterprise platform. Vibe’s algorithm, in real-time reporting, measures the positive and negative emotions of a work team.

Finding Context

“Microscope”, image by Ryan Adams

Returning to KeenCorp, can their product actually detect any wrongdoing by applying text analysis? While they did not initially see it, the company’s system had identified a significant “inflection point” in Enron’s history on the June 28, 1999 date in question. Fastow said that was the day the board had discussed a plan called “LJM”, involving a group of questionable transactions that would mask the company’s badly under-performing assets while improving its financials. Eventually, LJM added to Enron’s demise. At that time, however, Fastow said that everyone at the company, including employees and board members, was reluctant to challenge this dubious plan.

KeenCorp currently has 15 employees and six key clients. Fastow is also one of their consultants and advisors. He also invested in the company when he saw their algorithm highlight Enron’s employees’ concerns about the LJM plan. He hopes to raise potential clients’ awareness of this product to help them avoid similar situations.

The company includes heat maps as part of its tool set to generate real-time visualizations of employee engagement. These can assist companies in “identifying problems in the workplace”. In effect, it generates a warning (maybe a warming, too), that may help to identify significant concerns. As well, it can assist companies with compliance of government rules and regulations. Yet the system “is only as good as the people using it”, and someone must step forward and take action when the heat map highlights an emerging problem.

Analyzing employees’ communications also presents the need for applying a cost/benefit analysis of privacy considerations. In certain industries such as finance, employees are well aware that their communications are being monitored and analyzed, while in other businesses this can be seen “as intrusive if not downright Big Brotherly”. Moreover, managers “have the most to fear” from text analysis systems. For instance, it can be used to assess sentiment when someone new is hired or given a promotion. Thus, companies will need to find a balance between the uses of this data and the inherent privacy concerns about its collection.

In addressing privacy concerns about data collection, KeenCorp does not “collect, store on report” info about individual employees. All individually identifying personal info is scrubbed away.

Text analysis is still in its early stages. There is no certainty yet that it may not register a false positive reading and that it will capture all emerging potential threats. Nonetheless it is expected to continue to expand and find new fields for application. Experts predict that among these new areas will be corporate legal, compliance and regulatory operations. Other possibilities include protecting against possible liabilities for “allegations of visa, fraud and harassment”.

The key takeaway from the current state of this technology is to ascertain the truth about employees’ sentiments not by snooping, but rather, “by examining how they are saying it”.

My Questions

  • “Message In a Bottle”, Image from Pixabay.com

    Should text analysis data be factored into annual reviews of officers and/or board members? If so, how can this be done and what relative weight should it be given?

  • Should employees at any or all levels and departments be given access to text analysis data? How might this potentially impact their work satisfaction and productivity?
  • Is there a direct, casual or insignificant relationship between employee sentiment data and up and/or down movements in market value? If so, how can companies elevate text analysis systems to higher uses?
  • How can text analysis be used for executive training and development? Might it also add a new dimension to case studies in business schools?
  • What does this data look like in either or both of short-term and long-term time series visualizations? Are there any additional insights to be gained by processing the heat maps into animations to show how their shape and momentum are changing over time?

 


1.  See also the May 20, 2015 Subway Fold post entitled A Legal Thriller Meets Quantum Physics: A Book Review of “Superposition” for the application of this science in a hard rocking sci-fi novel.

2These 10 Subway Fold posts cover other measurements of social media analytics, some including other applications of text analytics.

The Predictive Benefits of Analyzing Employees’ Communications Networks

Image from Pixabay

Image from Pixabay

In the wake of the destruction left by the Enron scandal and subsequent bankruptcy in the early 2000s, one of the more revelatory and instructive artifacts left behind was the massive trove of approximately 1,600,000 of the company’s corporate emails. Researchers from a variety of fields have performed all manner of extensive analyses on this “corpus” of emails as it known. Of particular interest was the structure and operations of this failed company’s communications network. That is, simply stated, extracting and examining who’s who and what’s what in this failed organization.

No other database of this type, size and depth had ever been previously available for such purposes. What the researchers have learned from this and its subsequent and significant influences in many public and private sectors was the subject of a fascinating article in MIT Technology Review posted on July 2, 2013 entitled The Immortal Life of the Enron E-mails by Jessica Lander. I highly recommend reading this.

[July 18, 2017 Update:  For a new deep and wide analysis of the Enron email database, see What the Enron E-Mails Say About Us, by Nathan Heller, in the 7/24/17 edition of The New Yorker.]

I immediately recalled this piece recently while reading a column posted on the Harvard Business Review blog on February 10, 2016 entitled What Work Email Can Reveal About Performance and Potential by Chantrelle Nielsen. This analytical processes and consulting projects it describes could be of highly practical value to all manners and sizes of organizations. I also suggest reading this in its entirety. I will summarize, annotate and pose some emoji-free questions of my own.

I believe this post will also provide a logical follow-on to the February 15, 2016 Subway Fold post entitled Establishing a Persuasive Digital Footprint for Competing in Today’s Job Market. That post covered the importance a job candidate’s digital presence before being hired while this post covers the predictive potential of an employee’s digital presence after they have become an employee and integrated themselves into an organization.

Data Generation

The author begins by focusing her attention upon the modern tools and platforms used in the workplace for people to communicate and collaborate such as Skype and Slack. More traditionally, there is email. While these modes are important, they can also be a “mixed blessing”. Careful management of these technologies can assist is determining which forms of “digital communications are productive” for both employers and their employees.

Most importantly, these systems produce huge volumes of data. As a result, some firms are developing “next generation products” containing analytical capabilities to deeply dive into these databases and the networks they support.¹,²

The author mentions that her former company, VoloMetrix, is engaged in this field and has been acquired by Microsoft. The examples and her article concern work done for the firm’s clients before it become part of MS. During this time, VoloMetrix worked for years “with executives in large enterprises” to enable them to discern patterns within employees’ digital communications.

Predicting Employee Performance

A “strong network” can be a predictive factor of an employee’s performance. For example, a software company looked at a year’s worth of anonymized employee email data across all job categories. The findings showed that:

  • The best performers were characterized by 36% larger in-house networks, when compared to average performers, where they connected “at least biweekly in small group messages”. (This criterion was used to determine “strong ties”).
  • The lower performers exhibited “6% smaller networks” when compared to average performers.

On an annual basis, the “size and strength” of employees’ networks proved to a better predictor of their performances than managers’ more traditional assessments. Thus, being “intensely engaged” in collaborating with their peers was a driver of their work performance.

This effect was likewise seen at other business-to-business sales concerns. For instance, at a software company the top 10 workers in sales were, on average, connected to 10 or more of their colleagues. Their internal networks proved to be 25% larger than the networks of low performers. When social graph data (used to visualize the structures of networks), was examined it frequently indicated that connections within a company were even more important than those outside of it.

Predicting Employee Potential

Some businesses use “engagement programs” to assist the careers of employees are seen as having high potential to become future leaders. For example, a utility company studied the networks of a few hundreds of these people. They discovered that:

  • Those people who “were often the most connected” were shown to have networks “52% larger than average”.
  • Nonetheless, there were still others within this same group having networks of “below average” dimensions.

Managers surveyed reported that the less connected workers also had “great skills or ideas”, but displayed “potentially less” extroversion³ or emotional intelligence 4 needed to become influential. Still, opportunities are available to assist these people to “gain a broader audience” with better connected “agents” who, in turn, can promote their ideas.

Furthermore, growing a large network only for its own sake is not always the optimal approach. Rather, some networks are “more effective” because of who they include. That is, if they include people who have higher degrees of influence.

Another client, a hardware company, advanced their analysis to examine the “composition and quality” of the networks assembled by their sales reps. Their findings indicated that:

  • The “involvement of certain sales roles” corresponded to a 10X increase in the size of deals with customers.
  • Some sales roles were characterized as “middlemen” and, as such, did not “clearly demonstrate” anyone’s personal leadership potential.

Synthesizing Two Approaches

As described above, two analytical approaches have emerged for examining and leveraging the insights gained from communications networks. Both can work well in conjunction with the other. First is awareness whereby business leaders:

  • Communicate the importance of building networks
  • Provide the network analytical tools
  • Maintain the “faith” that their employees will understand this message and act upon it

The second is the prediction of outcomes, most often by sales organization to determine “which deals will close”. While this currently is applied less often than the awareness approach, this situation is now changing.

The insights gained from studying communications networks which are then applied to help build better working relationships and performance, must be “used thoughtfully” while balancing human and technological factors. Moreover, for these to work properly and “make connections more meaningful and efficient”, effectively gathering sufficient data on how employees do their jobs and communicate with their peers is essential.

My Questions

  • What standards should be established to assess communication and collaboration networks? Should they be the same for all businesses and job types or varied from field to field? Should they be differentiated further from employer to employer within a field and then perhaps for every department and job title within the same firm? (For some excellent new reading on how professional networks compare in their breadth and effectiveness in different professions, I highly recommend reading another new article on The Harvard Business Review blog posted on February 19, 2016 entitled How Having an MBA vs. a Law Degree Shapes Your Network by Adina Sterling.)
  • How should “influential” members of a network be defined in a business environment? Is influencer marketing, where individuals with a significant online presence appear to have more influence upon others in their social networks and are thus given special attention by marketers, the correct model to consider?  If so, should businesses consider developing and applying the equivalent of a Klout score to their employees? (This is an online service that rates one’s relative influence across much of social media.)
  • Would it be helpful to a company’s workforce to make this data and analytics readily available to everyone on their internal network and, if so, what would be the benefits and/or drawbacks of doing so? Would access to one’s network’s shape and reach result in some unintended consequences such as pressuring workers to increase the size of their internal and external contacts?
  • Should rewards systems be piloted to see whether they can positively incentivize employees to nurture their networks? For example, for X amount of new contacts added that support a company’s goals, Y additional days off might be awarded.
  • Can network analytics be used to fairly or unfairly restrict workers with non-competition and non-disclosure clauses when they change jobs?

 


1.   Many of these 26 Subway Fold posts under the Category of Social Media also involve metrics and analytical systems for interpreting the voluminous data generated by a wide range of social media services.

2.  A thriving market exists today in enterprise search products that can index, search and unlock the valuable knowledge embedded deep within corporate email and other data platforms. Here is a list of vendors on Wikipedia.

3.  For a completely different and highly engaging analysis of the virtues of being an introvert in social and business environments, I highly recommend reading a recent bestseller entitled Quiet: The Power of Introverts in a World That Can’t Stop Talking (Broadway Books, 2013), by Susan Cain.

4.  The authoritative and highly regarded work on this subject is Emotional Intelligence: Why It Can Matter More Than IQ (Bantam Books, 2005), by Daniel Goleman.