The Growing Need to Standardize and Validate Online Education Credentials for the Job Market

"Graduation Caps", Image by John Walker

“Graduation Caps”, Image by John Walker

Near the end of The Matrix, right after Neo and Trinity have their epic battle with the agents on the rooftop , he turns to her and asks whether she “can fly that thing”, referring to a nearby helicopter. They need to do this in order proceed to rescue Morpheus. She doesn’t know how to … just quite yet. Then she takes out her mobile phone to call Apoc and ask him to quickly upload a program to her virtual self that will enable her to pilot the chopper.

The very first time I saw this groundbreaking sci-fi film, at the Regal Union Square Stadium 14 on Broadway and 13th Street in Manhattan, the audience laughed at the absurdity of this dialog. While they were utterly dazzled by the rest of the narrative and strikingly original special effects (especially the astonishing and brain-melting sequence known as “bullet time” where Neo fights and clearly proves he’s no neophyte), this was still an awkward moment because people were laughing at this otherwise captivating film.

While I doubt that anyone would still laugh at this line in today’s world of all things networked and digital, we still have not reached anywhere near the point where people can have new skills and knowledge uploaded right to our brains. Well, at least not anytime soon and, to say the least, doing so would redefine the whole notion of an “upgrade”.

Nonetheless, there has been an enormous revolution in the breadth and diversity of webwide learning platforms. These are now available to anyone anywhere anytime with online access and a desire to learn. The benefits and the potential of online education were first taken up here in a Subway Fold Post on February 15, 2015 entitled A Real Class Act: Massive Open Online Courses (MOOCs) are Changing the Learning Process. I have taken MOOCs on everything from content strategy to project management to basic programming and have learned a great deal from them.

Standards Still Lacking for Online Education Credentials

However, in today’s highly competitive economy and job market, employers are just not sure how to evaluate prospective workers when they list online courses on their resumes and discuss them at interviews. There is no standardization yet in the requirements and weighting of these credentials. This critical issue was taken up in a very timely and informative feature in the November 18, 2015 edition of The Wall Street Journal entitled Online Skills Are Hot, But Will They Land You a Job? by Lauren Weber. I will summarize and annotate it, and pose some of my own non-academic questions.

Employers are currently searching for people with latest “technical and digital skills”. As a result, there has been a significant increase in the services rendered by course providers including Udemy and Lynda.com, coding bootcamps, and MOOCs such as Coursera and edX. These online learning platforms aim to assist workers in enhancing their skills or to provide “experience they didn’t get in college”. Nonetheless, many managers still neither trust nor recognize these new providers and their course offerings.

According to Anthony Carnevale, the director of Georgetown University’s Center on Education and the Workforce, there is no central authority setting any standards for these online educational providers. Some of the job seekers who have taken these online classes are likewise frustrated by this situation.

Independent Groups Trying to Create Credential Standards

An effort to create such standards has recently been undertaken by a group of academic researchers with additional assistance from trade groups including the U.S. Chamber of Commerce Foundation. Support for this also includes a $2.25 million grant from the Lumina Foundation, whose stated goal is for 60% of Americans to gain post-high school training by 2025. This project involves creating an online registry for use by both employers and workers to research credentials. This is intended for either group to “see exactly what skills they reflect”.

The creation of this credential registry is currently being done as a joint project by George Washington University, Southern Illinois University and the American National Standards Institute. A pilot of the directory is expected to be rolled out sometime during mid-2016.

The working group plans to assure employers that an online educator’s credentials (or “badges”) are “a sign of rigorous training”, by surveying employers about the credentials held by employees in specific roles. This will be done in an effort to provide validation for particular courses and badges.

(I also searched and found a position paper entitled Connecting Credentials: Making the Case for Reforming the U.S. Credentialing System, published by The Lumina Foundation in June 2015. I highly recommend a click-through and full read of this for the clear and compelling case it makes for this project.)

Similar initiatives have also been developed by:

  • LinkedIn which is engaged in a pilot program in Phoenix and Denver. The company is canvassing area employers about the skills they are seeking and the credentials of the workers they have recently hired. Using this information, the job networking site will permit users to learn the skills they will need for a particular job and the classes and training that “recent hires in that role have had”. This service will launch in early 2016.
  • TechHire which is a new U.S. government venture launched earlier this year by the Obama administration, whose mission is to expedite training and employment opportunities “for people without traditional academic backgrounds”. It is expected to accelerate the validity of the credentials it is offering by persuading “employers to review their skill requirements” and coordinate with training providers of “nontraditional coursework” including coding boot camps and online classes.

Employer Initiative to Test Applicant’s Job-Specific Skills

Employers on their own initiatives may soon be testing job applicants’ tech and marketing skills with simulations. These could be given in conjunction with interviews. During an HR conference in 2014, a number of companies demo-ed such tests for a wide range of specific skills from “basic math to drafting legal contracts”.¹

According to Dennis Yang, the CEO of Udemy, if these gain wide acceptance, college degrees or technical certificates might no longer be relevant. Rather, for him, the two key criteria are the ability and the willingness to learn new things.

Currently, recruiters believe that badges and credentials from online education programs indicate someone’s receptivity to learning. For example, Melkeya McDuffie, the Senior Director of Talent Acquisition recently promoted an employee at Waste Management, Inc. partly because he had taken some relevant MOOCs on Coursera. She was impressed that he had taken the initiative to do so and could demonstrate his knowledge.²

My Questions

  • Would a hybrid of credential standardization and skills simulations be another viable approach? That is, could the groups involved in each of these efforts could inform, influence and shape each others’ work?
  • How would either or both of these processes be affected in jobs requiring state or federal licensing?
  • Should employees in certain jobs be somehow incentivized by their employers to take duly certified online courses in order to remain current in their fields? Should companies factor online courses taken into an employee’s annual performance review?

 


1.  See also a September 12, 2014 post on Lawyerist.com entitled The Legal Tech Audit Proves Lawyers Are Terrible at Technology, by Lisa Needham.

2.  See also an October 23, 2015 article in the Houston Chronicle entitled Waste Management Overhauls Its Recruiting by Sarah Scully, where Ms. McDuffie is also quoted several times.

IBM’s Watson is Now Data Mining TED Talks to Extract New Forms of Knowledge

"sydneytocairns_385", Image by Daniel Dimarco

“sydneytocairns_385”, Image by Daniel Dimarco

Who really benefited from the California Gold Rush of 1849? Was it the miners, only some of whom were successfully, or the merchants who sold them their equipment? Historians have differed as to the relative degree, but they largely believe it was the merchants.

Today, it seems we have somewhat of a modern analog to this in our very digital world: The gold rush of 2015 is populated by data miners and IBM is providing them with access to its innovative Watson technology in order for these contemporary prospectors to discover new forms of knowledge.

So then, what happens when Watson is deployed to sift through the thousands of incredibly original and inspiring videos of online TED Talks? Can the results be such that TED can really talk and, when processed by Watson, yield genuine knowledge with meaning and context?

Last week, the extraordinary results of this were on display at the four-day World of Watson exposition here in New York. A fascinating report on it entitled How IBM Watson Can Mine Knowledge from TED Talks by Jeffrey Coveyduc, Director, IBM Watson, and Emily McManus, Editor, TED.com was posted on the TED Blog on May 5, 2015. This was the same day that the newfangled Watson + TED system was introduced at the event. The story also includes a captivating video of a prior 2014 TED Talk by Dario Gil of IBM entitled Cognitive Systems and the Future of Expertise that came to play a critical role in launching this undertaking.

Let’s have a look and see what we can learn from the initial results. I will sum up and annotate this report, and then ask a few additional questions.

One of the key objectives of this new system is to enable users to query it in natural language. An example given in the article is “Will new innovations give me a longer life?”. Thus, users can ask questions about ideas expressed among the full database of TED talks and, for the results, view video excerpts where such ideas have been explored. Watson’s results are further accompanied by a “timeline” of related concepts contained in a particular video clip permitting users to “tunnel sideways” if they wish and explore other topics that are “contextually related”.

The rest of the article is a dialog between the project’s leaders Jeffrey Coveyduc from IBM and TED.com editor Emily McManus that took place at Watson World.  They discussed how this new idea was transformed into a “prototype” of a fresh new means to extract “insights” from within “unstructured video”.

Ms. McManus began by recounting how she had attended Mr. Dario’s TED Talk about cognitive computing. Her admiration of his presentation led her to wonder whether Watson could be applied to TED Talks’ full content whereby users would be able to pose their own questions to it in natural language. She asked Mr. Dario if this might be possible.

Mr. Coveyduc said that Mr. Dario then approached him to discuss the proposed project. They agreed that it was not just the content per se, but rather, that TED’s mission of spreading ideas was so compelling. Because one of Watson’s key objectives is to “extract knowledge” that’s meaningful to the user, it thus appeared to be “a great match”.

Ms. McManus mentioned that TED Talks maintains an application programming interface (API) to assist developers in accessing their nearly 2,000 videos and transcripts. She agreed to provide access to TED’s voluminous content to IBM. The company assembled its multidisciplinary project team in about eight weeks.

They began with no preconceptions as to where their efforts would lead. Mr. Coveyduc said they “needed the freedom to be creative”. They drew from a wide range of Watson’s existing technical services. In early iterations of their work they found that “ideas began to group themselves”. In turn, this led them to “new insights” within TED’s vast content base.

Ms. McManus recently received a call from Mr. Dario asking her to stop by his office in New York. He demo-ed the new system which had completely indexed the TED content. Moreover, he showed how it could display, according to her “a universe of concepts extracted” from the content’s core. Next, using the all important natural language capabilities to pose questions, they demonstrated how the results in the form of numerous short clips which, taken altogether, were compiling “a nuanced and complex answer to a big question”, as she described it.

Mr. Coveyduc believes this new system simplifies how users can inspect and inquire about “diverse expertise and viewpoints” expressed in video. He cited other potential areas of exploration such as broadcast journalism and online courses (also known as MOOCs*). Furthermore, the larger concept underlying this project is that Watson can distill the major “ideas and concepts” of each TED Talk and thus give users the knowledge they are seeking.

Going beyond Watson + TED’s accomplishments, he believes that video search remains quite challenging but this project demonstrates it can indeed be done. As a result, he thinks that mining such deep and wide knowledge within massive video libraries may turn into “a shared source of creativity and innovation”.

My questions are as follows:

  • What if Watson was similarly applied to the vast troves of video classes used by professionals to maintain their ongoing license certifications in, among others, law, medicine and accounting? Would new forms of potentially applicable and actionable knowledge emerge that would benefit these professionals as well as the consumers of their services? Rather than restricting Watson to processing the video classes of each profession separately, what might be the results of instead processing them together in various combinations and permutations?
  • What if Watson was configured to process the video repositories of today’s popular MOOC providers  such as Coursera or edX? The same as well for universities around the world who are putting their classes online. Their missions are more or less the same in enabling remote learning across the web in a multitude of subjects. The results could possibly hold new revelations about subjects that no one can presently discern.

Two other recent Subway Fold posts that can provide additional information, resources and questions that I suggest checking out include Artificial Intelligence Apps for Business are Approaching a Tipping Point posted on March 31, 2015, and Three New Perspectives on Whether Artificial Intelligence Threatens or Benefits the World posted on December 27, 2014.


*  See the September 18, 2014 Subway Fold post entitled A Real Class Act: Massive Open Online Courses (MOOCs) are Changing the Learning Process for the full details and some supporting links.