Companies are starving for data analytics and science skills. All the surveys we’re seeing have that same message. Among the most recent, a survey by CrowdFlower finds that 83% of data scientists themselves agree there aren’t enough data scientists to go around and this is up from 79% the year before.
So, for anyone who seeks to build their skills in the data analytics and data science area, the time is ripe. But skill-building both from an employer and a professional perspective takes significant investments of money and time. Before embarking on this journey, some trends need to be considered.
How much will ultimately be automated? There are some who believe that significant portions of data science and data analytics jobs will be automated, driven in large part by a need to get around skills shortages. Machine learning, for one, is this year’s hottest technology initiative. There are there other efforts underway as well though we are only at the beginning. DARPA, the US government agency that first designed the internet, is on the case. Adi Gaskell of The Horizons Tracker recently surfaced a request for proposal from DARPA called
Data-Driven Discovery of Models (D3M), which “aims to help in the development of automated means of crossing the data skills gap and allow non-experts to develop their own complex models. They will be empowered to do this via a significant level of automation of the back-end work behind such algorithms….This capability will enable subject matter experts to create empirical models without the need for data scientists, and will increase the productivity of expert data scientists via automation.”
Automating over the data skills gap
As Gaskill puts it , D3M seeks to do “for data science what WYSIWYG editors did for web development 20 years ago and visual programming environments for coding. DARPA believe that it will be akin to allowing relative novices to behave like virtual data scientists.”
Again, this is only the beginning of the process, so it may be some time before robot data scientists have the ability to ask questions of data that need to be asked, that tie into the goals of businesses.
Until that takes off, we’re going to need people to sift through and grasp the meaning of data, make the correlations, and draw conclusions.
There are some key technical and programming skills that are important. The CrowdFlower finds the skills most in demand include SQL (56%), Hadoop (49%), python (39%), Java (36%) and R (32%).
Andrew Abramson, executive data science recruiter with Safire Partners, explored some of the key skill areas in demand in a recent post. Hard skills in demand include “Python and R,” he says, noting that “neither of these are going anywhere for a while, and staying up-to-date with the latest trends and possibilities for Python and R will be critical, notably through packages that allow for data manipulation & modeling at scale such as pySpark and SparkR ― which has greatly expanded its capabilities with Spark 2.0.”
Most Valued Skills for Data Scientists
Data scientists and industry leaders “feel that Python will be the language of cleaning data while R will be the language of doing analysis,” he adds. Open-source tools will also predominate, he adds.
Soft skills are just as important as the hard skills, and Abramson advises building proficiency in three areas: “creativity, curiosity, and communication.” These skills “are critical to measuring a data scientist’s potential, their ultimate ceiling of achievement,” he says. “Being able to speak to an example of challenging existing best practices, exploring new alternatives, or introducing new initiatives is something I would look for when gauging soft skills, in addition to taking an abstract business issue and deriving an analytical solution for it.”
The effort to seek out these skills or develop them on a personal level is worth it, the CrowdFlower survey suggests. “The most salient takeaway was how excited our respondents were about the evolution of the field,” the report’s authors state, noting that 79% were satisfied with their careers. “They cited things in their own practice, how they saw their jobs getting more interesting and less repetitive, all while expressing a real and broad enthusiasm about the value of the work in their organization.”