Working in the Big Data: The 6 Wanted Professions
Mass storage, manipulation and data analysis…what are the phenomenon and technology behind this term and the different professions (technical and marketing) that are sought around the Big Data?
The 21st century witnessed an unprecedented explosion in the volume of data: books, studies, statistics, personal data, movies…
The market today calls this data explosion as “Big Data”. To successfully exploit “Big Data”, a single computer is no longer enough, the idea is no longer to centralize the storage and processing of data on a server, but to distribute their storage and to parallelize their processing on several computers .
The Big Data solutions allow Google to respond to the 6,000,000 requests sent to it per second , Yahoo to manage the 2,115,000 emails that are sent per second in the world and Facebook to share 2.46 million content per minute.
Based on the needs we have seen in companies, current trends and job vacancies on the Big Data, we found the following six hot profiles:
Data Engineer
What is behind the term “Data Engineer” is the idea of a professional specialized in data management. In other words, it is someone who knows how to connect to multiple data sources, cross data, perform data cleanup operations, filters, joins, manage data storage in different databases, and manage various kinds of data formats.
Clearly, the data engineer is the one who has mastered data management techniques and data storage in a very efficient way: fast and accessible from everywhere, and querying databases, it can write complex queries needed to solve business requirements of reporting, calculation of indicators, and exploitation of data at reporting goals.
Given the diversity of digital data formats and the explosive volume of data, this profile is increasingly sought after.
Data Scientist
There was a lot of enthusiasm around this profile. At one time it was even described as the “5-legged sheep of the digital economy” before the market realized that it was a little exaggerated. In reality, the idea behind this profile is someone who knows how to develop statistical algorithms to anticipate the behaviour of a variable, to recommend actions to be performed, or to categorize the data in their degree of similarity.
The models that it develops are vital to the functioning of certain companies, for example in e-commerce and social networks, it is the Data Scientist that develops the recommendation algorithms that revolve behind “people you might also know”, “products you might also buy” , “pages you might also like”. In the field of banking, data scientist develops scoring models that make it possible to lend money to an individual, to invest or not to invest on a project.
This business profile is the one that is most in vogue on the market today. If you want to focus on it, then you will need to specialize in statistical learning and data models.
Growth Hacker
This is another trade that comes straight from Silicon Valley in the United States. The Growth Hacker is someone at the intersection of marketing and Big Data which uses marketing techniques to quickly and significantly accelerate growth of a company, specifically of a start-up.
Growth Hacker is basically a marketing professional, but who knows how to do software development. It uses data models, database tools (SQL), abstraction languages to create new functionalities, its purpose is not the data analysis for decisional purposes, it seeks to create new products from Big Data and as marketing people, it strives to find customers for the company’s products; but it does so by using variants of home pages, viral factors, and the massive sending of e-mails.
Growth Hacker models the hypotheses and uses data management tools to query the databases regularly. If the company has not yet fully developed its product, the Growth Hacker ensures that virality is part of the product itself; if the start-ups already has a finished product, it studies precisely the data to discover what works in the product and allows to optimize the growth. This profile is very sought after in start-ups and in companies with a flexible economic model that have the concern to reinvent themselves constantly.
Developer
The developer refers to a software developer who can use Java, Scala or any other advanced language to develop business applications. The developer knows how to write MapReduce functions in java, knows how to execute parallel work, and knows how to do distributed development, service coordination, manage fault tolerance, make a system coherent and can even think about future improvements. This is one of the few profiles to be able to work on practically all categories of Big Data tools since the work intervenes upstream of them. This profile is also highly sought after, and especially now that companies are using Big Data for evaluation purposes.
Administrator
The administrator refers to a competency profile related to the administration of Big Data tools. Concretely, the administration consists of the tasks of setting up computers on which Big Data tools are installed (connection of computers, configuration, and installation of the operating system), installation and configuration of Big Data tools, new computers, managing failures (removing failed computers and replacing them with new ones, provisioning resources and resizing.) The administration also consists of managing security aspects, assigning permissions and permission levels to different administrator know how to use the Big Data administration tools. Obviously, no need to tell you that this profile is also a very sought after profile and that in each case requiring Big Data, there is always need of at least one administrator. If you would like to move to this profile, we recommend that you specialize in the areas of computer network, computer security and database administration.
Architect
This functional profile is more functional than technical and refers, on the one hand, to the ability to decide which “bricks” are needed to solve a specific problem, and on the other hand to the ability to integrate this set with the computer architecture existing company or to modify it so that it can integrate with it.
It is more a conceptual and functional work than a technical work. Usually, architects of this type of profile work in the urbanization of information systems; they are often referred to as information systems (IS) architects or IS urban planners. Unlike IS architects who provide the mapping of the company’s information system, Big Data architect provides the cartography of the tools to be used, shows the impact this will have in the architecture of the IS of the company and works with the decision-makers to set it up. If you want to follow this profile then, we recommend you specialize in project management, project management and database development in general.
These six profiles allow you to take the Big Data turn. Once you have chosen your professional profile, you will be able to develop it by means of one or more certifications, or a training program such as a Master’s degree or a vocational training.
