Theses and Main Ideas

Agile Data & DataOps

How are companies addressing the increasing demands of their business? Seeking to coordinate their Data and Analytics structures via the following initiatives:

  • Data must be delivered to those who need it when they need it, making sure it is secure.
  • Major challenge for DataOps: making development and operations groups collaborate and facilitate the feedback loop that speeds up data delivery.
  • DataOps must be focused mainly on automating data delivery, because the more human intervention there is, the more delay to make decisions based on data.
  • Introducing automation and cooperation in the data pipeline facilitates and accelerates data processes in a firm.
  • To be effective, a data team must be as unhindered as possible from the restraints of centralized governance and standardization.
  • Independent and isolated Data Pipelines and Data Pools (instead of a single huge Data Lake) contribute to a company´s adaptability to changes and openness to innovations:
  • It is easier to share and update a small Data Pool than the whole Data Lake (millions of times a day).
  • Companies need to orchestrate many Data Pipelines that necessitate connections with other Data Pipelines, sets and producers…
    • For example: pipelines from social media, providers, open data banks…
  • Machine Learning will enable projects to process even larger amounts of data, because…
    • An algorithm can learn to curate and improve data quality
    • Can also find more dataset correlations
    • And can minimize mistakes in data analysis.
  • As a result, humans save time extracting the most value from the data
  • To make more precise deductions and better decisions.
  • Establishing DataOps entails automating the development of Data Pipelines and Data Pools
  • Thus boosting applications that support the decision-making process
  • Helping teams and businesses evolve.
  • GDPR, AI-driven Analytics, Cloud Computing and related data management trends will mark the work of data professionals during 2019.

Adjuvant Technologies

2019 will be a year of transition in learning how to get the most value out of Big Data, searching for synergies with new related technologies:

  • Edge Computing can boost the collecting, processing and analyzing of data that IoT devices gather.
  • A full and consistent Edge Computing protects IoT deployment and the enterprise network
  • Because it limits the remote transmission of data, independently of whether your IoT escalates its data generation volume or not.
  • The benefits of implementing Edge Computing adequately are:
    • Lower latency and efficient resources use (uptime, yield and energy).
    • Machine Learning models react quickly to changes, improving the quality of their predictive insights.
    • Smaller investments are needed in heavy computer processing equipment or new industrial hardware.

Automation & AI

  • Companies believe that their customers prefer dealing with humans rather than with an AI algorithm
  • And customers feel deceived when they realize they deal with a hidden algorithm.
  • So companies try to hide or humanize their AI algorithms (Alexia, Siri, Cortana…).

Companies are acknowledging that:

  • Employees cannot compete with a machine in some tasks –e.g. elaborating more data or creating multiple insights– but can use it to enhance their own capabilities:
    • The synergy between human intelligence and Artificial Intelligence is the right approach, not eliminating human intelligence intervention.
  • Nowadays, using AI and ML for marketing is a “must”:
    • AI and ML allow marketers to automate, which helps them be more effective…
    • And allows them to personalize their products and services, providing unique customer journeys, at the same pace as competitors.
    • Digital marketers see AI as a threat that will replace them with a robotic counterpart more effective and less costly
    • AI has proved to be positive for the marketing industry, when it is controlled by professionals, instead of substituting them.
    • So marketing professionals should embrace AI, not fear it.
  • Automation’s impact may include “white-collar jobs”, which means not only simple or repetitive tasks:
    • Machine Learning models and Artificial Intelligence generate higher quality predictive insights, delivering greater operating efficiencies, including uptime, yield and energy savings.
  • Automation’s impact on human efficiency: robotic process automation (RPA) is being deployed in a wide range of companies:
    • For example, to capture data (instead a human typing data in a spreadsheet), curating and correlating it and sending notifications or alarms to certain stakeholders, periodically and at a high pace.
  • Condition-based data, condition monitoring and diagnostics enable industries to apply Predictive Maintenance, which uses IoT to monitor continuously the condition and performance of equipment during normal operations, thus reducing the likelihood of failures and preventing a production line from being stopped.
    • This approach is widely deployed all over the manufacturing industry.