Scenarios

Agile Data & DataOps

  • As the amount of data available for any company grows rapidly, organizations will need Data Ops.
  • It is not just an emergent discipline: it will be prevalent in 2019.
  • The trend now is using Data Ops practices&techniques all together:
    • Valuable data assets:
      • The Data Lake
      • Machine Learning and Deep Learning algorithms
      • The Data Warehouse
      • Even the insights…
    • Are not the most valuable assets derived from data…
    • But the better knowledge about:
      • How the market works
      • What the customers want
      • What the customers are going to pay for
    • This useful and actionable knowledge is the really valuable asset.
  • Fully distributed architecture, made of interconnected data pools, will substitute centralized, unified Data Lakes.
    • This means that companies will have one set of people working on data delivery, instead of many teams focused on separate parts of processing data.

Data Warehouse: it is the centralized structured data repository.

  • Structured data can be thought of as columns and rows.
  • Unstructured data (stored in the Data Lake) are images or audio recordings.
  • The Data Warehouse allow us to perform data mining, which means “discovering” new trends in the data, initially non-visible:
    • Data mining activity is carried out more than ever (the more data IoE generates, the more “mining” is needed)…
    • So Data Warehouses are not going to be replaced by Data Lakes…
    • On the contrary, Data Lakes complement Data Warehouses with non-structured data.
    • As the amount of data rises, it is inevitable to change from Data Warehouse to the cloud:
      • Which means no more “Processing Data Centers” on site.
  • The goal is for everyone in the company (developers, managers, business analysts, CXOs) to scan data quickly and make time-pressured decisions based on information that stands out
  • So this information must be available as quickly as possible, with the best quality.
  • Companies are spending huge budgets to obtain insights:
    • Data location, storage, processing, analytics…
    • Only to be used by a few people –that is, C-level— in a company with thousands of workers?
  • What is the real ROI of Agile Data?
  • It is a great idea if thousands of people can make the best possible decisions based on data.

Impact of Adjuvant Technologies

  • Automated diagnostics can help you to comply with the regulation:
    • Example: when a process is moving or storing personal data, risking GPDR non-compliance.
  • Industry 4.0 will monetize insights beyond being used in the same company, as customer, product, service and operational data may be sold to others:
    • Example: dataphones not only make payment transactions, they register where and when people spend their money…
    • They know how many people are spending money in a certain area at any moment…
    • How many companies may be interested in those insights?
  • Digital twins will be extremely important for the physical-to-digital-to- physical loop, saving costs in building, testing, avoiding mistakes…
  • As Automation and Artificial Intelligence develop and are used more widely, a question arises: who will be responsible for the decisions made by the machines?
    • The designer, the programmer, the developer of the Machine Learning algorithm, the data scientist?
    • Companies need to stablish accountability policies regarding the consequences of applying Machine Learning.
  • The functional areas most affected by the onset of management by technology will be:
    • Service Level Agreements (SLAs) management: Continuity, Capacity and Availability will change from observation to analysis and prediction.
    • Incidents (helpdesk’s first line) will be handled directly by users, with the help and guidance of automated assistants (which learn incrementally from different incidents).
    • Problem-solving will focus on validating results, instead of doing diagnostics.

Automation & AI

  • AI is embedded in any information system and tool:
    • For example: chatbots are advancing and the software is starting to be more malleable.
    • An information system doesn’t behave the same way over time, but it learns how the user interacts, adapting to him/her progressively:
      • Example: an incident management system or even our word processor.
  • Many industries will face low-qualified job cuts…
  • But new work roles and jobs will appear when we capitalize on such skills as collaboration, communication, creativity and problem solving.
  • The more technology and AI we apply, the less technical and repetitive tasks people will need to perform:
    • Automation will develop gradually, but in the next 5 years, with more AI automation, people will focus on creative rather than routine tasks.
  • People will still manage machine performances and teach them how to operate.
  • Human-led work will change: from dealing with incidents to forecasting and Analytics.
  • If you are not transparent with your clients about using AI algorithms, you may lose them:
    • They feel cheated when discovering that an algorithm, instead of a human being, is dealing with them.
    • At the same time, companies are starting to share data and use cases…
    • So they begin to unleash their AI algorithms and share them with other companies in order to find mutual benefits.
  • Phishing scams could get even worse, as hackers gather more and more data, and perform Analytics to learn how to simulate our identity.
  • Hackers start using AI like financial firms, to learn what illegal incomes are more profitable.
  • Fake news and propaganda are going to get worse, as natural language processing algorithms can write “as humans”, including jargon or light grammatical mistakes to simulate human writing.