The words behind the work, in plain English.

The fifth generation of cellular network technology, offering much higher bandwidth and lower latency than 4G and supporting far more connected devices per area. For data teams it makes real-time streaming from sensors, vehicles and mobile apps practical, which feeds edge analytics and IoT use cases.

A controlled experiment that randomly splits users between two or more versions of something, such as a web page, email or pricing offer, and compares a chosen metric to see which performs better. Done properly, it replaces opinion with evidence and shows whether a difference is real or just noise.

A guarantee that a database handles each transaction as atomic (all steps succeed or none do), consistent (rules are never broken), isolated (concurrent transactions do not interfere) and durable (committed changes survive crashes). It is essential for systems such as banking, orders and inventory, where a half-finished update would corrupt records.

Analysis that is specific, timely and relevant enough for someone to make a decision or take a concrete step based on it. A report full of interesting numbers is not actionable unless it tells the reader what has changed, why it matters and what option to choose next.

Ready-made artificial intelligence capabilities, such as language models, vision, speech or forecasting, offered by a cloud provider and consumed through an API or subscription. It lets organizations add AI features without training models or running GPU infrastructure themselves, though they must still manage cost, data privacy and vendor lock-in.

An AI system, usually built on a large language model, that can plan multi-step work and take actions through tools, such as searching data, calling APIs or updating records, toward a goal with limited human input. Agents need clear permissions, monitoring and human checkpoints, because their mistakes become actions rather than just wrong answers.

The policies, roles and controls an organization uses to make sure its AI systems are safe, fair, legal and aligned with business goals. It covers model inventories, risk assessments, testing, human oversight, monitoring and compliance with rules such as the EU AI Act.

A precise, step-by-step set of instructions a computer follows to solve a problem or complete a task, such as sorting records, finding a route or training a model. In machine learning the algorithm is the learning method, while the model is what it produces after being trained on data.

Amazon's cloud computing platform, the largest by market share, offering hundreds of on-demand services for compute, storage, databases, analytics, machine learning and generative AI. Companies use it to rent infrastructure by the hour instead of buying servers, scaling capacity up or down as demand changes.

Environments filled with sensors and connected devices that sense people's presence and needs and respond automatically without explicit commands. Examples include lighting that adjusts to occupancy, hospital rooms that monitor patients passively, and retail spaces that track foot traffic. It raises important questions about consent and privacy.

Historical data that has been collected, cleaned and organized to answer questions about trends, performance and behavior, rather than to run day-to-day transactions. It usually lives in a data warehouse or lakehouse and is shaped for reporting, forecasting and machine learning.

An open-source framework for storing and processing very large datasets across clusters of ordinary servers, built around the HDFS file system, the YARN resource manager and the MapReduce processing model. It popularized big data in the 2010s; many teams have since moved to cloud object storage and Spark.

An open-source engine for processing large datasets in parallel across a cluster, keeping data in memory where possible so jobs run much faster than disk-based MapReduce. It supports SQL, streaming, machine learning and graph workloads from Python, Scala, Java or R, and underpins platforms such as Databricks.

Spark's built-in machine learning library, providing distributed versions of common algorithms for classification, regression, clustering and recommendation, plus tools for feature preparation and pipelines. It is useful when training data is too large for a single machine and already lives in a Spark environment.

The process of planning, designing, building, testing and maintaining software that meets a business need, whether a mobile app, web portal or internal tool. Modern teams typically work in short iterative cycles, automate testing and deployment, and increasingly use AI coding assistants to speed up routine work.

The full set of software applications an organization builds, buys and maintains, viewed together so leaders can judge each one's cost, risk and business value. Reviewing the portfolio helps decide which systems to invest in, modernize, consolidate or retire.

The field of building computer systems that perform tasks normally requiring human judgment, such as understanding language, recognizing images, making predictions and planning actions. Today's AI is dominated by machine learning, including large language models, and works best when applied to clearly defined problems with good data and human oversight.

A data mining technique that finds items or events that frequently occur together and expresses them as if-then rules, such as customers who buy pasta often also buy sauce. Rules are judged by measures like support, confidence and lift, and are widely used for cross-selling and store layout.

Using software, scripts or machines to carry out repetitive tasks with little or no human involvement, from scheduled data pipelines to robotic process automation and AI agents. Good automation reduces errors and frees people for higher-value work, but it needs monitoring so failures are caught quickly.

A security and analytics approach that learns what normal activity looks like for users, devices or systems and flags meaningful deviations, such as a login at an unusual hour or a sudden spike in data downloads. It helps catch insider threats and compromised accounts that rule-based tools miss.

Datasets so large, fast-moving or varied that traditional databases and tools struggle to store and process them, often described by volume, velocity and variety. Handling it requires distributed storage and compute, such as cloud data lakes and Spark, and the real value comes from the questions it can answer.

A machine learning task where the model assigns each item to one of exactly two classes, such as fraud or not fraud, churn or stay, spam or not spam. Results are usually judged with precision, recall and related measures, since plain accuracy can mislead when one class is rare.

A probability distribution that describes how many successes to expect in a fixed number of independent trials, each with the same chance of success, such as the number of conversions from 1,000 site visits. It underpins many A/B test calculations and quality-control checks.

A shared digital ledger that records transactions in linked, cryptographically secured blocks copied across many computers, making past entries extremely hard to alter without detection. Beyond cryptocurrency it is used for supply chain traceability and digital credentials, but it adds cost and complexity a regular database often avoids.

The practice of using data, statistics and modeling to understand business performance and guide decisions, ranging from describing what happened to predicting what will happen and recommending what to do. It focuses on answering specific business questions rather than on the underlying technology.

Connecting an organization's applications, data and processes, internally and with partners, so information flows automatically between systems such as CRM, ERP and e-commerce. It is usually done through APIs, integration platforms and event streams, and it removes manual re-keying and the errors that come with it.

The tools and practices that turn an organization's data into reports, dashboards and analysis people use to track performance and make decisions. Modern BI combines a governed data layer with self-service tools such as Power BI or Tableau, and increasingly adds natural language questions answered by AI.

Software that holds a conversation with people through text or voice, answering questions or completing simple tasks. Older chatbots followed scripted rules; most new ones use large language models, often grounded in company documents so answers stay accurate. When a chatbot can also take actions in other systems, it becomes an AI agent.

A machine learning model that predicts which category an item belongs to, such as whether a loan will default or which product line a support ticket concerns. It typically outputs a probability for each class, and a threshold turns that score into a decision.

The share of people who click a link, ad or email after seeing it, calculated as clicks divided by impressions. It is a quick signal of how relevant and appealing a message is, but a high rate means little unless those clicks lead to conversions.

The percentage of sales opportunities or qualified leads that turn into paying customers over a given period. Tracking it by rep, channel or segment shows where the sales process works and where deals stall, and it is a key input to revenue forecasts.

Renting computing resources such as servers, storage, databases and AI services over the internet from providers like AWS, Microsoft Azure and Google Cloud, and paying only for what is used. It lets teams scale quickly and avoid buying hardware, but spending needs active management to stay under control.

An unsupervised machine learning technique that groups similar items together without predefined labels, based on how close they are in their features. Businesses use it for customer segmentation, grouping documents by topic and spotting unusual records that fit no group.

A system's ability to handle many users or processes working at the same time without corrupting shared data. In databases, concurrency control uses locks or versioning so that two people updating the same record do not overwrite each other's changes.

A variable that can take any value within a range, including fractions, such as temperature, revenue or time on site. It contrasts with a discrete variable, which takes only separate countable values, and the distinction affects which statistics and charts are appropriate.

The percentage of visitors or users who complete a desired action, such as making a purchase, signing up or requesting a demo, out of everyone who had the chance. It is one of the most direct measures of how well a website, campaign or funnel turns interest into results.

A type of deep learning network designed for grid-like data such as images, using small filters that scan the input to detect edges, textures and shapes. CNNs power image classification, defect detection and medical imaging, although transformer-based vision models now compete with them on many tasks.

A number between -1 and 1 that measures how strongly two variables move together in a straight-line relationship: values near 1 mean they rise together, near -1 that one falls as the other rises, and near 0 that there is little linear link. Correlation alone never proves cause and effect.

Analysis that combines data, methods and expertise from several fields, such as finance, operations and marketing, to answer a question none could answer alone. It often reveals causes and trade-offs hidden inside departmental silos, but it depends on shared definitions and well-integrated data.

The percentage of customers a business keeps over a set period, excluding new customers gained during that time. Because keeping customers usually costs less than winning new ones, retention is a core indicator of loyalty and long-term revenue health.

The use of data analysis and machine learning on security data, such as network traffic, logs and user activity, to detect, investigate and predict cyberattacks. It helps security teams sort through millions of events and focus on the small number that signal real threats.

The practice of protecting systems, networks, applications and data from unauthorized access, disruption and theft through technology, processes and training. It now also covers securing AI systems themselves, including risks such as prompt injection, data poisoning and leaks of sensitive data through models.

Evidence-based information about current and emerging attackers, their methods and their targets, gathered and analyzed so an organization can anticipate and defend against attacks. It ranges from technical indicators like malicious IP addresses to strategic reports on which groups target a given industry.

A single screen that displays the most important metrics and charts for a team or process, usually updated automatically from live data. A good dashboard is built around specific decisions and shows only what its audience needs, rather than every number available.

Facts, measurements and observations recorded in a form that can be stored and processed, including numbers, text, images, audio and sensor readings. On its own, data has little value; it becomes useful once it is organized, checked for quality and analyzed to answer a question.

Making business choices based on evidence from data and analysis rather than on intuition, habit or the loudest opinion. It does not remove judgment; it gives leaders a clearer view of options, trade-offs and likely results, and requires trustworthy data and people who can interpret it.

The degree to which data correctly reflects the real-world facts it describes, such as a customer's actual address or a product's true price. It is a core dimension of data quality, and inaccurate data quietly undermines reports, models and automated decisions.

The process of examining data to find patterns, answer questions and support decisions, using methods that range from simple summaries and charts to statistical modeling and machine learning. It is commonly grouped into descriptive, diagnostic, predictive and prescriptive analytics.

Removing or altering personal identifiers in a dataset so the individuals it describes cannot reasonably be identified, for example by masking, generalizing or aggregating fields. True anonymization is harder than it looks, because combining several innocuous fields can often re-identify people.

A systematic review of an organization's data and how it is collected, stored, used and accessed, to check quality, compliance and security. Audits find errors, unused or risky data and policy gaps, and produce evidence for regulators and leadership.

The assurance that data can be accessed by authorized people and systems when they need it, without unexpected outages or delays. It depends on reliable infrastructure, backups, replication and disaster recovery planning, and is one of the three pillars of information security alongside confidentiality and integrity.

An organized collection of data stored electronically and managed by software that lets people and applications store, retrieve and update it efficiently. Common types include relational databases that use tables and SQL, and NoSQL databases built for documents, key-value pairs, graphs or vectors.

Software that creates, manages and controls access to databases, handling storage, queries, security, backups and concurrent use. Examples include PostgreSQL, MySQL, Oracle Database and SQL Server, as well as NoSQL systems such as MongoDB.

An incident in which sensitive, protected or confidential data is accessed, copied or disclosed by someone who is not authorized to see it, whether through hacking, a misconfigured system or human error. Breaches often trigger legal notification duties, fines and lasting damage to customer trust.

Finding and fixing problems in a dataset, such as duplicates, typos, missing values, inconsistent formats and impossible values, so it can be trusted for analysis. It often takes more time than the analysis itself and works best when the fixes are automated and applied at the source.

Gathering data from sources such as applications, forms, sensors, surveys, logs and third parties for a defined purpose. Collecting only what is needed, with clear consent and documented origins, makes later analysis easier and reduces privacy and compliance risk.

A formal agreement between the producers and consumers of a dataset that specifies its schema, meaning, quality expectations, update frequency and ownership. Contracts are checked automatically, so a change upstream cannot silently break dashboards or models downstream.

Finding, exploring and understanding the data an organization has, including where it lives, what it means and who owns it. The term covers both business users searching a data catalog and the automated scanning of systems to locate sensitive information such as personal data.

A change over time in the data a machine learning model sees in production compared with the data it was trained on, such as new customer behavior or a shifted sensor reading. Drift quietly erodes accuracy, so teams monitor inputs and outputs and retrain models when it appears.

The full set of data sources, platforms, tools, people and partners that create, move and use data within and around an organization. Mapping the ecosystem shows where data comes from, where it is duplicated and where gaps or risks exist.

The discipline of designing, building and running the pipelines and platforms that move data from source systems into a reliable, usable form for analytics and AI. Data engineers handle ingestion, transformation, storage, orchestration and monitoring, so analysts and data scientists can work with data they trust.

Combining data from multiple sources about the same entities or events to produce information that is more complete and accurate than any single source. Common in defense, autonomous vehicles and sensor networks, it also applies to merging customer data from sales, support and web activity.

The policies, roles, standards and processes that determine who can do what with which data, and how its quality, security and compliance are maintained. Effective governance makes data easier to find and trust, and it now extends to how data is used to train and run AI systems.

A central platform that collects data from many systems and distributes it in a consistent, governed form to the applications and teams that need it. Unlike a warehouse built mainly for analysis, a hub focuses on sharing and synchronizing data, such as master customer or product records.

A meaningful finding drawn from analyzing data that explains what is happening or why, such as discovering that most churn happens in the first 60 days. An insight is valuable only when it is accurate, relevant to a decision and communicated clearly enough to act on.

Combining data from different systems into a consistent, unified view so it can be analyzed or used together, for example merging CRM, billing and web data into one customer record. Common methods include ETL and ELT pipelines, APIs, change data capture and data virtualization.

The overall accuracy, completeness and consistency of data across its whole life, including protection against unauthorized or accidental changes. It is enforced through database constraints, validation rules, access controls and audit trails.

A central repository that stores large amounts of raw data in its native format, structured or not, usually in low-cost cloud object storage, until it is needed. It is flexible and cheap, but without governance and cataloging it can turn into a disorganized swamp nobody trusts.

A data architecture that combines the low-cost, flexible storage of a data lake with the reliability, performance and governance features of a data warehouse, usually through open table formats such as Delta Lake or Apache Iceberg. It lets BI, data science and AI work from one copy of the data.

A record of where data comes from, how it moves between systems and what transformations it undergoes before reaching a report or model. Lineage makes it possible to trace errors to their source, assess the impact of changes and show regulators how a number was produced.

The overall practice of collecting, storing, organizing, protecting and maintaining data so it stays accurate, accessible and useful throughout its life. It spans architecture, integration, quality, security, metadata and governance, and provides the foundation for analytics and AI.

Changing the structure or content of data to make it more useful, such as sorting, filtering, joining, aggregating or reshaping tables, typically with SQL, Python or spreadsheet tools. In a different sense, the term also describes deliberately falsifying data, which is a security and integrity concern.

A smaller, focused subset of a data warehouse built for one department or subject area, such as sales or finance. Because it holds only relevant data, a data mart is simpler and faster for its users to query, but too many independent marts can create conflicting numbers.

An organizational approach in which business domains, such as sales or supply chain, own and publish their data as products, supported by a shared self-service platform and common governance rules. It aims to remove the bottleneck of a single central data team in large organizations.

Searching large datasets with statistical and machine learning techniques to discover patterns, relationships and anomalies that are not obvious at first glance. Typical uses include market basket analysis, customer segmentation and fraud detection.

Generating measurable economic value from data, either directly by selling or licensing data and data-driven services, or indirectly by using it to cut costs, raise revenue or improve products. Direct approaches require especially careful attention to privacy, consent and contracts.

A term with two common meanings: in database design, organizing tables to reduce duplicated data and update errors; in analytics and machine learning, rescaling numeric values to a common range so no single feature dominates because of its units. Context usually makes clear which is intended.

Continuous, automated monitoring of the health of data pipelines and datasets, tracking freshness, volume, schema changes, distributions and lineage. It alerts teams to broken or suspicious data before it reaches dashboards and models, much as application monitoring does for software.

Getting raw data ready for analysis or machine learning by collecting, cleaning, combining, transforming and labeling it. It is often the most time-consuming part of a data project, and the choices made here strongly affect the reliability of the results.

A structured review of how an organization collects, uses, stores, shares and deletes personal data, checked against laws such as GDPR and US state privacy acts and against its own policies. It identifies compliance gaps, unnecessary data and risky practices before regulators or attackers do.

A reusable, well-documented dataset or data service built for a defined set of consumers, with a named owner, quality guarantees and clear access rules. Treating data as a product means designing it around user needs and maintaining it over time, rather than producing one-off extracts.

Examining a dataset to understand its structure, content and quality, such as value ranges, distributions, missing values, duplicates and relationships between fields. Profiling is usually the first step before integrating or analyzing data, because it reveals problems early.

The combination of legal, organizational and technical measures that keep data safe from loss, corruption, misuse and unauthorized access, including backups, encryption, access controls and privacy policies. It covers both securing data and respecting the rights of the people it describes.

Laws that govern how organizations collect, use, store and share personal data and what rights individuals have over it, such as the EU's GDPR, California's CCPA/CPRA and sector rules like HIPAA. Newer rules such as the EU AI Act add obligations for AI systems.

How well data fits its intended use, commonly judged on accuracy, completeness, consistency, timeliness, validity and uniqueness. Poor quality leads to wrong reports, failed models and lost trust, so mature teams measure it continuously and assign clear ownership for fixing problems.

The same piece of data being stored in more than one place. Deliberate redundancy, such as backups and replicas, improves availability; accidental redundancy creates conflicting copies, wasted storage and confusion about which version is correct.

The degree to which data is consistently accurate, complete and delivered on time, so people and systems can depend on it without second-guessing. It is maintained through testing, monitoring, data contracts and clear ownership of each dataset.

An interdisciplinary field that uses statistics, programming, machine learning and domain knowledge to extract knowledge and predictions from data. Data scientists frame business problems, explore and model data, and communicate results that guide decisions or power products.

Protecting data from unauthorized access, corruption and theft throughout its life, using measures such as encryption, access controls, monitoring, masking and secure backups. It applies to data at rest, in transit and in use, including data sent to AI models.

A defined collection of related data, such as a table of customer records or a folder of labeled images, treated as a unit for analysis or model training. For machine learning, a dataset's size, diversity and label quality largely determine how well the resulting model performs.

Any origin from which data is obtained, such as a database, application, file, API, sensor, website or third-party provider. Knowing each source's owner, update frequency and reliability is essential for judging how far the resulting analysis can be trusted.

Converting data into a common, agreed format so it can be compared and combined, for example using one date format, consistent country codes or standard product categories. In statistics, it can also mean rescaling values to have a mean of zero and a standard deviation of one.

Communicating findings from data through a clear narrative supported by well-chosen visuals and context, so the audience understands what the numbers mean and what to do about them. It bridges the gap between analysis and action, especially for non-technical decision makers.

A plan that defines how an organization will collect, manage, govern and use data to reach its business goals, including priorities, architecture, roles, skills and investment. A sound strategy ties every data initiative to a business outcome and is a prerequisite for scaling AI.

Changing data from one format, structure or set of values into another so it fits the needs of a target system or analysis, for example converting currencies, joining tables or aggregating daily figures into monthly totals. It is the T in ETL and ELT pipelines.

Representing data graphically, through charts, maps and diagrams, so patterns, trends and outliers are easier to see and understand. Effective visualization matches the chart type to the question, avoids clutter and keeps scales honest.

A central database designed for analysis and reporting, holding cleaned, integrated and historical data from many operational systems in a consistent structure. Modern cloud warehouses such as Snowflake, BigQuery and Redshift separate storage from compute so they can scale on demand.

The hands-on work of transforming messy raw data into a structured, usable form, including cleaning, reshaping, merging and enriching it. The term is often used interchangeably with data munging and overlaps heavily with data preparation.

A model that makes predictions by asking a series of yes-or-no questions about the data, branching at each step until it reaches an outcome, much like a flowchart. Decision trees are easy to explain, and combining many of them in random forests or gradient boosting produces highly accurate models.

A branch of machine learning that uses neural networks with many layers to learn patterns directly from large amounts of data, such as images, audio and text. It powers modern speech recognition, computer vision and large language models, but needs substantial data and computing power.

Predicting how much of a product or service customers will want over a future period, using historical sales, seasonality, promotions, prices and external signals such as weather. Better forecasts reduce stockouts and excess inventory and improve staffing and purchasing decisions.

Promoting products and services through digital channels such as search engines, social media, email, websites and mobile apps. Because nearly every interaction can be measured, it relies heavily on analytics to track performance, target audiences and optimize spending.

The broad process of using digital technology, data and AI to change how an organization operates, serves customers and creates value. Successful transformation is as much about redesigning processes and building skills as about adopting new tools.

A set of steps connected by one-way links that never loop back to an earlier step. In data engineering, DAGs describe the order in which tasks must run, such as in Apache Airflow pipelines or the execution plans Spark builds, so dependencies are respected and failed steps can be rerun.

A variable that can take only separate, countable values, such as the number of orders, customers or defects, with nothing in between. It contrasts with a continuous variable, and the difference determines which statistical methods and charts are appropriate.

A database whose data is stored across multiple servers or locations but appears to users as a single system. Spreading data this way improves scalability and resilience, though it makes keeping every copy consistent and fast more complicated.

A way to deal with imbalanced training data by randomly removing examples from the larger class so the classes are more evenly represented. It speeds up training and helps the model notice the rare class, but it discards information, so it is often combined with reweighting.

A list of numbers that represents a piece of content, such as a word, document, image or product, so that similar items end up close together mathematically. Embeddings power semantic search, recommendations and retrieval-augmented generation, and are typically stored in vector databases.

Using real-time and historical data, such as call volumes, locations, weather and traffic, to plan for and manage emergencies. Agencies apply it to position ambulances, predict demand surges, route responders and coordinate resources during disasters, where faster decisions save lives.

A measure of how actively people interact with content, such as likes, comments, shares, clicks or time spent, relative to the number of people reached or following. Definitions vary by platform, so it matters to state which interactions and which denominator are used.

The straight-line distance between two points, calculated as the square root of the summed squared differences across each dimension. It is the default similarity measure in many clustering and nearest-neighbor methods, and works best when features are on comparable scales.

The first pass over a new dataset, where analysts summarize, chart and probe the data to learn its shape, catch errors and outliers, and form questions worth testing. It comes before formal modeling, and skipping it is a common reason models end up built on data nobody fully understood.

A pipeline pattern that copies raw data from source systems straight into a cloud warehouse or lakehouse and then reshapes it there, using the warehouse's own computing power. ELT has largely replaced older ETL designs on modern platforms because storage is cheap and keeping the raw data makes reprocessing easy.

A data integration process that pulls data from source systems, reshapes and cleans it in a separate processing step, then loads the result into a target such as a data warehouse. Many cloud teams now favor ELT, which loads raw data first and transforms it inside the warehouse.

Estimating values beyond the range of the data you actually have, such as projecting next year's sales from the last five years of results. It assumes past patterns will continue, so the likely error grows the further the estimate reaches past known data.

An input variable a machine learning model uses to make a prediction, such as account age, monthly spend or region in a customer churn model. Choosing and shaping good features, called feature engineering, often affects model quality more than the choice of algorithm.

Further training an existing pretrained model on a smaller, task-specific dataset so it performs better in a particular domain, style or task. For language models, fine-tuning is often weighed against retrieval-augmented generation and prompt design, which are cheaper to update when the underlying information changes.

Building statistical or machine learning models that estimate future values of a measure over time, such as weekly demand, call volume or cash flow. Useful forecast models report a range of likely outcomes rather than a single number, so planners can see how uncertain the estimate is.

Using rules, analytics and machine learning to spot transactions, claims or account activity that look deceptive, ideally before money or goods are lost. Systems usually score each event in real time and send the riskiest to human reviewers, balancing missed fraud against blocking legitimate customers.

AI systems that create new content, such as text, images, code, audio or video, in response to prompts, based on patterns learned from large training datasets. Business uses range from drafting and summarizing to coding assistants and AI agents, and outputs need review because these systems can give confident but wrong answers.

Analyzing location-based data, such as satellite and aerial imagery, GPS traces and mapped records, to understand what is happening where and how it changes over time. Uses range from defense and disaster response to choosing store sites and planning delivery routes.

Google Cloud's fully managed, serverless data warehouse, which runs SQL queries over very large datasets without any servers to set up. It bills by data scanned or by reserved capacity and includes built-in machine learning and geospatial functions, so how queries are written has a direct effect on cost.

Google's public cloud, offering computing, storage, databases, analytics and AI services, including BigQuery and the Vertex AI machine learning platform. It competes mainly with Amazon Web Services and Microsoft Azure, and many organizations run workloads across more than one of these providers.

A database that stores data as nodes, such as people, accounts or devices, and edges, the relationships between them, so connected data can be queried directly. It suits fraud rings, recommendations, supply networks and knowledge graphs, where questions follow chains of relationships that are slow to answer with table joins.

The storage layer of Apache Hadoop, which splits large files into blocks and keeps copies across many inexpensive servers so data survives the loss of a machine. Much of this kind of storage has since moved to cloud object stores, but HDFS still runs in many on-premises big data clusters.

The resource manager in Apache Hadoop; the name stands for Yet Another Resource Negotiator. YARN decides how a cluster's memory and processors are shared among jobs, so batch processing, Spark, SQL engines and streaming work can run side by side on the same hardware.

When a generative AI model produces output that sounds plausible but is false or unsupported, such as invented facts, citations or figures. Grounding answers in trusted sources, asking for citations and keeping people in the review loop for high-stakes uses all reduce the risk, but none removes it entirely.

The number of positions at which two strings or binary codes of equal length differ; for example, 1011 and 1001 differ in one position. It is used in error detection, in comparing records made of categories and in fast similarity search over compact binary fingerprints.

Using clusters of powerful processors, increasingly GPUs, working in parallel to solve problems too large or slow for ordinary servers, such as engineering simulations, genomics and training large AI models. HPC capacity is now rented on demand from cloud providers as well as run in dedicated on-premises supercomputers.

Extracting useful information from photos, scans or video frames, such as counting objects, reading text, measuring defects or classifying medical images. Most modern image analysis relies on deep learning models trained on labeled examples, though simpler rule-based methods still work in controlled settings like factory lines.

A dataset in which one outcome is far rarer than the others, such as fraud cases making up well under one percent of transactions. Models trained on it can look accurate while missing nearly every rare case, so teams use resampling, class weights and metrics like precision and recall instead of plain accuracy.

The stage where a trained model is put to work on new data to produce predictions, classifications or generated text. In production AI, inference accounts for most ongoing compute cost and response time, so model size and hosting choices matter as much as accuracy.

The network of physical devices, such as sensors, meters, vehicles, machines and wearables, that collect data and exchange it over the internet. IoT data usually arrives as high-volume, time-stamped streams, so it needs pipelines built for real-time ingestion and careful attention to device security.

Estimating unknown values that fall between known data points, such as filling a missing daily reading from the readings on either side of it. It is generally more reliable than extrapolation because the estimate stays within the range the data already covers.

The combined hardware, software, networks, data centers and cloud services an organization relies on to run its systems and store its data. Infrastructure decisions, such as cloud versus on-premises, shape the cost, security and scalability of every analytics and AI project built on top of them.

A simple machine learning method that classifies or estimates a new data point by finding the k most similar points in the training data and taking their majority label or average value. It needs no real training step but slows down on large datasets, and results depend heavily on the distance measure chosen.

The known answer attached to a training example in supervised learning, such as whether a past customer actually churned or the price a house actually sold for. The model learns to predict labels from features, so inaccurate or inconsistent labeling directly limits how good the model can become.

A very large neural network trained on huge amounts of text to predict and generate language, which lets it answer questions, summarize, translate, write code and follow instructions. LLMs power chat assistants and AI agents, and they work best when connected to an organization's own data and checked for accuracy.

A statistical method that fits a straight line, or a flat plane when there are several inputs, through data to describe how a numeric outcome changes as the inputs change. It is fast and easy to explain, which makes it a common baseline, but it assumes a roughly linear relationship and is sensitive to outliers.

A statistical model that estimates the probability of a yes-or-no outcome, such as whether a customer will renew, from one or more input variables. Despite its name it is used for classification, and it stays popular because the effect of each input on the odds is easy to interpret.

A branch of artificial intelligence in which software learns patterns from data to make predictions or decisions, instead of following rules a programmer wrote by hand. Common business uses include demand forecasting, churn prediction, fraud detection and document processing, and results depend on data quality as much as on the algorithm.

Studying malicious software to learn what it does, how it spreads, what it targets and how to detect and remove it. Analysts combine inspecting the code with running samples in an isolated sandbox to watch their behavior, then turn the findings into detection rules and threat intelligence.

The distance between two points found by adding up the absolute differences along each dimension, like counting city blocks on a street grid instead of measuring a straight line. It is common in clustering and nearest-neighbor methods and is less affected by one extreme difference than Euclidean distance.

Measuring and analyzing marketing activity across channels to see what drives awareness, leads and revenue, and where budget should shift. Typical work includes campaign attribution, customer segmentation and lifetime value modeling; stricter privacy rules and fewer third-party cookies have made first-party data central to it.

Gathering and analyzing information about customers, competitors and market conditions to guide decisions on products, pricing and positioning. It mixes primary research, such as surveys and interviews, with secondary sources like industry reports, and increasingly draws on behavioral data from digital channels.

The arithmetic average: the sum of all values divided by how many values there are. It is the most common summary of a numeric measure, but a few very large or small values can pull it far from what is typical, which is why the median is often reported alongside it.

The middle value when data is sorted from lowest to highest; with an even number of values, it is the average of the two in the middle. Because extreme values barely move it, the median is the better summary for skewed measures like household income or home prices.

Microsoft's public cloud platform, providing computing, storage, databases, analytics and AI services, including hosted access to OpenAI and other foundation models. It works closely with Microsoft 365, Entra ID and Power BI, which makes it a common choice for organizations already standardized on Microsoft tools.

Microsoft's business intelligence tool for connecting to data, building data models and publishing interactive reports and dashboards. It is part of Microsoft Fabric, the company's analytics platform, and is widely adopted because it fits closely with Excel, Teams and Azure.

A general distance formula that includes other common measures as special cases, controlled by a parameter p: p=1 gives Manhattan distance and p=2 gives Euclidean distance. Adjusting p changes how strongly a large difference in one dimension counts in clustering or nearest-neighbor models.

A pattern of missing data where the chance a value is missing depends on other recorded information, but not on the missing value itself. For example, younger survey respondents may skip an income question more often than older ones. Methods such as multiple imputation can usually handle this case well.

A pattern of missing data where gaps occur purely by chance, unrelated to any value in the dataset, such as readings lost to a random sensor glitch. It is the easiest case to handle, since dropping incomplete rows shrinks the sample but does not bias the results.

A pattern of missing data where the chance a value is missing depends on that value itself, such as people with very high debt being less likely to report it. It is the hardest case because the remaining data is biased, and correcting it usually needs domain knowledge or extra data collection.

Gaps in a dataset where a value should exist but was never recorded, was lost or was withheld. Whether you remove those rows, fill in estimates or flag them can change the results of an analysis, so it pays to understand why the values are missing before choosing a fix.

The practices and tools for reliably deploying, monitoring and updating machine learning models in production, borrowing ideas from software DevOps such as automation, version control and continuous delivery. It covers data and model versioning, automated retraining, performance monitoring and rollback, closing the gap between a working prototype and a dependable system.

The value that appears most often in a dataset, such as the most common shoe size sold. It is the only basic average that works for categories like product type or region, and a dataset can have more than one mode or none at all.

In machine learning, the result of training: a mathematical function with learned parameters that turns input data into a prediction, score or generated response. A model is only as reliable as the data it learned from, and it needs monitoring after deployment because real-world conditions change.

The share of a classification model's predictions that are correct, out of all the predictions it makes. It is easy to understand but misleading when one class dominates: a model that never flags fraud can be 99 percent accurate, so read it alongside precision and recall.

The gradual decline in a deployed model's accuracy as real-world data or relationships shift away from what it learned in training, for example when customer behavior changes after a price increase. Monitoring inputs and outcomes, and retraining on a schedule or when alerts fire, keeps drift in check.

A classification task where each item is assigned to one of three or more categories, such as routing support tickets to billing, technical or sales. It differs from binary classification, which has two outcomes, and from multi-label tasks, where one item can belong to several categories at once.

A widely used open-source relational database, owned by Oracle, that stores data in tables and is queried with SQL. It runs a large share of web applications and is offered as a managed service by all major cloud providers; MariaDB is a closely related community-developed fork.

The part of natural language processing that produces readable text from data or prompts, such as report summaries, product descriptions or chatbot replies. Older systems filled in templates; today most of this work is done by large language models, which write fluently but need checks for factual accuracy.

The field of AI that enables computers to work with human language in text and speech: reading, classifying, translating, summarizing and generating it. Business uses include document extraction, sentiment analysis, search and chatbots, and much of the field now runs on large language models.

An open-source Python library for teaching and experimenting with natural language processing, with tools for splitting text into words and sentences, tagging parts of speech and working with sample text collections. It is popular in education and research, while production systems more often use libraries such as spaCy or transformer-based models.

The part of natural language processing concerned with working out what text or speech means, including the intent behind it, the people and things it mentions and how ideas relate. It is what lets a support bot recognize that "I can't log in" and "my password reset isn't working" describe the same problem.

Collecting and analyzing data from network devices and traffic, such as flow records, logs and performance metrics, to understand how a network is performing and being used. It helps teams find bottlenecks, plan capacity and catch unusual activity that could signal a fault or an attack.

Network behavior that departs noticeably from the normal baseline, such as a sudden traffic spike, connections to unfamiliar destinations or data leaving at odd hours. Anomalies can come from equipment faults or misconfigurations as well as attacks, so detection tools flag them for investigation rather than treating each one as a threat.

How completely an organization can see the traffic, devices, users and applications on its networks, including cloud services and remote connections. Blind spots are where attackers and performance problems hide, so good visibility is a basic requirement for both security monitoring and troubleshooting.

A machine learning model built from layers of connected nodes, loosely inspired by neurons, that adjust the strength of their connections during training to recognize patterns. Networks with many layers make up deep learning, the basis of modern image recognition, speech recognition and large language models.

A category of databases that store data in forms other than fixed relational tables, such as documents, key-value pairs, wide columns or graphs. They suit flexible or fast-changing data and very large scale, often giving up some consistency guarantees or query flexibility in exchange for speed and easy scaling across servers.

A style of data processing designed for analysis, where queries summarize large volumes of historical data across dimensions such as time, product and region. OLAP workloads run in data warehouses and analytical engines and sit behind most business intelligence dashboards and reports.

A style of data processing built to handle many small transactions at the same time, quickly and reliably, such as orders, payments and account updates. OLTP databases put fast writes and data integrity first, so heavy reporting is usually moved to a separate analytical system to avoid slowing them down.

The current, frequently changing data that runs day-to-day business, such as open orders, inventory levels and account balances, usually held in transactional systems. It is typically copied into a warehouse or lake for analysis, while the live systems remain the source of truth.

A large US technology company best known for Oracle Database, one of the most widely used commercial relational databases, along with enterprise applications and Oracle Cloud Infrastructure. Many large organizations run core finance and operations systems on Oracle, so it is a frequent source system in data integration projects.

A data point that sits far from the rest, such as a $50,000 order in a store where nearly all orders are under $200. An outlier may be an error to correct or a real and important event, such as fraud, so investigate it before removing it.

A free, open-source relational database, often called Postgres, known for reliability, standards support and extensibility. Extensions add capabilities such as geospatial queries with PostGIS and vector search with pgvector, and every major cloud provider offers it as a managed service.

Of all the items a model flagged as positive, the share that really were positive: true positives divided by all predicted positives. High precision means few false alarms, which matters when each alert triggers costly follow-up, such as blocking a card or sending out an investigator.

A model's output for a new case it has not seen before, such as the chance a customer will cancel or next month's expected sales. Predictions are usually probabilities or estimates rather than certainties, so business rules decide which score or threshold should trigger an action.

Using historical data, statistics and machine learning to estimate what is likely to happen next, such as which customers will churn, which machines will fail or how much stock to order. It sits between descriptive reporting, which explains what happened, and prescriptive analytics, which recommends what to do.

The hands-on work of building, testing and tuning the statistical or machine learning models used in predictive analytics. It involves choosing features and algorithms, validating on data the model has not seen and confirming that performance holds up before the model goes into production.

Analytics that recommends a course of action, not just a forecast, by combining predictions with optimization, simulation or business rules. Examples include setting prices, scheduling staff or choosing reorder quantities; the recommendations are only as good as the goals and constraints the model is given.

Analyzing event logs from systems such as ERP, CRM and ticketing tools to reconstruct how business processes actually run, step by step. It exposes bottlenecks, rework loops and departures from the intended process, giving an evidence-based starting point for automation and improvement work.

An attack on AI applications in which malicious instructions are hidden in user input or in content the model reads, such as a web page or email, to make it ignore its rules, leak data or take unwanted actions. It is a leading security risk for AI agents and calls for layered defenses, not just careful prompts.

A general-purpose programming language that has become the standard for data science, machine learning and AI development, thanks to libraries such as pandas, scikit-learn and PyTorch. Its readable syntax makes it approachable for analysts as well as engineers, and it is widely used for data pipelines and automation.

A free programming language and environment built for statistics and data visualization, widely used in academia, life sciences and research-heavy analytics teams. Its package ecosystem, including the tidyverse and ggplot2, is strong for statistical modeling and publication-quality charts, though Python is more common for production machine learning.

A machine learning method that builds many decision trees, each on a random sample of the rows and features, and combines their votes or averages into one prediction. It is accurate, copes well with mixed data types and resists overfitting, which makes it a dependable choice for tabular business data.

Data in the form it was originally captured, before any cleaning, transformation or aggregation, such as unedited log files, sensor readings or system exports. Keeping raw data lets teams reprocess it when business rules change or errors come to light, which is why many pipelines store it untouched first.

Of all the items that really were positive, the share a model correctly caught: true positives divided by true positives plus false negatives. High recall means few misses, which matters when overlooking a case is costly, such as an illness or a fraudulent payment.

Software that suggests products, content or actions to each user based on their past behavior, the choices of similar users and the attributes of the items. Familiar from retail and streaming services, these systems also drive next-best-offer and cross-sell programs in banking, insurance and B2B sales.

A type of neural network that processes sequences one step at a time while carrying a memory of earlier steps, which suits time series, text and speech. Variants such as LSTMs were standard for language tasks until transformer models largely replaced them, though RNNs still appear in some forecasting and signal-processing work.

A family of statistical methods for estimating how an outcome changes as one or more other factors change, such as how price and advertising affect sales. It helps quantify relationships and make predictions, but on its own it shows association, not proof that one factor causes another.

In machine learning, a model that predicts a numeric quantity, such as a sale price, delivery time or energy demand, rather than a category. Its quality is usually judged by how far predictions land from actual values, using measures such as mean absolute error.

A type of machine learning in which an agent learns by trial and error, taking actions in an environment and receiving rewards or penalties until it finds a strategy that earns the most reward. It is used in robotics, game playing and resource scheduling, and to tune large language models using human feedback.

A database that organizes data into tables of rows and columns, linked to one another through shared keys and queried with SQL. Systems such as PostgreSQL, MySQL, SQL Server and Oracle enforce structure and consistency, which is why they remain the default for transactional business data.

A central place where data, code or documents are stored and managed, such as a data repository holding curated datasets or a Git repository tracking every change to a codebase. Good repositories control access and keep version history, so people can find the authoritative copy and see what changed.

The original core data structure in Apache Spark: a read-only collection of records split across a cluster so it can be processed in parallel and rebuilt automatically if a machine fails. Most Spark work now uses the higher-level DataFrame API, but RDDs remain available when engineers need low-level control.

A technique in which a language model first retrieves relevant passages from trusted sources, such as company documents or a knowledge base, and then bases its answer on them. It keeps answers current without retraining, reduces hallucination and allows citations, but answer quality depends heavily on how well documents are indexed and retrieved.

A measure of how much an investment returns relative to its cost, usually calculated as net gain divided by cost and shown as a percentage. For data and AI projects, a credible ROI depends on agreeing baseline metrics before work starts and counting ongoing costs such as maintenance and cloud usage.

Identifying potential threats to an organization, such as fraud, credit default, compliance breaches, cyberattacks or supplier failure, early enough to act on them. Modern approaches pair business rules with analytics and machine learning that watch data continuously and raise alerts when risk indicators move.

The structures, roles and policies that set how an organization decides which risks to accept, who owns them and how they are reported to leadership and the board. For AI, it increasingly covers model approval, documentation and oversight, guided by frameworks such as the NIST AI Risk Management Framework and the EU AI Act.

The ongoing process of identifying, assessing, prioritizing and responding to risks, whether by reducing, transferring, accepting or avoiding them. Data plays a growing part, from quantifying exposure to tracking early warning signs, and AI systems add their own risks to manage, such as biased outputs and data leakage.

Examining systems, networks, applications and logs to find weaknesses, misconfigurations and signs of compromise, then judging how serious each issue is. It includes vulnerability assessment, log review and alert investigation, and its findings guide which fixes and controls to prioritize.

A shared layer of business definitions between data storage and reporting tools, where metrics such as revenue or active customer are defined once so every dashboard and query uses the same logic. It also gives AI assistants consistent definitions to work from when answering questions about company data.

Using natural language processing to judge whether text such as reviews, survey comments, support tickets or social posts expresses positive, negative or neutral feeling, and sometimes specific emotions. It lets organizations track customer opinion at scale, though sarcasm, slang and mixed opinions still cause errors.

The foundation of Apache Spark that handles task scheduling, memory management, fault recovery and distributed data processing across a cluster. Spark's higher-level libraries, including Spark SQL, Structured Streaming, MLlib and GraphX, are all built on top of it.

Apache Spark's library for graph processing, used to analyze networks of connected items, such as social links or chains of transactions, with algorithms like PageRank across a cluster. It has seen little new development in recent years, and many teams now use GraphFrames or a dedicated graph database instead.

Apache Spark's capability for processing live data streams, such as clickstreams, IoT readings or transaction feeds, in small continuous batches. The original DStream-based Spark Streaming API is now legacy; new work uses Structured Streaming, which applies the same DataFrame and SQL interfaces used for batch data.

Technology that converts spoken audio into written text, also called speech-to-text. It powers voice assistants, call center transcription, meeting notes and dictation; modern deep learning models handle accents and background noise far better than earlier systems, though specialist vocabulary can still need tuning.

A measure of how spread out values are around their mean: a small standard deviation means values cluster tightly, a large one means they vary widely. In roughly bell-shaped data, about two-thirds of values fall within one standard deviation of the mean, which makes it useful for spotting unusual results.

Data organized into a fixed format of rows and columns with defined fields and types, such as sales transactions, customer records or spreadsheets. It is easy to store in relational databases and query with SQL, and most traditional reporting and analytics is built on it.

The standard language for querying and managing data in relational databases: retrieving, filtering, joining and aggregating rows, and creating or changing tables. First developed in the 1970s, it remains the most widely used data skill, and most cloud warehouses and big data engines accept SQL too.

A machine learning approach where a model learns from examples that already include the correct answer, such as past loans marked as repaid or defaulted, and then predicts the answer for new cases. Most business prediction tasks, including classification and regression, use it, which makes good labeled data essential.

Using data, forecasting and optimization models to balance cost, speed and resilience across sourcing, production, inventory and delivery. Typical decisions include where to hold stock, how much to order and which routes or suppliers to use, and recent disruptions have pushed resilience up the priority list alongside cost.

A data visualization and business intelligence platform, owned by Salesforce since 2019, for building interactive charts and dashboards with drag-and-drop tools. It is known for flexible visual exploration, connects to most databases, warehouses and spreadsheets, and now includes AI features that answer questions about data in plain language.

Automatically condensing long text, such as reports, contracts, call transcripts or email threads, into shorter summaries that keep the key points. Large language models now do this well, either by selecting key sentences or writing new ones, but summaries of critical documents should still be checked against the source.

Assessing who might attack an organization, what they are after, how they would likely do it and which assets are most exposed, so defenses can be prioritized. It draws on threat intelligence, incident history and vulnerability data, and for AI systems it also covers risks such as prompt injection and data poisoning.

Using threat intelligence, past incident data and machine learning to anticipate likely attacks or risky activity before they happen, such as which vulnerabilities attackers will exploit next. Its value lies in prioritization, helping security teams patch, monitor and harden the right systems first.

The process of teaching a machine learning model by feeding it data and repeatedly adjusting its internal parameters to reduce its errors. Training is usually the most compute-intensive phase, and models are then tested on data held back from training to confirm they work on cases they have not seen.

Information that does not fit neatly into rows and columns, such as emails, documents, images, audio, video and chat logs. It makes up most of the data organizations hold, and AI tools, especially large language models, have made it far more practical to search, classify and analyze.

A machine learning approach that finds patterns in data without labeled answers, such as grouping customers into segments or flagging unusual transactions. It is useful for exploration and anomaly detection, though people still need to judge whether the patterns it finds are meaningful.

A way to rebalance an imbalanced dataset by adding more examples of the rare class, either by duplicating existing records or creating synthetic ones with methods such as SMOTE. It helps a model learn the rare cases, but it should be applied only to training data so test results stay honest.

Designing products and services around how people actually use them, covering user research, information structure, interaction and visual design, and usability testing. For dashboards and AI tools, good UX design decides whether people trust and adopt the output, not just whether the underlying analysis is correct.

A database designed to store embeddings, which are numeric representations of text, images or other content, and quickly find the items closest in meaning to a query. Vector search is a core part of retrieval-augmented generation and semantic search, offered by dedicated products and as an extension to databases like PostgreSQL.

Technology that places users in a fully computer-generated 3D environment, usually through a headset that tracks head and hand movement. Beyond gaming, it is used for training simulations, product design reviews and exploring complex models, such as buildings or factory digital twins, at life-size scale.

Collecting and analyzing data on how visitors find and use a website, including traffic sources, page views, engagement and conversions, using tools such as Google Analytics 4. Consent rules and browser privacy controls now limit tracking, so many organizations add server-side and first-party data to fill the gaps.