Data Processing - Operations / Operator

Card Puncher Data Processing


A data processing function takes an input and creates an output in a pipeline.

A data operator is also known as:


It may involve various function such as, including:

  • Validation with data type conversion – Ensuring that supplied data is “clean, correct and useful”. See also: Data Quality
  • Sorting – “arranging items in some sequence and/or in different sets.”
  • Summarization – reducing detail data to its main points.
  • Aggregation – combining multiple pieces of data.
  • Analysis – the “collection, organization, analysis, interpretation and presentation of data.”.
  • Reporting / visualization – list detail or summary data or computed information.
  • Classification – separates data into various categories.
  • Filtering - transform or delete data
  • selection - select the data field

Discover More
Card Puncher Data Processing
Data Processing - (Pipeline | Compose | Chain)

A pipeline is a finite automata where: the data transition from one state to another via a series of transformations (work) A pipeline creates a composition relationship. A pipeline is also...
Card Puncher Data Processing
Data Processing - Arc

The arcs shows the data dependencies among operators Arcs are edge in a graph data flow.
Card Puncher Data Processing
Data Processing - Filtering Operations

filtering is a data processing operation that processes data through a filter where the data may be: deleted according to a predicate from a collection transformed (map operation) Therefore, you...
Card Puncher Data Processing
Data Processing - Selection

selection is a data processing operation. finding: the min, the max the median or even any kth element in sub-linear time
Card Puncher Data Processing
Data Processing - Transformation Rule

Transformation rule are rules that are applied to data during data processing. They are implemented via data processing operations.
How does the email system work? SMTP, POP, IMAP explained

This page gives you the foundations of the email system. At the end, you should be able to understand why when you configure your email client, you need to set up: The SMTP configuration And the...
Data System Architecture

LogStash is: * a metrics collector * a log collector. * with pipeline ability A Logstash pipeline is composed of the following elements; * input (produce the data) * filter (optional, process...
Data System Architecture
Logical Data Modeling - Classification (Taxonomy | Categorization) - Transversal Data Organization

Classification is an naming technique for organization where entity or relationship gets classified by giving them a nominal attribute known as a classifier. relationshipsgroups or categories The output...
Card Puncher Data Processing
Shell Data Processing - Filter (Stream Operator)

This page is pipeline operator in a shell language. They are known as filter in a shell language. It is a computer program or shell command (subroutine) that: read from standard input (stream)...
Card Puncher Data Processing
What is Data Processing (Data Integration)?

Card puncher Data processing is a more general term for manipulating data whereas data integration is the integration...

Share this page:
Follow us:
Task Runner