bat365 Homepage

Preprocessing Data

Data cleaning, smoothing, grouping

Data can require preprocessing techniques to ensure accurate, efficient, or meaningful analysis. Data cleaning refers to methods for finding, removing, and replacing bad or missing data. Detecting local extrema and abrupt changes can help to identify significant data trends. Smoothing and detrending are processes for removing noise and polynomial trends from data, while scaling changes the bounds of the data. Grouping and binning methods identify data characteristics by groups.

Apps

Data Cleaner

Preprocess and organize column-oriented data (Since R2022a)

Live Editor Tasks

Clean Missing Data	Find, fill, or remove missing data in the Live Editor (Since R2019b)
Clean Outlier Data	Find, fill, or remove outliers in the Live Editor (Since R2019b)
Find Change Points	Find abrupt changes in data in the Live Editor (Since R2019b)
Find Local Extrema	Find local maxima and minima in the Live Editor (Since R2019b)
Smooth Data	Smooth noisy data in the Live Editor (Since R2019b)
Find and Remove Trends	Find and remove polynomial or periodic trends from data in the Live Editor (Since R2019b)
Normalize Data	Center and scale data in the Live Editor (Since R2021b)
Pivot Table	Summarize tabular data in pivoted table in the Live Editor (Since R2023b)
Compute by Group	Summarize, transform, or filter by group in the Live Editor (Since R2021b)

Functions

expand all

Missing Data and Outliers

`anymissing`	Determine if any array element is missing (Since R2022a)
`ismissing`	Find missing values
`rmmissing`	Remove missing entries
`fillmissing`	Fill missing entries
`fillmissing2`	Fill missing entries in 2-D data (Since R2023a)
`missing`	Create missing values
`standardizeMissing`	Insert standard missing values
`isoutlier`	Find outliers in data
`filloutliers`	Detect and replace outliers in data
`rmoutliers`	Detect and remove outliers in data
`movmad`	Moving median absolute deviation

Detecting Change Points and Local Extrema

`ischange`	Find abrupt changes in data
`islocalmin`	Find local minima
`islocalmax`	Find local maxima

Smoothing Data and Finding Trends

`smoothdata`	Smooth noisy data
`smoothdata2`	Smooth noisy data in two dimensions (Since R2023b)
`movmean`	Moving mean
`movmedian`	Moving median
`detrend`	Remove polynomial trend
`trenddecomp`	Find trends in data (Since R2021b)

Normalizing and Scaling Data

`normalize`	Normalize data
`rescale`	Scale range of array elements

Grouping and Binning Data

`discretize`	Group data into bins or categories
`pivot`	Summarize tabular data in pivoted table (Since R2023a)
`groupsummary`	Group summary computations
`groupcounts`	Number of group elements (Since R2019a)
`groupfilter`	Filter by group (Since R2019b)
`grouptransform`	Transform by group
`histcounts`	Histogram bin counts
`histcounts2`	Bivariate histogram bin counts
`findgroups`	Find groups and return group numbers
`splitapply`	Split data into groups and apply function
`rowfun`	Apply function to table or timetable rows
`varfun`	Apply function to table or timetable variables
`accumarray`	Accumulate vector elements

Topics

Clean Messy and Missing Data in Tables
Standardize, fill, or remove missing values in tables, and reorganize tables by sorting rows and moving variables.
Split Data into Groups and Calculate Statistics
This example shows how to group data and apply statistics functions to each group.
Perform Calculations by Group in Table
Specify groups of data in tables and timetables, and perform calculations by group. Choose a function for group calculations using these recommendations.