Skip to content

Preprocessing Nodes

This page documents all preprocessing node types supported by Skyulf's NodeRegistry (resolved by FeatureEngineer at runtime).

Step schema

Every preprocessing step in the pipeline config uses:

{
  "name": "...",
  "transformer": "TransformerType",
  "params": { ... }
}

Where params is passed into the node's Calculator fit().

Decimal numeric columns

Numeric auto-selection includes Polars Decimal(precision, scale) columns and pandas object columns whose non-missing values are all decimal.Decimal. Numeric strings and mixed Decimal/string columns remain outside automatic numeric selection. Existing exclusions for booleans, binary values, constants and all-missing columns still apply according to the node's selection rules.

Selected Decimal values use float64 for numerical transformations such as scaling, power transforms and mean/median imputation. This has ordinary floating point precision, rather than exact decimal arithmetic. Source frames and unselected columns retain their original values; outlier filtering retains the original values of kept rows and their matching targets.

For example, MinMaxScaler with params={} now scales Decimal prices [100.00, 200.00, 300.00, 400.00] to [0.0, 0.3333, 0.6667, 1.0] on both engines. An explicit columns=[] still makes the scaler a no-op.

Refit pipelines whose previous automatic selection skipped Decimal columns. Existing fitted artifacts keep their original selection when replayed.

Splitters

Example step:

{"name": "split", "transformer": "TrainTestSplitter", "params": {"test_size": 0.2, "random_state": 42, "target_column": "target"}}

TrainTestSplitter

Splits a DataFrame (or (X, y) tuple) into SplitDataset(train, test, validation).

Config (params):

  • test_size: float (default 0.2)
  • validation_size: float (default 0.0)
  • random_state: int (default 42)
  • shuffle: bool (default True)
  • stratify: bool (default False)
  • target_column: str (required only when splitting a DataFrame and using stratify)

The node copies only these six settings into its fitted artifact. Unknown public keys produce a warning and are ignored; they do not change the split. For example, stratify_col="target" is a DataSplitter constructor argument, not a node setting. Use stratify=True and target_column="target" here. The artifact's type field and private underscore-prefixed routing metadata are ignored without warnings. Canvas sends the supported settings separately from its editor labels and identifiers.

Learned params: none (records the recognized config settings).

feature_target_split

Splits a DataFrame into (X, y) (or applies the split to each SplitDataset split).

Config:

  • target_column: str (required)

Learned params: none.

Cleaning

When columns is omitted, TextCleaning and AliasReplacement select text-like columns, including pandas categorical strings and Polars Enum columns. An explicit columns=[] keeps both nodes as no-ops. Enum categories do not restrict the cleaned output: for example, trim plus lowercase changes " YES " to "yes", while boolean alias replacement produces "Yes". Missing values stay missing and source frames are unchanged. Refit saved artifacts to include Enum columns that older automatic selections skipped.

Automatic text selection excludes pandas object columns containing Decimal numbers, matching native Polars Decimal behavior. Amounts such as Decimal("12.34") retain their values and dtype while nearby text is cleaned. Refit saved text-cleaning steps that selected Decimal columns and rerun from the original data; old artifacts keep their stored column selections.

Example step:

{
  "name": "clean_text",
  "transformer": "TextCleaning",
  "params": {
    "columns": ["free_text"],
    "operations": [
      {"op": "trim", "mode": "both"},
      {"op": "case", "mode": "lower"},
      {"op": "regex", "mode": "collapse_whitespace"}
    ]
  }
}

TextCleaning

Applies a list of string operations.

Config:

  • columns: list[str] (optional; auto-detects text-like columns)
  • operations: list[dict]
  • { "op": "trim", "mode": "both"|"leading"|"trailing" }
  • { "op": "case", "mode": "lower"|"upper"|"title"|"sentence" }
  • { "op": "remove_special", "mode": "keep_alphanumeric"|"keep_alphanumeric_space"|"letters_only"|"digits_only", "replacement": "" }
  • { "op": "regex", "mode": "collapse_whitespace"|"extract_digits"|"custom", "pattern": "...", "repl": "..." }

Learned params:

  • columns
  • operations

ValueReplacement

Replaces values in selected columns.

Config:

  • columns: list[str]
  • Either:
  • mapping: dict (global mapping) or dict[col -> mapping]
  • to_replace + value
  • replacements: list of {old, new} (converted into a mapping)

Learned params:

  • columns, mapping, to_replace, value

AliasReplacement

Normalizes common textual aliases (boolean/country/custom).

Config:

  • columns: list[str] (optional; auto-detects text-like columns)
  • alias_type: boolean | country | custom (also supports legacy mode)
  • custom_map: dict[str, str] (also supports legacy custom_pairs)

Learned params:

  • columns, alias_type, custom_map

InvalidValueReplacement

Replaces invalid numeric values.

Config:

  • columns: list[str]
  • rule: negative | zero | negative_to_nan | custom_range (also supports legacy mode)
  • replacement: any (default NaN)
  • min_value / max_value: used by custom_range

Active numeric rules and infinity replacement require numeric selected columns on both engines. Text, categorical, Boolean and temporal selections raise a ValueError naming the columns; convert numeric text explicitly before this node. The same check runs when applying a fitted artifact, so inference data that changes a selected column to text fails clearly without coercing values. Automatic selection excludes temporal columns. An empty selection or a configuration with no active rule or infinity replacement leaves data untouched.

Enabling replace_inf or replace_neg_inf alone leaves integer columns unchanged, including nullable signed/unsigned types and values larger than Float64 can represent exactly. Floating-point infinities still use the configured replacement, and an additional numeric rule still applies to integer columns.

Learned params:

  • columns, rule, replacement, min_value, max_value

Drop & Missing

Deduplicate

Config:

  • subset: list[str] | None
  • keep: first | last | none (mapped to False)

Learned params:

  • subset, keep

DropMissingColumns

Config:

  • missing_threshold: float (percentage; if > 0, drops columns with missing% >= threshold)
  • columns: list[str] (explicit columns to drop)

Learned params:

  • columns_to_drop, threshold

DropMissingRows

Config:

  • subset: list[str] | None
  • how: any | all (default any; filtering uses threshold when provided)
  • threshold: int | None (min non-null values)

Explicit null or unsupported how values raise ValueError during fitting and saved-artifact replay, even when a threshold currently takes precedence. Omit how to use the default; use only any or all when setting it explicitly.

Learned params:

  • subset, how, threshold

MissingIndicator

Adds {col}_missing indicator columns by default.

An explicit columns list selects only names present in the input. Missing names produce no flag in either execution or the predicted output schema. For example, selecting ghost when the input only contains x does not create ghost_missing; downstream references to that flag remain invalid. Automatic selection depends on the actual missing values and has no static output schema.

Generated names must be unique and must not collide with existing columns. Fitting and applying raise ValueError on a collision, including one introduced by new inference data. Rename the conflicting input column or change the selected columns.

Config:

  • columns: list[str] (optional; defaults to all columns with any missing values)
  • flag_suffix: str (default _missing)

Learned params:

  • columns, flag_suffix

Imputation

Example step:

{"name": "impute", "transformer": "SimpleImputer", "params": {"strategy": "median", "columns": ["age"]}}

SimpleImputer

Mean and median require numeric columns. An explicit selection containing text, including numeric-looking strings, raises a ValueError naming the incompatible columns on both engines. Select numeric columns, cast numeric strings first, or choose most_frequent/constant for text. Explicit binary, constant and Decimal numeric selections remain supported; automatic selection is unchanged.

Config:

  • strategy: mean | median | most_frequent | constant (also accepts mode)
  • columns: list[str] (optional; numeric auto-detection for mean/median)
  • fill_value: any (used for constant)

Learned params:

  • columns
  • strategy
  • fill_values: dict[col -> value]
  • missing_counts: dict[col -> count]
  • total_missing: int

KNNImputer

Config:

  • columns: list[str] (numeric)
  • n_neighbors: int (default 5)
  • weights: uniform | distance

Learned params:

  • columns
  • imputer_object (sklearn object; pickled in pipeline)
  • n_neighbors, weights

GroupImputer

Fills missing values from the statistic of the row's own group, for example the median employee count of the same industry. Statistics are learned on training rows only; a group that was not seen in training (or had no observed value) uses the overall training value. The group column itself is never filled.

Config:

  • group_by: str (required; the column that defines the groups)
  • strategy: mean | median | most_frequent (also accepts mode; constant is rejected — use SimpleImputer)
  • columns: list[str] (optional; numeric auto-detection for mean/median, all other columns for most_frequent)

Learned params:

  • group_by, strategy, columns
  • group_values: dict[col -> list of [group, value]]
  • fill_values: dict[col -> overall value]

Example step:

{"name": "fill_employees", "transformer": "GroupImputer",
 "params": {"columns": ["employees"], "group_by": "industry", "strategy": "median"}}

IterativeImputer

Config:

  • columns: list[str] (numeric)
  • max_iter: int (default 10)
  • estimator: BayesianRidge | DecisionTree | ExtraTrees | KNeighbors (matched case-insensitively; the canvas aliases bayesian_ridge, decision_tree, extra_trees, knn are accepted too)

Learned params:

  • columns
  • imputer_object (sklearn object; pickled in pipeline)
  • estimator

Encoding

When columns is omitted, the shared categorical selector includes native Polars Enum columns as well as strings and categorical columns. Refit a pipeline that previously skipped Enum inputs to learn their encoding.

Example step:

{"name": "encode", "transformer": "OneHotEncoder", "params": {"columns": ["city"], "drop_original": True, "handle_unknown": "ignore"}}

OneHotEncoder

Generated names must be unique and must not collide with retained input columns. Fitting and applying raise ValueError before returning ambiguous columns or replacing existing data. This also checks new inference columns and older fitted artifacts. Rename conflicting inputs or categories before encoding. A selected source that is removed by drop_original=True does not reserve its name; other retained columns do.

Config:

  • columns: list[str] (optional; auto-detects categorical columns)
  • drop_first: bool (default False)
  • max_categories: int (default 20)
  • handle_unknown: ignore | error (default ignore)
  • drop_original: bool (default True)
  • include_missing: bool (default False)

Learned params:

  • columns
  • encoder_object (sklearn OneHotEncoder)
  • feature_names: list[str]
  • drop_original, include_missing

DummyEncoder

Uses the same collision checks as OneHotEncoder. Generated columns are computed from the original source values before selected sources are dropped.

Newly fitted artifacts use stable Boolean and date/datetime category names on both engines, including nanosecond timestamps and timezone-aware instants. Naive midnight timestamps match equivalent dates; aware timestamps are normalized to UTC. Literal string categories keep their exact text. Older artifacts retain their original names and rendering. Refit the encoder and downstream model together before switching engines if a saved pipeline depends on the earlier Boolean or datetime names.

Config:

  • columns: list[str]
  • drop_first: bool

Learned params:

  • columns
  • categories: dict[col -> list[str]]
  • drop_first
  • category_key_version: identifies the rendering contract in new artifacts

OrdinalEncoder

Config:

  • columns: list[str]
  • handle_unknown: str (default use_encoded_value)
  • unknown_value: int/float (default -1)

Learned params:

  • columns
  • encoder_object (sklearn OrdinalEncoder)
  • categories_count

LabelEncoder

Encodes either target or selected feature columns.

Config:

  • columns: optional list[str]
  • if omitted, encodes the provided target y
  • if provided, encodes those feature columns (and also target if included)

Learned params:

  • encoders: dict[col or "target" -> sklearn LabelEncoder]
  • classes_count

Targets may be lists, NumPy arrays or a Series from the feature dataframe's engine. Encoded targets become native integer Series; existing Series names and pandas indexes are preserved. For example, fitting on ["yes", "no", "yes"] produces [1, 0, 1]. Feature-only encoding leaves the target container and values unchanged. Unknown target labels still use the configured missing_code.

TargetEncoder

Requires a target series (y).

In a FeatureEngineer or SkyulfPipeline, training rows are encoded with sklearn's cross-fitted TargetEncoder.fit_transform behavior. Test, validation, and inference rows use the fitted encoder's transform behavior. Skyulf uses deterministic five-fold cross-fitting with seed 42 whenever the training split supports it; smaller leakage-safe splits deterministically shrink to the largest supported fold count (for classification, capped by the smallest class size; for regression, capped by the training-row count). A one-row split, or a classification split where any target class appears only once, raises a clear error instead of leaking the row's target into its encoded value. Direct Calculator/Applier use remains an advanced API and uses the explicit fit/apply calls supplied by the caller.

Multiclass output names must be unique and must not collide with retained input columns. A collision raises ValueError during fitting, cross-fitted training, or later application, including with older artifacts. Rename the conflicting input column before encoding. Binary and regression encodings continue to replace their selected source columns.

Config:

  • columns: list[str]
  • smooth: auto or numeric
  • target_type: auto or explicit type

Learned params:

  • columns
  • encoder_object (sklearn TargetEncoder)

HashEncoder

Config:

  • columns: list[str]
  • n_features: int (default 10)

Learned params:

  • columns, n_features

Scaling

All scaling nodes accept columns (optional; numeric auto-detect) and return learned numeric arrays.

StandardScaler

Config:

  • columns: list[str]
  • with_mean: bool (default True)
  • with_std: bool (default True)

Learned params:

  • columns, mean, scale, var, with_mean, with_std

MinMaxScaler

Config:

  • columns: list[str]
  • feature_range: tuple (default (0, 1))

Learned params:

  • columns, min, scale, data_min, data_max, feature_range

RobustScaler

Config:

  • columns: list[str]
  • quantile_range: tuple (default (25.0, 75.0))
  • with_centering: bool (default True)
  • with_scaling: bool (default True)

Learned params:

  • columns, center, scale, quantile_range, with_centering, with_scaling

MaxAbsScaler

Config:

  • columns: list[str]

Learned params:

  • columns, scale, max_abs

Outliers

IQR

Filters rows outside per-column IQR bounds.

Config:

  • columns: list[str]
  • multiplier: float (default 1.5)

Learned params:

  • bounds: dict[col -> {lower, upper}]
  • warnings

ZScore

Config:

  • columns: list[str]
  • threshold: float (default 3.0)

Learned params:

  • stats: dict[col -> {mean, std}]
  • threshold, warnings

Winsorize

Clips values into per-column percentile bounds.

Config:

  • columns: list[str]
  • lower_percentile: float (default 5.0)
  • upper_percentile: float (default 95.0)

Learned params:

  • bounds: dict[col -> {lower, upper}]

ManualBounds

Filters rows outside user-provided bounds.

Config:

  • bounds: dict[col -> {lower, upper}]

Learned params:

  • bounds

ClipValues

Caps values at fixed, user-provided bounds and keeps every row. Unlike Winsorize, the limits are not learned from data; unlike ManualBounds, rows are not dropped, so the step also runs at prediction time.

Config:

  • bounds: dict[col -> {lower?, upper?}] (at least one side per column; finite numbers; numeric columns only)

Learned params:

  • bounds

Example step:

{"name": "cap_revenue", "transformer": "ClipValues",
 "params": {"bounds": {"revenue": {"lower": 0, "upper": 1_000_000}}}}

EllipticEnvelope

Learns a per-column EllipticEnvelope model and filters outliers.

Config:

  • columns: list[str]
  • contamination: float (default 0.01)

Learned params:

  • models: dict[col -> sklearn model]
  • contamination, warnings

Transformations

PowerTransformer

Config:

  • columns: list[str]
  • method: yeo-johnson | box-cox
  • standardize: bool

Learned params:

  • columns, lambdas, method, standardize, scaler_params

SimpleTransformation

Config:

  • transformations: list of {column, method, clip_threshold?}
  • methods include log, square_root, cube_root, reciprocal, square, exponential

Learned params:

  • transformations (passes through)

GeneralTransformation

Rules execute in list order, including when several rules target one column. A power rule learns from the result of the preceding rules on that column. For example, log followed by standardized yeo-johnson learns from the logged training values and produces a training mean near zero and standard deviation one. Later data reuses those fitted parameters. Existing artifacts keep their saved parameters; refit affected transformation steps and downstream models to correct previously learned statistics.

Config:

  • transformations: list of {column, method, standardize?, clip_threshold?}
  • methods include power transforms (box-cox, yeo-johnson) and the simple methods
  • standardize defaults to true for each power rule. Set it to false (clear Standardize result in the Canvas) to apply the power transform without centering and scaling its output.

Learned params:

  • transformations with fitted lambdas, the standardize choice and scaler_params when standardization is enabled. Later transforms reuse these training parameters; older artifacts without standardize default to true.

Bucketing (Binning)

GeneralBinning

Creates binned features with configurable strategies.

Generated output names must be unique and must not overwrite retained input columns. This is checked during fitting and later application. An empty suffix can replace the selected source only with drop_original=True; the binned output is retained. When duplicate edges collapse into fewer intervals, custom labels must describe those distinct intervals, not the discarded zero-width ones; otherwise fitting or applying raises ValueError.

Newly fitted range-label artifacts save their category text so pandas and Polars replay identical names. If the requested rounding would give different bins the same name, the stored labels use exact edge text to distinguish them. Older artifacts keep their original formatting. Refit binning and downstream encoders/models together before moving an affected legacy pipeline between engines.

Config:

  • columns: list[str] (numeric)
  • strategy: equal_width | equal_frequency | kmeans | custom | kbins
  • n_bins and strategy-specific keys:
  • equal_width_bins, equal_frequency_bins, duplicates
  • kbins_n_bins, kbins_strategy
  • custom_bins: dict[col -> edges]
  • custom_labels: dict[col -> labels]
  • output formatting:
  • output_suffix, drop_original, label_format, missing_strategy, missing_label, include_lowest, precision

missing_strategy="label" replaces missing and out-of-range bins with missing_label (default "Missing"). This includes nulls, NaN and values outside the fitted edges during later transformations. With this strategy Polars always returns strings, including in batches with no missing values: ordinal/bin-index codes become "0", "1", etc. Pandas keeps numeric codes alongside the text label in an object column. Range and custom labels retain their text. The default missing_strategy="keep" leaves missing bins as missing values.

Learned params:

  • bin_edges (dict[col -> edges])
  • range_labels (stored range text in newly fitted range-label artifacts)
  • output formatting settings

CustomBinning

Config:

  • columns: list[str]
  • bins: list[float] (shared edges)
  • plus output formatting keys (same as GeneralBinning)

Learned params:

  • bin_edges

KBinsDiscretizer

Wrapper around GeneralBinning with a KBins-style interface.

Config:

  • columns: list[str]
  • n_bins: int
  • strategy: uniform | quantile | kmeans

Learned params:

  • bin_edges

Casting

Casting

Config:

  • Either:
  • column_types: dict[col -> dtype]
  • or columns + target_type
  • coerce_on_error: bool (default True)

Text-to-datetime casts preserve valid date-only and timestamp strings on both engines. Invalid dates become missing by default; coerce_on_error=False raises. Boolean conversion recognizes the existing text aliases in Polars Categorical and Enum columns too, preserving nulls and invalid-token handling.

Learned params:

  • type_map, coerce_on_error

Feature Generation

PolynomialFeatures

(Alias: PolynomialFeaturesNode)

Config:

  • columns: list[str]
  • auto_detect: bool
  • degree: int
  • interaction_only: bool
  • include_bias: bool
  • include_input_features: bool
  • output_prefix: str

Learned params:

  • columns, degree, interaction_only, include_bias, include_input_features, output_prefix, feature_names

FeatureInteraction

(Alias: FeatureInteractionNode)

Generates 2-way/3-way/4-way multiplicative interaction columns between numeric columns. Interaction column names are deterministic and regularization-friendly: sorted column names joined with _x_ (e.g. x1_x_x2, x1_x_x2_x_x3, x1_x_x2_x_x3_x_x4), so the same combination of inputs always produces the same output name regardless of input column order.

Config:

  • columns: list[str]
  • degree: int (2, 3, or 4, default 2)
  • interaction_only: bool (default True; skips self-products like x1 * x1)
  • include_bias: bool (default False; adds a constant interaction_bias column of 1.0)

With interaction_only=False (clear Interaction Only in the Canvas), one column can generate its square, cube or fourth power. Two columns at degree 3 generate a*a*a, a*a*b, a*b*b and b*b*b. With interaction_only=True, at least degree distinct columns are needed to generate products.

Learned params:

  • columns, degree, interaction_only, include_bias, combinations, feature_names

FeatureGeneration

(Aliases: FeatureMath, FeatureGenerationNode)

Config:

  • operations: list[dict]
  • epsilon: float (default 1e-9)
  • allow_overwrite: bool

Supported operation_type values are arithmetic (the default), ratio, similarity, datetime_extract, and group_agg. Unsupported types raise ValueError during fitting and when replaying saved artifacts. In particular, polynomial is not a Feature Generation operation: use the separate PolynomialFeatures node for powers and interactions.

Arithmetic operations (add, subtract, multiply, divide) replace null and NaN operands with the operation's fillna value on both engines. Omitted or null fillna defaults to zero. For example, adding 2 to [1, NaN, 3] with fillna=10 produces [3, 12, 5]. Source columns retain their values. Rerun affected transformations and refit downstream models if they previously used arithmetic features containing unfilled Polars NaNs.

For ratio, the denominator is the sum of the selected denominator columns. If its absolute value is below epsilon, it is replaced by -epsilon when negative and +epsilon otherwise (including zero). This preserves the ratio's sign on both engines; for example, 1 / -1e-12 with the default epsilon produces -1e9. The saved epsilon is reused when applying a fitted node.

Missing operands (NaN or null) contribute zero to each ratio sum on both engines. Other selected operands still contribute normally: (NaN + 2) / 4 produces 0.5. An entirely missing denominator uses positive epsilon, and an entirely missing numerator sums to zero. Source columns retain their values; this rule applies to the generated ratio.

For datetime_extract, output_column names a single source/feature result exactly. With one source and multiple features, it becomes a prefix: output_column="calendar" with ["year", "month"] produces calendar_year and calendar_month. With multiple sources, names also include the source, such as calendar_start_year. Without output_column, the default is source_feature; an optional output_prefix is prepended to that default.

Datetime outputs now follow allow_overwrite (default False): a collision adds _1, _2, and so on until the name is available. Setting it to True permits replacement. For example, an existing dt_year is retained and a new year feature becomes dt_year_1. Names are resolved against the frame being transformed, as for other Feature Generation operations. Refit affected saved pipelines and downstream models if they depended on previously ignored names or overwritten columns.

For group_agg, null and NaN keys share a single training group on both engines. Later transformations reuse that fitted aggregate, including for missing keys; they never calculate it from the new batch's values. Keys not seen during training, including missing keys when training had none, produce missing output. The operation groups by one column. Legacy artifacts without fitted group statistics must be refitted before applying them.

Learned params:

  • operations, epsilon, allow_overwrite

Geo

GeoDistance

Computes the great-circle (haversine) or flat-plane (euclidean) distance between two lat/lon coordinate pairs and appends it as a new numeric column. Pure math — no optional geospatial dependency required, and runs natively on both the pandas and polars engines.

Haversine calculations remain finite for valid antipodal points: opposite points on Earth yield approximately 20015.114442 km. Missing coordinates remain missing rather than becoming a valid distance.

Config:

  • lat1_col: str
  • lon1_col: str
  • lat2_col: str
  • lon2_col: str
  • method: str ("haversine" default, or "euclidean" for a cheap flat-plane approximation)
  • unit: str ("km" default, or "mi")
  • output_column: str (default "", meaning automatic). Omit it or leave it empty to use "geo_distance_km" for kilometers or "geo_distance_mi" for miles.

The resolved name is stored during fitting and reused during inference. An explicit name always takes precedence, including names in existing saved artifacts. To retain an old miles pipeline that refers to "geo_distance_km", set that name explicitly before refitting; otherwise update downstream column references to "geo_distance_mi".

Learned params:

  • lat1_col, lon1_col, lat2_col, lon2_col, method, unit, output_column

H3Index

Computes the Uber H3 hierarchical hexagonal grid cell index for each lat/lon pair and appends it as a new string column, using h3.latlng_to_cell. Requires the optional h3 package (pip install skyulf-core[geo] or pip install h3); Calculator.fit() raises a clear ImportError if it isn't installed. The pandas engine computes row-wise (h3 has no vectorized API); the polars engine converts to pandas internally and back.

Config:

  • lat_col: str
  • lon_col: str
  • resolution: int (0-15, H3 grid resolution, default 9)
  • output_column: str (default "h3_index")

Learned params:

  • lat_col, lon_col, resolution, output_column

Feature Selection

VarianceThreshold

Config:

  • threshold: float (default 0.0)
  • columns: list[str] (optional; numeric)
  • drop_columns: bool (default True)

Learned params:

  • candidate_columns, selected_columns, variances, threshold, drop_columns

If every candidate is constant, all-null or at/below the threshold, the fitted selection is empty. Applying it removes all candidate columns while preserving unselected columns and the target; drop_columns=False preserves the input. Prediction reuses this fitted selection even if incoming values now vary. If no model features remain, adjust the selection before a downstream training step.

CorrelationThreshold

In the Canvas, choose Feature Selection > Correlation Threshold to remove highly correlated input features without selecting a target column. This method compares features with each other; supervised selection methods still require a target.

Config:

  • threshold: float (default 0.95)
  • correlation_method: pearson | spearman | kendall (default pearson)
  • columns: list[str] (numeric)
  • drop_columns: bool

Learned params:

  • columns_to_drop, threshold, method, drop_columns

UnivariateSelection

With problem_type="auto", string targets use classification whether stored as pandas object, StringDtype or string Categorical columns. Numeric targets retain the cardinality heuristic: up to ten distinct values means classification. Set problem_type explicitly to override inference. The same rules apply to ModelBasedSelection and the feature-selection facade.

Config:

  • target_column: str (if y not passed as tuple)
  • problem_type: auto | classification | regression
  • method: select_k_best | select_percentile | select_fpr | select_fdr | select_fwe | generic_univariate_select
  • selector parameters (depending on method): k, percentile, alpha, mode, param
  • scoring: score_func (e.g., f_classif, mutual_info_classif, …)
  • drop_columns: bool

Learned params:

  • selected_columns, candidate_columns, scores, pvalues (when available), plus selector config

ModelBasedSelection

Config:

  • target_column: str
  • problem_type: auto | classification | regression
  • method: select_from_model | rfe
  • estimator: auto | logistic_regression | random_forest | linear_regression
  • For select_from_model: threshold, max_features
  • For RFE: n_features_to_select, step
  • drop_columns: bool

Learned params:

  • selected_columns, candidate_columns, and method-specific metadata

feature_selection

A higher-level facade node that dispatches to the selection implementations.

Resampling

Oversampling

Config:

  • method: smote | adasyn | borderline_smote | svm_smote | kmeans_smote | smote_tomek
  • target_column: required if y is not provided as tuple
  • sampling_strategy: auto or dict
  • random_state: int
  • method-specific keys: k_neighbors, m_neighbors, kind, out_step, cluster_balance_threshold, density_exponent, n_jobs

For smote_tomek, k_neighbors configures the inner SMOTE step together with the chosen sampling strategy and random state. Tomek cleanup still runs after oversampling, so final class sizes can be smaller than the requested synthetic sample counts. Canvas exposes this setting as k Neighbors for SMOTE + Tomek and requires at least one neighbor. A class must contain more observations than its configured neighbor count.

Learned params: none (passes through config).

Undersampling

Config:

  • method: random_under_sampling | nearmiss | tomek_links | edited_nearest_neighbours
  • target_column: required if y not provided as tuple
  • sampling_strategy, random_state, replacement, version, n_neighbors, kind_sel, n_jobs

Learned params: none.

Inspection

DatasetProfile

Captures basic dataset stats without modifying data.

Numeric statistics cover supported signed/unsigned integer, float and Decimal columns on both engines. Binary, constant and entirely missing numeric columns remain visible, as do typed numeric columns in an empty frame. Boolean, categorical and temporal columns retain their metadata but have no numeric statistics. Statistics still use each engine's native describe() metrics and quantile conventions. Rerun the node to refresh older profile artifacts.

Config: none.

Learned params:

  • profile: rows/columns/dtypes/missing/numeric_stats

DataSnapshot

Captures the first N rows without modifying data.

Config:

  • n_rows: int (default 5)

Learned params:

  • snapshot: list[dict]