maxframe.learn.tree.DecisionTreeClassifier#

class maxframe.learn.tree.DecisionTreeClassifier(*, criterion='gini', max_depth=None, max_bins=32, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, min_impurity_decrease=0.0, max_memory_in_mb=256)[source]#

A distributed decision tree classifier.

This estimator implements a distributed version of the decision tree classifier using approximate split finding with feature binning, following the approach described in [1]. The algorithm is designed for large-scale datasets that do not fit in a single machine’s memory and runs on MaxCompute’s distributed execution engine.

Unlike scikit-learn’s DecisionTreeClassifier which uses an exact greedy algorithm to find optimal split points, this implementation discretizes continuous features into bins and finds approximate splits. This trades a small amount of split quality for the ability to efficiently aggregate statistics across distributed partitions.

Parameters:

  • criterion ({"gini", "entropy"}, default="gini") – The function to measure the quality of a split. Supported criteria are “gini” for the Gini impurity and “entropy” for the information gain.

  • max_depth (int, default=None) – The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples. Note that very deep trees may require many distributed communication rounds.

  • max_bins (int, default=32) –

    The maximum number of bins used to discretize continuous features. Higher values improve split approximation quality at the cost of increased memory usage and communication overhead. This parameter has no equivalent in scikit-learn and is specific to the distributed binning algorithm.

    Memory usage per node group is approximately n_features * max_bins * n_classes * 8 bytes. When the memory budget (controlled by max_memory_in_mb) is exceeded, fewer nodes are processed per iteration, increasing the number of data passes required.

  • min_samples_split (int or float, default=2) –

    The minimum number of samples required to split an internal node:

    • If int, then consider min_samples_split as the minimum number.

    • If float, then min_samples_split is a fraction and ceil(min_samples_split * n_samples) are the minimum number of samples for each split.

  • min_samples_leaf (int or float, default=1) –

    The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least min_samples_leaf training samples in each of the left and right branches.

    • If int, then consider min_samples_leaf as the minimum number.

    • If float, then min_samples_leaf is a fraction and ceil(min_samples_leaf * n_samples) are the minimum number of samples for each node.

  • min_weight_fraction_leaf (float, default=0.0) – The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided.

  • max_features (int, float, {"sqrt", "log2"} or None, default=None) –

    The number of features to consider when looking for the best split:

    • If int, then consider max_features features at each split.

    • If float, then max_features is a fraction and max(1, int(max_features * n_features)) features are considered.

    • If “sqrt”, then max_features=sqrt(n_features).

    • If “log2”, then max_features=log2(n_features).

    • If None, then max_features=n_features.

  • random_state (int, RandomState instance or None, default=None) – Controls the randomness of the estimator. The features are always randomly permuted at each split. When max_features < n_features, the algorithm will select max_features at random at each split before finding the best split among them. To obtain a deterministic behaviour during fitting, random_state has to be fixed to an integer.

  • min_impurity_decrease (float, default=0.0) –

    A node will be split if this split induces a decrease of the impurity greater than or equal to this value.

    The weighted impurity decrease equation is the following:

    N_t / N * (impurity - N_t_R / N_t * right_impurity
                    - N_t_L / N_t * left_impurity)

    where N is the total number of samples, N_t is the number of samples at the current node, N_t_L is the number of samples in the left child, and N_t_R is the number of samples in the right child.

    N, N_t, N_t_R and N_t_L all refer to the weighted sum, if sample_weight is passed.

  • max_memory_in_mb (int, default=256) – The maximum memory in MB to use for aggregating statistics for a group of nodes during training. If the memory budget is exceeded, fewer nodes are processed per iteration, increasing the total number of data passes. This parameter has no equivalent in scikit-learn and is specific to the distributed algorithm.

classes_#

The class labels (single output problem), or a list of arrays of class labels (multi-output problem).

Type:

ndarray of shape (n_classes,) or a list of such arrays

n_classes_#

The number of classes (single output problem), or a list containing the number of classes for each output (multi-output problem).

Type:

int or list

feature_importances_#

The impurity-based feature importances. Currently returns all zeros (not yet implemented). Tracked as a known limitation.

Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values).

Type:

ndarray of shape (n_features,)

max_features_#

The inferred value of max_features.

Type:

int

n_features_in_#

Number of features seen during fit.

Type:

int

feature_names_in_#

Names of features seen during fit. Defined only when X has feature names that are all strings.

Type:

ndarray of shape (n_features_in_,)

n_outputs_#

The number of outputs when fit is performed.

Type:

int

tree_#

The underlying Tree object. Please refer to help(maxframe.learn.tree._tree.Tree) for attributes of the Tree object.

Type:

Tree instance

See also

DecisionTreeRegressor

A distributed decision tree regressor.

RandomForestClassifier

A distributed random forest classifier.

Notes

Distributed Algorithm

This implementation uses a level-wise (breadth-first) training approach with approximate split finding, following the algorithm described in [1]:

  1. Continuous features are discretized into max_bins bins using approximate quantile thresholds computed from a sample of the data.

  2. At each level of the tree, sufficient statistics (per-bin class counts for classification, or sum/variance for regression) are aggregated across all distributed partitions and then reduced via a shuffle operation.

  3. Best splits are found by evaluating impurity improvement at each bin boundary using the merged statistics.

  4. Multiple nodes at the same level are processed simultaneously, grouped to fit within the max_memory_in_mb budget.

Limitations compared to scikit-learn

The following features are not supported due to fundamental incompatibility with the distributed level-wise training algorithm:

  • splitter: Only “best” splitting is supported. The “random” splitter (used by ExtraTrees) is not available because the binning approach already provides a form of discretization.

  • max_leaf_nodes: Best-first tree growth is incompatible with level-wise distributed training. Use max_depth to control tree size instead.

  • ccp_alpha: Cost-complexity pruning is not currently supported.

The following features are deferred to future versions:

  • monotonic_cst: Monotonicity constraints.

  • class_weight: Class weight balancing.

  • oob_score: Out-of-bag scoring (only applies to forests).

Performance Considerations

  • The number of distributed communication rounds equals the tree depth. Deep trees (max_depth > 15) may be expensive.

  • Increasing max_bins improves split quality but increases per-round memory and communication. For most datasets, 32-64 bins are sufficient.

  • The max_memory_in_mb parameter controls how many nodes are processed per iteration. Lower values increase the number of data passes but prevent out-of-memory errors.

References

Examples

>>> from maxframe.learn.tree import DecisionTreeClassifier
>>> from maxframe import new_session
>>> from odps import ODPS
>>> o = ODPS(...)
>>> session = new_session(o)
>>> clf = DecisionTreeClassifier(max_depth=5, max_bins=32)
>>> clf.fit(X_train, y_train)
DecisionTreeClassifier(max_bins=32, max_depth=5)
>>> predictions = clf.predict(X_test)
>>> predictions.execute()
>>> predictions.fetch()
__init__(*, criterion='gini', max_depth=None, max_bins=32, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, min_impurity_decrease=0.0, max_memory_in_mb=256)[source]#

Methods

__init__(*[, criterion, max_depth, ...])

execute([session, run_kwargs, extra_tileables])

fetch([session, run_kwargs])

fit(X, y[, sample_weight, execute, session, ...])

Fit a distributed decision tree classifier.

predict(X[, execute, session, run_kwargs])

Predict class for X.

predict_proba(X[, execute, session, run_kwargs])

Predict class probabilities for X.

score(X, y[, sample_weight])

Return the mean accuracy on the given test data and labels.

Attributes

tree_

The underlying Tree object.