maxframe.liteframe.LiteFrame#
- class maxframe.liteframe.LiteFrame(data=None, columns=None, dtype=None, chunk_size=None, gpu=None)[source]#
A performance-dedicated, lazy, distributed tabular data object with no index semantics.
LiteFrame provides a pandas-looking DataFrame API for large-scale data processing on MaxCompute. Unlike a traditional DataFrame, LiteFrame has no row index: all operations are column-based, single-column selection returns a LiteFrame (never a Series), and there is no implicit label alignment between independent frames. Arrow dtypes are used as first-class citizens so that integer columns with nulls stay nullable integer rather than being silently promoted to float.
By removing index overhead and embedding element-wise expressions in a small set of fusible operators, LiteFrame achieves lower planning cost and higher execution throughput.
Operations are lazy — they build a computation graph that is executed on demand via
.execute(). Row order is preserved through execution unless an operator explicitly reorders rows.- Parameters:
data (dict, pandas.DataFrame, tensor, or LiteFrame) – Input data. A
dictorpandas.DataFrameis converted internally to an Arrow-backed LiteFrame. A tensor is wrapped as a single-column (or multi-column) frame. An existing LiteFrame is returned as-is.columns (list of str, optional) – Column labels to use. When
datais a dict or DataFrame, this selects and orders the output columns.dtype (dtype, optional) – Data type to force on all columns.
chunk_size (int, optional) – Row-axis chunk size for distributed execution.
gpu (bool, optional) – Whether to use GPU execution.
See also
maxframe.liteframe.read_csvRead a CSV file into a LiteFrame.
maxframe.liteframe.read_parquetRead a Parquet file into a LiteFrame.
maxframe.liteframe.concatConcatenate LiteFrames along the row axis.
maxframe.liteframe.LiteFrame.from_tensorCreate a LiteFrame from a MaxFrame tensor.
Examples
>>> import maxframe.liteframe as mlf >>> lf = mlf.LiteFrame({'a': [1, 2, 3], 'b': [4, 5, 6]}) >>> lf.execute() a b 0 1 4 1 2 5 2 3 6
Methods
__init__([data, columns, dtype, chunk_size, gpu])add(other)Element-wise addition of
otherto the LiteFrame.add_prefix(prefix)Prefix all column labels with prefix.
add_suffix(suffix)Suffix all column labels with suffix.
agg([func, method, numeric_only, ...])Aggregate using one or more operations over columns.
all([method, numeric_only])Return whether all elements in each column are truthy.
any([method, numeric_only])Return whether any element in each column is truthy.
apply(func[, axis, result_type, dtypes, ...])Apply
funcper column (axis=0) or per row (axis=1).apply_chunk(func, *[, dtypes, skip_infer, ...])Apply func per execution chunk.
assign(**kwargs)Add new columns to LiteFrame.
astype(dtype[, errors])Cast a LiteFrame to a specified dtype.
between(left, right[, inclusive])Return boolean LiteFrame indicating whether each value is between left and right.
bfill([limit, columns])Backward-fill: propagate the next non-null value backward.
case_when(caselist)Replace values where conditions are True, evaluated in order.
clip([lower, upper])Trim values at input threshold(s).
copy()copy_from(obj)copy_to(target)count([method])Return the count of non-null values in each column.
cummax([skipna, columns])Cumulative max of column values, preserving row order.
cummin([skipna, columns])Cumulative min of column values, preserving row order.
cumprod([skipna, columns])Cumulative prod of column values, preserving row order.
cumsum([skipna, columns])Cumulative sum of column values, preserving row order.
diff([periods, columns])Compute
x[i] - x[i-periods]for each row.div(other)Element-wise true division (
lf / other).drop(columns[, errors])Drop columns from LiteFrame.
drop_duplicates([subset, keep, method])Return LiteFrame with duplicate rows removed.
dropna([axis, how, subset])Drop rows or columns with missing values.
duplicated([subset, keep, method])Return boolean mask marking duplicate rows.
eq(other)Element-wise equality comparison.
eval(expr[, inplace])Evaluate a string describing operations on LiteFrame columns.
execute([session])expanding([min_periods, columns, order_by, ...])Provide expanding window calculations.
explode(column)Explode list column(s) into multiple rows.
ffill([limit, columns])Forward-fill: propagate the last non-null value forward.
fillna([value])Fill missing values.
flat_apply(func, *[, dtypes, skip_infer, ...])Apply
functo each row, expanding it to 0-N output rows.floordiv(other)Element-wise floor division (
lf // other).from_local_df(df[, chunk_size, gpu, ...])Create a LiteFrame from a local pandas DataFrame or similar object.
from_tensor(tensor[, columns, gpu])Create a LiteFrame from a MaxFrame tensor.
ge(other)Element-wise greater-than-or-equal comparison.
groupby(by[, sort, dropna, group_keys])Group the LiteFrame by one or more columns.
gt(other)Element-wise greater-than comparison.
head([n])Return the first
nrows.isna()Detect missing values.
isnull()Detect missing values.
kurt([method, numeric_only])Return the unbiased kurtosis of each column.
le(other)Element-wise less-than-or-equal comparison.
lshift(other)Element-wise left bit shift (
lf << other).lt(other)Element-wise less-than comparison.
map(func, *[, na_action, dtype, skip_infer, ...])Apply func element-wise to every value in the LiteFrame.
mask(cond[, other])Replace values where cond is True with other.
max([method, numeric_only])Return the maximum of each column.
mean([method, numeric_only])Return the mean of each column.
median([method, numeric_only])Return the median of each column.
merge(right[, how, on, left_on, right_on, ...])Merge LiteFrame with another LiteFrame using a database-style join.
min([method, numeric_only])Return the minimum of each column.
mod(other)Element-wise modulo (
lf % other).mul(other)Element-wise multiplication (
lf * other).ne(other)Element-wise not-equal comparison.
notna()Detect non-missing values.
notnull()Detect non-missing values.
nunique([method, numeric_only, dropna])Return the number of distinct values in each column.
pct_change([periods, columns])Compute
(x[i] - x[i-periods]) / x[i-periods]for each row.pow(other)Element-wise exponentiation (
lf ** other).prod([method, numeric_only])Return the product of each column.
query(expr[, inplace])Query the columns of a LiteFrame with a boolean expression.
radd(other)Reverse element-wise addition (
other + lf).rdiv(other)Reverse element-wise true division (
other / lf).rebalance([axis, factor, num_partitions, rough])Make data more balanced across entire cluster.
rechunk(chunk_size[, reassign_worker, axis])Rechunk LiteFrame data.
rename(columns)Rename columns in LiteFrame.
replace([to_replace, value])Replace values in the LiteFrame.
rfloordiv(other)Reverse element-wise floor division (
other // lf).rlshift(other)Reverse element-wise left bit shift (
other << lf).rmod(other)Reverse element-wise modulo (
other % lf).rmul(other)Reverse element-wise multiplication (
other * lf).rolling(window[, min_periods, center, ...])Provide rolling-window calculations.
round([decimals])Element-wise rounding to
decimalsdecimal places.rpow(other)Reverse element-wise exponentiation (
other ** lf).rrshift(other)Reverse element-wise right bit shift (
other >> lf).rshift(other)Element-wise right bit shift (
lf >> other).rsub(other)Reverse element-wise subtraction (
other - lf).select(*args)Dispatch to select or select_expr based on argument types.
sem([method, numeric_only])Return the standard error of the mean of each column.
shift([periods, fill_value, columns])Shift rows by
periodspositions.size([method])Return the total number of rows.
skew([method, numeric_only])Return the unbiased skewness of each column.
sort([by, ascending, kind, na_position, ...])Sort the LiteFrame by the specified columns.
std([method, numeric_only])Return the sample standard deviation of each column.
sub(other)Element-wise subtraction (
lf - other).sum([method, numeric_only])Return the sum of each column.
to_csv(path[, sep, na_rep, float_format, ...])Write LiteFrame to a comma-separated values (csv) file.
to_json(path, *[, lines, orient, ...])Write a LiteFrame to JSON format (line-delimited by default).
to_lance(path, *[, mode, on, left_on, ...])Write a LiteFrame to a Lance dataset.
to_odps_table(table[, partition, ...])Write LiteFrame object into a MaxCompute (ODPS) table.
to_pandas([session])Convert the LiteFrame to a pandas DataFrame.
to_parquet(path[, engine, compression, ...])Write a LiteFrame to the binary parquet format, each chunk will be written to a Parquet file.
to_tensor([dtype])Convert the LiteFrame to a MaxFrame tensor.
transform(func[, axis, input_frame_type, ...])Transform per column (axis=0, default) or per row (axis=1).
transform_chunk(func, *[, input_frame_type, ...])Apply func per execution block (chunk).
var([method, numeric_only])Return the unbiased variance of each column.
where(cond[, other])Replace values where cond is False with other.
Attributes