maxframe.liteframe.LiteFrame#

class maxframe.liteframe.LiteFrame(data=None, columns=None, dtype=None, chunk_size=None, gpu=None)[source]#

A performance-dedicated, lazy, distributed tabular data object with no index semantics.

LiteFrame provides a pandas-looking DataFrame API for large-scale data processing on MaxCompute. Unlike a traditional DataFrame, LiteFrame has no row index: all operations are column-based, single-column selection returns a LiteFrame (never a Series), and there is no implicit label alignment between independent frames. Arrow dtypes are used as first-class citizens so that integer columns with nulls stay nullable integer rather than being silently promoted to float.

By removing index overhead and embedding element-wise expressions in a small set of fusible operators, LiteFrame achieves lower planning cost and higher execution throughput.

Operations are lazy — they build a computation graph that is executed on demand via .execute(). Row order is preserved through execution unless an operator explicitly reorders rows.

Parameters:
  • data (dict, pandas.DataFrame, tensor, or LiteFrame) – Input data. A dict or pandas.DataFrame is converted internally to an Arrow-backed LiteFrame. A tensor is wrapped as a single-column (or multi-column) frame. An existing LiteFrame is returned as-is.

  • columns (list of str, optional) – Column labels to use. When data is a dict or DataFrame, this selects and orders the output columns.

  • dtype (dtype, optional) – Data type to force on all columns.

  • chunk_size (int, optional) – Row-axis chunk size for distributed execution.

  • gpu (bool, optional) – Whether to use GPU execution.

See also

maxframe.liteframe.read_csv

Read a CSV file into a LiteFrame.

maxframe.liteframe.read_parquet

Read a Parquet file into a LiteFrame.

maxframe.liteframe.concat

Concatenate LiteFrames along the row axis.

maxframe.liteframe.LiteFrame.from_tensor

Create a LiteFrame from a MaxFrame tensor.

Examples

>>> import maxframe.liteframe as mlf
>>> lf = mlf.LiteFrame({'a': [1, 2, 3], 'b': [4, 5, 6]})
>>> lf.execute()
   a  b
0  1  4
1  2  5
2  3  6
__init__(data=None, columns=None, dtype=None, chunk_size=None, gpu=None)[source]#

Methods

__init__([data, columns, dtype, chunk_size, gpu])

add(other)

Element-wise addition of other to the LiteFrame.

add_prefix(prefix)

Prefix all column labels with prefix.

add_suffix(suffix)

Suffix all column labels with suffix.

agg([func, method, numeric_only, ...])

Aggregate using one or more operations over columns.

all([method, numeric_only])

Return whether all elements in each column are truthy.

any([method, numeric_only])

Return whether any element in each column is truthy.

apply(func[, axis, result_type, dtypes, ...])

Apply func per column (axis=0) or per row (axis=1).

apply_chunk(func, *[, dtypes, skip_infer, ...])

Apply func per execution chunk.

assign(**kwargs)

Add new columns to LiteFrame.

astype(dtype[, errors])

Cast a LiteFrame to a specified dtype.

between(left, right[, inclusive])

Return boolean LiteFrame indicating whether each value is between left and right.

bfill([limit, columns])

Backward-fill: propagate the next non-null value backward.

case_when(caselist)

Replace values where conditions are True, evaluated in order.

clip([lower, upper])

Trim values at input threshold(s).

copy()

copy_from(obj)

copy_to(target)

count([method])

Return the count of non-null values in each column.

cummax([skipna, columns])

Cumulative max of column values, preserving row order.

cummin([skipna, columns])

Cumulative min of column values, preserving row order.

cumprod([skipna, columns])

Cumulative prod of column values, preserving row order.

cumsum([skipna, columns])

Cumulative sum of column values, preserving row order.

diff([periods, columns])

Compute x[i] - x[i-periods] for each row.

div(other)

Element-wise true division (lf / other).

drop(columns[, errors])

Drop columns from LiteFrame.

drop_duplicates([subset, keep, method])

Return LiteFrame with duplicate rows removed.

dropna([axis, how, subset])

Drop rows or columns with missing values.

duplicated([subset, keep, method])

Return boolean mask marking duplicate rows.

eq(other)

Element-wise equality comparison.

eval(expr[, inplace])

Evaluate a string describing operations on LiteFrame columns.

execute([session])

expanding([min_periods, columns, order_by, ...])

Provide expanding window calculations.

explode(column)

Explode list column(s) into multiple rows.

ffill([limit, columns])

Forward-fill: propagate the last non-null value forward.

fillna([value])

Fill missing values.

flat_apply(func, *[, dtypes, skip_infer, ...])

Apply func to each row, expanding it to 0-N output rows.

floordiv(other)

Element-wise floor division (lf // other).

from_local_df(df[, chunk_size, gpu, ...])

Create a LiteFrame from a local pandas DataFrame or similar object.

from_tensor(tensor[, columns, gpu])

Create a LiteFrame from a MaxFrame tensor.

ge(other)

Element-wise greater-than-or-equal comparison.

groupby(by[, sort, dropna, group_keys])

Group the LiteFrame by one or more columns.

gt(other)

Element-wise greater-than comparison.

head([n])

Return the first n rows.

isna()

Detect missing values.

isnull()

Detect missing values.

kurt([method, numeric_only])

Return the unbiased kurtosis of each column.

le(other)

Element-wise less-than-or-equal comparison.

lshift(other)

Element-wise left bit shift (lf << other).

lt(other)

Element-wise less-than comparison.

map(func, *[, na_action, dtype, skip_infer, ...])

Apply func element-wise to every value in the LiteFrame.

mask(cond[, other])

Replace values where cond is True with other.

max([method, numeric_only])

Return the maximum of each column.

mean([method, numeric_only])

Return the mean of each column.

median([method, numeric_only])

Return the median of each column.

merge(right[, how, on, left_on, right_on, ...])

Merge LiteFrame with another LiteFrame using a database-style join.

min([method, numeric_only])

Return the minimum of each column.

mod(other)

Element-wise modulo (lf % other).

mul(other)

Element-wise multiplication (lf * other).

ne(other)

Element-wise not-equal comparison.

notna()

Detect non-missing values.

notnull()

Detect non-missing values.

nunique([method, numeric_only, dropna])

Return the number of distinct values in each column.

pct_change([periods, columns])

Compute (x[i] - x[i-periods]) / x[i-periods] for each row.

pow(other)

Element-wise exponentiation (lf ** other).

prod([method, numeric_only])

Return the product of each column.

query(expr[, inplace])

Query the columns of a LiteFrame with a boolean expression.

radd(other)

Reverse element-wise addition (other + lf).

rdiv(other)

Reverse element-wise true division (other / lf).

rebalance([axis, factor, num_partitions, rough])

Make data more balanced across entire cluster.

rechunk(chunk_size[, reassign_worker, axis])

Rechunk LiteFrame data.

rename(columns)

Rename columns in LiteFrame.

replace([to_replace, value])

Replace values in the LiteFrame.

rfloordiv(other)

Reverse element-wise floor division (other // lf).

rlshift(other)

Reverse element-wise left bit shift (other << lf).

rmod(other)

Reverse element-wise modulo (other % lf).

rmul(other)

Reverse element-wise multiplication (other * lf).

rolling(window[, min_periods, center, ...])

Provide rolling-window calculations.

round([decimals])

Element-wise rounding to decimals decimal places.

rpow(other)

Reverse element-wise exponentiation (other ** lf).

rrshift(other)

Reverse element-wise right bit shift (other >> lf).

rshift(other)

Element-wise right bit shift (lf >> other).

rsub(other)

Reverse element-wise subtraction (other - lf).

select(*args)

Dispatch to select or select_expr based on argument types.

sem([method, numeric_only])

Return the standard error of the mean of each column.

shift([periods, fill_value, columns])

Shift rows by periods positions.

size([method])

Return the total number of rows.

skew([method, numeric_only])

Return the unbiased skewness of each column.

sort([by, ascending, kind, na_position, ...])

Sort the LiteFrame by the specified columns.

std([method, numeric_only])

Return the sample standard deviation of each column.

sub(other)

Element-wise subtraction (lf - other).

sum([method, numeric_only])

Return the sum of each column.

to_csv(path[, sep, na_rep, float_format, ...])

Write LiteFrame to a comma-separated values (csv) file.

to_json(path, *[, lines, orient, ...])

Write a LiteFrame to JSON format (line-delimited by default).

to_lance(path, *[, mode, on, left_on, ...])

Write a LiteFrame to a Lance dataset.

to_odps_table(table[, partition, ...])

Write LiteFrame object into a MaxCompute (ODPS) table.

to_pandas([session])

Convert the LiteFrame to a pandas DataFrame.

to_parquet(path[, engine, compression, ...])

Write a LiteFrame to the binary parquet format, each chunk will be written to a Parquet file.

to_tensor([dtype])

Convert the LiteFrame to a MaxFrame tensor.

transform(func[, axis, input_frame_type, ...])

Transform per column (axis=0, default) or per row (axis=1).

transform_chunk(func, *[, input_frame_type, ...])

Apply func per execution block (chunk).

var([method, numeric_only])

Return the unbiased variance of each column.

where(cond[, other])

Replace values where cond is False with other.

Attributes

columns

Return the column labels of the LiteFrame.

dtypes

Return the dtype of each column.

iloc

ndim

Return the number of dimensions of the LiteFrame.

shape

values

Return the LiteFrame as a tensor.