maxframe.liteframe.LiteFrame.drop_duplicates#

LiteFrame.drop_duplicates(subset=None, keep='first', method='auto')#

Return LiteFrame with duplicate rows removed.

Parameters:

  • subset (str or list of str, optional) – Only consider certain columns for identifying duplicates, by default use all columns.

  • keep ({"first", "last", False}, default "first") –

    • first: drop duplicates except the first occurrence.

    • last: drop duplicates except the last occurrence.

    • False: drop all duplicates.

  • method ({"auto", "tree", "subset_tree", "shuffle"}, default "auto") – Tiling strategy for multi-chunk inputs.

Returns:

Deduplicated LiteFrame.

Return type:

LiteFrame

See also

duplicated

Return boolean mask marking duplicate rows.

sort

Sort the LiteFrame by columns.

Examples

>>> import maxframe.liteframe as mlf
>>> lf = mlf.LiteFrame({'a': [1, 1, 2], 'b': [4, 4, 6]})
>>> result = lf.drop_duplicates()
>>> result.execute()
   a  b
0  1  4
2  2  6