pyspark.pandas.Index.dropna

Index.dropna() → pyspark.pandas.indexes.base.Index

Return Index or MultiIndex without NA/NaN values

Examples

>>> df = ps.DataFrame([[1, 2], [4, 5], [7, 8]],
...                   index=['cobra', 'viper', None],
...                   columns=['max_speed', 'shield'])
>>> df
       max_speed  shield
cobra          1       2
viper          4       5
NaN            7       8

>>> df.index.dropna()
Index(['cobra', 'viper'], dtype='object')

Also support for MultiIndex

>>> midx = pd.MultiIndex([['lama', 'cow', 'falcon'],
...                       [None, 'weight', 'length']],
...                      [[0, 1, 1, 1, 1, 1, 2, 2, 2],
...                       [0, 1, 1, 0, 1, 2, 1, 1, 2]])
>>> s = ps.Series([45, 200, 1.2, 30, 250, 1.5, 320, 1, None],
...               index=midx)
>>> s
lama    NaN        45.0
cow     weight    200.0
        weight      1.2
        NaN        30.0
        weight    250.0
        length      1.5
falcon  weight    320.0
        weight      1.0
        length      NaN
dtype: float64

>>> s.index.dropna()  
MultiIndex([(   'cow', 'weight'),
            (   'cow', 'weight'),
            (   'cow', 'weight'),
            (   'cow', 'length'),
            ('falcon', 'weight'),
            ('falcon', 'weight'),
            ('falcon', 'length')],
           )

pyspark.pandas.Index.fillna pyspark.pandas.Index.isna