pandas DataFrame Index: set_index(), reset_index() and sort_index()

A pandas DataFrame index is the label axis for rows, not a guaranteed unique identifier or a stored column by default. The default RangeIndex usually displays 0, 1, 2, and so on. Use set_index() to label rows with an existing column, reset_index() to return to a default index, and sort_index() to order rows by their labels. The examples in this guide were run with pandas 2.2.3; the basic operations also appear in current pandas 3.x documentation.

Create a DataFrame with an ID column

import pandas as pd

employees = pd.DataFrame({
    "ID": [90, 40, 80, 98],
    "Name": ["Joe", "Parth", "Sunny", "Lauren"],
    "Dept": ["Manager", "Dev", "Admin", "Manager"],
    "Salary": [90000, 65000, 50000, 92000]
})

print(employees.index)  # RangeIndex(start=0, stop=4, step=1)

Pandas initially labels the four rows 0, 1, 2, 3. That index is a label axis. It is not automatically the same as a unique employee ID, and changing it does not necessarily change the physical row order.

1. set_index(): Use an existing column as the row index

by_id = employees.set_index("ID")
print(by_id.loc[90, "Name"])  # Joe
print(by_id.index.tolist())   # [90, 40, 80, 98]
print("ID" in by_id.columns)   # False

By default, set_index("ID") uses ID values as row labels and removes the ID column from the ordinary columns (drop=True). It returns a new DataFrame, leaving the original employees variable unchanged unless you explicitly reassign it or use an in-place operation. If you want the ID column retained too, pass drop=False.

The original PickupBrain video has been retained. Use the tested examples below when a historic pandas interface or parameter differs from your installed version.

Understand .loc vs .iloc with a custom index

print(by_id.loc[90, "Name"])   # Joe: label 90
print(by_id.iloc[0]["Name"])    # Joe: row position 0
print(by_id.loc[40, "Name"])   # Parth: label 40

.loc selects by index labels. .iloc selects by integer positions. With index labels 90, 40, 80, and 98, by_id.loc[0] is not the first row and would raise KeyError, while by_id.iloc[0] is the first row.

2. reset_index(): Restore sequential row labels

restored = by_id.reset_index()
print(restored.columns.tolist())
# ["ID", "Name", "Dept", "Salary"]

without_old_id = by_id.reset_index(drop=True)
print(without_old_id.columns.tolist())
# ["Name", "Dept", "Salary"]

With the default drop=False, reset_index() moves the old index into a normal column. With drop=True, it discards that index information. If the old index contains identifiers you need, preserve them by resetting without drop=True.

3. MultiIndex: Use two columns as row labels

multi = employees.set_index(["Dept", "ID"])
print(multi.index.names)  # ["Dept", "ID"]
print(multi.loc[("Manager", 90), "Name"])  # Joe

A MultiIndex has multiple levels. It can be useful for grouping data by department and ID, but you must supply the intended level labels for direct lookup. The append=True option appends new index levels to the existing index instead of replacing it.

4. sort_index() vs sort_values()

by_id_sorted = by_id.sort_index()
print(by_id_sorted.index.tolist())  # [40, 80, 90, 98]

by_salary = by_id.sort_values("Salary", ascending=False)
print(by_salary.index.tolist())  # [98, 90, 40, 80]

by_columns = by_id.sort_index(axis=1)
print(by_columns.columns.tolist())  # ["Dept", "Name", "Salary"]

sort_index() sorts index labels, whereas sort_values() sorts using values from a selected column. The default for sort_index() is to sort rows; axis=1 sorts column labels alphabetically. ascending=False reverses the requested sort order.

5. Do index labels have to be unique?

print(employees["ID"].is_unique)  # True

with_duplicates = pd.DataFrame({
    "ID": [1, 1], "Value": [2, 3]
}).set_index("ID")
print(with_duplicates.index.is_unique)  # False

An index does not have to be unique. Duplicate labels are allowed by default, and .loc[1] can then return multiple rows. If IDs should be unique, inspect df["ID"].is_unique or the resulting df.index.is_unique before relying on one-to-one lookups. Older tutorials recommend verify_integrity=True; that parameter is deprecated in pandas 3.0. Checking uniqueness explicitly is a clearer cross-version choice.

Quick comparison of index methods

Important parameter notes

• set_index(keys, drop=True, append=False, inplace=False): choose one or several existing columns, optionally retain them, or append index levels.
• reset_index(level=None, drop=False, inplace=False): reset all or selected index levels; preserve old labels as columns unless drop=True.
• sort_index(axis=0, ascending=True, inplace=False): sort labels; the older by= argument shown in some online examples is not valid for current sort_index().
• inplace=True: where supported, changes the DataFrame and the method typically returns None. Prefer explicit assignment for code that is easier to follow.

Troubleshooting index problems

• KeyError after set_index: check whether the column moved into the index. Use reset_index() to restore it as a normal column if needed.
• Unexpected row order: index values do not automatically reorder rows; use sort_index() when label order matters.
• Duplicate labels: use index.is_unique and inspect duplicates before using label lookups that expect a single row.
• Index unexpectedly lost: reset_index(drop=True) discards old labels rather than adding them as a column.
• MultiIndex selection error: check the index level names and pass a tuple of labels, or reset the index for simpler filtering.

Summary

Use set_index() to create meaningful row labels, reset_index() to restore columns and sequential labels, and sort_index() to sort the labels themselves. Learn the difference between .loc and .iloc early—it prevents many subtle data-selection errors. For authoritative parameter details, consult pandas set_index(), reset_index(), and sort_index().

C P Gupta

C P Gupta is a YouTuber and Blogger. He is expert in Microsoft Word, Excel and PowerPoint. His YouTube channel @pickupbrain is very popular and has crossed 9.9 Million Views.