python - KeyError: '[] not in index' when renaming pandas columns by index assignment

Question

I'm trying to rename a column and then use it, but I get the error KeyError: '[] not in index'.

What do I need to change and why?

import pandas as pd

url = 'https://github.com/hadley/tidy-data/blob/master/data/billboard.csv'

tracks = pd.read_csv(url, encoding='latin1')

tracks.head()

tracks.columns.values[1]

tracks.columns.values[1] = "artist"

tracks.head()

tracks2 = tracks[['year', 'artist', 'track', 'time']]
tracks2.head()

Traceback

Traceback (most recent call last):
  File "E:Anaconda3libsite-packagesIPythoncoreinteractiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-ed116e277ed9>", line 1, in <module>
    runfile('E:/PythonProjects/stack_overflow/test/test8.py', wdir='E:/PythonProjects/stack_overflow/test')
  File "E:Program FilesJetBrainsPyCharm 2020.1pluginspythonhelperspydev\_pydev_bundlepydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "E:Program FilesJetBrainsPyCharm 2020.1pluginspythonhelperspydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"
", file, 'exec'), glob, loc)
  File "E:/PythonProjects/stack_overflow/test/test8.py", line 15, in <module>
    tracks2 = tracks[['year', 'artist', 'track', 'time']]
  File "E:Anaconda3libsite-packagespandascoreframe.py", line 3030, in __getitem__
    indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
  File "E:Anaconda3libsite-packagespandascoreindexing.py", line 1266, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "E:Anaconda3libsite-packagespandascoreindexing.py", line 1316, in _validate_read_indexer
    raise KeyError(f"{not_found} not in index")
KeyError: '[] not in index'

Test Data

year,artist.inverted,track,time,genre,date.entered
2000,Destiny's Child,Independent Women Part I,3:38,Rock,2000-09-23
2000,Santana,"Maria, Maria",4:18,Rock,2000-02-12
2000,Savage Garden,I Knew I Loved You,4:07,Rock,1999-10-23
2000,Madonna,Music,3:45,Rock,2000-08-12
2000,"Aguilera, Christina",Come On Over Baby (All I Want Is You),3:38,Rock,2000-08-05

question from:https://stackoverflow.com/questions/65941256/keyerror-not-in-index-when-renaming-pandas-columns-by-index-assignment

Answer 1

• Tested in pandas 1.2.1
• Don't use tracks.columns.values[1] = "artist" to rename columns by index assignment because it results in inconsistent behavior.
  • If you remove or comment out the two lines with tracks.head() the code works.
  • However, if you have tracks.head() or print(tracks.head(), then the KeyError occurs.
• Use pandas.DataFrame.rename
  • tracks = tracks.rename(columns={'artist.inverted': 'artist'})

Test in PyCharm

Test 1

• Code in one file as follows
• With print(tracks.head(2))

import pandas as pd

print(pd.__version__)

URL = 'https://raw.githubusercontent.com/hadley/tidy-data/master/data/billboard.csv'

tracks = pd.read_csv(URL, encoding='latin1')

print(tracks.head(2))

tracks.columns.values[1]

tracks.columns.values[1] = "artist"

print(tracks.head(2))

tracks2 = tracks[['year', 'artist', 'track', 'time']]

print(tracks2.head())

Result

Python 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] on win32
runfile('E:/PythonProjects/stack_overflow/test/test8.py', wdir='E:/PythonProjects/stack_overflow/test')
1.2.1
   year  artist.inverted  ... x75th.week x76th.week
0  2000  Destiny's Child  ...        NaN        NaN
1  2000          Santana  ...        NaN        NaN
[2 rows x 83 columns]
   year           artist  ... x75th.week x76th.week
0  2000  Destiny's Child  ...        NaN        NaN
1  2000          Santana  ...        NaN        NaN
[2 rows x 83 columns]
Traceback (most recent call last):
  File "E:Anaconda3libsite-packagesIPythoncoreinteractiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-ed116e277ed9>", line 1, in <module>
    runfile('E:/PythonProjects/stack_overflow/test/test8.py', wdir='E:/PythonProjects/stack_overflow/test')
  File "E:Program FilesJetBrainsPyCharm 2020.1pluginspythonhelperspydev\_pydev_bundlepydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "E:Program FilesJetBrainsPyCharm 2020.1pluginspythonhelperspydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"
", file, 'exec'), glob, loc)
  File "E:/PythonProjects/stack_overflow/test/test8.py", line 17, in <module>
    tracks2 = tracks[['year', 'artist', 'track', 'time']]
  File "E:Anaconda3libsite-packagespandascoreframe.py", line 3030, in __getitem__
    indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
  File "E:Anaconda3libsite-packagespandascoreindexing.py", line 1266, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "E:Anaconda3libsite-packagespandascoreindexing.py", line 1316, in _validate_read_indexer
    raise KeyError(f"{not_found} not in index")
KeyError: '[] not in index'

Test 2

• Without print(tracks.head(2))

import pandas as pd

print(pd.__version__)

URL = 'https://raw.githubusercontent.com/hadley/tidy-data/master/data/billboard.csv'

tracks = pd.read_csv(URL, encoding='latin1')

tracks.columns.values[1]

tracks.columns.values[1] = "artist"

tracks2 = tracks[['year', 'artist', 'track', 'time']]

print(tracks2.head())

Result

Python 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] on win32
runfile('E:/PythonProjects/stack_overflow/test/test8.py', wdir='E:/PythonProjects/stack_overflow/test')
1.2.1
   year               artist                                  track  time
0  2000      Destiny's Child               Independent Women Part I  3:38
1  2000              Santana                           Maria, Maria  4:18
2  2000        Savage Garden                     I Knew I Loved You  4:07
3  2000              Madonna                                  Music  3:45
4  2000  Aguilera, Christina  Come On Over Baby (All I Want Is You)  3:38

Test in Jupyter Lab

• With tracks.head(1)

• Without tracks.head(1)