python - Extracting all specific rows (separately) from multiple csv files and combine rows to save as a new file

Question

I have a number of csv files. I need to extract all respective rows from each file and save it as a new file. i.e. first output file must contain first rows of all input files and so on.

I have done the following.

import pandas as pd
import os
import numpy as np
data = pd.DataFrame('', columns =['ObjectID', 'SPI'], index = np.arange(1,100))
path = r'C:UsersikraDesktopPandas'
i = 1
for files in os.listdir(path):
    if files[-4:] == '.csv': 
        for j in range(0,10, 1):
        #print(files) 
            dataset = pd.read_csv(r'C:UsersikraDesktopPandas'+'\'+files)
            spi1 = dataset.loc[j,'SPI'] 
            data.loc[i]['ObjectID'] = files[:]
            data.loc[i]['SPI'] = spi1
            data.to_csv(r'C:UsersikraDesktopPandasoutput\'+str(j)+'.csv') 
            i + 1

It works well when index (i.e. 'j' ) is specified. But when I tried to loop, the output csv file contains only first row. Where am I wrong?

question from:https://stackoverflow.com/questions/65921394/extracting-all-specific-rows-separately-from-multiple-csv-files-and-combine-ro

Answer 1

You better use append:

data = data.append(spi1)