Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
350 views
in Technique[技术] by (71.8m points)

python - Scraping PDF - Checking word frequency on specified keywords

I've created a parser that scrapes keywords from a PDF document. Currently, this scrapes the top keyword as well as shows the frequency (how many times) the words been repeated in the document.

At this point, I'm looking to check the frequency of specific keywords however when entering the desired keyword, it joins the word together with the top word and gives the same frequency.

Ideally, I'd like to be able to check the frequency of the keyword 1.) "GRI" 2.) "CDP"

Would great appreciate anyone's help here!

import pandas as pd
import textract
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
import nltk

nltk.download('stopwords')

pdffileobj=open('sample.pdf', 'rb')
pdfreader=PyPDF2.PdfFileReader(pdffileobj)
num_pages=pdfreader.numPages
count = 0
text= " "

while count < num_pages:
    pageObj = pdfreader.getPage(count)
    count +=1
    text += pageObj.extractText()

if text != "":
   text = text
else:
   text = textract.process(fileurl, method='tesseract', language='eng')

nltk.download('punkt')
tokens=word_tokenize(text)

punctuations = ['(',')',';',':','[',']',',','!','=','==','<','>','@','#','$','%','^','&','*','.','//','{','}','...','``','+',"''",]

stop_words = stopwords.words('english')

keywords = [word for word in tokens if not word in stop_words and not word in punctuations]

# print(keywords)
#At this point all the keywords in the document show up 

freq = pd.Series(' '.join(keywords).split()).value_counts()

#Print results show with frequency
print(freq)



与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

To get a single word's frequency from Pandas' value_counts() output, you just do

freq['word_you_want']

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...