Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
125 views
in Technique[技术] by (71.8m points)

python 3.x - I cant retrieve all rows from a table while web scraping

this is my code. I actually want to extract all the row from table contacted hosts. but I can only retrieve one row from the table. for loop doesn't work. or maybe I didn't understand it well. I am new to webscraping.

Scraping from: HYBRID Analysis with the following code:

import requests
from bs4 import BeautifulSoup

url = "https://www.hybrid-analysis.com/sample/a1b38a18decb253708e0198bfaaed97ef1c16fc061f1dc8c1ba00e98ef77092e/5ff1a85664a2e10f370a6c02"
r = requests.session()
page = r.get(url, headers = {'User-Agent': 'Mozilla/5.0'})
soup = BeautifulSoup(page.content, 'html.parser')
id1 = soup.find(id = 'contacted-hosts')
class1 = id1.find_all(class_='table table-striped small')

soup.find('div',{'class':'modal-dialog modal-xl'}).decompose()
soup.find('button',{'class':'btn btn-danger btn-xs btn-labeled smallest'}).decompose()
soup.find('div',{'class':'modal fade bs-example-modal-lg sample-network-traffic-modal'}).decompose()

#print(class1[0].find(class_='warning').get_text())
host1 = [item.find(class_='warning').get_text().strip().replace("
", "") for item in class1]
print(host1)

results in: https://i.stack.imgur.com/n2uLO.png

question from:https://stackoverflow.com/questions/65647535/i-cant-retrieve-all-rows-from-a-table-while-web-scraping

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
Waitting for answers

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...