Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
321 views
in Technique[技术] by (71.8m points)

python 3.x - Firefox selenium webscraper error: AttributeError: 'method_descriptor' object has no attribute 'today'

I have a selenium firefox webscraper that was functioning until recently, but all of a sudden I am getting the error mentioned in the title. I tried looking at other similar questions on here and elsewhere, but it seems like by importing datetime I shouldn't be getting this error? I'm obviously missing something, but I'm a little stumped. Any ideas on how to fix this are welcome.

from bs4 import BeautifulSoup as Bs
import datetime
import weasyprint
from classes.main import *
s = Session() # Keeps things stored in for future use

# If you look at the HTML, this is the "action" of the form (in this case happens to be the same as the form itself, not always true)
form_url = "https://online.portalberni.ca/WebApps/PIP/Pages/Search.aspx?templateName=permit%20reporting"

# Gets the HTML of the form
r = s.get(form_url)
html = Bs(r.text, "lxml")
form = html.find("form")

# Finds hidden inputs in the form that are necessary for a successful POST
hidden = form.find_all("input", {"type": "hidden"})
data = {i["name"]: i["value"] for i in hidden}

"""
There is javascript code that changes the form data before submission (onsubmit in the
form). I found this by using developer tools in chrome to see what the POST data actually
was, not by analyzing the javascript
"""
data["ctl00$FeaturedContent$ToolkitScriptManager1"] = "ctl00$FeaturedContent$updpnl_search|ctl00$FeaturedContent$btn_ViewReport"
data["__EVENTTARGET"] = ""
data["__EVENTARGUMENT"] = ""
data["__ASYNCPOST"] = "true"
data["ctl00$FeaturedContent$btn_ViewReport"] = "Search"

# Change to your date range
start = (datetime.date.today()-datetime.timedelta(days=7)).strftime("%m/%d/%Y")
#data["ctl00$FeaturedContent$txt_FromDate"] = "12/01/2020"
#data["ctl00$FeaturedContent$txt_ToDate"] = "12/10/2020"

# Submits the form
headers = {
    "Content-type": "application/x-www-form-urlencoded; charset=UTF-8",
    "Referer": "https://online.portalberni.ca/WebApps/PIP/Pages/Search.aspx?templateName=permit%20reporting",
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.84 Safari/537.36",
}
s.post(form_url, data=data, headers=headers)

# The page with the results you're looking for
results_url = "https://online.portalberni.ca/WebApps/PIP/Pages/PropBasedReportSelection.aspx?templateName=permit%20reporting"
r = s.get(results_url)

#print page as pdf
pdf = weasyprint.HTML(results_url).write_pdf()
open('/bm/tempdownload/portalberni.pdf', 'wb').write(pdf)```
question from:https://stackoverflow.com/questions/66054433/firefox-selenium-webscraper-error-attributeerror-method-descriptor-object-ha

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
Waitting for answers

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...