在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
开源软件名称(OpenSource Name):serpapi/google-search-results-python开源软件地址(OpenSource Url):https://github.com/serpapi/google-search-results-python开源编程语言(OpenSource Language):Python 98.2%开源软件介绍(OpenSource Introduction):Google Search Results in PythonThis Python package is meant to scrape and parse search results from Google, Bing, Baidu, Yandex, Yahoo, Home depot, Ebay and more.. using SerpApi. The following services are provided:
SerpApi provides a script builder to get you started quickly. InstallationPython 3.7+ pip install google-search-results Link to the python package page Quick startfrom serpapi import GoogleSearch
search = GoogleSearch({
"q": "coffee",
"location": "Austin,Texas",
"api_key": "<your secret api key>"
})
result = search.get_dict() This example runs a search about "coffee" using your secret api key. The SerpApi service (backend)
Alternatively, you can search:
See the playground to generate your code. Summary
Google Search API capabilitySource code. params = {
"q": "coffee",
"location": "Location Requested",
"device": "desktop|mobile|tablet",
"hl": "Google UI Language",
"gl": "Google Country",
"safe": "Safe Search Flag",
"num": "Number of Results",
"start": "Pagination Offset",
"api_key": "Your SERP API Key",
# To be match
"tbm": "nws|isch|shop",
# To be search
"tbs": "custom to be search criteria",
# allow async request
"async": "true|false",
# output format
"output": "json|html"
}
# define the search search
search = GoogleSearch(params)
# override an existing parameter
search.params_dict["location"] = "Portland"
# search format return as raw html
html_results = search.get_html()
# parse results
# as python Dictionary
dict_results = search.get_dict()
# as JSON using json package
json_results = search.get_json()
# as dynamic Python object
object_result = search.get_object() Link to the full documentation see below for more hands on examples. How to set SERP API keyYou can get an API key here if you don't already have one: https://serpapi.com/users/sign_up The SerpApi GoogleSearch.SERP_API_KEY = "Your Private Key" The SerpApi query = GoogleSearch({"q": "coffee", "serp_api_key": "Your Private Key"}) Example by specificationWe love true open source, continuous integration and Test Drive Development (TDD). We are using RSpec to test our infrastructure around the clock to achieve the best QoS (Quality Of Service). The directory test/ includes specification/examples. Set your api key. export API_KEY="your secret key" Run test make test Location APIfrom serpapi import GoogleSearch
search = GoogleSearch({})
location_list = search.get_location("Austin", 3)
print(location_list) it prints the first 3 location matching Austin (Texas, Texas, Rochester) [ { 'canonical_name': 'Austin,TX,Texas,United States',
'country_code': 'US',
'google_id': 200635,
'google_parent_id': 21176,
'gps': [-97.7430608, 30.267153],
'id': '585069bdee19ad271e9bc072',
'keys': ['austin', 'tx', 'texas', 'united', 'states'],
'name': 'Austin, TX',
'reach': 5560000,
'target_type': 'DMA Region'},
...] Search Archive APIThe search result are stored in temporary cached. The previous search can be retrieve from the the cache for free. from serpapi import GoogleSearch
search = GoogleSearch({"q": "Coffee", "location": "Austin,Texas"})
search_result = search.get_dictionary()
assert search_result.get("error") == None
search_id = search_result.get("search_metadata").get("id")
print(search_id) Now let retrieve the previous search from the archive. archived_search_result = GoogleSearch({}).get_search_archive(search_id, 'json')
print(archived_search_result.get("search_metadata").get("id")) it prints the search result from the archive. Account APIfrom serpapi import GoogleSearch
search = GoogleSearch({})
account = search.get_account() it prints your account information. Search Bingfrom serpapi import BingSearch
search = BingSearch({"q": "Coffee", "location": "Austin,Texas"})
data = search.get_dict() this code prints baidu search results for coffee as a Dictionary. https://serpapi.com/bing-search-api Search Baidufrom serpapi import BaiduSearch
search = BaiduSearch({"q": "Coffee"})
data = search.get_dict() this code prints baidu search results for coffee as a Dictionary. https://serpapi.com/baidu-search-api Search Yandexfrom serpapi import YandexSearch
search = YandexSearch({"text": "Coffee"})
data = search.get_dict() this code prints yandex search results for coffee as a Dictionary. https://serpapi.com/yandex-search-api Search Yahoofrom serpapi import YahooSearch
search = YahooSearch({"p": "Coffee"})
data = search.get_dict() this code prints yahoo search results for coffee as a Dictionary. https://serpapi.com/yahoo-search-api Search Ebayfrom serpapi import EbaySearch
search = EbaySearch({"_nkw": "Coffee"})
data = search.get_dict() this code prints ebay search results for coffee as a Dictionary. https://serpapi.com/ebay-search-api Search Home depotfrom serpapi import HomeDepotSearch
search = HomeDepotSearch({"q": "chair"})
data = search.get_dict() this code prints home depot search results for chair as Dictionary. https://serpapi.com/home-depot-search-api Search Youtubefrom serpapi import HomeDepotSearch
search = YoutubeSearch({"q": "chair"})
data = search.get_dict() this code prints youtube search results for chair as Dictionary. https://serpapi.com/youtube-search-api Search Google Scholarfrom serpapi import GoogleScholarSearch
search = GoogleScholarSearch({"q": "Coffee"})
data = search.get_dict() this code prints Google Scholar search results. Search Walmartfrom serpapi import WalmartSearch
search = WalmartSearch({"query": "chair"})
data = search.get_dict() this code prints Google Scholar search results. Search Youtubefrom serpapi import YoutubeSearch
search = YoutubeSearch({"search_query": "chair"})
data = search.get_dict() this code prints Google Scholar search results. Search Apple Storefrom serpapi import AppleAppStoreSearch
search = AppleAppStoreSearch({"term": "Coffee"})
data = search.get_dict() this code prints Google Scholar search results. Search Naverfrom serpapi import NaverSearch
search = NaverSearch({"query": "chair"})
data = search.get_dict() this code prints Google Scholar search results. Generic search with SerpApiClientfrom serpapi import SerpApiClient
query = {"q": "Coffee", "location": "Austin,Texas", "engine": "google"}
search = SerpApiClient(query)
data = search.get_dict() This class enables to interact with any search engine supported by SerpApi.com Search Google Imagesfrom serpapi import GoogleSearch
search = GoogleSearch({"q": "coffe", "tbm": "isch"})
for image_result in search.get_dict()['images_results']:
link = image_result["original"]
try:
print("link: " + link)
# wget.download(link, '.')
except:
pass this code prints all the images links, and download image if you un-comment the line with wget (linux/osx tool to download image). This tutorial covers more ground on this topic. https://github.com/serpapi/showcase-serpapi-tensorflow-keras-image-training Search Google Newsfrom serpapi import GoogleSearch
search = GoogleSearch({
"q": "coffe", # search search
"tbm": "nws", # news
"tbs": "qdr:d", # last 24h
"num": 10
})
for offset in [0,1,2]:
search.params_dict["start"] = offset * 10
data = search.get_dict()
for news_result in data['news_results']:
print(str(news_result['position'] + offset * 10) + " - " + news_result['title']) this script prints the first 3 pages of the news title for the last 24h. Search Google Shoppingfrom serpapi import GoogleSearch
search = GoogleSearch({
"q": "coffe", # search search
"tbm": "shop", # news
"tbs": "p_ord:rv", # last 24h
"num": 100
})
data = search.get_dict()
for shopping_result in data['shopping_results']:
print(shopping_result['position']) + " - " + shopping_result['title']) this script prints all the shopping results order by review order. Google Search By LocationWith SerpApi, we can build Google search from anywhere in the world. This code is looking for the best coffee shop per city. from serpapi import GoogleSearch
for city in ["new york", "paris", "berlin"]:
location = GoogleSearch({}).get_location(city, 1)[0]["canonical_name"]
search = GoogleSearch({
"q": "best coffee shop", # search search
"location": location,
"num": 1,
"start": 0
})
data = search.get_dict()
top_result = data["organic_results"][0]["title"] Batch Asynchronous SearchesWe do offer two ways to boost your searches thanks to
# Operating system
import os
# regular expression library
import re
# safe queue (named Queue in python2)
from queue import Queue
# Time utility
import time
# SerpApi search
from serpapi import GoogleSearch
# store searches
search_queue = Queue()
# SerpApi search
search = GoogleSearch({
"location": "Austin,Texas",
"async": True,
"api_key": os.getenv("API_KEY")
})
# loop through a list of companies
for company in ['amd', 'nvidia', 'intel']:
print("execute async search: q = " + company)
search.params_dict["q"] = company
result = search.get_dict()
if "error" in result:
print("oops error: ", result["error"])
continue
print("add search to the queue where id: ", result['search_metadata'])
# add search to the search_queue
search_queue.put(result)
print("wait until all search statuses are cached or success")
# Create regular search
while not search_queue.empty():
result = search_queue.get()
search_id = result['search_metadata']['id']
# retrieve search from the archive - blocker
print(search_id + ": get search from archive")
search_archived = search.get_search_archive(search_id)
print(search_id + ": status = " +
search_archived['search_metadata']['status'])
# check status
if re.search('Cached|Success',
search_archived['search_metadata']['status']):
print(search_id + ": search done with q = " +
search_archived['search_parameters']['q'])
else:
# requeue search_queue
print(search_id + ": requeue search")
search_queue.put(result)
# wait 1s
time.sleep(1)
print('all searches completed') This code shows how to run searches asynchronously. The search parameters must have {async: True}. This indicates that the client shouldn't wait for the search to be completed. The current thread that executes the search is now non-blocking which allows to execute thousand of searches in seconds. The SerpApi backend will do the processing work. The actual search result is defer to a later call from the search archive using get_search_archive(search_id). In this example the non-blocking searches are persisted in a queue: search_queue. A loop through the search_queue allows to fetch individual search result. This process can be easily multithreaded to allow a large number of concur |
2023-10-27
2022-08-15
2022-08-17
2022-09-23
2022-08-13
请发表评论