Posting form data with requests and python

Posting form data isn't working and since my other post about this wasn't working, I figured I would try to ask the question again so maybe I can get another perspective. I am currently trying to get the requests.get(url, data=q) to work. When I print, I am getting a page not found. I have resorted just to set variables and join them to the entire URL to make it work but I really want to learn this aspect about requests. Where am I making the mistake? I am using the HTML tag attributes name=search_terms and name=geo_location_terms for the form.

requests.get(url, data=q)

requests

name=search_terms

name=geo_location_terms

search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com"
q = 'search_terms': search_terms, 'geo_locations_terms': location
page = requests.get(url, data=q)
print(page.url)


2 Answers
2

You have few little mistakes in your code:

url = "https://www.yellowpages.com/search"

geo_location_terms

geo_locations_terms

requests.get

params

data

So, the final version of code:

import requests

search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"
q = 'search_terms': search_terms, 'geo_location_terms': location
page = requests.get(url, params=q)
print(page.url)


Result:

https://www.yellowpages.com/search?search_terms=Bars&geo_location_terms=New+Orleans%2C+LA


requests.get()

requests.post()

data

Besides the issues pointed by @Lev Zakharov, you need to set the cookies in your request, like this:

import requests

search_terms = "Bars"
location = "New Orleans, LA"
url = "https://www.yellowpages.com/search"

with requests.Session() as session:
session.headers.update(
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36',
'Cookie': 'cookies'
)

q = 'search_terms': search_terms, 'geo_locations_terms': location
response = session.get(url, params=q)

print(response.url)
print(response.status_code)


Output

https://www.yellowpages.com/search?search_terms=Bars&geo_locations_terms=New+Orleans%2C+LA
200


To get the cookies you can see the requests using some Network listener for instance using Chrome Developer Tools Network tab, then replace the value 'cookies'

'cookies'

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.