Why it is occuring KeyError: 'main' and how to solve it - django

I am having keyerror: 'main'. I have searched many sites, but still can't find any satisfactory answer to solve this error. I would really appreciate if someone can give me some pointers. Thanks in advance.
I have tried solving this by adding a function on the init.py which is suggested by a site. But it still didn't work.
https://forum.inductiveautomation.com/t/error-on-sys-modules/6431/2
code: view.py
from django.shortcuts import render
import requests
from .models import City
def index(request):
url = 'http://api.openweathermap.org/data/2.5/weather?q={}&units=imperial&appid=MYKey'
cities = City.objects.all() # return all the cities in the database
city = 'Dhaka'
# request the API data and convert the JSON to Python data types
city_weather = requests.get(url.format(city)).json()
weather_data = []
for city in cities:
# request the API data and convert the JSON to Python data types
city_weather = requests.get(url.format(city)).json()
weather_app = {
'city': city,
'temperature': city_weather['main']['temp'],
'description': city_weather['weather'][0]['description'],
'icon': city_weather['weather'][0]['icon']
}
# add the data for the current city into our list
weather_data.append(weather_app)
#context = {'weather' : weather_app}
context = {'weather_data': weather_data}
# returns the index.html template
return render(request, 'weather_app/index.html')
terminal:
(env) acer#acer-Aspire-V3-472P:~/DjangoProject/Weather$ python manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
May 03, 2019 - 06:48:01
Django version 2.2, using settings 'Weather.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Internal Server Error: /
Traceback (most recent call last):
File "/home/acer/DjangoProject/env/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/acer/DjangoProject/env/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/acer/DjangoProject/env/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/acer/DjangoProject/Weather/weather_app/views.py", line 30, in index
'temperature': city_weather['main']['temp'],
KeyError: 'main'
[03/May/2019 06:48:18] "GET / HTTP/1.1" 500 68977

You don't check that the data for the specific city is found. You loop through all cities in your database, and try to get the weather for each one; but you don't check that the result is actually returned. You should do:
for city in cities:
response = requests.get(url.format(city))
if response.status_code == 404:
continue
city_weather = response.json()
Also, you should check that you are formatting your URL properly. As it stands, you are inserting your City object directly into the URL - this will only work if you have defined a __str__ method that returns only the city name. It would be better to use the name directly:
response = requests.get(url.format(city.name)) # or whatever the name field is

I'm not sure. I have a suggestion. You may check the city_weatherdata (using print())before making weather_app to see if the key'main'is really in city_weather, maybe the key happened not to be in the data.

Related

escape element in a list when it return Error

In order to get better skills I'am making this scripte so it takes a list of website and create a dict and take every website and crawle it to find the "conatct-us" page, But I see that my script stops when one of the websites is not working so what I am trying to do is to escape that website and continue to the others
here's my code :
import requests
from bs4 import BeautifulSoup
from urlparse import urlparse
from mechanize import Browser
import re
headers = [('User-Agent','Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0')]
urls = 'http://www.officialusa.com/stateguides/chambers/georgia.html'
links_dict = []
response = requests.get(urls, headers)
bsObj = BeautifulSoup(response.text,'lxml')
for tag in bsObj.find_all('li'):
links_dict.append(tag.a.get('href'))
for ink in links_dict:
r = requests.get(ink)
#get domain name only
parsed_uri = urlparse(ink)
domain = parsed_uri.netloc
br = Browser()
br.set_handle_robots(False)
br.addheaders = headers
try:
br.open(str(ink))
for link in br.links():
siteMatch = re.compile(ink).search(link.url)
print link.url
except:
pass
Everything is okey with the other links
here's the Error:
Traceback (most recent call last):
File "/home/qunix/PycharmProjects/challange/crawel.py", line 20, in <module>
r = requests.get(ink)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 487, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='www.quitmangeorgia.org', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7facf68cca50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))
thank You !!
Try wrapping the line
r = requests.get(ink)
in a try catch like so:
try:
r = requests.get(ink)
except ConnectionError:
continue
This will mean that if the call to requests.get throws a ConnectionError as it does in your example, it will pass over to the next website in the list.

Exporting data from App Engine datastore as a Google Drive spreadsheet

I have an app engine application on which I mark my monthly expenses along with some comments or reason. I would like to export these data into a
Google Drive Spreadsheet. I use Django framework.
I had gone through the tutorials provided by Google here.
But they have implemented it using webapp2 and jinja. Moreover, the doc for Implementing Using Django seems way too obsolete since I do not use Django ORM.
Below is my code sample which I use to upload. I strongly apologize if what I paste below is rubbish. Please help.
from django.utils.datastructures import SortedDict
import os
from apiclient.discovery import build
from apiclient.http import MediaFileUpload
from oauth2client.appengine import OAuth2DecoratorFromClientSecrets
decorator = OAuth2DecoratorFromClientSecrets(os.path.join(os.path.dirname(__file__ ), 'clientSecrets.json'), 'https://www.googleapis.com/auth/drive')
drive_service = build('drive', 'v2')
class Exporter(object):
serializedObjects = []
mime_type = 'text/plain'
fileToExport = None
request = None
def __init__(self, serializedObjects, request):
self.serializedObjects = serializedObjects
self.request = request
def createCSV(self):
import csv
import StringIO
stdout = StringIO.StringIO()
writer = csv.writer(stdout)
for obj in self.serializedObjects:
for value in obj.values():
writer.writerow([value])
# I will get the csv produced from my datastore objects here.
# I would like to upload this into a Google Spreadsheet.
# The child class ExportToSpreadSheet tries to do this.
return stdout.getvalue()
class ExportToSpreadSheet(Exporter):
def __init__(self, *args, **kwargs):
super(ExportToSpreadSheet, self).__init__(*args, **kwargs)
self.mime_type = 'application/vnd.google-apps.spreadsheet'
def create(self):
import datetime
valueToDrive = self.createCSV()
media_body = MediaFileUpload(valueToDrive, mimetype=self.mime_type, resumable=True)
body = {
'title' : 'MyExpense_%s' % datetime.datetime.now().strftime('%d_%b_%Y_%H_%M_%S'),
'description' : '',
'mimeType' : self.mime_type
}
self.fileToExport = drive_service.files().insert(body=body, media_body=media_body, convert=True)
return self.fileToExport
#decorator.oauth_aware
def upload(self):
if decorator.has_credentials():
self.create()
self.fileToExport.execute(decorator.http())
return self.fileToExport
raise Exception('user does not have the credentials to upload to google drive.')
#decorator.oauth_aware only works on webapp.RequestHandler subclasses. Why I am saying it is because I got this error while I ran the code.
INFO 2013-09-19 11:28:04,550 discovery.py:190] URL being requested: https://www.googleapis.com/discovery/v1/apis/drive/v2/rest?userIp=%3A%3A1
ERROR 2013-09-19 11:28:05,670 main.py:13] Exception in request:
Traceback (most recent call last):
File "/home/dev5/divya/jk/MyApp/ItsMyTuition/SDK/google_appengine/lib/django-1.2/django/core/handlers/base.py", line 100, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/dev5/divya/jk/MyApp/ItsMyTuition/ItsMyTuition/src/tuition/json/ajaxHandler.py", line 27, in mainHandler
responseValues = funtionToCall(*args)
File "/home/dev5/divya/jk/MyApp/ItsMyTuition/ItsMyTuition/src/tuition/json/ajaxHandler.py", line 69, in export
uploadedFile = exporterInstance.upload()
File "/home/dev5/divya/jk/MyApp/ItsMyTuition/ItsMyTuition/src/oauth2client/appengine.py", line 770, in setup_oauth
self._create_flow(request_handler)
File "/home/dev5/divya/jk/MyApp/ItsMyTuition/ItsMyTuition/src/oauth2client/appengine.py", line 734, in _create_flow
redirect_uri = request_handler.request.relative_url(
AttributeError: 'ExportToSpreadSheet' object has no attribute 'request'
INFO 2013-09-19 11:28:05,777 module.py:593] default: "POST /ajaxCall/export HTTP/1.1" 200 964
Since I am using Django framework I cannot get a Request handler as they except.
How can I integrate or do it in my scenario? I would very much appreciate any code samples or relevant links I may have missed.
Moreover, the whole thing happens in an ajax call.
Thanks in advance.
Use mimeType=text/csv and during the upload, request a conversion from csv to Spreadsheets:
drive_service.files().insert(covert=True, body=body, media_body=media_body, convert=True)

How to correctly make a POST call to Facebook Graph API in django

I am trying to create an Facebook Event from my local application and I have a form where a user enters the event information. According to this, I have to make a post call to https://graph.facebook.com/USERID/events
To do so I've used urlopen as such
form_fields = {
"access_token": request.user.get_profile().access_token,
"name" : "name",
"start_time" : "01/01/2012"
}
urllib2.urlopen(update_url,urllib.urlencode(form_fields))
Running the code I get
HTTP Error 400: Bad Request
and after debugging the program the variable values are
update_url = str: https://graph.facebook.com/XXXXXXXX/events
form_fields = dict: {'access_token': u'XXXXXX', 'start_time': datetime.datetime(2012, 1, 6, 0, 0), 'location': 'someplace', 'name': u'ab'}
The update_url seems to be correct and I'm guessing the problem is with form_fields. So how should I give these fields to Facebook Graph API?
Traceback:
File "C:\Python27\lib\site-packages\django\core\handlers\base.py" in get_response
111. response = callback(request, *callback_args, **callback_kwargs)
File "C:\Users\Akshay\workspace\example\example\planpouchproto\views.py" in createPouch
20. graph.post('%s/events' %request.user.get_profile().facebook_id, **{"access_token": request.user.get_profile().access_token,"name" : cd['event_name'],"start_time" : "01/01/2012"})
File "C:\Users\Akshay\workspace\example\example\facepy\graph_api.py" in post
65. retry = retry
File "C:\Users\Akshay\workspace\example\example\facepy\graph_api.py" in _query
240. return load(method, url, data)[0]
File "C:\Users\Akshay\workspace\example\example\facepy\graph_api.py" in load
196. result = self._parse(response.content)
File "C:\Users\Akshay\workspace\example\example\facepy\graph_api.py" in _parse
282. error.get('code', None)
Exception Type: OAuthError at /pouch/
Exception Value:
Facepy makes the interaction with Facebook simpler, give it a try.
Posting an event to the graph would be something like
graph.post('USERID/events', form_fields)
To debug, you can test if authentication works by posting a comment:
graph = GraphAPI(request.user.get_profile().access_token)
graph.post(path="me/feed", message="hello FB", caption="hello FB", description="hello FB")
Then you can narrow down your problem.

django web app testing

I have the following in tests.py.
def setUp(self):
self.client = Client()
self.client.get('/homepage',{'join':'NPO2','siteid':1450})
self.client.session.save()
self.oraganisation_list = ['NPO1','NPO2','NPO3']
self.pay_recursion_list = ['annual','monthly','bi-annual','quarter']
def test_paytermpage(self):
for org in self.organisation_list:
response = self.client.get('',{'join':org,'siteid':1450})
self.failUnlessEqual(response.status_code,200)
self.assertTemplateUsed(response,'some.html')
def test_infopage(self):
for term in self.pay_recurstion_list:
response = self.client.post('',{'pay-term':term,'submit':'payterm'})
self.failUnlessEqual(response.status_code,200)
test_infopage() is failing and here is the traceback.
Traceback (most recent call last):
File "/var/lib/django/bsdata/shoppingcart/tests.py", line 50, in test_infopage
response = self.client.post('',{'pay-term':term,'submit':'payterm'})
File "/usr/lib/pymodules/python2.6/django/test/client.py", line 313, in post
response = self.request(**r)
File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py", line 92, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/var/lib/django/.../views.py", line 22, in start
term,costdict,webobj = costInfo(request)
File "/var/lib/django/...views.py", line 238, in getCostInfo
cost_dict = Site.objects.getDict(request.session['siteid'])
File "/var/lib/django/.../managers.py", line 16, in getLoadedDict
siteobj = Site.objects.get(pk=agent)
File "/usr/lib/pymodules/python2.6/django/db/models/manager.py", line 120, in get
return self.get_query_set().get(*args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/db/models/query.py", line 305, in get
% self.model._meta.object_name)
DoesNotExist: Site matching query does not exist.
I did debug to see what value 'agent' in siteobj = Site.objects.get(pk=agent) is getting its a valid integer.
Surprisingly both of them are working when tested from shell like this
setup_test_environment()
client = Client()
client.get('/shoppingcart',{'join':'NPO1','siteid':1450})
client.session.save()
oraganisation_list = ['NPO1','NPO2','NPO3']
pay_recursion_list = ['annual','monthly','bi-annual','quarter']
for org in oraganisation_list:
response = client.get('',{'join':org,'siteid':1450})
TestCase.failUnlessEqual(t,response.status_code,200)
for term in pay_recursion_list:
response = client.post('',{'pay-term':term,'submit':'payterm'})
TestCase.failUnlessEqual(t,response.status_code,200)
Sorry for too much info,didn't know how to explain better.
Any ideas would be highly helpful for this newbie. Thanks.
The Django test runner uses a different database than your production data. If your site requires a Site to exist, you should either add it in your Test setUp, or you should require a fixture that loads the site.
Try putting "SITE_ID = 1" in your settings. This is part of the sites framework.

testing django web app that uses cookies/session

In views.py:
get_dict = Site.objects.getDictionary(request.COOKIES['siteid'])
{gets a dictionary with site information based on id from cookie}
In tests.py:
from django.test import TestCase
class WebAppTest(TestCase):
def test_status(self):
response = self.client.get('/main/',{})
response.status_code # --->passed with code 200
response = self.client.get('/webpage/',{'blog':1})
response.status_code # ----> this is failing
In order to present blog page it goes to a view where it gets a dictionary using existing cookie, process it, renders templates, which works fine when running the app. But the tests are failing.Having never tested Django webapps I'm not sure how to test it right. Here is the traceback.
Traceback (most recent call last):
File "<console>", line 2, in <module>
File "/usr/lib/pymodules/python2.6/django/test/client.py", line 313, in post
response = self.request(**r)
File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py", line 92, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/var/lib/django/data/../webpage/views.py", line 237, in getCostInfo
get_dict = Site.objects.getDictionary(request.COOKIES['siteid'])
KeyError: 'siteid'
Went through some online samples but couldn't find something that deals in depth with cookies/sessions. Any ideas or directs to useful links are highly appreciated.
Take a look at the Persistent State section of the Django Testing docs.
In your case, I would expect your test to be something more like:
from django.test import TestCase
from django.test.client import Client
class WebAppTest(TestCase):
def setUp(self):
self.client = Client()
session = self.client.session
session['siteid'] = 69 ## Or any valid siteid.
session.save()
def test_status(self):
response = self.client.get('/main/',{})
self.assertEqual(response.status_code, 200)
response = self.client.get('/webpage/',{'blog':1})
self.assertEqual(response.status_code, 200)