I am trying to upload a csv file into Azure Devops workitem. While uploading the attachment step calling below rest api, I am facing an issue:
POST https://dev.azure.com/{organization}/{project}/_apis/wit/attachments?api-version=5.1
My code is as below:
with open('details.csv', 'r') as f:
details = f.read()
print(details) #Printing the CSV file as expected
ado_req_headers_ATT = {'Content-Type':'application/octet-stream'}
ADO_SEC_ATTA_URL = 'https://dev.azure.com/orgname/projectname/_apis/wit/attachments?fileName=details.csv&api-version=5.1-preview.3'
ado_req_attach_File = requests.post(url=ADO_SEC_ATTA_URL,headers=ado_req_headers_ATT,data=details, auth=('',ADO_AUTH_PAT))
print(ado_req_attach_File.text)
Same code is working when I am using python 3.8 in my local visual studio code but not working when I am using Azure Automation Runbook (python 2.7).
When I try to print text of the response body, I am getting below error:
print(ado_req_attach_File.text)UnicodeEncodeError: 'ascii' codec can't encode character u'\u221e' in position 6303: ordinal not in range(128)
Expected Output:
{"id":"facedff6-48c6-5479-894b-f7807f29b96e","url":"https://dev.azure.com/orgname/d93740f8-fe37-5433-bc8e-79c0a320d81b/_apis/wit/attachments/facedff6-48c6-5479-894b-f7807f29b96e?fileName=details.csv"}
This seems not related to Azure DevOps side, since your API works properly. And you have got the JSON response succeed.
For version 2.7 ,encoding is not set by default.
Please use below code as a first line of program and check if it works:
# -*- coding: utf-8 -*-
# Your code goes below this line
For python 3.x ,there is default encoding.hence there will be no issue of encoding.
Also take a look at similar issue here for more info: https://stackoverflow.com/a/39293287/5391065
Related
When running the following from my laptop:
import sys, os, boto3, json
payload = json.dumps({'query':query})
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('table1')
response = table.get_item(Key={'item': 'item1'})
I get the error:
botocore.exceptions.SSLError: SSL validation failed for https://dynamodb.eu-west-2.amazonaws.com/ [Errno 2] No such file or directory
It crashes on the last line with the "table.get_item(". This error appeared today for the first time. Prior to that, the same code had been running fine on the same laptop for the past two years. The same code still runs fine in AWS Lambda. The code is in Python 2.7.
I have been trying to resolve this issue for the past six hours. I have reinstalled boto3, botocore, awscli. Reconfigured aws cli. Updated pip and all the modules I use in Python.
Any help would be appreciated. Thank you.
Tesseract OCR on AWS Lambda via virtualenv
Scroll to Adapatations for tesseract 4.
I have used this link to create the executable and the dependency libraries for tesseract. I have zipped everything and dropped in S3.
I am using lambda to download this zip, extract the dependencies in to /tmp folder. Now I am planning to use these dependencies in my lambda(python3 platform).
I am getting this error
Response:
{
"errorMessage": "tesseract is not installed or it's not in your path",
"errorType": "TesseractNotFoundError",
This is happening cause of not setting the environmental variable.
I have tried to do it but cannot by pass this error.
# Setting the modules path
sys.path.insert(0, '/tmp/')
import boto3
import cv2
import numpy as np
import subprocess
os.environ['PATH'] = "{}:/tmp/pytesseract:/tmp/".format(os.environ['PATH'])
os.environ['TESSDATA_PREFIX'] = "/tmp/tessdata/"
import pytesseract
I have set the environmental variables like this in the lambda function. Still I am getting the same error. I have even tried setting the variables like shown in the image below. Still hard luck.
I am sure this lambda package works because I have created a new ec2 instance, downloaded the same zip file and extracted the libraries into /tmp/ folder. I wrote a basic test function for testing tesseract. This works.
import cv2
import pytesseract
import os
# os.environ['PATH'] = "{}:/tmp/pytesseract:/tmp/".format(os.environ['PATH'])
os.environ['LD_LIBRARY_PATH'] = '/tmp/lib:/tmp'
config = ('-l eng --oem 1 --psm 3')
im = cv2.imread('pytesseract/test-european.jpg', cv2.IMREAD_COLOR)
text = pytesseract.image_to_string(im, config=config)
print(text)
Can somebody tell me what did I do wrong with lambda.
I don't want to zip everything because my zip file is greater than 50 MB. Also I want to try downloading the packages/modules/binaries from S3 to lambda and make it work.
Apparently lambda doesn't allow you to make changes to the PATH variable.
Try adding this to your script
pytesseract.pytesseract.tesseract_cmd = r'/var/task/tesseract'
I'm just starting to work with AWS and I'm working on getting Anaconda set up on my AWS EC2 Instance. The entire set up process worked well but i'm getting stuck with inputting my password in jupyter.
FYI, here is the tutorial I have been using: https://chrisalbon.com/jupyter/run_project_jupyter_on_amazon_ec2.html
I manually changed the password in the ipython config file:
Even when inputting testing123 I get an invalid error message.
Have you guys dealt with this? Any help would be much appreciated.
I think you have understood the whole process but got confused in password section so I am only gonna point out what you did wrong.
Assuming You followed below blog as you said:
https://chrisalbon.com/jupyter/run_project_jupyter_on_amazon_ec2.html
In the section
Create a password for jupyter notebook
1 ipython
2 from IPython.lib import passwd
3 passwd()
4 'sha1:98ff0e580111:12798c72623a6eecd54b51c006b1050f0ac1a62d'
5 exit
I have added the line numbers for the sake of better explanation:
when you write passwd() which is in 3rd line
The AWS expects you to input a password and again ask you to verify the password you have given and then it gives you a SHA version of your password. like on 4th line: 'sha1:98ff0e580111:12798c72623a6eecd54b51c006b1050f0ac1a62d'
Now you have to copy this and keep it with yourself to paste it later in ipython config file.
so you should use below lines in your config to make it work for you:
c = get_config()
# Kernel config
c.IPKernelApp.pylab = 'inline' # if you want plotting support always in your notebook
# Notebook config
c.NotebookApp.certfile = u'/home/ubuntu/certs/mycert.pem' #location of your certificate file
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False #so that the ipython notebook does not opens up a browser by default
c.NotebookApp.password = u'sha1:98ff0e580111:12798c72623a6eecd54b51c006b1050f0ac1a62d' #the encrypted password we generated above
# It is a good idea to put it on a known, fixed port
c.NotebookApp.port = 8888
I tried to run tensorflow-wavenet on the google cloud ml-engine with gcloud ml-engine jobs submit training but the cloud job crashed when it was trying to read the json configuration file:
with open(args.wavenet_params, 'r') as f:
wavenet_params = json.load(f)
arg.wavenet_params is simply a file path to a json file which I uploaded to the google cloud storage bucket. The file path looks like this: gs://BUCKET_NAME/FILE_PATH.json.
I double-checked that the file path is correct and I'm sure that this part is responsible for the crash since I commented out everything else.
The crash log file doesn't give much information about what has happened:
Module raised an exception for failing to call a subprocess Command '['python', '-m', u'gcwavenet.train', u'--data_dir', u'gs://wavenet-test-data/VCTK-Corpus-Small/', u'--logdir_root', u'gs://wavenet-test-data//gcwavenet10/logs']' returned non-zero exit status 1.
I replaced wavenet_params = json.load(f) by f.close() and I still get the same result.
Everything works when I run it locally with gcloud ml-engine local train.
I think the problem is with reading files with gcloud ml-engine in general or that I can't access the google cloud bucket from within a python file with gs://BUCKET_NAME/FILE_PATH.
Python's open function cannot read files from GCS. You will need to use a library capable of doing so. TensorFlow includes one such library:
import tensorflow as tf
from tensorflow.python.lib.io import file_io
with file_io.FileIO(args.wavenet_params, 'r') as f:
wavenet_params = json.load(f)
I am running a flask app on an AWS EC2 server, and have been using boto to access data stored in dynamoDB. After accidentally adding boto.conf to a git commit (and push and pull on the server), I have found that my python code can no longer locate the boto.conf file. I rolled back the changes with git, but the problem remains.
The python module and boto.conf file exist in the same directory, but when the module calls
boto.config.load_credential_file('boto.conf')
I get the flask error IOError: [Errno 2] No such file or directory: 'boto.conf'.
As per Documentation:
I'm not really sure why you are using boto.config_load_credential_file. In general you can pick up the config in a file called either ~/.boto or /etc/boto.cfg.
You can also look at this questions from SO that also answers how to get the configuration for boto: Getting Credentials File in the boto.cfg for Python