After spending multiple hours, I created this sample model endpoint code I want to host on vertex.ai, these are the three files I have
1. Dockerfile
FROM python:3.7
USER root
ADD . /home/model-server/
WORKDIR /home/model-server/
RUN pip3 install --upgrade pip
RUN pip install -r ./requirements.txt
CMD exec gunicorn -b :5000 --max-requests 1 --graceful-timeout 300 -t 600 main:app
Main.py -- contains flask app
import json
from flask import Flask, request, Response, jsonify
app = Flask(__name__)
app.config['JSON_SORT_KEYS'] = False
#app.route('/health', methods=['GET'])
def health_check():
return Response(response=json.dumps({"status": 'healthy'}), status=200, mimetype="application/json")
#app.route('/vertex_predict/', methods=['POST'])
def main():
request_json = request.get_json()
request_instances = request_json['instances']
output = {'predictions':
[
{
'result' : "this is working with static output"
}
]
}
return jsonify(output)
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
Requirement.txt
pandas==1.1.1
numpy==1.21.5
flask==2.2.2
gunicorn==20.1.0
this is pretty simple code which we are able to put in artifact registry then
Import this as model in vertex.ai Model registry
Then trying to deloy it as endpoint
I tried multiple times but after almost 20 mins, it fails and showing this error all the time
jsonPayload: {
levelname: "ERROR"
logtag: "F"
message: "exec /bin/sh: exec format error"
}
the same flask app is working fine on local machine, not sure what is wrong here on vertex.ai. Can someone please help me here or guide the next steps
Related
I'm using Python 3.7.8
ModuleNotFound: No module named 'SpeechRecognition' appears when I try to run my project.
ModuleNotFound: No module named 'SpeechRecognition'
But it works well when I try it with python -m speech_recognition:
enter image description here
How to start flask app. I recommend to start like this. main.py is flask start script file.
# set flask start script file
export FLASK_APP=main.py
# run flask
python3 -m flask run
# main.py
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello, World!'
code
from flask import Flask, render_template, session, request
from flask_session import Session
app = Flask(__name__)
app.config["SESSION_PERMANENT"] = False
app.config["SESSION_TYPE"]= "filesystem"
Session(app)
#app.route("/", methods=["GET", "POST"])
def index():
if session.get("comments") is None:
session["comments"] = []
if request.method == "POST":
comment = request.form.get("comment")
session["comments"].append(comment)
CMD
(env) E:\COURSES\CS50W\lec2\exersice>set FLASK_APP=application.py
(env) E:\COURSES\CS50W\lec2\exersice>python -m flask run
E:\COURSES\CS50W\lec2\exersice\env\Scripts\python.exe: No module named flask
Check this out.
If you are not sure whether you have installed flask or not, try running only flask in the cmd or any terminal.If you get any commands about flask then you can be sure that it is installed or you can install by using
pip install flask
If you are concerned about the versions of the libraries you use for your application, then follow the above mentioned link!
I created a flask application that consists of MQTT client which simultaneously subscribe data from esp32 and store it in database.
from flask import Flask, redirect, render_template, request, session, abort,make_response,jsonify, flash, url_for
import paho.mqtt.client as mqtt
import json
import config
import db_access
#mqtt code
def on_message(client, userdata, message):
topic = message.topic
print("line 12 - topic checkpoint - ",topic)
msgDecode=str(message.payload.decode("utf-8","ignore"))
msgJson=json.loads(msgDecode) #decode json data
print("line 15 - json checkpoint - ",type(msgJson))
# deviceID = msgJson["DeviceID"]
# currentCounter = msgJson["Counter"]
# status = msgJson["Status"]
db_access.updateStatus(msgJson["DeviceID"],msgJson["Status"])
app = Flask(__name__,template_folder='templates')
'''Web portal routes'''
#app.route('/device/switch', methods=['POST'])
def switch():
#parameter parsing
deviceID = request.args.get('deviceID')
status = request.args.get('status')
statusMap = {"on":1,"off":0}
#MQTT publish
mqtt_msg = json.dumps({"deviceID":int(deviceID),"status":statusMap[status]})
client.publish(config.MQTT_STATUS_CHANGE_TOPIC,mqtt_msg)
time_over_flag = 0
loop_Counter = 0
while status != db_access.getDeviceStatus(deviceID):
time.sleep(2)
loop_Counter+=1
if loop_Counter ==2:
time_over_flag = 1
break
if time_over_flag:
return make_response(jsonify({"statusChange":False}))
else:
return make_response(jsonify({"statusChange":True}))
if __name__ == "__main__":
db_access.createUserTable()
db_access.insertUserData()
db_access.createDeviceTable()
db_access.insertDeviceData()
print("creating new instance")
client = mqtt.Client("server") #create new instance
client.on_message=on_message #attach function to callback
print("connecting to broker")
client.connect(config.MQTT_BROKER_ADDRESS)
client.loop_start()
print("Subscribing to topic","esp/#")
client.subscribe("esp/#")
app.run(debug=True, use_reloader=False)
This is code in ____init____.py
db_access.py is consist of database operations and config.py consist of configurations.
Will this work in apache?
also, I have no previous experience with WSGI
The problem with launching your code (as included) with a WSGI server, is the part in that last if block, only gets run when you execute that file directly with the python command.
To make this work, I'd try moving that block of code to the top of your file, around here:
app = Flask(__name__,template_folder='templates')
db_access.createUserTable()
db_access.insertUserData()
db_access.createDeviceTable()
db_access.insertDeviceData()
print("creating new instance")
client = mqtt.Client("server") #create new instance
client.on_message=on_message #attach function to callback
print("connecting to broker")
client.connect(config.MQTT_BROKER_ADDRESS)
client.loop_start()
print("Subscribing to topic","esp/#")
client.subscribe("esp/#")
I'd also rename that __init__.py file, to something like server.py as the init file isn't meant to be heavy.
Once you've done that, run it with the development server again and test that it works as expected.
Then install a WSGI server like gunicorn into your virtual enviornment:
pip install gunicorn
And launch the app with gunicorn (this command should work, assuming you renamed your file to server.py):
gunicorn --bind '0.0.0.0:5000' server:app
Then test again.
I am deploying my watson assistant chatbot on Flask + Gunicorn + Nginx.
I am able to successfully dockerize and run , but something is breaking my code. Multiple watson assistant sessions are being created while I send the messages to watson services. While I try to reply for an intent I get answer for another intent or slot or does not understand message
I have reviewed all the tutorials on digital ocean and github, but I think creating chatbot session should be handled differently.
app.py
from flask import Flask, render_template,Response,make_response,jsonify
import os
from ibm_watson import AssistantV2
import random
from random import randint
import json
#import report
from io import StringIO
app = Flask(__name__)
conversation = AssistantV2(
iam_apikey = 'key',
url='https://gateway.watsonplatform.net/assistant/api',
version='2019-05-19')
session = conversation.create_session("someid").get_result()
variables = None
#context_val = {}
#app.route('/')
#app.route('/index')
def chat():
return render_template('chat.html')
#app.route('/send_message/<message>')
def send_mesage(message):
text = ''
response = conversation.message(
assistant_id = 'id',
session_id= session['session_id'],input={'text': str(message),'options': {
'return_context': True}}
).get_result()
variables = response['output'].get('user_defined')
#context = response['context']['skills']['main skill']['user_defined']
for i in response['output']['generic']:
text = text+ i['text']+'\n'
return text
if __name__ == "__main__":
app.run(host='0.0.0.0')
wsgi.py
from app import app
if __name__ == "__main__":
app.run()
Dockerfile
FROM python:3.6
WORKDIR /app
ADD . /app
RUN chgrp -R 0 /app/app.log && chmod -R g=u /app/app.log
RUN pip install -r requirements.txt
EXPOSE 8080
CMD ["gunicorn", "-b", "0.0.0.0:8080", "app", "-p 8080:8080"]
RUN chmod 770 /app
USER 1001
When working with IBM Watson Assistant with the V2 API, you need to be aware of the following objects:
First, you create an Assistant. It manages the connection to Watson Assistant.
Next, a Session is a per user interaction within the chat.
Last, a Message flows to Watson within a session with a Response coming back.
You probably have seen this simple code sample in the docs, your own code is - on a general level - similar. To make it work, you need to create Watson sessions per user sessions, then send the messages as part of the corresponding session. That way, the chat context is kept correctly. Your code currently initialize Watson and creates a session once. You need to create a session per user. Look into session management.
I use celery in my django project. It works well on my MacBook and in a CentOS VM. When I run it in a docker container, the request which contains add.delay(add is a task) method is always blocked.
I created a demo project on github: https://github.com/fengyouchao/proj_test
My task:
#shared_task
def add(x, y):
return x + y
My view:
def index(request):
a = int(request.GET.get('a', 1))
b = int(request.GET.get('b', 2))
add.delay(a, b)
return HttpResponse("Hello world")
def hello(request):
return HttpResponse("hello")
In the demo project I created three services in docker-compose.yml:
web - The service which run "manage.py runserver 0.0.0.0:8000"
celery - The service which run "celery"
rabbitmq - The service wich run rabbitmq-server
Run services
docker-compose up
Test
curl localhost:8000 # blocked
curl localhost:8000/hello # OK
Run the django project in current system(use the same rabbitmq-server in docker container)
manage.py runserver 0.0.0.0:18000
Test
curl localhost:18000 # OK , and the "celery" service printed task logs
This problem has been bothering me for a long time, and I don't know where the problem is. I hope someone can help me. Thanks!
I just came across a similar issue,
I am using rabbitmq container as a broker so added CELERY_BROKER_URL in settings.py
when I run the add.delay() in manage.py django shell, inside the container it got struck but works fine in production
So I have added the following change, it started working
app = Celery('app', broker="amqp://rabbitmq")
I have faced the same issue, and fixed by it importing the app created on proj/proj/celery.py in my proj/proj/__init__.py like this:
from __future__ import absolute_import, unicode_literals
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app
__all__ = ('celery_app',)
You can see more information in Celery's first steps with django documentation.
Hope it helps!