SageMaker end point deployment keeps failing trying to find a dependency file.
In the below example, I'm using the "lables.txt" in a function load_lables() that gets called in the model_fn() function
project folder structure
--model (directory)
|--code (directory)
|--requirements.txt
|--train.py (entry point)
|--labels.txt
|--notebook_train_deploy.ipynb
train.py
def load_labels(file_name_category='labels.txt'):
labels = list()
with open(file_name_category) as label_file:
for line in label_file:
labels.append(line.strip().split(' ')[0][:])
_out_labels = tuple(labels)
return _out_labels
def model_fn(model_dir):
labels = load_labels()
num_labels = len(labels)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model_ft = models.resnet18(pretrained=True)
.
.
.
with open(os.path.join(model_dir, 'model.pth'), 'rb') as f:
model_ft .load_state_dict(torch.load(f))
model_ft .eval()
return model_ft .to(device)
notebook_train_deploy.ipynb
pytorch_model = PyTorchModel(model_data='s3://sagemaker-poc/model.tar.gz',
role=role,
source_dir='code',
entry_point='train.py',
framework_version='1.0.0',
dependencies=['./code/labels.txt']
)
predictor = pytorch_model.deploy(
instance_type='ml.t2.medium',
initial_instance_count=1)
ERROR
algo-1-esw8d_1 | [2020-04-29 19:33:40 +0000] [22] [ERROR] Error handling request /ping
algo-1-esw8d_1 | Traceback (most recent call last):
algo-1-esw8d_1 | File "/usr/local/lib/python3.6/dist-packages/sagemaker_containers/_functions.py", line 85, in wrapper
algo-1-esw8d_1 | return fn(*args, **kwargs)
algo-1-esw8d_1 | File "/usr/local/lib/python3.6/dist-packages/train.py", line 49, in model_fn
algo-1-esw8d_1 | labels = load_labels()
algo-1-esw8d_1 | File "/usr/local/lib/python3.6/dist-packages/train.py", line 25, in load_labels
algo-1-esw8d_1 | with open(os.path.join(file_name_category)) as label_file:
algo-1-esw8d_1 | FileNotFoundError: [Errno 2] No such file or directory: 'labels.txt'
Related
EDIT 1
to test, I just change primary from blue to purple
My scss seems to be compiled and get (return 200)
If I reload the page, I can see it is applyed as my primary button is briefly purple but immediatly "override" by intial blue color
what is wrong ?
I try to see if other bootstrap css was loaded but no
app architecture
- static
- bootstrap
- CACHE
- css
- theme.7ba97bdae4cb.css
- css
- js
- theme.scss
settings.py
STATIC_URL = '/static/'
STATIC_ROOT = 'static'
STATICFILES_DIRS = (
os.path.join(BASE_DIR,'project/static'),
)
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
]
COMPRESS_PRECOMPILERS = (
('text/x-scss', 'django_libsass.SassCompiler'),
)
I find the previous issue came from DEBUG set in settings.py (I cast in int and to be honnest do not remember excatly why)
I want to use SCSS in my Django project running with Docker in order to customize bootstrap
I have follow this tutorial https://www.accordbox.com/blog/how-use-scss-sass-your-django-project-python-way/
First I had error while building docker container that I manage to resolve.
But when I run it, and try to reach home page, I got an error that seems to deal with connection
source_comments must be bool, not 1 with {% compress css %} tag
I run logs to complete trace but I do not understand but error does not deal with this...
thanks for help
Internal Server Error: /
web | Traceback (most recent call last):
web | File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
web | response = get_response(request)
web | File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
web | response = self.process_exception_by_middleware(e, request)
web | File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
web | response = wrapped_callback(request, *callback_args, **callback_kwargs)
web | File "/usr/src/app/core/views.py", line 12, in home
web | return render(request, 'home.html', {
web | File "/usr/local/lib/python3.8/site-packages/django/shortcuts.py", line 36, in render
web | content = loader.render_to_string(template_name, context, request, using=using)
web | File "/usr/local/lib/python3.8/site-packages/django/template/loader.py", line 62, in render_to_string
web | return template.render(context, request)
web | File "/usr/local/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
web | return self.template.render(context)
web | File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 171, in render
web | return self._render(context)
web | File "/usr/local/lib/python3.8/site-packages/django/test/utils.py", line 96, in instrumented_test_render
web | return self.nodelist.render(context)
web | File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 937, in render
web | bit = node.render_annotated(context)
web | File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 904, in render_annotated
web | return self.render(context)
web | File "/usr/local/lib/python3.8/site-packages/django/template/loader_tags.py", line 150, in render
web | return compiled_parent._render(context)
web | File "/usr/local/lib/python3.8/site-packages/django/test/utils.py", line 96, in instrumented_test_render
web | return self.nodelist.render(context)
web | File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 937, in render
web | bit = node.render_annotated(context)
web | File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 904, in render_annotated
web | return self.render(context)
web | File "/usr/local/lib/python3.8/site-packages/compressor/templatetags/compress.py", line 131, in render
web | return self.render_compressed(context, self.kind, self.mode, forced=forced)
web | File "/usr/local/lib/python3.8/site-packages/compressor/templatetags/compress.py", line 107, in render_compressed
web | rendered_output = compressor.output(mode, forced=forced)
web | File "/usr/local/lib/python3.8/site-packages/compressor/css.py", line 49, in output
web | ret.append(subnode.output(*args, **kwargs))
web | File "/usr/local/lib/python3.8/site-packages/compressor/css.py", line 51, in output
web | return super(CssCompressor, self).output(*args, **kwargs)
web | File "/usr/local/lib/python3.8/site-packages/compressor/base.py", line 295, in output
web | output = '\n'.join(self.filter_input(forced))
web | File "/usr/local/lib/python3.8/site-packages/compressor/base.py", line 235, in filter_input
web | for hunk in self.hunks(forced):
web | File "/usr/local/lib/python3.8/site-packages/compressor/base.py", line 205, in hunks
web | precompiled, value = self.precompile(value, **options)
web | File "/usr/local/lib/python3.8/site-packages/compressor/base.py", line 276, in precompile
web | return True, filter.input(**kwargs)
web | File "/usr/local/lib/python3.8/site-packages/django_libsass.py", line 138, in input
web | return compile(filename=self.filename,
web | File "/usr/local/lib/python3.8/site-packages/django_libsass.py", line 125, in compile
web | output = sass.compile(**kwargs)
web | File "/usr/local/lib/python3.8/site-packages/sass.py", line 607, in compile
web | raise TypeError(
web | TypeError: source_comments must be bool, not 1
web | [28/Jan/2021 14:19:03] "GET / HTTP/1.1" 500 177528
web | ----------------------------------------
web | Exception happened during processing of request from ('127.0.0.1', 44218)
web | Traceback (most recent call last):
web | File "/usr/local/lib/python3.8/socketserver.py", line 650, in process_request_thread
web | self.finish_request(request, client_address)
web | File "/usr/local/lib/python3.8/socketserver.py", line 360, in finish_request
web | self.RequestHandlerClass(request, client_address, self)
web | File "/usr/local/lib/python3.8/socketserver.py", line 720, in __init__
web | self.handle()
web | File "/usr/local/lib/python3.8/site-packages/django/core/servers/basehttp.py", line 171, in handle
web | self.handle_one_request()
web | File "/usr/local/lib/python3.8/site-packages/django/core/servers/basehttp.py", line 179, in handle_one_request
web | self.raw_requestline = self.rfile.readline(65537)
web | File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
web | return self._sock.recv_into(b)
web | ConnectionResetError: [Errno 104] Connection reset by peer
web | ----------------------------------------
You probably have another bootstrap theme that is loaded
Im have some issue while running example Keras2DML code in this page. While running the code, i've got this error:
Traceback (most recent call last):
File "/home/fregy/kerasplayground/sysml/examplenn.py", line 12, in <module>
sysml_model = Keras2DML(spark, keras_model,input_shape=(3,224,224))
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/estimators.py", line 909, in __init__
convertKerasToCaffeNetwork(keras_model, self.name + ".proto")
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 201, in convertKerasToCaffeNetwork
jsonLayers = list(chain.from_iterable(imap(lambda layer: _parseKerasLayer(layer), kerasModel.layers)))
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 201, in <lambda>
jsonLayers = list(chain.from_iterable(imap(lambda layer: _parseKerasLayer(layer), kerasModel.layers)))
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 137, in _parseKerasLayer
ret = { 'layer': { 'name':layer.name, 'type':supportedLayers[layerType], 'bottom':_getBottomLayers(layer), 'top':layer.name, paramName:param[paramName] } }
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 112, in _getBottomLayers
return [ bottomLayer.name for bottomLayer in _getInboundLayers(layer) ]
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 70, in _getInboundLayers
for node in layer.inbound_nodes: # get inbound nodes to current layer
AttributeError: 'Conv2D' object has no attribute 'inbound_nodes'
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/SocketServer.py", line 230, in serve_forever
r, w, e = _eintr_retry(select.select, [self], [], [],
AttributeError: 'NoneType' object has no attribute 'select'
Im using Tensorflow-GPU 1.5 , and Keras 2.1.3 .
Thanks for trying out Keras2DML. The issue arises because the newer Keras versions renamed the attribute inbound_nodes to _inbound_nodes. This issue was fixed in yesterday's commit: https://github.com/apache/systemml/commit/9c3057a34c84d5bf1c698ad0a5c3c34d90412dbb.
Since you are using TensorFlow-GPU, you may want to check if TF grabs onto most of GPU memory when Keras model is compiled using nvidia-smi. If yes, here are two easy workarounds:
a. Hide GPUs from TF:
os.environ['CUDA_DEVICE_ORDER'] = 'PCI_BUS_ID'
os.environ['CUDA_VISIBLE_DEVICES'] = ''
import tensorflow as tf
b. Or minimize the overhead due to TensorFlow:
from keras.backend.tensorflow_backend import set_session
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
set_session(tf.Session(config=tf_config))
I have a problem with tests. When I run some tests I launch separately, they pass. When all together then fail.
#mock.patch(
'apps.abstract.validators.years_range_is_not_future', new=fake_years_range_is_not_future
)
def test_create_building_with_validation_of_foundation_period(self):
self.c.login(username=self.user.username, password='111')
response = self.c.post(
'/en/api/buildings/',
data={
'name': "New Building",
'foundation_period': {
'lower': MIN_INT_VALUE,
'upper': MAX_INT_VALUE
},
'stream': {
'uuid': s_uuid(self.stream)
}
}
)
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
I read about this problem here
why would a django test fail only when the full test suite is run?
and tried to patch the validator in the serializer file as shown here
#mock.patch(
'apps.buildings.api.serializers.years_range_is_not_future', new=fake_years_range_is_not_future
)
def test_create_building_with_validation_of_foundation_period(self):
..............................................................
but then I get an incomprehensible for me exception
Error
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/mock.py", line 1049, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'apps.buildings.api' has no attribute 'serializers'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/mock.py", line 1149, in patched
arg = patching.__enter__()
File "/usr/lib/python3.5/unittest/mock.py", line 1205, in __enter__
self.target = self.getter()
File "/usr/lib/python3.5/unittest/mock.py", line 1375, in <lambda>
getter = lambda: _importer(target)
File "/usr/lib/python3.5/unittest/mock.py", line 1062, in _importer
thing = _dot_lookup(thing, comp, import_path)
File "/usr/lib/python3.5/unittest/mock.py", line 1051, in _dot_lookup
__import__(import_path)
File "/home/env/project/apps/buildings/api/serializers.py", line 12, in <module>
from apps.communities.api.serializers import CommunityBriefSerializer
File "/home/env/project/apps/communities/api/serializers.py", line 297, in <module>
class CommunityOfficialRequestBuildingSerializer(BaseCommunityOfficialRequestSerializer):
File "/home/rp/env/project/apps/communities/api/serializers.py", line 299, in CommunityOfficialRequestBuildingSerializer
from apps.buildings.api.serializers import BuildingBriefSerializer
ImportError: cannot import name 'BuildingBriefSerializer'
help please understand what I'm doing wrong
project structure (__init__.py files not listed)
project
|__apps
|__communities
| |_api
| |_serializers.py
|
|__buildings
| |_api
| | |_serializers.py
| |
| |_tests
| |_test.py
|
|_abstract
|_validators.py
Seeing this,
Traceback (most recent call last):
File "/home/rp/env/project/apps/communities/api/serializers.py", line 299, in CommunityOfficialRequestBuildingSerializer
from apps.buildings.api.serializers import BuildingBriefSerializer
suggests that your import statement is inside a class or a def or some other statement.
Maybe the import statement is executed after you mocked apps.buildings.api.serializers. If you move this import to the top of the file, then BuildingBriefSerializer will probably become available before apps.buildings.api.serializers is mocked, and your tests will pass.
This would also explain why the tests run, when you run them individually.
I am using celery for my django project.
java_path = "/your/Java/jdk/home/java.exe
os.environ['JAVAHOME'] = java_path
st = POSTagger('/your/postagger/models/path/english-bidirectional-distsim.tagger','/your/postagger/jar/file/path/stanford-postagger.jar')
tag = st.tag([key])
here the key is a list of feature words.
I got following errors, when using celery to execute:
raised unexpected:
LookupError('\n\n===========================================================================\nNLTK
`was unable to find the java file!\nUse software specific configuration paramaters or set the JAVAHOME environment`
variable.\n===========================================================================',)
Traceback (most recent call last):
File "/Users/Envs/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/Users/Envs/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
return self.run(*args, **kwargs)
File "/Users/Envs/src/evolvelearning/tasks.py", line 772, in RunProgram
tag = st.tag([key])
File "/Users/Envs/lib/python2.7/site-packages/nltk/tag/stanford.py", line 59, in tag
return self.tag_sents([tokens])[0]
File "/Users/Envs/lib/python2.7/site-packages/nltk/tag/stanford.py", line 64, in tag_sents
config_java(options=self.java_options, verbose=False)
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 82, in config_java
_java_bin = find_binary('java', bin, env_vars=['JAVAHOME', 'JAVA_HOME'], verbose=verbose, binary_names=['java.exe'])
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 544, in find_binary
binary_names, url, verbose))
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 538, in find_binary_iter
url, verbose):
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 517, in find_file_iter
raise LookupError('\n\n%s\n%s\n%s' % (div, msg, div))
LookupError:
===========================================================================
NLTK was unable to find the java file!
Use software specific configuration paramaters or set the JAVAHOME environment variable.
===========================================================================
I have set java_path and javahome and I would like to know why still thess errors occur?
my environment is MAC.
The problem is the java_path, should be:
java_path = "/your/Java/jdk/home
I tried to group serveral apps into a "app" folder for better management. however i got the below error when i did the first migration.
./manage.py makemigrations
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
utility.execute()
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/core/management/__init__.py", line 342, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/core/management/commands/makemigrations.py", line 132, in handle
migration_name=self.migration_name,
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/migrations/autodetector.py", line 45, in changes
changes = self._detect_changes(convert_apps, graph)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/migrations/autodetector.py", line 128, in _detect_changes
self.old_apps = self.from_state.concrete_apps
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/migrations/state.py", line 166, in concrete_apps
self.apps = StateApps(self.real_apps, self.models, ignore_swappable=True)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/migrations/state.py", line 228, in __init__
self.render_multiple(list(models.values()) + self.real_models)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/migrations/state.py", line 296, in render_multiple
model.render(self)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/migrations/state.py", line 585, in render
body,
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/models/base.py", line 158, in __new__
new_class.add_to_class(obj_name, obj)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/models/base.py", line 299, in add_to_class
value.contribute_to_class(cls, name)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/models/fields/related.py", line 702, in contribute_to_class
super(ForeignObject, self).contribute_to_class(cls, name, virtual_only=virtual_only)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/models/fields/related.py", line 308, in contribute_to_class
lazy_related_operation(resolve_related_class, cls, self.remote_field.model, field=self)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/models/fields/related.py", line 85, in lazy_related_operation
return apps.lazy_model_operation(partial(function, **kwargs), *model_keys)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/models/fields/related.py", line 83, in <genexpr>
model_keys = (make_model_tuple(m) for m in models)
File "/Users/reonard/.virtualenvs/django-ito/lib/python2.7/site-packages/django/db/models/utils.py", line 14, in make_model_tuple
app_label, model_name = model.split(".")
ValueError: too many values to unpack
I added a print statement for the model splitted and the output was:
auth.User
auth.Group_permissions
auth.User_groups
auth.User_user_permissions
app.ticketmgr.BasePriority
It appears that the "app.ticketmgr.BasePriority" has an extra "dot". Actually the app name is "ticketmgr" but put in an "app" folder for better managerment. May i know what should i do to make the migrations work? is there any extra settings should be done to support putting serveral apps in a "app" folder ?
Below is my app folder, there are 2 apps (comm, ticketmgr) in it.
|____app
| |______init__.py
| |____comm
| | |______init__.py
| | |____admin.py
| | |____apps.py
| | |____migrations
| | | |______init__.py
| | |____models.py
| | |____role_required.py
| | |____tests.py
| | |____views.py
| |____ticketmgr
| | |______init__.py
| | |____admin.py
| | |____apps.py
| | |____forms.py
| | |____migrations
| | | |____0001_initial.py
| | | |______init__.py
| | |____models.py
| | |____tests.py
| | |____views.py
Nothing more is needed. Just add "app.ticketmgr", "app.comm" to INSTALLED_APPS.
BasePriority does not appear in your project directories. Maybe it's a model?
The installed app line should be:
app.ticketmgr
Since BasePriority does not exists, django gets 3 arguments instead of the 2 required for app_label,model_name. The "BasePriority" is the third argument, which excepts.