I'm basically attempting py2/3 compatibility by trying to add a a fullmatch method to Python2 re compiled patterns.
I've seen https://stackoverflow.com/a/2982/2694385 on how to add an attribute to an existing instance.
I'm creating the method as given in https://stackoverflow.com/a/30212799/2694385.
Example code
import re
import six
regex = re.compile('some pattern') # Better if we keep this same
if six.PY2:
def fullmatch(self_regex, string, flags=0):
return self_regex.match(string, flags=flags)
regex = re.compile(r'(?:' + regex.pattern + r')\Z', flags=regex.flags)
import types
bound = types.MethodType(fullmatch, regex)
# AttributeError here. None of the following three lines work
regex.fullmatch = bound
regex.__setattr__('fullmatch', bound)
setattr(regex, 'fullmatch', bound)
That ain't gonna work - regex objects are created on the C side and they don't represent your regular instances so you cannot modify their signature from within Python. For example, if you try to extend _sre.SRE_Pattern:
import _sre
class CustomPattern(_sre.SRE_Pattern): pass
You'll get an AttributeError complaining that such object doesn't exist in the _sre module. If you try to cheat it with:
import re
tmp_pattern = re.compile("")
sre_pat = type(tmp_pattern)
class CustomPattern(sre_pat): pass
it will give you a TypeError complaining that _sre.SRE_Pattern (which now 'temporarily' exists as it's being created ad-hoc) is not an acceptable base type.
What you can do instead is to create a full wrapper around the re module (or at least add the missing structures to it) and handle the Python version differences on the Python's side, although I think it's just not worth it.
P.S. If you're not using six anywhere else, there no reason for the overhead just to check your Python version - you can use sys.version_info.major < 3 instead.
See nlpia.regexes.Pattern for something similar to what you want -- a Frankenstein mashup of _sre.Pattern with a fullmatch() method. This monkey-patching "inheritance" approach works in Python 2 and 3.
import re
import regex
class Pattern:
""" "Inherits" _sre.SRE_Pattern and adds .fullmatch() method
>>> pattern = Pattern('Aaron[ ]Swartz')
>>> pattern.match('Aaron Swartz')
<_sre.SRE_Match object; span=(0, 12), match='Aaron Swartz'>
>>> pattern.fullmatch('Aaron Swartz!!')
>>> pattern.match('Aaron Swartz!!')
<_sre.SRE_Match object; span=(0, 12), match='Aaron Swartz'>
"""
def __init__(self, pattern):
self._compiled_pattern = re.compile(pattern)
for name in dir(self._compiled_pattern):
if not name in set(('__class__', '__init__', 'fullmatch')):
attr = getattr(self._compiled_pattern, name)
setattr(self, name, attr)
def fullmatch(self, *args, **kwargs):
return regex.fullmatch(self._compiled_pattern.pattern, *args, **kwargs)
Related
I am trying to figure out how I can print an indexed variable in SymPy to make it look cleaner than below. I am not able to use Mathjax for some reason, so I apologize that there is just a photo to work with.
import sympy
from sympy import *
init_printing(use_latex='mathjax')
S = IndexedBase('S')
i,j,t = Idx('i'),Idx('j'),Idx('t')
S[i]
As my previous comment states, it is currently not supported in the existing latex printer.
However, you can manually implement _latex(self, expr) for Idx, or use a custom printer:
from sympy import *
from sympy.printing.latex import LatexPrinter
class CustomLatexPrinter(LatexPrinter):
def _print_Idx(self, expr):
return expr.name
#classmethod
def printer(cls, expr, **kwargs):
return cls(kwargs).doprint(expr)
init_printing(use_latex='mathjax', latex_printer=CustomLatexPrinter.printer)
All I do here is simple implement _print_Idx to return the label as a string (through the name property) and provide a printer function to match the signature init_printing requires for a latex_printer.
Then, following your example
An object has been serialized by pickle, and it will be used by a model which would be placed at anyplace (under any directory). Since the object is frequently used and kind of a part of the model, I want to have the model contain the pickle file (place the file under a directory of the model) as a variable.
./data/constant.py
object = pickle.load(open('object.pkl'))
./data/object.pkl
./code/model01.py
from ..data import constant
# or
# from __future import absolute_import
# from model.data import constant
object = constant.object
./code/model02.py
from ..data import constant
object = constant.object
The problem is obviously that python will search object.pkl under ./code/(and anywhere I use the function of the model outside of the model) rather than ./data/.
Am I doing it right? Any better solutions? Thanks.
I think this question may be duplicated (this is a very common issue) but I cannot find any related archive here. If so, please help redirect me there.
Doing a little bit of path manipulation should work:
In module constant.py:
import os
path = os.path.dirname(os.path.abspath(__file__))
obj = pickle.load(open(os.path.join(path, 'object.pkl')))
Looks like you want object to be part of the module constant.
One way would be just putting the pickled object in constant.py:
my_object = pickle.loads(pickled_object) # don't use the name `object` it is a built-in
Note the s in loads.
pickled_object needs to be placed inside constant.py before the line shown above. It has to be a byte string.
You can create it either directly from the object:
pickled_object = pickle.dumps(obj)
or take it form the pickled file and past it in.
Example
Pickle your object:
>>> import pickle
>>> obj = [1, 2, 3]
>>> pickle.dumps(obj)
b'\x80\x03]q\x00(K\x01K\x02K\x03e.'
Now, in constant.py:
pickled_object = b'\x80\x03]q\x00(K\x01K\x02K\x03e.'
my_object = pickle.loads(pickled_object)
As a result, my_object is [1, 2, 3] and can be accessed via constant.my_object
I would like to provide default behaviour for a class as illustrated below.
import numpy as np
class Test:
def __init__(self, my_method=None):
self.my_method = my_method or np.min
Test().my_method([1, 2, 3]) # >>> 1
The code works as expected. To keep all the default values together for easier code maintenance I wanted to change the code to
import numpy as np
class Test:
default_method = np.min
def __init__(self, my_method=None):
self.my_method = my_method or Test.default_method
Test().my_method([1, 2, 3]) # >>> TypeError
but the call to my_method fails with the error message unbound method amin() must be called with Test instance as first argument (got list instance instead). Oddly, the code works as expected if I use the builtin min rather than np.min, i.e. the following works as expected.
import numpy as np
class Test:
default_method = min # no np.
def __init__(self, my_method=None):
self.my_method = my_method or Test.default_method
Test().my_method([1, 2, 3]) # >>> 1
What am I missing?
Any function stored as an attribute on a class object is treated as a method by Python. On Python 2, that means it requires the first argument to be an instance of the class (which will be passed automatically if the attribute is requested via an instance). On Python 3, unbound methods no longer check their arguments in that way (so your code would work as written).
To work around the issue on Python 2, try wrapping the default_method value with staticmethod:
class Test(object):
default_method = staticmethod(np.min)
#...
This might not be a bad idea even on Python 3, since you'll also be able to use self.default_method rather than explicitly naming the class.
As for why the code worked with min but not np.min, that's because they are implemented differently. You can see that from their types:
>>> type(min)
<class 'builtin_function_or_method'>
>>> type(np.min)
<class 'function'>
Regular functions (like np.min) act as descriptors when they're attributes of a class (thus getting the "binding" behavior that was causing your issue). Builtin functions like min don't support the descriptor protocol, so the issue doesn't come up.
In my Django v1.6.5 project running on Python v2.7.x, I have a Model that returns its configuration as a string. I need the returned string to be a gettext_lazy object, so I can evaluate it in any language required later.
from __future__ import unicode_literals
from django.utils.translation import ugettext_lazy as _, string_concat
...
class MyModel(models.Model):
key = models.CharField(...)
value = models.CharField(...)
#property
def config_str(self):
return _('some configuration')
This seems to work fine in these scenarios:
Static string: (see above) - works!
String concatenation: return string_concat(self.key, _(' equals '), self.value) - works!
What is not working, is using gettext_lazy with placeholders, a la:
return _('“%(key)s” equals “%(value)s”' % {key: self.key, value: self.value})
or using the .format() mechanism:
return _('“{key}” equals “{value}”').format(key=self.key, value=self.value)
When I do this, my .po file does contain:
#, python-format
msgid "“%(key)s” equals “%(value)s”" or
msgid "«{key}» equals «{value}»"
but even when I populate this Eg.:
msgstr "«%(key)s» est égal à «%(value)s»" or
msgstr "«{key}» est égal à «{value}»"
and I run compilemessages, the translation seems to be ignored. When I translate the promise returned by the model instance, I always get an EN string with the placeholders filled E.g., '“foo” equals “bar”'. Note, I get an EN string even when the first calling context is FR (for example). This tells me that the translations just aren't even occurring. It is my theory that by the time I eval the lazy object, gettext is looking for the literal string "“foo” equals “bar”" (for example) in the translation catalog rather than something with placeholders and named values.
With this in mind, I've also tried wrapping the whole format() in the lazy object like this:
return _('“{key}” equals “{value}”'.format(key=self.key, value=self.value))
But it seems to have made zero difference. =/
I can get by with string_concat() for now, but sometimes, the placeholders will need to be moved around in some translations, so I'd like to figure this out.
I'm beginning to think that one simply cannot use placeholders with gettext_lazy.
NOTE: I have reviewed django: Translation with variables inside, but a) that has no accepted answer and b) he's using gettext, not gettext_lazy.
OK, the solution here is to provide an extra layer of laziness (Thanks, Django core dev: Florian Apolloner AKA “apollo13”).
Here's my modified function that WORKS:
from django.utils import six
from django.utils.functional import lazy
class MyModel(models.Model):
key = models.CharField(...)
value = models.CharField(...)
#property
def configuration_string(self):
def wrapper():
return _('“{key}” equals “{value}”').format(
key=self.key,
value=self.value
)
return lazy(
wrapper,
six.text_type
)
The only thing is, where I use this, I must remember to eval the wrapper function as follows:
from django.utils.encoding import force_text
config = my_model_instance.configuration_string
# NOTE: Evaluate the lazy function wrapper inside the force_text()
config_str = force_text(config())
Now, in my case, I need to support cases where 3rd party devs write the function configuration_string returning either the lazy function wrapper, a lazily evaluated translation string or just a regular string, so I use:
import types
from django.utils.encoding import force_text
from django.functional import Promise
config = my_model_instance.configuration_string
if isinstance(config, types.FunctionType):
config_str = force_text(config())
elif isinstance(config, Promise):
config_str = force_text(config)
else:
config_str = config
Thanks again to Apollo13 for guidance!
I had a very similar problem and found that using gettext_noop instead of gettext_lazy worked for me available since Django 1.11.
I have a class that transforms some values via a user-specified function. The reference to the function is passed in the constructor and saved as an attribute. I want to be able to pickle or make copies of the class. In the __getstate__() method, I convert the dictionary entry to a string to make it safe for pickling or copying. However, in the __setstate__() method I'd like to convert back from string to function reference, so the new class can transform values.
class transformer(object):
def __init__(self, values=[1], transform_fn=np.sum):
self.values = deepcopy(values)
self.transform_fn = transform_fn
def transform(self):
return self.transform_fn(self.values)
def __getstate__(self):
obj_dict = self.__dict__.copy()
# convert function reference to string
obj_dict['transform_fn'] = str(self.transform_fn)
return obj_dict
def __setstate__(self, obj_dict):
self.__dict__.update(obj_dict)
# how to convert back from string to function reference?
The function reference that is passed can be any function, so solutions involving a dictionary with a fixed set of function references is not practical/flexible enough. I would use it like the following.
from copy import deepcopy
import numpy as np
my_transformer = transformer(values=[0,1], transform_fn=np.exp)
my_transformer.transform()
This outputs: array([ 1. , 2.71828183])
new_transformer = deepcopy(my_transformer)
new_transformer.transform()
This gives me: TypeError: 'str' object is not callable, as expected.
You could use dir to access names in a given scope, and then getattr to retrieve them.
For example, if you know the function is in numpy:
import numpy
attrs = [x for x in dir(numpy) if '__' not in x] # I like to ignore private vars
if obj_dict['transform_fn'] in attrs:
fn = getattr(numpy, obj_dict['transform_fn'])
else:
print 'uhoh'
This could be extended to look in other modules / scopes.
If you want to search in the current scope, you can do the following (extended from here):
import sys
this = sys.modules[__name__]
attrs = dir(this)
if obj_dict['transform_fn'] in attrs:
fn = getattr(this, obj_dict['transform_fn'])
else:
print 'Damn, well that sucks.'
To search submodules / imported modules you could iterate over attrs based on type (potentially recursively, though note that this is an attr of this).
I think if you are asking the same question, I came here for.
The answer is simply use eval() to evaluate the name.
>>> ref = eval('name')
This is going to return what 'name' references in the scope where the eval is
executed, then you can determine if that references is a function.