Python. Hint for variable - python-2.7

I'm using PyCharm as editor. For example, I have next function:
def get_instance():
# method without doc sctring in some module
# returns instance of MyClass
return some_var
And I have second file which calls get_instance():
var = get_instance()
How I can to define type of data for value? I want to use autocomplete for variable.
For example in php it will be like this:
/** #var MyClass $var */
$var = getInstance();
After this I will be see all methods and properties of $var. How to declare docstring for python variable? I know about :type and :rtype, but if I need declare type for specific variable? Can I declare a few types? What I should do?
Note: I can't do any changes with get_instance().

I don't think you can do this via docstring, but there is a mechanism called annotation. You can append whatever expression you want to parameters following a colon : and to the function declaration itself using an arrow ->. For example:
def getInstance(param: str) -> MyClass
return some_var
I have added an input parameter for purposes of illustration here. In this case str and MyClass are objects that represent the expected input and return types, respectively. They are never used in your code directly, just stashed away in an func_annotations attribute for end users that care about that sort of things (like editors).

You can do it in the same way as description properties of class. Here example:
from example_module import MyClass
var = get_instance()
"""
:type var: MyClass
"""
Also you can use description without import like this:
var = get_instance()
"""
:type var: example_module.MyClass
"""
The second way is using PEP 484:
test = my() # type: dict
Examples of autocomplete in Pycharm:

Related

Surprising behaviour using TemplateEngine with variable "URL" interpreted as class

Given the following Groovy code:
def engine = new SimpleTemplateEngine()
def propMap = [ URL: "http://stackoverflow.com",URL2: "http://stackoverflow.com"]
def result = engine.createTemplate('''
${URL}
${URL2}
''').make(propMap) as String
println(java.net.URL)
the output is
class java.net.URL
http://stackoverflow.com
Somehow the URL ends up being interpreted as class java.net.URL (which Groovy seems to be auto-importing), but why? And can a variable named URL used in this context?
Groovy is making several default imports, which also includes java.net. Import java.net.URL apparently shadows your local variable.
You could use this to explicitly tell Groovy to use your variable instead of java.net.URL.
${this.URL}
${URL2}
I also tried to use alias for import like this:
import java.net.URL as JavaURL
but it didn't really help, because both implicit (URL) and explicit (JavaURL) imports were used.

self parameter in Django [duplicate]

Consider this example:
class MyClass:
def func(self, name):
self.name = name
I know that self refers to the specific instance of MyClass. But why must func explicitly include self as a parameter? Why do we need to use self in the method's code? Some other languages make this implicit, or use special syntax instead.
For a language-agnostic consideration of the design decision, see What is the advantage of having this/self pointer mandatory explicit?.
To close debugging questions where OP omitted a self parameter for a method and got a TypeError, use TypeError: method() takes 1 positional argument but 2 were given instead. If OP omitted self. in the body of the method and got a NameError, consider How can I call a function within a class?.
The reason you need to use self. is because Python does not use special syntax to refer to instance attributes. Python decided to do methods in a way that makes the instance to which the method belongs be passed automatically, but not received automatically: the first parameter of methods is the instance the method is called on. That makes methods entirely the same as functions, and leaves the actual name to use up to you (although self is the convention, and people will generally frown at you when you use something else.) self is not special to the code, it's just another object.
Python could have done something else to distinguish normal names from attributes -- special syntax like Ruby has, or requiring declarations like C++ and Java do, or perhaps something yet more different -- but it didn't. Python's all for making things explicit, making it obvious what's what, and although it doesn't do it entirely everywhere, it does do it for instance attributes. That's why assigning to an instance attribute needs to know what instance to assign to, and that's why it needs self..
Let's say you have a class ClassA which contains a method methodA defined as:
def methodA(self, arg1, arg2):
# do something
and objectA is an instance of this class.
Now when objectA.methodA(arg1, arg2) is called, python internally converts it for you as:
ClassA.methodA(objectA, arg1, arg2)
The self variable refers to the object itself.
Let’s take a simple vector class:
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
We want to have a method which calculates the length. What would it look like if we wanted to define it inside the class?
def length(self):
return math.sqrt(self.x ** 2 + self.y ** 2)
What should it look like when we were to define it as a global method/function?
def length_global(vector):
return math.sqrt(vector.x ** 2 + vector.y ** 2)
So the whole structure stays the same. How can me make use of this? If we assume for a moment that we hadn’t written a length method for our Vector class, we could do this:
Vector.length_new = length_global
v = Vector(3, 4)
print(v.length_new()) # 5.0
This works because the first parameter of length_global, can be re-used as the self parameter in length_new. This would not be possible without an explicit self.
Another way of understanding the need for the explicit self is to see where Python adds some syntactical sugar. When you keep in mind, that basically, a call like
v_instance.length()
is internally transformed to
Vector.length(v_instance)
it is easy to see where the self fits in. You don't actually write instance methods in Python; what you write is class methods which must take an instance as a first parameter. And therefore, you’ll have to place the instance parameter somewhere explicitly.
When objects are instantiated, the object itself is passed into the self parameter.
Because of this, the object’s data is bound to the object. Below is an example of how you might like to visualize what each object’s data might look. Notice how ‘self’ is replaced with the objects name. I'm not saying this example diagram below is wholly accurate but it hopefully with serve a purpose in visualizing the use of self.
The Object is passed into the self parameter so that the object can keep hold of its own data.
Although this may not be wholly accurate, think of the process of instantiating an object like this: When an object is made it uses the class as a template for its own data and methods. Without passing it's own name into the self parameter, the attributes and methods in the class would remain as a general template and would not be referenced to (belong to) the object. So by passing the object's name into the self parameter it means that if 100 objects are instantiated from the one class, they can all keep track of their own data and methods.
See the illustration below:
I like this example:
class A:
foo = []
a, b = A(), A()
a.foo.append(5)
b.foo
ans: [5]
class A:
def __init__(self):
self.foo = []
a, b = A(), A()
a.foo.append(5)
b.foo
ans: []
I will demonstrate with code that does not use classes:
def state_init(state):
state['field'] = 'init'
def state_add(state, x):
state['field'] += x
def state_mult(state, x):
state['field'] *= x
def state_getField(state):
return state['field']
myself = {}
state_init(myself)
state_add(myself, 'added')
state_mult(myself, 2)
print( state_getField(myself) )
#--> 'initaddedinitadded'
Classes are just a way to avoid passing in this "state" thing all the time (and other nice things like initializing, class composition, the rarely-needed metaclasses, and supporting custom methods to override operators).
Now let's demonstrate the above code using the built-in python class machinery, to show how it's basically the same thing.
class State(object):
def __init__(self):
self.field = 'init'
def add(self, x):
self.field += x
def mult(self, x):
self.field *= x
s = State()
s.add('added') # self is implicitly passed in
s.mult(2) # self is implicitly passed in
print( s.field )
[migrated my answer from duplicate closed question]
The following excerpts are from the Python documentation about self:
As in Modula-3, there are no shorthands [in Python] for referencing the object’s members from its methods: the method function is declared with an explicit first argument representing the object, which is provided implicitly by the call.
Often, the first argument of a method is called self. This is nothing more than a convention: the name self has absolutely no special meaning to Python. Note, however, that by not following the convention your code may be less readable to other Python programmers, and it is also conceivable that a class browser program might be written that relies upon such a convention.
For more information, see the Python documentation tutorial on classes.
As well as all the other reasons already stated, it allows for easier access to overridden methods; you can call Class.some_method(inst).
An example of where it’s useful:
class C1(object):
def __init__(self):
print "C1 init"
class C2(C1):
def __init__(self): #overrides C1.__init__
print "C2 init"
C1.__init__(self) #but we still want C1 to init the class too
>>> C2()
"C2 init"
"C1 init"
Its use is similar to the use of this keyword in Java, i.e. to give a reference to the current object.
Python is not a language built for Object Oriented Programming unlike Java or C++.
When calling a static method in Python, one simply writes a method with regular arguments inside it.
class Animal():
def staticMethod():
print "This is a static method"
However, an object method, which requires you to make a variable, which is an Animal, in this case, needs the self argument
class Animal():
def objectMethod(self):
print "This is an object method which needs an instance of a class"
The self method is also used to refer to a variable field within the class.
class Animal():
#animalName made in constructor
def Animal(self):
self.animalName = "";
def getAnimalName(self):
return self.animalName
In this case, self is referring to the animalName variable of the entire class. REMEMBER: If you have a variable within a method, self will not work. That variable is simply existent only while that method is running. For defining fields (the variables of the entire class), you have to define them OUTSIDE the class methods.
If you don't understand a single word of what I am saying, then Google "Object Oriented Programming." Once you understand this, you won't even need to ask that question :).
First of all, self is a conventional name, you could put anything else (being coherent) in its stead.
It refers to the object itself, so when you are using it, you are declaring that .name and .age are properties of the Student objects (note, not of the Student class) you are going to create.
class Student:
#called each time you create a new Student instance
def __init__(self,name,age): #special method to initialize
self.name=name
self.age=age
def __str__(self): #special method called for example when you use print
return "Student %s is %s years old" %(self.name,self.age)
def call(self, msg): #silly example for custom method
return ("Hey, %s! "+msg) %self.name
#initializing two instances of the student class
bob=Student("Bob",20)
alice=Student("Alice",19)
#using them
print bob.name
print bob.age
print alice #this one only works if you define the __str__ method
print alice.call("Come here!") #notice you don't put a value for self
#you can modify attributes, like when alice ages
alice.age=20
print alice
Code is here
self is an object reference to the object itself, therefore, they are same.
Python methods are not called in the context of the object itself.
self in Python may be used to deal with custom object models or something.
It’s there to follow the Python zen “explicit is better than implicit”. It’s indeed a reference to your class object. In Java and PHP, for example, it's called this.
If user_type_name is a field on your model you access it by self.user_type_name.
I'm surprised nobody has brought up Lua. Lua also uses the 'self' variable however it can be omitted but still used. C++ does the same with 'this'. I don't see any reason to have to declare 'self' in each function but you should still be able to use it just like you can with lua and C++. For a language that prides itself on being brief it's odd that it requires you to declare the self variable.
The use of the argument, conventionally called self isn't as hard to understand, as is why is it necessary? Or as to why explicitly mention it? That, I suppose, is a bigger question for most users who look up this question, or if it is not, they will certainly have the same question as they move forward learning python. I recommend them to read these couple of blogs:
1: Use of self explained
Note that it is not a keyword.
The first argument of every class method, including init, is always a reference to the current instance of the class. By convention, this argument is always named self. In the init method, self refers to the newly created object; in other class methods, it refers to the instance whose method was called. For example the below code is the same as the above code.
2: Why do we have it this way and why can we not eliminate it as an argument, like Java, and have a keyword instead
Another thing I would like to add is, an optional self argument allows me to declare static methods inside a class, by not writing self.
Code examples:
class MyClass():
def staticMethod():
print "This is a static method"
def objectMethod(self):
print "This is an object method which needs an instance of a class, and that is what self refers to"
PS:This works only in Python 3.x.
In previous versions, you have to explicitly add #staticmethod decorator, otherwise self argument is obligatory.
Take a look at the following example, which clearly explains the purpose of self
class Restaurant(object):
bankrupt = False
def open_branch(self):
if not self.bankrupt:
print("branch opened")
#create instance1
>>> x = Restaurant()
>>> x.bankrupt
False
#create instance2
>>> y = Restaurant()
>>> y.bankrupt = True
>>> y.bankrupt
True
>>> x.bankrupt
False
self is used/needed to distinguish between instances.
Source: self variable in python explained - Pythontips
Is because by the way python is designed the alternatives would hardly work. Python is designed to allow methods or functions to be defined in a context where both implicit this (a-la Java/C++) or explicit # (a-la ruby) wouldn't work. Let's have an example with the explicit approach with python conventions:
def fubar(x):
self.x = x
class C:
frob = fubar
Now the fubar function wouldn't work since it would assume that self is a global variable (and in frob as well). The alternative would be to execute method's with a replaced global scope (where self is the object).
The implicit approach would be
def fubar(x)
myX = x
class C:
frob = fubar
This would mean that myX would be interpreted as a local variable in fubar (and in frob as well). The alternative here would be to execute methods with a replaced local scope which is retained between calls, but that would remove the posibility of method local variables.
However the current situation works out well:
def fubar(self, x)
self.x = x
class C:
frob = fubar
here when called as a method frob will receive the object on which it's called via the self parameter, and fubar can still be called with an object as parameter and work the same (it is the same as C.frob I think).
In the __init__ method, self refers to the newly created object; in other class methods, it refers to the instance whose method was called.
self, as a name, is just a convention, call it as you want ! but when using it, for example to delete the object, you have to use the same name: __del__(var), where var was used in the __init__(var,[...])
You should take a look at cls too, to have the bigger picture. This post could be helpful.
self is acting as like current object name or instance of class .
# Self explanation.
class classname(object):
def __init__(self,name):
self.name=name
# Self is acting as a replacement of object name.
#self.name=object1.name
def display(self):
print("Name of the person is :",self.name)
print("object name:",object1.name)
object1=classname("Bucky")
object2=classname("ford")
object1.display()
object2.display()
###### Output
Name of the person is : Bucky
object name: Bucky
Name of the person is : ford
object name: Bucky
"self" keyword holds the reference of class and it is upto you if you want to use it or not but if you notice, whenever you create a new method in python, python automatically write self keyword for you. If you do some R&D, you will notice that if you create say two methods in a class and try to call one inside another, it does not recognize method unless you add self (reference of class).
class testA:
def __init__(self):
print('ads')
def m1(self):
print('method 1')
self.m2()
def m2(self):
print('method 2')
Below code throws unresolvable reference error.
class testA:
def __init__(self):
print('ads')
def m1(self):
print('method 1')
m2() #throws unresolvable reference error as class does not know if m2 exist in class scope
def m2(self):
print('method 2')
Now let see below example
class testA:
def __init__(self):
print('ads')
def m1(self):
print('method 1')
def m2():
print('method 2')
Now when you create object of class testA, you can call method m1() using class object like this as method m1() has included self keyword
obj = testA()
obj.m1()
But if you want to call method m2(), because is has no self reference so you can call m2() directly using class name like below
testA.m2()
But keep in practice to live with self keyword as there are other benefits too of it like creating global variable inside and so on.
self is inevitable.
There was just a question should self be implicit or explicit.
Guido van Rossum resolved this question saying self has to stay.
So where the self live?
If we would just stick to functional programming we would not need self.
Once we enter the Python OOP we find self there.
Here is the typical use case class C with the method m1
class C:
def m1(self, arg):
print(self, ' inside')
pass
ci =C()
print(ci, ' outside')
ci.m1(None)
print(hex(id(ci))) # hex memory address
This program will output:
<__main__.C object at 0x000002B9D79C6CC0> outside
<__main__.C object at 0x000002B9D79C6CC0> inside
0x2b9d79c6cc0
So self holds the memory address of the class instance.
The purpose of self would be to hold the reference for instance methods and for us to have explicit access to that reference.
Note there are three different types of class methods:
static methods (read: functions),
class methods,
instance methods (mentioned).
The word 'self' refers to instance of a class
class foo:
def __init__(self, num1, num2):
self.n1 = num1 #now in this it will make the perimeter num1 and num2 access across the whole class
self.n2 = num2
def add(self):
return self.n1 + self.n2 # if we had not written self then if would throw an error that n1 and n2 is not defined and we have to include self in the function's perimeter to access it's variables
it's an explicit reference to the class instance object.
from the docs,
the special thing about methods is that the instance object is passed as the first argument of the function. In our example, the call x.f() is exactly equivalent to MyClass.f(x). In general, calling a method with a list of n arguments is equivalent to calling the corresponding function with an argument list that is created by inserting the method’s instance object before the first argument.
preceding this the related snippet,
class MyClass:
"""A simple example class"""
i = 12345
def f(self):
return 'hello world'
x = MyClass()
I would say for Python at least, the self parameter can be thought of as a placeholder.
Take a look at this:
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
p1 = Person("John", 36)
print(p1.name)
print(p1.age)
Self in this case and a lot of others was used as a method to say store the name value. However, after that, we use the p1 to assign it to the class we're using. Then when we print it we use the same p1 keyword.
Hope this helps for Python!
my little 2 cents
In this class Person, we defined out init method with the self and interesting thing to notice here is the memory location of both the self and instance variable p is same <__main__.Person object at 0x106a78fd0>
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
def say_hi(self):
print("the self is at:", self)
print((f"hey there, my name is {self.name} and I am {self.age} years old"))
def say_bye(self):
print("the self is at:", self)
print(f"good to see you {self.name}")
p = Person("john", 78)
print("the p is at",p)
p.say_hi()
p.say_bye()
so as explained in above, both self and instance variable are same object.

Construct a callable object from a string representing its name in Python

Let's say I have a class like this
class MyClass(AnotherClass):
def __init__(self, arg1, arg2):
self.arg1 = arg1
self.arg2 = arg2
def f(self):
#dostuff
And I have the string "my class"
str = "my class"
I can do this:
class_name_from_str = str.title().replace(" ", "") # now class_name_from_str is "MyClass"
How could I make class_name_from_str callable? I want something like this:
callable_obj = some_process(class_name_from_str)
o = callable_obj(arg1, arg2)
o.f()
I'm using Python 2.7, by the way
UPDATE: As Matti Virkkunen suggested in a comment, a good solution is to create a dict to map strings with classes
my_calls_dict = {'my class' : MyClass, 'my other class' : MyOtherClass}
It kind of depends on where your class resides. If it's in the current module
globals()[class_name]
should get you the class. If it's in a module, replace globals() with the module (possibly imported via __import__() - if you want the module name to be dynamic as well).
This would be a good time to think about whether this is a good idea though. Wouldn't it be clearer to, for instance, make a dict of names and corresponding classes? If your .title().strip() thing is part of your actual code, I honestly can't think of what it might be trying to do.

Extending SWIG builtin classes

The -builtin option of SWIG has the advantage of being faster, and of being exempt of a bug with multiple inheritance.
The setback is I can't set any attribute on the generated classes or any subclass :
-I can extend a python builtin type like list, without hassle, by subclassing it :
class Thing(list):
pass
Thing.myattr = 'anything' # No problem
-However using the same approach on a SWIG builtin type, the following happens :
class Thing(SWIGBuiltinClass):
pass
Thing.myattr = 'anything'
AttributeError: type object 'Thing' has no attribute 'myattr'
How could I work around this problem ?
I found a solution quite by accident. I was experimenting with metaclasses, thinking I could manage to override the setattr and getattr functions of the builtin type in the subclass.
Doing this I discovered the builtins already have a metaclass (SwigPyObjectType), so my metaclass had to inherit it.
And that's it. This alone solved the problem. I would be glad if someone could explain why :
SwigPyObjectType = type(SWIGBuiltinClass)
class Meta(SwigPyObjectType):
pass
class Thing(SWIGBuiltinClass):
__metaclass__ = Meta
Thing.myattr = 'anything' # Works fine this time
The problem comes from how swig implemented the classes in "-builtin" to be just like builtin classes (hence the name).
builtin classes are not extensible - try to add or modify a member of "str" and python won't let you modify the attribute dictionary.
I do have a solution I've been using for several years.
I'm not sure I can recommend it because:
It's arguably evil - the moral equivalent of casting away const-ness in C/C++
It's unsupported and could break in future python releases
I haven't tried it with python3
I would be a bit uncomfortable using "black-magic" like this in production code - it could break and is certainly obscure - but at least one giant corporation IS using this in production code
But.. I love how well it works to solve some obscure features we wanted for debugging.
The original idea is not mine, I got it from:
https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader
The basic idea is to access the const dictionary in the swig-created type object as a non-const dictionary and add/override any desired methods.
FYI, the technique of runtime modification of classes is called monkeypatching, see https://en.wikipedia.org/wiki/Monkey_patch
First - here's "monkeypatch.py":
''' monkeypatch.py:
I got this from https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader,
his comment: "found this from Armin R. on Twitter, what a beautiful gem ;)"
I made a few changes for coding style preferences
- Rudy Albachten April 30 2015
'''
import ctypes
from types import DictProxyType, MethodType
# figure out the size of _Py_ssize_t
_Py_ssize_t = ctypes.c_int64 if hasattr(ctypes.pythonapi, 'Py_InitModule4_64') else ctypes.c_int
# python without tracing
class _PyObject(ctypes.Structure):
pass
_PyObject._fields_ = [
('ob_refcnt', _Py_ssize_t),
('ob_type', ctypes.POINTER(_PyObject))
]
# fixup for python with tracing
if object.__basicsize__ != ctypes.sizeof(_PyObject):
class _PyObject(ctypes.Structure):
pass
_PyObject._fields_ = [
('_ob_next', ctypes.POINTER(_PyObject)),
('_ob_prev', ctypes.POINTER(_PyObject)),
('ob_refcnt', _Py_ssize_t),
('ob_type', ctypes.POINTER(_PyObject))
]
class _DictProxy(_PyObject):
_fields_ = [('dict', ctypes.POINTER(_PyObject))]
def reveal_dict(proxy):
if not isinstance(proxy, DictProxyType):
raise TypeError('dictproxy expected')
dp = _DictProxy.from_address(id(proxy))
ns = {}
ctypes.pythonapi.PyDict_SetItem(ctypes.py_object(ns), ctypes.py_object(None), dp.dict)
return ns[None]
def get_class_dict(cls):
d = getattr(cls, '__dict__', None)
if d is None:
raise TypeError('given class does not have a dictionary')
if isinstance(d, DictProxyType):
return reveal_dict(d)
return d
def test():
import random
d = get_class_dict(str)
d['foo'] = lambda x: ''.join(random.choice((c.upper, c.lower))() for c in x)
print "and this is monkey patching str".foo()
if __name__ == '__main__':
test()
Here's a contrived example using monkeypatch:
I have a class "myclass" in module "mystuff" wrapped with swig -python -builtin
I want to add an extra runtime method "namelen" that returns the length of the name returned by myclass.getName()
import mystuff
import monkeypatch
# add a "namelen" method to all "myclass" objects
def namelen(self):
return len(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen
x = mystuff.myclass("xxxxxxxx")
print "namelen:", x.namelen()
Note that this can also be used to extend or override methods on builtin python classes, as is demonstrated in the test in monkeypatch.py: it adds a method "foo" to the builtin str class that returns a copy of the original string with random upper/lower case letters
I would probably replace:
# add a "namelen" method to all "myclass" objects
def namelen(self):
return len(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen
with
# add a "namelen" method to all "myclass" objects
monkeypatch.get_class_dict(mystuff.myclass)['namelen'] = lambda self: return len(self.getName())
to avoid extra global variables

Python API C++ : "Static variable" for a Type Object

I have a small question about static variable and TypeObjects.
I use the API C to wrap a c++ object (let's call it Acpp) that has a static variable called x.
Let's call my TypeObject A_Object :
typedef struct {
PyObject_HEAD
Acpp* a;
} A_Object;
The TypeObject is attached to my python module "myMod" as "A". I have defined getter and setters (tp_getset) so that I can access and modify the static variable of Acpp from python :
>>> import myMod
>>> myA1 = myMod.A(some args...)
>>> myA1.x = 34 # using the setter to set the static variable of Acpp
>>> myA2 = myMod.A(some other args...)
>>> print myA2.x
34
>>> # Ok it works !
This solution works but it's not really "clean". I would like to access the static variable in python by using the TypeObject and not the instances :
>>> import myMod
>>> myMod.A.x = 34 # what I wish...
Does anybody have an idea to help me ?
Thanks in advance.
Essentially, what you're trying to do is define a "static property". That is, you want a function to be called when you get/set an attribute of the class.
With that in mind, you might find this thread interesting. It only talks about Python-level solutions to this problem, not C extension types, but it covers the basic principles.
To implement the solution proposed in that thread for a C extension type, I think you'd have to initialize tp_dict and add to it an entry for "x" whose value is an object that implements __get__ appropriately.
You could add a dummy 'x' field in the A_Object and create a pair of set/get methods. When you access the dummy 'x' field, the method would redirect the call to the static 'x' field.