I have recently tried to use Great expectations in a data creation pipeline but I run into a problem with validator object.
I followed this tutorial to set things up: https://docs.greatexpectations.io/docs/deployment_patterns/how_to_use_great_expectations_in_databricks/
I created my validator instance with following:
validator = context.get_validator(
batch_request=batch_request_1,
expectation_suite_name=expectation_suite_name,
)
Then I tried to set some expectations manually or with profiler but got those errors:
validator.expect_column_values_to_not_be_null(column="col")
AttributeError: 'Validator' object has no attribute 'expect_column_values_to_not_be_null'
profiler = UserConfigurableProfiler(profile_dataset=validator)
AttributeError: 'Validator' object has no attribute 'expect_column_values_to_be_in_type_list'
It looks like my instance does not have those methods and indeed when I hit tab to check what options I have with validator there are many other expect_column_.... But none of those 2.
Can you please advise me how to fix my validator object?
Related
I am trying to understand how polymorphic resources work in django-rest-framework-json-api and I get the following error both on my own project and the example project provided by django-rest-framework-json-api. My models are created using django-polymorphic. The example project I am trying to run is https://github.com/django-json-api/django-rest-framework-json-api/tree/master/example.
Endpoint : localhost:8000/projects
{
"errors": [
{
"detail": "Incorrect relation type. Expected on of [artProject, researchProject], received None.",
"source": {
"pointer": "/data"
},
"status": "409"
}
]
}
Project page's form used to create a new instance doesn't have any field that I can specify the type of the Project. It only shows the topic field. So I am also wondering how I can create an Art project for instance.
In the documentation it says the type is determined by the resource_name in Meta data of either view, serializer or model. But type is none in the returned json data. It is because serializer.instance = None which is used to determine the type when force_type_resolution is set to True. It is set to True to force polymorphic serializers to resolve the type based on instance.
So I can't seem to fix this issue, maybe I am missing some configuration or there really is a bug in the code but I have no idea. Can someone please tell me how to solve this if you have come across this issue or know how to get the example app working with polymorphic resources?
I was initially trying to create instances in the browser which threw the error above and didn't let me choose child types. I tested this on Postman and specified the type and it worked. I had the impression that I could even create an instance of Project as well (Parent instance itself, not a child instance) but as error message clearly says, I can't.
I set up a simple Ember Twiddle to show you my error that is occurring when trying to update a model.
It's considerable that I'm using ember-cli-mirage for mocking the data.
According to the docs, I created a shorthand route that should handle the PUT request.
It does, but with the error: Your handler for the url /api/shops/1 threw an error: Cannot convert undefined or null to object
When using the JSONAPISerializer, everything is working with shorthands (mirage/config.js) and I'm able to update models, but in my case I have to use the RESTSerializer with serialized IDs in the responses.
The request payload when I'm sending the model's attrs are without Id at the end of the property name, f.e.:
// attrs object in PUT request
{
name: "Shop 1",
city: "1" // belongsTo relationship,
}
Now Mirage is trying to find those properties on the respective database model that has to be updated, but cannot find it, because in the database it's cityId and not just city...
I also found this issue report and it’s working, but I was hoping I could avoid something like this. As far as I can remember, in previous versions of ember-cli-mirage (v0.1.x) it was also not needed to override the normalize method in the serializer to be able to make use of the RestSerializer with serializedIds…
My question is:
Is there a way to stick to shorthand route handlers only, or do I really have to write a helper or other custom solution only because I have to use the RestSerializer?
That would be really sad, but at least I would know then.
Thanks for your support!
Short answer: it looks like you need the custom serializer for now until the bug fix for it is merged.
Long answer: that issue looks to be an issue that occurred in the 0.2 -> 0.3 upgrade for Mirage, likely because of underlying DB changes made in Mirage. It'll probably get fixed, but for now you'll need to work around it.
So I'm working on some unit tests and relational fixtures.
I'm creating a model dynamically like:
$model = CActiveRecord::model('Post');
$post = $model->findByPk(1);
But after that I cannot for some reason get $post->id. I traced the problem to CActiveRecord class:
public function __get($name)
{
if(isset($this->_attributes[$name]))
return $this->_attributes[$name];
...
Where $name = "id". It says that $this->_attributes[$name] does not exist! As a matter of fact _attributes is empty.
My Post class does not define id (or any other properties) as a public property and I don't want to do so either. I just let the AR map it to table columns for me.
What am I missing?
Edit 1
My fixtures are regular Yii fixtures - nothing really special about them.
What differs is the way I load them really. I extended the CDbFixtureManager to be able to specify the order in which they should be loaded by overloading load() method. Only thing of interest that actually fails is that in the fixtures that have foreign keys I use the following:
'comment1' => array('post_id' => $this->getRecord('Post', 'post1')->id);
That's where it fails. getRecord returns the actual Post record (since I know the Post fixture has already been successfully loaded and exists in DB), but on the ->id part I get an exception about that attribute not existing.
If I go into Post model and add public $id; to it, then everything works! But I'm not sure if it's good practice to go about declaring all properties public like that.
If you look at this page carefully:
http://www.yiiframework.com/doc/guide/1.1/en/test.unit
you'll see that they use an array form for retrieving fixtures:
$this->posts['sample1']['id']
There is an alias defined in their fixture array for each record and fixture items aren't loaded as models really ...
Does that help? If not, it would be helpful to see your fixture file :-)
I think I found the root cause of this issue for me. While my FixtureManager was using the testdb DBConnection, the models still used the regular one.
For whatever reason, my debugger was giving me misleading errors like the one described in my original post.
Once I was able to set the DBConnection of all Models in the unit test the puzzle snapped into place and everything is now working smoothly!
I am very new to unit testing and am probably doing something wrong, but when I simulate a post to update a model via the admin backend it seems like my save_model method in my AdminForm isn't being called. I am trying to test this method - what am I doing wrong?
My second, less relevant question is in general how can I make sure a method is being called when I use unit testing? Is there some way to list all the methods that were hit?
Below is the code my test is running. In my save_model method in my AdminForm for this model, I set this model's foobar attribute to the username of the currently signed in user. Below is my test:
self.client = Client()
self.client.login(username='username',password='password')
# self.dict is a dictionary of field names and values for mymodel to be updated
response = self.client.post('/admin/myapp/mymodel/%d/' % self.mymodel.id, self.dict)
self.assertEqual(response.status_code,200) # passes
self.assertEqual(self.mymodel.foobar,'username') # fails
self.client.logout()
It fails because it says that self.mymodel.foobar is an empty string. That was what it should have been before the update. No value for foobar is passed in the self.dict but my save_model method is designed to set it on its own when the update happens. It is also worth noting that my code works correctly and save_model seems to work fine, just my test is failing. Since I am a total noob at TDD, I'm sure the issue is with my test and not my code. Thoughts?
From the code it looks like the problem is that, after posting the form, you don't reload self.mymodel from the database. If you hold a reference to a model object stored in the database, and one or more of the fields on that object is changed in the database, then you will need to reload the object from the database to see the updated values. As detailed in this question, you can do this with something like:
self.mymodel = MyModelClass.objects.get(id=self.mymodel.id)
To answer your second question, probably the most useful way to see what is happening would be to use logging to output what is happening in your save_model method - this will not only help you debug the issue during testing, but also if you encounter any issues in this method when running your application. The django guide to logging gives an excellent introduction:
https://docs.djangoproject.com/en/dev/topics/logging/
I am new to Django and Python, building my first app using TDD... I wanted to copy an instance of Task, a model object. I used the following code, which works correctly during the tests:
import copy
class Task(models.Model):
...
def make_copy(self):
new_task = copy.copy(self)
new_task.save()
return new_task
But when running this code 'normally', in the server, I noticed it was not working: no new object was created. I found out I had to add new_task.id = None just before saving, and I understand the reason for this...
But if copy.copy does not know about Django and thus will not change the id itself, why is it the case that the object returned has id == None during the tests?
It sounds like your test case is not fully matching the usage in your "normal" use case.
The id field is set for objects that exist in the database. If you pass your make_copy() method an object with id set, it will appear to fail because it's not making a new database object, it's just saving an existing object (through the Python copy, with an existing id).
I'd guess your test case is passing Task objects to make_copy() with id of None (and therefore appearing to work), while in "normal" usage, the objects are coming in with id set. (You could test this hypothesis with a simple print or assert statement).
One simple solution might be to set id to None right after your copy operation. That way, a new database object is always created.
Finally, someone else with the same situation: http://www.nerdydork.com/copy-model-object-in-django.html