I'm having trouble with a type conversion problem in Clojure, using Hugsql. I'm new to Clojure and newer to SQL, and I'd appreciate any help I can get.
We recently migrated our PostgreSQL db so one column is a json array instead of a string -- the migration looks like:
ALTER TABLE customers ALTER id TYPE JSON USING json_build_array(id);
But now when I make query for that data, the type is a PGobject and not a collection like I was hoping. While I'm looking for:
["id-123"]
the actual returned value is
#object[org.postgresql.util.PGobject 0x7ff0434e "[\"id-123\"]"]
I spent some time looking up how to do the type conversion, but had no luck. How can I convert the type of this response to a collection or sequence in Clojure?
Thanks for your help!
It turns out that this is much easier than I was making it out to be.
You can just use (.getValue my-pgobject) and it will be converted to a string like:
"[\"id-123\"]", which can then be parsed with something like cheshire.core/parse-string
Related
So right now, my model is declared with serializers.DateTimeField(input_formats=["iso-8601"]) and the value I get when printing is:
2020-10-03 11:15:00+00:00
Ideally that'd be
2020-10-03T11:15:00+00:00
or even more ideally, if it doesn't require a ton of work,
2020-10:03T11:15:00Z
How do I accomplish this? Or when its sent over the network/to front-end, does this happen automatically?
I believe that you are getting the default string evaluation of a datetime function here. What you want to do when printing is override the default string function. In your serializer, you want to pass a format argument (look here), and construct the string using a combination of the python date formats (date formats). This formatting makes python date strings fully customizable.
For you, your string should look something like... %Y-%m-%dT%I:%M:%SZ
I am writing simple SELECT queries which involve parsing out date from a string.
The dates are typed in by users manually in a web application and are recorded as string in database.
I am having CASE statement to handle various date formats and use correct format specifier accordingly in TO_DATE function.
However, sometimes, users enter something that's not a valid date(e.g. 13-31-2013) by mistake and then the entire query fails. Is there any way to handle such rougue records and replace them with some default date in query so that the entire query does not fail due to single invalid date record?
I have already tried regular expressions but they are not quite reliable when it comes to handling leap years and 30/31 days in months AFAIK.
I don't have privileges to store procedures or anything like that. Its just plain simple SELECT query executed from my application.
This is a client task..
The DB will give you an error for an invalid date (the DB does not have a "TO_DATE_AND_FIX_IF_NOT_CORRECT" function).
If you've got this error- it means you already tried to cast something to an invalid date.
I recommend doing the migration to date on your application server, and in the case of exception from your code - send a default date to the DB.
Also, that way you send to the DB an object of type DbDate and not a string.
That way you achieve two goals:
1. The dates will always be what you want them to be (from the client).
2. You close the door for SQL Injection attacks.
It sounds like in your case you should write the function I mentioned...
it should look something like that:
Create or replace function TO_DATE_SPECIAL(in_date in varchar2) return DATE is
ret_val date;
begin
ret_val := to_date(in_date,'MM-DD-YYYY');
return ret_val;
exception
when others then
return to_date('01-01-2000','MM-DD-YYYY');
end;
within the query - instead of using "to_date" use the new function.
that way instead of failing - it will give you back a default date.
-> There is not IsDate function .. so you'll have to create an object for it...
I hope you've got the idea and how to use it, if not - let me know.
I ended up using crazy regex that checks leap years, 30/31 days as well.
Here it is:
((^(0?[13578]|1[02])[\/.-]?(0?[1-9]|[12][0-9]|3[01])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^(0?[469]|11)[\/.-]?(0?[1-9]|[12][0-9]|30)[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?(0?[1-9]|1[0-9]|2[0-8])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?29[\/.-]?(((18|19|20){0,1}(04|08|[2468][048]|[13579][26]))|2000|00)$))
It is modified version of the answer by McKay here.
Not the most efficient but it works. I'll wait to see if I get a better alternative.
We are developing the module in tryton based on GNU Health.We got the following error :
ProgrammingError: operator does not exist character varying = bigint
Hint: No opreator matches the given name and argument type(s). You might need to add explicit type casts
As best as I can vaguely guess from the limited information provided, in this query:
"SELECT name,age,dob,address FROM TABLENAME WHERE pmrn=%s" % (self.pmrn)
you appear to be doing a string substitution of a value into a query.
First, this is dangerously wrong, and you should never ever do it without an extremely good reason. Always use parameterized queries. psycopg2 supports these, so there's no excuse not to. So do all the other Python interfaces for PostgreSQL, but I'm assuming you're using psycopg2 because basically everyone does, so go read the usage documentation to see how to pass query parameters.
Second, as a result of failing to use parameterized queries, you aren't getting any help from the database driver with datatype handling. You mentioned that pmrn is of type char - for which I assume you really meant varchar; if it's actually char then the database designers need to be taken aside for a firm talking-to. Anyway, if you substitute an unquoted number in there your query is going to look like:
pmrn = 201401270001
and if pmrn is varchar that'll be an error, because you can't compare a text type to a number directly. You must pass the value as text. The simplistic way is to put quotes around it:
pmrn = '201401270001'
but what you should be doing instead is letting psycopg2 take care of all this for you by using parameterized queries. E.g.
curs.execute("SELECT name,age,dob,address FROM TABLENAME WHERE pmrn=%s", (self.pmrn,))
i.e. pass the SQL query as a string, then a 1-tuple containing the query params. (You might have to convert self.pmrn to str if it's an int, too, eg str(self.pmrn)).
All,
I'm trying to basically do keyword notifications, so whenever an object with a name is created, anyone who wants to be notified of any of the words in this name will be.
e.g.
Records:
keyword: Hello
keyword: World
New Name: "Hello World"
Returns both records
I've created a query that correctly works for this in sqlite, and I know how to translate it across databases.
SELECT * FROM table t
WHERE "a constant string" LIKE "%" || t.field || "%";
I've determined that within django, one can use F() objects to compare one field to another field, like so:
Entry.objects.filter(n_comments__gt=F('n_pingbacks'))
Now anyone know how to replace the first field with a constant string? Like so:
Entry.objects.filter("constant string"__icontains=F('n_pingbacks'))
Or am I going about this backwards?
It doesn't look particularly pretty but it can be done all with standard filters.
from django.db.models import ExpressionWrapper, CharField, Value
Entry.objects.annotate(
my_const=ExpressionWrapper(Value("a constant string"),
output_field=CharField())
).filter(my_const__contains=F('n_pingbacks'))
You can do this by providing a dict of arguments, e.g.:
Entry.objects.filter(**{("%s__icontains" % constant_string):F('n_pingbacks')})
Try using '.extra' to select your const as field, than using myconst__contains, like:
queryset.extra(select={'myconst': "'this superstring is myconst value'"}).filter(myconst__contains=F('myfield'))
Do not forget to put constant value in apostrophes inside double qoutation marks.
But, will somebody help me put it into Q object? =)
UPD:
Suddenly, it fails because of the following issue:
https://code.djangoproject.com/ticket/13363
Maybe, they will fix it.
UPD: You can filter by fields that added with '.annotate', but I don't know, how to put constant here instead of aggregation. Maybe, with creation of custom aggreation function, like here:
http://coder.cl/2011/09/custom-aggregates-on-django/
UPD: I made custom aggregator, this logic seems to be correct, because the query I got from queryset is quite similar to I wanted it to be, but, unfortunately, there is another issue: 16731 (sorry not providing full url, not enough rep, see another ticket above).
UPD(last): I have managed to do this using monkeypatching of the following:
django.db.models.sql.Query.query_terms
django.db.models.fields.Field.get_prep_lookup
django.db.models.fields.Field.get_db_prep_lookup
django.db.models.sql.where.WhereNode.make_atom
Just defined custom lookup 'starts', which has reverse logic of 'startswith'
You should not try to fight with django ORM. It covers the most often things but your case is not the one. Just use extra to get what you need (or even raw).
I am using James Bennetts code (link text) to create a dynamic form. Everything is working ok but I have now come to the point where I need to save the data and have become a bit stuck. I know I can assess the data returned by the form and simply save this to the database as a string but what I'd really like to do is save what type of data it is e.g. date, integer, varchar along with the value so that when it comes to viewing the data I can do some processing on it depending on what type it is e.g. get dates greater than last week.
So my question is how do I access what database type the form element is based on what type of form element it is e.g. a django.forms.IntegerField has a database field type of int, django.forms.DateField would be a date field and django.forms.ChoiceField would be a varchar field?
Why do you want to know, what kind of database field you are using? Are you storing information from the form through raw sql? You should have some model, that you are storing information from the form and it will do all the work for you.
Maybe you could show some form code? Right now it's hard to determine, what exactly you are trying to do.
I can not understand the exact problem, so forgive me if I get things wrong.
If you are using models, then you don't need to know about database-level data types. They are defined by django according to your model fields.
However, since you are talking about dynamic forms (I've read the article), you are probably not working with models, at least not directly. In that case, it should not matter as well, because you are using form validation so, for example, you can be absolutely sure that an integer comes out of a forms.IntegerField field, unicode comes out of forms.CharField and so on.
If you are writing your database-interaction routies by hand (raw sql), then you have to map python-types to db-types yourself, for example <type 'int'> goes to a column of type integer (or something), <type 'datetime.datetime'> goes to a datetime type of column (or not, this example is arbitrary) and so on. When you are using models, django does this type of mapping for you in a database-engine-independent way.
Either way, you, yourself are defining the datatypes on the python side and you or django must also define the datatypes on the db side. The choice of those types is, at times, not an automatic 1:1 type of decision, but, rather, a design decision, based on what this data is used for in your application.
Sorry, if this makes little sense, but, I must admit, that I don't quite understand the problem behind your question.
If you're using James' code, then you don't get a Model out of the form per se, rather a list of form field elements. That means that you can't save the data as a Model instance.
I think you have two choices; bundle the whole form into a JSON object and save that into a LONGTEXT variable in your database, or save each form element into a row of the database on its own, saving it into a BLOB entry. In this case, you'll need to 'pickle' the object before saving it. If you pickle the object and save it into the database, when you retrieve it and unpickle it, you'll have all the python class information associated with the object.
Trying to make this clearer - if you have the bytes; 2009-11-28 21:34:36.516176, is this a str or a datetime object? You can't tell if it's stored in the database as a VARCHAR or LONGTEXT. Which is the core of your question - you do get object information if you save it as a pickled object though.
By extension, you could save your whole Form object into the database, either as a JSON object, or pickle the object and save that.
I'm struggling with something very similar at the moment, as I'm trying to put together a dynamic form system, and thinking of going the 'individual form field element, pickled' and then saved into the database. So I'll be watching how this question works out! :)