Get elements from LazyLoadCollection - doctrine-orm

I have found Doctrine\Common\Collections\Criteria to be a very useful concept, if they worked for me.
In a symfony controller, I am calling this code:
$criteria = Criteria::create()
->where(Criteria::expr()->gt('position', 0))
->orderBy(['riskPosition', Criteria::ASC]);
$positions= $this->getDoctrine()->getRepository(DataCategory::class)->matching($criteria);
dump($positions->count()); // dumps 1, correct!
dump($positions);
foreach($positions as $r)
dump($r); // -> Unrecognized field: 0
dump($positions) gives
LazyCriteriaCollection {#881 ▼
#entityPersister: JoinedSubclassPersister {#849 ▶}
#criteria: Criteria {#848 ▼
-expression: Comparison {#836 ▶}
-orderings: array:2 [▶]
-firstResult: null
-maxResults: null
}
-count: 1
#collection: null
#initialized: false
}
As soon as I access an element of the returned array, I get an error
ORMException::unrecognizedField(0)
in vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php (line 1193)
But as soon as I want to access the elements (e.g. loop and dump) I get some error like An exception has been thrown during the rendering of a template ("Unrecognized field: 0").
As far as I have studied the code, the problem is that the query results have not been fetched from the database. Only count() works. How can I trigger this fetch?
Does it matter that my entity has #ORM\InheritanceType("JOINED")?
This code (circumventing the use of Criteria) does give correct results, but I'd like to use Criteria:
$riskPositions = $this->getDoctrine()->getRepository(DataCategory::class)
->createQueryBuilder('p')
->where('p.position > 0')
->orderBy('p.position', 'ASC')
->getQuery()
->execute();

The issue is caused by line:
->orderBy(['riskPosition', Criteria::ASC]);
Doctrine\Common\Collections\Criteria `s orderBy accepts an array argument where
Keys are field and values are the order, being either ASC or DESC.
github link
Apparently, there is a mistake at doctrine s documentation.
So doctrine thinks that "0", which is the 1st key of the array argument, is the field to sort by, but cannot find it.
To solve, change the above line to:
->orderBy(['riskPosition' => Criteria::ASC]);

Related

Randomly set one-third of na's in a column to one value and the rest to another value

I'm trying to impute missing values in a dataframe df. I have a column A with 300 NaN's. I want to randomly set 2/3rd of it to value1 and the rest to value2.
Please help.
EDIT: I'm actually trying to this on dask, which does not support item assignment. This is what I have currently. Initially, I thought I'll try to convert all NA's to value1
da.where(df.A.isnull() == True, 'value1', df.A)
I got the following error:
ValueError: need more than 0 values to unpack
As the comment suggests, you can solve this with Series.where.
The following will work, but I cannot promise how efficient this is. (I suspect it may be better to produce a whole column of replacements at once with numpy.choice.)
df['A'] = d['A'].where(~d['A'].isnull(),
lambda df: df.map(
lambda x: random.choice(['value1', 'value1', x])))
explanation: if the value is not null (NaN), certainly keep the original. Where it is null, replace with the corresonding values of the dataframe produced by the first lambda. This maps values of the dataframe (chunks) to randomly choose the original value for 1/3 and 'value1' for others.
Note that, depending on your data, this likely has changed the data type of the column.

TypeError during executemany() INSERT statement using a list of strings

I am trying to just do a basic INSERT operation to a PostgreSQL database through Python via the Psycopg2 module. I have read a great many of the questions already posted regarding this subject as well as the documentation but I seem to have done something uniquely wrong and none of the fixes seem to work for my code.
#API CALL + JSON decoding here
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers
)
The loop creates a list of strings that looks like this when printed:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
I am just trying to have the code INSERT these strings as 1 row each into the table.
The error specified when running is:
TypeError: not all arguments converted during string formatting
I tried changing the INSERT to:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", (activeUsers,) )
But that seems like it's merely treating the entire list as a single string as it yields:
psycopg2.DataError: value too long for type character varying(30)
What am I missing?
First in the code you pasted:
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
Is not the right way to accomplish what you are trying to do.
first list is a reserved word in python and you shouldn't use it as a variable name. I am assuming you meant ulist.
if you really need access to the index of an item in python you can use enumerate:
for x, item in enumerate(ulist):
but, the best way to do what you are trying to do is something like
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append(str(item['name']))
Your first try was:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
Your second attempt was:
(['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo'], )
What I think you want is:
[['b2ong'], ['dune'], ['drble'], ['drars'], ['feman'], ['got'], ['urbo']]
You could get this many ways:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", [ [a] for a in activeUsers] )
or event better:
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append([str(item['name'])])
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers)

couchbase view using multiple keys to get result

I have the following document
{
"Credit_Amount": 99,
"Acc_no": 138,
"Job_No": "esmwga",
"Source_No": "x",
"Temp": 1017,
"Document_No": "gaf",
"Debit_Amount": 67,
"User_Id": "xirbzsewiw"
}
and my map function is this
function (doc, meta) {
if(doc.Type == "GLEntry")
{
emit([doc.Acc_no,doc.User_Id],[doc.Credit_Amount,doc.Debit_Amount]);
}
}
and this is my reduce function
function(key,values,rereduce){
var sum1=0,sum2=0;
for(var i=0;i<values.length;++i)
{
sum1+=values[i][0];
sum2+=values[i][1];
}
return ([sum1,sum2])
}
when I pass this key
[138,"xirbzsewiw"]
group level 2
I get this output
[ 99, 67 ]
But When I give this as key
[138]
group level 1
I get empty result. But what I have understood is it will group using only acc number when I give group level 1 so it should give same output. Am I doing something wrong?
Abhi is correct, the result set for your specified key is empty so the reduce is also empty. You can check that by querying with reduce=false.
You are probably confused from another question you asked where you are using startkey and endkey to get a range. With startkey and endkey you do not need to specify exact keys, the first partial match will be treated as the start or end. In your example if you query with startkey=[138]&endkey=[139]&inclusive_end=false you should see the result you expect.
Here is what I think is probably happening here:
Your map function emits keys like: [138,"xirbzsewiw"]
You need to pass key that your map function generates as "keys" while doing reduce operations. In 2nd case, you're passing [138], which isn't a legitimate key so it isn't matching anything in index and hence you aren't seeing any output.
group_level does filtering on output i.e. numbers of indexes upto which you want to get from an array of strings. You might want to look at understanding group level view queries, if you haven't already to have better understanding of group_level

"IN" predicate with Criteria filtering isn't working

I'm trying to filter a doctrine collection directly with Criteria in order to avoid all collection-loading when I'm searching only for certain elements.
public function bar()
{
$entity = ...; // not useful for the example
$property = ...; // not useful for the example but is a getter for a collection
$ids = [22, 13]; // just for the sake of this example
$criteria = Criteria::create()->where(Criteria::expr()->in("id", $ids));
return $this->foo($entity, $property, $criteria);
}
public function foo($entity, $property, Criteria $criteria)
{
return $this->propertyAccessor->getValue($entity, $property)->matching($criteria);
}
Above code will produce this DQL (I'll show only last and relevant part)
AND te.id = ?' with params [22,13]:
And so, this error
Notice: Array to string conversion
I suppose that wrong DQL is generated here, but I don't know why.
Does anyone got a clue?
This is the collection property I'm trying to match on
/**
* #ORM\ManyToMany(targetEntity="Vendor\Bundle\Entity\Foo")
* #ORM\JoinTable(name="foo_bar",
* joinColumns={#ORM\JoinColumn(name="foo_bar_id", referencedColumnName="id", onDelete="cascade")},
* inverseJoinColumns={#ORM\JoinColumn(name="foo_id", referencedColumnName="id")}
* )
*/
protected $foo;
Edit
If I try to dump $criteria object, I obtain this
Criteria {#10958 ▼
-expression: Comparison {#10956 ▼
-field: "id"
-op: "IN"
-value: Value {#10948 ▼
-value: array:2 [▼
0 => 22
1 => 13
]
}
}
-orderings: []
-firstResult: null
-maxResults: null
}
So this seems to be correct. I'm also aware I could use "or" expressions, but I would prefer to understand why this issue is taking place.
Edit2
I've changed $criteria as follows
$criteria = Criteria::create()
->where(Criteria::expr()->eq('id', 22))
->orWhere(Criteria::expr()->eq('id', 13));
As now, query is correct but collection isn't: it is empty but shouldn't be (if I run manually that query from mysql cli I obtain what I expect).
So ...
BONUS QUESTION
Can I use Criteria only inside entities getters or repository if I collection isn't already loaded?
Edit3 - Bonus question
It seems that if you don't have collection already loaded, matching criteria is pretty useless. Infact if I loop onto collection elements and call some getters and, then, I use criteria, resulting collection is what I was searching for (but, of course, ALL collection elements are now loaded)
I've found this
https://github.com/doctrine/doctrine2/issues/4910
It seems that is a "well-known" issue in ManyToMany collection situations

How to query multi values of enum field in ActiveRecord?

mymodel.rb
enum status: { posted: 1, failed: 2, suspended: 3 }
mycontroller.rb
def filter_params
params.fetch(:mymodel, {}).
permit(
:status => []
)
end
And i have params like mymodel[:status] => ["failed", "suspended"]
How can i get all results by status is failed and suspended
Something like: Mymodel.where(status: filter_params[:status])
Thanks a lot!
And when a call:
#mymodel = Mymodel.new(filter_params)
I got this error:
'["failed", "suspended"]' is not a valid status
When running a query, you need to supply the ordinal values for the enum attribute. So instead of strings like 'failed' or 'suspended', you need to query using their integer values.
Luckily, you can access a hash to easily map all the statuses to integers from your filter_params hash:
values = Mymodel.statuses.values_at(*Array(filter_params[:status]))
With that you can run your query to get all records which have any of the filtered statuses:
Mymodel.where(status: values)
You don't want to scatter that piece of code all over the place though, so I recommend you'd implement this as a scope in your model:
class Mymodel < ActiveRecord::Base
enum status: { posted: 1, failed: 2, suspended: 3 }
scope :for_statuses, ->(values) do
return all if values.blank?
where(status: statuses.values_at(*Array(values)))
end
end
Note that the return all if values.blank? line makes it possible to throw in nil or an empty array without breaking your query.
You can now easily query the records:
Mymodel.for_statuses(filter_params[:status])
Note that you cannot create a record which has multiple statuses. enum only restricts the values that can be assigned, but you can assign only one, otherwise you get the not a valid status error.
See the Rails documentation for more information about enum.
In Rails 5 you can now pass a string array to the query, e.g:
Mymodel.where(status: ['failed', 'suspended'])
For earlier versions, just convert your array values to symbols:
statuses = filter_params[:status].map(&:to_sym)
Mymodel.where(status: statuses)