I have a query which shows 3566 results what is ok.
When I use paginate on it, result is 18 but in console I see that query which it runs is ok
this is my controller
def listcontractors
#listcons = Contract.paginate(:page => params[:page], :per_page => 50).joins(:contractor)
.select("contractors.id,name,ico,city,country,count(resultinfo_id)")
.group("contractors.id,name,ico,city,country")
.order("name")
end
this is query I see in console, when I put it in psql result is ok
(22.2ms) SELECT COUNT(*) AS count_all,
contractors.id,name,ico,city,country AS
contractors_id_name_ico_city_country FROM "contractors" INNER JOIN
"contracts" ON "contracts"."contractor_id" = "contractors"."id" GROUP
BY contractors.id,name,ico,city,country Contractor Load (30.8ms)
SELECT contractors.id,name,ico,city,country,count(resultinfo_id) as
count FROM "contractors" INNER JOIN "contracts" ON
"contracts"."contractor_id" = "contractors"."id" GROUP BY
contractors.id,name,ico,city,country ORDER BY name LIMIT 50 OFFSET
1050
when I remove .paginate part from the query, result is ok
my models are
class Contract < ActiveRecord::Base
belongs_to :resultinfo
belongs_to :contractor
end
class Contractor < ActiveRecord::Base
has_many :contracts
end
I tried to switch query to Contractor.joins(:contract) but issue was same, with paginate result is much lower than it should be
any idea why this happens?
thanks
thanks to gmcnaughton I created this solution
ids = Contractor.order("name").pluck(:id)
#listcons = ids.paginate(:page => params[:page], :per_page => 50)
#groupedcons = Contractor.joins(:contracts)
.where(id: #listcons)
.select("contractors.id,name,ico,city,country,count(resultinfo_id)")
.group("contractors.id,name,ico,city,country")
.order("name")
and I had to add to initializers require 'will_paginate/array' because otherwise it shows undefined total_pages method for an array
Mixing paginate and group is tricky. paginate sets an OFFSET and LIMIT on the query, which get applied to the result of the GROUP BY -- rather than limiting what records will get grouped.
If you want to paginate through the all Contracts, then group each page of 50 results (one page at a time), try this:
def listcontractors
# get one page of contract ids
ids = Contract.paginate(:page => params[:page], :per_page => 50).pluck(:id)
# group just the items in that page
#listcons = Contract.where(id: ids)
.select("contractors.id,name,ico,city,country,count(resultinfo_id)")
.group("contractors.id,name,ico,city,country")
.order("name")
end
Hope that helps!
Related
Let us we have a Rails 4.2.x app and we have two tables posts and authors, and we want to use Arel to get the posts authored by an author with name == 'Karl'.
(In this case we could be happy with Active Record joins but this is just to keep the example simple.)
posts = Arel::Table.new :posts
authors = Arel::Table.new :authors
my_query = posts.project(Arel.star)
.join(authors)
.on(posts[:author_id].eq(authors[:id]))
.where(authors[:name].eq('Karl'))
> my_query.class
=> Arel::SelectManager
Now we could get back an array (of class Array) of posts by doing:
> Post.find_by_sql my_query
[master] Post Load (3.1ms) SELECT * FROM "posts" INNER JOIN "authors"
ON "posts"."author_id" = "authors"."id"
WHERE "authors"."name" = 'Karl'
=> [#<Post:0x005612815ebdf8
id: 7474,
...
]
So we do get an array of posts, not an active record relation:
> Post.find_by_sql(my_query).class
=> Array
Also injecting the manager into Post.where won't work
> Post.where my_query
=> #<Post::ActiveRecord_Relation:0x2b13cdc957bc>
> Post.where(my_query).first
ActiveRecord::StatementInvalid: PG::SyntaxError:
ERROR: subquery must return only one column
SELECT "posts".* FROM "posts"
WHERE ((SELECT * FROM "posts" INNER JOIN "authors" ON "posts"."author_id" = "authors"."id" WHERE "authors"."name" = 'Karel'))
ORDER BY "posts"."id" ASC LIMIT 1
I am thinking I must be missing something. In short: how do you get an active record relation from a select manager like my_query above (or another select manager accomplishing the same thing).
You can't get ActiveRecord::Relation from Arel::SelectManager neither from sql string. You have two ways to load data through ActiveRecord:
Do all query logic in Arel. In this case you can't use any of ActiveRecord::Relation methods. But you have same functionality in Arel. In your example you may set limit through Arel:
my_query.take(10)
Other way is to use Arel in ActiveRecord::Relation methods. You may rewrite your query like this:
posts = Arel::Table.new :posts
authors = Arel::Table.new :authors
join = posts.join(authors).
on(posts[:author_id].eq(authors[:id])).
join_sources
my_query = Post.
joins(join).
where(authors[:name].eq('Karl'))
> my_query.class
=> ActiveRecord::Relation
In this case you may use my_query as ActiveRecord::Relation
I currently have the following models: MinorCategory > Product > Review
On a view, I show the 12 MinorCategories that have the most reviews. This view is very slow to respond, and I think it is a problem with how I do the query.
Here is my current code:
class MinorCategory < ActiveRecord::Base
has_many :products
has_many :reviews, through: :products
...
def count_reviews
self.reviews.count
end
...
end
class Review < ActiveRecord::Base
belongs_to :product, touch: true
...
end
class HomeController < ApplicationController
#categories = MinorCategory.all.sort_by(&:count_reviews).reverse.take(12)
end
So that is basically it. In the view itself I go through each #categories and display a few things, but the query in the controller is what seems to be slow. From SkyLight:
SELECT COUNT(*) FROM "reviews" INNER JOIN "products" ON "reviews"."product_id" = "products"."id" WHERE "products"."minor_category_id" = ? ... avg 472ms
I am not good with sql or active record, and still pretty new to Ruby on Rails. I've spent a couple hours trying other methods, but I can not get them to work so I thought I would check here.
Thank you in advance to anybody that has a moment.
You need some basic SQL knowledge to better understand how database queries work, and how to take advantage of a DBMS. Using ActiveRecord is not an excuse to not learn some SQL.
That said, your query is very inefficient because you don't use the power of the database at all. It's a waste of resources both on the Ruby environment and on the database environment.
The only database query is
MinorCategory.all
which extracts all the records. This is insanely expensive, especially if you have a large number of categories.
Moreover, self.reviews.count is largely inefficient because it is affected by the N+1 query issue.
Last but not least, the sorting and limiting is made in the Ruby environment, whereas you should really do it in the database.
You can easily obtain a more efficient query by taking advantage of the database computation capabilities. You will need to join the two tables together. The query should look like:
SELECT
minor_categories.*, COUNT(reviews.id) AS reviews_count
FROM
"minor_categories" INNER JOIN "reviews" ON "reviews"."minor_category_id" = "minor_categories"."id"
GROUP BY
minor_categories.id
ORDER BY
reviews_count DESC
LIMIT 10
which in ActiveRecord translates as
categories = MinorCategory.select('minor_categories.*, COUNT(reviews.id) AS reviews_count').joins(:reviews).order('reviews_count DESC').group('minor_categories.id').limit(10)
You can access a single category count by using reviews_count
# take a category
category = categories[0]
category.reviews_count
Another approach that doesn't require a JOIN would be to cache the counter in the category table.
I was hoping this would be simple, but in essence I would like to count the number of Users who have certain attributes through a join table in Rails 4.
I have a table called Views which holds three columns:
t.references :viewer
t.references :viewed
t.boolean :sentmessage, default: false
I then have the fields references as:
belongs_to :viewer, :class_name => "User"
belongs_to :viewed, :class_name => "User"
Each user record is then associated with a number of other records like Stats, Questions and a number of others. I'm interested in effectively counting how many viewers of a viewed record are Male or Female (and other search fields) which is data all held in User.stat.gender.name etc.
I'm trying to use a group statement but have no idea how to drill down and count the number of Males etc. I've tried:
#results = View.where(viewed: 63).group("viewer.stat.gender")
But this is so wrong it's frightening.
Any help to do this would be appreciated.
I worked it out finally. For anyone else who is interested:
View.where(viewed_id: 63).joins(viewer: {stat: :gender}).group("name").count
Didn't realise what an INNER JOIN was but some research and some trial and error means I can now show information about the users who have visited.
My models:
OrderStatus
belongs_to Order
Order
has_one OrderStatus
belongs_to Logo
Logo
has_many Orders
I would like to perform a query on Logo model attribute named artwork:
OrderStatus.includes({:order => :logo}).where(:order => {:logo => {:artwork => search_artwork}})
but it basically throws an error:
SQLite3::SQLException: no such column: order.logo: SELECT COUNT(DISTINCT "order_statuses"."id") FROM "order_statuses" LEFT OUTER JOIN "orders" ON "orders"."id" = "order_statuses"."order_id" LEFT OUTER JOIN "logos" ON "logos"."id" = "orders"."logo_id" WHERE "order"."logo" = '---
:artwork: xxxxxxx'
I can't see the reason of this error.
EDIT
After extensive searching I realised that where part should be using table names (that is, plurals), so my code should be
#order_statuses = OrderStatus.includes(:order => [:logo]).where(:orders => {:logos => {:artwork => search_artwork}})
but I still see the SQLite3 exception error
SQLite3::SQLException: no such column: orders.logos: SELECT COUNT(DISTINCT "order_statuses"."id") FROM "order_statuses" LEFT OUTER JOIN "orders" ON "orders"."id" = "order_statuses"."order_id" LEFT OUTER JOIN "logos" ON "logos"."id" = "orders"."logo_id" WHERE "orders"."logos" = '---
:artwork: xxxxxxxx
'
So the quick answer to
I still see the SQLite3 exception error ... no such column: orders.logos"
Is that your orders table doesn't have a column named logos, it has a column named logo_id, which was created in your migration you mentioned in your comment.
The reason this error is happening is that Rails doesn't understand (as far as I know) nested values in a .where clause.
Perhaps more importantly, I think what you're (probably) more interested in is:
how do I write logic that will give me all the OrderStatus records related to the Logo with an artwork value equal to search_artwork?
If so, adding a has_one :through relationship to your Logo model will make this really easy!
OrderStatus
belongs_to Order
Order
has_one OrderStatus
belongs_to Logo
Logo
has_many Orders
has_many OrderStatuses, :through => :Order # This is what you're adding!
Then, your query should be as simple as:
Logo.where(artwork: search_artwork).includes(:orderStatus)
EDIT: oops the above is wrong
Logo.where(artwork: search_artwork).order_statuses
For more information on Rails relationships and using has_many :through, see:
http://guides.rubyonrails.org/association_basics.html#the-has-many-through-association
I have a channel model with 2 associations, "contents" and "subscriptions".
In the channel index the user has the possibility of ordering the channels by number of subscriptions or number of approved contents.
While in development everything seems to work properly (by observation of the results, can be malfunctioning and be a question of not enough data to see it properly), in staging the results are random, sometimes showing them properly, sometimes don't.
At first I wasn't using delta indexes and thought the problem could be there so every time I approve a content I call:
Delayed::Job.enqueue(DelayedRake.new("ts:index"), queue: "sphinx")
Since the subscriptions don't have indexes, I don't reindex every time I create one ( should I do it? )
Then I started using delta indexes in the channel and I still get the same problems:
ThinkingSphinx::Index.define :channel, with: :active_record, delta: true do
# fields
indexes :name, sortable: true
indexes description
# attributes
has created_at, sortable: true
has approved, type: :boolean
has public, type: :boolean
join subscriptions
has "COUNT(subscriptions.id)", as: :subscription_count, type: :integer, sortable: true
join contents.approved
has "COUNT(contents.id)", as: :content_count, type: :integer, sortable: true
end
And here is the search call in the controller:
def index
if params[:order_by].present?
#channels = Channel.search params[:search],
order: "#{params[:order_by]} DESC",
page: params[:page], per_page: 6
else
#channels = Channel.search params[:search],
order: :name,
page: params[:page], per_page: 6
end
end
Summarising, my questions would be:
1. Are my channel indexes well formed?
2. Should subscriptions by indexed as well or is it enough to join them in my channel index?
3. Should I run reindex after I create a subscription / approve a content or the delta index in the channel deals with that since I have those two controllers joined in the channel index?
Your index looks fine, but if you're using deltas (and I think that's the wisest approach here, to have the data up-to-date), then you want to fire deltas for the related channels when a subscription or content is created/edited/deleted. This is covered in the documentation (see the "Deltas and Associations" section), but you'd be looking at something like this in both Subscription and Content:
after_save :set_channel_delta_flag
after_destroy :set_channel_delta_flag
# ...
private
def set_channel_delta_flag
channel.update_attributes :delta => true
end
Given you're using Delayed Job, I'd recommend investigating ts-delayed-delta to ensure delta updates are happening out of your normal HTTP request flow. And I highly recommend not running a full index after every change - that has the potential of getting quite slow quite quickly (and adding to the server load unnecessarily).