I came across an old project previously written for Sitecore 6.4 and now updated to Sitecore 7.2.
There is a fast query that does not return results:
1. fast:/sitecore/content/Home/About Us/News//*[##templatename='Newsletter']
I tried to tweak the query and these two are working fine:
2. fast:/sitecore/content/Home/About Us/News/descendant::*[##templatename='Newsletter']
3. fast:/sitecore/content/Home/About Us/News/Newsletters//*[##templatename='Newsletter']
The Newsletter items are not direct children of Newsletters item either, there is another layer in between.
So Why the query 1 does not work while 2 & 3 return exactly what I need?
Check your web.config setting for FastQueryDescendantsDisabled.
Rebuild your Descendents table via Control Panel, Databases, Clean Up Databases
Reference: http://sdn.sitecore.net/upload/sdn5/developer/using%20sitecore%20fast%20query/using%20sitecore%20fast%20query001.pdf
Check to see if the number of descendants of "News" is under whatever the maximum returned items size is set to in the web config (<setting name="Query.MaxItems" value="100" />).
While you aren't returning items while looking at the descendants, it is possible that the older version of Sitecore only looks at the max number of items (which would be a bug). The fact that query 3 works and query 1 doesn't is says to me that this is likely the issue. Query 2 is trying to do the same thing, but queries 1 and 3 also happen to be the same syntax. Since all three are meant to do the exact same thing (the only exception being that /News/Newsletters is looking from deeper root item), I would expect that this is a bug.
You can test this by setting the value of the Query.MaxItems setting to a ridiculously high number, like 5000 (note that you should change the value back after testing, as this can greatly degrade performance). If query 1 now returns items, then this is your issue. Otherwise, try setting the value higher. If it still doesn't return values after that, then this is not your issue.
Let me know if you have any questions. Good luck and happy coding! :)
Related
Power BI newbie question here.
Whenever I add a Group By step with a Text.Combine() or a Max() aggregate, applying changes or refreshing data results in the aforementioned exception.
My datasource is a D365 dataverse connection, all queries run just fine until I add a step to group and aggregate. As an example, starting with a very simple query with 2 columns (demandId, kor_subcontractorbillnumber) I want to concatenate in a csv column all billNumbers related to a given demandId :
= Table.Group(#"Table Buffer", {"demandId"}, {{"BillNumbers", each Text.Combine([kor_subcontractorbillnumber],", "), type nullable text}})
As seen in the attached screenshot, the preview on screen seems correct : the expected result is displayed in the BillNumbers column, and no error is reported in the column quality indicators. All is fine...until I click Apply, which raises the exception.
I tried to clean the columns as much as possible before grouping (removing empty values, errors, duplicates, etc.), as well as adding an extra step to store results in a table buffer before grouping but with no luck.
Browsing through SO I found that similar issues could be related to :
Wrong relationship cardinalities : does not apply here I guess since everything is correct in the buffer table until I group
Power Bi Desktop update : some users have reported in the past that an update broke something and gave the same exception. In my case, the issue started occurring after upgrading to July 2022 version and unfortunately it seems I can't downgrade to a previous version. I've started using PowerBi in June and do not have much experience to detect whether the july update actually broke something, though some reports ceased functioning short time after the update.
Even stranger : If I remove the last step (Group By) and I create a new query referencing this one... I can add a Group By step and apply my changes...until I Refresh my report : at this point all the embedded queries fail with the same exception, even those absolutely unrelated with my changes.
Could anyone explain me what I'm doing wrong or if you have experienced the same behavior with the last version of Power Bi desktop ( 2.107.841.0 64-bit), which could point me to the right direction ?
Thanks for your help !
After many tries, I eventually stumbled upon a workaround: instead of the Group By step, I clicked on the very last step of my query and selected 'Extract Previous'. This created a new query (result of all previous steps), and I was able to perform my Group By on this new query without any errors.
I have no idea how this is different from adding the Group By at the end of the first query... but the exception is gone. Kind of a code smell anyway...I mark my own question as answered in case it can help someone, but I'd more than happy if someone could shed some light on the underlying reason of this issue.
In our Sitecore (6.6) implementation we use Lucene indexing. In our PROD server, index bilding process is very slow. At the moment it has 5000+ entries to waiting in the index queue.
Queries I used (in master database),
select * from Properties (check the index last run time)
select * from History where created > 'last index updated time'
As a result of this delay, data gets created do not reflect their changes in the website. Also this queue keeps increasing. When the site takes offline, index building catch up after a while.
Its a heavy read intensive website.
We encountered CPU going high issues, but now they have been sorted. We thought index building was lagging because of the CPU high issue. But now the CPU is running around 30-40%. Still the lucene indexing queue increase rate is high.
How can I solve this issue? Please help.
You need to set up a database maintenance task, so that you regularly flush your History table. If you have sites that are index heavy, this table can grow excessively large. I think the default job cleans this table out with everything that is older than 30 days - you could set this much lower. Like 1 day, or a couple of days.
This article on SDN covers most of the standard maintenance tasks: http://sdn.sitecore.net/Articles/Administration/Database%20Maintenance.aspx
More general information about searching, indexing and performance here: http://sdn.sitecore.net/upload/sitecore6/65/sitecore_search_and_indexing_sc60-65-a4.pdf#search=%22clean%22
I think you need to take a step back and ask the question as to why there is such a large number of entries being added to the history table to begin with, before looking at what configuration changes to Sitecore can be made.
You should trace through your code in your development environment based on each of the use cases for your implementation, to find all calls to the Sitecore API where an item is:
Added into the Sitecore Tree
Edited - the changing of any fields item including security, presentation, workflow, publishing restrictions, etc.
Duplicated
Deleted from the Sitecore Tree
Moved to a new location.
Has a new version is added
Has a version removed
As you are going through, make sure that all edit actions to an item are performed with in a single Sitecore.Data.Items.Item.Editing.BeginEdit() and Sitecore.Data.Items.Item.Editing.EndEdit() call whenever possible, so that the changes are performed as a single edit action instead of multiple. Every time Sitecore.Data.Items.Item.Editing.EndEdit() is called, a new record will be inserted into the history table so unnecessary edits will only cause the history table size to increase.
If you are duplicating an item using the Sitecore.Data.Items.Item.CopyTo() method, remember that all versions of the item will be duplicated as well as the item's descendants. This means that the history table will have a record in it for every version of the item that was copied. If you only require the latest version and therefore removing older versions from the new item after it was created, again you should be aware that removing a version from an item will result in a record inserted into the history table for each version deleted.
If you have minimized all of the above actions to the bare minimum that is required to make the system functional, you should find that the Lucene Indexing will keep up-to-date pretty well without having to change Sitecore's default index configuration.
My sitecore item hierarchy is like this,
/sitecore/content/Public/level1/level2/level3/Item1/Images/Image1
I tried following query in X-Path builder
/sitecore/content/Public/level1/level2/level3/Item1/Images/Image1/*
Does not return any result...
but, 1 level above returns results, (All images Image1, Image 2 etc.)
/sitecore/content/Public/level1/level2/level3/Item1/Images/*
Following returned Image1 as well,
/sitecore/content/Public/level1/level2/level3/Item1/Images//*
(Data and query executed in the master database)
Is this a sitecore query limitation where you can't query single items which has more that 8 levels?
Check the setting of the <setting name="Query.MaxItems" value="100" /> property in the web.config. It is standard set to return only the first 100 items. Raise the limit and see if this might have something to do with your issue.
I don't know about some restrictions and think this should work. Your query also looks ok. Are you sure that you have some items available under "Image1"? If you only have the described hierarchy with no childs under "Image1", then you won't get any items returned by the query.
Your last query (with //*) also say that only "Image1" is returned. Because the // defines a recursive query, also childs of "Image1" should be returned, if there are any.
We are querying a large SP 2007 document library with over 12,000 documents using the Lists web service, for document comparison purposes.
All queries are built using CAML, to limit the results returned by one of the fields on the list.
In general, the CAML query will return no more than 200 records.
Unfortunately, we are finding that one query will return 20 documents, and the exact same query will return 23 documents 15 minutes later.
As this crawl occurs after hours, it is not possible that documents have been added during that time.
Has anyone experienced similar issues ?
If you're using the Lists.GetListItems method, try setting the RowLimit parameter to something larger.
rowLimit A string that specifies the number of items, or rows, to
display on a page before paging begins. If supplied, the value of this
parameter overrides the row limit set in the view specified by the
viewName parameter or the row limit set in the default view for the
list.
If you don't specify, it will use the limit for the default view which is probably 200 judging by your question.
I don't understand the second part of your question. The search index uses a completely separate web service and you'll never use CAML to query the search index.
Turns out that the issue was related to hardware errors on one of our front end web servers.
This caused a validation failure for some of the List Items.
I have a couple hundred of image thumbnails, 15k each. I want to display 20 or so on each page.
Would django.core.paginator suffice for the pagination of these pages? I.e., will it return only those images displayed on the current page? (And if not, what would be a good way to do this?) Thank you.
Depends, because there is one big limitation from the RDBMS (which affects all databases, including MySQL, Postgres, etc.).
django.core.paginator takes a QuerySet which represent any kind of SQL query and adds a LIMIT clause to just get a couple of entries from the database. This approach works well for many kinds of applications, but might become a serious problem if you have a lot of entries. The particular problem is, that whenever you access the 800th page, the database will actually fetch 801*20 entries and then drop the first 800*20 entries again to return the last twenty.
Unfortunately, there is no easy way to solve this problem. In a lot of cases, a next/prev button might be enough so you can write your own pagination which does operate on after-keys instead of page numbers. For example, if the last entry currently displayed by the user has the key "D" you show a next button which links to /next?after=D and then use a SQL query like SELECT * FROM objects WHERE key >DORDER BY key LIMIT 20. The advantage of this approach is, that you can add an index on objects.key which speed up things significantly.
The other approach requires, that you add an additional, indexed (!) column page_num to your table. Then you can perform SQL queries like SELECT * FROM objects WHERE page_num=800 ORDER BY key. With that approach, you can still access all pages randomly, but you have to maintain the page_num column. This might be easy if data is mostly appended at the end and is more complicated if you want to delete/insert elements from the middle efficiently.
So, I would start with django.core.paginator because it's just about 1 line of code. But keep an eye on the response times of your paginated views and the slowquery log from your database. If your database server can't handle the load anymore, you will have to choose one of the techniques mentioned above. Choose solution 2 if random page access is an requirement and solution 1 otherwise (because it's much simpler).
PS: And yes, django.core.paginator will work correctly. :)