Can anyone explain to me what state the data should be in for a healthy sitecore instance in each database?
for example:
We currently have an issue with publishing in a 2 server setup.
Our staging server hosts the SQL instance and the authoring / staging instance of sitecore.
We then have a second server to host just the production website for our corp site.
When I look in the master database the PublishQueue table is full of entries and the same table in the web database is empty.
Is this correct?
No amount of hitting publish buttons is changing that at the moment.
How do I determine what the state of an item is in both staging and production environments without having to write an application on top of the sitecore API which I really don't have time for?
This is a normal behavior for the Publish Queue of the Web Database to be blank. The reason is because changes are made on the Master database which will add an entry in the Publish Queue.
After publishing, the item will not be removed from the Publish Queue table. It is the job of the CleanupPublishQueue to cleanup the publish queue table.
In general, tables WILL be different between the two databases as they are used for different purposes. Your master database is generally connected to by authors and the publishing logic, while the web database is generally used as a holding place for the latest published version of content that should be visible.
In terms of debugging publishing, from the Sitecore desktop, you can swap between 'master' and 'web' databases in the lower right corner and use the Content Editor to examine any individual item. This is useful for spot checking individual items have been published successfully.
If an item is missing from 'web', or the wrong version is in 'web', you should examine the following:
Publishing Restrictions on the item: Is there a restriction applied to the item or version that prevents it from publishing at this time?
Workflow state: Is the item/version in the final approved workflow state? You can use the workbox to do a quick check for items needing approval.
Connection strings: Is your staging system connection strings setup to connect to the correct 'web' used by the production delivery server?
The Database table [PublishQueue] is a table where all save and other mutations are stored. This table is used by a Incremental Publish. Sitecore get all the items from the PublishQueue table that were modified more recently than the last incremental publish date. The PublishQueue tabel is not used by a full publish
So it is okay that this Table contain a lot of records on the Master. The web database has the same database scheme. (not the same data, web contain only one version of a item, optimize for performance) The PublishQueue on the web is Empty this is normal.
To Know the state of an item compair the master version with the web version, there can be more than 1 webdatabase, The master database do not know the state/version of the web database
Related
I have configured API Manager 2.0.0 & API Manager Analytics Pack to use MySQL databases.
For each server, there exists a WSO2AM_STATS_DB. I have given these differing names on my MySQL server. I have also pointed my datasources in master-datasources.xml(for APIM) & stats-datasources.xml(for Analytics) to the relevant databases.
I couldn't find any relevant schema(dbscripts) for these databases in their respective packs.
On running, the Analytics database is populated but the APIM database isn't and throws an exception. The Analytics database not only gets the schema but also the invocation details of my API.
I am unable to get the stats on my dashboard though.
Previously, I (unwittingly) configured the h2-repository stats database to be the same for both servers (due to the folder structure) and was able to get all the statistics on my dashboard in the publisher.
Other configurations I have tried :
On the MySQL Server, pointed it to the same database (the Analytics one with the schema) but with no results on my dashboard (after waiting for a while).
Both datasources (WSO2AM_STATS_DB) in 2 servers should be pointed to the same database. There are no database scripts for this. Tables are created automatically.
By default in both servers, Stats DB path comes like this. (note ../ part)
<url>jdbc:h2:../tmpStatDB/WSO2AM_STATS_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000;AUTO_SERVER=TRUE</url>
So if you extract both servers to the same directory as mentioned in this doc, both datasources will be pointing to the same database (inside tmpStatDB) like this.
/parent_dir
|__wso2am-2.0.0/
|__wso2am-analytics-2.0.0/
|__tmpStatDB/
So, what happens here is, wso2am-analytics writes stats data to shared database, then apim reads it and shows data on its databases.
I am working with Sitecore Intranet Portal. I am using web database for CD.
If a user change his email Id. How would I publish this to master database.
I am using this code to publish item from web to master db.
// The publishOptions determine the source and target database,
// the publish mode and language, and the publish date
var publishOptions =
new PublishOptions(Database.GetDatabase("web"), Database.GetDatabase("master"),
PublishMode.SingleItem,
item.Language,
DateTime.Now);
var publisher = new Publisher(publishOptions);
// Choose where to publish from
publisher.Options.RootItem = item;
// Publish children as well?
publisher.Options.Deep = true;
// Do the publish!
publisher.Publish();
It would be nice to know the correct procedure.
To publish from web to master is a bad practice.
This kind of content is named User Generated Content. I suggested to use this approach :
https://sitecore.unic.com/2015/07/16/user-generated-content-in-a-security-hardened-environment
Or you can use webservices. This allows all content (even user-generated) to be authored in your CM instance. This allows you to leverage the capabilities of the platform for workflow, publishing, etc.
Either use an external database or use Sitecore users for a certain domain and store everything in the core database. The core database is commonly shared between all environments
You can make use of package to transfer your item from web to master. You can follow the following steps:
Create your item package from web.
Use the tool that i have developed to convert the destination of the items. That is, change from web to master. The tool is found on marketplace at Sitecore package modifier.
Install the modified package on the master database.
Another solution is to have a schedule job that create the package from web, modified it and install it on the master database. This hence require no manual intervention.
I prefer to use this method than the sitecore transfer method because the transfer method tends to timeout or cause an error if there are lots of items to transfer.
The best practice to store the user information in the Core database. Sitecore is using ASP.NET membership to manage the user accounts. I recommend you to store all the user info in the core DB.
Sitecore doesn't provide any option to publish data from web to master, however, it has provides you an option to transfer data from web DB to master DB.
Check out my blog Transfer Items from Web to Master Database for complete details on transferring the data from one DB to another DB
Step 1: Login to Sitecore desktop mode.
Step 2: Select the source database from where you need to transfer the items. If you want to transfer from Web to Master then select Web Database in the right-hand bottom corner.
Step 3: Open the content editor and select the item which you would like to transfer.
Step 4: Right click on the Item and select Copying>>Transfer and click on the transfer button
Step 5: In the Transfer window verify if the source item is selected properly or not.
Step 6: Click on Next to continue. In this window first thing is to select the destination database to which you need to transfer. The second thing is to select the parent item or the destination folder where you need to place this item. In my case, I need to select Layout because sublayouts are present in the layouts item.
Step 7: Click on next, here you will get an option which says include subitems. Select the checkbox if you need to restore the subitems also else uncheck the box to transfer only the selected item.
Step 8: Click on transfer button and it will start the transfer process. Close the dialog box once the transfer is completed.
We are having an issue with the EventQueue table growing very fast at times, up to 3k records a second, and never clearing records (30 million as of right now). Our environment has the following set up:
Sitecore 7.2
4 CD servers and 1 CM server
All four CD servers are load balanced.
CD1 and CD2 are pointed to DB1 server CD3 and CD4 are pointed to DB2
server There are 2 Publishing targets (one for each DB) Merge
Replication is setup for the Core db across all servers (CM, CD's)
EventQueue is enabled
I have a few questions so I will break them down into separate line items.
When a publish is issued for all CD servers is the updated content sent directly from the CM db to the CD db's (all of the correct tables) or is it sent to the EventQueue table in the CD db and the CD server has a job/task that looks at the table and updates as needed.
Depending on answer to the first question, if there are 2 CD servers pointing to the same DB how do they know if they should process the EventQueue table (wont they each process the table and be duplicating efforts)
Why isn't the EventTable table cleared? How is is cleared, when is it cleared?
On CM publish, the publish request is sent to the EventQueue table on the CD db where it is processed as per the instance's publishing schedule.
The InstanceName column in the EventQueue table stores the unique name of each Sitecore instance (by default this is Machine Name + IIS Instance Name, but can be set in web.config). This enables events to be picked up by an individual CD instance in a load balanced environment.
The EventQueue table is cleared by a Sitecore task defined in the <scheduling> element in the web.config, although I've seen this misbehave in the past. By default, it is set as follows:
<agent type="Sitecore.Tasks.CleanupEventQueue, Sitecore.Kernel" method="Run" interval="04:00:00">
<DaysToKeep>1</DaysToKeep>
</agent>
I've previously run into high loads on the EventQueue and PublishQueue tables and would recommend trying the following (some of which were suggested from Sitecore support):
Reduce the interval of the CleanupEventQueue agent (above)
Reduce the DaysToKeep setting on the CleanupEventQueue (also CleanupPublishQueue wouldn't hurt)
Create a scheduled SQL job to run the clean up script outlined in the CMS Tuning Guide (Page 10: http://sdn.sitecore.net/upload/sitecore7/70/cms_tuning_guide_sc70-usletter.pdf)
Finally, from Sitecore support:
Sitecore recommends that the number of rows (entries) in the History, PublishQueue, and EventQueue tables would be less than 1000.
I was trying to create a report from master database in analytics reports. (Stimulsoft Report Designer)
As it explains in the reports cookbook, I have created a "mrt file" (Report UI) and a report definition item in Engagement analytics.
I have configured the datasource item as query item
(/sitecore/system/Settings/Analytics/Reports SQL Queries/Visit Pages).
It worked.
But then I tried with a query using the master database, in the SQL query item I specifically mentioned the database as 'testProjectMaster' to point to master database. It did not work!
Then I figured out that in "/sitecore/system/Settings/Analytics/Reports SQL Queries/Visit Pages" item and other query items, it does not specify the database, that means by default sitecore queries the analytics database.
Is this a limitaion in sitecore, cant we query the master databse for reports? Are there any good resources to follow on creating reports?
I suggest taking the SQL from the Visit Pages report and running it in SQL Server Management Studio. There, you will be able to quickly see what's preventing your query from running. If I had to venture a guess, I would suspect that your SQL user account does not have db_datareader access to the master database.
The default SQL queries provided by Sitecore assume that the DMS is configured as the default database in the connection string. This, however, does not prevent you from querying other databases or doing cross-database joins like so:
SELECT TOP 100 * FROM Pages
INNER JOIN Sitecore_Master.dbo.Items AS MasterItems ON Pages.ItemId = MasterItems.ID
A word of caution.. from my experience, this can really slow down your reports as it does not take advantage of indexing and creating indexed views doesn't work across multiple databases.
I work with SQL Server database with ODBC, C++. I want to detect modifications in some tables of the database: another application inserts or updates rows and I have to detect all these modifications. It does not have to be the immediate trigger, it is acceptable to use polling to periodically check database tables for modifications.
Below is the way I think this can be done, and need your opinions whether this is the standard/right way of doing this, or any better approaches exist.
What I've thought of is this: I add triggers in SQL Server, which, on any modification, will insert the identifiers of modified/added rows into special table, which I will check periodically from my application. Suppose there are 3 tables: Customers, Products, Services. i will make three additional tables: Change_Customers, Change_Products, Change_Services, and will insert the identifiers of modified rows of the respective tables. Then I will read these Change_* tables from my application periodically and delete processed records.
Now if you agree that above solution is right, I have another question: Is it better to have separate Change_* tables for each of my tables I wish to monitor, or is it better to have one fat Changes table which will contain the changes from all tables.
Query Notifications is the technology designed to do exactly what you're describing. You can leverage Query Notifications from managed clients via the well known SqlDependency class, but there are native Ole DB and ODBC ways too. See Working with Query Notifications, the paragraphs about SSPROP_QP_NOTIFICATION_MSGTEXT (OleDB) and SQL_SOPT_SS_QUERYNOTIFICATION_MSGTEXT (ODBC). See The Mysterious Notification for an explanation how Query Notifications work.
This is the only polling-free solution that work with any kind of updates. Triggers and polling for changes has severe scalability and performance issues. Change Data Capture and Change Tracking are really covering a different topic (synchronizing datasets for occasionally connected devices, eg. Sync Framework).
Change Data Capture(CDC)--http://msdn.microsoft.com/en-us/library/cc645937.aspx
First you will need to enable CDC in database
::
USE db_name
GO
EXEC sys.sp_cdc_enable_db
GO
Enable CDC on table then
:: sys.sp_cdc_enable_table
Then you can query changes
If your version of Sql Server is 2005 - you may use Notification Services
If your Sql Server is 2008+ - there is most preferrable way to use triggers and log changes to log tables and periodically poll these tables from application to see the changes