I am using Sitecore 8.2 Update 6. Earlier my CM, Processing and Reporting roles are on single CM server. Now I just need to use separate Processing server and my Reporting and CM will be on one server.
I have configured my processing server as mentioned in the following url:
https://doc.sitecore.net/sitecore_experience_platform/82/setting_up_and_maintaining/xdb/configuring_servers/configure_a_processing_server
and configured my connection strings as per the following url:
https://doc.sitecore.net/sitecore_experience_platform/81/setting_up_and_maintaining/xdb/configuring_servers/database_connection_strings_for_configuring_servers
Now I have couple of questions:
1) IS there any change required in my CM or CD to know about my separate processing server
2) How can I test whether my processing server is doing the required tasks.
Thanks,
Nicks
Your CM and CD do not need to know about the processing server, but you need to make sure that processing functions are not enabled on the CM or CD.
You will know if processing is working by looking at the logs and seeing if the pipelines are executing and not throwing errors.
You will also see analytics data being processed and showing up in the reporting database. If you are not seeing analytics data, this is an indication you might have errors in processing.
Note that there are several possible reasons reporting data might not be working, but if it is succeeding at getting your new analytics data than processing is running.
Related
We have a pipeline to extract embeddings (feature vectors) from images stored in Cloud Storage bucket and insert into a BigQuery table.
We're consistently getting SDK harness sdk-0-1 disconnected. errors when the Dataflow job runs on N1 type VM instances.
Error message from worker:
Data channel closed, unable to send additional data to SDK sdk-0-3
SDK harness sdk-0-1 disconnected.
SDK harness sdk-0-2 disconnected.
SDK harness sdk-0-0 disconnected.
Data channel closed, unable to receive additional data from SDK sdk-0-3
SDK harness sdk-0-1 disconnected.
SDK harness sdk-0-2 disconnected.
Data channel closed, unable to receive additional data from SDK sdk-0-1
Notes
N2 machines work fine but N1 fails somewhat surprising because N1 is Google-default machine.
Jobs run slower on N1 machines and sometimes appear to fail due to these errors.
Using a larger VM (more memory, CPU and disk) didn't resolve the errors.
We also have another pipeline to extract embeddings from text and using lapse model which has the same errors on both N1 and N2 machines
Diagnostics tab: No errors found during this interval.
We're creating DF job templates (Apache Beam 2.40 Python), storing them on Cloud Storage and using API to launch new jobs.
We're batching the items before giving them to the stage where embeddings are extracted. Reducing batch size didn't matter.
Pipeline option sdk_worker_parallelism changed from 0 (default) to 1 and didn't change anything.
Auto-scaling disabled (max_worker=1) and same errors.
Reshuffle stage removed from the pipe
There are disconnect errors e.g. SDK harness sdk-0-0 disconnected.
but no data channel errors e.g. The Data channel closed, unable to send additional data to SDK sdk-0-3
This error message could be due to a wide variety of causes which cannot be easily detected unless the error message is accompanied by the other behavior. This error could be due to any number of listed errors in this documentation.
For getting more information about the error, it can be investigated in the Diagnostics table which can be seen in the below image.
The Diagnostics table shows the timeline and possible recommendations for your pipeline for the errors that occurred. You can view the job metrics to monitor your Dataflow Jobs.
I am using config update and cloud functions for communication between mobile application and esp32 device by following the example here, but when I am sending config update messages frequently some of them are not sending; say out of 5 only 3 config update messages are going, I have two questions:
1) How frequently we can send config update to avoid some missing updates.
2) Is there any alternative way to communicate between cloud functions and IoT device.
According to the docs: [IoT docs]
Configuration updates are limited to 1 update per second, per device.
However, for best results, device configuration should be updated much
less often — at most, once every 10 seconds.
The update rate is calculated as the time between the most recent
server acknowledgment and the next update request.
If your operations are mostly configuration updates I cannot think another alternative that could perform better.
Coldfusion 2016
Microsoft Server 2012
Oracle 12
ODBC connection
I turned on profiling and monitoring and now I can see that there are requests that are stuck and cannot be terminated by the CF monitor; Some are over 200k seconds.
I know I can increase the number of simultaneous requests but I want to solve the underlying problem. As I read the stack traces of these “zombie requests” they are getting stuck on and some are in but some are not. I ran the query in my oracle client and they resolve instantly.
Is there a way to terminate these requests or prevent this from happening at all?
EDIT: The server monitor does not treat these requests as slow or hung, the alerts are not triggering for any of these. Honestly, they should have be going off constantly considering how many of these there are.
Also, the execution time is a mere .003 seconds so what happened? Why doesn't ColdFusion know this?
An example of a "zombie"
The active query that is stuck
We have a similar situation with a different database engine - redbrick, which runs on a unix server. We solved it as follows.
We set up a cron job on the database server to run every 5 minutes. This job uses a combination of unix and awk commands.
This job runs a query against the system table that looks for queries that have been running for more than 120 seconds, where the database account is the one used by ColdFusion. Records are outputted to a file. Something like this:
print "alter system cancel user command userName process " $1 ";"
$1 comes from the query and is the process Id we want to stop.
Then we run the file, which executes all those alter system commands.
With a different database engine, and possible different OS for the database server, the details would be different, but the approach should work.
Edit Starts Here
To prevent recurrence, look at the pages that call the ones with the long running queries. If impatient users are able to repeatedly click something because nothing is happening, do something about that. You can use javascript to make the link/button go away. Alternatively, you can go to an intermediate page with a display for the user and something that carries them through to the real page.
I get the following error:Service Invocation Exeption i am working with Version 8.7 IBM InfoSphere DataStage and QualityStage Designer and using a server job and there, i have 1 sequential file, web service, sequential file.
Any idea what could be the reason of this error ?
Make sure you have chosen proper DataStage job type and your stage that operates on web service is configured properly.
You should also check the DataStage logs to get more information about root cause of the error.
I've got a django site that is producing a csv download. The content of the csv is dictated by user defined parameters. It's possible that users will set parameters that require significant thinking time on the server. I need a way of sustaining the http connection so the browser doesn't kick up an error message. I heard that it's possible to send intermittent http headers to do this. Can anyone point me in the right direction to set this up on a django site?
(unfortunatly I'm stuck with the possibility of slow reports - improving my sql won't mitigate this)
Don't do it online. Trigger an offline task, use a bit of Javascript to repeatedly call a view that checks if the task has finished, and redirect to the finished file when it's ready.
Instead of blocking the user and it's browser for 20 minutes (which is not a good idea) do the time-consuming task in the background. When the task will finish and generate the result simply notify the user so that he/she will just need to download the ready result.