Trying to run Presto coordinator server with discovery server embedded on AWS CDH4 cluster
config.properties:
coordinator=true
datasources=jmx
http-server.http.port=8000
presto-metastore.db.type=h2
presto-metastore.db.filename=var/db/MetaStore
task.max-memory=1GB
discovery-server.enabled=true
discovery.uri=http://ip-10-0-0-11:8000
When server starts it can't register itself with discovery (relevant logs):
2013-11-08T19:38:38.193+0000 WARN main Bootstrap Warning: Configuration property 'discovery.uri' is deprecated and should not be used
2013-11-08T19:38:38.968+0000 INFO main Bootstrap discovery-server.enabled false true
2013-11-08T19:38:38.975+0000 INFO main Bootstrap discovery.uri null http://ip-10-0-0-11:8000 Discovery service base URI
2013-11-08T19:38:40.916+0000 ERROR Discovery-0 io.airlift.discovery.client.CachingServiceSelector Cannot connect to discovery server for refresh (collector/general): Lookup of collector failed for http://ip-10-0-0-11:8000/v1/service/collector/general
2013-11-08T19:38:42.556+0000 ERROR Discovery-1 io.airlift.discovery.client.CachingServiceSelector Cannot connect to discovery server for refresh (presto/general): Lookup of presto failed for http://ip-10-0-0-11:8000/v1/service/presto/general
2013-11-08T19:38:43.854+0000 INFO main org.eclipse.jetty.server.AbstractConnector Started SelectChannelConnector#0.0.0.0:8000
Tried to also run standalone Discovery server, same effect. Looks that listener is started after registration attempt is made.
I was wondering if someone would notice this in the logs :) It's actually not a problem. The error appears because the discovery client starts before the discovery server is ready. You'll see "succeeded for refresh" shortly after in the logs which shows that it's working. We will fix the log message eventually but it's purely a cosmetic issue.
Related
I am new at Redis Enterprise and can't fix this problem:
I have a Redis Enterprise cluster (v.6.0) in AWS with two nodes. When I have only one node I can enter UI, but after adding other (second) nodes always throws me out to the login page after entering credentials. Meanwhile, the cluster works fine (information is taken from rladmin).
In what direction I should investigate the issue?
P.S.: Can this error from logs cause an issue?
ERROR redis_mgr MainThread: Connect failed: connect: connection failed: Error 2 connecting to unix socket: /var/opt/redislabs/run/ccs.sock. No such file or directory.: retrying
Possibly, this solution will help anybody:
the reason was that ALB before UI didn't use sticky sessions.
the solution was to enable a sticky session and it works.
After a power outage I got both of my View Connection Servers unbootable with BSOD and I could not recovery it and also I don't have backup of it.
After all steps below I could not get things fixed, all VMs are "Agent Unreachable"
Created a new VM for connection server (WITH THE SAME NAME AS THE OLD "VIEWCS01")
Before doing it correctly I connected the base disk, and not the last
snapshot, to the new VM and broke up the whole thing with the error
"file system specific implementation of ioctl [file] failed", I solved
this correcting the CID - https://kb.vmware.com/s/article/1007969
Installed Windows (Same version)
As in https://kb.vmware.com/s/article/76770:
Installed Connection Server (Same Version)
Restored LDF backup
Removed all View Connection Servers and all Security Servers (vdmadmin -S -s -r viewcs01, viewss01 and viewcs02)
Uninstalled the connection server
Reinstalled the connection server reusing the AD LDS
I did removed the "viewcs01" because with my previous tests I was not removing it, I think because of >this, after the recovery steps done no console was opening, In previous tests I also not using the old >machine name, instead of it I was using "viewcs03".
Ok, Console opened, I changed the vCenter credentials (Just putting password was not working with error - https://kb.vmware.com/s/article/60152 - Log below:
2020-12-31T18:23:48.108-02:00 ERROR (18F8-0D0C) <MessageFrameWorkDispatch> [ws_java_bridgeDLL] BCryptDecrypt FAILED, status={Data Error}
An error in reading or writing data occurred. (0xC000003E)
2020-12-31T18:23:48.109-02:00 ERROR (18F8-0C54) <VCHealthUpdate> [SecurityManagerUtil] decryptAsText: com.vmware.vdi.crypto.SecurityManagerException: decrypt: Cannot decrypt: Cipher scheme decryption failed.
2020-12-31T18:23:48.109-02:00 DEBUG (18F8-0C54) <VCHealthUpdate> [ServiceConnection25] Connecting instance VCHealth Test instance at URL https://vcenterd.DOMAIN.net:443/sdk
Corrected Composer credentials, and added license.
All machines are "Agent Unreachable" - Connection Server Log below:
2020-12-31T18:23:49.160-02:00 DEBUG (18F8-1A6C) <DesktopControlJMS> [DesktopTracker] CHANGEKEY message from agent/bda3fbe6-029c-41f8-b9f8-017af574f56b accepted as key and thumbprints match machine record
2020-12-31T18:23:49.162-02:00 DEBUG (18F8-1A6C) <DesktopControlJMS> [DesktopTracker] found broker thumbprints: 0f:9e:80:5d:f6:33:c7:1b:a2:d5:8c:9a:9f:12:45:16:0f:6f:c0:2b:46:8d:d0:33:62:87:53:a9:48:8d:57:8c#SHA_256;51:c5:d0:44:02:7f:ca:6d:5a:ad:5b:f6:8d:f5:11:23:e8:aa:e1:91:d0:5c:ff:71:3b:fb:e2:4b:f4:12:5e:d5#SHA_256
2020-12-31T18:23:49.162-02:00 WARN (18F8-1A6C) <DesktopControlJMS> [JMSMessageSecurity] Failed to sign message: Cannot sign message
2020-12-31T18:23:49.162-02:00 DEBUG (18F8-1A6C) <DesktopControlJMS> [DesktopTracker] CHANGEKEY message from agent/bda3fbe6-029c-41f8-b9f8-017af574f56b result: true (success)
Excerpt from VM agent log:
2020-12-31T19:53:44.322-03:00 DEBUG (1EDC-0FA8) <Thread-4> [AgentJmsConfig] Using paired signing key
2020-12-31T19:53:44.322-03:00 DEBUG (1EDC-0FA8) <Thread-4> [AgentMessageSecurityHandler] Configuring message security (ENHANCED).
2020-12-31T19:53:44.369-03:00 DEBUG (1EDC-0FA8) <Thread-4> [BrokerUpdateUtility] Published CHANGEKEY request
2020-12-31T19:53:59.386-03:00 DEBUG (1EDC-0FA8) <Thread-4> [BrokerUpdateUtility] Timeout waiting for success response
2020-12-31T19:59:33.944-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] Using connection broker viewcs01.DOMAIN.net
2020-12-31T19:59:33.944-03:00 DEBUG (1430-2494) <MessageFrameWorkDispatch> [MessageFrameWork] KeyVault service got operation=getEndEntityCertificates, ok=1, msecs=0
2020-12-31T19:59:33.944-03:00 DEBUG (1430-2494) <MessageFrameWorkDispatch> [MessageFrameWork] KeyVault service got operation=getEndEntityCertificates, ok=1, msecs=0
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] username for swiftmq connection is: agent/90916ab8-704c-4fe3-a605-c4a7745b246e
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [AgentJmsConfig] Skipping pair operation: already paired
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [AgentMessageSecurityHandler] Configuring message security (ENHANCED).
2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] Re-connecting using secure port 4002
2020-12-31T19:59:34.381-03:00 DEBUG (1430-2780) <SwiftMQ-ConnectorPool-2> [AgentSSLSocketFactory] Received cert with subject cn=router/viewcs01
2020-12-31T19:59:34.381-03:00 WARN (1430-2780) <SwiftMQ-ConnectorPool-2> [AgentSSLSocketFactory] Certificate thumbprint verification failed, no matching thumbprint. Presented identity: router/viewcs01
2020-12-31T19:59:34.381-03:00 DEBUG (1430-2558) <Thread-4> [JmsManager] Unable to connect to JMS server viewcs01.DOMAIN.net com.vmware.vdi.logger.Logger.debug(Logger.java:44)
javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Unexpected certificate: router/viewcs01
2020-12-31T19:59:34.381-03:00 WARN (1430-2558) <Thread-4> [JmsManager] Unable to connect to any listed host. The agent will continue to retry: [viewcs02.DOMAIN.net, viewcs01.DOMAIN.net]
Reinstalled the agent and also tried the command below, as mentioned in https://kb.vmware.com/s/article/2038679, nothing has worked at all.
vdmadmin -A -d desktop-pool-name -m name-of-machine-in-pool -resetkey
Update to question:
Some piece (That I don't have now) of the log lead me to This KB, so I have uninstalled the Connection Server, remove all certificates and reinstalled it again, nothing changed.
After reading the following links [1], [2], [3]:
I've changed the security mode to Mixed, nothing changed.
But after change it to mixed and after reinstalling the Agent (I've reinstalled it before change to mixed and it didn't worked) from the VM have turned it to Available, I still not able to access machine with tunnel errors (Changed tunnel configurations also).
Updated the Connection Server to 7.13, stopped to open console.
Started the whole process from zero, now machine name is viewcs04, not worked also.
For any who encounter the same problem, I decided to create a new Connection Server and I will create manual pools, so people can work and I will migrate everyone to new linked-clone pools.
Just to mention, I cannot just create new pools, all pools are dedicated and have manually installed applications, printers etc.
I have been following the documentation in every step, and I didn't face any errors. Configured, deployed and made a subscription to hello/world topic just as the documentation detailed. However, when I arrived at the testing step here: https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-check.html
No messages were showing up on the IoT console (subscription view hello/world)! I am using Greengrass core daemon which runs on my Ubuntu machine, it is active and listens to port 8000. I don't think there is anything wrong with my local device because the group was deployed successfully and because I see the communications going both ways on Wireshark.
I have these logs on my machine: /home/##/Desktop/greengrass/ggc/var/log/system/runtime.log:
[2019-09-28T06:57:42.492-07:00][INFO]-===========================================
[2019-09-28T06:57:42.492-07:00][INFO]-Greengrass Version: 1.9.3-RC3
[2019-09-28T06:57:42.492-07:00][INFO]-Greengrass Root: /home/##/Desktop/greengrass
[2019-09-28T06:57:42.492-07:00][INFO]-Greengrass Write Directory: /home/##/Desktop/greengrass/ggc
[2019-09-28T06:57:42.492-07:00][INFO]-Group File Directory: /home/##/Desktop/greengrass/ggc/deployment/group
[2019-09-28T06:57:42.492-07:00][INFO]-Default Lambda UID: 122
[2019-09-28T06:57:42.492-07:00][INFO]-Default Lambda GID: 127
[2019-09-28T06:57:42.492-07:00][INFO]-===========================================
[2019-09-28T06:57:42.492-07:00][INFO]-The current core is using the AWS IoT certificates with fingerprint. {"fingerprint": "90##4d"}
[2019-09-28T06:57:42.492-07:00][INFO]-Will persist worker process info. {"dir": "/home/##/Desktop/greengrass/ggc/ggc/core/var/worker/processes"}
[2019-09-28T06:57:42.493-07:00][INFO]-Will persist worker process info. {"dir": "/home/##/Desktop/greengrass/ggc/ggc/core/var/worker/processes"}
[2019-09-28T06:57:42.494-07:00][INFO]-No proxy URL found.
[2019-09-28T06:57:42.495-07:00][INFO]-Started Deployment Agent to listen for updates. [2019-09-28T06:57:42.495-07:00][INFO]-Connecting with MQTT. {"endpoint": "a6##ws-ats.iot.us-east-2.amazonaws.com:8883", "clientId": "simulators_gg_Core"}
[2019-09-28T06:57:42.497-07:00][INFO]-The current core is using the AWS IoT certificates with fingerprint. {"fingerprint": "90##4d"}
[2019-09-28T06:57:42.685-07:00][INFO]-MQTT connection successful. {"attemptId": "GVko", "clientId": "simulators_gg_Core"}
[2019-09-28T06:57:42.685-07:00][INFO]-MQTT connection established. {"endpoint": "a6##ws-ats.iot.us-east-2.amazonaws.com:8883", "clientId": "simulators_gg_Core"}
[2019-09-28T06:57:42.685-07:00][INFO]-MQTT connection connected. Start subscribing. {"clientId": "simulators_gg_Core"}
[2019-09-28T06:57:42.685-07:00][INFO]-Deployment agent connected to cloud.
[2019-09-28T06:57:42.685-07:00][INFO]-Start subscribing. {"numOfTopics": 2, "clientId": "simulators_gg_Core"}
[2019-09-28T06:57:42.685-07:00][INFO]-Trying to subscribe to topic $aws/things/simulators_gg_Core-gda/shadow/update/delta
[2019-09-28T06:57:42.727-07:00][INFO]-Trying to subscribe to topic $aws/things/simulators_gg_Core-gda/shadow/get/accepted
[2019-09-28T06:57:42.814-07:00][INFO]-All topics subscribed. {"clientId": "simulators_gg_Core"}
[2019-09-28T06:58:57.888-07:00][INFO]-Daemon received signal: terminated. [2019-09-28T06:58:57.888-07:00][INFO]-Shutting down daemon.
[2019-09-28T06:58:57.888-07:00][INFO]-Stopping all workers.
[2019-09-28T06:58:57.888-07:00][INFO]-Lifecycle manager is stopped.
[2019-09-28T06:58:57.888-07:00][INFO]-IPC server stopped.
/home/##/Desktop/greengrass/ggc/var/log/system/localwatch/localwatch.log:
[2019-09-28T06:57:42.491-07:00][DEBUG]-will keep the log files for the following lambdas {"readingPath": "/home/##/Desktop/greengrass/ggc/var/log/user", "lambdas": "map[]"}
[2019-09-28T06:57:42.492-07:00][WARN]-failed to list the user log directory {"path": "/home/##/Desktop/greengrass/ggc/var/log/user"}
Thanks in advance.
I had a similar issue on another platform (Jetson Nano). I could not get a response after going through the AWS instructions for setting up a simple Lambda using IOT Greengrass. In my search for answers I discovered that AWS has a qualification test script for any device you connect.
It goes through an automated process of deploying and testing a lambda function(as well as other functionality) and reports results for each step and docs provide troubleshooting info for failures.
By going through those tests I was able to narrow down the issues with my setup, installation, and configuration. The testing docs give pointers to troubleshoot test results. Here is a link to the test: https://docs.aws.amazon.com/greengrass/latest/developerguide/device-tester-for-greengrass-ug.html
If you follow the 'Next Topic' links, it will take you through the complete test. Let me warn you that its extensive, and will take some time, but for me it gave a lot of detailed insight that a hello world does not.
when I ran the workflow manager getting the error message at add host to service bus farm.
We have the SharePoint as standalone, OS is Windows server 2012 r2
SQL server 2016 developer.
Followed below two url's for installing
https://collab365.community/configuring-sharepoint-2013-to-support-workflow-management/
https://www.c-sharpcorner.com/article/workflow-manager-configuration-for-sharepoint-server-2013/ unable to under stand the issue where exactly.
please find the below log file
[Verbose] [12/10/2018 4:43:54 PM]: Service Bus services starting.
[Progress] [12/10/2018 4:43:54 PM]: Service Bus services starting.
[Error] [12/10/2018 4:53:55 PM]: System.Management.Automation.CmdletInvocationException: Starting service Service Bus Message Broker failed: Time out has expired and the operation has not been completed. ---> Microsoft.ServiceBus.Commands.Common.Exceptions.OperationFailedException: Starting service Service Bus Message Broker failed: Time out has expired and the operation has not been completed.
at Microsoft.ServiceBus.Commands.Common.SCMHelper.StartService(String serviceName, Nullable1 waitTimeout, String hostName)
at Microsoft.ServiceBus.Commands.ServiceBusConfigHelper.StartSBServices(String hostName, Nullable1 waitTimeout)
at Microsoft.ServiceBus.Commands.AddSBHost.ProcessRecordImplementation()
--- End of inner exception stack trace ---
at System.Management.Automation.Runspaces.AsyncResult.EndInvoke()
at System.Management.Automation.PowerShell.EndInvoke(IAsyncResult asyncResult)
at Microsoft.Workflow.Deployment.ConfigWizard.CommandletHelper.InvokePowershell(Command command, Action`3 updateProgress)
at Microsoft.Workflow.Deployment.ConfigWizard.ProgressPageViewModel.AddSBNode(FarmCreationModel model, Boolean isFirstCommand)
please let me know how to resolve this issue for installing the workflowmanager.
what worked for me was enabling TLS 1.0 in the registry.
in my case I don't have registry of client but only enabled the server one
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Client]
"Enabled"=dword:00000001
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Server]
"Enabled"=dword:00000001
fyi... I was stopped the Service Bus Message Broker while the workflow manager configuration wizard was running in the "add host to service bus fam" task, then the changes the wizard complete successfully. I hope so much you can resolve this issue :)
this is the link where I fund the answers http://answersweb.azurewebsites.net/MVC/Post/Thread/e6667e72-36db-44d7-bcb9-0d537cd19542?category=workflow and is the CRBenson post, thank you very much
I had almost same issue. Installing the correct patch fixed the issue.
Complete details on below thread.
http://fixingsharepoint.blogspot.com/2021/02/service-bus-gateway-service-stuck-at.html
Test cluster of two brokers, WKA membership scheme, PostgreSQL message store, working fine for a couple of days, then throwing following errors:
TID: [] [] [2016-07-19 12:09:24,738] ERROR {org.wso2.andes.server.protocol.MultiVersionProtocolEngine} - Error establishing session {org.wso2.andes.server.protocol.MultiVersionProtocolEngine}
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at org.apache.mina.transport.socket.nio.SocketIoProcessor.read(SocketIoProcessor.java:218)
at org.apache.mina.transport.socket.nio.SocketIoProcessor.process(SocketIoProcessor.java:198)
at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$400(SocketIoProcessor.java:45)
at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:485)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Thread.java:745)
Startup of Message Broker looks fine, no errors, JDBC connection to PostgreSQL DB is ok, Registry mount looks ok. Then after that error appears in wso2carbon.log several times/minute.
Anyone any ideas? As far as I know nothing's changed and I don't know what it's trying to connect to.
This usually happens when client's whom connected to MB tries to create connections per message. jms is heavy connection and not recommended to create connections per each message. Therefore, please go through client implementation and verify connections are not created per message.
If by any chance you are using wso2 esb to publish/subscribe queues/topics to mb there is a property "transport.jms.CacheLevel" connection caching in esb axis2.xml.Read the documentation and use appropriate caching level for your usecase.
There was bug in connection caching property to be ignored in esb 4.8.1 which is currently fixed in 4.9.0 as well.
These are the possible cases I can think of with the given information. If you need more info please provide a detailed usecase.