I have a WebApp with a WebJob (monitoring a queue with QueueTrigger) in Azure that has been working fine for over a year. I'm trying to re-organize some of my Azure resources and would like the WebJob to monitor a different Queue than it has been.
I've created a new (non-classic) storage account and changed my code to insert new messages into the new Queue. I can see new messages showing up in the new Queue but the WebJob is never triggered. (The old queue is in a classic storage account. I don't think that matters, but it has crossed my mind. I have seen some older posts that make me think this used to be a problem, but some newer ones that make me think it's OK.)
My code is pretty straightforward (almost straight out of tutorial). It wants the Queue connection strings in app.config for both AzureWebJobsDashboard and AzureWebJobsStorage, which I have done.
var host = new JobHost();
host.RunAndBlock();
To verify that the new code is being successfully deployed, I have deleted the old WebJob in the Azure portal and verified the file dates in the App_Data\Jobs... folder are current and I've looked at the value of "AzureWebJobsStorage" in the deployed config file and it is the new Queue's connection string.
I finally thought to manually insert a message into the old queue (that none of my code is pointing at any more). Sure enough - when I do that, the WebJob is triggered and runs.
I think changing the connection string values in App.Config should be all that's needed to have it "watch" a new queue, but that doesn't seem to be enough. Does anyone know what else would need to be changed?
So I figured this out. Not only do you have to have the Connection String entries (pointing to the queue) in the App.Config of the WebJob project, but also for the Web Application. I had configured AzureWebJobsDashboard and AzureWebJobsStorage in the Application Settings section of the Web App in the Azure portal.
Those were still pointing to the old queue, and apparently those were the important ones. Once I updated those settings to point to the new queue, it worked as it should.
Related
Below is the program that I am using to publish a message to SNS. There is no issue when I execute this program(simple Java or Spring boot) in my local system. The message is published on the topic.
But, the actual program runs inside a thread in a spring boot project. The thread runs every 1 minute. Whenever the thread runs, the below program has to publish the message. This spring boot project deployed inside a standalone Unix server. From the server, it is not publishing messages. There is no exception. The logs are showing a successful handshake to the Amazon network.
I tried AmazonSNS and SnsClient classes as well.
AmazonSNSAsync snsClient1 = AmazonSNSAsyncClient.asyncBuilder()
.withCredentials(new AWSStaticCredentialsProvider(
new BasicAWSCredentials("access-key",
"secret-key")))
.withRegion("us-east-1")
.build();
log.info("STATUS: snsClient1 formed");
PublishRequest publishReq =
new PublishRequest()
.withTopicArn("sns_topic-arn")
.withMessage("mes: "+message);
log.info("STATUS: publishReq formed");
snsClient1.publishAsync(publishReq);
I tried the below things already.
moved the program out of the thread
tried with a new topic
used the latest dependency of SNS
However, when I published a message from the Unix box using AWS CLI, the message was published successfully.
Did anyone face a similar issue? If not, you can share any alternative way to hit the SNS from a Java program if you know.
The issue is partially fixed. AWS SDK was throwing an error, not an exception. That was the cause of the code block. Once, I catch with the "Throwable" keyword, the program is not stopping.
Later, I used the bom concepts to avoid the transitive dependency version conflicts and converted all the AWS SDK 1 code into SDK 2 code. Now, the message is posted on the SNS topic. But, the weird thing is that SDK2 also throws a similar error. I am trying to fix it.
Reference:
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/javav2/example_code/sns/src/main/java/com/example/sns/PublishTopic.java
Meanwhile, you can track the issue on the below page as well.
https://github.com/aws/aws-sdk-java/discussions/2867
I disabled/deleted a GOCD go-agent. It seems I can no longer attempt to register this go-agent, as it doesn't appear in the agents list on service start.
Is this agent getting blacklisted somewhere since it was deleted? Just tried to disable/delete another go-agent that was working just fine, and it also no longer appears in the list on service start.
I checked the cruise-config.xml, with no mentioned of the deleted agent. The UI doesn't show deleted agents either.
If you disable an agent on in the GoCD's server agent administration page, it is shown grayed out at the end of the list. You can re-enable it.
If you delete and agent, GoCD truly forgets about it.
If you then, on an agent machine, start an agent that you had previously deleted, it'll show up in the agent list again in state Pending, and you have to enable it.
If the agent doesn't show up, it's worth checking the agent's log (/var/log/go-agent/*.log) and possibly the server's log if there are any cues about why the agent doesn't register.
Finally, you can try to delete /var/lib/go-agent/config/ (make a backup first) on the agent. Restarting the agent then allocates a new agent UUID, so the GoCD server will see it as a new agent.
Just wanted to know from a high level how I would accomplish this.
I thought that when a user opens the application, I will keep track of the last opened time in a Dynamo DB table.
Then I could have a background worker constantly check and see if anybody hasn't used their app in 3 or 4 days and then send a push notification, ie, "you haven't used your app in a while, why don't you open it up and do XYZ."
From a very high level, there are two possible ways:
1.) Local notifications (you don't need AWS for this):
You can schedule a local notification, every time the user opens up the app (or better - every time the user brings the app to foreground). It works like: User opens app -> cancel old scheduled notification if existing -> schedule new notification for "in 3 or 4 days" -> ready :-)
You can use something like this: https://github.com/zo0r/react-native-push-notification (see section Sheduled Notifications).
2.) You could do it with remote notifications (https://aws.amazon.com/sns/):
You can go the way you proposed. Then you have to store an entry in your db with the push notification token of the device and the last time the app was opened. Your worker then has to check and send the push message to the device using a service like SNS.
I would recommend 1.) over 2.) because you are independent from the users internet connection when getting the app opening info. In 2.) you can miss the opening info, when the user opens the app without internet connection. Also 2.) is more expensive then 1.) when you scale your app.
An advantage of 2.) would be, that you are more flexible when and what you send in your notification, since you can edit it on server side. 1.) would mean that it is coded in your app (at least until you build a synchronization mechanism for the variables) :-)
The scenario here is that we have a service bus queue and a web job. The web job reads the message from the service bus queue and calls a logic up which then goes on and does other stuff.
The problem we are facing is that after the web job reads the message from the service bus, it occasionally doesn't delete it after, which constantly causes the logic app to be called and flood our database with data.
Here is the message in question as seen from azure management studio:
https://gyazo.com/7f57b460421d1bb4a69fcb8b5a9ff01f
As you can see, there is no lock time on the message. I have tried to play around with the settings to no avail.
When i manually try to delete that message from azure management studio it is also unsuccessful but there is no error message received.
Does anyone know what is going on here? I feel like this is a problem with the queue itself as opposed to a bug in our code since 2-3 tools that i have used are unable to delete this message from the queue.
It looks like the message is only deleted after a specific time (does not go to the dead-letter queue however).
Thanks
So just for information, i figured my own issue out. When the file scraper job runs, it puts a message in the service bus. The webjob now that runs and picks up that file stores the file that it just picked up locally as well as on blob storage.
The problem was that webjob keeps a queue of what it processes locally which was never cleared so every time the webjob run, it was processing all previous files as well.
What does "Pending Restart" mean? I have stopped and restarted my WebJob numerous times and that doesn't seem to fix it. Does it mean I have to restart my website? What caused my job to get in this state in the first place? Is there any way I can prevent this from happening in the future?
Usually, it means that the job fails to start (an exception?). Look in the jobs dashboard for logs.
Also, make sure that if the job is continuous, you actually have an infinite loop that keeps the process alive.
To add to Victor's answer, the continuous WebJob states are:
Initializing - The site was just started and the WebJob is doing it's initialization process.
Starting - The WebJob is starting up the process/script.
Running - The WebJob's process is running.
PendingRestart - The WebJob's process exited (for any good or bad reason) in less than 2 minutes since it started, for a continuous WebJob it's considered that something was probably not right with it (some exception during start-up probably as mentioned by Victor), at this point the system is waiting for 60 seconds before it'll restart the WebJob process (hence the name "pending restart").
Stopped - The WebJob was stopped (usually from the Azure portal) and is currently not running and will not be running until it is started again, best way to see this is as disabled.
Also, take a look at the webjob log, it should hold cue to what's been happening.
if the JOB is set to run continuously, once the process exits (say you are polling a queue and it's empty) the job shuts down and status changes to "pending restart". Azure Scheduler will typically restart the process in 60 seconds.
Try changing the target framework to .NET 4.5. This same issue was fixed for me when I changed the target framework from 4.6.1 to 4.5.
Had the same problem, found out that i need to keep my webjob alive, so I put a continous loop to keep it alive.
It means that application is failing after start. Check the App Service Application setting might have some problem. .. In my case i am passing date as a configuration and i entered wrong date like 20160431
This is just because Webjobs is failing or giving exception.
Make sure the Webjobs is continuous and you can check that in the log where it failing and can make the changes.
Process went down, waiting for 60 seconds
Status changed to Pending Restart
In my case, we got the deployment package prepared through the Visual Studio folder publish and deployed along with WebApp. Package created through 'Folder publish' lacked 'run.cmd' file in which command to invoke the console application (.exe) is available; this file is automatically created when we directly publish to Azure WebJob from Visual Studio.
After manually adding this to a package folder, the issue got fixed.