the azure webjob with runmode set to "onDemand" keeps running and I am not able to stop it.
I don't see anything that needs to be handled but the job.
{
"$schema": "http://schemastore.org/schemas/json/webjob-publish-settings.json",
"webJobName": "ScheduledJob",
"runMode": "OnDemand"
}
ScheduledJob Triggered Running n/a
the only way to restarted is by restarting the web service. Then start the job manually. And then it keeps running. It does not stop.
What is going on with this webjob?
Update1:
I am using the code from Pnp Partner package which can be found here.
As the code is two long I am just providing the code in the program.cs file.
For the rest please have a look at the I posted above.
static void Main()
{
var job = new PnPPartnerPackProvisioningJob();
job.UseThreading = false;
job.AddSite(PnPPartnerPackSettings.InfrastructureSiteUrl);
job.UseAzureADAppOnlyAuthentication(
PnPPartnerPackSettings.ClientId,
PnPPartnerPackSettings.Tenant,
PnPPartnerPackSettings.AppOnlyCertificate);
job.Run();
#if DEBUG
Console.ReadLine();
#endif
}
In your code, the PnPPartnerPackProvisioningJob class is inheritted from TimerJob class.
In TimerJob class, there is not a stop method. And if timer job has started executing, you can not really stop it unless you restart web jobs. For more details, you could refer to this article.
So if your requirement is to cancel a job, you will need to delete the timer job definition. However if timer job has started executing, you can not really STOP it unless you reset IIS or stop Sharepoint Windows Timer Service.
Related
I have the following WebJob project where I'm trying to deploy a TimerTrigger WebJob function, however I cannot get it to run on a scheduled basis when deploying it via "Publish As Azure WebJob..." in Visual Studio 2017.
Program.cs
class Program
{
static void Main()
{
var config = new JobHostConfiguration();
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
}
config.UseTimers();
var host = new JobHost(config);
host.RunAndBlock();
}
}
Functions.cs
public class Functions
{
public static async Task ProcessAsync([TimerTrigger("0 */3 * * * *")] TimerInfo timerInfo, TextWriter log)
{
...
}
}
webjob-publish-settings.json
{
"$schema": "http://schemastore.org/schemas/json/webjob-publish-settings.json",
"webJobName": "TestWebJob",
"runMode": "OnDemand"
}
Settings.job
{ "schedule": "0 */3 * * * *" }
The documentation for this is pretty non-existent, and it's baffling to why Azure supports Scheduled CRON TimerTrigger's but doesn't actually include them as an option when deploying.
Is this possible?
If you have created a schedule webjob manually, I think you probably have found it will generate a settings.job to set the schedule.Then the SCHEDULE in the portal read the schedule and show it. And if you deploy a TimerTrigger webjob with VS2017, it won't generate this file because you have define the TimerTrigger function.
Then I did some tests to show it.Firstly I create a webjob with TimerTrigger and deploy it, it will show same result just like yours with n/a SCHEDULE. Then I kill the webjob process and upload a settings.job then refresh(not the refresh in in the portal) the page, then the SCHEDULE change to CRON expression. And if you delete the file, it will change back.
As for the log, in my opinion it's also caused by the settings.job, if you have this file it will trigger this webjob every x minutes, and if you don't have it will trigger the function every x minutes in a webjob.
If you still have questions, please let me know.
It seems that the above code is working. However, it runs completely differently to how you would imagine if you're familiar with running "Scheduled" WebJobs manually.
If you were to run them manually, you would usually see the Schedule at the top level, along with the Status updating every x minutes, etc:
and you would also see the logs update at parent level, like so:
However, when deploying it using the above method via Visual Studio 2017, you only ever get the WebJob running once for the duration of it's lifetime. As a result you would only ever get one parent log in the logs list too.
Though if you click into this, you will see individual logs for each scheduled function log:
Hopefully this will make sense for other people who are looking into setting up WebJobs :)
If I apply [Singleton] and [NoAutomaticTrigger] attributes and publish the webjob, it goes to pending restart state.
We want to solve multiple instance issue which is occurring in a method.
Please help.
it goes to pending restart state.
In your case, you need to check the reason why webjob goes to pending restart state.
There are lots of reasons that goes to pending restart state. maybe due to an issues or webjob thread is finished needs to restart. We could check it with Webjob log.
Before publish it to azure, we make sure that it works correctly locally and add the appsetting AzureWebJobsDashboard and AzureWebJobsStorage with storage connection string then we could get the webjob log from webjob dashboard.
If you publish it as continuous type webjob, and method is executed completely. And the status will become to pending restart. It is a normal behavior.
[Singleton] and [NoAutomaticTrigger] attributes could work correctly, please refer to the following demo code.
static void Main()
{
JobHost host = new JobHost();
host.Call(typeof(Functions).GetMethod("CreateQueueMessage"), new { value = "Hello world!" + Guid.NewGuid() });
}
[Singleton]
[NoAutomaticTrigger]
public static void CreateQueueMessage(TextWriter logger,string value,[Queue("outputqueue")] out string message)
{
message = value;
logger.WriteLine("Creating queue message: ", message);
Console.WriteLine(message);
}
I registered my task app in Spring Cloud Data Flow, created a definition for it and the status shows 'unknown'. I created the stream and trying to launch the task through task-sink and I get an error:
java.lang.IllegalStateException: failed to resolve MavenResource:
How to launch a task from the task-sink? Am I missing something? Any help is appreciated. Another question I have is how do I access the payload sent via TaskLaunchRequest in my task?
S1 http | step1: transformer-rabbit | log
S2 :S1.step1 > filter --expression=payload.contains('CUSTADDRMODRQ_V15') | task-processor | task-sink
task-sink is launching the task provided by the uri in the TaskLaunchRequest. It is looking for the resource as shown in the log
OUT Using manager EnhancedLocalRepositoryManager with priority 10.0 for /home/vcap/.m2/repository
OUT Using transporter HttpTransporter with priority 5.0 for https://repo.spring.io/libs-snapshot and finally failing.
The task is deployed in our repository and as mentioned I registered and created the definition for it as well.
This one is in cf environment and I am using SCDF server 1.0.0.M4.
In the application.properties for the task-sink i am providing maven.remote.repositories.snapshots.url=**
task create fis-ifx-event-task --definition "fis-event-task"
My goal is launching the task from the stream.
Thanks for the information. I am in fact using the BUILD-SNAPSHOT as I am unable to enable taks in 1.0.0M4 version. Here is the one I am using spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-20160808.144306-116. I am able to register and create task definitions. The status of the task definition is showing as 'unknown' even when I am using the sample task module provided by your team. But when I initiate the flow of the stream and when task-sink tries to launch the task, it is unable to find the maven resource. When I create the task definition, does the task module gets deployed? I don't see any app in Pivotal Apps Manager. As mentioned earlier, I provided maven.remote.repositories.snapshot.url in the application.properties file for the task-sink application. Another thing I observed is when I launch the task manually from dataflow shell it gives an error CF-UnprocessableEntity(10008): The request is semantically invalid: Unknown field(s): 'staging_disk_in_mb', 'staging_memory_in_mb' and also a message saying 'Source is empty'. Presently the task is supposed to print the timestamp and is not dependent on any input.
TaskProcessor code:
#EnableBinding(Processor.class)
#EnableConfigurationProperties(TaskProcessorProperties.class)
public class TaskProcessor {
#Autowired
private TaskProcessorProperties processorProperties;
public TaskProcessor() {
}
#Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
#ELI(level = "info", eventType = ELIEventType.INBOUND)
public Object setupRequest(String message) {
Map<String, String> properties = new HashMap<String, String>();
properties.put("payload", message);
TaskLaunchRequest request = new TaskLaunchRequest(processorProperties.getUri(), null, properties, null);
return new GenericMessage<>(request);
}
}
TaskSink code:
#SpringBootApplication
#EnableTaskLauncher
#EnableBinding(Sink.class)
#EnableConfigurationProperties(TaskSinkProperties.class)
public class FisIfxEventTaskSinkApplication {
public static void main(String[] args) {
SpringApplication.run(FisIfxEventTaskSinkApplication.class, args);
}
}
I provided the stream I am using earlier in the post. Sink is receiving the TaskLaunchRequest with uri and payload as you can see here and unable to launch the task.
OUT registering [40, java.io.File] with serializer org.springframework.integration.codec.kryo.FileSerializer
2016-08-10T16:08:55.02-0600 [APP/0]
OUT Launching Task for the following resource TaskLaunchRequest{uri='maven://com.xxx:fis.ifx.event-task:jar:1.0-SNAPSHOT', commandlineArguments=[], environmentProperties={payload={"statusCode":0,"fisT
opic":"CustomerDataUpdated","payloadId":"CUSTADDRMODR``Q_V15","customerIds":[1597304]}}, deploymentProperties={}}
Before I begin, you have a number of questions here. In the future, it's better to break them up into multiple questions so that they are easier to find by other users and easier to answer. That being said:
A little context on the current state of things
In order to understand how things will work, it's important to understand the current state of things. The current releases of the software involved are:
Pivotal Cloud Foundry (PCF) - 1.7.12. This version is required for any task support.
Spring Cloud Task (SCT) - 1.0.2.RELEASE
Spring Cloud Data Flow CF (SCDF) - 1.0.0.BUILD-SNAPSHOT (current as of the date of this post).
Currently PCF 1.7.12+ has all the capabilities to run tasks. You can create v3 applications (the type of application used to launch a task), run it as a task, etc. However, the tooling around that functionality is not currently complete. There is no support for v3 applications in Apps Manager or the CLI. There is a plugin for the CLI that is more of a dev tool that can be used to help with some functions (it will show you logs, etc), but it is not fully functional and requires a specific version of the CLI to work [1]. This is one of the reasons that the task functionality within PCF is still considered experimental.
Spring Cloud Task is currently GA and supports all the functionality needed to effectively run tasks on CF. However, it's important to note that SCT doesn't handle orchestration so the actual launching of tasks on CF is the responsibility of either the user, or Spring Cloud Data Flow (the easier route).
Spring Cloud Data Flow's Cloud Foundry server implementation currently has functionality to launch tasks on PCF in the latest snapshots. We have validated this against 1.7.12 as well as the development branch of 1.8.
The task workflow within SCDF
Tasks are fundamentally different from stream applications within the context of SCDF. When you create a stream definition, you are given the option to deploy it. What this does is it actually downloads the Spring Boot über jars and deploys them to PCF as long running processes. If they go down, PCF, will relaunch them as expected, etc.
Tasks on the other hand, are not deployed. They are launched. The difference is that while you create a task definition, there is nothing deployed until you click launch. And when the task completes, the software is shut down and cleaned up. So while a stream definition may have states, it's really a one to one relationship between the definition and the deployed software. Where with a task, you can launch a task definition as many times as you want.
Your issues
Reading through your post, I see a few things that you are struggling with. Let me see if I can help:
Task Definitions within SCDF and launching them via a stream - When launching a task from a stream, the task registry within SCDF is not used. The sink expects the URL for the resource to be within the TaskLauchRequest.
Apps Manager and tasks - As mentioned above, there is no support for v3 applications in Apps Manager yet so you won't be able to see your tasks there.
Viewing the logs - In order to debug what's going wrong with launching your task on CF, you're going to want to view the logs. To do so, use the v3 CLI plugin mentioned above to view them. It's important to note that you can only tail live logs with the plugin, not view logs that have previously been rendered. Because of that, when testing, you'll want to tail the logs as soon as the app is created, before it's launched.
Error in SCDF Shell - The error you received from the SCDF shell (CF-UnprocessableEntity(10008):...) leads me to wonder if you have both the correct version of PCF (1.7.12+) and the correct version of the following other libraries:
spring-cloud-deployer-cloudfoundry - The latest snapshots
cf-java-client - 2.0.0.M10+
reactor-core - 3.0.0.RC1+
I hope this helps!
[1] https://github.com/cloudfoundry/v3-cli-plugin
Task support is not available in 1.0.0.M4 release of SCDF's CF-server. In this release, the task commands/REST-APIs should be disabled - see here. And for that reason, you wouldn't see any docs related to Tasks in the 1.0.0.M4 reference guide.
That said, the Task support is available/enabled in the BUILD-SNAPSHOT release. If you're locally building the CF-server and upon pushing it to CF, you could take advantage the task commands in the shell to create and launch task definitions.
I'm developing a web application based on the Yii2 framework.
Every 12 hours the application needs to run a batch process to update some DB tables and it takes 5-10 seconds. In order to do that, I created a console command (say ./yii dummy/index) that is called by the windows task scheduler using a .bat script. However, while running the task, the application Web GUI is still running. Does Yii2 autonomously stops any web interaction while executing the task or should I lock manually the system to avoid any inconsistency issues? If is that so, how can I stop web interaction while executing the console process and restart them when it's completed?
Console and web apps are completely autonomous. Running a console command will in no way prevent the web app from running.
The easiest way to solve this problem would be to set some kind of mutex in your console command and check for it in your web application. For that, Yii2 offers a variety of mutex classes all derived from yii\mutex\Mutex.
You add a mutex to your console and web app configs:
'mutex' => [
'class' => 'yii\mutex\MysqlMutex',
],
In your console command you need to acquire the mutex (allow it to wait for a few seconds):
$mutexResult = Yii::$app->mutex->acquire('example-mutex', 10);
if ($mutexResult) {
echo 'Could not acquire lock'.PHP_EOL;
Yii::$app->end();
}
And when you're done, release it:
Yii::$app->mutex->release('example-mutex');
In your web application you then check if the mutex is available. I believe current application template will generate controllers that extend the yii\web\Controller class. You should create your own base controller class that extends yii\web\Controller and define its beforeAction method:
class BaseController extends \yii\web\Controller
{
public function beforeAction($action)
{
$mutexResult = Yii::$app->mutex->acquire('example-mutex', 20);
if ($mutexResult) {
Yii::$app->mutex->release('example-mutex');
} else {
echo 'Console app is running';
return false;
}
return parent::beforeAction($action);
}
}
Then extend all your web application's controllers from this BaseController class.
This way Yii will check for this flag before any action is executed.
What does "Pending Restart" mean? I have stopped and restarted my WebJob numerous times and that doesn't seem to fix it. Does it mean I have to restart my website? What caused my job to get in this state in the first place? Is there any way I can prevent this from happening in the future?
Usually, it means that the job fails to start (an exception?). Look in the jobs dashboard for logs.
Also, make sure that if the job is continuous, you actually have an infinite loop that keeps the process alive.
To add to Victor's answer, the continuous WebJob states are:
Initializing - The site was just started and the WebJob is doing it's initialization process.
Starting - The WebJob is starting up the process/script.
Running - The WebJob's process is running.
PendingRestart - The WebJob's process exited (for any good or bad reason) in less than 2 minutes since it started, for a continuous WebJob it's considered that something was probably not right with it (some exception during start-up probably as mentioned by Victor), at this point the system is waiting for 60 seconds before it'll restart the WebJob process (hence the name "pending restart").
Stopped - The WebJob was stopped (usually from the Azure portal) and is currently not running and will not be running until it is started again, best way to see this is as disabled.
Also, take a look at the webjob log, it should hold cue to what's been happening.
if the JOB is set to run continuously, once the process exits (say you are polling a queue and it's empty) the job shuts down and status changes to "pending restart". Azure Scheduler will typically restart the process in 60 seconds.
Try changing the target framework to .NET 4.5. This same issue was fixed for me when I changed the target framework from 4.6.1 to 4.5.
Had the same problem, found out that i need to keep my webjob alive, so I put a continous loop to keep it alive.
It means that application is failing after start. Check the App Service Application setting might have some problem. .. In my case i am passing date as a configuration and i entered wrong date like 20160431
This is just because Webjobs is failing or giving exception.
Make sure the Webjobs is continuous and you can check that in the log where it failing and can make the changes.
Process went down, waiting for 60 seconds
Status changed to Pending Restart
In my case, we got the deployment package prepared through the Visual Studio folder publish and deployed along with WebApp. Package created through 'Folder publish' lacked 'run.cmd' file in which command to invoke the console application (.exe) is available; this file is automatically created when we directly publish to Azure WebJob from Visual Studio.
After manually adding this to a package folder, the issue got fixed.