Calling Webjob from Azure Data Factory pipeline throwing HTTP 409 conflict exception error - azure-webjobs

Question

I have a OnDemand triggered webjob that I want to trigger through ADF copy activity using HTTP linked service. Here is the linked service:-
{
"name": "LS_WebJob",
"properties": {
"hubName": "yas-cdp-adf_hub",
"type": "Http",
"typeProperties": {
"url": "https://cust-app.scm.azurewebsites.net/api/triggeredwebjobs/ConsoleApplication1/run",
"authenticationType": "Basic",
"username": "$custdata-app",
"password": "**********"
}
}
}
Input Dataset
{
"name": "ZZ_Inp_Webjob",
"properties": {
"published": false,
"type": "Http",
"linkedServiceName": "LS_WebJob",
"typeProperties": {
"requestMethod": "Post",
"requestBody": "Hey Buddy"
},
"availability": {
"frequency": "Day",
"interval": 1,
"style": "StartOfInterval"
},
"external": true,
"policy": {}
}
}
Output Dataset
{
"name": "ZZ_Out_WebJob",
"properties": {
"published": false,
"type": "AzureBlob",
"linkedServiceName": "LS_ABLB",
"typeProperties": {
"fileName": "webjob.json",
"folderPath": "yc-cdp-container/Dummy/temp",
"format": {
"type": "TextFormat"
}
},
"availability": {
"frequency": "Day",
"interval": 1,
"style": "StartOfInterval"
}
}
}
Pipeline
{
"name": "ZZ-PL-WebJob",
"properties": {
"description": "This pipeline copies data from an HTTP Marina WiFi Source URL to Azure blob",
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "HttpSource"
},
"sink": {
"type": "BlobSink",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
}
},
"inputs": [
{
"name": "ZZ_Inp_Webjob"
}
],
"outputs": [
{
"name": "ZZ_Out_Webjob"
}
],
"policy": {
"timeout": "01:00:00",
"concurrency": 1
},
"scheduler": {
"frequency": "Day",
"interval": 1,
"style": "StartOfInterval"
},
"name": "WebjobSourceToAzureBlob",
"description": "Copy from an HTTP source to an Azure blob"
}
],
"start": "2017-04-10T01:00:00Z",
"end": "2017-04-10T01:00:00Z",
"isPaused": false,
"hubName": "yas-cdp-adf_hub",
"pipelineMode": "Scheduled"
}
}
My webjob is a simple C# application:-
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("My Team Rocks!");
}
}
}
When I am executing the pipeline, the webjob is being successfully triggered. However the pipeline fails with HTTP 409 conflict error.
Copy activity encountered a user error at Source side:
ErrorCode=UserErrorFailedToReadHttpFile,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed
to read data from http source
file.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The
remote server returned an error: (409) Conflict.,Source=System,'.

Answer 1

Try adding in the gateway name to the linked service json. Refer to this link How to integrate a WebJob within an Azure Data Factory Pipeline .