Naming Blob Dynamically for WebJob on a Schedule - azure-webjobs

I have a web job which is creating a blob based on the return value of a WebClient call. This is working fine. But as you can see from the Blob attribute (see code below), the name of the file is static. So, it is getting overwritten every time in blob storage.
Function class:
public class Functions
{
private static int _retryCount;
private static readonly int _retryLimit = int.Parse(ConfigurationManager.AppSettings["retryLimit"]);
private static readonly string _ghostRestfullUri = ConfigurationManager.AppSettings["ghostRestfullUri"];
[NoAutomaticTrigger]
public static void LightUpSite([Blob("ghost/response.json")] out string output, TextWriter logger)
{
_retryCount = 0;
output = string.Empty;
do
{
try
{
using (var request = new WebClient())
{
var response = request.DownloadString(_ghostRestfullUri);
_retryCount++;
output = response;
break;
}
}
catch(Exception exception)
{
logger.WriteLine("Job failed. Retry number:{0}", _retryCount);
}
} while (_retryCount < _retryLimit);
}
}
Main menu:
public class Program
{
static void Main()
{
var host = new JobHost();
host.Call(typeof(Functions).GetMethod("LightUpSite"));
}
}
How can I use placeholders to dynamically name the incoming file?
I have already tried the following:
ghost/{name}
ghost/{BlobName}
Other things to note:
This job is run on a schedule, so the host does not run and block
This job does not get invoked by a trigger, it just wakes up and runs;
Because the source is not coming from a message queue object or a uploaded file, I can’t figure out how I am supposed to name this blob.
Perhaps somehow using the blob storage API directly?

To name an output blob dynamically use IBinder as shown in this sample
To name an input blob dynamically as in a call from from Host.Call just pass the name of blob as argument:
static void Main()
{
var host = new JobHost();
host.Call(typeof(Functions).GetMethod("LightUpSite"), new {blobArgumentName= "container/blob"});
}

Related

Running BeamSql WithoutCoder or Making Coder Dynamic

I am reading data from file and converting it to BeamRecord But While i am Doing Query on that it Show Error-:
Exception in thread "main" java.lang.ClassCastException: org.apache.beam.sdk.coders.SerializableCoder cannot be cast to org.apache.beam.sdk.coders.BeamRecordCoder
at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.registerTables(BeamSql.java:173)
at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:153)
at org.apache.beam.sdk.extensions.sql.BeamSql$QueryTransform.expand(BeamSql.java:116)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:533)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:465)
at org.apache.beam.sdk.values.PCollectionTuple.apply(PCollectionTuple.java:160)
at TestingClass.main(TestingClass.java:75)
But When I am Providing it a Coder Then It Runs Perfectly.
I am little confused that if I am reading data from a file the file data schema changes on every run because I am using templates so is there any way I can use Default Coder or Without Coder, i can Run the Query.
For Reference Code is Below Please Check.
PCollection<String> ReadFile1 = PBegin.in(p).apply(TextIO.read().from("gs://Bucket_Name/FileName.csv"));
PCollection<BeamRecord> File1_BeamRecord = ReadFile1.apply(new StringToBeamRecord()).setCoder(new Temp().test().getRecordCoder());
PCollection<String> ReadFile2= p.apply(TextIO.read().from("gs://Bucket_Name/FileName.csv"));
PCollection<BeamRecord> File2_Beam_Record = ReadFile2.apply(new StringToBeamRecord()).setCoder(new Temp().test1().getRecordCoder());
new Temp().test1().getRecordCoder() --> Returning HardCoded BeamRecordCoder Values Which I need to fetch at runtime
Conversion From PColletion<String> to PCollection<TableRow> is Below-:
Public class StringToBeamRecord extends PTransform<PCollection<String>,PCollection<BeamRecord>> {
private static final Logger LOG = LoggerFactory.getLogger(StringToBeamRecord.class);
#Override
public PCollection<BeamRecord> expand(PCollection<String> arg0) {
return arg0.apply("Conversion",ParDo.of(new ConversionOfData()));
}
static class ConversionOfData extends DoFn<String,BeamRecord> implements Serializable{
#ProcessElement
public void processElement(ProcessContext c){
String Data = c.element().replaceAll(",,",",blank,");
String[] array = Data.split(",");
List<String> fieldNames = new ArrayList<>();
List<Integer> fieldTypes = new ArrayList<>();
List<Object> Data_Conversion = new ArrayList<>();
int Count = 0;
for(int i = 0 ; i < array.length;i++){
fieldNames.add(new String("R"+Count).toString());
Count++;
fieldTypes.add(Types.VARCHAR); //Using Schema I can Set it
Data_Conversion.add(array[i].toString());
}
LOG.info("The Size is : "+Data_Conversion.size());
BeamRecordSqlType type = BeamRecordSqlType.create(fieldNames, fieldTypes);
c.output(new BeamRecord(type,Data_Conversion));
}
}
}
Query is -:
PCollectionTuple test = PCollectionTuple.of(
new TupleTag<BeamRecord>("File1_BeamRecord"),File1_BeamRecord)
.and(new TupleTag<BeamRecord>("File2_BeamRecord"), File2_BeamRecord);
PCollection<BeamRecord> output = test.apply(BeamSql.queryMulti(
"Select * From File1_BeamRecord JOIN File2_BeamRecord "));
Is thier anyway i can make Coder Dynamic or I can Run Query with Default Coder.

Using Persistent Flash Message Library for ColdFusion

I am trying to use a library for showing Flash Messages https://github.com/elpete/flashmessage But I am having trouble getting it working correctly. The documentation isn't that great and I am new to ColdFusion. I want to have the ability to have persistent error messages across pages. Specifically during checkout so when the user needs to go back or a validation error occurs the message will appear. According to the documentation:
The FlashMessage.cfc needs three parameters to work:
A reference to your flash storage object. This object will need
get(key) and put(key, value) methods. A config object with the
following properties: A unique flashKey name to avoid naming
conflicts. A reference to your containerTemplatePath. This is the view
that surrounds each of the individual messages. It will have
references to a flashMessages array and your messageTemplatePath. A
reference to your messageTemplatePath. This is the view that
represents a single message in FlashMessage. It will have a reference
to a single flash message. The name is chosen by you in your container
template. Create your object with your two parameters and then use it
as normal.
I am getting the error
the function getMessages has an invalid return value , can't cast null value to value of type [array]
I had this script somewhat working at one point but it seems very finicky. I believe it is my implementation of it. I am hoping someone here can help me figure out where I went wrong. Or give me some pointers because I am not sure I am even implementing it correctly.
This is What I have in my testing script:
<cfscript>
alertStorage = createObject("component", 'alert');
config = {
flashKey = "myCustomFlashKey",
containerTemplatePath = "/flashmessage/views/_templates/FlashMessageContainer.cfm",
messageTemplatePath = "/flashmessage/views/_templates/FlashMessage.cfm"
};
flash = new flashmessage.models.FlashMessage(alertStorage, config);
flash.message('blah');
flash.danger('boom');
</cfscript>
And inside of alert.cfc I have:
component {
public any function get(key) {
for(var i = 1; i < ArrayLen(session[key]); i++) {
return session[key][i];
}
}
public any function put(key, value) {
ArrayAppend(session.myCustomFlashKey, value);
return true;
}
public any function exists() {
if(structKeyExists(session,"myCustomFlashKey")) {
return true;
} else {
session.myCustomFlashKey = ArrayNew();
return false;
}
}
}
The Flash Message Component looks like this:
component name="FlashMessage" singleton {
/**
* #flashStorage.inject coldbox:flash
* #config.inject coldbox:setting:flashmessage
*/
public FlashMessage function init(any flashStorage, any config) {
instance.flashKey = arguments.config.flashKey;
singleton.flashStorage = arguments.flashStorage;
instance.containerTemplatePath = arguments.config.containerTemplatePath;
instance.messageTemplatePath = arguments.config.messageTemplatePath;
// Initialize our flash messages to an empty array if it hasn't ever been created
if (! singleton.flashStorage.exists(instance.flashKey)) {
setMessages([]);
}
return this;
}
public void function message(required string text, string type = "default") {
appendMessage({ message: arguments.text, type = arguments.type });
}
public any function onMissingMethod(required string methodName, required struct methodArgs) {
message(methodArgs[1], methodName);
}
public any function render() {
var flashMessages = getMessages();
var flashMessageTemplatePath = instance.messageTemplatePath;
savecontent variable="messagesHTML" {
include "#instance.containerTemplatePath#";
}
setMessages([]);
return messagesHTML;
}
public array function getMessages() {
return singleton.flashStorage.get(instance.flashKey, []);
}
private void function setMessages(required array messages) {
singleton.flashStorage.put(
name = instance.flashKey,
value = arguments.messages
);
}
private void function appendMessage(required struct message) {
var currentMessages = getMessages();
ArrayAppend(currentMessages, arguments.message);
setMessages(currentMessages);
}
}

Salesforce Apex web service test class error

I'm working on Salesforce Apex web service to create record. Below is the apex web service class that I have written:
global class createCloudRecord {
global class projectInputs{
webService Integer ProjectID;
webService String ProjectName;
}
webService static Boolean createSFRecords(List<projectInputs> inputs) {
cv__Designation__c desg = new cv__Designation__c();
desg.cv__Active__c = true;
desg.cv__Default__c = false;
desg.cv__Description__c = 'Test Desc';
desg.OwnerId = '002B0000000K9soIAD';
desg.Name = inputs[0].ProjectName;
desg.cv__ExternalId__c = string.valueof(inputs[0].ProjectID);
insert desg;
return true;
}
}
It's working fine and creating records on SF cloud via SOAP API call. Now I have to written the test class for above web service with code coverage of min 75%.
Below is the code that I have written for my test class:
#isTest
private class createCloudRecordTest {
static testMethod void testCreateSFRecords() {
createCloudRecord.projectInputs project = new createCloudRecord.projectInputs();
project.ProjectID = 12345;
project.ProjectName = 'TestProject';
createCloudRecord.createSFRecords(project);
}
}
But this is showing an error for this line createCloudRecord.createSFRecords(project);:
Error: Compile Error: Method does not exist or incorrect signature.
Anyone has any idea how can I make this working.
Thanks!
I got the solution for my question. The problem was in my web service I've defined param as list but above in test class I'm passing param as single record.
So it should be something like below:
#isTest
private class createCloudRecordTest {
static testMethod void testCreateSFRecords() {
createCloudRecord.projectInputs project = new createCloudRecord.projectInputs();
project.ProjectID = 12345;
project.ProjectName = 'TestProject';
list<createCloudRecord.projectInputs> projects = new list<createCloudRecord.projectInputs>();
projects.add(project);
createCloudRecord.createSFRecords(projects);
}
}

Web Service Serialization produces an unwanted root in SOAP body

History:
A WSDL was provided to me with which I generated a service reference.
The object to serialize and send to the web service to consume, was a strongly, complex-typed class.
Instead of sending a strongly, complex-typed class, I wanted to send an XMLDocument instead.
So I modified the service and I am left with this:
[System.CodeDom.Compiler.GeneratedCodeAttribute("wsdl", "4.0.30319.17929")]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Web.Services.WebServiceBindingAttribute(Name = "B2MML_ProcessProductionPerformance_MaterialConsumption_OBBinding", Namespace="http://company.com/M2D/Manufacturing/ManufacturingExecution/GoodsIssue/pi")]
public partial class B2MML_ProcessProductionPerformance_MaterialConsumption_OBService : System.Web.Services.Protocols.SoapHttpClientProtocol {
private System.Threading.SendOrPostCallback B2MML_ProcessProductionPerformance_MaterialConsumption_OBOperationCompleted;
public B2MML_ProcessProductionPerformance_MaterialConsumption_OBService()
{
}
public event B2MML_ProcessProductionPerformance_MaterialConsumption_OBCompletedEventHandler B2MML_ProcessProductionPerformance_MaterialConsumption_OBCompleted;
[System.Web.Services.Protocols.SoapDocumentMethodAttribute("http://sap.com/xi/WebService/soap1.1", OneWay=true, Use=System.Web.Services.Description.SoapBindingUse.Literal, ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Bare)]
public void B2MML_ProcessProductionPerformance_MaterialConsumption_OB(XmlDocument ProcessProductionPerformance)
{
this.Invoke("B2MML_ProcessProductionPerformance_MaterialConsumption_OB", new object[] {
ProcessProductionPerformance});
}
public System.IAsyncResult BeginB2MML_ProcessProductionPerformance_MaterialConsumption_OB(XmlDocument ProcessProductionPerformance, System.AsyncCallback callback, object asyncState)
{
return this.BeginInvoke("B2MML_ProcessProductionPerformance_MaterialConsumption_OB", new object[] {
ProcessProductionPerformance}, callback, asyncState);
}
public void EndB2MML_ProcessProductionPerformance_MaterialConsumption_OB(System.IAsyncResult asyncResult)
{
this.EndInvoke(asyncResult);
}
public void B2MML_ProcessProductionPerformance_MaterialConsumption_OBAsync(XmlDocument ProcessProductionPerformance)
{
this.B2MML_ProcessProductionPerformance_MaterialConsumption_OBAsync(ProcessProductionPerformance, null);
}
public void B2MML_ProcessProductionPerformance_MaterialConsumption_OBAsync(XmlDocument ProcessProductionPerformance, object userState)
{
if ((this.B2MML_ProcessProductionPerformance_MaterialConsumption_OBOperationCompleted == null))
{
this.B2MML_ProcessProductionPerformance_MaterialConsumption_OBOperationCompleted = new System.Threading.SendOrPostCallback(this.OnB2MML_ProcessProductionPerformance_MaterialConsumption_OBOperationCompleted);
}
this.InvokeAsync("B2MML_ProcessProductionPerformance_MaterialConsumption_OB", new object[] {
ProcessProductionPerformance}, this.B2MML_ProcessProductionPerformance_MaterialConsumption_OBOperationCompleted, userState);
}
private void OnB2MML_ProcessProductionPerformance_MaterialConsumption_OBOperationCompleted(object arg)
{
if ((this.B2MML_ProcessProductionPerformance_MaterialConsumption_OBCompleted != null))
{
System.Web.Services.Protocols.InvokeCompletedEventArgs invokeArgs = ((System.Web.Services.Protocols.InvokeCompletedEventArgs)(arg));
this.B2MML_ProcessProductionPerformance_MaterialConsumption_OBCompleted(this, new System.ComponentModel.AsyncCompletedEventArgs(invokeArgs.Error, invokeArgs.Cancelled, invokeArgs.UserState));
}
}
public new void CancelAsync(object userState)
{
base.CancelAsync(userState);
}
}
[System.CodeDom.Compiler.GeneratedCodeAttribute("wsdl", "4.0.30319.17929")]
public delegate void B2MML_ProcessProductionPerformance_MaterialConsumption_OBCompletedEventHandler(object sender, System.ComponentModel.AsyncCompletedEventArgs e);
Problem Decription:
I am able to send an XMLDocument successfully, however, if my XMLDocument looks like this:
<ProcessProductionPerformance xmlns="SomeNS">
<ApplicationArea>Some data here</ApplicationArea>
<DataArea>Some data there</DataArea>
</ProcessProductionPerformance>
I am seeing the below in the SOAP Body (seen via Fiddler):
<ProcessProductionPerformance>
<ProcessProductionPerformance xmlns="SomeNS">
<ApplicationArea>Some data here</ApplicationArea>
<DataArea>Some data there</DataArea>
</ProcessProductionPerformance>
</ProcessProductionPerformance>
Information:
1) I do not have access to modify the web service
2) I've tried to pack XMLDocument ApplicationArea and XMLDocument DataArea individually into a class and try to serialize the class, I end up with this in the SOAP Body:
<ProcessProductionPerformance xmlns="SomeNS">
<ApplicationArea>
<ApplicationArea xmlns="SomeNS">Some data here</ApplicationArea>
<ApplicationArea>
<DataArea>
<DataArea xmlns="SomeNS">Some data there</DataArea>
<DataArea>
</ProcessProductionPerformance>
3) I suspect it's to do with the SOAPBindingUse or SOAPBindingStyle perhaps? I did not change this at all because I don't know much about it.
4) I'm just a kid new to C#.. Please have mercy.
EDIT:
Okay it seems that the enums SoapBindingUse.Literal and SoapParameterStyle.Bare have a major role to play in this. But I'm still stuck because my use of this has been correct all along.
https://msdn.microsoft.com/en-us/library/vstudio/2b4bx2t6%28v=vs.100%29.aspx
So what am I missing?
By the way, not sure if you humans care about the client but I'm calling the service like this:
var processProductionPerformance = new XmlDocument();
processProductionPerformance.LoadXml(#xmlText);
var sendPerformanceToSap = new B2MML_ProcessProductionPerformance_MaterialConsumption_OBService
{
//Url = Link here,
//Credentials since
};
sendPerformanceToSap.B2MML_ProcessProductionPerformance_MaterialConsumption_OBAsync(processProductionPerformance);

Custom webservice in Umbraco 4.5 giving odd error

We've created a custom webservice in Umbraco to add (async) files and upload them. After upload the service is called with node and file-information to add a new node to the content tree.
At first our main problem was that the service was running outside of the Umbraco context, giving strange errors with get_currentuser.
Now, we inherit the umbraco BaseWebService from the umbraco.webservices dll and we've set all acces information in the settings file; we authenticatie before doing anything else using (correct and ugly-hardcoded) administrator.
When we now execute the webservice (from the browser or anything else) we get:
at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters)
at umbraco.cms.businesslogic.CMSNode.setupNode()
at umbraco.cms.businesslogic.web.Document.setupNode()
at umbraco.cms.businesslogic.CMSNode..ctor(Int32 Id)
at umbraco.cms.businesslogic.Content..ctor(Int32 id)
at umbraco.cms.businesslogic.web.Document..ctor(Int32 id)
at FileUpload.AddDocument(String ProjectID, String NodeID, String FileName)*
Where AddDocument is our method. The node (filename w/o extension) does not exist in the tree (not anywhere, it's a new filename/node). We've cleared the recycle bin, so it's not in there either.
Are we missing something vital, does anyone has a solution?
Below is the source for the webservice;
using umbraco.cms.businesslogic.web;
using umbraco.BusinessLogic;
using umbraco.presentation.nodeFactory;
using umbraco.cms.businesslogic.member;
using umbraco.cms;
/// <summary>
/// Summary description for FileUpload
/// </summary>
[WebService(Namespace = "http://umbraco.org/webservices/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
public class FileUpload : umbraco.webservices.BaseWebService //System.Web.Services.WebService
{
private string GetMimeType(string fileName)
{
string mimeType = "application/unknown";
string ext = System.IO.Path.GetExtension(fileName).ToLower();
Microsoft.Win32.RegistryKey regKey = Microsoft.Win32.Registry.ClassesRoot.OpenSubKey(ext);
if (regKey != null && regKey.GetValue("Content Type") != null)
mimeType = regKey.GetValue("Content Type").ToString();
return mimeType;
}
[WebMethod]
public string HelloWorld() {
return "Hello World";
}
[WebMethod]
public void AddDocument(string ProjectID, string NodeID, string FileName)
{
Authenticate("***", "***");
string MimeType = GetMimeType(FileName); //"application/unknown";
// Create node
int nodeId = 1197;
string fileName = System.IO.Path.GetFileNameWithoutExtension(#"*****\Upload\" + FileName);
string secGroups = "";
//EDIT DUE TO COMMENT: Behavior remains the same though
Document node = umbraco.cms.businesslogic.web.Document.MakeNew(fileName.Replace(".", ""), new DocumentType(1049), umbraco.BusinessLogic.User.GetUser(0), nodeId);
secGroups = "Intern";
StreamWriter sw = null;
try
{
//EXCEPTION IS THROWN SOMEWHERE HERE
Document doc = NodeLevel.CreateNode(fileName, "Bestand", nodeId);
doc.getProperty("bestandsNaam").Value = fileName;
byte[] buffer = System.IO.File.ReadAllBytes(#"****\Upload\" + FileName);
int projectId = 0;
int tempid = nodeId;
//EXCEPTION IS THROWN TO THIS POINT (SEE BELOW)
try
{
Access.ProtectPage(false, doc.Id, 1103, 1103);
Access.AddMembershipRoleToDocument(doc.Id, secGroups);
}
catch (Exception ex)
{
// write to file
}
try
{
doc.Publish(umbraco.BusinessLogic.User.GetUser(0));
umbraco.library.UpdateDocumentCache(doc.Id);
umbraco.content.Instance.RefreshContentFromDatabaseAsync();
}
catch (Exception ex)
{
// write to file
}
System.IO.File.Delete(FileName);
}
catch (Exception ex)
{
// THIS EXCEPTION IS CAUGHT!!
}
}
public override umbraco.webservices.BaseWebService.Services Service
{
get { return umbraco.webservices.BaseWebService.Services.DocumentService; }
}
}
If anyone has a solution, pointer, hint or whatever; help is appreciated!!
TIA,
riffnl
We've rewritten the whole procedure (dumped all code and restart) and we've got it working now.
I think we've been messing around with the old code so much in trying to get it to work we were missing some key issues, because it functions.
Thanks for thinking along anyway!