Is is possible to set batch size on function level within a webjob?
I have multiple functions in a webjob, some of them depend on other external APIs which does not allow a high degree of parallelization.
I have seen only the Singleton attribute which is not exactly what I am looking for.
just figured out that this is possible with a custom QueueProcessorFactory I already use.
An example from MS is here:
https://github.com/Azure/azure-webjobs-sdk-samples/blob/master/BasicSamples/MiscOperations/CustomQueueProcessorFactory.cs
Having attributes for this would be nice ;-)
Alex
Yeah, custom QueueProcessor instances are designed were designed to be the "escape hatch" allowing you full control in advanced scenarios. We want to keep the mainline paths simple and easy to use, while allowing you to drop down and deeply customize when needed. Adding a bunch of override options on QueueTriggerAttribute itself would be possible, but could also complicate the programming model.
If you would like to suggest a change, I suggest you log issues in the public repo: https://github.com/Azure/azure-webjobs-sdk/issues
Thanks :)
Related
I whould like to use separate databases for runtime and history data without implementing a custom HistoryEventHandler. Does someone know how this is possible?
I read the camunda user guides but this did not help much because it only hints the custom implementation way.
Currently, everytime I query history data (about 2mil activity entries) the performance of the system drops as it kind of blocks the runtime, too. I'd like to avoid this without loosing the ability to query historic data.
That would be a really cool feature, but it is currently not supported. You will have to disable the default history and implement a custom handler.
Camunda BPM offers Optimize, which pulls the history data from the Engine to an Elastic Search database. If you are using the Enterprise version, it may be a way to solve it.
(Based on your comments to other answers, it appears that you're interested in learning more about custom HistoryEventHandler implementations. Thus, I'm adding this answer in the hope that it will help.)
Implementing a custom History Event Handler isn't difficult, but there are a few important points to keep in mind:
Unless you want to skip the storage of history information in the standard Camunda history tables, you'll want to use their CompositeHistoryEventHandler. This simply gives you the ability to use multiple HistoryEventHandler implementations.
Any HistoryEventHandler implementations will complete in the same threads as the ones executing process instances; thus, you will want to be cognizant of the performance impacts your custom HistoryEventHandler will have.
You may want to consider publishing your history events through a message bus or messaging system to allow for reliable delivery without impacting Camunda workflow instance performance.
Finally, it may make sense to use your custom HistoryEventHandler along with Camunda's default HistoryEventHandler and their functionality for deleting process instances after a period of time. This would allow you to use their querying capabilities for some period of time without having the history stack up (and thus slowing down your system).
I would like to ask you if it is possible to create constraint in Neo4j(cypher) with the usage of regex.
To be specific, I have lot of nodes which serves as IPs and I would like to ensure, that each node(property ip_address) is having proper format for IP address.
If the answer is no, is there any workaround ? The only one which currently comes to my mind, is to check every node in programming language before adding them to Neo4j
This isn't currently available in an easy-to-apply constraint form.
While the recommended approach when you need specific formatting is to handle this at the application layer, you could create a trigger that could check if a newly-added node of the given label has the correct formatting and fail out if not.
This does take some additional work and testing however.
TransactionEventHandlers are used to implement this. Here's the TransactionEventHandler java interface you'll need to implement.
Alternately you can use triggers in APOC Procedures to implement this with Cypher.
Is there any method to identify from which source an API is called? source refer to IOS application, web application like a page or button click( Ajax calls etc).
Although, saving a flag like (?source=ios or ?source=webapp) while calling api can be done but i just wanted to know is there any other better option to accomplish this?
I also feel this requirement is weird, because in general an App or a web application is used by n number of users so it is difficult to monitor those many API calls.
please give your valuable suggestions.
There is no perfect way to solve this. Designating a special flag won't solve your problem, because the consumer can put in whatever she wants and you cannot be sure if it is legit or not. The same holds true if you issue different API keys for different consumers - you never know if they decide to switch them up.
The only option that comes to my mind is to analyze the HTTP header and see what you can deduce from it. As you probably know a typical HTTP header looks something like this:
You can try and see how the requests from all sources differ in your case and decide if you can reliably differentiate between them. If you have the luxury of developing the client (i.e. this is not a public API), you can set your custom User-Agent strings for different sources.
But keep in mind that Referrer is not mandatory and thus it is not very reliable, and the user agent can also be spoofed. So it is a solution that is better than nothing, but it's not 100% reliable.
Hope this helps, also here is a similar question. Good luck!
What are some common use cases for implementing CFML Custom Tag (not CFX tag)? In 3 yrs of my CF exp I've never written one. Would someone please enlighten me, under which use case / situation would one choose custom tag over cfc / udf?
Remember that custom tags were, at one time, the only method available to extend CFML (up until version 4) - UDFs came later (CF 5) and CFCs later still (CF MX). They're not as commonly used as they once were for the simple reason that there are more options.
Custom tags are basically procedural in nature in a language that, with CFCs, become more and more OO in practice. This is another reason that they're not very common.
But there's still cases where they come in handy (but are never required) - mostly for interface work. The ability to create both a start and end state can definately come in handy. A simple example could be a "wrapper" for page content the opening tag might add the HTML header and page navigation while the closing tag would add the footer and end the page.
In this way your page content could be nothing more than:
<cfmodule... >
Page Content!
</cfmodule>
Of course there are other ways to do this as well - but sometimes the classics still have value. ;^)
Look at the CFUniform project for a great example of custom tag usage. Custom Tags are great when building reusable pieces for the UI portion of an application.
I think that, for the most part, custom tags have mostly fallen by the wayside since UDFs, CFCs, and integration with Java (and to a lesser degree .NET) allowed easier and more straightforward ways to do similar things.
Looking back to when I started in CF5, I can think of several examples. A good one might be CFX)Zip, which allowed interaction with Zip files before that was available directly through CF.
The only use I can think of offhand in a more modern context would be to provide precompiled code that wasn't written in Java or .NET, such as proprietary doodads written in C. That's a pretty niche use, though.
Honestly, I imagine at this point they exist more or backwards comatibility than anything else.
Ever since CFCs came out I've stopped using custom tags simply because of the overhead. They take too long to initiate and execute. But like #Jim Davis said, they may be useful where you need to write a tag that wraps around other content.
But in a well defined solution, you can do way with them all together.
Historically operating system directory-structures have been trees:
C:
Windows
System32
Program Files
Common Files
Internet Explorer
And the REST architecture emulates the same thing:
http://...//Thomas/
http://...//Thomas/Mexico/Year2003/Photos
http://...//Thomas/Mexico/Year2007/Photos
http://...//Thomas/Finland/Year2005/Photos
http://...//Thomas/Finland/Year2010/Photos
http://...//Thomas/Finland/Year2010/Videos
http://...//Thomas/USA/Year2005/Photos
But, looking the current structure, I need to make searches:
All pictures that are not from
Finland?
All pictures taken in 2005?
All pictures in timeline?
It is not efficient to do a REST-interface with every tree-hierarchy combinations. You need more efficient information management; you need an attribute-system rather than a tree-structure.
(Oh, why the operating systems are not based on attributes?)
StackOverflow and Google seem to use attributes and syntax with "+"-marks like:
http://www.stackoverflow.com/Tags/asp.net+iis7
http://www.google.com/search?&q=iis7+asp.net
Today's frameworks like WCF and ASP.NET MVC have a good support for RESTful tree-structures. But is there support for attribute-structures? Wouldn't you call an attribute-structure still REST?
I would like to make an attribute-WebService and use it with a LINQ in Silverlight-client... Which is the best way to start? :-)
In order to create an effective REST interface you need to identify the resources that make sense for your client application. If you look at you use cases:
All pictures that are not from Finland?
All pictures taken in 2005?
All pictures in timeline?
The question you need to answer, is if this requires three resources or just one. I am assuming you want to have more than just these three queries, so therefore the most flexible solution is to define a generic resource which is a "collection of pictures".
/Thomas/pictures
From here, you want to be able limit contents of this resource by using query parameters.
/Thomas/pictures?country=not-finland
/Thomas/pictures?year=2005
In the case of the third item it may make sense to create a separate resource for that item.
/Thomas/PictureTimeline
There are other scenarios where it may make sense to create additional resource such as
/Thomas/FavouritePictures
The important thing is to identify what key concepts of your application you want to model as resources and then assign those resources an URL. Trying to do REST design via the URL space is going to make you bang your head against the wall.
What you are looking for are URI matrix parameters:
http://www.w3.org/DesignIssues/MatrixURIs.html
When to use query parameters versus matrix parameters?.