We're working now on the design of a new API for our product, which will be exposed via web services. We have a dispute whether we should use strict parameters with well defined types (my opinion) or strings that will contain XML in whatever structure needed. It is quite obvious that ideally using a strict signature is safer, and it will allow our users to use tools like wsdl2java. OTOH, our product is developing rapidly, and if the parameters of a service will have to be changed, using XML (passed as a string or anyType - not complex type, which is well defined type) will not require the change of the interface.
So, what I'm asking for is basically rule of thumb recommendations - would you prefer using strict types or flexible XML? Have you had any significant problems using either way?
Thanks,
Eran
I prefer using strict types. That gives you access to client tools that make that end of the job much easier. You also state that if the messaging changes, the string approach will not require changing the interface. Personally, I see this as a disadvantage, not an advantage. If the interface changes, you will know very quickly which clients need to be updated.
Strings containing XML is an extremely bad idea and asking for trouble. Use messages that have a defined schema.I had to rewrite significant portions of an app that used a lot of XML internally instead of types. It was horribly slow and impossible to figure out what was happening.
Related
What are the benefits of the "Convention over Configuration" paradigm in web development? And are there cases where sticking with it don't make sense?
Thanks
Convention states that 90% of the time it will be a certain way. When you deviate from that convention then you can make changes...versus forcing each and every user to understand each and every configuration parameter. The idea is that if you need it to differ you will search it out at that point in time versus trying to wrap your head around all the configuration parameters when it often times has no real value.
IMHO it always makes sense. Making convention the priority over explicit configuration is ideal. Again if someone has a concern, they will force themselves to investigate the need.
I think the benefit is simple: No configuration necessary. You don't need to define locations for this-or-that type of resource, for example, for the app/framework to find them itself.
As for cases where it does not make sense: any situation where it will be fairly frequent that alternative configurations would be required, or where it makes sense that a developer/admin would need to 'opt-in' to some behavior explicitly (for example, to prevent unintended and unexpected side-effects that could have security implications).
The benefit of convention over configuration paradigm in web development the productivity since you won't be required to configured to set all the rules and there are less decision that a programmer has to make. This is evident when using the .NET Framework.
The most obvious benefit is that you will have to write lesser code. Let's take case of Java Persistence API. When you define a POJO having attributes and corresponding setters/getters, it's a simple class. But the moment you annotate it with #javax.persistence.Entity it becomes an entity object (table) which can get persisted in DB. Now this was achieved by just a simple annotation, no other config file.
Another plus point is, all your logic is at one place and in one language (i.e. you get rid of separate xml).
I think this wikipedia article has explained it very well:
Convention over configuration (also known as coding by convention) is
a software design paradigm used by software frameworks that attempts
to decrease the number of decisions that a developer using the
framework is required to make without necessarily losing flexibility.
The concept was introduced by David Heinemeier Hansson to describe the
philosophy of the Ruby on Rails web framework, but is related to
earlier ideas like the concept of "sensible defaults" and the
principle of least astonishment in user interface design.
The phrase essentially means a developer only needs to specify
unconventional aspects of the application. For example, if there is a
class Sales in the model, the corresponding table in the database is
called "sales" by default. It is only if one deviates from this
convention, such as the table "product sales", that one needs to write
code regarding these names.
When the convention implemented by the tool matches the desired
behavior, it behaves as expected without having to write configuration
files. Only when the desired behavior deviates from the implemented
convention is explicit configuration required.
I'm designing a new API and I'm struggling with some decisions. I've read tons of blogs on SOAP vs REST and I used the popular APIs (Paypal, Amazon, etc.) as my guidelines.
I ended up with 2 endpoints in my API: one for SOAP and one for REST (XML). The SOAP one looks pretty good, but the XML interface looks somewhat strange. I'm calling it "strange" because I ended up with namespaces in some of my tags. For example:
[sample1]
<EnvelopeRequest xmlns:c1='http://foobar/CarrierX'>
<Weight>1.0</Weight>
<PostmarkDate>5/3/2013</PostmarkDate>
<c1:ShippingMethod>Ground</c1:ShippingMethod>
<c1:Notification>a#b.com</c1:Notification>
</EnvelopeRequest>
[sample2]
<EnvelopeRequest xmlns:cs='http://foobar/SpecialCarrier'>
<Weight>1.0</Weight>
<PostmarkDate>5/3/2013</PostmarkDate>
<cs:Shape>Flat</cs:Shape>
</EnvelopeRequest>
The reason the XML interface has namespaces is because it is auto-generated from the class definition (which has some inheritance). We are using WCF btw. That works just fine for SOAP (the WSDL is derived from the same class), because SOAP hides all the ugliness in the client proxies. However, after looking at many REST/XML services, I don't think I've seen namespaces being used too often. This also kinda scares me because I'm thinking that I would love to have a JSON interface in the near future, and JSON doesn't support namespaces.
My decision to make the API SOAP friendly came from the fact that many of our customers use Enterprise solutions which thrive on SOAP. But lately, with the growing popularity of Python and Ruby, which new clients seem to adopt more often, I'm starting to second guess my initial decision. The main thing that bothers me is the namespaces in the XML interface, but is it really an issue? Are namespaces in a REST/XML API such a big no-no that I should change my design?
If I do change my design, then my (2 previous) requests would look like so:
[sample1]
<EnvelopeRequest>
<Weight>1.0</Weight>
<PostmarkDate>5/3/2013</PostmarkDate>
<CarrierX>
<ShippingMethod>Ground</ShippingMethod>
<Notification>a#b.com</Notification>
</CarrierX>
</EnvelopeRequest>
[sample2]
<EnvelopeRequest>
<Weight>1.0</Weight>
<PostmarkDate>5/3/2013</PostmarkDate>
<SpecialCarrier>
<Shape>Flat</Shape>
</SpecialCarrier>
</EnvelopeRequest>
And yes, this would allow me to have a JSON interface in the future.
Removing namespaces would be a problem if by doing so you create the possibility of ambiguity in a given message. Is it possible for someone somewhere to create an EnvelopeRequest message with a Shape element that might be interpreted (by code or by people reading the message) in more than one way? The reason to introduce namespaces is to preclude this possibility. Tools like WCF's auto-generator are not able to answer this question in the general case so they err on the side of caution.
Only you can know the set of possible valid messages. In my experience, it's usually preferable to remove namespaces for the sake of not confusing your users/clients. There are a few reasons why I might change that preference:
I expect my message format to be used widely and intermixed with other formats. (A good example is the Atom syndication format)
I'm using someone else's widely used (and namespaced) format and planning to intermix it with my own (e.g. embedding XHTML inside my message).
I expect to embed a message of a given format inside a message of the same format (e.g. XSLT stylesheets that generate XSLT stylesheets).
In that latter case, you might find it convenient (though not absolutely necessary) to use namespaces to separate the inner message from the message that is carrying it by using different prefixes. I don't think any of these cases apply very often.
I would ponder why you have namespace in the first place, those are some strange payloads.
But, disregarding that, no, the namespaces are not a big deal. Namespaces almost inevitably run afoul with XPath and XSL (since they tend to be namespace aware), but when consuming the document wholesale, a lot of times folks just ignore the namespace component completely, so in the end there's no difference.
I would clean up the namespaces for the sake of cleaning them up semantically, but not necessarily for the sake of the consumers. From a practical stand point, it's not that big a deal.
We have an application which uses XSLT to format XML data for display as XHTML.
The system is able to cope with arbitrary XML schemas, so Schemas and XSLTs need to be uploaded by users of the system. Clearly this is a task which is only allowed to Admin level users, however it's also a pretty large bulls-eye to aim at so I'm trying to make it more secure.
I should mention that we're using Saxon 9.0 B
Is there any standard way to sanitise user supplied XSLT? So far I have identified three possible issues although I am concious that there may be more which I simply haven't thought of:
xsl:import and document() functions can get at the server file system. This is pretty easy to lock down using a custom URI Resolver so I'm pretty confident I have this covered
output can contain javascript. I'm thinking of using something like OWASP Anti-Samy to white-list the allowed output tags.
XSLT can call java functions. This is the one which is currently causing me a headache. I don't want to turn the capability off altogether (although at the moment I can't even see how to do that) because we're using it. My preferred solution would be to be able to lock down the acceptable java namespaces so that only known safe functions can be executed. I am open to other suggestions though.
The gold standard would be a standard library which just handles all known XSLT based vulnrabilities, but failing that any suggestions on tackling the issues listed above (especially 3) would be much apprieciated.
Thanks in advance
Saxon has a configuration option to disable use of "reflexive" (dynamically loaded) extension functions. This doesn't prevent use of "integrated" extension functions which have been explicitly registered in the configuration via an API. This seems to meet your requirement of allowing the service provider to register extension functions, but not allowing the stylesheet author to do so.
You can be even more selective if you want by defining your own JavaExtensionFunctionFactory to control how extension function calls are bound. This is fairly low-level system programming and you'll probably need to study the source code to see which methods you need to override to meet your needs.
As well as document(), you need to consider collection(), unparsed-text(), xsl:result-document. In all cases there are Saxon hooks that allow you to control the behaviour.
I don't think uploading and executing anybody's XSLT on the server is something sensible to do.
There are things that one can't prevent or detect, such as a Denial Of Service attack like:
Endless recursion that eats up all available memory and crashes the server with stack overflow.
Transformation that takes many minutes or hours -- as the halting problem is undecidable, we don't know if this is an intentional perpetual loop, or accidental programmer error, or a computation that may or maynot converge.
There are certainly many other exploits, such as referencing a recursively defined entity ...
We are currently building a pile of SOAP Web Service to front the access of various backend systems.
While defining our Request/Response message XML, we see multiple services needing the ‘Account’ object with different ‘mandatory/optional’ fields.
How should we define and enforce the validation of these ‘mandatory/optional’ fields on the same Message? I see these options
1) Enforce validation with XSD by creating different 'Account' Complexe Type
Pros : Design time clarity.
Cons : proliferation of Object Type, Less reuse of Object,
2) Enforce validation with XSD by Extending+Restriction a single base 'Account' type
Pros : Design time clarity.
Cons : Not sure of the support of the Extend+Restriction feature (java, .Net)
3) Using a single 'Account' type and enforcing validation in runtime (ie in the Code).
Pros: Simple
Cons: No design time validation. Need to communicate field requirements via a specification doc.
What are you’re thoughts on that?
I would have to assume that: i) some of what you would call optional fields are actually fields that are not applicable (don't make sense) to all accounts and ii) we're not talking trivial scenarios (like two type of accounts with 2 fields each-kind of thing).
Firstly, I would say that unless you're really lucky, from a requirements perspective, then you're going to end up with some sort of "validation in runtime" no matter what option you're going with. XML Schema can't express some common data validation requirements, such as cross field validation; or simply because the data in your XML is not sufficient to feed the rules to validate the integrity of the message (the data in the message being a subset on what's available at the time the XML is being un/marshalled).
Secondly, I would avoid deriving new complex types through restricton; from an authoring perspective you don't achieve much in terms of reuse, and you might end up with problems in how that is interpreted by your XSD to code tooling. I like to think that the original intention of deriving through restriction was to provide a tool for people to use in xsd:redefine scenarios; for people that wouldn't want to fiddle with XML Schemas that were authored by someone else. If one owns (authors) the schema, one can work around the need to restrict by defining the "lesser" object first and extend from that.
As to the "proliferation of objects", you are kind of getting that with option #2 as well (when compared with #1); what I mean by that, all the tools I know will create a class for each named (global) complex type you have in your XSD; so if you have to have three type of accounts, you'll have three for scenario #1, and four, or so, if you choose to extend from one, or so, base classes; a worst case scenario for the later would be when you need three specializations (concrete if you wish); anyway, from my experience, the difference in real life scenarios is not something that would really tip the decision one way or the other.
Extending base types in XML Schema is good for reuse; however, reuse brings coupling; if you're analysing this from a forward/backward compatibility point of view, extending something in the base type could mess up some of the unmarshalling (deserialization) of the XML for clients of your service(s) that don't want to change their code base, yet you want to maintain only one Web Service endpoint for all; in this case, a forward-compatibility strategy that relies on an xsd:any at the end of a compositor (xsd:sequence) would be rendered useless in your first release that goes and extends your base type.
There is even more; because of this, I don't think there's a correct answer, just for the criteria you seem to imply by setting your pro/cons.
All of my preferred options below assume that you put high value on the requirement to ensure forward/backward compatibility of your services, and you want to minimize the cost of your clients having to deal with your services (because of XML Schema changes).
I would say that if all your domain (accounts in particular) can be fully modeled (assume no future change basically) and that there is enough commonality to justify reuse, then go with option #2. Otherwise, go with option #1 since I have yet to see things that don't change...
If the modeling of your domain can be done 80% or more (or some number that you think is high) and that there is enough commonality to justify reuse, then I would still go with option #2, with the caveat that any future extensions for common attributes across accounts, must be applied for each individual account (basically turning your option into a hybrid, by doing #1).
For anything else, I would go #1. Whew, I can't believe I wrote all of this...
we are currently thinking about how to design an interface for other systems.
My co-worker would like to implement a generic interface (for e.g. doIt(JSONArray)) where you put the desired information you would like to do inside a JSONObject, so that calls would e.g. look like this:
doIt('{"method":"getInformation", "id":"1234", "detailLevel": "2"}')
doIt('{"method":"getEmployeeInfo", "EmployeeId":"4567", "company": "Acme Inc."}')
(i used ' and " in this example just for demonstration purposes. I know that i had to escape the " in the real system).
This method will then be accessable via http, so that i would like http://mysite/doIt?parm={JSONObject}
My approach is to use different interfaces with their respective parameters so that I would have a getInformation(1234,2) and a getEmployeeInfo(4567,"Acme Inc.") interface. So for access via http my scheme would look like: http://mysite/getInformation?id=1234&detailLevel=2 and http://mysite/getEmployeeInfo?employeeId=4567&company=acmeinc.
For the clients accessing our service we want to provide special libraries that encapsulate the bevahiour. E.g. there will a client java-lib which translates a client-call getEmployeeInfo(..) either to
http://mysite/doIt?parm={'{"method":"getEmployeeInfo", "EmployeeId":"4567", "company": "Acme Inc."}'}
or to
http://mysite/getEmployeeInfo?employeeId=4567&company=acmeinc.
and then return the result.
So for clients it will be completely transparent how the backend works if they use the library which handles the "translation".
What do you think are the pros and cons of each idea? I like my approach better because it looks "cleaner". But that is just a feeling which is difficult to argue about. Perhaps you can give me (or him) some thoughts about the design and also touch areas (scalability, security,...) or provide useful links about this matter
I'd probably vote for the JSON solution, even if they are more or less equivalent. (Both easily extendable, standard, future-proof solutions.)
The reasons for choosing JSON:
There are a plethora of different libraries for different platforms that help you build correct objects, check that the string data is valid, etc.
Unmarshalling of JSON data into objects. Some libraries (for example Gson) can automatically marshal and unmarshal JSON into objects. Saves you from writing your own code, and you get the benefit of using code that has been tested by others.
Support for new interfaces. Suppose that you change your transport method to sockets, ftp(!) or whatever. You could still send the JSON objects to you backend using another transport.
I realize this question is old, but I think the answers here would guide developers down the wrong path.
In my experience you should always lean towards the more specific methods. Generic methods are difficult to test, difficult to wrap your head around and provide no (or minimal) IDE/compiler support. Such an api you are describing does not tell the user anything about what it will do.
Your own suggested api design is much better.
That being said, its a balancing act.
The JSON solution could be better because you can send complex object easier
But here it's just a little syntax detail, let the boss choose (or do a vote) and build your software.