Automated browser compatibility validation - unit-testing

Even though the HTML/CSS page is 100% valid according to http://validator.w3.org/
the page may still be broken in some browsers (*cough* IE7 *cough*).
Is there a unittest / validation tool that parses the html, css and javascript and searches the DOM for known browser bugs. Like using "position: relative" in combination with jQuery.slideUp()

That's hard with static analysis. You could have position: relative in a class you applied via an event handler to several elements, one of which you later selected by id and called slideUp() on. The validation tool you want would need the facilities of a browser and need to trigger all kinds of events to find those dangerous patterns.
Because Microsoft browsers are so excellent and are likely to give you a handful of new problems no one else has ever encountered before, it's hard to avoid spending time testing by hand; and on your way to finding crashes they crafted just for you, you'll catch the more mundane ones.

Related

Client-side and server-side rendering of HTML page in a Django single page app

this question could be a duplicate but I have a specific use case.
The app is a single page Django app. The user can choose from several options in a drop-down box. Depending on the choice, a few input boxes will need to be rendered. As an example, if the user chooses to order a pizza, the options in another drop-down menu should be related to the toppings whereas an order for drinks should provide options related to the type or brands. The resulting number of input boxes could be 5 with one option or 10 with a different option.
I can think of rendering the page using JS or by using Python in the back-end. Because this app will be a commercial one, I do not need SEO as users need to log into the app first. I also want to minimize the amount of code that a competitor can 're-use' from my work.
Will client-side rendering open up security problems? If not, is client-side the better way to go?
This question is more of a theoretical/opinion-based nature than technical, but let me provide some answers.
Will client-side rendering open up security problems?
Generally, web application security is a server-side concern, not client-side. You can do things like input validation on the client-side, but the minimum practice for security is to sanitize, validate, and authenticate all request data anyway, so the client-side checks are more of for convenience and improved user experience than security. I'm not saying that there are no such things as client-side security concerns, but it's something I don't think is generally a cause of worry. Client-side rendering specifically and especially doesn't sound like something to be careful about: regardless of what your client-side code does, whatever <form> and <input> markup it generates, your server-side code should always handle the submitted data as if it could be malicious.
Is client-side the better way to go?
There are so many more factors to consider in order to answer this, so it's largely a matter of opinion. But since you're asking about Django, then you might want to reduce overall development friction by maximizing Django's features and design—and Django, in my view, is largely a static markup-first framework, meaning minimal use (at first, at least) of client-side JavaScript. Django Forms and Class-Based Views (CBV), for example, work well together to allow rapid development of non-single-page applications.
Your specific use case of an initial drop-down choice determining the main form to be presented could be developed very rapidly in the traditional Django way by giving up your single-page-application requirement, and just providing some initial menu page that will lead to the different views and forms (pizza vs. drinks, etc.), the latter of which you could build rapidly with the help of CBVs. (By the way, your specific use case doesn't seem too unique, actually. It's just the fundamental issue of complexity for which we have programming concepts such as polymorphism and inheritance in object-oriented programming—hence the appropriateness of CBVs.)
I know that single-page applications are nice, and is the fashionable thing nowadays, but I think people underestimate the speed of old-fashioned HTML applications. And by speed I mean not just the user's client-side experience (HTML pages load rather rapidly with HTTP2 and CDNs and all the other modern Web infrastructure tech these days), but also development time.
Besides, you can always just add single-page-like experiences in a progressive manner. Django is particularly suited to an agile-style development strategy where you'd build initial functionality rapidly without much client-side JS, and then just add rich client-side experiences (using React or Vue or something similar) where it will add the most value for users.
I also want to minimize the amount of code that a competitor can 're-use' from my work.
I don't know the full context, but generally I wouldn't worry about this. If you won't do much client-side rendering, then there won't be much client-side code to ‘steal’. But even if you do, unless you specifically write your client-side code in a way that maximizes reusability (either for yourself or for others), I think coders tend anyway to write highly-coupled code, which is to say, your client-side code will tend to be highly dependent on your server-side code's specifics, which means poor reusability. Your competitors could copy your client-side code all they want, but the cost of making it work with their own back-end will be so high that it wouldn't be worth it, they'll just want to write their own.

What could break when migrating from Adobe ColdFusion to an alternative CFML engine?

We are currently using Adobe ColdFusion 9 for a rather large application. We are thinking about moving to Railo or Blue Dragon.
What problems will we run into?
Will it require a large amount of refactoring or will most CFML code just work on the new system?
Do alternative engines provide support for most all official tags, or are they more limited?
In short, how divergent are these alternatives from the official language?
Is there anything we can do to make this process less painful (like upgrading to CF11 first or removing/avoiding certain features)?
My question is similar to What Notable Differences are there between Railo, Open Bluedragon, and Adobe Coldfusion?, but while that is concerned with practical differences I'm asking more specifically about practicality of transition/implementation.
It all depends on your code and the specific Adobe ColdFusion functionality that you are using. For the most part each CFML iteration supports the same tags/functionality. Where they deviate from the Adobe product is usually documented and explained. You need to dive into your code base and look specifically at the features you are using and compare those to the CFML engine of your choosing. Or you can just download and spin-up the alternate CFML engine, drop your code base in it and see what breaks.
As an example from Railo - CFML Compatibility
Railo tries to adhere the CFML standard as good as possible, Still there are some differences like missing tags and functions or a slightly different behavior. This page and the ones below should describe the incompatibilities.
And I have to question what you are basing this comment on? "and especially it's very uncertain future with them". You are running ColdFusion 9. Adobe has implemented two major version releases since then (10 and 11) and are currently working on the future release.
There are two main areas that can prove problematic when migrating from Adobe ColdFusion to Railo:
Use of feature areas that are not supported by Railo
Sloppy CFML code
The former includes integration with Microsoft technologies, such as Exchange and Sharepoint, as well as Office document manipulation; PDF forms and some of the more sophisticated document manipulations; UI "widget" integration. There are third party extensions for some of the Microsoft integrations, e.g., cfSpreadsheet, but for PDF-related stuff you'll need to roll your own using Java libraries (PDF forms and high quality HTML to PDF conversion are Adobe specialties so be prepared to do quite a bit of work in your migration if you rely on these). As for the UI "widgets", you're better off doing that the "right way" so if you rely on those, you should read ColdFusion UI The Right Way.
The latter is a harder issue to nail down. The differences are not well documented - except in experience posts to mailing lists and blogs by people who've made the transition to Railo - but they include things like:
Using scope names as variables (Railo treats scopes as reserved names for performance reasons)
Embedding comments inside tags, e.g., <cfif x gt y <!--- check boundary --->> (I've seen things like this in older CFML code and was surprised it worked).
Reliance on automatic creation of nested struct elements, e.g., a.b.c = 0 when a has not been declared.
Reliance on long-deprecated features, e.g., parameterExists().
There are many other small differences: Railo is generally stricter about syntax and semantics than Adobe ColdFusion, and often those decisions are driven by performance concerns in that compatibility with Adobe ColdFusion would make Railo slower.
Full disclosure here: I have used Railo pretty much exclusively for five years and I used to run the US arm of Railo's consulting business. That said, you need to consider that Railo is a small company (despite the backing of five fairly large former Adobe partners) with just a handful of people working on the engine, and very little awareness of the product outside the more leading edge portion of the CFML community. By comparison, Adobe have a large team and a marketing budget. Your concerns about the difficulty of finding developers will not be addressed by switching to Railo - to gain access to a larger developer pool, you'd really need to switch to a more popular language, not just a different engine.
Finally, a word about Blue Dragon's engine, specifically Open BlueDragon: the maintainers of that project have stated publicly several times that compatibility with the other engines (Adobe, Railo) is not a primary concern for them, and indeed there are a lot of modern language features that they still don't support or at least don't support in a compatible manner. Last I checked, full-script components were on that list despite having been supported in Adobe ColdFusion and Railo for many years (by which I mean using component { ... } rather than the <cfcomponent><cfscript> .. </cfscript></cfcomponent> form). The BlueDragon dialect of CFML has been steadily diverging over the years so unless you have very old school CFML, that would still run on CFMX7 / ACF8, you probably won't have much success trying to migrate to Open BlueDragon.
There are a couple good answers here and I appreciate the advice given in them. When I asked this question I was looking for something a little more specific, so now that I've had the chance to really play around with migrating our app to Railo I thought I should come back and list out the issues we've run into and, just as importantly, the severity and workarounds. Hopefully this will help others considering making the jump:
cfMessageBox:
cfMessageBox is not a supported tag in Railo. The best solution we've come up with is to create a new custom tag called MessageBox.cfm, then drop it into “{railo-install}/lib/railo-server/context/library/tag/”. This will allow it to be recognized as a core tag and referenced via “”, which saves us from updating hundreds of templates that call it. This, of course, requires us to create a message box custom tag from the ground up.
cfDiv:
cfDiv seems to be throwing a JS error when used to bind to a JS function. I'm going to guess that this is because JS binding is not officially supported (given that I can't find any reference in the official docs), and while ACF allows it as delayed execution, Railo simply doesn’t accept it. We could just create a custom tag that generates a JS setTimeout as described in (1) above, which solved our problem, but applications that actually use this tag for its intended purpose may have a more difficult road ahead.
cfWindow:
There appears to be limited support for cfWindow in Railo. Specifically, new windows need manually shown, and the destroy methods do not exist. Various other bugs appeared as well. We decided that it made more sense to just move to JQuery based modals.
cfLayout:
cfLayout support is questionable. It is based on JQuery and not Ext-JS like ACF’s version. This causes a problem because we run JQuery 1.10 right now and the built-in tag doesn’t appear to work beyond JQuery 1.8. In fact, I could not find any JQuery version within which the tag worked perfectly. We decided that it may be best to, again, just write our own custom tag based on JQuery.
cfDocument:
cfDocument works differently in Railo and seems to require more strict HTML. I found a lot of helpful information here, though as of yet I haven't actually gotten any of my cfDocument calls to work as expected.
Relative cfLocations:
cfLocations that began with a “../” and backtracked beyond the webroot would throw a weird Java error. This ended up being a bug in Tomcat, and was patched by the Railo team in version 4.3.1.003. If you download an older Railo version you may run into this issue and need to update all of your cfLocation calls.
Oracle Thin Client:
Our database guy reported to me that he setup the Oracle Thin Client, because the OCI client is not natively supported in Railo. I found this, which might be relevant, but I don't have the expertise to say for sure.
Documentation:
ACF Livedocs are sometimes aggravating as they don't touch on the more important intricacies of how some tags are implemented, but Railo's version is the definition of minimalist. I think it's fair to say that Railo has no docs specifying each tag and function and that they leave you to rely on Adobe for that, which causes a serious issue when you need to know how the two implementations differ.
In the end it seems like, as predicted by previous answers, the UI tags were the bulk of our issues. Based on previous comments I was hoping for better implementations of them that may just require a tweak here and there, but (at least for our needs) the Railo versions seem borderline non-functional and it looks like we would need to replace them completely. For us, this may not be realistic, though we are still tossing the idea around.
To be fair, here are some of the good points from our research and testing:
Performance:
Although compatibility problems have prevented me from doing much performance testing, initial spot checks show approximately a 50% decrease in execution time for most pages.
Debugging:
The debugging options in Railo are quite amazing. There are far more options for formatting, including specifying different formats for different developers (IP addresses). One incredible feature is the inclusion of a comma delimited list of query fields that were actually used in the page: this could allow you to effectively develop based on a "select *" query and simply copy and paste the fieldlist into the query at the end of development, which would save a lot of time with views as large as the ones we're using.
Cost:
This is one of the larger reasons we decided to look into alternatives. Switching just a few Enterprise licensed ACF servers over to Railo would save $20k+ over upgrading to the newest version of ACF. Further, with the performance increases you could see an even greater savings in hardware requirements. A side effect of this point is that one can keep far more up to date without the constant cost/benefit analysis of licensing costs holding up upgrades.
Support:
Without a support contract, it doesn't seem like Adobe responds to user concerns. I've had a production impacting bug reported since ACF 9 which still hasn't been fixed. Yet the Railo community is one of the most helpful and responsive I've ever seen, and developers have even responded directly to concerns and bug reports I've raised.
Longevity:
This is a highly opinionated point, of course, but while Adobe seems to be relegating ACF to the shadows more and more with each new version, Railo appears to be dedicated to growing the community. Combined with its open source nature I think this makes it a safer bet for future support in the long term, even if that support is just us taking development into our own hands when needed.
For a number of reasons, including divergent CFML compatibility, we did not even get to the testing stage with Blue Dragon.

What are my options for white-listing HTML in ColdFusion?

I want to allow my users to input HTML.
Requirements
Allow a specific set of HTML tags.
Preserve characters (do not encode ã into ã, for example)
Existing options
AntiSamy. Unfortunately AntiSamy encodes special characters and breaks requirement 2.
Native ColdFusion functions (HTMLCodeFormat() etc...) don't work as they encode HTML into entities, and thus fail requirement 1.
I found this set of functions somewhere, but I have no way of telling how secure this is: http://pastie.org/2072867
So what are my options? Are there existing libraries for this?
Portcullis works well for Cold Fusion for attack-specific issues. I've used a couple of other regex solutions I found on the web over time that have worked well, though they haven't been nearly as fleshed out. In 15 years (10 as a CMS developer) nothing I've built has been hacked....knock on wood.
When developing input fields of any type, it's good to look at the problem from different angles. You've got the UI side, which includes both usability and client-side validation. Yes, it can be bypassed, but javascript-based validation is quicker, more responsive, and rates higher on the magical UI scale than backend-interruption method or simply making things "disappear" without warning. It will speed up the back-end validation because it does the initial screening. So, it's not an "instead of" but an "in-addition to" type solution that can't be ignored.
Also on the UI front, giving your users a good quality editor also can make a huge difference in the process. My personal favorite is CKeditor simply because it's the only one that can handle Microsoft Word code on the front-side, keeping it far away from my DB. It seems silly, but Word HTML is valid, so it won't setoff any red flags....but on a moderately sized document it will quickly overload a DB field insert max, believe it or not. Not only will a good editor reduce the amount of silly HTML that comes in, but it will also just make things faster for the user....win/win.
I personally encode and decode my characters...it's always just worked well so I've never changed practice.

What are the gotchas with ColdFusion?

Background:
I have a new site in the design phase and am considering using ColdFusion. The Server is currently set-up with ColdFusion and Python (done for me).
It is my choice on what to use and ColdFusion seems intriguing with the tag concept. Having developed sites in PHP and Python the idea of using a new tool seems fun but I want to make sure it is as easy to use as my other two choices with things like URL beautification and scalability.
Are there any common problems with using ColdFusion in regards to scalability and speed of development?
My other choice is to use Python with WebPy or Django.
ColdFusion 9 with a good framework like Sean Cornfeld's FW/1 has plenty of performance and all the functionality of any modern web server development language. It has some great integration features like exchange server support and excel / pdf support out of the box.
Like all tools it may or may not be the right one for you but the gotchas in terms of scalability will usually be with your code, rarely the platform.
Liberally use memcached or the built in ehache in CF9, be smart about your data access strategy, intelligently chunk returned data and you will be fine performance wise.
My approach with CF lately involves using jQuery extensively for client side logic and using CF for the initial page setup and ajax calls to fill tables. That dramatically cuts down on CF specific code and forces nice logic separation. Plus it cuts the dependency on any one platform (aside from the excellent jQuery library).
To specifically answer your question, if you read the [coldfusion] tags here you will see questions are rarely on speed or scalability, it scales fine. A lot of the questions seem to be on places where CF is a fairly thin layer on another tool like Apache Axis (web services) and ExtJs (cfajax) - neither of which you need to use. You will probably need mod-rewrite or IIS rewrite to hide .cfm
Since you have both ColdFusion and Python available to you already, I would carefully consider exactly what it is you're trying to accomplish.
Do you need a gradual learning curve, newbie-friendly language (easy for someone who knows HTML to learn), great documentation, and lots of features that make normally difficult tasks easy? That sounds like a job for ColdFusion.
That said, once you get the basics of ColdFusion down, it's easy to transition into an Object Oriented approach (as others have noted, there are a plethora of MVC frameworks available: FW/1, ColdBox, Fusebox, Model-Glue, Mach-ii, Lightfront, and the list goes on...), and there are also dependency management (DI/IoC) frameworks (my favorite of which is ColdSpring, modeled after Java's Spring framework), and the ability to do Aspect-Oriented Programming, as well. Lastly, there are also several ORM frameworks (Transfer, Reactor, and DataFaucet, if you're using CF8 or earlier, or add Hibernate to the list in CF9+).
ColdFusion also plays nicely with just about everything else out there. It can load and use .Net assemblies, provides native access to Java classes, and makes creating and/or consuming web services (particularly SOAP, but REST is possible) a piece of cake. (I think it even does com/corba, if you feel like using tech from 1991...)
Unfortunately, I've got no experience with Python, so I can't speak to its strengths. Perhaps a Python developer can shed some light there.
As for url rewrting, (again, as others have noted) that's not really done in the language (though you can fudge it); to get a really nice looking URL you really need either mod_rewrite (which can be done without .htaccess, instead the rules would go into your Apache VHosts config file), or with one of the IIS URL Rewriting products.
The "fudging" I alluded to would be a url like: http://example.com/index.cfm/section/action/?search=foo -- the ".cfm" is in the URL so that the request gets handed from the web server (Apache/IIS) to the Application Server (ColdFusion). To get rid of the ".cfm" in the URL, you really do have to use a URL rewriting tool; there's no way around it.
From two years working with CF, for me the biggest gotchas are:
If you're mainly coding using tags (rather than CFScript) and formatting for readability, be prepared for your output to be filled with whitespace. Unlike other scripting languages, the whitespace between statements are actually sent to the client - so if you're looping over something 100 times and outputting the result, all the linebreaks and tabs in the loop source code will appear 100 times. There are ways around this but it's been a while - I'm sure someone on SO has asked the question before, so a quick search will give you your solution.
Related to the whitespace problem, if you're writing a script to be used with AJAX or Flash and you're trying to send xml; even a single space before the DTD can break some of the more fussy parsing engines (jQuery used to fall over like this - I don't know if it still does and flash was a nightmare). When I first did this I spent hours trying to figure out why what looked like well formed XML was causing my script to die.
The later versions aren't so bad, but I was also working on legacy systems where even quite basic functionality was lacking. Quite often you'll find you need to go hunting for a COM or Java library to do the job for you. Again, though, this is in the earlier versions.
CFAJAX was a heavy, cumbersome beast last time I checked - so don't bother, roll your own.
Other than that, I found CF to be a fun language to work with - it has its idiosyncracies like everything else, but by and large it was mostly headache free and fast to work with.
Hope this helps :)
Cheers
Iain
EDIT: Oh, and for reasons best known to Adobe, if you're running the trial version you'll get a lovely fat HTML comment before all of your output - regardless of whether or not you're actually outputting HTML. And yes, because the comment appears before your DTD, be prepared for some browsers (not looking at any one in particular!) to render it like crap. Again - perhaps they've rethought this in the new version...
EDIT#2: You also mentioned URL Rewriting - where I used to work we did this all the time - no problems. If you're running on Apache, use mod_rewrite, if you're running on IIS buy ISAPI Rewrite 3.
do yourself the favor and check out the CFWheels project. it has the url rewriting support and routes that you're looking for. also as a full stack mvc framework, it comes with it's own orm.
It's been a few years, so my information may be a little out of date, but in my experience:
Pros:
Coldfusion is easy to learn, and quick to get something up and running end-to-end.
Cons:
As with many server-side scripting languages, there is no real separation between persistence logic, business logic, and presentation. All of these are typically interwoven throughout a typical Coldfusion source file. This can mean a lot more work if you want to make changes to the database schema of a mature application, for example.
There are some disciplines that can be followed to make things a little more maintainable; "Fusebox" was one. There may be others.

Web 2.0 and dial-up: how make it as painless as possible?

I'm trying to put a workable plan together for a charity that could really make good use of a forum and a wiki, but a crucial part of its operations happen in parts of the world where dial-up connection dominates and probably will continue to do so for the foreseeable future.
This site was recommended as one that behaves well even on a dial-up connection, so I thought I'd ask for some help here!
The site I want to hook this on to is using Drupal. Anyone out there with experiences like this who could maybe help?
Behaving well on dial-up involves sitting down and optimizing your HTML, CSS, and images to be as small as possible, and then ensuring that your server is sending sane HTTP headers for caching. Make sure your CSS stylesheet is external, and shared across all pages. If dial-up is a major issue, you'll want to stick to a single stylesheet if possible. Avoid JavaScript, because those computers usually don't have the processing power for it either. If you must use JavaScript, jQuery is extremely small and very fast and highly recommended, but I suspect that for most content-oriented websites, it won't be necessary.
To be honest, if you produce valid XHTML/HTML5, valid CSS, and you follow all of the usual best practices for standards-based web design (no table layouts, semantic markup, etc), dial-up really won't be an issue. It'll just work.
To tweak the maximum performance out of your site you might want to install this and use it on your site when you are done with the initial development- ySlow - this will analyse your pages and highlight all the areas you can improve. It's really a great tool for optimising site download speeds.
You should be able to accomplish this, but to be honest you are going to loose a lot in the way of user experience by creating a dial-up friendly site. It basically means you have to do the following to optimize for the experience:
Keep JS to a minimum
Make sure the JS is minified.
Reduce large image requirements w/ CSS and some optimal planning of layout
Make sure caching is enabled in the headers so that new files only get downloaded when nessisary.
If you do all this, you should have a site that is acceptable on dialup.
There are already some hints on how to keep page sizes and load times down.
To complement this, you could use a software that simulates limited bandwith. This helps you test the speed of your site on dialup.
There are several available (just google "simulate dialup").
Sloppy e.g. seems quite usable.
You could also do what Google does for Gmail, i.e. provide 2 versions of your view, one for slow connections that uses plain old HTML, and one for faster connections. You could make the default one the slow one, but provide a link to enable the faster one.
Gmail also has a built-in mechanism that detects when you load the page whether it's going fast or not and will automatically revert to the plain HTML view if it's too slow, which is another fancier alternative.
Your main goal should be minimum page size (keep only HTML in pages, all styling information should be externalized in css files for caching, same for JavaScript in js files) and minimum round trips to server (full requests and post backs). Contrary to popular belief a JS heavy site could work like a charm if you perform a lot of heavy duty client side and keep the server roundtrips clean with the minimum amount of data needed (think JQuery and AJAX here with small partial renderings).
P.S. If u'r using .NET throw ViewState away.