I'm in the process of porting a Java desktop application to a ColdFusion web app. This desktop app made queries with very large result sets (thousands of text records) that, while being all right on the database side, could take a lot of memory on the client side if they were buffered. For this reason, the app explicitly tells the database driver to not buffer results too much.
Now that I'm working on the ColdFusion port, I'm being hit by the buffering problem. The ColdFusion page times out during the <cfquery> call, and I'm fairly sure this is because it tries to buffer everything.
Can I make an unbuffered query in ColdFusion?
If pagination is not an option (i.e., you're writing out a report for example), then you'll have to get low level with the java, using setFetchSize(). See this answer. Note that the code in the answer uses the DataSourceService, which, with latest security patches from Adobe, is no longer available on CF8. You'll have to figure out how to get a connection via the adminapi or create a connection outside of coldfusion. Or you could transition your datasource to use JNDI, and then you can lookup the resource yourself without using CF api's.
I'm almost certain that ColdFusion does not provide such a mechanism. As a language, it was meant to abstract the developer away from things like that.
I'd suggest that you look into a few options:
Re-work your query to use pagination, and run it in a loop.
Use the timeout attribute on the <cfquery> to prevent timeouts from happening
Use the CreateObject() syntax to instantiate a JDBC database connection.
With the last option, what you'd actually do is access the underlying Java classes to do the querying and getting results. Take a look at this article for a quick look at the CreateObject() function.
You can also look at the Adobe Livedocs for the function, but they don't really seem helpful.
I haven't tried to use CreateObject() to do querying with the Java database access classes, but I imagine that you can probably get it to work.
Related
I need to design and implement a Java web application that can be used by multiple users at the same time. The data that is handled by this application is going to be huge and may take about 5 minutes for a page to display the results(database records).
I had designed this application using HTML, Servlets and JSP. But when two users would try to get the records, only one user was able to view the results while the other faced an error.
I always thought a web application would take care of handling multiple users but this is not the case.
Any insights on this would be highly appreciated.
Thanks.
I always thought a web application would take care of handling multiple users but this is not the case.
They do if they're written correctly. Obviously yours is not. That's all we can tell you unless you give more information, most importantly details of the error shown to the second user.
One possibility is that everything is OK on the web layer but your DB access for the first user causes an exclusive lock so that the second user cannot access the data at the same time. This could be fixed by using non-exclusive read locks. How to do that depends mainly on what DB you're using.
Getting concurrency right requires you to choose the correct tools and use them correctly. It doesn't just happen magically because it's a web app.
What are are using to develop this web-application? If you are developing it in your own way from the start I must say you are trying to re-invent the same wheel which has been already created and enhanced by very solid frameworks.
I suggest you analyze your requirements thoroughly and study some available frameworks. Let them handle the things like multi threading and other aspects in the best possible manner.
Handling multiple request at a time is a container work and as an application developer we have to concentrate how we are handling and processing those requret being forwarded by the container.
I must suggest you to get some insight how web-application work and how request -response cycle happens
I want to trace user's actions in my web site by logging their requests to database as plain text in Django.
I consider to write a custom decorator and place it to every view that I want to trace.
However, I have some troubles in my design.
First of all, is such logging mecahinsm reasonable or because of my log table will be enlarging rapidly it causes some preformance problems ?
Secondly, how should be my log table's design ?
I want to keep keywords if the user call search view or keep the item's id if the user call details of item view.
Besides, IP addresses of user's should be kept but how can I seperate users if they connect via single IP address as in many companies.
I am glad to explain in detail if you think my question is unclear.
Thanks
I wouldn't do that. If this is a production service then you've got a proper web server running in front of it, right? Apache, or nginx or something. That can do logging, and can do it well, and can write to a form that won't bloat your database, and there's a wealth of analytical tools for log analysis.
You are going to have to duplicate a lot of that functionality in your decorator, such as when you want to switch it on or off, or change the log level. The only thing you'll get by doing it all in django is the possibility of ultra-fine control, such as only logging views of blog posts with id numbers greater than X or something. But generally you'd not want that level of detail, and you'd log everything and do any stripping at the analysis phase. You've not given any reason currently why you need to do it from Django.
If you really want it in a RDBMS, reading an apache log file into Postgres or MySQL or one of those expensive ones is fairly trivial.
One thing you should keep in mind is that SQL databases don't offer you a very good writing performance (in comparison with reading), so if you are experiencing heavy loads you should probably look for a better in-memory solution (eg. some key-value-store like redis).
But keep in mind, that, especially if you would use a non-sql solution you should be aware what you want to do with the collected data (just display something like a 'log' or do some more in-deep searching/querying on the data).
If you want to identify different users from the same IP address you should probably look for a cookie-based solution (if you are using django's session framework the session's are per default identified through a cookie - so you could just simply use sessions). Another solution could be doing the logging 'asynchronously' via javascript after the page has loaded in the browser (which could give you more possibilities in identifying the user and avoid additional load when generating the page).
Before you start flaming, I'm going to tell you that I am trying to convince myself that this is a bad idea.
Basically, I'm trying to create a website with some basic accounting functions. My friend, a consultant who only knows excel, asked if this could be an excel spreadsheet instead of a web interface.
I found myself thinking, why is excel not the better tool in this case? It has all the tools a web2.0 app has (scripting, access to a db, basic formatting), and is made for accounting. You could basically use excel to write a program that fetches data from an SQL database, populates some cells, and use formulas for the rest.
Is there precedent to using excel as one would use the browser to make an ajax-y web2 app?
Why is this a bad idea?
Update: just to clarify I meant this to be more a "rhetorical" discussion. I'm not sure why the rest of the message didn't clarify that...
Well, if you made it in excel, then it wouldn't really be a web application. It would be an Excel application.
Not that that's a bad thing though. If it makes more sense to build the app in Excel, then go for it.
The problems you'll have doing it in Excel will be the same as the problems you run into with any desktop vs. web application. For example, How do you handle deploying new versions of the app? How do you handle updating the app for new (or older) versions of Excel? etc.
ETA:
If you want to avoid the bugginess of Excel VBA and the headaches of managing a desktop application, you might want to look into the Google Spreadsheets API. You can use it to create/update Google docs spreadsheets on the fly, including formulas and lots of other spreadsheet goodness. Using a hybrid of web application and Google Spreadsheets might give you the best of both worlds, depending on what exactly you need to do.
After years of programming in Excel-VBA, the best answer that I can give you for not doing this:
Excel-VBA is buggy! It is probably the most bug-filled app that Microsoft has produced. It's great for some tasks, but forcing it to do a job for which it was never intended will lead to trouble.
I have a few spreadsheets that do similar things, (things Bill never intended), and without exception, they teeter on the brink of failure, and tend to crash with only the slightest provocation.
Sure, you can do it, but the headache is not worth it.
Excel was not meant to be used that way, so it will be painful.
Two better ideas for web-spreadsheet integration:
Take a look at Resolver One, a programmable Excel-compatible spreadsheet with an integrated web server.
Use Google Docs spreadsheets. There is an API to interact programmatically with them.
It's not a bad idea, but it does come with some limitations. If deployment isn't an issue for you, and you don't need "universal" access to the application, your solution will likely save you some time. A web application would certainly be a more elegant approach, particularly if you want to make the software available publicly.
The web is inherently based on HTML and added to with Databases, CSS and server-side languages and javascript (possibly others). As with most things, you should use the right tool for the job. If you want a website then you should use the tools for creating a website.
Excel is not meant to make websites, forcing it to do so will likely lead to more frustration than happiness.
Well, you're going to run it on the server side, you'd probably run into licencing and performance issues.
If you're delivering .xls to the client, then you'd need for all your clients to have Excel or something compatible. But you also lose control over your "site" - what's the point of visiting if you have everything you need locally?
So, the SQL statements would be on the client? That's rarely a good idea.
It's a bad idea for the same reason that writing a graphics editor with MS Access is bad, or coding a MMORPG using Powerpoint is bad :)
I would also say that once you stop using a browser as the client it ceases to be a "web application" - you are really just talking about an Excel sheet which fetches its data via HTTP.
+1 to everyone who said "an excel app is not a web app."
BUT... If you want to use the spreadsheet metaphor for server-side calculations for a web app, or if you want to access the library of financial functions that come with Excel from server-side code, you can use Excel Services. It's exposed via SOAP, interoperable, callable from any SOAP-capable platform.
It's server-capable. It does not actually load Excel on the server, but a non-GUI runtime of the functions.
I think that using Excel as Browser is bad idea, however i think better idea is to use Excel Control in Desktop App.
This way you can control the sql and connections. you can save data as often as you like. you could also implement some update mechanism. App would be more secure and harder to hack.
I think Excel freezes when you try to connect to some outside resource and this way you would control everything.
Did you start by getting user requirements for this application? It doesn't sound like they wanted a website to begin with. Sounds like they wanted an excel spreadsheet with macros.
I need a way to stash some data that is global to the browser. If I open a new window with a URL from my app, e.g. via a bookmark, I need to access some data that was created in another window and never sent to the server.
As far as I can tell the only thing that is global to the browser and not just a window, (like window.name), is a cookie. The problem I'm running into is if I set a cookie the cookie is then sent with every request to the server, but I don't ever want this data on the wire. Is there any way to set a cookie and just use it purely as a bucket for storing some data and never send that data to the server?
The HTML 5 storage API looks like exactly what you want here, but unfortunately it's only supported by a handful of browsers right now.
Is there any way to set a cookie and just use it purely as a bucket for storing some data and never send that data to the server?
No.
You'll need to look into a plugin that provides dedicated offline storage facility, or use the HTML5 storage API and tell everyone to upgrade their browsers
If you decide to go the plugin route, as far as I am aware you have 3 options:
Google Gears
Flash - it has an offline storage facility - you could write a small flash app to store things using this facility, then interop with it from javascript.
Silverlight also has offline storage - as with flash you could write a small app to do the storage, then interop with it from javascript.
I'd probably look into using flash first, as everyone already has it.
Development would likely be a lot easier if you were to use silverlight. It's not as widely installed, but it is spreading pretty rapidly. Last I heard* something like 30% of browsers had it installed which is pretty impressive.
Google gears would unfortunately be a distant third. People are going to be installing flash and silverlight for other reasons, but nobody has gears.
*This is an entirely unsubstantiated quote, but does seem to fit with what I've seen on various people's computers, etc.
Can you mandate that your users install Google Gears? It's a javascript API that lets you store local info- also lets you persist between sessions, which may be useful for your app.
Why not just read a field in the parent window using window.opener ? Or if you've three windows running - parent and two children which I think you might be implying then read/write to a hidden field in the parent from the children.
Sounds like your app is running 100% local, if that is the case the browser isn't the way to go anyway. Cookies can be easily deleted. If your app isn't local the webserver should be the one supplying information. Cookies are never the correct way to store sensitive information or information that should persist over longer amounts of time.
This'll be my first question on this platform. I've done lots of development using Flex, WebORB and ASP.NET. We have solved Concurrency problems with messaging (Pessimistic Concurrency Control). This works pretty good but it also makes the whole application dependent of the messaging. No messaging, no concurrency control.
I know that ASP.NET has version control in DataSets, but how would you go and use that if you are working on a RIA. It seems hard to go and store each dataset in the session of the client... So, if the Client would like need all products, I would need to store the dataset in the session of the client. When the client would change something to a product and save the product, I could then update the dataset (stored in the session) and try to save it...
Seems a lot of work and a lot of memory that will be used (because those products will be kept in the memory of the client, so the dataset needs to be kept on the server side session).
I think the most easy way would be to provide all DTO's with a version number. If the client would try to save a DTO, I could compare the version number with the one in the database.
Lieven Cardoen
This is something I've done before - as the original data was coming from an SQL Server database we just used a rowversion typed column in each DTO to determine if it had changed while the user was working on it.
At this point you can either barf on the error or try and figure out a way to merge the changes, but at least you can tell that it's changed underneath you :)