Specifically I'm trying to figure out a rough way to calculate a good value, relative to JVM settings, for CF Admin > Server Settings > Caching > Maximum number of cached templates.
So what I'm trying to figure out is whether or not it's possible to use the Admin API to get this information from ColdFusion? Or, if there's another way using Java for instance.
The only guidance I've found is from Adobe's documentation suggesting 1 template per megabyte allocated to the JVM.
Please no answers about profiling.
I know that you can see some of the template cache usage in the CF Server Monitor, but you may also be able to use something http://www.cftracker.net/page.cfm/features to see what's getting used in your cache.
When you say "JVM Settings" do you know which part of the generation you are interested in. The New or Young generation, Old or Tenured generation or the Permanent generation? Also, what are you looking to achieve?
Related
I have made an application in C++ and would like to know how to go about implementing a usage statistics system so that I may gather some data regarding how users use the program.
Eg. IP Address, Number of hours spent in application, and OS used.
In theory I know I can code this myself if I must, but I was wondering if there is a framework available to make this easier to do. Unfortunately I was unable to find anything on google.
Though there is no any kind of such framework, you could reduce the work you have to do (in order to retrieve all these information) by using some approaches and techniques, which I tried describe below. Please, anybody feel free to correct me.
Let's summarise, what groups of information do we need to complete the task:
User Environment Information. I suggest you to look at Microsoft's WMI infrastructure, in particular to WMI classes: Desktop, File System, Networking, etc. Using this classes in your application can help you retrieve almost all kind of system information. But if you don't satisfy with this, see #2.
Application and System Performance. Under these terms I mean overall system performance, processor's count, processes running in OS, etc. To retrieve these data you can use the NtQuerySystemInformation function. With its help, you will get an access to detailed SystemProcessInformation, SystemProcessorPerformanceInformation (retrieves info about each processor) information, and much more.
User Related Information. It's hard to find a framework to do such things, so I suggest you simply start writing code, having in mind your requirements:
counting how many times each button was pressed, each text field was changes, etc.
measuring delay time between consecutive actions in some kind of predefined sequences (for example, if you have a settings gui form and you expect from the user to fill very fast all required text fields, so using a time delay measuments can give you an information if the user acted as we expected from him or delayed after TextBox2 for a 5 minutes).
anything that could be interested to you.
So, how you could implement the last item (User Related Information) requirements? As for me, I'd do something like folowing (some may seem very hard to implement or too pointless):
- creating a kind of base Counter class and derive from it some controls (buttons, edits, etc).
- using a windows hooks for mouse or keybord while getting a child handle (to recognize a control, for example).
- using Callback class, which can do all "dirty" work (counting, measuring, performing additional actions).
You could store all this information either in a textfile or an SQLite database or there wherever you prefer.
I would recommend taking a look at DeskMetrics. This StackOverflow post summarizes the issue.
Building your own framework could take you months of development (apart from maintenance). With something like Trackerbird Software Analytics you can integrate a DLL with your app and start tracking in 30 minutes and you get all the cool real-time visualizations.
Disclaimer: I am affiliated with company.
Trying to build a simple like system in modx (which uses php snippets of code) I just need a button that logged in users can press which adds a 'like' to a resource.
Would it be best to update a custom table or TV? my thoughts are that if it is a template variable i can use getResource to sort by amount of likes.
Any thoughts on the best way to approach this or how to build this would help. My php knowledge is limited.
Depends how you are going to use it after and if you are storing more data than just a 'like' count. TV's are expensive on resources [even more so if you are going to whip through the entire resource set with getResources] so if you are going to do a lot of processing after the fact I would either look at a custom table ~or~ explore using property sets on your pages [I think it should be pretty easy to write a plugin that will update a page property]
I'd definitely go for a custom table.
While you could simply increment a numeric TV to count the amount of likes, you will come to a situation where anyone may be able to keep on liking a resource without limit - while you didn't specify the exact concept, that hardly can be desired. Using a custom table you could throw in a relational alias to the user ID that liked the resource, add a timestamp so you know when it happened, and let your fantasy run wild on additional features that are now open to you.
While not a hard requirement for custom tables, you will probably want to take the time to learn xPDO, which is the database abstraction layer MODX is based on. There's a great tutorial on the RTFM which walks you through it.
I am currently faced with the task of importing around 200K items from a custom CMS implementation into Sitecore. I have created a simple import page which connects to an external SQL database using Entity Framework and I have created all the required data templates.
During a test import of about 5K items I realized that I needed to find a way to make the import run a lot faster so I set about to find some information about optimizing Sitecore for this purpose. I have concluded that there is not much specific information out there so I'd like to share what I've found and open the floor for others to contribute further optimizations. My aim is to create some kind of maintenance mode for Sitecore that can be used when importing large columes of data.
The most useful information I found was on Mark Cassidy's blogpost http://intothecore.cassidy.dk/2009/04/migrating-data-into-sitecore.html. At the bottom of this post he provides a few tips for when you are running an import.
If migrating large quantities of data, try and disable as many Sitecore event handlers and whatever else you can get away with.
Use BulkUpdateContext()
Don't forget your target language
If you can, make the fields shared and unversioned. This should help migration execution speed.
The first thing I noticed out of this list was the BulkUpdateContext class as I had never heard of it. I quickly understood why as a search on the SND forum and in the PDF documentation returned no hits. So imagine my surprise when i actually tested it out and found that it improves item creation/deletes by at least ten fold!
The next thing I looked at was the first point where he basically suggests creating a version of web config that only has the bare essentials needed to perform the import. So far I have removed all events related to creating, saving and deleting items and versions. I have also removed the history engine and system index declarations from the master database element in web config as well as any custom events, schedules and search configurations. I expect that there are a lot of other things I could look to remove/disable in order to increase performance. Pipelines? Schedules?
What optimization tips do you have?
Incidentally, BulkUpdateContext() is a very misleading name - as it really improves item creation speed, not item updating speed. But as you also point out, it improves your import speed massively :-)
Since I wrote that post, I've added a few new things to my normal routines when doing imports.
Regularly shrink your databases. They tend to grow large and bulky. To do this; first go to Sitecore Control Panel -> Database and select "Clean Up Database". After this, do a regular ShrinkDB on your SQL server
Disable indexes, especially if importing into the "master" database. For reference, see http://intothecore.cassidy.dk/2010/09/disabling-lucene-indexes.html
Try not to import into "master" however.. you will usually find that imports into "web" is a lot faster, mostly because this database isn't (by default) connected to the HistoryManager or other gadgets
And if you're really adventureous, there's a thing you could try that I'd been considering trying out myself, but never got around to. They might work, but I can't guarantee that they will :-)
Try removing all your field types from App_Config/FieldTypes.config. The theory here is, that this should essentially disable all of Sitecore's special handling of the content of these fields (like updating the LinkDatabase and so on). You would need to manually trigger a rebuild of the LinkDatabase when done with the import, but that's a relatively small price to pay
Hope this helps a bit :-)
I'm guessing you've already hit this, but putting the code inside a SecurityDisabler() block may speed things up also.
I'd be a lot more worried about how Sitecore performs with this much data... assuming you only do the import once, who cares how long that process takes. Is this going to be a regular occurrence?
My company is at the very end of development process of some (awesome:)) web application. This app will be available as a online service for (hopefully) some significant number of users. This is our biggest django release so far and as we are preparing to release some question about deployment have to be answered.
Q1: how to determine required server parameters for predicted X number of users/Y hits per minute or other factor?
Q2: what hosting solution (shared/vps/dedicated) is worth considering?
Q3: what optimizations can be done at a first place?
I know that this is very subjective and dependent of size of a site, code quality and other factors but I'm very interested in your experiences with django (and not only django) deployment. Any hints, links, advices are kindly welcome. Thanks in advance.
What hosting solution you want to have depends also on how much you want to take of your server yourself (from upgrades etc to backup...), and you should decide if you want to have the responsibility or leave it to someone else.
I think you can only really determine the necessary requirements and bottlenecks in your applications through testing with the estimated load! Try to simulate as many requests.... as you expect - think about caching (where memcached is the best option you have)! If you try to cache things one great tool is the django debug toolbar (http://github.com/robhudson/django-debug-toolbar) which can show you also much about how many database hits you have (dont take the times it shows for that for granted, but analyse them and keep an eye on the number of hits) and eg. how many templates are rendered....
If your system grows, you can first of all think about serving your static media files from another location! Coming to the web server I have made great experiences using lighttpd instead of the fat apache, but I guess you have to evaluate that for yourself!
Also take in consideration what database backend to use, in shared envionments there's in most cases a bigger load on the mysql than on the postgres servers, but also evaluate what works best for you!
You can get some guesses here, but to get a halfway reasonable performance estimate you have to measure the performance of your application yourself. You should then be able to roughly extrapolate the performance on different hardware.
Most of the time the bottleneck is the database, you should get enough RAM to keep it in memory if possible.
"Web application" can encompass so many different things, we can really do no more than guess here.
As for optimization, if it fits to your needs implement some caching (e.g. with memchached), that can give you huge speed improvements.
What I am looking for is a tool that easily or automatically sends coldfusion error messages to their system.
Then I can use the web-based interface, to manage priorities, track who fixed what and so forth.
But I want to use this to help us deal with errors better, but also to show the importance of a bug tracking system to my fellow works.
System Requirements: Apache, Windows, Coldfusion 8 Standard, Sql Server 2005.
Financial Requirements: Free or Open Source
Goal Or Purpose: To encourage my fellow workers to want and use a bug tracking system.
Does this re-write make more sense?
Thanks
Craig
Wiki has a list of issue tracking software, maybe this list could help.
http://en.wikipedia.org/wiki/Comparison_of_issue_tracking_systems
You may be able to find a hosted service and use either email or web services to create the ticket using onError. With that said, a simple issue tracking app could be created for your site using the same DB used to drive the content. 2 or 3 tables would take care of the data storage and you're already using CF so the application layer is already there.
HTH.
I have been heavily using this type of a setup for several years by email only, and the last 3 years with a Bug Tracking Software.
I must say, the bug tracking software has made my life so much more peaceful. Nothing is left, forgotten, or slips through the cracks. It's easy to find trends in errors, and remember "all the times" it happened.
Our setup is like this:
1) Coldfusion + Appropriate framework with error reporting - It doesn't matter what you use. I have used Fusebox extensively and am making the transition to ColdBox. Both are very capable, in addition to Mach-II, FW/1, Model-Glue, etc. The key part you have to find in them is their ability to catch "onError", usualy in the application CFC.
2) Custom OnError Script - Wherever an error occurs, you want to capture the maximum amount of information about that error and email it in. What we do is, when an error occurs, we log the user out with a message of "oops, log in again". Before logging them out, the application captures the error and emails it to Fogbugz. Along with it, at the top we include the CGI variables for the IP address, browser being used, etc. Over time you will find the things you need to add.
3) Routing in Fogbugz. A 2 user version of Fogbugz is free, and hosted online. There are two main ways to submit bugs. One is to email one in at a time. So if an error happens 2000 times, you get 2000 emails, and 2000 cases. Not always the best to link them together, etc. They have a feature called BugzScout, which is essentially an HTTP address that you do a form post to with cfform with all of the same information you would have put into the email. There's plenty of documentation on this and something I've always wanted to get around to. I had a scenario of 2000 emails for the first time happen a few weeks ago so I'll be switching over to this.
Hope that helps. Share what you ended up doing and why so we all can learn too!
I'm surprised no one mentioned LighthousePro (http://lighthousepro.riaforge.org). Open source - 100% free - and ColdFusion. As the author I'm a bit biased though. :)
Hard question to answer not knowing what kind of restrictions are there? Do you have any permissions to install anything? Also most bug-tracking systems require some kind of database support.
I have a suggestion. You can put in place a basic bug-tracking system, that just allows people to create tickets, and allows you/someone else to close it.
More Windows based tools are mentioned here
Good open-source bug tracking / issue tracking sofware for Windows
Any reason why coldfusion specifically?
I really like Fogbugz from the makers of Stack Overflow. For one user it's quite reasonably priced. I enter some bugs manually and have others emailed in.
A lot of bug tracking software will expose SOAP methods for entering data into them.
For example, we used Axosoft's OnTime and that exposed some WSDL pages that I consumed in my application. I was told that Jira did as well.
There are few in CF411 list: Bug Tracking/Defect Tracking/Trouble Ticket/Help Desk Tools Written in CFML
We use HopToad. There is another bug-tracking app called LightHouse that integrates with HopToad so you can easily create a [bug] ticket from an incoming exception. HopToad has an API of which there are many clients, you want the CF based one:
http://github.com/timblair/coldfusion-hoptoad-notifier
Even if you dont use HopToad and you end up using a different service or roll your own, if you needed to write your own API client you could leverage the code or pattern(s) of the above HopToad client.
A lot of good information from everyone, and I really do appreciate the efforts given. But not the answer i was looking for. Which maybe means, that what i want does not exist, yet.
So i may have to roll my own solution...Or maybe integrate with another existing app...
Thank You all.