I'm trying to determine the optimal settings for my ColdFusion PRODUCTION server. The server has the following specs.
ColdFusion: Enterprise Version 10 O/S: Windows Server 2012R2
Standard Processor: Intel(R) Xeon(R) CPU E5-2660 v2 # 2.20GHz
Installed Memory (RAM): 20.0 GB System Type: 64-bit
Operating System, x64-based processor
My Java and JVM settings from the CFIDE are:
Minimum Heap Size (in MB): 2048 Maximum Heap Size (in MB): 4096
JVM Arguments
-server -XX:MaxPermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random
I have multiple websites running on this production server, all of which use ColdFusion. The database server is completely separate, so all that this server is responsible for is the ColdFusion application and web server processes.
The websites are completely data-driven, all pulling from the database located on my production database server. Lately, I've been seeing the ColdFusion service locking up, as it is maxing out the CPU. The memory is stable, it's only the CPU that is maxing out.
Can anyone make suggestions as to how I can tune it to improve overall performance while reducing strain on the CPU?
Java Version
java version "1.8.0_73" Java(TM) SE Runtime Environment (build
1.8.0_73-b02) Java HotSpot(TM) Client VM (build 25.73-b02, mixed mode)
Thank you!
Are you running all of these sites on a single instance of ColdFusion? If so, I would recommend running multiple instances of CF. Each instance can run the same JVM settings given your total available memory.
Minimum Heap Size (in MB): 2048
Maximum Heap Size (in MB): 4096
So that's a max of about 16GB of memory allocated to a total of 4 instances of CF. Then you balance out the volume of sites run on each instance based on site usage. You might have one that needs its own instance and the rest can be spread across the other three.
It's also possible to run all of the sites on all of the instances using a load balancer to pass requests from one instance to another. Either approach should insure that one site doesn't cause the rest to run poorly or not at all.
I have a fairly large Windows application (about 10 years old, written in c++) which works with SQL2000 Express (MSDE). It operates with database pretty extensively, but doesn't have performance issues. Due to SQL2000 MSDE compatibility issues with Windows 7/8 I want to migrate the application to SQL2014 Express.
All database access code is written in t/sql and as such the application migrates to SQL2014 without any code changes and all features work as expected. Except it's so badly slow it makes no sense to use the application under SQL2014. All select/update/insert queries take about 5-20 times more time to execute.
These are connection strings that I tried:
Provider=SQLOLEDB;Data Source=localhost\app;User ID=app_user;Password=password;
Provider=SQLOLEDB;Data Source=localhost\app;Trusted_Connection=yes;
I don't convert SQL2000 database to 2014 as the application creates a new database from scratch from scripts on its first run. Nothing fails, the default DB size is 12MB, the schema is pretty well optimised.
I also tried the same under SQL2008R2 Express - it's as slow as SQL2014 Express. Tried different PCs under Windows 7/8/8.1 - all the same.
The main detail which I noticed is that when I run the application under SQL2014 the most CPU consuming process in Windows Task Manager is "Local Security Authority Process". This process doesn't appear in Task Manager at all when I run it under SQL2000 MSDE and the application runs much faster. I guess LSA may be very heavy processing my "open connection" requests, but I don't know what to do about it.
The application is written is a way that it doesn't keep connections open, but creates them on demand and then releases. I tried to run SQL 2014 service under different accounts - it made no difference.
This process doesn't appear in Task Manager at all when I run it under SQL2000 MSDE and the application runs much faster. I guess LSA may be very heavy processing my "open connection" requests, but I don't know what to do about it.
Typically, lsass.exe (LSA) been used by IPSEC Services(PolicyAgent),
ProtectedStorage and Security Accounts Manager(SamSs)
Try to disable IPSEC Services(PolicyAgent)
Evolution of this question
This started as an attempt to find other recommendations for running Django on Linux, accessing SQL Server via Django-PyODBC, and supporting Unicode as competently as in installations running Django on Windows.
After failing to materialize with a good solution for ODBC drivers in Linux that would provide the same level of support for Unicode as the Windows ODBC driver; the question morphed into coping with the negative side-effect of not having an Apache daemon mode in Windows.
The question
If you run Apache+mod_wsgi on Windows, every time you deploy new Django code you are required to restart the Apache server. See Graham's answer for details on why.
How to run a dependable set of applications and services when you might be required to restart Apache, denying service until it completely restarts?
The issue
We use SQL Server 2005 and we need to support unicode characters and certain characters (like smart-quotes) generated in Microsoft Office applications.
Running Django atop SQL Server 2005 requires us to use Django-PyODBC. It works great on Windows/Linux/Mac OS X; but if you require unicode support, you are out of luck on Linux / Mac OS X - the ODBC drivers for SQL Server in Unix are in varied stage of unicode compliance. FreeTDS, the open source driver, works for some characters if you specify a client character encoding of UTF-8 (*); but doesn't support all characters.
In our tests, running Django on Windows 2003 and using the Microsoft ODBC driver allowed us to properly insert/update/select any character in several different languages, and the Microsoft smart characters from Office applications.
But running on Windows means that every time we deploy new code we are required to bounce Apache - which means a few seconds without service.
(*) The only way we managed to get to the point where some characters would be accepted with FreeTDS, frankly, was to add a client charset entry to freetds.conf:
[a_db_server]
host = a_db_server
port = 1433
tds version = 8.0
client charset = UTF-8
In relation to reloading on Windows when using Apache/mod_wsgi, read my response to:
Server software choice for Django live/staging
Might be bad form to accept one's own answer, but it suited our case and it may help others...
First and foremost: we gave up trying to find an ODBC driver that would work properly with PyODBC and support unicode as competently as the Microsoft native ODBC driver. FreeTDS works partially, and some of the commercial drivers out there just didn't cut it in our tests. We considered a ODBC-JDBC bridge, but never really tested it.
We also thought about using an ODBC router, but given the price for a multi-user solution, the potential for more complexity, and the low load scenario for our Django apps; we decided to just stick with running Django on Windows and try to cope with the requirement of restarting the Apache server every time a new version of any of the Django apps is deployed.
What we did to cope with the Apache restart and still provide service availability:
We deployed an Apache server as a proxy/load-balancer to a cluster of (at the moment) two Apache servers running on Windows
On each of the Windows Apache servers in the cluster we have all our Django apps running
When new code must be deployed to the servers we:
Disable one of the servers in the cluster via Apache's balancer-manager interface
Apply the updates and install any new Django apps in the disabled server
Test the modifications in the disabled server
Reenable the the server in the cluster via Apache's balancer-manager interface
Perform the same steps for the second server
As long as one of the servers in the cluster is enabled and running, the applications and services are available to our users - no interruption. At the same time we gained some load-balancing (in our case we really don't need it at this point).
The proxy Apache servers rewrites all the redirects and cookie response headers, so as long as people are accessing the services through the proxy, there are no modifications needed in the Django code whatsoever.
we are setting up new QE testing server. I guess host OS will be win2008
Which vmware server to choose - 1.x or 2.x ?
A year ago I tried vmware 1 server with Win2008 and it did not work at all.
However, Vmware server 2 did not seem to have the handy vmware console
VMware Server Console\vmware.exe
(is the new vmware server2 still just web based?)
we have a lot of vmware 1 images, are these ok for v2?
Or is it just better to go with HyperV?
Hyper-V Server or ESXi would probably be the best performing and most trouble free options - ie virtualisation not really dependant on the host operating system (but still free).
You can use tools like Vmdk2Vhd to convert image files. Be aware that you should boot the image in its old format first and uninstall vmware tools and some drivers like specific disk controller drivers (as per the instructions of the tool) before you convert it.
Going the enterprise route the System Center Virtual Machine Manager can do this mostly automatically, but then it's not a free solution anymore :) This platform can of course also do physical to virtual migrations for you...
...but alas, you can prepare a physical machine the same way as before converting a vmware image and then use normal imaging tools to clone it into a virtual machine. Or you can use Vmware's free converter tool and then convert the resulting vmdk image to vhd :)
Vmware Server v2 can use v1 images. Windows Server 2008 is supported only in v2. v2 also includes the Virtual Infrastructure Client which you can use instead of the web access ( the client replaces the old console from v1). This is the same client that is used with ESX.
vmware 2.0 crashed regularly on one server I have - however, hasn't crashed since the recent 2.0.1 update (but its only been a week).
I have w2K8 running under vmware 1.0.9 - by selecting Vista (experimental) as the OS setting. However, it's not under any load yet.
I'd recommend ESXi over VMware Server (you can migrate the images) purely for performance reasons. Server 2008 and Vista both run abysmally under VMware Server from my experience. With ESXi and a decent hardware RAID setup, things can be a bit more bearable.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Curious, how many of you develop under a VMware environment?
Is it popular for employers to setup vmware for everyone?
Seems like a great way to rollout new desktop computers and perform backups etc.
Just worried about the performance though (PC vmwares).
Update
I was just looking at vmware's site, 1.3 BILLION in sales..wow!
I almost exclusively use Virtual Machines for development and am very happy doing so. The flexibility of multiple sand-boxed environments is definitely worth a small trade in performance.
Clearly a VM will never give you the same results as running on a native system, but you should be able to get performance that's easily within 10-15% of the real thing. In my experience many of the performance problems people encounter are due to underspecced or poorly configured systems and VM;s.
I primarily develop with a Vista x64 virtual machine on a 2.4Ghz Core 2 Duo with 4GB of Ram. Of this I assign 2GB of RAM and two virtual core's to my main VM. If I'm running more than one VM I usually change this to 1-1.5GB and one core.
Here's some quick GeekBench test results; (Note than GeekBench results under OSX and Vista don't seem comparable, they're listed here to show the impact of configs on both systems).
Fresh boot, no active applications:
Native OSX - 3115
Native OSX running Vista 64 VM - 3042
Native Vista 64 (2.4GHz x 2, 4GB) - 2596
Vista 64 VM (2 VCore, 3GB) - 2362
Vista 64 VM (1 VCore, 2GB) - 1892
These are the most common reasons for poor VM performance in my experience;
Under-specced machines. Ideally you should be able to dedicate one core and 1GB of memory to each VM you plan to work in. Contrary to what you might read I've found that Vista runs within a few percent of XP with 1GB of memory.
Running too many things on your VM. Keep your email, web browsing and IM's to Mummy on your native OS.
On your VM turn off items such as screensavers, background apps and non-essential services. If your VM's are backed-up you may want to turn off system restore.
If possible have your VM's on a separate hard-drive than your native OS so their disc access is independent if one or the other starts paging.
Defrag your VM drive. It does make a difference.
VMware Workstation 6.5 runs like a champ on my older Athlon X2. I use Visual Studio on my host machine and have many VMs installed with various OS, framework and browser combinations. VMware Workstation adds VM debugging into Visual Studio as well, so I can just hit F6 to start my app in any one of my VMs and debug it under any OS I want. The only catch is that you need at least 4gb RAM to make it practical to use more than 1 VM at a time.
My company uses VMware to test our webapp using different browsers/OS versions. Everyone has at least 1 VM on their machine for this purpose. We all develop on the native machine, however -- even on a quad core machine with 4GB RAM, it takes about 20 minutes to do a clean build of our app! For me, I dislike using VM images because of how much paging they do. A few developers here have started using Linux has the host OS and running Windows VMs inside it and they get much better performance due to reduced paging (Linux is way better at memory and disk cache management, plus is has a better scheduler). The extra VMs for testing that would normally be run inside our Windows instance thus get moved to run side by side on the Linux host, which improves performance.
I switched to developing exclusively in VMs around the time I started doing work with technologies like BizTalk Server, Sharepoint, and betas/CTPs of various things...it just got to be impossible to have all the stuff co-exist on the same box.
Since switching I have enjoyed many other benefits to developing in a VM - snapshots, portability, dynamically marshaling resources, etc.
The ultimate benefit is due to VMWare having a presence on many different hosts operating systems, thus I am free to select the host OS of my choice - XP, Vista, Linux, OSX, etc.
Now I run OSX on a MacBook Pro, which allows me to do Mac and iPhone development as well as Windows development, all on the same box.
That is the long winded backstory that brings me to answering the question - as long as your hardware is decently spec'd you should not run into any performance problems - even doing crazy shit with BizTalk and SQL Server.
We use it where I work. We are even making a dvd with the appliance on it to reduce the time it takes new developers to get up to speed.
Regarding performance, I have seen a performance hit. It seems mostly limited by the hard drive if you have snapshots enabled. Of course after I moved my vm's to a VelociRaptor, even that performance hit is no longer noticable.
Oh, I develop ASP websites and C/C++ applications using Visual Studio 2005 and 2008.
Sadly, it's not yet "popular" in the sense of "common," but it's definitely "popular" in the sense of "enjoyed" by those who try it. As a consultant, I love it, since it allows me to swap tool chains in a matter of minutes and, at the end of an engagement, burn a DVD, throw it in the project file, and be done with it.
Several responders seem to be emphasizing the use of VMs for testing, where I think it is beginning to gain some traction, at least within more sophisticated shops. It's clearly a huge win for deployment and compatibility testing.
Depends on the employer, I suppose. On a machine that is adequately-equipped, VMWare (or any virtualization software) performs perfectly fine. On machines that you are more likely to be forced to use at the majority of programming jobs, not so much.
I personally do not use VMWare at work. My work machine barely has enough power to natively handle the tools I need to use.
Its very popular unless employer is cheap, i used it in a few companies. its great for .NET or any language where you have to check if the thing works on different OS versions/platforms. The most common way is not to use VMWare on your own computer but to remotely join it.
I've started using VMware for almost everything on my personal PC.
I keep my native Windows install for games only and have seperate VMs for everything else:
a general office workstatation (MSOffice, accounting software, general crapware). This one stays on almost all the time.
a WAMP stack dev environment
a MS stack dev environment
a throwaway environment for beta testing and toying around with things that might break the OS install.
Everything is pretty fast. I use a streamlined WinXP base install that takes up very little space/memory.
Disk I/O seems to be the bottleneck for me, but I feel we are only one generation (6 months?) away from quite affordable SSDs.
I couldnt go back to physical computing.
Once you start using VM's you'll never go back. I use VMware on a MacBook Pro for Windows and Linux development and I'm very happy with the result.
Observations:
get plenty of RAM. 4GB is quite usable, but 8 is better. You're a developer, you have a lot of apps and web pages open, right?
allocate 1 core to the VM - it's faster than 2.
follow VMware's recommendations for allocating RAM to the guests
use a virtual hard drive for the guest OS. It's much faster than running the guest from a BootCamp partition.
VMware doesn't have the WDDM driver needed to enable Aero.
when I did an eval, the VMware Linux host video drivers didn't seem nearly as fast as for Windows or OSX hosts. Video for Windows guests is noticeably slower on a Linux host vs the other two OS's. This was the main reason I chose Mac over a Linux machine.
In my development environment I use a couple of VM's. Usually one (linux) server per role (such as subversion, MySQL databases, web server, trac server, etc.. ). This way my primary machine remains clean and can't affect my work by running amok, and the data remains secure on the VM-host.
VmWare is quite high-level, for production I'd recommend using a more low-level, bare-metal solution, like Xen.
VMWare as a windows development environment runs terrible on my dual core with 2GB ram (XP guest, XP host). Even with nothing running on the host except for VMware, constant paging that takes about a minute to settle every time I switch applications. Heck, native VS2008 doesn't even run that great during intellisense-heavy use (occasional noticible lag). While using a fixed VM image as my day-to-day working environment has a ton of benefits, the second-to-second performance lag is just too frustrating.
My employer is buying me a nice 64bit system with a ton of ram so I'll revisit the subject in a month. For now I just reimage my machine every couple months.
...console development is obviously performs just fine. for server applications (deployment) where high memory applications aren't launching and closing vmware is lovely and performs fine.
I am doing some SharePoint development and I really love the flexibility that comes from using the VMPlayer on my laptop. I have an image with WSS and the VS2005 tool chain and another image with MOSS and VS2008/SQL server 2008 when I need to it to the max.
When the 2008 image became corrupt (to many beta version I guess) I could just delete it and create a new one from a prior backup.
Being able to develop in a server environment while on the train speakes for it self.
PS: It only takes 4 GB to run the VMWare and it performing really nice, even with a slow 5600 rpm disk drive
Personally I would love to use a virtualization solution for my day to day development because of the ability to test and develop on multiple operating systems simultaneously. However, since my day-to-day development involves quite a bit of opengl this currently isn't a workable solution because most of the time the OS on the VM will default back to software rendering due to the lack of drivers and hardware acceleration.
I develop under a VMWare version of my entire network, including; AD Server, DB Server, etc, needless to say the performance is terrible even on our VMWare server that is running 4gb of ram. But it does allow me to develop without fear of accidentally destroying my companies live databases or shutting down an important server in the real world. And if something crazy happens, no biggy, I can just roll it back to yesterday. If my entire network wasn't housed inside the VMWare environment the performance would be incredible, but running all those other systems really bogs it down a lot.
We tried going all-in with VMs, but found that SQL Server running multiple times on the same physical box basically bogged it down to uselessness. However, I don't think we've seen any serious issues once the DBs were removed from the VM stacks.
Virtualization on desktop / workstation: Sun Virtualbox or VPC. Easy, light. We share our favorite images, keep it causal, and sometime even sysprep them.
Main QA environments get serious with Manager. It's a beast to get working, but can't live without it. There's no way we could afford our test matrix in real machines, or maintain it without the template management. Without such a resource, there are probably things you should do and don't.
Long lived servers or QA DB: VM Ware ESX. (No short explanation).
We don't have perf problems with DBs and virtualization. Well, I did in Lab Manager - which is part of why DB's live on ESX in our shop. For I/O, our IT guys do magic with SAN, iSCSI, and high quality wire. It is certainly simpler to avoid perf problems on db servers if they are bare metal, and probably possible to squeeze out more perf from a dedicated host.
Which brings up what virtualization is and isn't for: Virtualization isn't for a scenario where you are maxing out your hardware already. For example, I don't use it dev on, because I need everything my dev box can give me. It's to replace dozens of underutilized, hard to provision physical servers, with dozens of easy to provision virtual clones on many fewer hosts. It allows hot swapping more capacity, or allows engineering flexibility.
I also have some late 90s computer games that I run in virtualized Windows 98.