I've been tasked with creating a deployable version of a ColdFusion web app to be installed on a clients server. I'm trying to find a way to give them a compiled version of our code, and my first inclination was to use the CFCompile utility that I found here. However, after running CFCompile, most of the code in the CFM files is still readable. The only thing that appears to be obfuscated at all is the actual ColdFusion code - all of the SQL Queries are still perfectly readable. (Example in the screenshot below)
The HTML and JavaScript are also still readable in the compiled code, but that doesn't matter as those can be seen in a web browser anyways.
Is there another way to distribute my source code in a format that is completely unreadable to the user? I'm guessing that for whatever method I choose, there will be some way of decompiling the code. That's not an issue, I just need to find a way to make it more difficult than opening the file and seeing the queries.
Hostek has a pretty good write up on the subject over on their site - How to Encrypt or Compile ColdFusion Files.
Basically, from that article:
Using cfcompile.bat
The cfcompile.bat utility will compile all .cfm and .cfc files within a given directory into Java bytecode. This has the effect of making your source code unreadable, and it also prevents ColdFusion from having to compile your ColdFusion files on first use which provides a small performance enhancement.
More details about using cfcompile.bat can be found in ColdFusion's Documentation
Using cfencode.exe
The cfencode.exe utility will apply basic encryption to a specific file or directory. If used to encrypt a directory, it will apply encryption to ALL files in the directory which can break any JS, CSS, images, or other non-ColdFusion files.
They do also include this note at the bottom:
Note: Encrypting your site files with cfencode does not guarantee absolute security of your source code, but it does add a layer of obfuscation to help prevent unauthorized individuals from viewing the source.
The article goes on to give basic instructions on how to use each.
Adobe has this note on their site regarding cfencode:
Note: You can also use the cfencode utility, located in the cf_root/bin directory, to obscure ColdFusion pages that you distribute. Although this technique cannot prevent persistent hackers from determining the contents of your pages, it does prevent inspection of the pages. The cfencode utility is not available on OS X.
I would also add that it will be trivial for anyone familiar with ColdFusion to decode anything encoded with this utility because they also provide the decoder.
I'm trying to optimize the pagespeed and trying to figure out what does what.
I've managed to merge and compress js/css files, so Google Pagespeed doesn't bother me with that anymore.
Now I still get the message that I can save some bytes by compressing "http://yourpage.com/" which basically means that the html/php-file itself is not compressed (I think)
Any idea how I can solve this?
Some additional information:
Using TYPO3 6.2.21 with the default .htaccess file enabled.
there is an extension for that: https://typo3.org/extensions/repository/view/sourceopt
That works very well. I use it in many projects.
Apparently I had to remove the '<IfModule mod_filter.c>' condition from my .htaccess file, now it's working. I read that my Apache version might be too old.
I use django-filebrowser and django-storages (w/ s3boto) to add file embed into my tinyMCE implementations for my django projects.
Recently I noticed that there is a bug, django-fileborwser does not save thumbnailed versions of uploaded files properly when using an S3 as a backend.
I stepped through the version creation function (in django-filebrowser), and found what I think to be the bug.
fp is at EOF. Use rewind option or seek() to data start.
which occurs because
# Rewing the file pointer back to 0, before saving to the storage object
tmpfile.seek(0)
isn't called before
# Existing Code
site.storage.save(version_path, tmpfile)
I reported the bug through the issue tracker with my suggested fix. Then the repo's author threw me for a loop and asked:
#fyaconiello shouldn´t that snippet be part of the s3-storage engine (with django-storages) instaed of adding it to the filebrowser?
I guess the file rewind could be done in either library. But which library does it belong in?
Should django-storages make sure all tmpfile pointers are rewound?
is it django-filebrowser's job as it is the implimentor?
See my full issue here
They provide a solution here: https://github.com/sehmaschine/django-filebrowser/issues/40
IMPORTANT: The accepted answer was accepted post-bounty, not necessarily because I felt it was the best answer.
I find myself doing things over and over when starting new projects. I create a folder, with sub-folders and then copy over some standard items like a css reset file, famfamfam icons, jquery, etc.
This got me thinking what the ideal starting template would be. The reason I'm asking is that I'm going through once again and am wondering what I should include in my template so that I don't have to go back in the future and do this all over again with every new site I start.
What I currently have follows:
Project Template Folder
index.html -- XHTML 1.0 Strict Doctype. Meta Tags. CSS/js Files Referenced.
css/
default.css -- Empty. Reserved for user-styles.
960/ -- 960 Grid System for CSS Layouts.
960.css
reset.css
text.css
js/
default.js -- Empty. Reserved for user-scripts.
jQuery/ -- Light-Weight Javascript Framework
jquery-1.3.1.min.js
img/
famfamfam/ -- Excellent collection of png icons
icons/
accept.png
add.png
...etc
I have a similar structure and naming convention but for CSS, I use BluePrint which I find is more extensible. Also prefer jQuery having recently switched from prototype. In addition I have a common.js file that is an extension with custom functions for jQuery.
A /db/ folder with .sql files containing schema definitions. A /lib/ folder for common middle-tier libraries.
I will also have a /src/ folder which will sometimes have raw files such as Photoshop templates, readme's, todo lists etc.
If you have a lot of projects with a lot of static content in common (e.g. jquery, css framework, etc) make yourself a media server to serve all these. Then, instead of creating a bunch of folder structure from a "template" all you do is include the right files in your project's html. If you really want a template, your template becomes one html file instead of a directory structure.
This also gives you an easy way to update the static media for your sites (e.g. moving to the next version of 960). you only have to do it in one place. Of course, you still have to make sure that your updates don't break existing sites! :)
You can make the scheme a bit more complicated if certain projects have overlapping needs but are different from others. Just have a directory at the top level of the server for each setup and to each setup corresponds one html "template". The main idea is to have to deal with only one copy of everything that is common.
You can certainly do this on a small VM (e.g. linode) for $20/mo or a virtual web-server on your current web server. You don't really need a server, for that matter, you just need a folder. However, I think you can have some significant performance gains by having a dedicated media servers. I'd recommend using a fine-tuned apache or nginx for this purpose.
As for site-specific static files, it is also a good idea that they live on the media server and the directory structure would probably be exactly what you have, but they would/should be empty directories.
My web development framework sits in a git repository. Common code, such as general purpose PHP classes gets developed in the master branch. All work for a particular website gets done on a branch, and then changes that will help in future work get merged back into master.
This approach works well for me because I have full revision control of all the websites, and if I happen to fix a bug or implement a new feature while working on a branch I can do the merge, and then everything benefits.
Here's what my template looks like:
/
|-.htaccess //mod_rewrite skeleton
|-admin/ //custom admin frontend to the CMS
|-classes/ //common PHP classes
|-dwoo/ //template system
|-config/ //configuration files (database, etc)
|-controllers/ //PHP scripts that handle particular URLs
|-javascript/
|-tinyMCE/
|-jquery/
|-modules //these are modules for our custom CMS
|-news/
|-mailing_list/
|-others
|-private/ //this contains files that won't be uploaded (.fla, .psd, etc)
|-.htaccess //just in case it gets uploaded, deny all
|-templates/ //template source files for dwoo
I use a similar layout, but with one major exception: all of these directories live under a top-level media/ directory. This is for a few reasons:
This directory is rsync'd to two other servers which handle all of the static media requests.
Having multiple hosts allows some browsers to make more parallel requests for support files.
The media/ directory has its own .htaccess file which strips off a psuedo directory from the path which is the date-time last modified of the image (or whatever).
A custom template tag (I have used this with 2 Django projects, but you could do it in PHP, etc.) generates urls which a) semi-randomly choose one of the media servers, b) add the time-based pseudo directory to the path, and c) give the object an Expires time of now + 10 years.
I think the structure is good. The addition of a few other folders depends on what type of work you are completing.
For freelancing and the like, the addition of PSD folders, client comments would be a nice addition.
A very MS skewed view, but my SOP right now is along the lines of:
documentation/
architecture/ (what you might call code documentation)
communications/ (important client docs)
spec/
whitepapers/
graphics/
*.psd
source/
com.mycompany.projectname.solutionA/
com.mycompany.projectname.solutionB/
com.mycompany.projectname.solutionC/
com.mycompany.projectname.solutionX/ (project in the business sense here)
businesslogic/
*.cs (or whatever)
(further projects - in the visual studio sense)
site/
handlers/ (rarely do I use actual .html these days)
modules/
resources/
img/ (pngs jpegs, gifs whatever)
skin/
icons/
backgrounds/
js/ (compressed when published)
library/ (standard code)
common/ (app specific code)
*.js (app specific code, hopefully nil)
css/
skinX/ (even if there is only "default")
extension.css
base.css
transforms/(always hidden from public by config or build process)
*.xslt
unittests/
mocks/
testmain.cs (or whatever)
thirdparty/
dependencies
I definitely love the idea of having a skeleton template folder like this, but if you use a few different technologies, definitely pay close attention to the structure. My VB.net folder structure has a totally different setup compared to PHP. It sounds like common sense, but I have seen people approach both the same way.
At work we use Code Igniter as a PHP framework for our web applications and have created a new project template which does exactly that: Simple directory structure, Blueprint CSS, jQuery and the Code Igniter application folder, filled with a couple of commonly used libraries (Authentication, some speciales models for often used databases...).
The main motto here is: It's always easier to delete components than to add them. So fill your template up.
(And when I'm starting a new project in my spare time I sorely miss that template...)
I think what you have here is great.... What you've listed is of course all about the public front end of your app. My only addition to this, is to keep all your backend code and source out of the public web space if possible, as the less things you have in the public space, the more secure your app is.
So I'd suggest you take your entire tree, and put it in:
httpdocs/(all you had in your project template folder)
then put all your backend code (e.g. php libraries, sql files, etc) in adjacent subdirectories:
httpdocs/(all you had in your project template folder)
phplibs/
sql/
etc.
And, even for your front end stuff, make sure you don't copy in any example files that may come with your front end libraries, as the examples themselves may have security problems that would allow people to XSS or otherwise compromise your site.
I have been using the following setup for a while now with great results:
/site: This is where my actual working website will live. I'll install my CMS or platform in this directory after the templates are created.
.htaccess (basic tweaks I usually find myself enabling anyway)
robots.txt (so I don't forget to disallow items like /admin later)
/source: Contains any comps, notes, documents, specifications, etc.
/templates: Start here! Create all static templates that will eventually need to be ported into the CMS or framework of /site.
/behavior
global.js (site-specific code; may be broken out into multiple files as needed)
/media: Images, downloadable files, etc. Organized as necessary
/style: I prefer modular CSS development so I normally end up with many stylesheet for each unique section of the website. This is cleaned up greatly with Blender - I highly recommend this tool!
behavior.css (any styling that requires a JS-enabled browser)
print.css (this eventually gets blended, so use #media print)
reset.css (Eric Meyer's)
screen.css (for #media screen, handheld)
/vendor: all 3rd party code (jQuery, shadowbox, etc.)
Blendfile.yaml (for Blender; see above)
template.html (basic starting template; can be copied and renamed for each unique template)
I like OPs as a default start point. your standard template should err on simplicity, with the ability to add complexity only if it's needed.
one addition:
/robots.txt
I'm using Mercurial for development of quite a large C++ project which takes about 30 minutes to get built from the scratch(while incremental builds are very quick).
I'm usually trying to implement each new feature in the new branch(using "hg clone") and I may have several new features developed during the day and it's quickly getting very boring to wait for the new feature branch to get built.
Are there any recipes to somehow re-use object files from other already built branches?
P.S. in git there are named branches within the same repository which make re-usage of the existing object files possible for the build system, however I prefer the simpler Mercurial separate branches model...
I suggest using ccache as a way to speed up compilation of (mostly) the same code tree. The way it works is as following:
You define a place to be used as the cache (and the maximum cache size) by using the CCACHE_DIR environment variable
Your compiler should be set to ccache ${CC} or ccache ${CXX}
ccache takes the output of ${CC} -E and the compilation flags and uses that as a base for its hash. As long as the compiler flags, source file and the headers are all unchanged, the object file will be taken from cache, saving valuable compilation time.
Note that this method speeds up compilation of any source file that eventually produces the same hash. If you share source files across projects, ccache will handle them as well.
If you already use distcc and wish to use it with ccache, set the CCACHE_PREFIX environment variable to distcc.
Using ccache sped up our source tree compilation around tenfold.
A simple way to speed up your builds could be to use a local "build directory" on your disk. This way you can checkout into this directory and start the build. The first time it will take the full time, but after that it will (hopefully) only rebuild the files where the source code changed.
My Localbranch extension was designed partly around this use case. It uses a single working directory, but I think it's simpler than git. It's essentially a mechanism for maintaining multiple repository clones under one working directory, where only one is active at a given time.
Woops, I missed your P.S. where you don't like having multiple named branches in the same repo and that you prefer separate clones.. sorry about that.
I too have somewhat large C++ projects and the clone-per-feature workflow didn't work for me very well. Firstly, I had to close down my Vim session and then reopen (many of the same) files once I've created the clone. Secondly, like you said, a lot of code must be recompiled unnecessarily. Thirdly, I have to keep track of where I've pushed to and pulled from - gets confusing when you start a new feature and then get sidetracked onto a new one. Before you know it you have many clones and not sure which ones need to be pushed back to your main.
You definitely don't want to use named branches (as I'm sure you know) to handle this as they are quite permanent.
What you need are bookmarks: https://www.mercurial-scm.org/wiki/BookmarksExtension
Bookmarks allow you to create lightweight (and otherwise anonymous) branches per feature by facilitating the naming of heads in your repo. These heads would normally be unnamed and you would have to look at the output of 'hg log' or use some graphical tool to find the revision numbers for the tip of your feature-branch. With bookmarks you can name them descriptive names like 'my-cool-feature' or 'bugfix-392'.
If you like the idea of bookmarks, I'd also recommend my own extension called 'tasks': http://bitbucket.org/alu/hgtasks. This extension works like bookmarks but adds some more functionality. It allows you created feature-branches (now called tasks) and suppress the pushing of incomplete tasks. This is handy when you have a few feature-branches at once. You may not be ready to push your 'my-cool-feature' task, but 'bugfix-392' is ready to go. Because tasks track a set of changesets (and not just one 'tip' changeset) there are some things you can do with tasks that you can't with bookmarks. See an example workflow here: http://x.zpuppet.org/2009/03/09/mercurial-tasks-extension/.
Mercurial also has local named branches, see the hg branch command.
If you insist on using hg clone to do branchy development, I guess you could try creating a folder link (shortcut under windows) in your repo to a shared obj folder. This will work with hg clone, but I'm not sure your build tool will pick it up.
Otherwise, you probably keep all your repos in one folder - just put your obj folder there (it shouldn't be under source control anyways, imo). Use relative paths to refer to it.
A word of warning: many .o symbol tables (or equivalent) contain the full path name of the source file. If that other file changes (or if the path is not visible from the new directory) you may encounter weirdness when debugging.