I have the following log configuration.
<RollingRandomAccessFile name="RollingReqAppender"
fileName="/usr/test/req.log"
filePattern="/usr/test/req.log.%d{yyyy-MM-dd-HH}.%i.gz"
immediateFlush="true" append="true">
<PatternLayout>
<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS Z}%m%n</Pattern>
</PatternLayout>
<Policies>
<SizeBasedTriggeringPolicy size="500 MB"/>
<TimeBasedTriggeringPolicy interval="1" modulate="true" />
</Policies>
</RollingRandomAccessFile>
When the log is rolled over (based on time or size), it is gzipped. Gzipping the file takes some time (seconds to minutes depending on load). The problem is that the file is gzipped in place, and while it is being gzipped, the partially gzipped file is stored in the same folder with the same name as the final gzip file. e.g.: /usr/test/req.log.2017-01-03-02.1.gz
This causes problems for the file's consumer, which attempts to process the partial file. I want to be able to distinguish between a partial gzipped file and a completed gzipped file. For example, I would like to setup a configuration that while the file is being gzipped, it is written with a different name e.g.: /usr/test/req.log.2017-01-03-02.1.temp.gz and once it is completely gzipped the name is changed to the final name /usr/test/req.log.2017-01-03-02.1.gz
How can I achieve this?
Log4j2 currently does not behave like you describe. So there is no configuration to make this happen.
Please raise a feature request on the Log4j2 JIRA issue tracker. The fastest way to achieve this is to provide a patch with unit test.
This feature was added to log4j2 version 2.9.0 https://issues.apache.org/jira/browse/LOG4J2-1766
Related
I'm serving unversioned files via fossil's uv function. Now, this works fine for files without file extension and for archives. But I need to serve a .txt file. The problem now is that it gets delivered as a HTML page including the fossil web layout around it.
Is there a way to tell fossil to not do that, and instead deliver it as a raw .txt file?
You can specify a mimetype parameter on the URL. For example, mimetype=application/octet-stream will cause it to be offered as download.
For example, instead of https://www.fossil-scm.org/index.html/uv/download.html, you’d put https://www.fossil-scm.org/index.html/uv/download.html?mimetype=application/octet-stream.
Fossil reacts to the following mimetypes by putting headers around them:
text/x-fossil-wiki
text/x-markdown
text/html
text/plain
Unfortunately, all other mimetypes appear to lead to the browser downloading the unversioned file instead of displaying it.
If that's a problem, you could try a mimetype of text with no suffix.
Otherwise, you can post on Fossil's support forum. Either as a question or as a feature request. :-)
As an example, I'm currently uploading items directly to an S3 bucket using a form. While I was testing, I didn't specify any expected filenames or extensions.
I uploaded a .png which produced this direct link:
https://s3-us-west-2.amazonaws.com/easyhighlighting2/2015-07-271438019663927upload94788
When I place this inside an img tag, it displays on a web page properly.
My question is, without an extension, how would my browser know what type of file it's loading? Inside the bucket, the file's metadata isn't even filled out.
Is there any way to get that file extension, programmatically?
I'm ready to try any clientside methods available; my server-side language is ColdFusion which is somewhat limiting, but I'm open to suggestions for that as well.
Okay, so after some more extensive digging, I found a method of retrieving the file's type that was only added since CF10 was released; that would explain the lack of documentation.
The answer lies in the FileGetMimeType function.
<cfset someVar = "https://s3-us-west-2.amazonaws.com/easyhighlighting2/2015-07-271438019663927upload94788">
<cfset FileType = FileGetMimeType(someVar)>
<cfoutput>#FileType#</cfoutput>
This code would output image/png - which is correct and has worked for every filetype I have tested thus far.
I'm surprised this kind of question hasn't popped up before, but this appears to be the best answer, at least for users of CFML.
Edit:
ColdFusion accomplishes this by either reading the contents of a file, or by trusting its extension. An implicit attribute, 'strict', is used in this function. If true, it reads the file's contents. If false, it uses the provided extension.
True is the default.
Link:
https://wikidocs.adobe.com/wiki/display/coldfusionen/FileGetMimeType
Check the Content-Type HTTP response header returned by Amazon S3.
For example, curl -I https://s3.amazonaws.com/path/to/file fetches only the headers.
I'm want to use CFFILE upload to detect only .txt file. I've tried to use file.clientfileext to detect the extension. When TXT is detected, I'm showing a pop up error message to users and delete the file. But I was told I should not even allow user's file to reach our server.
Then I use CFFILE accept attribute to only accept text/plain. This should do it but unfortunately on my test when I tried uploading non text file I got ColdFusion error:
The MIME type of the uploaded file application/pdf was not accepted by
the server. Only files of type text/plain can be uploaded. Verify
that you are uploading a file of the appropriate type.
I tried to use cftry and cfcatch but I still get the same error, this mainly due to the MIME Type that I don't know when the file is being uploaded by the browser.
I also found the same question in this forum and tried the suggested answer, it did not work, still got the same error message (see below)
I also found another posting in this forum that do not suggest the use of CF "accept" attribute. This link is provided for a further detail explanation: http://www.petefreitag.com/item/701.cfm
So my question is, since I'm still using CF8, I actually don't have many options to prevent my users from uploading other than .txt file securely?
If I can't use the accept attribute of the CFFILE, can I at least secure my file upload functionality by doing the following? but is doing it this way safe enough?
Upload the file to a temp folder that is not under the root dir
verify the file extension
change the file name even if the extension is detected to be a .txt
move the file to the destination file under the root dir
Even if I do these steps, I have to allowed the file to reach our server, the order is to NOT allow the file to reach our server.
Below is the answer/suggestion from previous question. But it doesn't work when I tested it:
<CFTRY>
<cflock name="write_lock" type="Exclusive" timeout="120">
<cffile action="upload" filefield="filepath" destination="#DestDir#"
nameconflict="Overwrite" attributes="Archive">
</cflock>
<CFCATCH>
<cfif FindNoCase("not accepted", cfcatch.Message)>
<script>
$(function(){
alert("Only the following file types are allowed: .jpg, .gif, .bmp,
.png.");
});
</script>
<cfabort />
<cfelse>
<!--- looks like non-MIME error, handle separately --->
<cfdump var="#cfcatch#" abort />
</cfif>
</CFCATCH>
</CFTRY>
I think your steps are reasonable if you don't like using the Accept attribute for validation. FYI you can set accept to .txt instead of the MIME types. The MIME type was determined by the client so it's safer to check the extension anyway.
The exception thrown by cffile failing attribute validation may not have a type, so the code you posted tried to detect it with FindNoCase() by looking at the exception's message. You can dump the exception out and find out why the FindNoCase() failed to catch the exception.
Make sure you treat whatever uploaded as something potentially malicious and do not process them (e.g. cfinclude them). Forcing the file extension to be .txt should be safe enough, but I'll let other security experts charm in.
You can use the below code:
<cffile action="upload" filefield="BidDoc"
destination="C:\upload\"
nameconflict="makeunique"
accept="text/plain">
The other mime types which you may use are:
application/pdf
application/msword
application/vnd.ms-excel
application/vnd.openxmlformats-officedocument.wordprocessingml.document
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
As discussed in this answer, there really is no 100% fool-proof way.
If you don't want to trust the "accept" attribute, I would suggest allowing the user to upload the file and then checking the mime type of the uploaded file using the cffile.contentType property. Check against whatever mime types you wish to allow/restrict and reject with the appropriate message. You may also choose to employ a check of the file extension as an added layer of error checking.
It must be noted that like file extensions, mime types can not be 100% trusted to be accurate as they can be edited by the user. But using a combination of checks you can be reasonably that most files uploaded are of the correct type.
Coldfusion will not prevent a file from being uploaded to a server. You can set a maximum file size but this is processed during the upload. The cffile tag kicks in after the file is uploaded. Furthermore it is rather difficult to really determine if a file is a text file or a jpg, exe, rar etc file. The following q & a may help:
Determining binary/text file type in Java?
In my opinion it is best to follow the tips given by pete freitag and use a java class to determine the file type. Then you can delete all non text files.
I am trying to download a binary file from a http: server. I am using the functions InternetOpenUrl() and then InternetReadFile() to download the file. Is it possible to know the file name before downloading?
What I am doing now to get the file name is- Once the download is complete, using GetFileVersionInfo() and from the buffer i am getting the OrginalFilename, then renaming the file to the OrginalFilename.
Is there any other way to get the file name before downloading?
Thanks
Vinod
Look at HttpQueryInfo. Look at the Content-Type and Content-Disposition headers.
You may have to use HTTP_QUERY_CUSTOM to get raw content-type if it just returns e.g. "text/plain".
To get all the headers (and thereby work out which one contains the information you want) you can use HTTP_QUERY_RAW_HEADERS_CRLF.
I have a JQUERY file upload plug-in which allows users to upload files to the Coldfusion server. The plugin submits the files to the server in a way that requires me to use GetHttpRequestData() for the files contents. Here's what I have so far in terms of handling the file data:
<cfparam name="URL.qqfile" type="string">
<cfset x = GetHttpRequestData()>
<cffile action="write" output="#x.content#" file="c:\temp\#URL.qqfile#">
This works, which is nice, but I can't seem to take this to the next step.
What I want to happen next is:
A. Determine the file's extension.
B. If it is an accepted ext defined by my app, (JPG,PNG,PDF, DOC, DOCX, etc...) upload it to the correct directory on the server. Then delete the temp file above
C. Use CFIMAGE to make a thumbnail if the file uploaded was an Image
How can I take the above through steps A-C with the GetHttpRequestData problem?
Thanks
A few tips:
Have a look at the result structure of GetHttpRequestData() via <cfdump>.
Pull out the necessary headers by accessing this struct. The Content-Type header usually contains the stuff you want to know. You can use the List functions (i.e. ListLen(), ListFirst(), ListLast(), ListRest() with appropriate delimiter chars) to easily parse the string.
Always use StructKeyExists() to safeguard against missing struct parts. Never take for granted anything that "typically" seems to be in this struct.
Don't blindly trust file extensions or the Content-Type header. Also look into the first few bytes of the uploaded file and compare them against a white list to confirm the file type.
Have a look at <cffile action="upload">.
Optionally, perfom a drive space test to assess if the uploaded data does not clog the server, or enforce limits in another way that suits you.
Read through the documentation of <cfimage>. It can't be that hard to use it to make thumbnails.