Extract data from excel and create a custom CSV - ColdFusion - coldfusion

I am receiving excel files from our client and I need to create a csv file from them. My issue is that the values in the excel files do not match our standards to create the csv file just by copying it. I need to get the values and put them in the correct order while I'm creating the file. I read through the livedocs for CF but I couldn't find anything the worked.
I was thinking to create a struct and pull the data out in it and then create the csv file based on my struct but I haven't done anything like this before and I am not sure if it is even possible. I also thought of using listGetAt instead of the struct but still haven't tried it.
Anyone did something like this before? I'm trying to hardcode as less values as possible because I see this becoming a future problem if we get a second client with the same issue.
Update (I changed my code around a lot the past few days so this is what I have at the moment)
<cfset DataDirectory = "E:\testfolder\test.xlsx">
<cfspreadsheet action="read" src="#DataDirectory#" excludeHeaderRow="true" headerrow="1" query="queryData">
<cfquery name="contact" datasource="#ds#">
select ClientBrandID
from ClientBrands
where surveyReferralID IN
(select surveyReferralID from clientAdmin where username = '#res#')
</cfquery>
<cfscript>
dataFile = structNew();
StructInsert(datafile,"ClientBrandID",contact.ClientBrandID);
StructInsert(datafile,"surveyType", queryData.surveyType);
</cfscript>
<cfscript>
///We need an absolute path, so get the current directory path.
theFile= "E:\testfolder\NewTest.csv";
//Create a new Excel spreadsheet object and add the query data.
theSheet = SpreadsheetNew("PatientData");
SpreadsheetAddRow(theSheet, "ClientBrandID,SurveyType,Location,ClientContactID,FirstName,LastName,HomePhone,WorkPhone,CellPhone,Email"); <!---This is the header of the new CSV --->
SpreadsheetAddRows(theSheet, "datafile.clientbrandid, datafile.surveytype"); <!--- This is my latest failed attempt. Tried to pass in data from my struct, probably the quotes are the issue but haven't tried it without them --->
</cfscript>
<!--- Write the spreadsheet to a file, replacing any existing file. --->
<cfspreadsheet action="write" filename="#theFile#" format="csv" name="theSheet" overwrite=true >
All of these operations will need to be in a cfloop in order to go through all of the files in my test folder. Right now I am hardcoding a single file to fix the issue at hand before I do anything else. Also, I am missing another loop in between that will need to go through all the values of the file. It should be something like <cfloop query='queryData'> which is the query that Im getting from cfspreadsheet.

You could make it accept any order but you'll still need to hard code the expected column headers into an array or something and considering it is just for your internal use I would just add the data back to the CSV in the order it's supposed to be in. Last time I made a spreadsheet from a query I did it like this. Perhaps it will help.
<cfscript>
sMySpreadSheet = spreadsheetNew();
column = 1;
<!--- header row --->
<!--- Remember what is expected from SpreadsheetSetCellValue --->
<!--- SpreadsheetSetCellValue(spreadsheetObj, value, row, column) --->
spreadsheetSetCellValue(sMySpreadSheet ,"First Name",1,column); column++;
spreadsheetSetCellValue(sMySpreadSheet ,"Last Name",1,column); column++;
spreadsheetSetCellValue(sMySpreadSheet ,"DOB",1,column); column++;
spreadsheetSetCellValue(sMySpreadSheet ,"Phone Number",1,column); column++;
spreadsheetSetCellValue(sMySpreadSheet ,"Number of Kids",1,column); column++;
<!--- data rows --->
<!--- if you're not using a header row with real titles you can just use the
default col_x
example: spreadsheetSetCellValue(sMySpreadSheet , queryData.col_1[q], q+1, column); column++; --->
for(q=1; q LTE queryData.recordCount; q++){
column = 1;
spreadsheetSetCellValue(sMySpreadSheet , queryData.first[q], q+1, column); column++;
spreadsheetSetCellValue(sMySpreadSheet , queryData.last[q], q+1, column); column++;
spreadsheetSetCellValue(sMySpreadSheet , queryData.dob[q], q+1, column); column++;
spreadsheetSetCellValue(sMySpreadSheet , queryData.phoneNumber[q], q+1, column); column++;
spreadsheetSetCellValue(sMySpreadSheet , queryData.kidCount[q], q+1, column);
}
<!--- make it purdy (optional) --->
spreadsheetFormatRow(queryData, {fgcolor="light_cornflower_blue"},1);
</cfscript>
If you want to simply add to a CSV you can do something like this (wrap your stuff in " if you have them qualified):
<cfsetting enableCFoutputOnly = "Yes">
<cfsaveContent variable = "myCSV">
<cfset newline = #chr(13)#&#chr(10)#>
<cfoutput>First Name,Last Name,DOB,Phone Number,Number of Kids#newline#</cfoutput>
<cfoutput query="queryData">#queryData.first#,#queryData.last#,#queryData.dob#,#queryData.phoneNumber#,#kqueryData.idCount##newline#/cfoutput>
</cfsaveContent>
</cfsetting>
<cffile action = "append" file = "[your file]" output = "#myCSV#">
If cfsavecontent causes white spaces issues and cfsetting doesn't help, here is another alternative.
<cfset myCSV = "First Name,Last Name,DOB,Phone Number,Number of Kids#newline#">
<cfloop query="queryData">
<cfset myCSV &= "#queryData.first#,#queryData.last#,#queryData.dob#,#queryData.phoneNumber#,#queryData.kidCount##newline#">
</cfloop>

Here are some suggestions based on the edited question. I'm making them even though the edited question does not make it clear what you are trying to achieve.
Regarding, "All of these operations will need to be in a cfloop in order to go through all of the files in my test folder", you can use cfdirectory on that folder. It will return a ColdFusion query through which you can loop.
The rest of your code looks more complicated than it needs to be. Reading the spreadsheet gives you a query object - so far so good.
Your database query, named contact looks supicious. You refer to a variable named res but it's not clear where that comes from. If it comes from the spreadsheet, then a value list might be more appropriate. By the way, use query parameters.
It seems you want your csv file to combine data from the spreadsheet and the database query. If that's the case, a query of queries might be useful.
If it were me, I'd use cffile action="write" to create my csv file. I would also surround each value with double quotes in case any of them contained commas.

Related

ColdFusion 11 DateFormat

I am moving one of our applications from ColdFusion 9.01 to ColdFusion 11 and encountered a situation where I cannot get the date formatted the way I want it using "DateFormat". I read through the docs since things have changed in CF versions, but I honestly can't figure out why this isn't working. It worked beautifully in CF 9. I know it's probably something very easy, but I am just not seeing it.
The query (Oracle DB) provides me a list of the last 30 days and the loop is simply to reformat the date output from "2014-07-01 00:00:00.0" to a more friendly looking display of 01-Jul-2014 except that I cannot get it to format as "dd-mmm-yyyy" it just spits back the original output from the query. I hard coded the date where normally there would be a cfquerparam. Any ideas?
<cfquery name="qryDateArray" datasource="#request.db#">
select trunc(to_date('07/01/2014', 'mm/dd/yyyy') + 1 - rownum) as ref_date
from dual connect by rownum <= 30
</cfquery>
<cfloop from="1" to="#qryDateArray.recordcount#" index="j">
<cfset qryDateArray.ref_date[j] = DateFormat(qryDateArray.ref_date[j], "dd-mmm-yyyy")>
</cfloop>
<cfoutput>
<cfdump var="#qryDateArray#">
</cfoutput>
I could not test this on CF11 since I do not have it handy. I did verify that your code though returns results as you explained when I ran it on my CF10 environment here. So what you can do is add a column to the query object and define it as a varchar and add your formatted data to that. This in turn dumped out the formatted dates.
<cfquery name="qryDateArray" datasource="#request.db#">
select trunc(to_date('07/01/2014', 'mm/dd/yyyy') + 1 - rownum) as ref_date
from dual connect by rownum <= 30
</cfquery>
<cfset aryData = [] />
<cfloop from="1" to="#qryDateArray.recordcount#" index="j">
<cfset ArrayAppend(aryData, DateFormat(qryDateArray.ref_date[j], "dd-mmm-yyyy")) />
</cfloop>
<cfset QueryAddColumn(qryDateArray, "STRDATE", "VarChar", aryData) />
<cfoutput>
<cfdump var="#qryDateArray#">
</cfoutput>
If dependent on the query column names then could use something like Ben's method explained here to do some renaming of the columns: http://www.bennadel.com/blog/357-ask-ben-changing-coldfusion-query-column-names.htm
It'd be great if you'd given us a portable test case rather than one that relies on your database, but I suspect it is because ColdFusion has become more rigid with its type management of query columns.
So CF considers your ref_date column to be of type date, so when you try to put the formatted string back into the query column, CF tries (and succeeds) to convert the string back into a date.
Aside:
I have to wonder why you don't format the data string in the DB from the outset, and just return it the way you need it, rather than returning something else, then looping over the thing to adjust it..?

Images and formatting not working on a multi-tabbed XLSx file generated with ColdFusion

The code below will generate the file with two notable errors:
The SpreadSheetSetColumnWidth appears to stop working after 21 lines on the second tab
The image does not appear on the second tab
Hopefully this is just a rookie mistake and not something along the lines of the date formatting issue. Any help or constructive criticism to fix/improve would be greatly welcomed.
<!--- We cannot directly reference the image --->
<cfimage source="img/A2LA_Logo.jpg" name="A2LA_Logo">
<!--- We need an absolute path, so get the current directory path. --->
<cfset theFile=GetDirectoryFromPath(GetCurrentTemplatePath()) & "Final_Report.xlsx">
<cfscript>
/*********************************************************************** Page 1 ***********************************************************************/
//Create a new Excel spreadsheet object - SpreadsheetNew("Tab name", "yes=2007+ no=2003");
Tab1 = SpreadsheetNew("Final Report Page 1","yes");
SpreadSheetSetRowHeight(Tab1,1,45);
SpreadsheetSetCellValue(Tab1,"Final Test Report",1,1); SpreadsheetMergeCells(Tab1,1,1,1,10);
SpreadsheetAddImage(Tab1, A2LA_Logo,"jpg","1,12,2,20");
SpreadsheetSetCellValue(Tab1,"Confidential",1,21); SpreadsheetMergeCells(Tab1,1,1,21,30);
</cfscript>
<cfscript>
/*********************************************************************** Page 2 ***********************************************************************/
//Create a new Excel spreadsheet object - SpreadsheetNew("Tab name", "yes=2007+ no=2003");
Tab2 = SpreadsheetNew("Final Report Page 2","yes");
SpreadSheetSetRowHeight(Tab2,1,45);
SpreadsheetSetCellValue(Tab2,"Final Test Report",1,1); SpreadsheetMergeCells(Tab2,1,1,1,10);
SpreadsheetAddImage(Tab1, A2LA_Logo,"jpg","1,12,2,20");
SpreadsheetSetCellValue(Tab2,"Confidential",1,21); SpreadsheetMergeCells(Tab2,1,1,21,30);
</cfscript>
<!--- There must be a better way --->
<cfscript>
for(index=1; index LTE 30; index++) {SpreadSheetSetColumnWidth(Tab1,index,3);}
for(index=1; index LTE 30; index++) {SpreadSheetSetColumnWidth(Tab2,index,3);}
</cfscript>
<!--- Write the spreadsheet to a file, replacing any existing file, then append the other tabs. --->
<cfspreadsheet action="write" filename="#theFile#" name="Tab1" overwrite=true>
<cfspreadsheet action="update" filename="#theFile#" name="Tab2">
<!--- Needs to be at the bottom --->
<cfheader name="Content-disposition" value="attachment;filename=Final_Report.xlsx">
<cfcontent type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" file="#theFile#">
<!--- Clean up your mess after it is served to the browser --->
<cffile action="delete" file="#theFile#">
(From the comments)
The images show up when saved separately. So I am guessing it is an update bug. My usual recommendation is "do not use update if you can avoid it". Create a single workbook, add multiple sheets and save it with a single write action.
Update: As for why, in your original code, you are creating two separate workbook objects. Conceptually think of it like creating two separate Excel files, with one sheet each. Then trying to merge them into one file:
c:/path/to/MyWorkbookNamedTab1.xlsx
| Sheet1 /
c:/path/to/MyWorkbookNamedTab2.xlsx
| Sheet1 /
That is different than creating a single workbook, with multiple sheets (like in the link above):
c:/path/to/MySingleWorkbook.xlsx
| Sheet1 / Sheet2 /
There is no easy way to merge separate files (or workbooks). You basically have to copy everything from one workbook into another (which is a lot more involved than it sounds). That is what action=update does. Somewhere in the copying process it loses one of the images. I am not sure if it that is due to a CF bug one in POI.
In any case, the simplest solution is to avoid the need for copying altogether. If you stick with a single workbook (and add multiple sheets to it) then there is no need for copying or merging. Everything is already contained in one workbook. So all action=write has to do is save it to disk, exactly "as is".

Export to Excel and formatting cells using ColdFusion 10

I have a query which I'd like to output in a spreadsheet in Excel. I'd like some of the cell columns to be formatted in a certain way, which is thousands grouping and in Number format so that sums, additions etc can be done on that row without any further alteration.
I have read through the documentation but it has left me a bit confused on how to output to Excel in the first place.
I started out with a comment but it will be easier to read as an answer.
What have you tried? Have you read through the CFSpreadsheet documentation? Should be pretty straight forward. One of the parameters to the CFSpreadsheet tag is 'query'. Why not start with that and see how it formats the columns for you by default to see what needs tweaking.
Here is an example taken directly from the referenced documentation page:
<cfquery name="courses" datasource="cfdocexamples">
SELECT CORNUMBER, DEPT_ID, COURSE_ID, CORNAME
FROM COURSELIST
</cfquery>
<cfscript>
//Use an absolute path for the files. --->
theDir=GetDirectoryFromPath(GetCurrentTemplatePath());
theFile=theDir & "courses.xls";
//Create an empty ColdFusion spreadsheet object. --->
theSheet = SpreadsheetNew("CourseData");
//Populate the object with a query. --->
SpreadsheetAddRows(theSheet,courses);
</cfscript>
<!--- Write the sheet to a file --->
<cfspreadsheet action="write" filename="#theFile#" name="theSheet" sheetname="courses" overwrite=true>
See the documentation for SpreadsheetFormatColumn, SpreadsheetFormatColumns, SpreadsheetFormatRow and SpreadsheetFormatRows to read about formatting particular cells.
You just need to use the cfspreadsheet tag to create the file, and you can format the cells with the spreadsheetFormat* functions. You can find an example of how to do this at Ray Camden's site.

Coldfusion Query loop works in cf10 but not 9

Why does the following work in CF10 but not CF9?
<cfset out="">
<cfif isQuery( arguments.values ) >
<cfloop query="#arguments.values#" >
<cfset out = '#out#<option value="#value#">#label#</option>'>
</cfloop>
</cfif>
CF9 states that "Complex object types cannot be converted to simple values." for the line containing the cfloop. I'm using the Coldbox framework and it's debugger information shows that arguments.values is a query with Label & Value columns.
Prior to CF10, the query attribute of cfloop can only be a string - the name of the query - not the variable itself.
So, when you put #arguments.values# it is trying to convert the complex query object to a string, to obtain a name, which is where the error comes from.
It works in CF10 because the attribute has been updated to also allow a query value.
side notes:
This line of code can be simplified:
<cfset out = '#out#<option value="#value#">#label#</option>'>
to:
<cfset out &= '<option value="#value#">#label#</option>'>
Also you very likely should be using HtmlEditFormat* on at least label, and perhaps value too.
*(or encodeForHtml if it only needs to work in CF10+)

cffeed function: show just most recent post?

We have a function to pull RSS feeds and display the post on a ColdFusion page using this code:
<cfset rssUrl = "rss1">
<cffeed action="read" source="#rssUrl#" query="fitness" properties="info">
<cfset rssUrl2 = "rss2">
<cffeed action="read" source="#rssUrl2#" query="nutrition" properties="info">
<cfif #fitness.PUBLISHEDDATE# gt #nutrition.PUBLISHEDDATE#>
<cfset entries="fitness">
<cfelse>
<cfset entries="nutrition">
</cfif>
Output done via:
<cfoutput query="#entries#">
Problem is, the RSS feed has several posts and we only want to show one. Any thoughts on how to get it to pull and display only the most recent post? (We want the feed to have multiple posts, so right now our non ideal solution is to set maximum posts per feed to 1)
cfoutput/query=".." will go over an entire query. If you only want to do the first row, use:
<cfoutput>
Title from row 1: #somequery.title[1]#
</cfoutput>
Basically - array notation on the column. Make sense?
There's nothing wrong with Ray's answer, but here are some other options.
<cfoutput query="#entries#" maxrows="1">
Offers the least disruption to your existing code and, should you decide to change the number of rows displayed (like, via a user setting) it's an easy change.
OR
If you copy the query object rather than the query name (which isn't actually a copy but a copy by reference)
<cfset entries = fitness>
instead of
<cfset entries = "fitness">
you can do this
<cfoutput>
#entries.columnName1#
#entries.columnName2#
<!--- etc. --->
</cfoutput>
which will, by default, display only the first row of the query.