How can I write R ggplot charts into palantir foundry file system? - cloud-foundry

looking for a way to use foundry code repository to write a visualisation created via an R-Script using ggplot2 into the foundry file system.

Great question! This is answered in the documentation here: https://www.palantir.com/docs/foundry/code-workbook/transforms-unstructured/#example-saving-a-plot-to-a-pdf
It is possible to write to an output FileSystem. This can be useful to write non-tabular data formats including images, PDFs, text files, and so on.
Call new.output() to instantiate a TransformOutput. Learn more about the FileSystem API.
You can only write files using TransformOutput in nodes that are saved as datasets. You cannot write files using TransformOutput in the console.
Here is an example with a scatterplot (saved to a PDF to maintain good resolution):
plot_pdf <- function() {
library(ggplot2)
theme_set(theme_bw()) # pre-set the bw theme
data("midwest", package = "ggplot2")
# Scatterplot
gg <- ggplot(midwest, aes(x=area, y=poptotal)) +
geom_point(aes(col=state, size=popdensity)) +
geom_smooth(method="loess", se=F) +
xlim(c(0, 0.1)) +
ylim(c(0, 500000)) +
labs(subtitle="Area Vs Population",
y="Population",
x="Area",
title="Scatterplot",
caption = "Source: midwest")
output <- new.output()
output_fs <- output$fileSystem()
pdf(output_fs$get_path("my pdf example.pdf", 'w'))
plot(gg)
}

Related

Wrong data coming through on Windows, but not Mac R-Studio Shiny

When I pull in data from a csv:
descriptions <- read.csv("descriptions.csv")
on a Windows machine, and use outputText
output$textDesc <- renderText({ descriptions[1,1] })
The output shows a "1".
When I use the same code on a Mac, I get the data that is in descriptions[1,1] which is a string of information that I want displayed.
How can I get the text in descriptions[1,1] to show up on the Windows machine?

OPENCV - How to use Haar cascade Github xml files in OpenCV?

I am starting to learn object detection in OpenCV 3.4.2 (.Net C++ 2017).
I am very interested in detecting strawberries in pictures (at the moment I am really interested in detecting just strawberries). I know OpenCV has some pre-trained Haar cascade files in OpenCV directory, but there are not .xml files for strawberries (there are for body parts instead).
So I decided to search on Google, to try to find trained strawberry Haar cascade .xml files. I found this .xml file XML strawberry file but I get error -49 when I try to execute the program. I have executed correctly the program using OpenCV files, but I cannot execute correctly when I try with GitHub XML file.
I have found this thread here on StackOverflow StackOverFlow thread about GitHub XML files in OpenCV and a user claims that it's not possible to use GitHub XML files into OpenCV.
My question is about if there is a way to use the XML GitHub file I have posted in this thread in OpenCV or I need to train my own XML file? I would like to use GitHub file.
Edit(1)
I have found this link strawberry detection in OpenCV where, if you look at the source code, it seems that the same strawberry_classifier.xml is being used. I don't know if the name of the file is just a coincidence (Github filename and the filename shown in the source code of the 3rd link are exactly the same). At least it seems that the programmer (from the 3rd link) has obtained some results while using the (apparently) same .xml file that I want to use. But I don't know how to use that strawberry_classifier.xml file.
Python dev here,
I'm late, but in case anyone still wants to see an answer:
The classifier from GitHub works perfectly fine as shown in this Python code (Sorry, I didn't do it in C++, but I think it won't be much different)
The script uses your webcam as the image source. You can show the webcam some images of strawberries, and it will recognize it:
import cv2 #import library
#define Haar Cascade Classifier
Strawberry_Classifier = cv2.CascadeClassifier(r"C:\Users\Strawberry.xml")
VideoCapture = cv2.VideoCapture(0) #capture video from camera
#set video size
VideoCapture.set(3, 540)
VideoCapture.set(4, 360)
while True:
#Connect video and convert
Connection_Success, Video = VideoCapture.read() #returns a bool and video array in one tuple (sucess, video array)
RGB_video = cv2.cvtColor(Video, cv2.COLOR_BGR2RGB) #converts to suitable format
Detect_Strawberry = Strawberry_Classifier.detectMultiScale(RGB_video, 1.3, 13) #MODIFY THIS FOR LESS/MORE DETECTION ACCURACY
#Detect Strawberry
for(x,y,w,h) in Detect_Strawberry: #x,y width, height
cv2.rectangle(Video, (x, y), (x + w, y + h), (0, 255, 0), 3) #Put a rectangle around Strawberry
cv2.imshow("Window", Video) #show video
#quit if q is pressed, quit
QuitKey = cv2.waitKey(30)
if QuitKey == ord("q"):
VideoCapture.release()
cv2.destroyAllWindows()

LibXL: excel cross-sheet formulas not updated

I'm dealing with a problem with libXL and Office365.
I created an Excel file with Office 365: a simple formula which shows the content of a cell from another sheet. Then I proceeded to write something in that source cell through libXl. When I open the output file the formula is not calculated until I press CTRL+ALT+SHIFT+F9.
If I create the xlsx file from Office 2013 then the formula is correctly updated.
Couldn' find anything on their website whether O365 is supported or not.
Here's the code to reproduce the issue, (I can provide the two input xlsx files if needed):
#include "stdafx.h"
#include "libxl.h"
using namespace libxl;
int main()
{
Book* book = xlCreateXMLBook();
// xlsx file created by Office 2013
if (book->load(L"office2013.xlsx"))
{
Sheet* sheet = book->getSheet(0);
if (sheet)
sheet->writeNum(2, 2, 42);
book->save(L"okay.xlsx"); // works correctly when opened
}
// xlsx file created by O365
if (book->load(L"office365.xlsx"))
{
Sheet* sheet = book->getSheet(0);
if (sheet)
sheet->writeNum(2, 2, 42);
book->save(L"bugged.xlsx"); // must press CTRL+ALT+SHIFT+F9 to see '42' in the second sheet
}
book->release();
return 0;
}
This is the source sheet (number 42 written by the above code): https://i.stack.imgur.com/hp7Ti.png
This is the not working formula (written in Excel): https://i.stack.imgur.com/BhGW2.png
Thank you
Do you have auto calculate on in the Office 365 Excel?
Check this article: https://support.office.com/en-ie/article/change-formula-recalculation-iteration-or-precision-f38c7793-0367-41ce-b892-dfe54946bd76#__toc305944076
I have been fooled in some vbs-script I've created for Excel , with calculation off as default for some Office 365 installations.
Check out formulars -> Calculation and check if everything is automatic on the workbook there.
Edit: Please note that I've zero experience with LibXL, but plenty in vbs for Excel. I also program c++, but I find vbs as a quicker way to get in goal for my projects :)
May this might help setting calculation on with
Calculation = xlCalculationAutomatic
then setting off when required with
Calculation = xlCalculationManual

Tensorboard: No graph definition files were found.

In my Python code I execute
train_writer = tf.summary.FileWriter(TBOARD_LOGS_DIR)
train_writer.add_graph(sess.graph)
I can see 1.6MB file created in E:\progs\tensorboard_logs (and no other file)
but then when I execute
tensorboard --logdir=E:\progs\tensorboard_logs
it loads, but says: "No graph definition files were found." when I click on Graph.
Additionally, running tensorboard --inspect --logdir=E:\progs\tensorboard_logs
displays
Found event files in:
E:\progs\tensorboard_logs
These tags are in E:\progs\tensorboard_logs:
audio -
histograms -
images -
scalars -
Event statistics for E:\progs\tensorboard_logs:
audio -
graph
first_step 0
last_step 0
max_step 0
min_step 0
num_steps 1
outoforder_steps []
histograms -
images -
scalars -
sessionlog:checkpoint -
sessionlog:start -
sessionlog:stop -
This is TF 1.01 or so, on Windows 10.
I had similar issue. The issue occurred when I specified 'logdir' folder inside single quotes instead of double quotes. Hope this may be helpful to you.
egs: tensorboard --logdir='my_graph' -> Tensorboard didn't detect the graph
tensorboard --logdir="my_graph" -> Tensorboard detected the graph
In Tensorflows dealing with graphs, there are three parts:
1) creating the graph
2) Writing the graph to event file
3) Visualizing the graph in tensorboard
Example: Creating graph in tensorflow
a = tf.constant(5, name="input_a")
b = tf.constant(3, name="input_b")
c = tf.multiply(a,b, name="mul_c")
d = tf.add(a,b, name="add_d")
e = tf.add(c,d, name="add_e")
sess = tf.Session()
sess.run(c) <--check, value should be 15
sess.run(d) <--check, value should be 8
sess.run(e) <--check, value should be 23
Writing graph in event file
writer = tf.summary.FileWriter('./tensorflow_examples', sess.graph)
It is very important to specify a directory(in this case, the directory is tensorflow_examples), where the event file will be written to.
writer = tf.summary.FileWriter('./', sess.graph) didnt work for me, because the shell command => tensorboard --logdir expects a directory name.
After executing this step, verify if event file has been created in specified directory.
Visualizing graph in Tensorboard
Open terminal(bash), under working directory type:
tensorboard --logdir='tensorflow_examples' --host=127.0.0.1
Then open a new browser in http://127.0.0.1:6006/ or http://localhost/6006 and now tensorboard shows the graph successfully.
The problem might be the parameter --logdir. make sure you have type the correct
example:
in the code:
writer = tf.summary.FileWriter('./log/', s.graph)
open powershell
cd to your work directory and type
tensorboard --logdir=log
you can also use --debug to see if there is a problem in finding the log file. if you see:
TensorBoard path_to_run is: {'C:\\Users\\example\\log': None} that means it can not find the file.
You may need to change the powershell directory to your log file. And the logdir need not the single quotation marks.(Double quotation marks or without the quotes will be both OK)

weka: how to generate libsvm training parameter

I am running libsvm through weka. Its output accuracy looks good to me, so I am planning to write a svm model by myself. However, weka didn't generate any training parameter, such as number of support vector. Therefore i cannot do anything. Searching the web, i found somebody said it would generate some parameters like the following:
optimization finished, #iter = 27
nu = 0.058475864943863545
obj = -1.871013102744184, rho = -0.19357337828800944
nSV = 9, nBSV = 0 `enter code here`
Total nSV = 9
but how come i didn't see any of them? any step that i missed? please help me. Thanks a lot.
Weka writes the output you mentioned to stderr.
So if you have started weka.sh or weka.bat from a terminal (or "command window" if you are on Windows), you should see that output appear in your terminal window after clicking "classify"
If you want to have access to this information via scripts, you can
redirect the output to a file and read in that file.
Here is how to edit the startup file weka.sh / weka.bat.
Edit this line (it is probably the last line) in order to write log info to a file instead of the terminal window:
java -cp $CP -Xmx8092m weka.gui.GUIChooser 2>>/opt/weka-stable/weka.log &
You can also add a properties file to your home directory to add more fine-grained behaviour.
https://weka.wikispaces.com/Properties+file
(You probably can also access information via the Weka Java API somehow, but you did not ask for that)