Shiny implementation - shiny

Hello my question is about user interactivity with my code, I developed this simple loop for time series using the great forecast Hybrid package, here is the rmd for it
The strange language is Portuguese sorry I was too lazy to translate the whole thing it shouldn't matter anyway.
```{r Primeira Vez, clique no play, include=FALSE, include=FALSE}
setwd("~/R")
install.packages("forecastHybrid")
setwd("~/R")
library(forecastHybrid)
```
```{r Inputs}
# A dataset variable in the global enviroment
Data<- SemZero
#How to save a witout dummies regression
ComoSalvar<- "Exemplo2.csv"
#How to save a with dummies regression
ComoSalvarReg<- "ExemploparaDummies.csv"
#Where to save them.
OndeSalvar <- "~/R"
#Month,Year, and Day variable for the ts
Mes<- 9
Ano<- 2012
Dia<- 1
#Frequency
Freq<- 12
#Forecast Period.
Forecast = 12
#Confidance intervals
IC<- c(0)
#Variables in the Data dataset that will be used for the lapply regression, usually I would read an excel file with headers.
VStart = 1
VFinish = 2
#Simple regressor dataset, they can be matrix as well as line since de data.fram combines everyone example data.frame(OddMonths,Christmas,WasTrumpPresident?,RainInThatSeason)
Regressores<- data.frame(0)
```
```{r Logic for rolling test and data simplification}
setwd(OndeSalvar)
if(IC[1] > 0) pi<-TRUE else
pi<- FALSE
if (nrow(Data) >= Freq*3 + Forecast*2) {
Weights <- "cv.error"
Multiplicador<-floor((nrow(Data)-Forecast*2)/Freq) } else
if (nrow(Data) <= Freq*3 + Forecast*2) {
Weights <- "cv.error"
Multiplicador<-3 } else
if (nrow(Data) >= Freq*2.5 + Forecast*2){
Weights <- "cv.error"
Multiplicador = 2.5} else
Weights <- "equal"
```
```{r The bunk of the regression process}
if (nrow(Regressores) < nrow(Data)) {
my_forecast1 <- try({function(x){
print(x)
print(summary(x))
names(x)
x[is.na(x)] <- 0; x
if(sum(abs(x)) < Freq){
Model<- "aenst"} else
if(mean(x[1:Freq]) == 0)Model<- "aenst" else Model<- "aenstf"
x<- ts(x, start = c(Ano, Mes,Dia), frequency = Freq)
hm<-hybridModel(x, models = Model, lambda = NULL,
a.args = list(trace = FALSE,test = "kpss", ic ="aicc", max.P = 2, max.p = 9,max.q=9,max.Q = 2,max.d = 2,max.D = 2,start.p = 9,start.P = 2,start.Q = 2,start.q = 9,allowdrift = TRUE,allowmean = TRUE
#Se tiver tempo apague o # abaixo para uma maior qualidade no modelo arima.
#,stepwise = FALSE,parallel = TRUE,num.cores = NULL
),
e.args = list(ic = "aicc"),
n.args = list(repeats = nrow(Data)),
s.args = NULL,
t.args = NULL,
weights = Weights,
errorMethod = "RMSE",cvHorizon = Forecast,windowSize = frequency(x)*Multiplicador, horizonAverage = FALSE,
verbose = TRUE)
lapply(seq_along(x), function(i) paste(names(x)[[i]], x[[i]]))
fcast1<- forecast(hm,h = Forecast,level = IC,PI = pi)
return(fcast1)
}})
Listas<- lapply(Data[,VStart:VFinish], try(my_forecast1))
if (pi == FALSE) ListaResultado<- as.data.frame(lapply(Listas, '[[', 'mean')) else
ListaResultado <- Listas
write.csv(ListaResultado , file = ComoSalvar)
} else {
my_forecastreg <- function(x){
print(x)
print(summary(x))
names(x)
x[is.na(x)] <- 0; x
x<- ts(x, start = c(Ano, Mes,Dia), frequency = Freq)
hmreg <- hybridModel(x ,models = "ans",
a.args = list(xreg = Regressores[1:nrow(Data),],trace = TRUE,test = "kpss", ic ="aicc", max.P = 2, max.p = 9,max.q=9,max.Q = 2,max.d = 2,max.D = 2,start.p = 9,start.P = 2,start.Q = 2,start.q = 9,allowdrift = TRUE,allowmean = TRUE
#Se tiver tempo apague o # abaixo para uma maior qualidade no modelo arima.
#,stepwise = FALSE,parallel = TRUE,num.cores = NULL
),
n.args = list(xreg = Regressores[1:nrow(Data),], repeats= nrow(Regressores)),
s.args = list(xreg = Regressores[1:nrow(Data),], method = "arima"))
fcast2<- forecast(hmreg,h = Forecast,level = IC,PI = pi, xreg = Regressores[nrow(Data):(nrow(Data)+Forecast-1),])
return(fcast2)
}
Listas2<- lapply(Data[,VStart:VFinish], try(my_forecastreg))
if (pi == FALSE) ListaResultado2<- as.data.frame(lapply(Listas2, '[[', 'mean')) else
ListaResultado2 <- Listas2
write.csv(ListaResultado2 , file = ComoSalvarReg)
}
```
I want to develop something to get the user input and run the regressions, my end user doesn't usually like how ugly a R Markdown file looks, I was looking into using shiny, but i dont know a few details
Who runs this regressions if I upload the whole thing successfully to shiny? My computer,the server, the user, I have no idea?
Can the user input go into the users own global environment so that the whole thing could be kept as a strictly offline process(using Shiny as a beautification app that substitutes this input chunk?)
Can someone please give an example of an Shiny app that does something similar?
Can the user read.xlsm into shiny server, or use his global environment to define a data for the shiny app to use as input?
Also is the thief package possible to implement on this lapply function as a way to increase the forecast quality, I would of course Drop the stlm and theta option from the model as they behave rather poorly in a wide range of simulations that I performed with toy sets, the stlm crashes on cross validation with a few observations and the theta model just doesn't work.
Can someone teach me how to on error inside the function ignore the variable and just keep applying the function to the next variable? or change the model to something less problematic my solution was to try to catch these cases where the model would crash and drop the theta model before it happens but it is just an ugly hack to the underlying problem.
Also if you see something ugly in the code itself feel free to criticize.

Related

R code works in script but not R markdown

#The code below works fine in my scripts but not in R markdown.
library(tidyverse)
library(scales)
age <- kaggle_2020_Survey %>%
transmute(Q1 = as.factor(Q1)) %>%
filter(!is.na(Q1)) %>%
count(Q1) %>%
mutate(perc = n/sum(n)*100)
ggplot(age, aes(x = Q1, y = n)) + geom_col(fill = "darkblue", alpha =.7) +
geom_text (aes(x = Q1, y = n, label = paste0(round(perc,1), "%"),hjust = -.3), size = 3)
+
coord_flip() + labs(title = "Age of participants", x = "Percent", y = "Number",
subtitle = "Highest age group: 22-24") +
theme_classic()
#This is the error I am getting:
enter image description here
It could be that the object is in your environment but you forgot to include something like
library(tidyverse)
kaggle_2020_Survey <- read_csv("kaggle_2020_Survey.csv")
where you load in the data. When you knit a rmarkdown file it starts a new session each time so if the data isn't called in before you start doing stuff then it will throw that error
This answer goes into other solutions.

How do I get the word-embedding matrix from ft_word2vec (sparklyr-package)?

I have another question in the word2vec universe.
I am using the 'sparklyr'-package. Within this package I call the ft_word2vec() function. I have some trouble understanding the output:
For each number of sentences/paragraphs I am providing to the ft_word2vec() function, I always get the same amount of vectors. Even, if I have more sentences/paragraphs than words. For me, that looks like I get the paragraph-vectors. Maybe a Code-example helps to understand my problem?
# add your spark_connection here as 'spark_connection = '
# create example data frame
FK_data = data.frame(sentences = c("This is my first sentence",
"It is followed by the second sentence",
"At the end there is the last sentence"))
# move the data to spark
sc_FK_data <- copy_to(spark_connection, FK_data, name = "FK_data", overwrite = TRUE)
# prepare data for ft_word2vec (sentences have to be tokenized [=list of words instead of one string in each row])
sc_FK_data <- ft_tokenizer(sc_FK_data, input_col = "icd_long", output_col = "tokens")
# split data into test and trainings sets
partitions <- sc_FK_data %>%
sdf_random_split(training = 0.7, test = 0.3, seed = 123456)
FK_train <- partitions$training
FK_test <- partitions$test
# given a trainings data set (FK_train) with a column "tokens" (for each row = a list of strings)
mymodel = ft_word2vec(
FK_train,
input_col = "tokens",
output_col = "word2vec",
vector_size = 15,
min_count = 1,
max_sentence_length = 4444,
num_partitions = 1,
step_size = 0.1,
max_iter = 10,
seed = 123456,
uid = random_string("word2vec_"))
# I tried to get the data from spark with:
myemb = mymodel %>% sparklyr::collect()
Has somebody had similar experiences? Can someone explain what exactly the ft_word2vec() function returns? Do you have an example on how to get the word embedding vectors with this function? Or does the returned column indeed contain the paragraph vectors?
my colleague found a solution! If you know how to do it, the instructions really begin to make sense!
# add your spark_connection here as 'spark_connection = '
# create example data frame
FK_data = data.frame(sentences = c("This is my first sentence",
"It is followed by the second sentence",
"At the end there is the last sentence"))
# move the data to spark
sc_FK_data <- copy_to(spark_connection, FK_data, name = "FK_data", overwrite = TRUE)
# prepare data for ft_word2vec (sentences have to be tokenized [=list of words instead of one string in each row])
sc_FK_data <- ft_tokenizer(sc_FK_data, input_col = "icd_long", output_col = "tokens")
# split data into test and trainings sets
partitions <- sc_FK_data %>%
sdf_random_split(training = 0.7, test = 0.3, seed = 123456)
FK_train <- partitions$training
FK_test <- partitions$test
# CHANGES FOLLOW HERE:
# We have to use the spark connection instead of the data. For me this was the confusing part, since i thought no data -> no model.
# maybe we can think of this step as an initialization
mymodel = ft_word2vec(
spark_connection,
input_col = "tokens",
output_col = "word2vec",
vector_size = 15,
min_count = 1,
max_sentence_length = 4444,
num_partitions = 1,
step_size = 0.1,
max_iter = 10,
seed = 123456,
uid = random_string("word2vec_"))
# now that we have our model initialized, we add the word-embeddings to the model
w2v_model = ml_fit(w2v_model, sc_FK_EMB)
# now we can collect the embedding vectors
emb = word2vecmodel$vectors %>% collect()

How to adjust table size in markdown knitr

I'm trying to use a package expss, and I followed some of the examples, but my table is huge. How can I adjust the table so that it is small, like in the link?
For the tables in the link please see: expss introduction
My tables have wide margins:
I think I need to make changes to the setup, but I don't know how. My set up is:
library(knitr)
knitr::opts_chunk$set(echo = TRUE)
Try to use htmlTable css.cell argument:
library(expss)
data(mtcars)
mtcars = apply_labels(mtcars,
mpg = "Miles/(US) gallon",
cyl = "Number of cylinders",
disp = "Displacement (cu.in.)",
hp = "Gross horsepower",
drat = "Rear axle ratio",
wt = "Weight (1000 lbs)",
qsec = "1/4 mile time",
vs = "Engine",
vs = c("V-engine" = 0,
"Straight engine" = 1),
am = "Transmission",
am = c("Automatic" = 0,
"Manual"=1),
gear = "Number of forward gears",
carb = "Number of carburetors"
)
cro(mtcars$am, mtcars$vs) %>%
htmlTable(css.cell = "padding: 0px;")

Joining of curve fitting models

I have this 7 quasi-lorentzian curves which are fitted to my data.
and I would like to join them, to make one connected curved line. Do You have any ideas how to do this? I've read about ComposingModel at lmfit documentation, but it's not clear how to do this.
Here is a sample of my code of two fitted curves.
for dataset in [Bxfft]:
dataset = np.asarray(dataset)
freqs, psd = signal.welch(dataset, fs=266336/300, window='hamming', nperseg=16192, scaling='spectrum')
plt.semilogy(freqs[0:-7000], psd[0:-7000]/dataset.size**0, color='r', label='Bx')
x = freqs[100:-7900]
y = psd[100:-7900]
# 8 Hz
model = Model(lorentzian)
params = model.make_params(amp=6, cen=5, sig=1, e=0)
result = model.fit(y, params, x=x)
final_fit = result.best_fit
print "8 Hz mode"
print(result.fit_report(min_correl=0.25))
plt.plot(x, final_fit, 'k-', linewidth=2)
# 14 Hz
x2 = freqs[220:-7780]
y2 = psd[220:-7780]
model2 = Model(lorentzian)
pars2 = model2.make_params(amp=6, cen=10, sig=3, e=0)
pars2['amp'].value = 6
result2 = model2.fit(y2, pars2, x=x2)
final_fit2 = result2.best_fit
print "14 Hz mode"
print(result2.fit_report(min_correl=0.25))
plt.plot(x2, final_fit2, 'k-', linewidth=2)
UPDATE!!!
I've used some hints from user #MNewville, who posted an answer and using his code I got this:
So my code is similar to his, but extended with each peak. What I'm struggling now is replacing ready LorentzModel with my own.
The problem is when I do this, the code gives me an error like this.
C:\Python27\lib\site-packages\lmfit\printfuncs.py:153: RuntimeWarning:
invalid value encountered in double_scalars [[Model]] spercent =
'({0:.2%})'.format(abs(par.stderr/par.value))
About my own model:
def lorentzian(x, amp, cen, sig, e):
return (amp*(1-e)) / ((pow((1.0 * x - cen), 2)) + (pow(sig, 2)))
peak1 = Model(lorentzian, prefix='p1_')
peak2 = Model(lorentzian, prefix='p2_')
peak3 = Model(lorentzian, prefix='p3_')
# make composite by adding (or multiplying, etc) components
model = peak1 + peak2 + peak3
# make parameters for the full model, setting initial values
# using the prefixes
params = model.make_params(p1_amp=6, p1_cen=8, p1_sig=1, p1_e=0,
p2_ampe=16, p2_cen=14, p2_sig=3, p2_e=0,
p3_amp=16, p3_cen=21, p3_sig=3, p3_e=0,)
rest of the code is similar like at #MNewville
[![enter image description here][3]][3]
A composite model for 3 Lorentzians would look like this:
from lmfit import Model, LorentzianModel
peak1 = LorentzianModel(prefix='p1_')
peak2 = LorentzianModel(prefix='p2_')
peak3 = LorentzianModel(prefix='p3_')
# make composite by adding (or multiplying, etc) components
model = peak1 + peaks2 + peak3
# make parameters for the full model, setting initial values
# using the prefixes
params = model.make_params(p1_amplitude=10, p1_center=8, p1_sigma=3,
p2_amplitude=10, p2_center=15, p2_sigma=3,
p3_amplitude=10, p3_center=20, p3_sigma=3)
# perhaps set bounds to prevent peaks from swapping or crazy values
params['p1_amplitude'].min = 0
params['p2_amplitude'].min = 0
params['p3_amplitude'].min = 0
params['p1_sigma'].min = 0
params['p2_sigma'].min = 0
params['p3_sigma'].min = 0
params['p1_center'].min = 2
params['p1_center'].max = 11
params['p2_center'].min = 10
params['p2_center'].max = 18
params['p3_center'].min = 17
params['p3_center'].max = 25
# then do a fit over the full data range
result = model.fit(y, params, x=x)
I think the key parts you were missing were: a) just add models together, and b) use prefix to avoid name collisions of parameters.
I hope that is enough to get you started...

intField does not display changes

I am writing a script to simplify a tedious task when using Vray, but I am stuck with the intFields that are supposed to allow the user to type in a int value that triggers an certain action when hitting the button. I simplified the code to only the necessary parts. No matter what I change the value to, it is always 0 in the Script Editor output.
import maya.cmds as cmds
idManagerUI = cmds.window(title='Vray ID Manager', s = False, wh = (300,500))
cmds.columnLayout(adj = True)
cmds.text (l = 'type in MultimatteID to select matching shaders \n or specify ObjectID to select matching objects \n __________________________________________ \n')
cmds.text (l = 'MultimatteID: \n')
cmds.intField( "MultimatteID", editable = True)
MultimatteIdButton = cmds.button(l = 'Go!', w = 30, h = 50, c = 'multimatteChecker()')
cmds.text (l = '\n')
cmds.showWindow(idManagerUI)
MultimatteIdInput = cmds.intField( "MultimatteID", q = True, v = True)
def multimatteChecker():
print MultimatteIdInput
Three things:
First, as written you can't be sure that the intField MultimatteID is actually getting the name you think it should have. Maya widget names are unique, like maya object names -- you may name it MultimatteID but actually get back a widget named MultimatteID2 because you have an undeleted window somewhere (visible or not) with a similarly named control.
Second, the code you pasted queries the value of the control immediately after the window is created. It should always print out the value you gave it on creation.
Finally -- don't use the string version of command assignment in your button. It's unreliable when you move from code in the listener to working scripts.
This should do what you want:
idManagerUI = cmds.window(title='Vray ID Manager', s = False, wh = (300,500))
cmds.columnLayout(adj = True)
cmds.text (l = 'type in MultimatteID to select matching shaders \n or specify ObjectID to select matching objects \n __________________________________________ \n')
cmds.text (l = 'MultimatteID: \n')
# store the intField name
intfield = cmds.intField( "MultimatteID", editable = True)
cmds.text (l = '\n')
# define the function before assigning it.
# at this point in the code it knows what 'intfield' is....
def multimatteChecker(_):
print cmds.intField( intfield, q = True, v = True)
#assign using the function object directly
MultimatteIdButton = cmds.button(l = 'Go!', w = 30, h = 50, c = multimatteChecker)