I am trying to generate a large number of html files using rmarkdown::render in a loop, with Parameterized reports.
After generating a number of files, it stalls, and I have to restart RStudio. I can generate each individual file in itself; it is not at the same file it stalls each time when I try running the loop.
There is no error message, making it hard for me to debug.
I have tried the following, none of which helped:
Closing all other programs; reducing the memory used.
Adding knitr::knit_meta(clean = TRUE) before render
Adding clean = T inside render
Calling render with callr::r
Including rm([[data]]); gc() at the end of the .rmd file that is called by render
Any other ideas of how to try and solve this issue?
Taking This from R Markdown: The Definitive Guide Where an example is used to render multiple files. Every file would have a it's own name regarding it's param region and year.
render_report = function(region, year) {
rmarkdown::render(
"MyDocument.Rmd", params = list(
region = region,
year = year
),
output_file = paste0("Report-", region, "-", year, ".html")
)
}
Related
My Opencart 3.0 is running very slow. In the network tab in the chrome browser inspector, it records 23.02 seconds for the category page to load.
When I try to debug it, I could see that the slow loading happens here
catalog/model/catalog/product.php - public function getProduct($product_id)
Inside this method, when I comment out this line in the returned array - 'product_id' => $query->row['product_id'], the loading speed comes to 7s.
The method getProduct($product_id) is called in this method getProducts($data = array()). The part where the method is called looks like below.
$query = $this->db->query($sql);
foreach ($query->rows as $result) {
$product_data[$result['product_id']] = $this->getProduct($result['product_id']);
}
When I comment out this line
//$product_data[$result['product_id']] = $this->getProduct($result['product_id']);
The loading speed goes to 3.5s.
What I could not figure out is why this part of the code takes time to return - 'product_id' => $query->row['product_id']
I will appreciate your help.
It looks like your issues comes from getProduct() and getProducts() methods in the catalog/model/catalog/product.php file. Especially this row:
'product_id' => $query->row['product_id']
So you could check your DB performance, how much time is taking to complete this query.
If it takes too much time and your columns are not indexed, you could index them.
You could also try to limit the number of rows being retrieved in the query by using a LIMIT clause or a WHERE clause.
You could use Opencart cache system, to reduce time.
On first sign it looks more like DB related problem, instead of code problem.
i want display PDF file into region , i tried that by call application process using below code but always same file open.( plsql dynamic content region)
DECLARE
V_URL VARCHAR2(2500);
BEGIN
V_URL :='f?p=&APP_ID.:1:&APP_SESSION.:APPLICATION_PROCESS=display_emp_blob:::FILE_ID:' ||:P6_ID;
Sys.htp.p('<p align="center">');
sys.htp.p('<iframe src="'||V_URL||'"width="99%" height="1000">');
sys.htp.p('</iframe>');
sys.htp.p('</p>');
END;
and the application process code in below
CREATE OR REPLACE PROCEDURE OPEN_FILE (P_ID NUMBER)
IS
vBlob blob;
vmimetype varchar2(50);
BEGIN
SELECT ORG_FILES.FILE_CONTENT,MIME_TYPE INTO vBlob, vmimetype
FROM ORG_FILES
WHERE ID =P_ID ;
sys.HTP.init;
owa_util.mime_header(vmimetype,false);
htp.p('Content-Length: ' || dbms_lob.getlength(vBlob));
owa_util.http_header_close;
wpg_docload.download_file(vBlob);
apex_application.stop_apex_engine;
exception
when no_data_found then
null;
END;
How i can open different PDF file into region based a value in ITEM (P6_ID) .
I think the problem you have is that the browser caches the file.
You can specify the time the browser caches with the "Cache-control" header option. Below, you have the code that I use (I have this code in the application process, not in the database):
sys.htp.init;
sys.owa_util.mime_header( 'application/pdf', FALSE );
sys.htp.p('Content-length: ' || sys.dbms_lob.getlength( v_blob));
sys.htp.p('Content-Disposition: inline; filename="'|| v_filename || '"' ); -- "attachment" for download, "inline" for display
sys.htp.p('Cache-Control: max-age=3600'); -- in seconds. Tell the browser to cache for one hour, adjust as necessary
sys.owa_util.http_header_close;
sys.wpg_docload.download_file( v_blob );
apex_application.stop_apex_engine;
You can also try some lazy load, which is the way I access my files (It may be that the way to access your file is also part of the problem). This way you make the page load without waiting for the user and then it loads and shows the file. I don't use the iframe tag but the embed tag. The way to do it is as follows:
Create a region with static content with this html
<div id="view_pdf"></div>
creates a dynamic action when the page loads, which executes javascript and add the following code
$('#view_pdf').html('');
var url = 'f?p=&APP_ID.:1:&APP_SESSION.:APPLICATION_PROCESS=display_emp_blob:::FILE_ID:' + apex.item('P6_ID').getValue();
var preview = document.createElement('embed');
preview.type = "application/pdf";
preview.width="100%";
preview.height="625px";
preview.src = url;
$("#view_pdf").append(preview);
You can modify the values depending on what you need. The embed tag uses the default way to view pdf files from browsers.
Also if what you want is to change the pdf without reloading the page you must use the previous javacript in a dynamic action when you change the value of the item.
I hope you find it useful.
My apologies, on the rare occasion I use this region type, I always think it can be refreshed.
https://spendolini.blogspot.com/2015/11/refreshing-plsql-regions-in-apex.html
The solution is to create a classic report that calls a PL/SQL function that returns your HTML.
SELECT package_name.function_name(p_item => :P1_ITEM) result FROM dual
I am rying to use tensorboard embeddings page in order to visualize the results of word2vec. After debugging, digging of lots of codes i came to a point that tensorboard successfully runs, reads the confguration file, reads the tsv files but now the embeddings page does not show data.
( the page is opened , i can see the menus , items etc) this is my config file:
embeddings {
tensor_name: 'word_embedding'
metadata_path: 'c:\data\metadata.tsv'
tensor_path: 'c:\data\tensors2.tsv'
}
What can be the problem?
The tensor file originally is 1gb. in size, if i try that file , the app crashes becasue of the memory. So i copy and paste 1 or 2 pages of the original file into tensor2.tsv and use this file. May be this is the problem. May be i need to create more data by copy/ paste.
thx
tolga
Try following code snippet to get visualized word embedding in tensorboard. Open tensorboard with logdir, check localhost:6006 for viewing your embedding.
tensorboard --logdir="visual/1"
# code
fname = "word2vec_model_1000"
model = gensim.models.keyedvectors.KeyedVectors.load(fname)
# project part of vocab, max of 100 dimension
max = 1000
w2v = np.zeros((max,100))
with open("prefix_metadata.tsv", 'w+') as file_metadata:
for i,word in enumerate(model.wv.index2word[:max]):
w2v[i] = model.wv[word]
file_metadata.write(word + '\n')
# define the model without training
sess = tf.InteractiveSession()
with tf.device("/cpu:0"):
embedding = tf.Variable(w2v, trainable=False, name='prefix_embedding')
tf.global_variables_initializer().run()
path = 'visual/1'
saver = tf.train.Saver()
writer = tf.summary.FileWriter(path, sess.graph)
# adding into projector
config = projector.ProjectorConfig()
embed= config.embeddings.add()
embed.tensor_name = 'prefix_embedding'
embed.metadata_path = 'prefix_metadata.tsv'
# Specify the width and height of a single thumbnail.
projector.visualize_embeddings(writer, config)
saver.save(sess, path+'/prefix_model.ckpt', global_step=max)
Update 2
I accepted an answer and asked a different question elsewhere, where I am still trying to get to the bottom of this.
I don't think that one-lining this query is the answer, as I am still not getting the required results (and multi-lining queries is allowed in .mof, as shown in the URLs in comments to the answer ...
Update
I rewrote the query as a one-liner as suggested, but still got the same error! As it was still talking about lines 11-19 I knew there must be another issue. After saving a new file with the change, I reran mofcomp and it appears to have loaded, but the event which I have subscribed to simply does not work.
I really feel that there is not enough documentation on this topic and it is hard to work out how I am meant to debug this - any help on this would be much appreciated, even if this means using a different more appropriate method.
I have the following .mof file, which I would like to use to register an event on my system :
#pragma namespace("\\\\.\\root\\subscription")
instance of __EventFilter as $EventFilter
{
Name = "Event Filter Instance Name";
Query = "Select * from __InstanceCreationEvent within 1 "
"where targetInstance isa \"Cim_DirectoryContainsFile\" "
"and targetInstance.GroupComponent = \"Win32_Directory.Name=\"c:\\\\test\"\"";
QueryLanguage = "WQL";
EventNamespace = "Root\\Cimv2";
};
instance of ActiveScriptEventConsumer as $Consumer
{
Name = "TestConsumer";
ScriptingEngine = "VBScript";
ScriptText =
"Set objFSO = CreateObject(\"Scripting.FileSystemObject\")\n"
"Set objFile = objFSO.OpenTextFile(\"c:\\test\\Log.txt\", 8, True)\n"
"objFile.WriteLine Time & \" \" & \" File Created\"\n"
"objFile.Close\n";
// Specify any other relevant properties.
};
instance of __FilterToConsumerBinding
{
Filter = $EventFilter;
Consumer = $Consumer;
};
But whenever I run the command mfcomp myfile.mof I am getting this error:
Parsing MOF file: myfile.mof
MOF file has been successfully parsed
Storing data in the repository...
An error occurred while processing item 1 defined on lines 11 - 19 in file myfile.mof:
Error Number: 0x80041058, Facility: WMI
Description: Unparsable query.
Compiler returned error 0x80041058
This error appears to be caused by incorrect syntax in the query, but I don't understand where I have gone wrong with this - is anyone able to advise?
There are no string concatenation or line continuation characters being used in building "Query". To keep it simple, you could put the entire query on one line.
So I have an instance of Liquidsoap, which I am using to stream to an Icecast server.
I'd like to record any live broadcasts that take place automatically, which I am now doing and is working well.
What I'd like to do is use the metadata (specifically the songname) of the live show when creating the mp3 archive.
#!/usr/local/bin/liquidsoap
set("log.file",true)
set("log.file.path","/var/log/liquidsoap/radiostation.log")
set("log.stdout",true)
set("log.level",3)
#-------------------------------------
set("harbor.bind_addr","0.0.0.0")
#-------------------------------------
backup_playlist = playlist("/home/radio/playlists/playlist.pls",conservative=true,reload_mode="watch")
output.dummy(fallible=true,backup_playlist)
#-------------------------------------
live_dj = input.harbor(id="live",port=9000,password="XXX", "live")
date = '%m-%d-%Y'
time = '%H:%M:%S'
output.file(%mp3, "/var/www/recorded-shows/#{Title} - Recorded On #{date} At #{time}.mp3", live_dj, fallible=true)
#time_stamp = '%m-%d-%Y, %H:%M:%S'
#output.file(%mp3, "/var/www/recorded-shows/live_dj_#{time_stamp}.mp3", live_dj, fallible=true)
#-------------------------------------
on_fail = single("/home/radio/fallback/Enei -The Moment Feat DRS.mp3")
#-------------------------------------
source = fallback(track_sensitive=false,
[live_dj, backup_playlist, on_fail])
# We output the stream to icecast
output.icecast(%mp3,id="icecast",
mount="myradio.mp3",
host="localhost", password="XXX",
icy_metadata="true",description="cool radio",
url="http://myradio.fm",
source)
I have added #{title} where I would like my song title to appear, sadly though I am unable to get this populate.
My Dj's use BUTT and the show title is connected as part of their connection, so the data should be available pre recording.
Any advice is much appreciated!
This is far from being as easy as it seems.
The title metadata is dynamic, thus not available as a variable on script initialization
The filename argument of output.file is compiled when the script is initialized
A solution would consist in:
Defining a variable reference title to populate with live metadata
Output to a temporary file
Rename the file on close using on_close argument with output.file (in this case, we can just prepend the title)
This would give the following code (on a Linux box, change mv with ren on Windows):
date = '%m-%d-%Y'
time = '%H:%M:%S'
# Title is a reference
title = ref ""
# Populate with metadata
def get_title(m)
title := m['title']
end
live_dj = on_metadata(get_title,live_dj)
# Rename file on close
def on_close(filename)
# Generate new file name
new_filename = "#{path.dirname(filename)}/#{!title} - #{basename(filename)}"
# Rename file
system("mv '#{filename}' '#{new_filename}'")
end
output.file(on_close=on_close, %mp3, "/var/www/recorded-shows/Recorded On #{date} At #{time}.mp3", live_dj, fallible=true)
I tested a similar scenario and it works just well. Just beware that this will create a new file every time a DJ disconnects or updates the title. Also keep in mind that time stamps will be resolved by output.file.
This is based on the following example from a Liquidsoap dev: https://github.com/savonet/liquidsoap/issues/661#issuecomment-439935854)