Use d3.queue() to load .tsvs when working within Django - django

I am trying to convert my old website to use Django. However, I don't know how to successfully load my data in d3 when working inside of Django frameworks.
I know the D3 visualization works because it renders it on the old website frameworks. It appears to just be an issue of how do I properly call the pathing for the data files.
I have tried various call methods to the files by making duplicate copies and placing them in different directories. But so far I can't figure out how to call the paths correct!
Here is the original set of code:
queue()
.defer(d3.json, "../core/world_countries.json")
.defer(d3.tsv, "worldData.tsv")
.await(ready);
Here are two different method calls I have tried
queue()
.defer(d3.json, "world_countries.json")
.defer(d3.tsv, "{% static 'data/worldData.tsv' %}")
.await(ready)
2 different errors occur:
GET http://127.0.0.1:8000/web_app/world_countries.json 404 (Not Found)
GET http://127.0.0.1:8000/web_app/%7B%%20static%20'data/worldData.tsv'%20%%7D 404 (Not Found)

Solved it!
Change the pathing call to be: "/static/data/world_countries.json"
Code looks like this:
queue()
.defer(d3.json, "/static/data/world_countries.json")
.defer(d3.tsv, "/static/data/worldData.tsv")
.await(ready);

Related

In-repo addon writing public files on build causes endless build loop on serve

I'm having difficulty with my in-repo addon writing to appDir/public. What I'd like to do is write out a JSON file on each build to be included in the app /dist. The problem I'm running into is when running "ember serve", the file watcher detects the new file and rebuilds again, causing an endless loop.
I've tried writing the JSON file using preBuild() and postBuild() hooks, saving to /public, but after build, the watcher detects it and rebuild over and over, writing a new file again each time. I also tried using my-addon/public folder and writing to that, same thing.
The only thing that partially works is writing on init(), which is fine, except I don't see the changes using ember serve.
I did try using the treeForPublic() method, but did not get any further. I can write the file and use treeForPublic(). This only runs once though, on initial build. It partially solves my problem, because I get the files into app dist folder. But I don't think ember serve will re-run treeForPublic on subsequent file change in the app.
Is there a way to ignore specific files from file watch? Yet still allow files to include into the build? Maybe there's an exclude watch property in ember-cli-build?
Here's my treeForPublic() , but I'm guessing my problems aren't here:
treeForPublic: function() {
const publicTree = this._super.treeForPublic.apply(this, arguments);
const trees = [];
if (publicTree) {
trees.push(publicTree);
}
// this writes out the json
this.saveSettingsFile(this.pubSettingsFile, this.settings);
trees.push(new Funnel(this.addonPubDataPath, {
include: [this.pubSettingsFileName],
destDir: '/data'
}));
return mergeTrees(trees);
},
UPDATE 05/20/2019
I should probably make a new question at this point...
My goal here is to create an auto-increment build number that updates both on ember build and ember serve. My comments under #real_ates's answer below help explain why. In the end, if I can only use this on build, that's totally ok.
The answer from #real_ate was very helpful and solved the endless loop problem, but it doesn't run on ember serve. Maybe this just can't be done, but I'd really like to know either way. I'm currently trying to change environment variables instead of using treeforPublic(). I've asked that as a separate question about addon config() updates to Ember environment:
Updating Ember.js environment variables do not take effect using in-repo addon config() method on ember serve
I don't know if can mark #real_ate's answer as the accepted solution because it doesn't work on ember serve. It was extremely helpful and educational!
This is a great question, and it's often something that people can be a bit confused about when working with broccoli (I know for sure that I've been stung by this in the past)
The issue that you have is that your treeForPublic() is actually writing a file to the source directory and then you're using broccoli-funnel to select that new custom file and include it in the build. The correct method to do this is instead to use broccoli-file-creator to create an output tree that includes your new file. I'll go into more detail with an example below:
treeForPublic: function() {
const publicTree = this._super.treeForPublic.apply(this, arguments);
const trees = [];
if (publicTree) {
trees.push(publicTree);
}
let data = getSettingsData(this.settings);
trees.push(writeFile('/data/the-settings-file.json', JSON.stringify(data)));
return mergeTrees(trees);
}
As you will see the most of the code is exactly the same as your example. The two main differences are that instead of having a function this.saveSettingsFile() that writes out a settings file on disk we now have a function this.getSettingsData() that returns the content that we would like to see in the newly created file. Here is the simple example that we came up with when we were testing this out:
function getSettingsData() {
return {
setting1: 'face',
setting2: 'my',
}
}
you can edit this function to take whatever parameters you need it to and have whatever functionality you would like.
The next major difference is that we are using the writeFile() function which is actually just the broccoli-file-creator plugin. Here is the import that you would put at the top of the file:
let writeFile = require('broccoli-file-creator');
Now when you run your application it won't be writing to the source directory any more which means it will stop constantly reloading 🎉
This question was answered as part of "May I Ask a Question" Season 2 Episode 2. If you would like to see us discuss this answer in full you can check out the video here: https://youtu.be/9kMGMK9Ur4E

How to use Polymer 2 Build Process?

I know this might be a duplicate of this question but there was no clear answer ever.
Polymer 2 Framework has a quite good documentation so far but when it comes to the build process there`s not enough explanation.
I've successfully created my own element, also with external scripts referenced and everything runs fine with
polymer serve --open
But I've spent so much time to get this built and to include the output into a minimal HTML template without success. Is it really that tricky?
I`ve even tried again with the empty element template, no chance. This template has a nearly blank polymer.json:
{
"lint": {
"rules": [
"polymer-2"
]
}
}
If I build that right away with
polymer build
(should use default build behaviour) then I get a default build folder with an index.html and bower_components folder. There`s no reference to my created custom element ("Hello ...").
What do I have to do to get a final build of this "Hello Something" template and include it into a minimum html page?
There are a lot of different ways to go from single component to app with that single component, and I'm not going to say 100% that this is the best, but it should work. "What do I have to do to get a final build of this "Hello Something" template and include it into a minimum html page?", is a pretty flexible request, so even in the below there could be lots of alternatives for you, but the following would be my suggestion. We'll start in the command line, from the Desktop or a folder where you keep you projects.
mkdir minimal-html-page
cd minimal-html-page
polymer init
// here make sure you choose `polymer-2-application` other questions re of little consequence in this specific use case
bower install {yourComponent} --save
// in that case that you haven't published to github, copy and paste also works. Paste into the `src` directory if that's the case.
atom .
// or whatever editor you prefer
Now you're in your project, and there is just one change needed to get you going.
index.html
Change the current HTML import
<link rel="import" href="/src/build-test-app/build-test-app.html">
to link to your component.
At this point you should be free to use polymer build to have the project built, then it'll structure your code to be deployed in other applications via something that will look like
<script src="/bower_components/webcomponentsjs/webcomponents-loader.js"></script>
<link rel="import" href="/src/build-test-app/build-test-app.html">
<build-test-app></build-test-app>
This is the most basic of builds and won't account for any x-browser or performance goals you might have in your project. You'll also see in the build/default directory all the files you'd need to copy/paste to use the component elsewhere.
If you do have x-browser expectations, you can solve them fairly easily by using the polymer build --compile command. This will create slightly more complex embed code to manage either side of the ES5/6 capability boundary and will look something like:
<script>!function(e){var r=e.babelHelpers={};r.typeof="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},r.classCallCheck=function(e,r){if(!(e instanceof r))throw new TypeError("Cannot call a class as a function")},r.createClass=function(){function e(e,r){for(var t=0;t<r.length;t++){var n=r[t];n.enumerable=n.enumerable||!1,n.configurable=!0,"value"in n&&(n.writable=!0),Object.defineProperty(e,n.key,n)}}return function(r,t,n){return t&&e(r.prototype,t),n&&e(r,n),r}}(),r.defineEnumerableProperties=function(e,r){for(var t in r){var n=r[t];n.configurable=n.enumerable=!0,"value"in n&&(n.writable=!0),Object.defineProperty(e,t,n)}return e},r.defaults=function(e,r){for(var t=Object.getOwnPropertyNames(r),n=0;n<t.length;n++){var o=t[n],i=Object.getOwnPropertyDescriptor(r,o);i&&i.configurable&&void 0===e[o]&&Object.defineProperty(e,o,i)}return e},r.defineProperty=function(e,r,t){return r in e?Object.defineProperty(e,r,{value:t,enumerable:!0,configurable:!0,writable:!0}):e[r]=t,e},r.extends=Object.assign||function(e){for(var r=1;r<arguments.length;r++){var t=arguments[r];for(var n in t)Object.prototype.hasOwnProperty.call(t,n)&&(e[n]=t[n])}return e},r.get=function e(r,t,n){null===r&&(r=Function.prototype);var o=Object.getOwnPropertyDescriptor(r,t);if(void 0===o){var i=Object.getPrototypeOf(r);return null===i?void 0:e(i,t,n)}if("value"in o)return o.value;var a=o.get;if(void 0!==a)return a.call(n)},r.inherits=function(e,r){if("function"!=typeof r&&null!==r)throw new TypeError("Super expression must either be null or a function, not "+typeof r);e.prototype=Object.create(r&&r.prototype,{constructor:{value:e,enumerable:!1,writable:!0,configurable:!0}}),r&&(Object.setPrototypeOf?Object.setPrototypeOf(e,r):e.__proto__=r)},r.instanceof=function(e,r){return null!=r&&"undefined"!=typeof Symbol&&r[Symbol.hasInstance]?r[Symbol.hasInstance](e):e instanceof r},r.newArrowCheck=function(e,r){if(e!==r)throw new TypeError("Cannot instantiate an arrow function")},r.objectDestructuringEmpty=function(e){if(null==e)throw new TypeError("Cannot destructure undefined")},r.objectWithoutProperties=function(e,r){var t={};for(var n in e)r.indexOf(n)>=0||Object.prototype.hasOwnProperty.call(e,n)&&(t[n]=e[n]);return t},r.possibleConstructorReturn=function(e,r){if(!e)throw new ReferenceError("this hasn't been initialised - super() hasn't been called");return!r||"object"!=typeof r&&"function"!=typeof r?e:r},r.set=function e(r,t,n,o){var i=Object.getOwnPropertyDescriptor(r,t);if(void 0===i){var a=Object.getPrototypeOf(r);null!==a&&e(a,t,n,o)}else if("value"in i&&i.writable)i.value=n;else{var u=i.set;void 0!==u&&u.call(o,n)}return n},r.slicedToArray=function(){function e(e,r){var t=[],n=!0,o=!1,i=void 0;try{for(var a,u=e[Symbol.iterator]();!(n=(a=u.next()).done)&&(t.push(a.value),!r||t.length!==r);n=!0);}catch(e){o=!0,i=e}finally{try{!n&&u.return&&u.return()}finally{if(o)throw i}}return t}return function(r,t){if(Array.isArray(r))return r;if(Symbol.iterator in Object(r))return e(r,t);throw new TypeError("Invalid attempt to destructure non-iterable instance")}}(),r.taggedTemplateLiteral=function(e,r){return Object.freeze(Object.defineProperties(e,{raw:{value:Object.freeze(r)}}))},r.temporalRef=function(e,r,t){if(e===t)throw new ReferenceError(r+" is not defined - temporal dead zone");return e},r.temporalUndefined={},r.toArray=function(e){return Array.isArray(e)?e:Array.from(e)},r.toConsumableArray=function(e){if(Array.isArray(e)){for(var r=0,t=Array(e.length);r<e.length;r++)t[r]=e[r];return t}return Array.from(e)}}("undefined"==typeof global?self:global);</script>
<script>if (!window.customElements) { document.write('<!--'); }</script>
<script type="text/javascript" src="/bower_components/webcomponentsjs/custom-elements-es5-adapter.js"></script>
<!--! do not remove -->
<script src="/bower_components/webcomponentsjs/webcomponents-loader.js"></script>
<link rel="import" href="/src/build-test-app/build-test-app.html">
<build-test-app></build-test-app>
The <!--! do not remove --> part is super not kidding and it integral to making sure the compiled code works in ES6 environments.
Beyond that, there is a whole world of things that you can do to tune this up for production deployment. I'd research https://www.polymer-project.org/2.0/docs/tools/polymer-cli-commands#build and https://www.polymer-project.org/2.0/docs/tools/polymer-json to get a better handle on that.
Re polymer build "This command is for app projects only." (https://www.polymer-project.org/2.0/docs/tools/polymer-cli-commands#build)
Also, shell is required. See https://www.polymer-project.org/2.0/docs/tools/polymer-json#shell.

Coldfusion and SVNKit Log

I am trying to use SVNKit to get a Log of the SVN Entries in Coldfusion. I downloaded the latest SVNKit jar files and threw them in the lib folder under WEB-INF/lib.
Here is my code that should return an Array Of Log Entries but this code is returning a Null Pointer exception in Coldfusion 9.0.2.
SVNURL = createObject('java','org.tmatesoft.svn.core.SVNURL').parseURIEncoded(svnurl);
drf = createObject("java","org.tmatesoft.svn.core.internal.io.dav.DAVRepositoryFactory");
drf.setup();
rf = drf.create(SVNURL);
SVNWCUtil = createObject("java","org.tmatesoft.svn.core.wc.SVNWCUtil");
authManager = SVNWCUtil.createDefaultAuthenticationManager(user,pass);
rf.setAuthenticationManager(authManager);
log = rf.log(JavaCast("String[]",[]),JavaCast("null",""),JavaCast("long",10),JavaCast("long",15),true,true);
rf.closeSession();
When running this code, I receive the following Error.
The system has attempted to use an undefined value, which usually indicates a programming error, either in your code or some system code.
Null Pointers are another name for undefined values.
Which points to this line..
log = rf.log(JavaCast("String[]",[]),JavaCast("null",""),JavaCast("long",10),JavaCast("long",15),true,true);
I moved this same code over to Railo, and everything is running fine. I just cannot see why ACF is choking on the log() method.
I was using the Printing Out Repository History example on the SVNKit Wiki to start me off.
Any suggestions on getting it to work in Adobe Coldfusion would be greatly appreciated. I did not test on CF10.
I wasn't using the JavaCast("boolean",true) for the last two arguments in the log() function. After that, everything worked fine.
Got to remember to check and use JavaCast()!
log = rf.log(JavaCast("String[]",[]),JavaCast("null",""),JavaCast("long",10),JavaCast("long",15),JavaCast("boolean",true),JavaCast("boolean",true));

Getting an incorrect permissions screen in the webview Blackberry 10 Cascades Beta 3 SDK in Dev Alpha Simulator

I am trying to make dynamically generated html 5 graphs show up in a webview in Blackberry 10 Cascades. I have confirmed the html5 that I have generated, draws the correct graphs. My problem is that when I try to implement this in the Blackberry 10 Cascades Beta 3 SDK (using the Blackberry 10 Dev Alpha Simulator), the webview that is supposed to show the graph, just looks like this:
Here is the code that leads to this error:
//html_ already contains the html-5 code to make the graph at this point in the code
//This is the file path to a local file that is actually accessable in the emulator
//and not just from Windows
//
QFile *chartFile = new QFile("app/native/assets/data/chart.html");
if (chartFile->open(QIODevice::WriteOnly)) {
chartFile->write(html_.toUtf8());
chartFile->flush();
chartFile->close();
}
if (chartFile) delete chartFile;
if (graphView_) {
graphView_->setHtml("");
graphView_->setUrl(QUrl::fromLocalFile("app/native/assets/data/chart.html"));
}
I checked the permissions of that file, put they are all Allow (777 permissions for those who know Unix style permissions).
I added access_internet to the bar-descriptor.xml, eventhough my app was already able to access remote sites, just to see if that would fix it, but it did not.
I've been searching around trying to find a solution to this problem, but I have not.
If anyone could help me out with this, it would be greatly appreciated.
-------------------------------------------------------
Update:
I changed the code to set the html directly, now I have this:
if (graphView_) {
graphView_->setHtml(html_, QUrl("app/native/assets/data/chart.html"));
}
But nothing shows. It seems I have the wrong relative path relative to my base url.
My base url is this: QUrl("app/native/assets/data/chart.html")
My relative paths all begin with: ./Highcharts/js/...
My relative paths are located under: app/native/assets/data/Highcharts/js
It seems to me that I this should work, but when I do this, I just a blank screen, as if it can not find my relative paths. So I don't know what's going on here either.
I found a solution that works. I'm using the first approach, not the updated approach, but instead of
graphView_->setUrl(QUrl("app/native/assets/data/chart.html"));
I'm using:
graphView_->setUrl(QUrl("local:///assets/data/chart.html"));
And I have left the rest of the code the same, and it works.

Using hogan.js with express.js + vhosts

What is the correct way to use hogan.js with express.js?
I've tried the following:
var hogan = require('hogan.js')
...
app.set('view engine', 'hogan');
followed by
app.register('.hogan', hogan);
But I end up with the following error:
500 Error: Cannot find module 'hogan'
TJ put out a library called consolidate.js ( https://github.com/visionmedia/consolidate.js ) but I'm having trouble getting it to work with Express 2.5.8. After spending the day trying to figure this out I also came across a library called hulk-hogan.js ( https://github.com/quangv/hulk-hogan ) and another called hogan-express ( http://allampersandall.blogspot.com/2011/12/hoganjs-expressjs-nodejs.html ). But, do I really need all that?
If the solution can not be as simple as setting the templating engine with app.set() and app.register(), it would be great if someone could help me understand why. I'm using Hogan on the client and it's working great, it would just be so much better if I could also use it on the server.
UPDATE: Turns out there are two issues here.
While this is not causing the 500 error, Express does not work with Hogan out of the box (see: Linus G Thiel's answer below)
What seems to be causing the 500 error is that I'm using a virtual host and when I call res.render(), my res.render() call is actually calling the res.render() of a different virtual host on my same server.
Adding the full Express error dump. It looks like my app ('dataviz') is trying to use the render call from a different app ('datavizblocks')? Again, the two apps are virtual hosts on the same server.
dataviz 8000
Error: Cannot find module 'hogan.js'
at Function._resolveFilename (module.js:332:11)
at Function._load (module.js:279:25)
at Module.require (module.js:354:17)
at require (module.js:370:17)
at View.templateEngine (/localhost/datavizblocks/node_modules/express/lib/view/view.js:134:38)
at Function.compile (/localhost/datavizblocks/node_modules/express/lib/view.js:68:17)
at ServerResponse._render (/localhost/datavizblocks/node_modules/express/lib/view.js:417:18)
at ServerResponse.render (/localhost/datavizblocks/node_modules/express/lib/view.js:318:17)
at /localhost/dataviz/routes/section.js:325:7
at callbacks (/localhost/dataviz/node_modules/express/lib/router/index.js:272:11)
dataviz 8000
Error: Cannot find module 'hogan.js'
at Function._resolveFilename (module.js:332:11)
at Function._load (module.js:279:25)
at Module.require (module.js:354:17)
at require (module.js:370:17)
at View.templateEngine (/localhost/datavizblocks/node_modules/express/lib/view/view.js:134:38)
at Function.compile (/localhost/datavizblocks/node_modules/express/lib/view.js:68:17)
at ServerResponse._render (/localhost/datavizblocks/node_modules/express/lib/view.js:417:18)
at ServerResponse.render (/localhost/datavizblocks/node_modules/express/lib/view.js:318:17)
at /localhost/dataviz/routes/section.js:325:7
at callbacks (/localhost/dataviz/node_modules/express/lib/router/index.js:272:11)
The 500 error goes away when I comment out the datavizblock vhost, or when I switch the order of the vhost declarations around to have the dataviz vhost declared after datavizblocks vhost (of course, this then causes problems for the datavizblocks vhost)
Apologies ahead of time for the confusing question, but I was really confused when I came across this issue and never expected that switching to Hogan would have conflicts with virtual hosting.
The issue is that Express requires an interface from template engines, where the template engine is expected to have a compile method, and that compile method is expected to return a function which can be called with the template data.
Hogan has a compile method, but it returns a template object which has a render method. You need to expose that render method to Express, and this seems to be what the hogan-express module does. It shouldn't have to be that involved though, I think this will work (I have only tested it slightly, might be some gotchas?):
var express = require('express'),
hogan = require('hogan.js'),
app = express.createServer();
app.set('view engine', 'hogan');
app.register('hogan', {
compile: function() {
var t = hogan.compile.apply(hogan, arguments);
return function() {
return t.render.apply(t, arguments);
}
}
});
Basically, we are just creating our own object that has a compile method that maps to Hogan's render method.
This expects your templates to be named e.g. index.hogan.
As Linus said, you need an adapter to use Hogan with Express. consolidate works fine as long as you don't need support for partials or layouts (they are working on it but I don't know when it will be ready).
I was in the same spot you're in a few months ago and found hulk-hogan's and express-hogan's documentations to be quite confusing so I coded my own wrapper that has support for partials, layouts, template caching and can be plugged in Express in one line of code. You can check it out here: h4e - templating with hogan for express