How do I chain stored processes together using only EG4.3? - sas

I'm creating an HTML based data dictionary using stored processes in EG4.3 running on 9.2 BI Suite on Unix.
We have had our middle tier web servers effectively turned off, so this STP can only be used inside EG itself.
Using a plethora of puts, I have created the HTML pages that I require in four stored processes:
The main page, that takes as a prompt input, a wildcard for (for example) a column in question and returns in HTML a list of datasets where this column appears.
From this, the user can select a library, table or column to drill down into more details
A stored process that returns in HTML a list of datasets in a selected library;
A stored process that returns in HTML a list of columns in a selected table;
A stored process that returns in HTML a more detailed listing of a columns attributes.
The first STP works great and I can see a nicely formatted listing of information, with library, dataset and column links.
My problem is as follows: When I click on a link, I would like that to run the requisite STP, and return the HTML page to the same Results tab that my initial report is in.
I've checked endless sas articles and can't find an answer.
Here is the code I'm using to generate a link to the other STP's (where chars[i] is an a-z value from a dataset):
/* Get the root session ID*/
hrefroot = symget('_THISSESSION') || '&_PROGRAM=/Shared Data/Team/stored_process/';
/* Get the next stp path (neither ?&_PROGRAM= nor the below seem to work)*/
hrefnextstp = "?_PROGRAM=/Shared Data/Team/stored_process/";
// build up the HTML using puts //
stpval="stpddlibrary";
stplink=cats('', chars[i],'');
put stplink;
How, with the above in mind, can I chain together multiple stored processes using just EG?
There's no &_URL seen as though I'm not using the web, so that ruled that out.
I hope I've articulated that well..

The problem you are encountering is that the HTML page is trying to navigate to a web service to pull the next page. However, EG isn't able to function as a web server to respond the browsers request.
The stored process code is best used via the web mid-tier or the SAS MS Office plug-in. If those two tools are not available, you might consider moving the logic out of the SAS Stored Proc and into EG directly and use the response to drive a second flow in your eg project. If you are looking to drive a web page you can call sas via Java and the SAS IOM Bridge and talk to SAS the same way EG does but your UI in this case would be build in Java. Another suggestion, try using vbscript to drive EG via COM.
Here is a link to controlling EG via COM:
http://www.sascommunity.org/wiki/Not_Just_for_Scheduling:_Doing_More_with_SAS_Enterprise_Guide_Automation
The code you provided would work well to build the url as serviced by the SAS Mid-Tier stored proc web service. Stored in a SAS Stored proc you can build the url to the mid-tier pointing at other stored procs using the SAS provided web interface for stored procs to see how the parameters and url is constructed.
If your mid-tier web service was online, you might also be able to call the stored procedure in your code. Calling it via proc http, here is an example from sas docs:
proc http in=in out=out url="http://localhost.com/rsm/rstool"
method="post"
ct="application/x-www-form-urlencoded";
run;

Related

Importing data from Smartsheet into Power BI using web and not connector

The PowerBI connector doesn't work, since it defaults to the IE browser, and our IE browser is locked down by our IT department (can't connect to smartsheet site). So, I need to use the "web" API option to get my smartsheet data. I have the data coming in, but I'm brand new at this and I'm having trouble organizing it. I have 8 columns that are repeating as rows, and the cell data contains a bunch of metadata values that I don't want. How do I get this so the columns become columns again, with the cell data as rows (with just the face values) and not repeating?
screenshot of data in PowerBI
I think you need to read a good part of the documentation in order to get used to the responses you get from Smarsheet, in brief you get a Json data structure.
First read, https://smartsheet-platform.github.io/api-docs/#how-to-read-a-sheet-response
Then you need to make the json connection to PowerBI like this example, https://www.mssqltips.com/sqlservertip/4621/using-power-bi-with-json-data-sources-and-files/

Tableau Desktop: Live Vs Extract and Publishing to Tableau Server

I have a question that I can't seem to find a straightforward answer to. I am loading a roughly 20GB CSV file into Tableau Desktop to create some worksheets for future internal views. Where I am stumbling is whether to first use Extract or Live data source. The data itself will not change, only the reports or worksheets generated based on the data. The computations within the worksheets on Tableau Desktop take forever to complete.
On to the publishing to Tableau Server. I am under the assumption that I must upload my locally stored file to the server for future use. Would it be better to find a network drive to have the Tableau Server data source point to?
In short, what is the best method to manipulate a large dataset on Tableau Desktop and present on Tableau Server? Additionally, what Regex methodology does Tableau follow, as it doesn't seem to follow standards I use for Python.
Thanks!
When publishing a workbook with a file-based data source, choose the Include external files option in the publishing to server dialog box. This will eliminate the need to have the file in a networked location accessible by the server.
This approach only works if the data doesn't change. It remains static and embedded in the workbook. Under this option, if the data changes and you want your viz to reflect changes, you would need to update the data in Desktop and republish.
As you mentioned, you have to connect/load your 20GB csv file into Tableau Desktop for proper visualizations, if I understand your requirement correctly.
Please find below steps to do the same:
Yes, you have to manually update your data in that csv file whenever you want to show the same on Tableau (Note: Make sure "Name" of entire csv file & columns remain same)
After this on opening Tableau, you have to click on "Refresh data source" to get visualization of latest data present in csv file
enter image description here
For this case, I would say connection (LIVE / EXTRACT) won't help as this there is no role of this, However please use EXTRACT technique as per my understanding (means extract will update once we load latest data)

Framework selection for a new project?

Problem Context
We have a set of excel reports which are generated from an excel input provided by the user and then fed into SAS for further transformation. SAS pulls data from Teradata database and then there is a lot of manipulation that happens with the input data & data pulled from Teradata. Finally, a dataset is generated which can either be sent to the client as a report, or be used for populating Tableau dashboard. Also the database is being migrated from Teradata to Google Cloud (Big Query EDW) as the Teradata pulls from SAS used to take almost 6-7 hours
Problem Statement
Now we need to automate this whole process, by creating front end for the user to upload the input files and from there on the process should trigger and in the end the user should receive the excel file or Tableau dashboard as an attachment in a mail.
Can you suggest what technologies should be used in the front end & middle tier to make this process feasible is least possible time with google cloud platform as the backend?
Can an R shiny front end be a solution given that we need to communicate with a Google Cloud backend ?
I have got suggestion from people that Django will be a good framework to accomplish this task. What are your views on this ?

Dynamic Power BI OData data source location

We have a web application which is installed on customer sites.
They get to choose the name of the web server and the port that it runs on.
The web application exposes some data via an OData endpoint.
The web application also serves some template Power BI files (.pbit)
At the moment, when the customers run the pbit files for the first time, the data refresh fails and they have to relink their data source.
We use data source referencing and store the location of the endpoint in a parameters table but it's still a pain for the customers to have to change this value, and, since we (the web application) know the endpoint URL, we'd like to be able to update the pbit file with the appropriate setting.
Ideally we'd like an API to be able to manipulate the pbit file, or maybe to be able to supply parameters to Power BI desktop, or even load the pbit file (which is a .zip file as I'm sure you know!), and alter the appropriate part within it.
Has anyone actually managed to do programatically or dynamically change a data source URI without needing manual intervention?
Thanks.
I would use the Query Parameters feature for this (more details in the link below). Note as of the July 2016 update, the source for parameters can now be another Query.
So I would build that "parameter allowed values" query by pointing at an Azure SQL instance or some other web-visible source that helps them choose the right value.
https://powerbi.microsoft.com/en-us/blog/deep-dive-into-query-parameters-and-power-bi-templates/
https://powerbi.microsoft.com/en-us/blog/power-bi-desktop-july-feature-summary/

Web query with Power BI desktop returns empty table

I'm trying to import data from a web page to Power BI Desktop. The URL is the following:
https://casa.sapo.pt/Venda/Apartamentos/Maia/?sa=13
It contains data about housing prices, characteristics, etc.
The query returns an empty table, although if I browse to the page with any browser and without authentication or login, I see it contains the data I'm looking to analyse so I guess the publisher has disabled somehow the query I'm trying to make.
Does anyone know a workaround?
Power BI / Power Query gives quite disappointing results against that fairly typical page structure. It only seems to work well against "Web 1.0" style HTML tables. There are so few of those around (outside of Wikipedia) that it is almost pointless ...
You could try Chris Webb's ExpandAll function (link below) that will expand out all the data available.
http://blog.crossjoin.co.uk/2014/05/21/expanding-all-columns-in-a-table-in-power-query/
Then you can exclude all the non ".Text" columns and merge them together to get all the text from the page in one column - but it's hard to make sense of it. You also lose a lot of the HTML content eg links, image URLs.