PROC SCAPROC Doesn't read Views created using PROC SQL - sas
I'm building a tool to automate lineage between SAS jobs using SCAPROC and during my testing I found out that if a view is created using proc sql, SCAPROC doesn't capture the input for it and I end up with Jobs that doesn't have inputs.
Does anyone have any idea how to get around this?
Thanks
Views don't involve processing input data, so it makes sense SCAPROC would delay resolving it until it's actually used. While it's unfortunate that it's inconsistent with the data step views, it's not insurmountable.
With this code:
proc scaproc;
record 'h:\record.txt' attr;
run;
data test;
set sashelp.class;
run;
data testv/view=testv;
set sashelp.class;
run;
data result;
set testv;
run;
proc sql;
create view testv_sql as
select * from sashelp.class;
quit;
data result_sql;
set testv_sql;
run;
proc scaproc;
write;
run;
You see this output, around the data step where we use the view:
/* JOBSPLIT: TASKSTARTTIME 16JUL2021:09:46:59.00 */
/* JOBSPLIT: DATASET INPUT MULTI WORK.TESTV_SQL.VIEW */
/* JOBSPLIT: LIBNAME WORK V9 '...' */
/* JOBSPLIT: DATASET INPUT SEQ #C00001.CLASS.DATA */
/* JOBSPLIT: LIBNAME #C00001 V9 '...' */
/* JOBSPLIT: CONCATMEM #C00001 SASHELP */
/* JOBSPLIT: LIBNAME SASHELP V9 '( ... )' */
/* JOBSPLIT: DATASET INPUT SEQ SASHELP.CLASS.DATA */
/* JOBSPLIT: LIBNAME SASHELP V9 '( ... )' */
/* JOBSPLIT: DATASET OUTPUT SEQ WORK.RESULT_SQL.DATA */
/* JOBSPLIT: LIBNAME WORK V9 '...' */
/* JOBSPLIT: FILE OUTPUT h:\record.txt */
/* JOBSPLIT: ATTR WORK.TESTV_SQL.VIEW INPUT VARIABLE:Name TYPE:CHARACTER LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.TESTV_SQL.VIEW INPUT VARIABLE:Sex TYPE:CHARACTER LENGTH:1 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.TESTV_SQL.VIEW INPUT VARIABLE:Age TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.TESTV_SQL.VIEW INPUT VARIABLE:Height TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.TESTV_SQL.VIEW INPUT VARIABLE:Weight TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR #C00001.CLASS.DATA INPUT VARIABLE:Name TYPE:CHARACTER LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR #C00001.CLASS.DATA INPUT VARIABLE:Sex TYPE:CHARACTER LENGTH:1 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR #C00001.CLASS.DATA INPUT VARIABLE:Age TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR #C00001.CLASS.DATA INPUT VARIABLE:Height TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR #C00001.CLASS.DATA INPUT VARIABLE:Weight TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR SASHELP.CLASS.DATA INPUT VARIABLE:Name TYPE:CHARACTER LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR SASHELP.CLASS.DATA INPUT VARIABLE:Sex TYPE:CHARACTER LENGTH:1 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR SASHELP.CLASS.DATA INPUT VARIABLE:Age TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR SASHELP.CLASS.DATA INPUT VARIABLE:Height TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR SASHELP.CLASS.DATA INPUT VARIABLE:Weight TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.RESULT_SQL.DATA OUTPUT VARIABLE:Name TYPE:CHARACTER LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.RESULT_SQL.DATA OUTPUT VARIABLE:Sex TYPE:CHARACTER LENGTH:1 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.RESULT_SQL.DATA OUTPUT VARIABLE:Age TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.RESULT_SQL.DATA OUTPUT VARIABLE:Height TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: ATTR WORK.RESULT_SQL.DATA OUTPUT VARIABLE:Weight TYPE:NUMERIC LENGTH:8 LABEL: FORMAT: INFORMAT: */
/* JOBSPLIT: SYMBOL GET SYS_SQL_UNPUT_TRACE */
/* JOBSPLIT: SYMBOL GET SQLMAGIC */
/* JOBSPLIT: ELAPSED 19 */
/* JOBSPLIT: PROCNAME DATASTEP */
/* JOBSPLIT: STEP SOURCE FOLLOWS */
That gives you the input for that data step - the dataset passed through the view - which would be the proper input for that data step.
If this is a big problem for you, you could create a full pass-through view in the data step just before the PROC SQL view - then treat it in your lineage as the true input (and ignore the SQL created one).
data result_sql;
set testv_pass;
run;
That would then give you what you need (the input and the output).
Related
Chartjs - Fill dates between given labels
I've been experimenting with Chartjs for several hours now and I'm not sure if this is possible. I have some data that is sorted date wise but not all dates are included. For example: const salesChart = new Chart(document.getElementById('graph-sales'), { type: 'line', data: { labels: ['Aug 23','Aug 26','Aug 31','Sep 02'], datasets: [{ label: 'Sales', borderColor: '#ccc', data: [2750.00,1100.00,3080.00,4320.00], }] } } ); Chartjs plots this data as 4 datapoints and joins them with a line like so This is fine, but I want intermediate days to be added on the chart with datapoint value being 0. So essentially it would be like passing this data: labels: ['Aug 23','Aug 24','Aug 25','Aug 26','Aug 27','Aug 28','Aug 29','Aug 30','Aug 31','Sep 01','Sep 02'], datasets: [{ label: 'Sales', borderColor: '#ccc', data: [2750.00,0,0,1100.00,0,0,0,0,3080.00,0,4320.00], }] I've looked into timescales but can't get them to work. The docs says I need a time adapter but there's no example of it in use so I'm not sure what that means.
It would probably be easier to do this on the backend serving up the data; but this code works if you want to do it in JavaScript on the frontend. My method was to use a defined self-executing function which returns an array containing two arrays. The first array is called finalLabels and contains all date strings between, and including, the dates provided in the original labels. Which gives us ['Aug 23', 'Aug 24', 'Aug 25', 'Aug 26', 'Aug 27', 'Aug 28', 'Aug 29', 'Aug 30', 'Aug 31', 'Sep 01', 'Sep 02'] The second returned array is called finalDatas and contains all of the original data values at the same index of the original label; and a value of zero where the value wasn't previously defined. Which gives us: [2750, 0, 0, 1100, 0, 0, 0, 0, 3080, 0, 4320] Working Codepen: https://codepen.io/vpolston/pen/NWMgwOw My account doesn't have ability to embed pictures, but this is what you end up with: image of final chart created. JS const labels = ['Aug 23','Aug 26','Aug 31','Sep 02']; const data = [2750.00,1100.00,3080.00,4320.00]; const datasets = (function () { const months = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul","Aug", "Sep", "Oct", "Nov", "Dec"]; /* define start and end date from labels and create date objects */ const startDateString = labels[0]; const endDateString = labels[labels.length - 1]; const startDateObj = new Date(startDateString); const endDateObj = new Date(endDateString); /* create empty dates array to hold dates within range */ let dates = []; /* create currentDateObj var to increment in while loop loop through, add currentDateObj to array, increment until false */ let currentDateObj = new Date(startDateString); while( currentDateObj <= endDateObj ){ /* format date to match the provided label and push to dates array */ let dateString = currentDateObj.toDateString(); let month = months[currentDateObj.getMonth()]; let day = currentDateObj.getDate(); if( day < 10 ){ day = '0' + day }; let date = month + ' ' + day; dates.push(date); /* increment CurrentDateObj */ currentDateObj.setDate(currentDateObj.getDate() + 1); }; /* use counter to loop through original datas */ let valueExistsCounter = 0; let finalLabels = []; let finalDatas = []; for( const [index, date] of dates.entries() ){ if( labels.includes(date) ){ /* if date was provided in labels get the data value */ finalLabels.push(date); finalDatas.push(data[valueExistsCounter]); valueExistsCounter += 1 } else { /* set date value to 0 */ finalLabels.push(date); finalDatas.push(0); } }; return [finalLabels, finalDatas] }()); const finalLabels = datasets[0]; const finalDatas = datasets[1]; /* now we can build the chart */ const ctx = document.getElementById('myChart').getContext('2d'); const myChart = new Chart(ctx, { type: 'line', data: { labels: finalLabels, datasets: [{ label: 'Sales', 'fill': true, borderColor: '#ccc', backgroundColor: 'rgba(204,204,204,0.5)', tension: 0.2, data: finalDatas }] }, options: { scales: { x: { grid: { color: 'rgba(204,204,204,0.1)' } }, y: { grid: { color: 'rgba(204,204,204,0.1)' } } } } }); HTML <!-- Canvas --> <div class="chartContainer"> <canvas id="myChart"></canvas> </div> <!-- include Chart.js 3.9.1 --> <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/3.9.1/chart.js" integrity="sha512-d6nObkPJgV791iTGuBoVC9Aa2iecqzJRE0Jiqvk85BhLHAPhWqkuBiQb1xz2jvuHNqHLYoN3ymPfpiB1o+Zgpw==" crossorigin="anonymous" referrerpolicy="no-referrer"></script> CSS body { background-color: #0d1117; } .chartContainer { width: 800px; height: 400px; } I think that covers it? If that was helpful please mark answered.
change date format in tooltip of chart.js
I am using the chart.js to generate a chart. All works fine ! But how can I format the tooltip information? It should be: 28.04.2020 - 05:00
You need to define time.tooltipFormat for your xAxis. See Moment.js for the allowed format tokens. options: { ... scales: { xAxes: [{ type: 'time', time: { unit: 'hour', tooltipFormat: 'DD.MM.YYYY - HH:mm' } ... }] } ... }
Change the background color of tooltip dynamically using chart color
I have a line chart with 4 datasets, I want to show different tooltip backgrounds for all 4 lines. but tooltip doesn't support dynamic background color, anyway how to do this?
Working "dynamic" example. ***keep in mind the access to data related to your data structure. Using tooltip-model https://www.chartjs.org/docs/2.7.3/configuration/tooltip.html#tooltip-model "hello world" example - Change all tooltips background to red: tooltips: { custom: function(tooltipModel) { tooltipModel.backgroundColor = "red"; } }, Code Snippet: var data = { labels: ["Africa", "Asia", "Europe", "America"], datasets: [{ /* data */ label: "Population (millions)", backgroundColor: ["red", "blue","green", 'purple'], data: [1000,1500,2000, 2200] }] }; var options = { title: { text: 'Dynamically change tooltip background example', display: true }, tooltips: { titleFontSize: 20, borderWidth: 2, borderColor: "white", displayColors: false, /* if true, color boxes are shown in the tooltip */ /*########### Custom model ###########*/ custom: function(tooltipModel) { /* if data & datasets not empty & tooltip available */ if (tooltipModel.opacity !== 0 && data.labels.length && data.datasets.length) { /* get dataPoints index */ var index = tooltipModel.dataPoints[0].index; /* get dataPoints datasetIndex */ var dataSetIndex = tooltipModel.dataPoints[0].datasetIndex; /* get the current color on index and datasetIndex position */ var color = data.datasets[dataSetIndex].backgroundColor[index]; /* set backgroundColor */ tooltipModel.backgroundColor = color; }; } }, scales: { xAxes: [{ stacked: true }], yAxes: [{ stacked: true }] } }; var myChart = new Chart(document.getElementById("chart"), { type: 'bar', data: data, options: options }); <canvas id="chart" width="800" height="450"></canvas> <script src="https://cdn.jsdelivr.net/npm/chart.js#2.8.0"></script> The outline of the code "if" (To avoid console errors): /*########### Custom model ###########*/ custom: function(tooltipModel) { /* if data & datasets not empty & tooltip available */ if (tooltipModel.opacity !== 0 && data.labels.length && data.datasets.length) { /* do something */ console.log(tooltipModel.dataPoints[0]); /* return object */ console.log Return object: Object { datasetIndex: 0, index: 1, label: "Asia", value: "1500", x: 338.6845703125, xLabel: "Asia", y: 215.28, yLabel: 1500 } Than we use dot (.) notation to access the object values. console.log(tooltipModel.dataPoints[0].index); /* return 1 */ We use this index "anchor" to get the correct index for backgroundColor array: console.log(data.datasets[dataSetIndex].backgroundColor[index]); /* return "blue" Last step is to use this color value: /* set backgroundColor */ tooltipModel.backgroundColor = color; UI Usefull to hide color boxes: displayColors: false, /* if true, color boxes are shown in the tooltip */ https://www.chartjs.org/docs/2.7.3/configuration/tooltip.html#tooltip-configuration codepen: https://codepen.io/ezra_siton/pen/dyoQeGe?editors=1011
I also had same problem today.Solution is implementing custom tooltip method but you dont need to create custom tooltip from scratch. colorArray=["blue","red","green"]; tooltips: { custom: function(tooltipModel) { tooltipModel.backgroundColor=this.colorArray[tooltipModel.dataPoints[0].index]; }, }, This code worked for me.Depends on which tooltip you click it will bring the index of color from colorArray.
Error with Angular 2 and Jasmine "Error at Injection Error"
I am trying to run Angular 2 tests with webpack and Karma but am getting an error: Chrome 57.0.2987 (Windows 7 0.0.0) HeaderBarComponent should have a defined component FAILED Error at injectionError (webpack:///~/#angular/core/#angular/core.es5.js:1231:21 <- test/index.js:1558:86) [angular] at noProviderError (webpack:///~/#angular/core/#angular/core.es5.js:1269:0 <- test/index.js:1596:12) [angular] at ReflectiveInjector_._throwOrNull (webpack:///~/#angular/core/#angular/core.es5.js:2770:0 <- test/index.js:3097:19) [angular] at ReflectiveInjector_._getByKeyDefault (webpack:///~/#angular/core/#angular/core.es5.js:2809:0 <- test/index.js:3136:25) [angular] at ReflectiveInjector_._getByKey (webpack:///~/#angular/core/#angular/core.es5.js:2741:0 <- test/index.js:3068:25) [angular] at ReflectiveInjector_.get (webpack:///~/#angular/core/#angular/core.es5.js:2610:0 <- test/index.js:2937:21) [angular] at DynamicTestModuleInjector.NgModuleInjector.get (webpack:///~/#angular/core/#angular/core.es5.js:3557:0 <- test/index.js:3884:52) [angular] at resolveDep (webpack:///~/#angular/core/#angular/core.es5.js:10930:0 <- test/index.js:11257:45) [angular] at createClass (webpack:///~/#angular/core/#angular/core.es5.js:10791:0 <- test/index.js:11118:91) [angular] at createDirectiveInstance (webpack:///~/#angular/core/#angular/core.es5.js:10627:21 <- test/index.js:10954:37) [angular] at createViewNodes (webpack:///~/#angular/core/#angular/core.es5.js:11977:33 <- test/index.js:12304:49) [angular] at createRootView (webpack:///~/#angular/core/#angular/core.es5.js:11882:0 <- test/index.js:12209:5) [angular] at callWithDebugContext (webpack:///~/#angular/core/#angular/core.es5.js:13013:25 <- test/index.js:13340:42) [angular] at Object.debugCreateRootView [as createRootView] (webpack:///~/#angular/core/#angular/core.es5.js:12474:0 <- test/index.js:12801:12) [angular] Chrome 57.0.2987 (Windows 7 0.0.0): Executed 1 of 1 (1 FAILED) ERROR (0.949 secs / 0.531 secs) header-bar.spec.ts: import { TestBed, async, inject } from '#angular/core/testing'; import { RouterModule, Router} from '#angular/router'; import { FlexLayoutModule } from '#angular/flex-layout'; import { MaterialModule } from '#angular/material'; import { HeaderBarComponent } from '../../../src/app/components/layout/header-bar/header-bar.ts'; describe('HeaderBarComponent', () => { var component; beforeEach(() => { TestBed.configureTestingModule({ imports: [RouterModule, MaterialModule, FlexLayoutModule], declarations: [HeaderBarComponent] }); **// error occurs on this line** const fixture = TestBed.createComponent(HeaderBarComponent); component = fixture.componentInstance; }); it('should have a defined component', () => { expect(component).toBeDefined(); }); }); If I comment out "TestBed.createComponent(HeaderBarComponent)" the test will run but I need that line to be able to test the component don't I? Looks like a lot of the Angular docs use that line..
It's looks like you didn't write extra functions: ... beforeEach(async(() => { TestBed.configureTestingModule({ //Mainly you don't need RouterModule here imports: [MaterialModule, FlexLayoutModule], declarations: [HeaderBarComponent] }) //Do not forget to compile the component .compileComponents(); const fixture = TestBed.createComponent(HeaderBarComponent); component = fixture.componentInstance; //Do not forget to call that method when u need // to trigger change detection fixture.detectChanges(); })); ...
"Table has no rows" Error in Google Charts Histogram
I am working with Google Histogram chart. It working fine with some data sets but not for other data sets. And it raise an error "Table has no rows" even my input is correct. Here i am reading a csv file column wise and pass to visualization page. for eg: I am reading 2 csv column here and passing to visualization page. Here my input to Google histogram is var inputdata1 = [["val","d"],["val","2"],["val","2"],["val","1"],["val","2"],["val","2"],["val","1"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","1"],["val","2"]]; and this working fine and gives histogram for me. while I am passing other 2 columns.Here my input to Google histogram is var inputdata2 = [["val","b"],["val","3"],["val","3"],["val","3"],["val","5"],["val","1"],["val","12"],["val","7"],["val","11"],["val","1"],["val","7"],["val","6"],["val","16"],["val","11"],["val","21"],["val","12"],["val","1"],["val","22"],["val","16"],["val","1"],["val","21"],["val","11"],["val","6"],["val","11"],["val","15"],["val","12"],["val","12"]]; while executing this, it raise an error that "Table has no rows" . Please check my fiddle. Any help would be greatly appreciated. Thank you in Advance.
In fact neither inputdata1 nor inputdata2 contain JSON data that are supported by histogram chart. According to the documentation the following formats are supported: Data Format There are two ways to populate a histogram datatable. When there's only one series: var data = google.visualization.arrayToDataTable([ ['Name', 'Number'], ['Name 1', number1], ['Name 2', number2], ['Name 3', number3], ... ]); ...and when there are multiple series: var data = google.visualization.arrayToDataTable([ ['Series Name 1', 'Series Name 2', 'Series Name 3', ...], [series1_number1, series2_number1, series3_number1, ...], [series1_number2, series2_number2, series3_number2, ...], [series1_number3, series2_number3, series3_number3, ...], ... ]); Having said that you might want to convert the second column into number format: var inputJson = [["val","b"],["val","3"],["val","3"],["val","3"],["val","5"],["val","1"],["val","12"],["val","7"],["val","11"],["val","1"],["val","7"],["val","6"],["val","16"],["val","11"],["val","21"],["val","12"],["val","1"],["val","22"],["val","16"],["val","1"],["val","21"],["val","11"],["val","6"],["val","11"],["val","15"],["val","12"],["val","12"]]; var chartJson = inputJson.map(function(item,i){ if(i == 0) return item; else { return [item[0],parseInt(item[1])]; } }); var data = google.visualization.arrayToDataTable(chartJson); Once the data is converted the chart will be rendered properly. Working example google.load('visualization', '1.1', { 'packages': ['corechart'] }); google.setOnLoadCallback(drawStuff); function drawStuff() { var inputJson = [["val","b"],["val","3"],["val","3"],["val","3"],["val","5"],["val","1"],["val","12"],["val","7"],["val","11"],["val","1"],["val","7"],["val","6"],["val","16"],["val","11"],["val","21"],["val","12"],["val","1"],["val","22"],["val","16"],["val","1"],["val","21"],["val","11"],["val","6"],["val","11"],["val","15"],["val","12"],["val","12"]]; var chartJson = inputJson.map(function(item,i){ if(i == 0) return item; else { return [item[0],parseInt(item[1])]; } }); var data = google.visualization.arrayToDataTable(chartJson); //The below input data works fine. //var data = google.visualization.arrayToDataTable([["val","d"],["val","2"],["val","2"],["val","1"],["val","2"],["val","2"],["val","1"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","2"],["val","1"],["val","2"]]); // Set chart options var options = { width: 400, height: 300, histogram: { bucketSize: 0.1 } }; // Instantiate and draw our chart, passing in some options. var chart = new google.visualization.Histogram(document.getElementById('chart_div')); chart.draw(data, options); }; <script type="text/javascript" src="http://www.google.com/jsapi"></script> <script src="chart.js"></script> <div id="chart_div"></div>