I'm trying to relabel a bunch of Power BI reports and I am doing that with this script.
(Source) =>
let
res = List.Accumulate(
Text.ToList(Source),
[result="", index=0, source=Source],
fnAccumulator
),
IsUpper = (txt) => txt <> "" and txt = Text.Upper(txt) and txt <> Text.Lower(txt),
fnAccumulator =
(state as record, current as text) as record =>
let
prevCharacter =
if state[index]=0 then
""
else
Text.At(state[source], state[index] - 1),
prevCharacter2 =
if state[index]<=1 then
""
else
Text.At(state[source], state[index] - 2),
nextCharacter =
if state[index] = Text.Length(state[source]) - 1 then
""
else
Text.At(state[source], state[index] + 1),
aggregatedResult =
if state[index]=0 or current = "" then
current
else
if IsUpper(current) and
(not IsUpper(prevCharacter) or
(IsUpper(prevCharacter2) and not IsUpper(nextCharacter))) then
state[result] & " " & current
else
state[result] & current,
resultRecord =
if aggregatedResult = null then
null
else
[result = aggregatedResult, index = state[index]+1, source = state[source]]
in
resultRecord
in
res[result]
The problem is after running this on every table and applying the changes all the relationships in the report end up nuked.
Before
After
My thought process is that because my script isn't taking into account SID's and renaming those when they should stay the same this is breaking the relationships. Would appreciate either an alternative way of mass renaming these columns or if there is a way to add an exclusion statement into my script that will ignore any and all SIDs
Related
Good afternoon all,
I'm trying to call all of the results within an API that has:
6640 total records
100 records per page
67 pages of results (total records / records per page)
This is an ever growing list so I've used variables to create the above values.
I can obviously use the $Skip ODATA expression to get any one of the 67 pages by adding the expression to the end of the URL like so (which would skip the first 100, therefore returning the 2nd page:
https://psa.pulseway.com/api/servicedesk/tickets/?$Skip=100
What I'm trying to do though is to create a custom function that will loop through each of the 67 calls, changing the $Skip value by an increment of 100 each time.
I thought I'd accomplished the goal with the below code:
let
Token = "Token",
BaseURL = "https://psa.pulseway.com/api/",
Path = "servicedesk/tickets/",
RecordsPerPage = 100,
CountTickets = Json.Document(Web.Contents(BaseURL,[Headers = [Authorization="Bearer " & Token],RelativePath = Path & "count"])),
TotalRecords = CountTickets[TotalRecords],
GetJson = (Url) =>
let Options = [Headers=[ #"Authorization" = "Bearer " & Token ]],
RawData = Web.Contents(Url, Options),
Json = Json.Document(RawData)
in Json,
GetPage = (Index) =>
let Skip = "$Skip=" & Text.From(Index * RecordsPerPage),
URL = BaseURL & Path & "?" & Skip,
Json = GetJson(URL)
in Json,
TotalPages = Number.RoundUp(TotalRecords / RecordsPerPage),
PageIndicies = {0.. TotalPages - 1},
Pages = List.Transform(PageIndicies, each GetPage(_))
in
Pages
I got all happy when it successfully made the 67 API calls and combined the results into a list for me to load in to a Power Query table, however what I'm actually seeing is the first 100 records repeated 67 times.
That tells me that my GetPage custom function which handles the $Skip value isn't changing and is stuck on the first one. To make sure the Skip index was generating them properly I duplicated the query and changed the code to load in the $Skip values and see what they are, expecting them all to be $Skip=0, what I see though is the correct $Skip values as below:
Image showing correct Skip values
It seems everything is working as it should be, only I'm only getting the first page 67 times.
I've made a couple of posts on other community site around this issue before but I realise the problem I was (poorly) describing was far too broad to get any meaningful assistance. I think now I've gotten to the point where I understand what my own code is doing and have really zoomed in to the problem - I just don't know how to fix it when I'm at the final hurdle...
Any help/advice would be massively appreciated. Thank you.
Edit: Updated following #RicardoDiaz answer.
let
// Define base parameters
Filter = "",
Path = "servicedesk/tickets/",
URL = "https://psa.pulseway.com/api/",
Token = "Token",
Limit = "100",
// Build the table based on record start and any filters
GetEntityRaw = (Filter as any, RecordStart as text, Path as text) =>
let
Options = [Headers=[ #"Authorization" = "Bearer " & Token ]],
URLbase = URL & Path & "?bearer=" & Token & "&start=" & RecordStart & "&limit=" & Text.From(Limit),
URLentity = if Filter <> null then URLbase & Filter else URLbase,
Source = Json.Document(Web.Contents(URLentity, Options)),
Result = Source[Result],
toTable = Table.FromList(Result, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
toTable,
// Recursively call the build table function
GetEntity = (optional RecordStart as text) as table =>
let
result = GetEntityRaw(Filter, RecordStart, Path),
nextStart = Text.From(Number.From(RecordStart) + Limit),
nextTable = Table.Combine({result, #GetEntity(nextStart)}),
check = try nextTable otherwise result
in
check,
resultTable = GetEntity("0")
in
resultTable
As I couldn't test your code, it's kind of hard to provide you a concrete answer.
Said that, please review the generic code I use to connect to an api and see if you can find where yours is not working
EDIT: Changed api_record_limit type to number (removed the quotation marks)
let
// Define base parameters
api_url_filter = "",
api_entity = "servicedesk/tickets/",
api_url = "https://psa.pulseway.com/api/",
api_token = "Token",
api_record_limit = 500,
// Build the table based on record start and any filters
fx_api_get_entity_raw = (api_url_filter as any, api_record_start as text, api_entity as text) =>
let
api_url_base = api_url & api_entity & "?api_token=" & api_token & "&start=" & api_record_start & "&limit=" & Text.From(api_record_limit),
api_url_entity = if api_url_filter <> null then api_url_base & api_url_filter else api_url_base,
Source = Json.Document(Web.Contents(api_url_entity)),
data = Source[data],
toTable = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
toTable,
// Recursively call the build table function
fxGetEntity = (optional api_record_start as text) as table =>
let
result = fx_api_get_entity_raw(api_url_filter, api_record_start, api_entity),
nextStart = Text.From(Number.From(api_record_start) + api_record_limit),
nextTable = Table.Combine({result, #fxGetEntity(nextStart)}),
check = try nextTable otherwise result
in
check,
resultTable = fxGetEntity("0"),
expandColumn = Table.ExpandRecordColumn(
resultTable,
"Column1",
Record.FieldNames(resultTable{0}[Column1]),
List.Transform(Record.FieldNames(resultTable{0}[Column1]), each _)
)
in
expandColumn
QUESTION TO OP:
Regarding this line:
Result = Source[Result],
Does the json return a field called result instead of data?
I need to split my products into a total of 120 predefined price clusters/buckets. These clusters can overlap and look somewhat like that:
As I dont want to write down all of these strings manually: Is there a convenient way to do this in M or DAX directly using a bit of code?
Thanks in advance!
Dave
With m-Query you can create a function. Open the query editor. Richt click and create empty query. Create function (ignore warning) and call it : RowGenerator.
Open advanced editor and past the following code:
let
Bron = (base as number, start as number, end as number) => let
Bron = Table.FromList(List.Generate(() => start, each _ <= end, each _ + 1), Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Aangepaste kolom toegevoegd" = Table.AddColumn(Bron, "Aangepast", each Number.ToText(base) & " - " & Number.ToText([Column1]))
in
#"Aangepaste kolom toegevoegd"
in
Bron
This function creates a table where base is your first number and start, end the range.
Add another empty query, open the advanged editor and paste:
let
Bron = List.Generate(() => 0, each _ < 5, each _ + 1),
#"Geconverteerd naar tabel" = Table.FromList(Bron, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Aangeroepen aangepaste functie" = Table.AddColumn(#"Geconverteerd naar tabel", "test", each RowGenerator(_[Column1], _[Column1] + 1, 5)),
#"test uitgevouwen" = Table.ExpandTableColumn(#"Aangeroepen aangepaste functie", "test", {"Column1", "Aangepast"}, {"Column1.1", "Price Cluster"}),
#"Kolommen verwijderd" = Table.RemoveColumns(#"test uitgevouwen",{"Column1", "Column1.1"})
in
#"Kolommen verwijderd"
This creates first a list of 5 rows, then it calls the previous made function for each row and the last step is to expend the rows and remove the not needed columns.
Enjoy:
You can create this bucket by DAX (New Table):
Table = SELECTCOLUMNS(
GENERATE(SELECTCOLUMNS(GENERATESERIES(0,10,1),"FirstPart",[Value]), SELECTCOLUMNS(GENERATESERIES(0,10,1),"SecondPart",[Value]))
,"Bucket", [FirstPart] & " - " & [SecondPart]
)
Table = SELECTCOLUMNS(
GENERATE(SELECTCOLUMNS(GENERATESERIES(0,9,1),"FirstPart",[Value]), TOPN([FirstPart], SELECTCOLUMNS(GENERATESERIES(1,9,1),"SecondPart",[Value]), [SecondPart],ASC))
,"Bucket", [FirstPart] & " - " & [SecondPart]
)
I've created a function that cleans my data of extra columns with null values. There should always be 15 columns after this however occasionally there is more or less and when this happens those tables should just be removed.
I've tried just skipping all those rows and returning an empty table but when I try to expand those tables I get an error "Cannot convert the value false to type Number."
(tbl as table) =>
let
ColumnNames = Table.ColumnNames(tbl),
RemoveNullColumns = Table.SelectColumns(tbl, List.Select(ColumnNames, each List.MatchesAny(Table.Column(tbl, _), each _ <> null))),
CheckColumns = Table.Skip(RemoveNullColumns, Table.ColumnCount(RemoveNullColumns) <> 15)
in
CheckColumns
See if this works for you. Removes any columns containing a null and returns tbl only if there are 15 remaining columns
(tbl as table) =>
let ColumnNames = Table.ColumnNames(tbl),
ReplacedValue = Table.ReplaceValue(tbl,null,"imanull",Replacer.ReplaceValue,ColumnNames ),
UnpivotedColumns = Table.UnpivotOtherColumns(ReplacedValue, {}, "Attribute", "Value"),
FilteredRows = Table.SelectRows(UnpivotedColumns, each ([Value] = "imanull")),
NonNullColumns= List.Difference(ColumnNames,List.Distinct(FilteredRows[Attribute])),
Results = if List.Count (NonNullColumns) <> 15 then null else Table.SelectColumns(tbl,NonNullColumns)
in Results
In Power BI Desktop i have a table from an excel file and i want to split a row based on a division between the value of a specific column and a default number.
In more details lets assume tha we have a table like this :
if the default value we want to devide column Amount is 50,then the desirable result would be something like that :
Do you have any idea how can i implement that in Power query editor or with dax?
Thanks
Tested this in Power Query for Excel, but hopefully should work for you in Power BI too. If you create a function like:
divisionToList = (numberToDivide as number, numberToDivideBy as number) as list =>
let
divisionResult = numberToDivide / numberToDivideBy,
isResultValid = (divisionResult >= 0) and (Number.Mod(divisionResult, 1) = 0),
errorIfInvalid = Error.Record("Cannot create a list with " & Text.From(divisionResult) & " items", Number.ToText(numberToDivide) & " / " & Number.ToText(numberToDivideBy) & " = " & Text.From(divisionResult), null),
listOrError = if isResultValid then List.Repeat({divisionResult}, divisionResult) else error errorIfInvalid
in listOrError,
It should divide two numbers and return a list of length d in which each element is d (d is the result of the division). This list can then, in the context of a table, be expanded into new rows.
There is some basic error handling in the function for cases where the division yields a problematic number (since you can't have a list with, for example, 5.1 elements or -1 elements). You can change/remove this handling if necessary.
I think this code below takes me from your first image to your second image -- and hopefully will give you some idea on how to go about achieving this.
let
mockData = Table.FromColumns({{200, 400}, {"A", "B"}}, type table [Amount = number, Description = text]),
defaultValue = 50, // Not sure what logic is required for arriving at this figure, so have simply assigned it.
divisionToList = (numberToDivide as number, numberToDivideBy as number) as list =>
let
divisionResult = numberToDivide / numberToDivideBy,
isResultValid = (divisionResult >= 0) and (Number.Mod(divisionResult, 1) = 0),
errorIfInvalid = Error.Record("Cannot create a list with " & Text.From(divisionResult) & " items", Number.ToText(numberToDivide) & " / " & Number.ToText(numberToDivideBy) & " = " & Text.From(divisionResult), null),
listOrError = if isResultValid then List.Repeat({divisionResult}, divisionResult) else error errorIfInvalid
in listOrError,
invokeFunction = Table.TransformColumns(mockData, {{"Amount", each divisionToList(_, defaultValue), type list}}),
expanded = Table.ExpandListColumn(invokeFunction, "Amount")
in
expanded
In database I have partitioning table by column 'status' for better performance. My database administrator ask me about put in query value for that column directly in sql (not bind by parameter).
I can change binding by set hint QueryHints.BIND_PARAMETERS on false, but then all parameters are inside sql.
Can I set not bind only on 'status' parameter ?
Example result when BIND_PARAMETERS = true
SELECT t0.* FROM S_JOBS_ORG_UNIT_CFG t0
WHERE ((((t0.ORG_ID = ?) AND (t0.SCHEDULER_NEXT_ACTIVATION < SYSDATE)) AND (t0.ACTIVE = ?))
AND NOT EXISTS (SELECT ? FROM S_JOBS t1 WHERE (((t1.ORDER_ID = t0.ORDER_ID) AND (t1.ORG_ID = t0.ORG_ID)) AND NOT ((t1.STATUS = ?)))) )
bind => [472100, Y, 1, E]
and result when BIND_PARAMETERS = false
SELECT t0.* FROM S_JOBS_ORG_UNIT_CFG t0
WHERE ((((t0.ORG_ID = 472100) AND (t0.SCHEDULER_NEXT_ACTIVATION < SYSDATE)) AND (t0.ACTIVE = Y))
AND NOT EXISTS (SELECT 1 FROM S_JOBS t1 WHERE (((t1.ORDER_ID = t0.ORDER_ID) AND (t1.ORG_ID = t0.ORG_ID)) AND NOT ((t1.STATUS = E)))) )
Code:
Query jobOrgUnitCfgQuery = entityManager.createQuery(
"SELECT c FROM JobOrgUnitCfg c WHERE c.orgId = :orgId and c.schedulerNextActivation < current_timestamp and c.active = :active and " +
" not exists (SELECT j FROM Job j WHERE j.orderId = c.orderId and j.orgId = c.orgId and j.status <> 'E')");
jobOrgUnitCfgQuery.setParameter("orgId", orgId);
jobOrgUnitCfgQuery.setParameter("active", Boolean.TRUE);
return jobOrgUnitCfgQuery.getResultList();
I think your best bet is just to programmatically build your query like you're doing with a hard coded status and escape the other paramaters manually to avoid SQL Injection.