optimising multiple if statements in pyspark - if-statement

I am trying to convert if statements written in sas to pyspark;
Below is the code:
Query here is they directly mention column name and perform if function.
However if we use pyspark, we would use it df[colname].
There are lot of if statements; is their any way to cover this in a single syntax/statement.
if S1= "" and S2= "" then
do
S1= S0- S3;
S2= 0;
end;
if CS1= "" and CS2= "" then
do
CS1= CS0 - CS3;
CS2= 0;
end;
if S1< 0 then
S1= 0;
if S2< 0 then
S2= 0;
if CS1< 0 then
CS1= 0;
if CS2< 0 then
CS2= 0;
if IMP1= "" then
IMP1= 0;
if IMP2= "" then
IMP2= 0;
if CIMP1 = "" then
CIMP1 = 0;
if CIMP2 = "" then
CIMP2 = 0;

I didn't understand your question quite right. From what I understand, we can do the same thing in pyspark also ("they directly mention column name and perform if function"). Here's your sample code converted. It is generated by an automated tool called SPROCKET and millions of such 'if' conditions can be converted under a minute with it.
.withColumn('S1', expr("""case
when (S1= '' and S2= '') then S0- S3
else S1 end"""))
.withColumn('S2', expr("""case
when (S1= '' and S2= '') then 0
else S2 end"""))
.withColumn('CS1', expr("""case
when (CS1= '' and CS2= '') then CS0 - CS3
else CS1 end"""))
.withColumn('CS2', expr("""case
when (CS1= '' and CS2= '') then 0
else CS2 end"""))
.withColumn('S1', expr("""case
when (S1< 0) then 0
else S1 end"""))
.withColumn('S2', expr("""case
when (S2< 0) then 0
else S2 end"""))
.withColumn('CS1', expr("""case
when (CS1< 0) then 0
else CS1 end"""))
.withColumn('CS2', expr("""case
when (CS2< 0) then 0
else CS2 end"""))
.withColumn('IMP1', expr("""case
when (IMP1= '') then 0
else IMP1 end"""))
.withColumn('IMP2', expr("""case
when (IMP2= '') then 0
else IMP2 end"""))
.withColumn('CIMP1', expr("""case
when (CIMP1 = '') then 0
else CIMP1 end"""))
.withColumn('CIMP2', expr("""case
when (CIMP2 = '') then 0
else CIMP2 end"""))

Related

Check Valid Time in Userform

I am trying to error check in a userform where I want to be able to check that the time that was entered is a valid time expression (00:00 or 00 PM) as I am sending it through a TimeValue command for the output so that the time is standardized. If the value entered is not a valid expression I will have an error message display and allow to reenter a valid expression. I am not sure if this is something I can accomplish with Regex or if there is a simpler option. I have attached my code below.
Private Sub CommandButton_OK_Click()
Dim emptyrow As Long
'Error Check
If NameText.Value = "" Then
MsgBox "Please Enter Valid Name", vbOKOnly
Exit Sub
End If
'Make Sheet 1 Active
Sheet1.Activate
'Determine emptyRow
emptyrow = WorksheetFunction.CountA(Range("E:E")) + 1
'Convert Time of Inspection to Time Value
TimeText.Value = TimeValue(TimeText.Value)
'Transfer Info
Cells(emptyrow, 5).Value = DateText.Value
Cells(emptyrow, 6).Value = NameText.Value
Cells(emptyrow, 7).Value = ShiftText.Value
Cells(emptyrow, 8).Value = TimeText.Value
If CornNo.Value = True Then
Cells(emptyrow, 9).Value = "No"
Else
Cells(emptyrow, 9).Value = "Yes"
End If
If SurgeNo.Value = True Then
Cells(emptyrow, 10).Value = "No"
Else
Cells(emptyrow, 10).Value = "Yes"
End If
If MillNo.Value = True Then
Cells(emptyrow, 11).Value = "No"
Else
Cells(emptyrow, 11).Value = "Yes"
End If
If FBedNo.Value = True Then
Cells(emptyrow, 12).Value = "No"
Else
Cells(emptyrow, 12).Value = "Yes"
End If
If DDGOutNo.Value = True Then
Cells(emptyrow, 13).Value = "No"
Else
Cells(emptyrow, 13).Value = "Yes"
End If
If DDGInNo.Value = True Then
Cells(emptyrow, 14).Value = "No"
Else
Cells(emptyrow, 14).Value = "Yes"
End If
Unload Me
End Sub
It looks like you may want to user error handling with the On Error GoTo Label statement, I think that this will get what you want done
Private Sub CommandButton_OK_Click()
If nameText.Value = "" Then GoTo InvalidName
'' Do some stuff
On Error GoTo InvalidTime
Let TimeText.Value = TimeValue(TimeText.Value)
On Error GoTo 0
'' Do some more stuff
Let Cells(emptyrow, 9).Value = IIf(CornNo.Value = True, "No", "Yes")
'' Do even more stuff
Exit Sub
InvalidName:
MsgBox "Please Enter a Name", vbInformation + vbOKOnly, "Error"
Exit Sub
InvalidTime:
MsgBox "Please Enter a Valid Time", vbInformation + vbOKOnly, "Error"
Exit Sub
End Sub

Extract text which begin with the same two characters and end in a numeric character

I would like to extract a code from a larger extract of text, the constants I have is the Code will either start with WP or MD and end in a Numeric value and example of the patterns the code can be in are below;
WP0053
WP053
WP_053
WP_0053
WP 053
WP 0053
MDC_308
WP6
WP6.1
MDC_0308
Please see image of expected output below;
Any help would be much appreciated
Public Function GetCode(data As String) As String
startpos = InStr(data, "WP")
If startpos = 0 Then startpos = InStr(data, "MD")
fisrtNumPos = 0
For i = startpos To Len(data)
If fisrtNumPos = 0 And LastNumPos = 0 Then
If IsNumeric(Mid(data, i, 1)) Then
fisrtNumPos = i
End If
Else
If Not IsNumeric(Mid(data, i, 1)) Then
LastNumPos = i
Exit For
End If
End If
Next i
Endpos = LastNumPos - startpos
GetCode = Mid(data, startpos, Endpos)
End Function
Add this Code in any Module and try.
If your code doesn't have space then the below code will help.
Sub Test()
Data = "A850085_MDC-WP-01003_SRI Phase 2 - Programme Manager - Dionysios Psachoulias"
startpos = InStr(Data, "WP")
If startpos = 0 Then startpos = InStr(Data, "MD")
fisrtNumPos = 0
LastNumPos = 0
For i = startpos To Len(Data)
If fisrtNumPos = 0 And LastNumPos = 0 Then
If IsNumeric(Mid(Data, i, 1)) Then
fisrtNumPos = i
End If
Else
If Not IsNumeric(Mid(Data, i, 1)) Then
LastNumPos = i
Exit For
End If
End If
Next i
Endpos = LastNumPos - startpos
Debug.Print Mid(Data, startpos, Endpos)
End Sub
This should work now. But if the text contains "MD" followed by "WP" then it will take the code from WP only.
for example:
data= "A850085_WPC-MD-01003_SRI Phase 2 - Programme Manager - Dionysios Psachoulias"
then the result will be
result= "WPC-MD-01003"
Try something along the lines of this:-
Dim cell
Dim tmp as string
For each cell in activesheet.columns(1).usedrange.cells
If InStr(1, cell.Value, "_MDC_", vbTextCompare) > 0 Then
tmp = Right(cell.Value, Len(cell.Value) - InStr(1, cell.Value, "_MDC_", vbTextCompare))
tmp = Left(tmp, InStr(1, tmp, " ", vbTextCompare) - 1)
cell.offset(0,2).value = tmp
End If
next cell

Lua: Coloring "Win" status fails

I've had this function for almost two years now, and I can't seem to figure out why it's not working for colorizing. Here's the entire function, but you'll see the core parts that aren't working below.
function showscoreboard()
local function len(arg)
return string.len(arg)
end
local function tbuff(arg)
if len(arg) < 3 then
return arg.." "
else
return arg
end
end
local function sbuff(arg)
if len(arg) < 2 then
return " "..arg
else
return arg
end
end
local function cteam(t,s)
local status = s or nil
local forecolor = ""
if status == "p" then
forecolor = "yellow"
elseif status == "w" then
forecolor = "cyan"
else
forecolor = "limegreen"
end
return "<color fore="..forecolor..">"..t.."</color>"
end
local function bcolor(i)
local i = i or 0
if i%2 == 1 then
return "maroon"
else
return "navy"
end
end
local scorestring = ""
local allteams = {["ATL"]=0,["WAS"]=0,["MIA"]=0,["CLE"]=0,["OAK"]=0,["SD"]=0,["IND"]=0,["NYJ"]=0,["TEN"]=0,["SEA"]=0,["PHI"]=0,["DEN"]=0,["GB"]=0,["BUF"]=0,["TB"]=0,["PIT"]=0,["MIN"]=0,["HOU"]=0,["DET"]=0,["TB"]=0,["CAR"]=0,["CHI"]=0,["STL"]=0,["NYG"]=0,["ARI"]=0,["NO"]=0,["KC"]=0,["SF"]=0,["NE"]=0}
local byeweek = ""
for _,v in ipairs(nflscores.ss) do
allteams[v[5]] = 1
allteams[v[7]] = 1
end
for i,v in pairs(allteams) do
if v == 0 then
byeweek = byeweek .. "<color white>".. i .."</color>\r"
end
end
for i,v in ipairs(nflscores.ss) do
local hteam = v[7]
local ateam = v[5]
local qgame = v[3]
local hscre = v[8] or 0
local ascre = v[6] or 0
if v[4] then
qtime = "<color white>Time: "..v[4].."</color>"
else
qtime = ""
end
local gposs = v[9] or ""
if gposs ~= "" then
if gposs == hteam then
hteam = cteam(tbuff(hteam),"p")
ateam = cteam(tbuff(ateam))
else
ateam = cteam(tbuff(ateam),"p")
hteam = cteam(tbuff(hteam))
end
else
hteam = cteam(tbuff(hteam))
ateam = cteam(tbuff(ateam))
end
if qgame == "Final" or qgame == "final overtime" then
if hscre > ascre then
hteam = cteam(tbuff(hteam),"w")
ateam = cteam(tbuff(ateam))
elseif hscre < ascre then
ateam = cteam(tbuff(ateam),"w")
hteam = cteam(tbuff(hteam))
else
ateam = cteam(tbuff(ateam))
hteam = cteam(tbuff(hteam))
end
if qgame == "Final" then
qgame = "<color cyan>F</color>"
elseif qgame == "final overtime" then
qgame = "<color cyan>F/OT</color>"
end
elseif qgame == "Pregame" then
qgame = "<color cyan>Pre</color>"
elseif qgame == "Halftime" then
qgame = "<color white>"..qgame.."</color>"
else
qgame = "<color white>Q"..qgame.."</color>"
end
scorestring = scorestring .. "<color back="..bcolor(i) .. ">".. v[1] .. ": " .. ateam .. "<color white>: " .. sbuff(ascre) .. "</color> <color black>#</color> " .. hteam .. "<color white>: ".. sbuff(hscre) .."</color></color> " .. qgame .. " " .. qtime .. "\r"
end
return scorestring .. "<color white>Bye week:</color>\r"..byeweek
end
The part that isn't working properly is:
if hscre > ascre then
hteam = cteam(tbuff(hteam),"w")
ateam = cteam(tbuff(ateam))
elseif hscre < ascre then
ateam = cteam(tbuff(ateam),"w")
hteam = cteam(tbuff(hteam))
else
ateam = cteam(tbuff(ateam))
hteam = cteam(tbuff(hteam))
end
The function for cteam is:
local function cteam(t,s)
local status = s or nil
local forecolor = ""
if status == "p" then
forecolor = "yellow"
elseif status == "w" then
forecolor = "cyan"
else
forecolor = "limegreen"
end
return "<color fore="..forecolor..">"..t.."</color>"
end
Now, it colors the "p" status just fine. But when the status changes to "w", it fails, and for the life of me, I cannot figure out why. Am I missing something? Could this code be a lot cleaner?
Edit: I haven't found the issue to the problem, but apparently the "elseif status == "w" statement is completely bypassed. When the games are being played, the correct team in possession shows yellow. However, after the game is over, both teams are lime green, as if no score was higher than the other.
2nd Edit: The error listed in the first answer has been corrected. Still, it doesn't solve the issue. I'm still quite at a loss.
Nothing jumps out and I can't test here but here are some things to check:
You mention that cteam works correctly during the game, and that it's only once the game is over that cteam doesn't give the correct final result. So the logic of cteam is correct. The problem must be in the code that calls cteam: does cteam ever get called with s equals "w": this would never happen if hscre and ascre are always the same. Also there is a typo in the branch code that calls cteam:
if hscre > ascre then
hteam = cteam(tbuff(hteam),"w")
ateam = cteam(tbuff(ateam))
elseif hscre < ascre then
ateam = cteam(tbuff(ateam,"w")) -- ERR
hteam = cteam(tbuff(hteam))
else
ateam = cteam(tbuff(ateam))
hteam = cteam(tbuff(hteam))
end
The line that is tagged ERR should be:
ateam = cteam(tbuff(ateam),"w")
About cleaning up the code: post your question on StackOverflow's code review forum.

What regexp to use for removing addresses from emailaddresses using GAS

I can't get the regExp syntax into my head, so if would like to ask a simple question here.
I have emailaddresses
name1 < somename1#domain1.com>, name2 < somename2#domain2.com>
I would like to keep name1, name2 but remove the rest
What expression to use?
Basically I want to use a RegExp for
while (to != '')
{
var indexOne = to.indexOf('<');
var indexTwo = to.indexOf('>');
if ((indexOne > 0) && (indexTwo > indexOne))
{
to = to.substr(0, indexOne - 1) + to.substr(indexTwo, to.length - indexTwo - 1);
}
else break;
}
How about this
var s = "name1 < somename1#domain1.com>, name2 < somename2#domain2.com>, name3 < somename3#domain3.com>";
var p = /\s?<(.*?)>/g
console.log(s.replace(p, '')); // name1, name2, name3
Working jsBin

Validation for Form and QueryString in ASP Classic using Regex. Almost working but missing something?

I'm trying to add some Input Validation in Classic ASP by using the function/code seen below.
The only one that looks like it's working correctly is the "text" type. the others I keep getting errors or it just does not filter correctly.
I'm trying to understand what I'm doing wrong please help me.
Valid Data Types: "email", "integer", "date", "string" and "text".
The first three are obvious, the last two have slight differences.
The "email" should only allow numbers and leters, and the following characters "#" , "-" , "." , "_"
The "date" should validate by running IsDate and if True then allow if False DON'T.
The "string" should validate text-based querystrings, allowing only letters, numbers, _, - and .
Whereas "text" is any free-form text form field type content.
The "integer" should only allow numbers and a period (.)
Usage Example: <input type="text" value="<%=MakeSafe("test#test.com</HTML>1234.5",integer,50)%>">
Eg: MakeSafe(dataInput,dataType,dataLength)
<%
'// CODE BY: dB Masters
'// FOUND AT: http://successontheweb.blogspot.com/2008/03/input-validation-for-security-in.html
Function MakeSafeConvert(encodeData)
encodeData = replace(encodeData,"&", "&")
encodeData = replace(encodeData,"'", "'")
encodeData = replace(encodeData,"""", """)
encodeData = replace(encodeData,">", ">")
encodeData = replace(encodeData,"<", "<")
encodeData = replace(encodeData,")", ")")
encodeData = replace(encodeData,"(", "(")
encodeData = replace(encodeData,"]", "]")
encodeData = replace(encodeData,"[", "[")
encodeData = replace(encodeData,"}", "}")
encodeData = replace(encodeData,"{", "{")
encodeData = replace(encodeData,"--", "--")
encodeData = replace(encodeData,"=", "=")
MakeSafeConvert = encodeData
End Function
Function MakeSafe(dataInput,dataType,dataLength)
Dim regex, validInput, expressionmatch
regex = ""
validInput = "1"
If dataType = "string" And Len(dataInput) > 0 Then
regex = "^[\w-\.]{1,"& dataLength &"}$"
ElseIf dataType = "email" And Len(dataInput) > 0 Then
regex = "^[\w-\.]+#([\w-]+\.)+[\w-]{2,6}$"
ElseIf dataType = "integer" And Len(dataInput) > 0 Then
regex = "^\d{1,"& dataLength &"}$"
ElseIf dataType = "date" And Len(dataInput) > 0 Then
If Not IsDate(dataInput) Then validInput = "0" End If
ElseIf dataType = "text" And Len(dataInput) > 0 Then
If Len(dataInput) > dataLength Then validInput = "0" End If
End If
If Len(regex) > 0 And Len(dataInput) > 0 Then
Set RegExpObj = New RegExp
RegExpObj.Pattern = regex
RegExpObj.IgnoreCase = True
RegExpObj.Global = True
RegExpChk = RegExpObj.Test(dataInput)
If Not RegExpChk Then
validInput = "0"
End If
Set RegExpObj = nothing
End If
If validInput = "1" And Len(dataInput) > 0 Then
MakeSafe = MakeSafeConvert(dataInput)
ElseIf Len(dataInput) = 0 Then
MakeSafe = ""
Else
Response.Write "<h2>Processing Halted.</h2>"
Response.End
End If
End Function
%>
EXAMPLE CODE AND ERROR(S):
When I test this using the code:
<%=MakeSafe("test#test.com1234.5",email,50)%>
* Does NOT Validate Anything.*
I don't get an error message but it DOES NOT Validate anything.
**The OUTPUT IS : test#test.com1/27/20121234.5
SHOULD BE ONLY: test#test.com**
When I test this using the code:
<%=MakeSafe("test#test.com1/27/20121234.5",date,50)%>
I don't get an error message but it DOES NOT Validate anything.
The OUTPUT IS : test#test.com1/27/20121234.5
SHOULD BE ONLY: 1/27/2012
The other two give me this error message:
<%=MakeSafe("test#test.com1234.5",string,50)%>
* ERROR!!! Wrong number of arguments or invalid property assignment: 'string'
<%=MakeSafe("test#test.com1234.5",integer,50)%>
* ERROR!!! Syntax error
Thank you so much for any help that you provide...
If it's not a typo then your fault was in the second parameter of the function call.
You call the function like:
<%=MakeSafe("test#test.com1234.5",email,50)%>
which is wrong because you should "..." the second parameter too. This should work:
<%=MakeSafe("test#test.com1234.5","email",50)%>