Is it possible to convert/parse SqlBackendKey to Real Text?
e.g Model
UserInfo
fullName Text
ageBracket Text Maybe
gender Text
createdOn UTCTime default=now()
updatedOn UTCTime Maybe
deriving Show
User
userInfoId UserInfoId
username Text
password Text
UniqueUsername username
deriving Show
on my code:
-- let say the output of this
-- userInfoId = 1 -- this is a
-- username = admin
-- paswword = admin
userData <- runDB $ selectFirst [UserUsername ==. uname , UsersPassword ==. pword] [LimitTo 1]
case userData of
Nothing -> ....
Just (Entity _ u) -> do
let userInfoId = T.pack (show (userUserInfoId u))
$(logInfo) userInfoId
.....
the output of the $(logInfo) userInfoId
[INFO] UserInfoKey {unUserInfoKey = SqlBackendKey {unSqlBackendKey = 1}}
I only need the number "1" (One).
You can use fromSqlKey.
fromSqlKey (userUserInfoId u)
Related
I have my model from models.persistentmodels
...
Thing
title Text
price Int
kosher Bool
optionalstuff [Text] Maybe
createdat UTCTime
updatedat UTCTime
deriving Show
...
It contains two time fields, which are UTCTime.
I am receiving via AJAX what is almost a Thing, in JSON. But the user JSON should not have createdat and updatedat or kosher. So we need to fill them in.
postNewEventR = do
inputjson <- requireCheckJsonBody :: Handler Value
...
-- get rawstringofthings from inputjson
...
let objectsMissingSomeFields = case (decode (BL.fromStrict $ TE.encodeUtf8 rawstringofthings) :: Maybe [Object]) of
Nothing -> error "Failed to get a list of raw objects."
Just x -> x
now <- liftIO getCurrentTime
-- Solution needs to go here:
let objectsWithAllFields = objectsMissingSomeFields
-- We hope to be done
let things = case (eitherDecode $ encode objectsWithAllFields) :: Either String [Thing] of
Left err -> error $ "Failed to get things because: " <> err
Right xs -> xs
The error "Failed to get things" comes here because the JSON objects we parsed are missing fields that are needed in the model.
Solution
let objectsWithAllFields = Import.map (tackOnNeccessaryThingFields now True) objectsMissingSomeFields
So we take the current object and tack on the missing fields e.g. kosher and createdat.
But there is some strange difference in the way UTCTime is read vs aeson's way to parse UTCTime. So when I print UTCTime in to a Aeson String, I needed to print out the UTCTime into the format that it is expecting later:
tackOnNeccessaryThingFields :: UTCTime -> Bool -> Object -> Object
tackOnNeccessaryThingFields t b hm = G.fromList $ (G.toList hm) <> [
("createdat", String (pack $ formatTime defaultTimeLocale "%FT%T%QZ" t)),
("updatedat", String (pack $ formatTime defaultTimeLocale "%FT%T%QZ" t)),
("kosher", Bool b)
]
tackOnNeccessaryThingFields _ _ _ = error "This isn't an object."
After this fix, the object has all the fields needed to make the record, so the code gives [Thing].
And also the code runs without runtime error, instead of failing to parse the tshow t as UTCTime.
Note:
This aeson github issue about this problem seems to be closed but it seems to be not any more permissive: https://github.com/bos/aeson/issues/197
Thanks to Artyom:
https://artyom.me/aeson#records-and-json-generics
Thanks to Pbrisbin:
https://pbrisbin.com/posts/writing_json_apis_with_yesod/
Thanks to Snoyman:
For everything
I have text data inside a caloumn of dataset as shown below
Record Note 1
1 Amount: $43,385.23
Mode: Air
LSP: Panalpina
2 Amount: $1,149.32
Mode: Ocean
LSP: BDP
3 Amount: $1,149.32
LSP: BDP
Mode: Road
4 Amount: U$ 3,234.01
Mode: Air
5 No details
I need to extract each of the details inside the text data and write them into new column as shown below how to do it in python
Expected Output
Record Amount Mode LSP
1 $43,385.23 Air Panalpina
2 $1,149.32 Ocean BDP
3 $1,149.32 Road BDP
4 $3,234.01 Air
5
Is this possible. how can this be do
Write a custom function and then use pd.apply() -
def parse_rec(x):
note = x['Note']
details = note.split('\n')
x['Amount'] = None
x['Mode'] = None
x['LSP'] = None
if len(details) > 1:
for detail in details:
if 'Amount' in detail:
x['Amount'] = detail.split(':')[1].strip()
if 'Mode' in detail:
x['Mode'] = detail.split(':')[1].strip()
if 'LSP' in detail:
x['LSP'] = detail.split(':')[1].strip()
return x
df = df.apply(parse_rec, axis=1)
import re
Amount = []
Mode = []
LSP = []
def extract_info(txt):
Amount_lst = re.findall(r"amounts?\s*:\s*(.*)", txt, re.I)
Mode_lst = re.findall(r"Modes??\s*:\s*(.*)", txt, re.I)
LSP_lst = re.findall(r"LSP\s*:\s*(.*)", txt, re.I)
Amount.append(Amount_lst[0].strip() if Amount_lst else "No details")
Mode.append(Mode_lst[0].strip() if Mode_lst else "No details")
LSP.append(LSP_lst[0].strip() if LSP_lst else "No details")
df["Note"].apply(lambda x : extract_info(x))
df["Amount"] = Amount_lst
df["Mode"]= Mode_lst
df["LSP"]= LSP_lst
df = df[["Record","Amount","Mode","LSP"]]
By using regex we can extract information such as the above code and write down to separate columns.
fake = {'EmployeeID' : [0,1,2,3,4,5,6,7,8,9],
'State' : ['a','b','c','d','e','f','g','h','i','j'],
'Email' : ['a','b','c','d','e','f','g','h','i','j']
}
fake_df = pd.DataFrame(fake)
I am trying to define a function that returns a Series of strings of all email addresses of employees in states. The email addresses should be separated by a given delimiter. I think I will use ";".
Arguments:
- DataFrame
- delimiter (;)
Do I have to use for loop?? to be honest, I don't even know how to start on this..
====EDITION
After done with coding, I should run
emails = getEmailListByState(fake_df, ", ")
for state in sorted(emails.index):
print "%15s: %s" % (state, emails[state])
and should get something like
a: a
b: b
c: c,d
d: e
e: f,g
as my output
If I understand the problem properly you are looking for groupby state,get the emails and apply join i.e joining the emails based on the state i.e
fake = {'EmployeeID' : [0,1,2,3,4,5,6,7,8,9],
'State' : ['NZ','NZ','NY','NY','ST','ST','YK','YK','YK','YK'],
'Email' : ['ab#h.com','bab#h.com','cab#h.com','dab#h.com','eab#h.com','fab#h.com','gab#h.com','hab#h.com','iab#h.com','jab#h.com']
}
fake_df = pd.DataFrame(fake)
ndf = fake_df.groupby('State')['Email'].apply(', '.join)
Output:
State
NY cab#h.com, dab#h.com
NZ ab#h.com, bab#h.com
ST eab#h.com, fab#h.com
YK gab#h.com, hab#h.com, iab#h.com, jab#h.com
Name: Email, dtype: object
If you want that in a method then
def getEmailListByState(df,delim):
return df.groupby('State')['Email'].apply(delim.join)
emails = getEmailListByState(fake_df, ", ")
for state in sorted(emails.index):
print( "%15s: %s" % (state, emails[state])
I get this as a response to an API hit.
1735 Queries
Taking 1.001303 to 31.856310 seconds to complete
SET timestamp=XXX;
SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
38 Queries
Taking 1.007646 to 5.284330 seconds to complete
SET timestamp=XXX;
show slave status;
6 Queries
Taking 1.021271 to 1.959838 seconds to complete
SET timestamp=XXX;
SHOW SLAVE STATUS;
2 Queries
Taking 4.825584, 18.947725 seconds to complete
use marketing;
SET timestamp=XXX;
SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
I have extracted this out of the response html and have it as a string now.I need to retrieve values as concisely as possible such that I get a map of values of this format Map(Query -> T1 to T2 seconds) Basically what this is the status of all the slow queries running on MySQL slave server. I am building an alert system over it . So from this entire paragraph in the form of String I need to separate out the queries and save the corresponding time range with them.
1.001303 to 31.856310 is a time range . And against the time range the corresponding query is :
SET timestamp=XXX; SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
This information I was hoping to save in a Map in scala. A Map of the form (query:String->timeRange:String)
Another example:
("use marketing; SET timestamp=XXX; SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified xyz ;"->"4.825584 to 18.947725 seconds")
"""###(.)###(.)\n\n(.*)###""".r.findAllIn(reqSlowQueryData).matchData foreach {m => println("group0"+m.group(1)+"next group"+m.group(2)+m.group(3)}
I am using the above statement to extract the the repeating cells to do my manipulations on it later. But it doesnt seem to be working;
THANKS IN ADvance! I know there are several ways to do this but all the ones striking me are inefficient and tedious. I need Scala to do the same! Maybe I can extract recursively using the subString method ?
If you want use scala try this:
val regex = """(\d+).(\d+).*(\d+).(\d+) seconds""".r // extract range
val txt = """
|1735 Queries
|
|Taking 1.001303 to 31.856310 seconds to complete
|
|SET timestamp=XXX; SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
|
|38 Queries
|
|Taking 1.007646 to 5.284330 seconds to complete
|
|SET timestamp=XXX; show slave status;
|
|6 Queries
|
|Taking 1.021271 to 1.959838 seconds to complete
|
|SET timestamp=XXX; SHOW SLAVE STATUS;
|
|2 Queries
|
|Taking 4.825584, 18.947725 seconds to complete
|
|use marketing; SET timestamp=XXX; SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
""".stripMargin
def logToMap(txt:String) = {
val (_,map) = txt.lines.foldLeft[(Option[String],Map[String,String])]((None,Map.empty)){
(acc,el) =>
val (taking,map) = acc // taking contains range
taking match {
case Some(range) if el.trim.nonEmpty => //Some contains range
(None,map + ( el -> range)) // add to map
case None =>
regex.findFirstIn(el) match { //extract range
case Some(range) => (Some(range),map)
case _ => (None,map)
}
case _ => (taking,map) // probably empty line
}
}
map
}
Modified ajozwik's answer to work for SQL commands over multiple lines :
val regex = """(\d+).(\d+).*(\d+).(\d+) seconds""".r // extract range
def logToMap(txt:String) = {
val (_,map) = txt.lines.foldLeft[(Option[String],Map[String,String])]((None,Map.empty)){
(accumulator,element) =>
val (taking,map) = accumulator
taking match {
case Some(range) if element.trim.nonEmpty=> {
if (element.contains("Queries"))
(None, map)
else
(Some(range),map+(range->(map.getOrElse(range,"")+element)))
}
case None =>
regex.findFirstIn(element) match {
case Some(range) => (Some(range),map)
case _ => (None,map)
}
case _ => (taking,map)
}
}
println(map)
map
}
I am trying to create a code that takes the user input, compares it to a list of tuples (shares.py) and then prints the values in a the list. for example if user input was aia, this code would return:
Please list portfolio: aia
Code Name Price
AIA Auckair 1.50
this works fine for one input, but what I want to do is make it work for multiple inputs.
For example if user input was aia, air, amp - this input would return:
Please list portfolio: aia, air, amp
Code Name Price
AIA Auckair 1.50
AIR AirNZ 5.60
AMP Amp 3.22
This is what I have so far. Any help would be appreciated!
import shares
a=input("Please input")
s1 = a.replace(' ' , "")
print ('Please list portfolio: ' + a)
print (" ")
n=["Code", "Name", "Price"]
print ('{0: <6}'.format(n[0]) + '{0:<20}'.format(n[1]) + '{0:>8}'.format(n[2]))
z = shares.EXCHANGE_DATA[0:][0]
b=s1.upper()
c=b.split()
f=shares.EXCHANGE_DATA
def find(f, a):
return [s for s in f if a.upper() in s]
x= (find(f, str(a)))
print ('{0: <6}'.format(x[0][0]) + '{0:<20}'.format(x[0][1]) + ("{0:>8.2f}".format(x[0][2])))
shares.py contains this
EXCHANGE_DATA = [('AIA', 'Auckair', 1.5),
('AIR', 'Airnz', 5.60),
('AMP', 'Amp',3.22),
('ANZ', 'Anzbankgrp', 26.25),
('ARG', 'Argosy', 12.22),
('CEN', 'Contact', 11.22)]
I am assuming a to contain values in the following format 'aia air amp'
raw = a # just in case you want the original string at a later point
toDisplay = []
a = a.split() # a now looks like ['aia','air','amp']
for i in a:
temp = find(f, i)
if(temp):
toDisplay.append(temp)
for i in toDisplay:
print ('{0: <6}'.format(i[0][0]) + '{0:<20}'.format(i[0][1]) + ("{0:>8.2f}".format(i[0][2])))
Essentially what I'm trying to do is
Split the input into a list
Do exactly what you were doing for a single input for each item in that list
Hope this helps!