Related
When I use the RegexTokenizer from pyspark.ml.feature to tokenize sentences column in my dataframe to find all the word characters, I get the opposite of what I would get when the python re package is used for the same sentence. Here is the sample code:
from pyspark.sql import SparkSession
from pyspark.ml.feature import RegexTokenizer
spark = SparkSession.builder \
.master("local") \
.appName("Word list") \
.getOrCreate()
df = spark.createDataFrame(data = [["Hi there, I have a question about RegexTokenizer, Could you
please help me..."]], schema = ["Sentence"])
regexTokenizer = RegexTokenizer(inputCol="Sentence", outputCol="letters", pattern="\\w")
df = regexTokenizer.transform(df)
df.first()['letters']
This gives the following output:
[' ', ', ', ' ', ' ', ' ', ' ', ' ', ', ', ' ', ' ', ' ', ' ', '...']
On the other hand if I use the re module on the same sentence and use the same pattern to match the letters, using this code here:
import re
sentence = "Hi there, I have a question about RegexTokenizer, could you
please help me..."
letters_list = re.findall("\\w", sentence)
print(letters_list)
I get the desired output as per the regular expression pattern as:
['H', 'i', 't', 'h', 'e', 'r', 'e', 'I', 'h', 'a', 'v', 'e', 'a',
'q', 'u', 'e', 's', 't', 'i', 'o', 'n', 'a', 'b', 'o', 'u', 't',
'R', 'e', 'g', 'e', 'x', 'T', 'o', 'k', 'e', 'n', 'i', 'z', 'e',
'r', 'c', 'o', 'u', 'l', 'd', 'y', 'o', 'u', 'p', 'l', 'e', 'a',
's', 'e', 'h', 'e', 'l', 'p', 'm', 'e']
I also found that I need to use \W instead of \w in pySpark to solve this problem. Why is this difference? Or have I misunderstood the usage of pattern argument in RegexTokenizer?
From what the documentation on RegexTokenizer says, on creation it has a parameter called gaps. In one mode, the regexp matches gaps (true and is the default), in other it matches tokens (not the gaps, false).
Try setting it manually to the value you need: in your case, gaps = false.
Here is my code:
#Alphabet class
class Alphabet(object):
def __init__(self, s):
self.s = s
def __str__(self):
return "Before: " + str(self.s)
#Define your Vowels class here
class Vowels:
def __init__(self,vowelList):
self.vowelList = vowelList
def __str__(self):
return "Invoking the method in Vowels by passing the Alphabet object\nAfter: " + str(vowelList)
def addVowels(self,a_obj):
for letter in a_obj:
if letter in 'aeiou':
vowelList.append(letter)
l = ','.join(vowelList)
a1 = Alphabet('A,B,C,E,I')
print a1
b = Vowels(a1)
b.addVowels(a1)
print (a2)
Right now, all it is printing is "Before: A,B,C,E,I", but I am trying to take a string of letters separated by commas (i.e. a_obj), extract the vowels from the string, then append the result to a list. I have looked at other answers regarding finding and printing only the vowels, which is why I have the for loop and if statement in addVowels, but no luck. Just to note,Vowels is supposed to be a container class for Alphabet.
When trying to get the output...the below code gives me
a1 = Alphabet('A,B,C,E,I')
print a1
a2 = Vowels(a1)
print a2
ouput:
Before: A,B,C,E,I
Invoking the method in Vowels by passing the Alphabet object
After: []
it seems like it isn't passing the letters from Alphabet...
You can create the list and get rid of the commas in one line by using split.
>>> "a,b,c,d,e,f,g,h,i,j".split(",")
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
From there you can remove the consonants by only keeping the vowels.
You can use a for loop:
letterList = ['a', 'b', 'c', 'd']
vowelList = []
for letter in letterList:
if letter in 'aeiou':
vowelList.append(letter)
Or you can use list comprehension:
letterList = ['a', 'b', 'c', 'd']
vowelList = [letter for letter in letterList if letter in 'aeiou']
Example of how this would work for your code:
class Vowels(object):
def __init__(self, vowelList):
self.vowelList = vowelList
lettersList = self.vowelList.s.split(",")
self.vowelList = [letter for letter in self.lettersList if letter in 'aeiou']
I'm using this code, and it works for me.
def getVowels(text):
vowel_letters = []
vowel_list = ['a', 'e', 'i', 'o', 'u', 'A', 'E', 'I', 'O', 'U',]
for vowels in text:
if vowels in vowel_list:
vowel_letters.append(vowels)
return vowel_letters
print(getVowels('Hi, How are you today!'))
## Output: ['i', 'o', 'a', 'e', 'o', 'u', 'o', 'a']
let = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']
Corresponds letters with certain values
These values are place holders
letMap = {'a' : '100', 'b' : '172', 'c' : '936', 'd' : '238', 'e' : '235','f' : '576', 'g' : '111', 'h' : '811', 'i' : '892', 'j' : '401','k' : '777' , 'l' : '209', 'm' : '123', 'n' : '901', 'o' : '378', 'p' : '189', 'q' : '045', 'r' : '888', 's': '700','t' : '381', 'u' : '109', 'v' : '837', 'w' : '378', 'x' : '192', 'y' : '456', 'z' : '521'}
for y in range(0,25):
ran = random.randint(1,9999999999999999999999999999999999999999999999999999)
letMap[let[myLet]] = ran
# This will not convert the integers into strings
str(letMap[let[myLet]])
myLet += 1
time.ctime()
time.sleep( .04 )
time.ctime()
in my little example (see below) it worked fine. Be aware that you should take integers as value in letMap and you did not assigned 'myLet' in your loop. Probably it should by the running index. So just replace 'myLet' to 'y' and kill the line myLet += 1.
Here is little my example:
let=['a','b','c']
letMap={'a':100,'b':123,'c':5}
letMap[let[0]]
str(letMap[let[1]])
prints:
100 (int)
'123' (str)
Hope this helps.
Cheers,
Viktor
I am trying to get a single match for the first consonant or consonant cluster in an input. Then the program should move the consonant to the beginning of the word and add "ay" at the end.
Here is my code
import re
consonants = [ 'bl', 'cl', 'fl', 'gl', 'pl', 'sl', 'br', 'cr', 'dr', 'fr', 'gr','pr', 'tr', 'sc', 'sk', 'sm', 'sn', 'sp', 'st', 'sw', 'tw','b', 'c', 'd', 'f', 'g', 'h', 'j', 'k', 'l', 'm', 'n', 'p', 'q', 'r', 's', 't', 'v', 'w', 'x', 'y', 'z']
word1 = str(input("Type something"))
word2 = re.split(r'[b-df-hj-np-tv-z]' or '[bl]''[cl]''[fl]', word1)
if any(consonants in word2 for consonants in consonants):
print(word2[1] + word2[0] + word2[2] + "ay")
The output does not appear in the interactive console.
Right, Python does not do "magic"; or is a well-defined operator which takes two boolean expressions and produces a boolean expression, not something which magically combines two regular expression strings into a new regular expression string. (You have to remember that you're talking to a computer, and computers are very stupid!)
To do the pig latin game you'll probably want to just gather a substring of non-vowels and then check whether it's 0-length (starts with a vowel) or not.
Just solved the program.
import re
words1 = input("Input Sentence:")
b1 = re.search(r"([^aeoiu]*)([aeoiu]*)([^aeoiu]*)([aeoiu]*)([^aeoiu]*)", words1)
b2 = b1.group(1)
b3 = b1.group(2)
b4 = b1.group(3)
b5 = b1.group(4)
b6 = b1.group(5)
if b5 != 5:
print(b3 + b4 + b5 + b6 + b2 + "ay")
got a question about this error:
TypeError: rot13() takes exactly 1 argument (2 given)
Which occurs on this code:
def get(self): <-- called on every get request
ch = self.rot13("abc")
def rot13(input): <-- fairly untested rot 13 ;)
alpha = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l',
'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']
escaped = escape(input)
ciphered = ""
for char in escaped:
originalIndex = alpha.index(char)
newIndex = (originalIndex + 13) % 26
ciphered = chipered + alpha[newIndex]
Do not know why there is this error. I'm just handing one parameter there.
It seems that you're missing this:
def rot13(self, input):
… That's because rot13() appears to be a method inside a class, not a stand-alone function, so it needs to receive self.