I want to configure a config.toml file in meta-toolkit with default filter chain but specifying each of the filters. I plan to make modifications to some of the filters so I want to have a baseline filter chain.
I have tried the following attributes:
[[analyzers]]
method = "ngram-word"
ngram = 1
[[analyzers.filter]]
type = "icu-tokenizer"
[[analyzers.filter]]
type = "lowercase"
[[analyzers.filter]]
type = "alpha"
[[analyzers.filter]]
type = "length"
min = 2
max = 35
[[analyzers.filter]]
type = "list"
filename = "../data/lemur-stopwords.txt"
[[analyzers.filter]]
type = "porter2-stemmer"
[[analyzers.filter]]
type = "empty-sentence"
I'm getting:
token_stream_exception: what(): file required for list_filter config
The attribute should be file instead of filename.
[[analyzers.filter]]
type = "list"
file = "../data/lemur-stopwords.txt"
Related
I am getting an error "unhashable dict" for tmp_str = [{category : name }]. category and name are variables.
Tried options are
tmp_str = [{category : name }]
tmp_str = {category : name }
Complete code is here
def isAttributesIntheName(file_name,message):
table = db_client.Table(table_name)
count = table.item_count
json_msg = json.dumps(message)
print (json_msg)
numCount = 0
loop = 0
print (count)
for numCount in range (count):
name = table.scan()['Items'][numCount]['name']
result = file_name.lower().find(name)
category = table.get_item(Key={'name':name})
if result > 0:
tmp_str = [{category : name }]
json_str1 = dict(**json_str,**{tmp_str })
loop = loop + 1
print (json.dumps(json_str1))
For the specific error you are asking about, the variable category (results of querying your database table) is not a hashable type - e.g. is mutable type such as a dict or list. Dict keys need to be immutable so they can be hashed form a hash key.
It looks like you are querying a DynamoDB and the datatype returned by get_item is a dict, and within that dict there is a key 'Item' that has the data you are requesting in another dict. Here are the docs for get_item:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.get_item
Here is more info on "hashable":
What does "hashable" mean in Python?
I did the code for insert records from so_parts table to so_bo table using Query...How can I use ORM method to do this kind of job. Is there any other way(best)to do that? Here is my code`
`
#api.multi
def save_rapair_parts(self, vals):
#get todays date and convert it to string
created_date = datetime.datetime.today().strftime("%m/%d/%Y")
str_date = str(created_date)
so_p_id = self.so_p_id.id
bo_status = self.bo_status
so_part_t = self.so_part_t
so_part_sno = self.so_part_sno
product = self.so_part_product
product_str = 'Repair '+str(product)
part_id = self.id
bench_order_table.search(['id','bo_sno','created_date','bo_number','rep_description','bo_status'])
#insert details intoso bench orders
`
if so_part_t=='r_b':
try:
sequence = self.env['ir.sequence'].next_by_code('so.benchorder') or '/'
str_sequence = str(sequence)
query = """SELECT so_work_authorization FROM my_depots_so WHERE id=%d """ % (so_p_id)
self.env.cr.execute(query)
result = self.env.cr.fetchall()
result_number = json.dumps(result, ensure_ascii=False)
strip_number = result_number.strip('\' \" [] ')
work_auth_no = str(strip_number)
work_auth_no += "-"
work_auth_no += str_sequence
insert ="""INSERT INTO my_depots_so_bo(id,so_bo_id,bo_sno,created_date,bo_number,rep_description,bo_status) values %s """
parameters = (part_id,so_p_id,so_part_sno,str_date,work_auth_no,product_str,bo_status)
self.env.cr.execute(insert,(parameters,))
my_depots_bo(id,bo_sno,created_date,bo_number,rep_description,bo_status) values %s """
# self.env.cr.execute(insert_query, (parameters,))
except Exception:
print "Error in inserting values"`
yes there is a better way because when you use ORM
method you also checks access right for user to:
for your select query:
rec = self.env['my.depots.so'].search_read(['id', '=', so_p_id], ['so_work_authorization'])
if rec:
rec = rec[0] # search_read return a list of dictionary
so_work_authorization = rec['so_work_authorization']
# and do what ever you want with the result
# to create
# call create method witch accept a dictionary
# field_name : value
new_rec = self.env['my.depots.so.bo'].create({
'so_bo_id': so_p_id, # many2one must be an integer value
'bo_sno': bo_nso_value,
'bo_number': value_of_number,
# ....
# ....
# add al field
}) # create return the new created record as model object
for inserting use: self.env['model.name'].create(vals)
for updating use : self.env['model.name'].write(vals)
using ORM method makes sure that user don't pass the security access rigths
Hope you get the idea
Sorry for the long title, didn't know how to ask it:
I am working with ExactTarget Salesforce Marketing API, trying to iterate over multiple dictionary objects from the API call but some of them are nested and have the same name as the other API responses and I am getting confused on how to iterate over the same named variables into a dataframe.
This is the output of the API Call:
(ClickEvent){
Client =
(ClientID){
ID = 11111111
}
PartnerKey = None
CreatedDate = 2016-07-12 00:40:17
ModifiedDate = 2016-07-12 00:40:17
ID = 11111111
ObjectID = "11111111"
SendID = 11111111
SubscriberKey = "azfull#usa.net"
EventDate = 2016-07-12 00:40:17
EventType = "Click"
TriggeredSendDefinitionObjectID = None
BatchID = 1
URLID = 11111111
URL = aaa.com
I want to create a separate dataframe column for the "ID" under "ClientID" but I am running into the trouble of another variable already being named "ID". How can I iterate over "ClientID" and get the ID value plus also get the other values and place them in the dataframe?
My code has been able to place the data in the dataframe but I am not getting the particular Client ID. this is what output looks like now:
BatchID ClientID CreatedDate \
0 1 (ClientID){\n ID = 10914162\n } 2016-02-23 13:08:59
1 1 (ClientID){\n ID = 10914162\n } 2016-02-23 13:11:49
As you can see only want the ID number not the other garbage under "ClientID"
Code:
import ET_Client
import pandas as pd
try:
debug = False
stubObj = ET_Client.ET_Client(False, debug)
## Modify the date below to reduce the number of results returned from the request
## Setting this too far in the past could result in a very large response size
retrieveDate = '2016-07-11T13:00:00.000'
#ET call for clicks
print '>>>ClickEvents'
getClickEvent = ET_Client.ET_ClickEvent()
getClickEvent.auth_stub = stubObj
getResponse = getClickEvent.get()
ResponseResults = getResponse.results
#print ResponseResults
Client = []
partner_keys = []
created_dates = []
modified_date = []
ID = []
ObjectID = []
SendID = []
SubscriberKey = []
EventDate = []
EventType = []
TriggeredSendDefinitionObjectID = []
BatchID = []
URLID = []
URL = []
for ClickEvent in ResponseResults:
Client.append(str(ClickEvent['Client']))
partner_keys.append(ClickEvent['PartnerKey'])
created_dates.append(ClickEvent['CreatedDate'])
modified_date.append(ClickEvent['ModifiedDate'])
ID.append(ClickEvent['ID'])
ObjectID.append(ClickEvent['ObjectID'])
SendID.append(ClickEvent['SendID'])
SubscriberKey.append(ClickEvent['SubscriberKey'])
EventDate.append(ClickEvent['EventDate'])
EventType.append(ClickEvent['EventType'])
TriggeredSendDefinitionObjectID.append('TriggeredSendDefinitionObjectID')
BatchID.append(ClickEvent['BatchID'])
URLID.append(ClickEvent['URLID'])
URL.append(ClickEvent['URL'])
df = pd.DataFrame({'ClientID': Client, 'PartnerKey': partner_keys,
'CreatedDate' : created_dates, 'ModifiedDate': modified_date,
'ID':ID, 'ObjectID': ObjectID,'SendID':SendID,'SubscriberKey':SubscriberKey,
'EventDate':EventDate,'EventType':EventType,'TriggeredSendDefinitionObjectID':TriggeredSendDefinitionObjectID,
'BatchID':BatchID,'URLID':URLID,'URL':URL})
print df
I have been trying this solution but not working:
for ClickEvent in ResponseResults():
if 'ClientID' in ClickEvent:
ID.append(ClickEvent['Client']:
print Client
Thank you in advance.
-EDIT-
The output of the API call above is exactly how the systems outputs it, how should I make it an actual JSON response?
Data frame I want to look like this:
BatchID ClientID CreatedDate \
0 1 111111111 2016-02-23 13:08:59
1 1 111111111 2016-02-23 13:11:49
Just dont want other stuff in the "ClientID" portion of the data I submitted above. Hope this helps.
Instead of appending the entire Client object to your list :
Client.append(str(ClickEvent['Client']))
Have you tried storing just the ID field of the object? Maybe something like:
Client.append(str(ClickEvent['Client']['ID']))
I have a models class with a select list / positive integer field.
class AchievementDetails(models.Model, FillableModelWithLanguageVersion):
ACADEMIC_ACHIEVEMENT = 1
COMMERCIAL_ACHIEVEMENT = 2
PERSONAL_ACHIEVEMENT = 3
PROFESSIONAL_ACHIEVEMENT = 4
SPORTING_ACHIEVEMENT = 5
OTHER_ACHIEVEMENT_TYPE = 6
ACHIEVEMENT_TYPES = (
(ACADEMIC_ACHIEVEMENT, _('Academic Details')),
(COMMERCIAL_ACHIEVEMENT, _('Commercial Achievement')),
(PERSONAL_ACHIEVEMENT, _('Personal Achievement')),
(PROFESSIONAL_ACHIEVEMENT, _('Professional Achievement')),
(SPORTING_ACHIEVEMENT, _('Sporting Achievement')),
(OTHER_ACHIEVEMENT_TYPE, _('Other Achievement Type')),
)
....
achievement_type = models.PositiveIntegerField(choices=ACHIEVEMENT_TYPES)
....
The above set up adds in the following select list option as the default option:
<option value="">---------</option>
I have been instructed not to include SELECT_TYPE = '' into the ACHIEVEMENT_TYPES.
I am using django-parsley for client side validation, so the option value must be a blank value.
I have two questions:
1. How do I replace the default text of '---------' with 'Select Achievement Type'
2. How do I remove the default option value when the form is in the edit template?
I got this to work using the:
validators=[MinValueValidator(1)]
This is how I used it:
Include the import in the models.py file:
from django.core.validators import MinValueValidator
Include the types with the zero value included to the models.py file:
SELECT_ACHIEVEMENT_TYPE = 0
ACADEMIC_ACHIEVEMENT = 1
COMMERCIAL_ACHIEVEMENT = 2
PERSONAL_ACHIEVEMENT = 3
PROFESSIONAL_ACHIEVEMENT = 4
SPORTING_ACHIEVEMENT = 5
OTHER_ACHIEVEMENT_TYPE = 6
WRITE_MY_OWN_ACHIEVEMENT_TYPE_DESCRIPTION = 7777 # 7777 triggers a hidden text field to be displayed.
DISPLAY_ONLY_ACHIEVEMENT_DESCRIPTION_WITH_PROMPT = 8888
DISPLAY_ONLY_ACHIEVEMENT_DESCRIPTION_WITHOUT_PROMPT = 9999
ACHIEVEMENT_TYPES = (
(SELECT_ACHIEVEMENT_TYPE, _('Select Type')),
(ACADEMIC_ACHIEVEMENT, _('Academic Achievement')),
(COMMERCIAL_ACHIEVEMENT, _('Commercial Achievement')),
(PERSONAL_ACHIEVEMENT, _('Personal Achievement')),
(PROFESSIONAL_ACHIEVEMENT, _('Professional Achievement')),
(SPORTING_ACHIEVEMENT, _('Sporting Achievement')),
(OTHER_ACHIEVEMENT_TYPE, _('Other Achievement Type')),
(WRITE_MY_OWN_ACHIEVEMENT_TYPE_DESCRIPTION, _('Write my own Type description')),
(DISPLAY_ONLY_ACHIEVEMENT_DESCRIPTION_WITH_PROMPT, _('Display only Description with prompt')),
(DISPLAY_ONLY_ACHIEVEMENT_DESCRIPTION_WITHOUT_PROMPT, _('Display only Description without prompt'))
)
Include the field in the models.py file with the choices, default and MinValueValidator:
....
achievement_type = models.PositiveIntegerField(choices=ACHIEVEMENT_TYPES, default=SELECT_ACHIEVEMENT_TYPE, validators=[MinValueValidator(1)])
....
In the forms.py file, include the error_messages meta data to override the error message (for Django 1.5+):
.....
error_messages = {
'achievement_type': {'validate_min': _('This field is required.')},
}
....
I am attempting to alter a previous script that utilizes biopython to fetch information about a species phylum. This script was written to retrieve information one species at a time. I would like to modify the script so that I can do this for 100 organisms at a time.
Here is the initial code
import sys
from Bio import Entrez
def get_tax_id(species):
"""to get data from ncbi taxomomy, we need to have the taxid. we can
get that by passing the species name to esearch, which will return
the tax id"""
species = species.replace(" ", "+").strip()
search = Entrez.esearch(term = species, db = "taxonomy", retmode = "xml")
record = Entrez.read(search)
return record['IdList'][0]
def get_tax_data(taxid):
"""once we have the taxid, we can fetch the record"""
search = Entrez.efetch(id = taxid, db = "taxonomy", retmode = "xml")
return Entrez.read(search)
Entrez.email = ""
if not Entrez.email:
print "you must add your email address"
sys.exit(2)
taxid = get_tax_id("Erodium carvifolium")
data = get_tax_data(taxid)
lineage = {d['Rank']:d['ScientificName'] for d in
data[0]['LineageEx'] if d['Rank'] in ['family', 'order']}
I have managed to modify the script so that it accepts a local file that contains one of the organisms I am using. But I need to extend this to a 100 organisms.
So the idea was to generate a list from the file of my organisms and somehow separately fed each item generated from the list into the line taxid = get_tax_id("Erodium carvifolium") and replace "Erodium carvifolium" with my organisms name. But I have no idea how to do that.
Here is the sample version of the code with some of my adjustments
import sys
from Bio import Entrez
def get_tax_id(species):
"""to get data from ncbi taxomomy, we need to have the taxid. we can
get that by passing the species name to esearch, which will return
the tax id"""
species = species.replace(' ', "+").strip()
search = Entrez.esearch(term = species, db = "taxonomy", retmode = "xml")
record = Entrez.read(search)
return record['IdList'][0]
def get_tax_data(taxid):
"""once we have the taxid, we can fetch the record"""
search = Entrez.efetch(id = taxid, db = "taxonomy", retmode = "xml")
return Entrez.read(search)
Entrez.email = ""
if not Entrez.email:
print "you must add your email address"
sys.exit(2)
list = ['Helicobacter pylori 26695', 'Thermotoga maritima MSB8', 'Deinococcus radiodurans R1', 'Treponema pallidum subsp. pallidum str. Nichols', 'Aquifex aeolicus VF5', 'Archaeoglobus fulgidus DSM 4304']
i = iter(list)
item = i.next()
for item in list:
???
taxid = get_tax_id(?)
data = get_tax_data(taxid)
lineage = {d['Rank']:d['ScientificName'] for d in
data[0]['LineageEx'] if d['Rank'] in ['phylum']}
print lineage, taxid
The question marks refer to places where I am stumped as what to do next. I don't see how I can connect my loop to replace the ? in get_tax_id(?). Or do I need to somehow append each of the items in the list so that they are modified each time to contain get_tax_id(Helicobacter pylori 26695) and then find some way to place them in the line containing taxid =
Here's what you need, place this below your function definitions, i.e. after the line that says: sys.exit(2)
species_list = ['Helicobacter pylori 26695', 'Thermotoga maritima MSB8', 'Deinococcus radiodurans R1', 'Treponema pallidum subsp. pallidum str. Nichols', 'Aquifex aeolicus VF5', 'Archaeoglobus fulgidus DSM 4304']
taxid_list = [] # Initiate the lists to store the data to be parsed in
data_list = []
lineage_list = []
print('parsing taxonomic data...') # message declaring the parser has begun
for species in species_list:
print ('\t'+species) # progress messages
taxid = get_tax_id(species) # Apply your functions
data = get_tax_data(taxid)
lineage = {d['Rank']:d['ScientificName'] for d in data[0]['LineageEx'] if d['Rank'] in ['phylum']}
taxid_list.append(taxid) # Append the data to lists already initiated
data_list.append(data)
lineage_list.append(lineage)
print('complete!')