I'm trying to plot the voltage/current coming out of a device, with these being written to a file and then displayed on the same plot, which updates in real time. Currently the code does this, but it pops up with an error message each time the graph updates which seems to be slowing the pace at which the program takes measurements.
The error seems to be occurring each time the graph updates i.e. when plt.draw() is executed within the While loop, making me think that it occurs in def animate_readings(i), though I've included all pertinent pieces of code just in case.
plt.ion()
...
fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)
ax1.clear()
ax2 = ax1.twinx()
ax2.clear()
plt.xlim(-7,3)
ax1.set_ylabel('Voltage V')
ax2.set_ylabel('Current mA')
ax1.set_ylim(-0.5,27)
ax2.set_ylim(-5, 400, 10)
Vpatch = mpatches.Patch(color='blue', label='Voltage')
Cpatch = mpatches.Patch(color='red', label='Current')
plt.legend(handles=[Vpatch, Cpatch])
plt.draw()
plt.pause(0.1)
"Is used to assign properties to the graph used in FuncAnimation"
def animate_readings(i):
plt.xlim(c-8,c+2)
ax1.plot(AVdata, color='b')
ax2.plot(ACdata, color='r')
Artists = namedtuple ('Artists', ('AVdata', 'ACdata'))
returnable = Artists(
ax1.plot(AVdata, color='b'),
ax2.plot(ACdata, color='r')
)
return returnable
...
while 1:
print 1
"Below if statement names and creates file at beginning of operation"
ani = animation.FuncAnimation(fig, animate_readings, interval=200, blit=True)
...
if c % 1 == 0:
print "2b"
try:
plt.draw()
plt.pause(0.1)
print "2c"
except :
pass
print 2
c = c + 1
The error which pops up after print "2b" executes is as follows. Nominally the loop should be running ~3x a second but, due to the error it only runs every 3 seconds and, in a recent extended test, only recorded readings every 30 seconds.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/matplotlib/cbook/__init__.py", line 388, in process
proxy(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/matplotlib/cbook/__init__.py", line 228, in __call__
return mtd(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/matplotlib/animation.py", line 1026, in _start
self._init_draw()
File "/usr/local/lib/python2.7/dist-packages/matplotlib/animation.py", line 1750, in _init_draw
self._draw_frame(next(self.new_frame_seq()))
File "/usr/local/lib/python2.7/dist-packages/matplotlib/animation.py", line 1778, in _draw_frame
a.set_animated(self._blit)
AttributeError: 'list' object has no attribute 'set_animated'
Related
I have three variables I want to write in a tab delimited .csv, appending values each time the script iterates over a key value from the dictionary.
Currently the script calls a command, regex the stdout as out then assigns the three defined regex groups to individual variables for writing to .csv labeled first second and third. I get a __exit_ error when I run the below script.
/note I've read up on csv.writer and I'm still confused as to whether I can actually write multiple variables to a row.
Thanks for any help you can provide.
import csv, re, subprocess
for k in myDict:
run_command = "".join(["./aCommand", " -r data -p ", str(k)])
process = subprocess.Popen(run_command,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = process.communicate()
errcode = process.returncode
pattern = re.compile('lastwrite|(\d{2}:\d{2}:\d{2})|alert|trust|Value')
grouping = re.compile('(?P<first>.+?)(\n)(?P<second>.+?)([\n]{2})(?P<rest>.+[\n])',
re.MULTILINE | re.DOTALL)
if pattern.findall(out):
match = re.search(grouping, out)
first = match.group('first')
second = match.group('second')
rest = match.group('rest')
with csv.writer(open(FILE, 'a')) as f:
writer = csv.writer(f, delimiter='\t')
writer.writerow(first, second, rest)
Edit: Requested in the comments to post entire traceback, note the line listed in traceback will not match the above code as this is not the entire script.
Traceback (most recent call last):
File "/mydir/pyrr.py", line 60, in <module>
run_rip()
File "/mydir/pyrr.py", line 55, in run_rip
with csv.writer(open('/mydir/ntuser.csv', 'a')) as f:
AttributeError: __exit__
Answer: Using the below comment I was able to write it as follows.
f = csv.writer(open('/mydir/ntuser.csv', 'a'),
dialect=csv.excel,
delimiter='\t')
f.writerow((first, second, rest))
The error is pretty clear. The with statement takes a context manager, i.e., an object with an __enter__ and an __exit__ method, such as the object returned by open. csv.writer does not provide such an object. You are also attempting to create the writer twice:
with open(FILE, 'a') as f:
writer = csv.writer(f, delimiter='\t')
writer.writerow(first, second, rest)
The with ... f: is like a try...except...finally that guarantees that f is closed no matter what happens, except you don't have to type it out. open(...) returns a context manager whose __exit__ method is called in that finally block you don't have to type. That is what your exception was complaining about. open returns an object that has __exit__ properly defined and can therefore handle normal exit and exceptions in the with block. csv.writer does not have such a method, so you can't use it in the with statement itself. You have to do it in the with block following the statement, as I've shown you.
What I'm trying to achieve here is to submit multiple jobs to Gearman and print the results returned by the workers once they are done with processing the job.
I've read through the examples on:
https://pythonhosted.org/gearman/1to2.html#client-multiple-tasks
https://pythonhosted.org/gearman/client.html
I then tried to implement the check_request_status in the following way:
list_of_jobs = []
for i in xrange(1,4,1):
list_of_jobs.extend(dict(task='run_task', data=str(i)))
completed_requests = gm_client.wait_until_jobs_completed(submitted_requests,
poll_timeout=30.0)
for job_request in completed_requests:
if job_request.complete:
print job_request.result
elif job_request.timed_out:
print "Job %s timed out!" % job_request.unique
elif job_request.state == JOB_UNKNOWN:
print "Job %s connection failed!" % job_request.unique
I'm getting the following error which I can't seem to figure out even after much Google-fu, searching through the Gearman Google Groups and poring through other people's implementation of Gearman:
Traceback (most recent call last):
File "supervisor.py", line 16, in <module>
completed_requests = gm_client.submit_multiple_jobs(list_of_jobs)
File "/usr/local/lib/python2.7/dist-packages/gearman/client.py", line 48, in submit_multiple_jobs
requests_to_submit = [self._create_request_from_dictionary(job_info, background=background, max_retries=max_retries) for job_info in jobs_to_submit]
File "/usr/local/lib/python2.7/dist-packages/gearman/client.py", line 169, in _create_request_from_dictionary
job_unique = job_info.get('unique')
AttributeError: 'str' object has no attribute 'get'
Does anyone know what's going on here?
Turns out the error is a simple one. In the for-loop above, simply use the append function instead of extend for adding a dict to the list_of_jobs array:
list_of_jobs.append(dict(task='run_task', data=str(i)))
I'm trying to run a celery (3.1.17) task that executes further tasks in a group but I always run into errors. This is how I set up the code:
from celery import task, group
#task
def daily_emails():
[...]
all_tasks = []
for chunk in range(0, users.count(), 1000):
some_users = users[chunk:chunk+1000]
all_tasks.append(write_email_bunch.subtask(some_users, execnum))
job = group(all_tasks)
# result = job.apply_async()
# job.get()
result = job.delay()
print result
results = result.join()
print results
print "done writing email tasks"
count = sum(results)
print count
#task
def write_email_bunch(some_users, execnum):
[...]
return len(some_users) - skipped_email_count
And this is the output:
<GroupResult: 3d766c85-21af-4ed0-90cb-a1ca2d281db1 [69527252-8468-4358-9328-144f727f372b, 6d03d86e-1b69-4f43-832e-bd27c4dfc092, 1d868d1b-b502-4672-9895-430089e9532e]>
Traceback (most recent call last):
File "send_daily_emails.py", line 8, in <module>
daily_emails()
File "/var/www/virtualenvs/nt_dev/local/lib/python2.7/site-packages/celery/app/task.py", line 420, in __call__
return self.run(*args, **kwargs)
File "/var/www/nt_dev/nt/apps/emails/tasks.py", line 124, in daily_emails
results = result.join()
File "/var/www/virtualenvs/nt_dev/local/lib/python2.7/site-packages/celery/result.py", line 642, in join
interval=interval, no_ack=no_ack,
File "/var/www/virtualenvs/nt_dev/local/lib/python2.7/site-packages/celery/result.py", line 870, in get
raise self.result
TypeError: write_email_bunch() argument after ** must be a mapping, not long
So I get a GroupResult but somehow Im unable to join it or further process it.
And when I use write_email_bunch.s(some_users, execnum) I get this exception:
File "/var/www/virtualenvs/nt_dev/local/lib/python2.7/site-packages/celery/result.py", line 870, in get
raise self.result
TypeError: 'tuple' object is not callable
How would I wait for all the Group Tasks to be completed to continue afterwards?
job.get() gives me this exception:
TypeError: get expected at least 1 arguments, got 0
subtask takes a tuple of args, a dict of kwargs and task options so it should be called like this:
all_tasks.append(write_email_bunch.subtask((some_users, execnum)))
note that we are passing it a tuple containing the args
Also you shouldn't wait on a task inside a task - this can cause deadlocks. In this case I reckon daily_emails does not need to be a celery task - it can be a regular function that creates a canvas object and runs apply async.
def daily_emails():
all_tasks = []
for chunk in range(0, users.count(), 1000):
some_users = users[chunk:chunk+1000]
all_tasks.append(write_email_bunch.subtask(some_users, execnum))
job = group(all_tasks)
result = job.apply_async()
return result.id
In addition to the other answer you could be using chunks here:
http://docs.celeryproject.org/en/latest/userguide/canvas.html#chunks
#app.task
def daily_emails():
return write_email.chunks(users, 1000).delay()
#task
def write_email(user):
[...]
It may be beneficial to do it manually if getting several objects at once
from the db is important. You should also consider that the model objects will be serialized here, to avoid that you can send the pk only and refetch the model in the task, or send the fields that you care about (like email address or whatever is required to send that email to the user).
I am trying to get all the tweets from the previous day. And to address the rate limit by Twitter, I implemented two sets of codes.
if counter == 4000:
time.sleep(60*20) # wait for 20 min every time 4,000 tweets are extracted
counter == 0
continue
I looked at the output file and usually I get the rate limit message when I have about 5500-6500 tweet entities extracted. So to be conservative, I set that every time 4000 tweets (and the associated extracted fields) are extracted, I paused it for 20 min (to cover the Twitter's designated 15-min interval).
I also found someone else's attempt to address the same issue using the following code:
except tweepy.TweepError:
time.sleep(60*20)
continue
It is supposed to pause the script when there is a TweepError, I tested it, but it didn't seem to work, but I included it anyway.
The error I got (after extracting 10,700 tweet entities) is as follows:
Traceback (most recent call last):
File "C:\Users\User\Dropbox\Python exercises\_Scraping\Social media\TweepyModule\TweepyTut1.18.py", line 32, in <module>
since='2014-09-15', until='2014-09-16').items(999999999): # changeable here
File "C:\Program Files Extra\Python27\lib\site-packages\tweepy\cursor.py", line 181, in next
self.current_page = self.page_iterator.next()
File "C:\Program Files Extra\Python27\lib\site-packages\tweepy\cursor.py", line 99, in next
data = self.method(max_id=self.max_id, parser=RawParser(), *self.args, **self.kargs)
File "C:\Program Files Extra\Python27\lib\site-packages\tweepy\binder.py", line 230, in _call
return method.execute()
File "C:\Program Files Extra\Python27\lib\site-packages\tweepy\binder.py", line 203, in execute
raise TweepError(error_msg, resp)
tweepy.error.TweepError: {"errors":[{"message":"Rate limit exceeded","code":88}]}
[Finished in 1937.2s with exit code 1]
Here is my code:
import tweepy
import time
import csv
ckey = ""
csecret = ""
atoken = ""
asecret = ""
OAUTH_KEYS = {'consumer_key':ckey, 'consumer_secret':csecret,
'access_token_key':atoken, 'access_token_secret':asecret}
auth = tweepy.OAuthHandler(OAUTH_KEYS['consumer_key'], OAUTH_KEYS['consumer_secret'])
api = tweepy.API(auth)
searchTerms = '"good book"'
counter = 0
for tweet in tweepy.Cursor(api.search, q=searchTerms,
since='2014-09-15', until='2014-09-16').items(999999999): # changeable here
try:
'''print "Name:", tweet.author.name.encode('utf8')
print "Screen-name:", tweet.author.screen_name.encode('utf8')
print "Tweet created:", tweet.created_at'''
placeHolder = []
placeHolder.append(tweet.author.name.encode('utf8'))
placeHolder.append(tweet.author.screen_name.encode('utf8'))
placeHolder.append(tweet.created_at)
with open("TweetData_goodBook_15SEP2014_all.csv", "ab") as f: # changeable here
writeFile = csv.writer(f)
writeFile.writerow(placeHolder)
counter += 1
if counter == 4000:
time.sleep(60*20) # wait for 20 min everytime 4,000 tweets are extracted
counter == 0
continue
except tweepy.TweepError:
time.sleep(60*20)
continue
except IOError:
time.sleep(60*2.5)
continue
except StopIteration:
break
I'm trying to use the Python package ComplexNetworkSim, which inherits from networkx and SimPy, to simulate an agent-based model of how messages propagate within networks.
Here is my code:
from ComplexNetworkSim import NetworkSimulation, NetworkAgent, Sim
import networkx as nx
#define constants for our example of states
NO_MESSAGE = 0
MESSAGE = 1
class Message(object):
def __init__(self,topic_pref):
self.relevance = topic_pref
class myAgent(NetworkAgent):
def __init__(self, state, initialiser):
NetworkAgent.__init__(self, state, initialiser)
self.state = MESSAGE
self.topic_pref = 0.5
def Run(self):
while True:
if self.state == MESSAGE:
self.message = self.Message(topic_pref, self, TIMESTEP)
yield Sim.hold, self, NetworkAgent.TIMESTEP_DEFAULT
elif self.state == NO_MESSAGE:
yield Sim.hold, self, NetworkAgent.TIMESTEP_DEFAULT
# Network and initial states of agents
nodes = 30
G = nx.scale_free_graph(nodes)
states = [MESSAGE for n in G.nodes()]
# Simulation constants
MAX_SIMULATION_TIME = 25.0
TRIALS = 2
def main():
directory = 'test' #output directory
# run simulation with parameters
# - complex network structure
# - initial state list
# - agent behaviour class
# - output directory
# - maximum simulation time
# - number of trials
simulation = NetworkSimulation(G,
states,
myAgent,
directory,
MAX_SIMULATION_TIME,
TRIALS)
simulation.runSimulation()
if __name__ == '__main__':
main()
(There may be other problems downstream with this code and it is not fully tested.)
My problem is that the myAgent object is not properly calling the method Run as an attribute. Specifically, this is the error message that I get when I try to run the above code:
Starting simulations...
---Trial 0 ---
set up agents...
Traceback (most recent call last):
File "simmessage.py", line 55, in <module>
main()
File "simmessage.py", line 52, in main
simulation.runSimulation()
File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/ComplexNetworkSim-0.1.2-py2.7.egg/ComplexNetworkSim/simulation.py", line 71, in runSimulation
self.runTrial(i)
File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/ComplexNetworkSim-0.1.2-py2.7.egg/ComplexNetworkSim/simulation.py", line 88, in runTrial
self.activate(agent, agent.Run())
AttributeError: 'myAgent' object has no attribute 'Run'
Does anybody know why this is? I can't figure how my code differs substantially from the example in ComplexNetworkSim.
I've run your code on my machine and there the Run method gets called.
My best guess is what Paulo Scardine wrote, but since i can't reproduce the problem i can't actually debug it.