i am a new Clojure/Cascalog user trying to migrate some pig scripts, but often i get an error like the following in repl.
FlowException local step failed cascading.flow.planner.FlowStepJob.blockOnJob (FlowStepJob.java:191)
"with-debug" gives some more information but still no root cause of the issue. Any ideas on how to improve this?
I agree that the stacktraces are sometimes very unhelpful. One thing I can suggest is writing unit tests: http://sritchie.github.com/2011/09/30/testing-cascalog-with-midje.html which narrows down significantly where your actual problem lies.
If your query works for basic cases but fails on big data you can add a trap, to see what inputs are causing a failure:
(<- .... (:trap (hfs-textline "s3://.../errors" :sinkmode :replace)))
Related
I get this error message whenever I try to run code that uses the .dae extension.
I've been getting this error on code for a problem I've been working on. Eventually, I ran code that I found online for a different problem and got the same error. This makes me think that it's some error with how I'm running the solver (ipopt) rather than the code itself
Here's a link to the code from GitHub that I'm getting an error message with: https://github.com/Pyomo/pyomo/blob/master/examples/dae/Optimal_Control.py
I've been evoking the solve with:
pyomo solve --solver=ipopt filename.py
Most of the DAE examples come with a second Python script for running the model. Models formulated with Pyomo.DAE are meant to be run from Python scripts and not using the pyomo solve command line. The optimal control problem you're looking at can be solved by using the command
python run_Optimal_Control.py
If you take a look at this file you'll see the syntax for applying a discretization scheme to the dynamic model, creating a solver object using the SolverFactory, solving the model, and plotting the results.
I'm trying to set up an error detection system in c++ when doing a mysql_query, to check whether the query failed and if so try again (I'm using the mysql libray for Linux). The function mysql_query returns an int which is set to zero for success, and non-zero if failed (the error numbers are available at https://dev.mysql.com/doc/refman/5.0/en/mysql-query.html).
My first problem is that mysql_query only returns 0 or 1 for me, and this would be fine until I try to update a database entry with a similar content to what's already in the database. It also returns an error for that, which is fine on the principle, but as I can't differentiate it from an normal error, my current error detection system would try to repeat the query until it stops returning an error, which never happens. You'll tell me I could check what's in the DB first and then compare it to what I try to update before doing the query, and you would be right, but the code is already very complex and adding this level of complexity on the top would take quite some time.
So I'm wondering if there is a simpler answer to that? Thanks
Like #Mat wrote just use mysql_errno() (see Mysql Docs)
It returns 0 for success or an error code for errors.
Just so you guys know where this error actually came from, if you ever have the same.
I repetitively had mysql errors 2013 and 2006 when performing queries from different threads at the same time, but only after running a certain number of requests. I tried pretty much everything explained here.
Apparently for some reason the connection crashes and I can't do much to avoid it. The only thing that worked for me is to enable automatic reconnect using mysql_options and the option MYSQL_OPT_RECONNECT. It seems like a work around, more than a real fix, but at least my program doesn't crash now!
I am using PC-Lint for my C++ project.
Is there a way to switch off all error and warning messages by default, so I can then reintroduce the required messages explicitly?
I have read the chapter of the PC-Lint manual entitled "Error Inhibition Options" and the best I could find was setting the wLevel to -w0 No messages (except for fatal errors)
Yes, it is possible, you can simply use -e* or -w0. However, the manual truly states (Chapter 16. Living with Lint):
DO NOT simply suppress all warnings with something
like: -e* or -w0 as this can disguise hard errors and make subsequent diagnosis very difficult.
So, yes, you can use it if your code is basically cleaned, and you want to review recent changes for a certain set of messages. But if you want to start cleaning your code, and are swamped with messages because of the default warning level -w3, I suggest to start using -w1, and resolve all issues there; most of the warnings/errors given at level one indicate problems with finding all header files, having al implicit macros set properly, and/or mimicking the compiler you use normally in a sufficiently precise way.
As always, I hesitate to advertise my own work, but if you want, take a look at my "How to wield PC Lint" PDF, where I have documented detailed instructions to handle the initial deployment of PC Lint and tackling the many warnings/errors/infos/notes you may be buried under.
When I started introducing PC-Lint to a new project I did the following:
As suggested by Johan Bezem, ran a -w1 level check over the whole thing. This doesn't actually find any new problems, but checks that your program is syntactically valid and finds any configuration issues. Nothing major, assuming your project compiles already.
Run the test again with -w2 level. This found 53,000 issues, which was a bit much to tackle in one go.
Pick a typical bad file, then suppress any errors that seem
irrelevant or non-urgent (eg. error 525: (Warning -- Negative indentation from line xxx)
adding -e525 to the command line or config file, until you find one that seems serious.
In my case this was
error 442: (Warning -- for clause irregularity: testing direction inconsistent with increment direction), i.e. a 'for' loop
that looked like it should be counting up was actually counting
down.
Reset the test level back to -w1 but added in the critical problem by number, -w1 +e442 in this case. Re-run it over the whole project then fix all the instances of that problem.
Back to stage 2 and try again.
This combination of fixing actual problems and suppressing likely false alarms soon gets your numbers under control.
So that everything gets better over time we also implement a script that does a thorough (full -w2 or -w3) check on any files that are created or modified.
I also found the tool LintProject very helpful as it can do an entire Visual Studio solution in one go, with tables with numbers of errors and worst offenders!
today I ran into an error and have no clue how to fix it.
Error: App with label XYZ could not be found. Are you sure your INSTALLED_APPS setting is correct?
Where XYZ stands for the app-name that I am trying to reset. This error shows up every time I try to reset it (manage.py reset XYZ). Show all the sql code works.
Even manage.py validate shows no error.
I already commented out every single line of code in the models.py that I touched the last three months. (function by function, model by model) And even if there are no models left I get this error.
Here http://code.djangoproject.com/ticket/10706 I found a bugreport about this error. I also applied one the patches to allocate the error, it raises an exception so you have a trace back, but even there is no sign in what of my files the error occurred.
I don't want to paste my code right now, because it is nearly 1000 lines of code in the file I edited the most.
If someone of you had the same error please tell me were I can look for the problem. In that case I can post the important part of the source. Otherwise it would be too much at once.
Thank you for helping!!!
I had a similar problem, but I only had it working after creating an empty models.py file.
I was running Django 1.3
Try to clean up all your build artifacts: build files, temporary files and so on. Also ./manage.py test XYZ will show you stack trace. Later try to run python with -m pdb option and step through the code to see where you fail and why.
You don't specify which server you're using. With Apache you'll almost certainly need a restart for things to take effect. If you're using the development one try restarting that. If this doesn't work you may need to give us some more details.
I'd also check your paths as you may have edited one file but you may be using a different one.
Plus check what's still in your database, as some of your previous versions may be interfering.
Finally as a last resort I'd try a clean install (on another django instance) and see if that goes cleanly, if it does then I'd know that I'd got a conflict, if not then the problem's in the code.
I have frequently encounter the following debugging scenario:
Tester provide some reproduce steps for a bug. And to find out where the problem is, I try to play with these reproduce steps to get the minimum necessary reproduce steps. Sometimes, luckily I found that when do a minor change to the steps, the problem is gone.
Then the job turns to find the difference in code workflow between these two reproduce steps. This job is tedious and painful especially when you are working on a large code base and it go through a lot code and involve lots of state changes which you are not familiar with.
So I was wondering is there any tools available to compare "code workflow". As I've learned the "wt" command in WinDbg, I thought it might be possible to do it. For example, I can run the "wt" command on some out most functions with 2 different reproduce steps and then compare the difference between outputs. Then it should be easy to found where the code flow starts to diverge.
But the problem with WinDBG is "wt" is quite slow (maybe I should use a log file instead of output to screen) and not very user-friendly (compared with visual studio debugger) ... So I want to ask you guys is there any existing tools available . or is it possible and difficult to develop a "plug-in" for visual studio debugger to support this functionality ?
Thanks
I'd run it under a profiler in "coverage" mode, then use diff on the results to see which parts of the code were executed in one run by not the other.
Sorry, I don't know of a tool which can do what you want, but even if it existed it doesn't sound like the quickest approach to finding out where the lower layer code is failing.
I would recommend to instrument your layer's code with high-level logs so you can know which module fails, stalls, etc. In debug, your logger can write to file, to output debug window, etc.
In general, failing fast and using exceptions are good ways to find out easily where things go bad.
Doing something after the fact is not going to cut it, since your problem is reproducing it.
The issue with bugs is seldom some interal wackiness but usually what the user's actually doing. If you log all the commands that the user enters then they can simply send you the log. You can substitute button clicks, mouse selects, etc. This will have some cost but certainly much less than something that keeps track of every method visited.
I am assuming that if you have a large application that you have good logging or tracing.
I work on a large server product with over 40 processes and over one million lines of code. Most of the time the error in the trace file is enough to identify the location of problem. However sometimes the error I see in the trace file is caused by some earlier code and the reason for this can be hard to spot. Then I use a comparative debugging technique:
Reproduce the first scenario, copy the trace to a new file (if the application is multi threaded ensure you only have the trace for the thread that does the work).
Reproduce the second scenario, copy the trace to a new file.
Remove the timestamps from the log files (I use awk or sed for this).
Compare the log files with winmerge or similar, to see where and how they diverge.
This technique can be a little time consuming, but is much quicker than stepping through thousand of lines in the debugger.
Another useful technique is producing uml sequence diagrams from trace files. For this you need the function entry and exit positions logged consistently. Then write a small script to parse your trace files and use sequence.jar to produce uml diagrams as png files. This is a great way to understand the logic of code you haven't touched in a while. I wrapped a small awk script in a batch file, I just provide trace file and line number to start then it untangles the threads and generates the input text to sequence.jar then runs its to create the uml diagram.