So I'm using libsvn on windows in a C++ application. I have several svn trees with which I'm using the api. I assume the whole initialization and setup is correct since all other operations work as expected.
I use svn_client_log4 (also tried svn_client_log with exactly the same results) and usually when my callback is called, I get the correct info. However, my problem is that every now and then, the revision number I get is -1 instead of the full number. The strange thing is that this only happens on one of the tree I'm querying and always that same one.
It's not very consistent, happens usually once per day and it's very hard to debug because whenever I try to step through the code, it usually doesn't happen. Now I guess -1 is used as the head revision and technically it might not be wrong, but I'm looking for the actual number.
I've seen a couple of different ways of grabbing the latest revision with the api so I'll give them a go, but I still feel it's strange that this happens only sometimes. Perhaps someone has experienced it before or knows what could be wrong?
You need to check the error message from the actual svn_client_log call. The return type contains information about the error (there's a char* called message).
I suspect you may be slamming the server, which would result in sporadic issues that won't show up in (much slower) debugging.
Related
This might sound strange, but I would like to alter my local Node.js version and modify the Promise implementation to add a new source instance property.
global.Promise = class SourcePromise extends Promise {
constructor(params) {
super(params)
this.source = new Error('This is where this promise was created').stack
}
}
This would be to help me debug an error occurring on my Nuxt app, but only on the server. I'm able to catch the error by listening to the unhandledRejection event, but the error returned is not an Error object, it is simply undefined so I have no clue where it's coming from. The callback of unhandledRejection also returns the promise so I tried to add the code snippet above at the very beginning of nuxt start script to be able to log the source like:
process.on('unhandledRejection', (error, promise) => {
console.log('Unhandled Rejection:', error?.stack)
console.log('Promise source:', promise.source)
})
but promise.source is also undefined. If I log console.log(Promise.resolve().source) from any script, it works and I get the source so the only explanation I have in mind would be that the promise is created in a child process where my Promise extension is not defined.
To sum up, since it's happening in a separate process and I can't identify which one, the only way I see of implementing my SourcePromise globally in all Node processes would be to change the Promise definition directly in my local version of Node. Is it even possible?
I'm on macOS Monterey 12.3.1 using nvm v0.38.0
EDIT
I ended up cloning Node.js from Github to be able to build it locally and use it to start my Nuxt server. The problem is: it's written in C++ which I don't understand. I think I found where the promise constructor is defined which calls NewJsPromise that seems to be defined here, but I'll need help from a C++ developer since I still don't know how or where to add the stack...
(V8 developer here.) Bad news up front: I don't know exactly how to do what you're asking, and I don't have the time to figure it out for you.
But since you're not getting other answers so far, I'll try to say a few things that might help you make progress:
I don't think this is a "separate process" issue. JS objects, including Promises, can't travel between processes. If you see it in one process, then it was created in that same process.
There's more than one piece of code in V8 that creates Promises, so to be sure that you'll catch the promise in question, you'll have to update all of them:
NewJSPromise in src/builtins/promise-misc.tq, which you've found, is the Torque (not C++!) implementation, used for both the JavaScript Promise constructor and several other builtins. Note that if you put your modifications only into the PromiseConstructor, that would skip the other uses of the helper, so be sure to update NewJSPromise.
There's the C++ version in Factory::NewJSPromise in src/heap/factory.cc (which is used, probably among other things, for V8's API, i.e. any case where Node itself or a custom Node add-on creates Promises).
There's inlining support in the optimizing compiler in PromiseBuiltinReducerAssembler::ReducePromiseConstructor in src/compiler/js-call-reducer.cc (which could potentially be dealt with by disabling compiler support for inlining Promise creation).
The general outline of what you'd do is:
Update the Promise object definition to include another field.
Look at how Error objects are created to see how to get the stack. Error objects employ some fancy "lazy creation" scheme for stack traces for performance reasons; it may be easier to follow that example (to minimize divergence), or it may be easier to simplify it (because you don't care about performance). Note that AFAICT Error objects are always created in C++; you'd have to figure out how to get stacks to Torque-created Promises (off the top of my head I don't have a good suggestion for that).
Update all places where Promises are created (see above) to initialize the new field accordingly.
I strongly suspect that there are less time consuming ways to debug your original problem. (I dunno, maybe just audit all places where you're dealing with promises, and check that they all have rejection handlers? That might take hours, but quite possibly fewer hours than the V8 modification project sketched above.)
Good luck with your investigation!
I have a C++ application that crashes with segfault with some unknown customer data. Customer refuses to share his input data. Is it possible to figure out where did error happen?
When Java application crashes on end-user side it usually produces a stack trace that can help developer to figure out where is the error in program and what program invariants where broken.
But what should C++ developer do in this case? Should I recompile application with some compiler option so it provides some diagnostics when error happens?
If you don't have the input data required to recreate the problem (for whatever reason...including difficult customers) and you don't have core/minidumps, there is not much you can do. I've been in many situations such as this. My recourse was to recreate what I thought was the execution path based on interviewing the customer and then just do a meticulous code review to find possibilities of error conditions. I would test every candidate condition and eventually find the problem. This is painful, time consuming, and the main prerequisite is that you are able to read code nearly like you're reading your native language.
Begin Story Time
I worked somewhere that had a crash bug randomly manifest in a multi-tenant system. No amount of logging, core dumps, etc. would help us find it. Finally I reviewed the code (line. by. line. for multiple thousands of lines) and noticed that the developer was constructing a std::string instance from a char* sequence passed to the ctor. It was DEEP down in the parts of the code that hardly ever changed, so correlating the issue to recent changes was just a set of false leads. I asked the developer, "Are all your char arrays null terminated?" Answer: "No." Me: "Well we are then randomly reading memory until it finds a null, and apparently sometimes the heap has a lot of contiguous non-zero memory." Handling the char array bounds differently resulted in fixing the problem.
End Story Time
While you can't find a single way to find all bugs, there is a defensive design you can apply that is quite simple. Most people put it in the code once they get burned by this type of situation. The approach is to add support for different levels of logging verbosity and essentially instrument your code with log outputs that don't execute unless the code is set to use the correct level of verbosity. Turning the verbosity level up until the bug is recreated gives you at least some idea of where it is happening. Often customers will not have a problem sharing redacted log data (assuming there is sensitive data in the logs). Load the logs in Splunk or something similar (if the customer doesn't already aggregate their logs in an analysis tool) and you'll have an easier time reviewing the data.
Unfortunately with C++ you don't get nice stack traces and post-mortem data for free (in general). You have to add these post-mortem troubleshooting capabilities into your design up front. Most of the design gets driven from the expected deployment environment and user personas of your code, so add "difficult customer" as a persona and start coding. :)
Everywhere I read that glDebugMessageCallback is the better solution to get errors, so I implemented it. So far so good. And I am getting indeed more detailed information about what's going on. Currently on NVIDIA it looks f.e. like this:
DebugLog: In source API, type OTHER, id 131185, severity : NONE,
message Buffer detailed info: Buffer object 3 (bound to
GL_VERTEX_ATTRIB_ARRAY_BUFFER_BINDING_ARB (0), and
GL_ARRAY_BUFFER_ARB, usage hint is GL_STREAM_DRAW) will use VIDEO
memory as the source for buffer object operations.
Really nice, indeed- however, I miss one thing- I can't indicate here where exactly this happened.
If I use old style method with a macro like this:
#define CHECK_GL_ERROR() CheckGLError(__FILE__, __LINE__)
and
glGetError()
it's far less detailed and a mess in the code- BUT! I can track it more easily down to the line or call it happened, at least when debugging myself.
Of course, if I reduce the log level to something more severe, it is probably also easier to identify the origin, since there are less functions in question, yet, depending on code I find this a bit imprecise to find a specific function.
So my question is now- is there a way to tell what exactly the callback triggered, the function or the perhaps the line in the code, like in the old method (that is, now without adding a manual breakpoint/debug)??
I would find that very handy, especially when considering a situation in which someone who is maybe just using the software could provide me a log only for a problem I can't reproduce myself.
PS: Could someone enlighten me what "id" is for? I found a lot of tutorials and explanations, also read the docs but I still don't see of what use it is for debugging.
There are two ways that help you identify where the error comes from:
First, you can glEnable(GL_DEBUG_OUTPUT_SYNCHRONOUS) to ensure that errors are thrown in the scope of the function that produces them. When you now set a breakpoint in the error callback function, you'll see through the callstack where the error originates.
Second: OpenGL allows you to associate names and scopes with each OpenGL object. This allows you to specify names for, let's say, a buffer. Have a look here for more details.
I have embedded Lua in a C++ application. I need to be able to kill rogue (i.e. badly written scripts) from hogging resources.
I know I will not be able to cater for EVERY type of condition that causes a script to run indefinitely, so for now, I am only looking at the straightforward Lua side (i.e. scripting side problems).
I also know that this question has been asked (in various guises) here on SO. Probably the reason why it is constantly being re-asked is that as yet, no one has provided a few lines of code to show how the timeout (for the simple cases like the one I described above), may actually be implemented in working code - rather than talking in generalities, about how it may be implemented.
If anyone has actually implemented this type of functionality in a C++ with embedded Lua application, I (as well as many other people - I'm sure), will be very grateful for a little snippet that shows:
How a timeout can be set (in the C++ side) before running a Lua script
How to raise the timeout event/error (C++ /Lua?)
How to handle the error event/exception (C++ side)
Such a snippet (even pseudocode) would be VERY, VERY useful indeed
You need to address this with a combination of techniques. First, you need to establish a suitable sandbox for the untrusted scripts, with an environment that provides only those global variables and functions that are safe and needed. Second, you need to provide for limitations on memory and CPU usage. Third, you need to explicitly refuse to load pre-compiled bytecode from untrusted sources.
The first point is straightforward to address. There is a fair amount of discussion of sandboxing Lua available at the Lua users wiki, on the mailing list, and here at SO. You are almost certainly already doing this part if you are aware that some scripts are more trusted than others.
The second point is question you are asking. I'll come back to that in a moment.
The third point has been discussed at the mailing list, but may not have been made very clearly in other media. It has turned out that there are a number of vulnerabilities in the Lua core that are difficult or impossible to address, but which depend on "incorrect" bytecode to exercise. That is, they cannot be exercised from Lua source code, only from pre-compiled and carefully patched byte code. It is straightforward to write a loader that refuses to load any binary bytecode at all.
With those points out of the way, that leaves the question of a denial of service attack either through CPU consumption, memory consumption, or both. First, the bad news. There are no perfect techniques to prevent this. That said, one of the most reliable approaches is to push the Lua interpreter into a separate process and use your platform's security and quota features to limit the capabilities of that process. In the worst case, the run-away process can be killed, with no harm done to the main application. That technique is used by recent versions of Firefox to contain the side-effects of bugs in plugins, so it isn't necessarily as crazy an idea as it sounds.
One interesting complete example is the Lua Live Demo. This is a web page where you can enter Lua sample code, execute it on the server, and see the results. Since the scripts can be entered anonymously from anywhere, they are clearly untrusted. This web application appears to be as secure as can be asked for. Its source kit is available for download from one of the authors of Lua.
Snippet is not a proper use of terminology for what an implementation of this functionality would entail, and that is why you have not seen one. You could use debug hooks to provide callbacks during execution of Lua code. However, interrupting that process after a timeout is non-trivial and dependent upon your specific architecture.
You could consider using a longjmp to a jump buffer set just prior to the lua_call or lua_pcall after catching a time out in a luaHook. Then close that Lua context and handle the exception. The timeout could be implemented numerous ways and you likely already have something in mind that is used elsewhere in your project.
The best way to accomplish this task is to run the interpreter in a separate process. Then use the provided operating system facilities to control the child process. Please refer to RBerteig's excellent answer for more information on that approach.
A very naive and simple, but all-lua, method of doing it, is
-- Limit may be in the millions range depending on your needs
setfenv(code,sandbox)
pcall (function() debug.sethook(
function() error ("Timeout!") end,"", limit)
code()
debug.sethook()
end);
I expect you can achieve the same through the C API.
However, there's a good number of problems with this method. Set the limit too low, and it can't do its job. Too high, and it's not really effective. (Can the chunk get run repeatedly?) Allow the code to call a function that blocks for a significant amount of time, and the above is meaningless. Allow it to do any kind of pcall, and it can trap the error on its own. And whatever other problems I haven't thought of yet. Here I'm also plain ignoring the warnings against using the debug library for anything (besides debugging).
Thus, if you want it reliable, you should probably go with RB's solution.
I expect it will work quite well against accidental infinite loops, the kind that beginning lua programmers are so fond of :P
For memory overuse, you could do the same with a function checking for increases in collectgarbage("count") at far smaller intervals; you'd have to merge them to get both.
I am looking for a binary diff tool which allows me to mask parts of the binary files so they won't be taken into account.
The challenge I am facing is the following: Once we finished actively developing a branch of our embedded software we put it into maintenance. This means it is still build once a week to ensure that it doesn't erode and is not buildable any more. This is necessary so that once we urgently need to fix a bug or add a little feature we can concentrate on this and not need to solve other problems as well.
With our current approach we know that the branch still builds without errors but we do not know whether the produced binaries are is still the same. So I thought comparing a “golden master” binary against what was build and decide whether they are identical would be an approach to this. Anyhow, the files will never be identical to the bit because e.g. build dates are part of the binaries.
Does anyone have an idea or has even solved this puzzle before? I guess I am not the first developer facing this challenge.
Regards,
Mark
The Linux from Scratch project ran into this problem when they were transitioning from Glibc 2.2 to Glibc 2.3, I think in 2001. They developed some scripts and techniques to reduce the amount of noise, but it was still only reducing the amount of noise. They still needed to manually inspect the remaining differences. Anyway, if you dig around the mailinglist archives from around 2001 to 2003, you should find some pointers.
However, aren't you going at this the wrong way? I mean, the interesting question is not so much whether it still builds identical binaries, but rather whether it still works, isn't it? So, it is much more interesting whether it still passes the testsuite rather than whether it is still the same binary.
After all, nobody cares whether it is still the same binary, if there is no obversable difference in behavior. And if it's broken, then nobody is going to take "Yes, I know, but it is broken exactly the same way it was last week, and I can prove it!" as an excuse.