I got this list of items (it's in a sql script) and I would like to reorder it by number :
from this :
,user_1
,user_2
,user_3
,name_1
,name_2
,name_3
to this
,user_1
,name_1
,user_2
,name_2
,user_3
,name_3
I use sql server management studio 2008 so I have ability to replace using regex but I don't know if that kind of manipulation is even possible with regular expressions.
Just copy paste them in excel, then sort and then copy paste back to ssms.
It's that simple :)
I think you need to add a bit more description for this to really make sense.
Perhaps post the SQL script?
Is this data stored in a single varchar field and this is the reason you are looking for a regex solution?
You can easily parse the comma-seperated values using a regex, but you would need some other function to sort that result and it can fairly quickly get messy to do this in SQL.
In general I would say this problem is better handled outside of the SQL statement - eg. process this in your favorite programming/scripting language after getting the result back from the SQL.
Also this problem indicates a design problem with the database layout, if in any way possible the preferred way to solve this would probably be to restructure it.
Related
Not sure where else to go so thought I would ask the group.
I have Apache Calcite working as a SQL parser for my application - I am trying to parse MySQL. However it is unable to handle certain SQL statements. The issue seems to be with any SQL statement that contains a variable denoted by "##" so something like :
SELECT ##session.auto_increment_increment AS auto_increment_increment
fails in the parser. I appreciate this is MySQL specific but was wondering if there is a way to "handle" these ##'s to at least get them into the Node tree so I can provide a more useful response than throw an exception.
There is an open request for this feature, CALCITE-5066. Probably the best way to "handle" these ## variables is to implement the feature.
I'm not being facetious. A quick 'hack' solution will likely trip up if ## characters appear in comments or character literals. So it's better to handle this by modifying the parser. And once you've modified the parser, if you want it to stay working you should write tests and contribute it back to the project.
I have inherited a large base of SAS code. I need to reverse engineer to create some mapping document, so that given a field in the final output dataset, we can easily trace it all the way back to one of the inputs.
I can create it by hand, but can SAS automatically generate something like this?
No, I don't think there is any ready-made automated way of doing this.
Bear in mind that it is possible to create variables and pass them through a whole series of procs and data steps without mentioning them by name anywhere in the source code. Some sort of run-time analysis is therefore unavoidable.
Reeza's suggestion of using proc scaproc will yield some useful information for code executed within a single self-contained job running in a single SAS session, and the ATTR option in the record statement might be of some help to you when tracing the lineage of variables, but I'm afraid that however you approach this, it's going to take quite a lot of work.
I will first describe the problem and then what I currently look at, in terms of libraries.
In my application, we have a set of variables that are always available. For example: TOTAL_ITEMS, PRICE, CONTRACTS, ETC (we have around 15 of them). A clients of the application would like to have certain calculations performed and displayed, using those variables. Up until now, I have been constantly adding those calculations to the app. It's pain in the butt, and I would like to make it more generic by way of creating a template, where the user can specify a set of formulas that the application will parse and calculate.
Here is one case:
total_cost = CONTRACTS*PRICE*TOTAL_ITEMS
So, want to do something like that for the user to define in the template file:
total_cost = CONTRACTS*PRICE*TOTAL_ITEMS and some meta-date, like screen to display it on. Hence they will be specifying the formula with a screen. And the file will contain many formulas of this nature.
Right now, I am looking at two libraies: Spirit and matheval
Would anyone make recommendations what's better for this task, as well as references, examples, links?
Please let me know if the question is unclear, and I will try to further clarify it .
Thanks,
Sasha
If you have a fixed number of variables it may be a bit overkill to invoke a parser. Though Spirit is cool and I've been wanting to use it in a project.
I would probably just tokenize the string, make a map of your variables keyed by name (assuming all your variables are ints):
map<const char*,int*> vars;
vars["CONTRACTS"] = &contracts;
...
Then use a simple postfix calculator function to do the actual math.
Edit:
Looking at MathEval, it seems to do exactly what you want; set variables and evaluate mathematical functions using those variables. I'm not sure why you would want to create a solution at the level of a syntax parser. Do you have any requirements that MathEval does not fulfill?
Looks like it shouldn't be too hard to generate a simple parser using yacc and bison and integrate it into your code.
I don't know about matheval, but boost::spirit can do that for you pretty efficiently : see there.
If you're into template metaprogramming, you may want to have a look into Boost::Proto, but it will take some time to get started using it.
How can one allow code snippets to be entered into an editor (as stackoverflow does) like FCKeditor or any other editor while preventing XSS, SQL injection, and related attacks.
Part of the problem here is that you want to allow certain kinds of HTML, right? Links for example. But you need to sanitize out just those HTML tags that might contain XSS attacks like script tags or for that matter even event handler attributes or an href or other attribute starting with "javascript:". And so a complete answer to your question needs to be something more sophisticated than "replace special characters" because that won't allow links.
Preventing SQL injection may be somewhat dependent upon your platform choice. My preferred web platform has a built-in syntax for parameterizing queries that will mostly prevent SQL-Injection (called cfqueryparam). If you're using PHP and MySQL there is a similar native mysql_escape() function. (I'm not sure the PHP function technically creates a parameterized query, but it's worked well for me in preventing sql-injection attempts thus far since I've seen a few that were safely stored in the db.)
On the XSS protection, I used to use regular expressions to sanitize input for this kind of reason, but have since moved away from that method because of the difficulty involved in both allowing things like links while also removing the dangerous code. What I've moved to as an alternative is XSLT. Again, how you execute an XSL transformation may vary dependent upon your platform. I wrote an article for the ColdFusion Developer's Journal a while ago about how to do this, which includes both a boilerplate XSL sheet you can use and shows how to make it work with CF using the native XmlTransform() function.
The reason why I've chosen to move to XSLT for this is two fold.
First validating that the input is well-formed XML eliminates the possibility of an XSS attack using certain string-concatenation tricks.
Second it's then easier to manipulate the XHTML packet using XSL and XPath selectors than it is with regular expressions because they're designed specifically to work with a structured XML document, compared to regular expressions which were designed for raw string-manipulation. So it's a lot cleaner and easier, I'm less likely to make mistakes and if I do find that I've made a mistake, it's easier to fix.
Also when I tested them I found that WYSIWYG editors like CKEditor (he removed the F) preserve well-formed XML, so you shouldn't have to worry about that as a potential issue.
The same rules apply for protection: filter input, escape output.
In the case of input containing code, filtering just means that the string must contain printable characters, and maybe you have a length limit.
When storing text into the database, either use query parameters, or else escape the string to ensure you don't have characters that create SQL injection vulnerabilities. Code may contain more symbols and non-alpha characters, but the ones you have to watch out for with respect to SQL injection are the same as for normal text.
Don't try to duplicate the correct escaping function. Most database libraries already contain a function that does correct escaping for all characters that need escaping (e.g. this may be database-specific). It should also handle special issues with character sets. Just use the function provided by your library.
I don't understand why people say "use stored procedures!" Stored procs give no special protection against SQL injection. If you interpolate unescaped values into SQL strings and execute the result, this is vulnerable to SQL injection. It doesn't matter if you are doing it in application code versus in a stored proc.
When outputting to the web presentation, escape HTML-special characters, just as you would with any text.
The best thing that you can do to prevent SQL injection attacks is to make sure that you use parameterized queries or stored procedures when making database calls. Normally, I would also recommend performing some basic input sanitization as well, but since you need to accept code from the user, that might not be an option.
On the other end (when rendering the user's input to the browser), HTML encoding the data will cause any malicious JavaScript or the like to be rendered as literal text rather than executed in the client's browser. Any decent web application server framework should have the capability.
I'd say one could replace all < by <, etc. (using htmlentities on PHP, for example), and then pick the safe tags with some sort of whitelist. The problem is that the whitelist may be a little too strict.
Here is a PHP example
$code = getTheCodeSnippet();
$code = htmlentities($code);
$code = str_ireplace("<br>", "<br>", $code); //example to whitelist <br> tags
//One could also use Regular expressions for these tags
To prevent SQL injections, you could replace all ' and \ chars by an "innofensive" equivalent, like \' and \, so that the following C line
#include <stdio.h>//'); Some SQL command--
Wouldn't have any negative results in the database.
I have inherited a large legacy ColdFusion app. There are hundreds of <cfquery>some sql here #variable#</cfquery> statements that need to be parameterized along the lines of: <cfquery> some sql here <cfqueryparam value="#variable#"/> </cfquery>
How can I go about adding parameterization programmatically?
I have thought about writing some regular expression or sed/awk'y sort of solution, but it seems like somebody somewhere has tackled such a problem. Bonus points awarded for inferring the sql type automatically.
There's a queryparam scanner that will find them for you on RIAForge: http://qpscanner.riaforge.org/
There is a script referenced here: http://www.webapper.net/index.cfm/2008/7/22/ColdFusion-SQL-Injection that will do the majority of the heavy lifting for you. All you have to do is check the queries and make sure the syntax will parse properly.
There is no excuse for not using CFQueryParam, apart from it being much more secure, it is a performance boost and the best way to handle quoted values in character based column types.
Keep in mind that you may not be able to solve everything with <cfqueryparam>.
I've seen a number of examples where the order by field name is being passed in the query string, which is a slightly trickier problem to solve as you need to validate that in a more "manual" way.
<cf_inputFilter
scopes = "FORM,COOKIE,URL"
chars = "<,>,!,&,|,%,=,(,),',{,}"
tags="script,embed,applet,object,HTML">
We used this to counteract a recent SQL injection attack. We added it to the Application.cfm file for our site.
I doubt that there is a solution that will fit your needs exactly. The only option I see is to write your own recursive search that builds a report for you or use one of the apps/scripts that people have listed above. Basically, you are going to have to edit each page or approve all of the automated changes.