I have tried to solve this problem by reading old questions and by googles help.
I writing a short script in matlab where the user types in a equation and then plot the data by using eval.
But I want to check if the equation is right and uses the right variables and so...
I have three variables, X,Y,Z with upper case, so for example 'X+Y-Z-7.5' is a solid equation, but 'XB-Z' isn't. Just 'X' is also a solid "equation"...
How can I write the expression? Here is what I have...
regexp(test,'(X|Y|Z)$|(X|Y|Z|\d)&&(+|-|*|/|)')
My next plan is to do like,
if regexp(test,'(X|Y|Z)$|(X|Y|Z|\d)&&(+|-|*|/|)') == 1
disp ('Correct')
end
So I want the regexp return if the string matches the whole expression, not just startindex. I have problem to fix that too.
Please, I'm stuck.
One potential solution (if you have the Symbolic Math Toolbox) is to simply rely on that to determine whether the equations are valid.
You can use symvar to extract all symbols used in the equation and compare these to the variables you allow.
allowed = {'X', 'Y', 'Z'};
vars = symvar(userinput);
tf = ismember(vars, allowed);
if ~all(tf)
disp('Invalid Variables Used');
end
This is likely going to be much more robust than attempting to create regular expressions as it relies on MATLAB's internal parser.
Related
I'm new to this site and this is my first time to ask here.
My problem is I want to check if my string follows a correct pattern or syntax. I'm doing it with C++ String (std::string). I have already done this using C-Style string, however, I want to do it this time in C++ String. Sample problem below:
Input: 2y'' + 3y' - 2y = 0
or y'' = 4y
I want to check if the derivative input is in correct syntax like (a)y'' + (b)y' + (c)y = 0, a second order homogeneous equation. However, I still want to input a non-standard form equation like the second sample input that can be transposed and make it to standard form.
What I did before with it is remove all the white spaces, loop the entire string and check every index. Eg. if 'y' is found the next char should be '\'' or an arithmetic symbol like '-' or '+' or '=' then if it does not match, then, it must return false.
Or maybe I am just implementing this wrong. I'm new to programming and just taking a computer science course. Note: Sorry for my bad English and sorry if I did not written my code here. Its just way too long.
Regular expressions might be the answer. They're commonly used for checking whether a string first a format, or finding parts of a string that do.
RegExr is a great tool to both learn and test your regular expressions.
I want to get a specific digit based on the right string.
This stretch of string is in body2.txt
string = "<li>3 <span class='text-info'>quartos</span></li><li>1 <span class='text-info'>suíte</span></li><li>96<span class='text-info'>Área Útil (m²)</span></li>"
with open("body2.txt", 'r') as f:
area = re.compile(r'</span></li><li>(\d+)<span class="text-info">Área Útil')
area = area.findall(f.read())
print(area)
output: []
expected output: 96
You have a quote mismatch. Note carefully the difference between 'text-info' and "text-info" in your example string and in your compiled regex. IIRC escaping quotes in raw strings is a bit of a pain in Python (if it's even possible?), but string concatenation sidesteps the issue handily.
area = re.compile(r'</span></li><li>(\d+)<span class='"'"'text-info'"'"'>Área Útil')
Focusing on the quotes, this is concatenating the strings '...class', "'", 'text-info', "'", and '>.... The rule there is that if you want a single quote ' in a single-quote raw string you instead write '"'"' and try to ignore Turing turning in his grave. I haven't tested the performance, but I think it might behave much like '...class' + "'" + 'text-info' + "'" + '>.... If that's the case, there is a bunch of copying happening behind the scenes, and that strategy has a quadratic runtime in the number of pieces being concatenated (assuming they're roughly the same size and otherwise generally nice for such an analysis). You'd be better off with nearly any other strategy (such as ''.join(...) or using triple quoted raw strings r'''...'''). It might not be a problem though. Benchmark your solution and see if it's good enough before messing with alternatives.
As one of the comments mentioned, you probably want to be parsing the HTML with something more powerful than regex. Regex cannot properly parse arbitrary HTML since it can't parse arbitrarily nested structures. There are plenty of libraries to make the job easier though and handle all of the bracket matching and string munging for you so that you can focus on a high-level description of exactly the data you want. I'm a fan of lxml. Without putting a ton of time into it, something like the following would be roughly equivalent to what you're doing.
from lxml import html
with open("body2.txt", 'r') as f:
tree = html.fromstring(f.read())
area = tree.xpath("//li[contains(span/text(), 'Área Útil')]/text()")
print(area)
The html.fromstring() method parses your data as html. The tree.xpath method uses xpath syntax to query that parsed tree. Roughly speaking it means the following:
// Arbitrarily far down in the tree
li A list node
[*] Satisfying whatever property is in the square brackets
contains(span/text(), 'Área Útil') The li node needs to have a span/text() node containing the text 'Área Útil'
/text() We want any text that is an immediate child of the root li we're describing.
I'm working on a pretty small amount of text here and don't know what your document structure is in the general case. You could add or change any of those properties to better describe the exact document you're parsing. When you inspect an element, any modern browser is able to generate a decent xpath expression to pick out exactly the element you're inspecting. Supposing this snippet came from a larger document I would imagine that functionality would be a time saver for you.
This will get the right digits no matter how / what form the target is in.
Capture group 1 contains the digits.
r"(\d*)\s*<span(?=\s)(?=(?:[^>\"']|\"[^\"]*\"|'[^']*')*?\sclass\s*=\s*(?:(['\"])\s*text-info\s*\2))\s+(?=((?:\"[\S\s]*?\"|'[\S\s]*?'|[^>]?)+>))\3\s*Área\s+Útil"
https://regex101.com/r/pMATkj/1
I have a pandas df that contains many string formulas that I would like to be able to parse and eventually solve. I came across parse_expr and initially seemed like it would work for my problem but now I'm not so sure.
An example string formula might look like this:
A = B + C; D = A*.2;
parse_expr would seem to work well if i had a system of equations and I may not be using this correctly. As it stands, parse_expr throws an "invalid syntax" error I believe because of the equal sign. Can anyone tell if its possible to solve this problem using parse_expr or if there is another approach I should try?
SymPy cannot parse a bunch of semicolon-separated formulas at once, so the string needs to be split first. It will need to be split again at =, assuming all formulas have = in them. After parsing each side of =, you can combine them with Eq, which is SymPy's equation object; or use them somehow else.
from sympy import S, Eq
str = "A = B + C; D = A*.2;"
result = [Eq(*map(S, f.split("="))) for f in str.split(";")[:-1]]
The result is [Eq(A, B + C), Eq(D, 0.2*A)]
I use S, short for sympify; parse_expr could be used similarly, and it has a few options that are not needed here.
parse_expr is based on the Python tokenizer, but it has several extensions. These extensions take the form of functions that take a list of tokens, a locals dictionary, and a globals dictionary, and return a modified list of tokens. These are passed as a tuple to parse_expr, like parse_expr(expression, transformations=(transformation1, transformation2, ...)).
It's probably easiest to just take a look at the source of the sympy.parsing.sympy_parser submodule to see the existing transformations and how they work. Some of the transformations that are there will probably be useful to you. In this case, you would want a transformation that transforms the = token into something else (actually there's already a transformation function convert_equals_sign in the sympy_parser submodule that does this). You assumedly also want to handle *. somehow.
I've also written a guide on Python tokenization which may be helpful here: https://www.asmeurer.com/brown-water-python
If your syntax is too far off from Python's then it will be challenging to use parse_expr, since it only works with Python's tokenizer. In that case, you'd need to generate your own grammar and parser (e.g., using antlr) for your DSL and parse it into something that can then be transformed into a SymPy expression.
Original problem:
I am reading something from regex format, expands it and writing it out. This list can become huge while writing it out.
While writing it out, I do not have the original regex data. So, I will have to create regexes out of the strings which I have.
A couple of cases while reading and writing:
Say, read regular expression is:
abc/*
Since 'abc' can have only 'A', 'B', 'C', 'D'(Have this list with me), Above would be translated to list of strings as
"abc/A", "abc/B", "abc/C", "abc/D" -- 1
Say, another read regular expression is:
def/*/A
Since 'def' can have only 'x', 'y', 'x'(Have this list with me), Above would be translated to list of strings as
"def/x/A", "def/x/A", "def/x/A" -- 2
I have already said that I do not have original regular expressions now. All I have is list of strings. I will have to create regexes out of statements number 1 and 2.
From number 1, I should get
abc/*
From number 2, I should get
def/*/A
which are the original.
Question: Which data structure would be efficient to solve this problem. I have thought of using tries & Aho–Corasick algorithm but could not get a clear solution at top of my head till now.
I would be happy to expand the question more in case it is not clear. Please consider that * will not match /, //, or anything except characters.
This is a hard problem to solve.
Good possible solution:
Step 1: Convert list of strings into a finite automation. As I mentioned, I shall use 'Aho–Corasick algorithm' to do that.
Step 2: Transform the finite automation into a regular expression... Something like this: https://cs.stackexchange.com/questions/2016/how-to-convert-finite-automata-to-regular-expressions
I am writing a program which will tokenize the input text depending upon some specific rules. I am using C++ for this.
Rules
Letter 'a' should be converted to token 'V-A'
Letter 'p' should be converted to token 'C-PA'
Letter 'pp' should be converted to token 'C-PPA'
Letter 'u' should be converted to token 'V-U'
This is just a sample and in real time I have around 500+ rules like this. If I am providing input as 'appu', it should tokenize like 'V-A + C-PPA + V-U'. I have implemented an algorithm for doing this and wanted to make sure that I am doing the right thing.
Algorithm
All rules will be kept in a XML file with the corresponding mapping to the token. Something like
<rules>
<rule pattern="a" token="V-A" />
<rule pattern="p" token="C-PA" />
<rule pattern="pp" token="C-PPA" />
<rule pattern="u" token="V-U" />
</rules>
1 - When the application starts, read this xml file and keep the values in a 'std::map'. This will be available until the end of the application(singleton pattern implementation).
2 - Iterate the input text characters. For each character, look for a match. If found, become more greedy and look for more matches by taking the next characters from the input text. Do this until we are getting a no match. So for the input text 'appu', first look for a match for 'a'. If found, try to get more match by taking the next character from the input text. So it will try to match 'ap' and found no matches. So it just returns.
3 - Replace the letter 'a' from input text as we got a token for it.
4 - Repeat step 2 and 3 with the remaining characters in the input text.
Here is a more simple explanation of the steps
input-text = 'appu'
tokens-generated=''
// First iteration
character-to-match = 'a'
pattern-found = true
// since pattern found, going recursive and check for more matches
character-to-match = 'ap'
pattern-found = false
tokens-generated = 'V-A'
// since no match found for 'ap', taking the first success and replacing it from input text
input-text = 'ppu'
// second iteration
character-to-match = 'p'
pattern-found = true
// since pattern found, going recursive and check for more matches
character-to-match = 'pp'
pattern-found = true
// since pattern found, going recursive and check for more matches
character-to-match = 'ppu'
pattern-found = false
tokens-generated = 'V-A + C-PPA'
// since no match found for 'ppu', taking the first success and replacing it from input text
input-text = 'u'
// third iteration
character-to-match = 'u'
pattern-found = true
tokens-generated = 'V-A + C-PPA + V-U' // we'r done!
Questions
1 - Is this algorithm looks fine for this problem or is there a better way to address this problem?
2 - If this is the right method, std::map is a good choice here? Or do I need to create my own key/value container?
3 - Is there a library available which can tokenize string like the above?
Any help would be appreciated
:)
So you're going through all of the tokens in your map looking for matches? You might as well use a list or array, there; it's going to be an inefficient search regardless.
A much more efficient way of finding just the tokens suitable for starting or continuing a match would be to store them as a trie. A lookup of a letter there would give you a sub-trie which contains only the tokens which have that letter as the first letter, and then you just continue searching downward as far as you can go.
Edit: let me explain this a little further.
First, I should explain that I'm not familiar with these the C++ std::map, beyond the name, which makes this a perfect example of why one learns the theory of this stuff as well as than details of particular libraries in particular programming languages: unless that library is badly misusing the name "map" (which is rather unlikely), the name itself tells me a lot about the characteristics of the data structure. I know, for example, that there's going to be a function that, given a single key and the map, will very efficiently search for and return the value associated with that key, and that there's also likely a function that will give you a list/array/whatever of all of the keys, which you could search yourself using your own code.
My interpretation of your data structure is that you have a map where the keys are what you call a pattern, those being a list (or array, or something of that nature) of characters, and the values are tokens. Thus, you can, given a full pattern, quickly find the token associated with it.
Unfortunately, while such a map is a good match to converting your XML input format to a internal data structure, it's not a good match to the searches you need to do. Note that you're not looking up entire patterns, but the first character of a pattern, producing a set of possible tokens, followed by a lookup of the second character of a pattern from within the set of patterns produced by that first lookup, and so on.
So what you really need is not a single map, but maps of maps of maps, each keyed by a single character. A lookup of "p" on the top level should give you a new map, with two keys: p, producing the C-PPA token, and "anything else", producing the C-PA token. This is effectively a trie data structure.
Does this make sense?
It may help if you start out by writing the parsing code first, in this manner: imagine someone else will write the functions to do the lookups you need, and he's a really good programmer and can do pretty much any magic that you want. Writing the parsing code, concentrate on making that as simple and clean as possible, creating whatever interface using these arbitrary functions you need (while not getting trivial and replacing the whole thing with one function!). Now you can look at the lookup functions you ended up with, and that tells you how you need to access your data structure, which will lead you to the type of data structure you need. Once you've figured that out, you can then work out how to load it up.
This method will work - I'm not sure that it is efficient, but it should work.
I would use the standard std::map rather than your own system.
There are tools like lex (or flex) that can be used for this. The issue would be whether you can regenerate the lexical analyzer that it would construct when the XML specification changes. If the XML specification does not change often, you may be able to use tools such as lex to do the scanning and mapping more easily. If the XML specification can change at the whim of those using the program, then lex is probably less appropriate.
There are some caveats - notably that both lex and flex generate C code, rather than C++.
I would also consider looking at pattern matching technology - the sort of stuff that egrep in particular uses. This has the merit of being something that can be handled at runtime (because egrep does it all the time). Or you could go for a scripting language - Perl, Python, ... Or you could consider something like PCRE (Perl Compatible Regular Expressions) library.
Better yet, if you're going to use the boost library, there's always the Boost tokenizer library -> http://www.boost.org/doc/libs/1_39_0/libs/tokenizer/index.html
You could use a regex (perhaps the boost::regex library). If all of the patterns are just strings of letters, a regex like "(a|p|pp|u)" would find a greedy match. So:
Run a regex_search using the above pattern to locate the next match
Plug the match-text into your std::map to get the replace-text.
Print the non-matched consumed input and replace-text to your output, then repeat 1 on the remaining input.
And done.
It may seem a bit complicated, but the most efficient way to do that is to use a graph to represent a state-chart. At first, i thought boost.statechart would help, but i figured it wasn't really appropriate. This method can be more efficient that using a simple std::map IF there are many rules, the number of possible characters is limited and the length of the text to read is quite high.
So anyway, using a simple graph :
0) create graph with "start" vertex
1) read xml configuration file and create vertices when needed (transition from one "set of characters" (eg "pp") to an additional one (eg "ppa")). Inside each vertex, store a transition table to the next vertices. If "key text" is complete, mark vertex as final and store the resulting text
2) now read text and interpret it using the graph. Start at the "start" vertex. ( * ) Use table to interpret one character and to jump to new vertex. If no new vertex has been selected, an error can be issued. Otherwise, if new vertex is final, print the resulting text and jump back to start vertex. Go back to (*) until there is no more text to interpret.
You could use boost.graph to represent the graph, but i think it is overly complex for what you need. Make your own custom representation.