I have a problem with a string in C++ which has several words in Spanish. This means that I have a lot of words with accents and tildes. I want to replace them for their not accented counterparts. Example: I want to replace this word: "había" for habia. I tried replace it directly but with replace method of string class but I could not get that to work.
I'm using this code:
for (it= dictionary.begin(); it != dictionary.end(); it++)
{
strMine=(it->first);
found=toReplace.find_first_of(strMine);
while (found!=std::string::npos)
{
strAux=(it->second);
toReplace.erase(found,strMine.length());
toReplace.insert(found,strAux);
found=toReplace.find_first_of(strMine,found+1);
}
}
Where dictionary is a map like this (with more entries):
dictionary.insert ( std::pair<std::string,std::string>("á","a") );
dictionary.insert ( std::pair<std::string,std::string>("é","e") );
dictionary.insert ( std::pair<std::string,std::string>("í","i") );
dictionary.insert ( std::pair<std::string,std::string>("ó","o") );
dictionary.insert ( std::pair<std::string,std::string>("ú","u") );
dictionary.insert ( std::pair<std::string,std::string>("ñ","n") );
and toReplace strings is:
std::string toReplace="á-é-í-ó-ú-ñ-á-é-í-ó-ú-ñ";
I obviously must be missing something. I can't figure it out.
Is there any library I can use?.
Thanks,
I disagree with the currently "approved" answer. The question makes perfect sense when you are indexing text. Like case-insensitive search, accent-insensitive search is a good idea. "naïve" matches "Naïve" matches "naive" matches "NAİVE" (you do know that an uppercase i is İ in Turkish? That's why you ignore accents)
Now, the best algorithm is hinted at the approved answer: Use NKD (decomposition) to decompose accented letters into the base letter and a seperate accent, and then remove all accents.
There is little point in the re-composition afterwards, though. You removed most sequences which would change, and the others are for all intents and purposes identical anyway. WHat's the difference between æ in NKC and æ in NKD?
First, this is a really bad idea: you’re mangling somebody’s language by removing letters. Although the extra dots in words like “naïve” seem superfluous to people who only speak English, there are literally thousands of writing systems in the world in which such distinctions are very important. Writing software to mutilate someone’s speech puts you squarely on the wrong side of the tension between using computers as means to broaden the realm of human expression vs. tools of oppression.
What is the reason you’re trying to do this? Is something further down the line choking on the accents? Many people would love to help you solve that.
That said, libicu can do this for you. Open the transform demo; copy and paste your Spanish text into the “Input” box; enter
NFD; [:M:] remove; NFC
as “Compound 1” and click transform.
(With help from slide 9 of Unicode Transforms in ICU. Slides 29-30 show how to use the API.)
I definitely think you should look into the root of the problem. That is, look for a solution that will allow you to support characters encoded in Unicode or for the user's locale.
That being said, your problem is that you're dealing with multi-character strings. There is std::wstring but I'm not sure I'd use that. For one thing, wide characters aren't meant to handle variable width encodings. This hole goes deep, so I'll leave it at that.
Now, as for the rest of your code, it is error prone because you mix the looping logic with translation logic. Thus, at least two kinds of bugs can occur: translation bugs and looping bugs. Do use the STL, it can help you a lot with the looping part.
The following is a rough solution for replacing characters in a string.
main.cpp:
#include <iostream>
#include <string>
#include <iterator>
#include <algorithm>
#include "translate_characters.h"
using namespace std;
int main()
{
string text;
cin.unsetf(ios::skipws);
transform(istream_iterator<char>(cin), istream_iterator<char>(),
inserter(text, text.end()), translate_characters());
cout << text << endl;
return 0;
}
translate_characters.h:
#ifndef TRANSLATE_CHARACTERS_H
#define TRANSLATE_CHARACTERS_H
#include <functional>
#include <map>
class translate_characters : public std::unary_function<const char,char> {
public:
translate_characters();
char operator()(const char c);
private:
std::map<char, char> characters_map;
};
#endif // TRANSLATE_CHARACTERS_H
translate_characters.cpp:
#include "translate_characters.h"
using namespace std;
translate_characters::translate_characters()
{
characters_map.insert(make_pair('e', 'a'));
}
char translate_characters::operator()(const char c)
{
map<char, char>::const_iterator translation_pos(characters_map.find(c));
if( translation_pos == characters_map.end() )
return c;
return translation_pos->second;
}
You might want to check out the boost (http://www.boost.org/) library.
It has a regexp library, which you could use.
In addition it has a specific library that has some functions for string manipulation (link) including replace.
Try using std::wstring instead of std::string. UTF-16 should work (as opposed to ASCII).
I could not link the ICU libraries but I still think it's the best solution. As I need this program to be functional as soon as possible I made a little program (that I have to improve) and I'm going to use that. Thank you all for for suggestions and answers.
Here's the code I'm gonna use:
for (it= dictionary.begin(); it != dictionary.end(); it++)
{
strMine=(it->first);
found=toReplace.find(strMine);
while (found != std::string::npos)
{
strAux=(it->second);
toReplace.erase(found,2);
toReplace.insert(found,strAux);
found=toReplace.find(strMine,found+1);
}
}
I will change it next time I have to turn my program in for correction (in about 6 weeks).
If you can (if you're running Unix), I suggest using the tr facility for this: it's custom-built for this purpose. Remember, no code == no buggy code. :-)
Edit: Sorry, you're right, tr doesn't seem to work. How about sed? It's a pretty stupid script I've written, but it works for me.
#!/bin/sed -f
s/á/a/g;
s/é/e/g;
s/í/i/g;
s/ó/o/g;
s/ú/u/g;
s/ñ/n/g;
/// <summary>
///
/// Replace any accent and foreign character by their ASCII equivalent.
/// In other words, convert a string to an ASCII-complient string.
///
/// This also get rid of special hidden character, like EOF, NUL, TAB and other '\0', except \n\r
///
/// Tests with accents and foreign characters:
/// Before: "äæǽaeöœoeüueÄAeÜUeÖOeÀÁÂÃÄÅǺĀĂĄǍΑΆẢẠẦẪẨẬẰẮẴẲẶАAàáâãåǻāăąǎªαάảạầấẫẩậằắẵẳặаaБBбbÇĆĈĊČCçćĉċčcДDдdÐĎĐΔDjðďđδdjÈÉÊËĒĔĖĘĚΕΈẼẺẸỀẾỄỂỆЕЭEèéêëēĕėęěέεẽẻẹềếễểệеэeФFфfĜĞĠĢΓГҐGĝğġģγгґgĤĦHĥħhÌÍÎÏĨĪĬǏĮİΗΉΊΙΪỈỊИЫIìíîïĩīĭǐįıηήίιϊỉịиыїiĴJĵjĶΚКKķκкkĹĻĽĿŁΛЛLĺļľŀłλлlМMмmÑŃŅŇΝНNñńņňʼnνнnÒÓÔÕŌŎǑŐƠØǾΟΌΩΏỎỌỒỐỖỔỘỜỚỠỞỢОOòóôõōŏǒőơøǿºοόωώỏọồốỗổộờớỡởợоoПPпpŔŖŘΡРRŕŗřρрrŚŜŞȘŠΣСSśŝşșšſσςсsȚŢŤŦτТTțţťŧтtÙÚÛŨŪŬŮŰŲƯǓǕǗǙǛŨỦỤỪỨỮỬỰУUùúûũūŭůűųưǔǖǘǚǜυύϋủụừứữửựуuÝŸŶΥΎΫỲỸỶỴЙYýÿŷỳỹỷỵйyВVвvŴWŵwŹŻŽΖЗZźżžζзzÆǼAEßssIJIJijijŒOEƒf'ξksπpβvμmψpsЁYoёyoЄYeєyeЇYiЖZhжzhХKhхkhЦTsцtsЧChчchШShшshЩShchщshchЪъЬьЮYuюyuЯYaяya"
/// After: "aaeooeuueAAeUUeOOeAAAAAAAAAAAAAAAAAAAAAAAaaaaaaaaaaaaaaaaaaaaaaaBbCCCCCCccccccDdDDjddjEEEEEEEEEEEEEEEEEEeeeeeeeeeeeeeeeeeeFfGGGGGgggggHHhhIIIIIIIIIIIIIiiiiiiiiiiiiJJjjKKkkLLLLllllMmNNNNNnnnnnOOOOOOOOOOOOOOOOOOOOOOooooooooooooooooooooooPpRRRRrrrrSSSSSSssssssTTTTttttUUUUUUUUUUUUUUUUUUUUUUUUuuuuuuuuuuuuuuuuuuuuuuuYYYYYYYYyyyyyyyyVvWWwwZZZZzzzzAEssIJijOEf'kspvmpsYoyoYeyeYiZhzhKhkhTstsChchShshShchshchYuyuYaya"
///
/// Tests with invalid 'special hidden characters':
/// Before: "\0\0\000\0000Bj��rk�\'\"\\\0\a\b\f\n\r\t\v\u0020���oacu\'\\\'te�"
/// After: "00000Bjrk'\"\\\n\r oacu'\\'te"
///
/// </summary>
private string Normalize(string StringToClean)
{
string normalizedString = StringToClean.Normalize(NormalizationForm.FormD);
StringBuilder Buffer = new StringBuilder(StringToClean.Length);
for (int i = 0; i < normalizedString.Length; i++)
{
if (CharUnicodeInfo.GetUnicodeCategory(normalizedString[i]) != UnicodeCategory.NonSpacingMark)
{
Buffer.Append(normalizedString[i]);
}
}
string PreAsciiCompliant = Buffer.ToString().Normalize(NormalizationForm.FormC);
StringBuilder AsciiComplient = new StringBuilder(PreAsciiCompliant.Length);
foreach (char character in PreAsciiCompliant)
{
//Reject all special characters except \n\r (Carriage-Return and Line-Feed).
//Get rid of special hidden character, like EOF, NUL, TAB and other '\0'
if (((int)character >= 32 && (int)character < 127) || ((int)character == 10 || (int)character == 13))
{
AsciiComplient.Append(character);
}
}
return AsciiComplient.ToString().Trim(); // Remove spaces at start and end of string if any
}
Related
I am trying to print Unicode characters in C++. My Unicode characters are Old Turkic, I have the font. When I use a letter's code it gives me another characters. For example:
#include <iostream>
#include <string>
using namespace std;
int main()
{
string str = "\u10C00" // My character's unicode code.
cout << str << endl;
return 0;
}
This snipped gives an output of another letter with a 0 just after its end.
For example, it gives me this (lets assume that I want to print 'Ö' letter):
A0
But when I copied and pasted my actual letter to my source snippet, from character-map application in ubuntu, it gives me what I want. What is the problem here? I mean, I want use the character code way "\u10C00", but it doesn't work properly. I think this string is too long, so it uses the first 6 characters and pops out the 0 at the end. How can I fix this?
After escape /u must be exactly 4 hexadecimal characters. If you need more, you should use /U. The second variant takes 8 characters.
Example:
"\u00D6" // 'Ö' letter
"\u10C00" // incorrect escape code!
"\U00010C00" // your character
std::string does not really support unicode, use std::wstring instead.
but even std::wstring could have problems since it does not support all sizes.
an alternative would be to use some external string class such as Glib::ustring if you use gtkmm or QString in case of Qt.
Almost each GUI toolkit and other libraries provide it's own string class to handle unicode.
I have to following string which i get from share point :
\u002fsites\u002fblabla\u002fShared Documents\u002fkittens.xml
and i'm trying to convert it to :
/sites/blabla/Shared Documents/kittens.xml
I googled it and found that it is Unicode encoded, but i couldn't find anything that converts it, technically i can write a small function that converts all the "\u002f" to "/" but i don't think it is the right thing to do.
If any one can shed some light on this matter it would be very helpful.
Thanks
I found a library that solved my problem, and i want to share the solution in case someone else will stumble upon this question.
The library called ICU
And the use is very simple :
icu::UnicodeString converter = icu::UnicodeString(in.c_str(),in.length());
icu::UnicodeString newUstring = converter.unescape();
newUstring.toUTF8String(out);
when in and out are string.
I don't whether there's a ready function in C++ to do that or not , but you can do that by this algorithm :
1-find "\" character in your first string using find function in string class,change it to "/"
2- omit from this point to 4 nexts(incluse u002) and so on...
#include <iostream>
#include <string>
using namespace std;
string process(string str){
string result="";
for (int i=0;i<str.size();i++){
if(str[i]=='\\' ) {
result.push_back('//');
i+=5;
}
else
result.push_back(str[i]);
}
return result;
}
int main(){
string path;
getline(cin,path);
cout<<process(path)<<endl;
}
If the string is a string literal, the compiler should take
care of this translation; a sequence \uxxxx
is a "universal character name". If it's data you're reading
from an outside source, I'm not aware of any standard function
which would convert it; you'll have to write your own. (And
getting it right isn't that trivial, once you add all of the
necessary error checking.)
I have two words and both are of the type std::string and they are unicode words. they are the same, I mean when I write them to some file they both have the same representation. but when I call word1.compare(word2), I dont get the right result. why they are not the same?
or should I use another function instead of compare to compare two unicode strings?
thanks
ifstream myfile;
string term = "";
myfile.open("homograph.txt");
istream_iterator<string> i(myfile);
multiset<string> s(i, istream_iterator<string>());
for(multiset<string>::const_iterator i = s.begin(); i != s.end(); i = s.upper_bound(*i))
{
term = *i;
}
pugi::xml_document doc;
std::ifstream stream("words0.xml");
pugi::xml_parse_result result = doc.load(stream);
pugi::xml_node words = doc.child("Words");
for (pugi::xml_node_iterator it = words.begin(); it != words.end(); ++it)
{
std::string wordValue = as_utf8(it->child("WORDVALUE").child_value());
if(!wordValue.compare(term))
{
o << wordValue << endl;
}
}
the first word is "term" and the second word is wordValue;
the overload function of as_utf8() is :
std::string wordNet::as_utf8(const char* str)
{
return str;
}
In Unicode (and UTF-8 is Unicode), there is the problem of composition. A token like é can be represented by its own code point, or by the code point e followed by ´. It could be that one is encoded using precomposition (é) and the other using decomposition (e´). Both will usually be displayed the same way. To avoid the problem, one should normalize strings on one of these composition types.
Of course, there could be another problem, but this is one of the problems that can make equal looking strings not compare as equal. OTOH, if your text does not have any characters outside ASCII, this is hardly the problem.
The correct way to compare the strings is to normalize them first. You can do this in Python with the unicodedata module.
The Unicode Standard Technical Appendix #15 describes composition and normalization in detail.
Unicode is more complicated than you think. There are combining characters, invisible code points and what not. If two strings look the same when printed, it doesn't mean they are byte-to-byte identical.
To take all complications of Unicode into account, you need to use a Unicode-aware string library. One such library is ICU. The C++ standard library is most definitely not Unicode-aware. It probably can correctly count characters in a UTF-8 strings, but that's about it.
Try using std::wstring instead.
is there anyway to find out how many times a word repeated in a text .
the text is in character arrays (char[])
text = this is a book,and this book
is about book.
word = book
result = 3
Because this is clearly homework and not tagged as such, I'll give you a solution you clearly can't submit as your assignment because your teacher will know you got it on the internet.
There were no requirements such as ignoring punctuation, so I've allowed myself to write a version that only works for clearly separated words and thus inserted spaces in your sample text string.
#include <algorithm>
#include <iostream>
#include <iterator>
#include <sstream>
#include <string>
// Count clearly separated occurrences of `word` in `text`.
std::size_t count ( const std::string& text, const std::string& word )
{
std::istringstream input(text);
return (std::count(std::istream_iterator<std::string>(input),
std::istream_iterator<std::string>(), word));
}
int main ( int, char ** )
{
const char text[] = "this is a book , and this book is about book .";
const char word[] = "book";
std::cout << count(text, word) << std::endl;
}
Output:
3
You might want to implement this using std::string and here is a sample for you to start from.
The simplest way would be to loop through the string, counting the number of times that you find the word that you're looking for. I'm sure that you could use a function in <algorithm> to do it fairly easily, but if you have to ask whether it's possible to do this in C++, I wouldn't think that you're advanced enough to try using the algorithm library, and doing it yourself would be more instructional anyway.
I would suggest using std::string though if you're allowed to (since this question does sound like homework, which could carry additional restrictions). Using std::string is easier and less error-prone than char arrays. It can be done with both though.
It is possible.
You have an array of characters. Try to do the search on a piece of paper, character by character:
First character is a T. This is not a b, so it can't be the first character of "book"
Second character is a h, so again, it is not b...
[...]
The next character is a b... Oah, this could be it. Is the next character a o? YES!!! And then next another o???... etc. etc..
When you can do it this way, you will be able to use C++ to do it.
Remember that you can access the n-th character in an array by using the [] operator:
char c = array[5] ; // c is the 6th character in the array
Now, going toward the C++ way would be, at first, to use a std::string instead of an array of chars, and use the strings methods. Google for std::string methods, and I guess you should find somes that you could use...
So you should manage to write some code that will iterate each character until the end
I guess this should be more than enough.
The point of your homework (because everyone here knows this is a homework question) is more about searching for the solution than finding it: This is not rote learning.
I doubt anyone on Stack Overflow remembers the solution to this classical problem. But I guess most will know how to find one solution. You need to learn the "how to find" mindset, so get your compiler and try again...
P.S.: Of course, if you know little or nothing of C++, then you're screwed, and you could start by Googling some C++ Tutorials.
I have a task to implement "void makeAmbigram(char*)" that will print on screen ambigram of latin string or return something like 'ambigram not possible'. Guess it's just about checking if string contains only of SNOXZHI and printing string backwards. Or am I wrong ?
I'm a complete noob when dealing with cpp so that's what I've created :
#include <iostream>
using namespace std;
char[]words;
char[]reversed;
char[] ret_str(char* s)
{
if(*s != '\0')
ret_str(s+1);
return s;
}
void makeAmbigram(char* c)
{
/* finding chars XIHNOZS and printing ambigram */
}
int main()
{
cin>>words;
reversed = ret_str(words);
makeAmbigram(reversed);
return 0;
}
I can reverse string but how to check if my reversed string contains only needed chars ?
I've found some function but it's hard or even imposible to implement it for greater amount of chars : www.java2s.com/Code/C/String/Findcharacterinstringhowtousestrchr.htm
You need to allocate space in your arrays or use std::vector. The arrays word and reversed are just pointers and no space is allocated. The C++ language does not support dynamic arrays; however, the STL provides std::vector which dynamically allocates space as required.
Change:
char[]words;
char[]reversed;
To:
#define MAX_LETTERS 64
char words[MAX_LETTERS + 1]; // + 1 for terminating nul character ('\0')
char reversed[MAX_LETTERS + 1];
Or:
#include <string>
std::string words;
std::string reversed;
Or:
#include <vector>
std::vector<char> words;
std::vector<char> reversed;
As far as the ambigram rules go, you need to talk to your instructor. Also, if this is homework, add a tag indicating so.
Hint: The std::string data type has some reverse iterators which may be of use to you.
std::string has an entire family of member functions along the lines of find_first_of. You can pass in a string containing all the letters your ambigram test requires, and they'll find whether any of those letters are present in the source string.
The complete list of string functions is available here.
As for the definition of ambigrams, given the wiki page you've included in the question...you need to check if a letter is legible if viewed upside down, for eg. u/n, w/m, d/p, q/b and so on. There are of course more complex rules was well, for eg. 'ui' can resemble 'm' if viewed upside down.
However, if you're only required to check if your string contains only SNOXZHI, you can look into a regular expression (regex) for the same, and compare input string character-wise to your regex.