Replace special characters to make JSON API output valid - regex

Question

I'm working in my R script with the Twitter REST API 1.1 (user_timeline.json). I collect a large amount of tweets.
Unfortunately, the texts contain a lot of special characters like \n, ^ or single \. So far, I was able to replace them with str_replace_all or gsub before importing them via fromJSON (jsonlite package):
correctJSON <- function(string) {
string <- str_replace_all(string, pattern = perl('\\\\(?![tn"])'), replacement = " ")
string <- str_replace_all(string, pattern = "\n", replacement = " ")
string <- str_replace_all(string, pattern = "\r", replacement = " ")
string <- str_replace_all(string, pattern = "\\^", replacement = " ")
return(string)
}
Now I have a string with special characters like \xed or \xa0. When trying to import it (via fromJSON(correctJSON(string))), I get as an error of the correctJSON function:
Fehler in parseJSON(txt) : lexical error: invalid bytes in UTF8 string.
uch sind.Mutig von bd. Seiten�������������������������������
(right here) ------^
The tweet containing the problematic characters is AFAICS:
[{\"created_at\":\"Fri Feb 07 18:35:02 +0000 2014\",\"id\":431858659656990721,\"id_str\":\"431858659656990721\",\"text\":\"RT #FHubersr: #peteraltmaier //die Schwarz-Grünen werden zeigen, daß sich Ökologie und Ökonomie vertragen und kein Widerspruch sind.Mutig v…\",\"source\":\"Twitter for iPhone\",\"truncated\":false,\"in_reply_to_status_id\":null,\"in_reply_to_status_id_str\":null,\"in_reply_to_user_id\":null,\"in_reply_to_user_id_str\":null,\"in_reply_to_screen_name\":null,\"user\":{\"id\":378693834,\"id_str\":\"378693834\"},\"geo\":null,\"coordinates\":null,\"place\":null,\"contributors\":null,\"retweeted_status\":{\"created_at\":\"Fri Feb 07 18:32:30 +0000 2014\",\"id\":431858022366064640,\"id_str\":\"431858022366064640\",\"text\":\"#peteraltmaier //die Schwarz-Grünen werden zeigen, daß sich Ökologie und Ökonomie vertragen und kein Widerspruch sind.Mutig von bd. Seiten\xed\xa0\xbd\xed\xb1\x8d\xed\xa0\xbd\xed\xb8\x8e\",\"source\":\"Twitter for iPhone\",\"truncated\":false,\"in_reply_to_status_id\":431845492579123201,\"in_reply_to_status_id_str\":\"431845492579123201\",\"in_reply_to_user_id\":378693834,\"in_reply_to_user_id_str\":\"378693834\",\"in_reply_to_screen_name\":\"peteraltmaier\",\"user\":{\"id\":2172292811,\"id_str\":\"2172292811\"},\"geo\":null,\"coordinates\":null,\"place\":null,\"contributors\":null,\"retweet_count\":3,\"favorite_count\":4,\"favorited\":false,\"retweeted\":false,\"lang\":\"de\"},\"retweet_count\":3,\"favorite_count\":0,\"favorited\":false,\"retweeted\":false,\"lang\":\"de\"}]
I already tried a lot of things but even after reading some threads here I cannot come up with a solution which can replace all problematic special characters.
Note: It's quite funny that when I want to import the single tweet via fromJSON, I do not get an error. But as soon as I import the correctJSON-string, it throws the error. But I need correctJSON because of the many \n appearances...
PS: I only pasted the problematic tweet. Here you can see the whole output of my API call also containing this one: https://p.mehl.mx/?53c04753c247a48a#5w+HtSCYpcjRwSk0PdsP3P1w3u+Z22/f6GKMJRoW//8=
Thanks for help!

Answer 1

Okay, I found a possible answer myself which works for the first 5000 tweets I gathered so far:
correctJSON <- function(string) {
string <- str_replace_all(string, pattern = "[^[:print:]]", replacement = " ")
string <- str_replace_all(string, pattern = perl('\\\\(?![tn"])'), replacement = " ")
return(string)
}
The regex [^[:print:]] is suitable for special characters like \xed, \n and maybe also \U..... Only for single \ you'll need the second (perl) regex.
So it works for now, hopefully also for the many upcoming tweets I'll import. I'll edit if something unexpected happens.