String replace on a very large file - regex

Question

I have a giant text file that is JSON. You can see it here: http://api.mtgdb.info/cards/. I have saved this JSON to a file called cards.json.
In cards.json, I need to escape every single quote ' with a backslash \.
So I need to replace ' with \'.
Usually this is trivial in any editor, however the file is too large. How can I escape all single quotes in this string?
What I've tried:
I tried using sed. My command was sed s/\'/\\\'/ cards.json > cards_cleaned.json. However the cards_cleaned.json file did not have any escaped ', it was just an exact copy of cards.json. Sed works when i do sed s/\'/foobar/ cards.json > cards_cleaned.json, so I'm assuming something is wrong with my escaping backslashes.
I tried using vim. I opened cards.json in vim $ vi cards.json. Then I tried a global string replace using :%s/'/\'/g. This did not change anything in the file.

Answer 1

While #anubhava's or #gboffi's answers works, they produces and INVALID JSON.
JSON allows only few characters after the backslash:
\"
\\
\/
\b
\f
\n
\r
\t
\u four-hex-digits
e.g. the part of the following original (correct) JSON
[
{
"description" : "Whenever a land enters the battlefield, Ankh of Mishra deals 2 damage to that land's controller.",
"rarity" : "Rare",
"name" : "Ankh of Mishra"
}
]
you want to get
[
{
"description" : "Whenever a land enters the battlefield, Ankh of Mishra deals 2 damage to that land\'s controller.",
"rarity" : "Rare",
"name" : "Ankh of Mishra"
}
]
#e.g. instead of the land's want land\'s
But this is an INVALID JSON.
So, if you (for some strange reason) want have the backslash, you need to use double \\, such:
[
{
"description" : "Whenever a land enters the battlefield, Ankh of Mishra deals 2 damage to that land\\'s controller.",
"rarity" : "Rare",
"name" : "Ankh of Mishra"
}
]
Solution (for both)
with perl
perl -pE "s/'/\\\'/g" < mtg_cards.json > cards.malformed.json
#changes "land's" to wrong "land\'s"
and
perl -pE "s/'/\\\\'/g" < mtg_cards.json > card_with_double_BS.json
#changes "land's" to "land\\s"
Ps: Because your file is only one long (30MB) line, the vim has some problems. You can pretty print (fold and indent) the JSON, before editing. Many tools here, i'm using the json_xs command from the JSON_XS perl package. After "prettyfying" you can use the vim safely.

Answer 2

You need to use double quotes in the shell to avoid quoting the single quote character, but the you have to be careful because the shell, for a double quoted string, use the backslash as a quoting character
$ echo "eoieriou'iouou'oiuiouiuo"|sed "s/'/\\'/g"
eoieriou'iouou'oiuiouiuo
and the command that sed is trying to execute is s/'/\'/g but sed quoting character is the backslash, so that you substitute each single quote with a single quote...
We have to quote the backslash also when it arrives to sed, so let's try
$ echo "eoieriou'iouou'oiuiouiuo"|sed "s/'/\\\\'/g" # Four (4) backslashes in a row
eoieriou\'iouou\'oiuiouiuo
$
That's OK, isn't it? because sed is instructed to do s/'/\\'/g so that the quoted character, from the POV of sed, is the backslash itself...
Please note that the quotes, single or double, are not special characters from the POV of sed, they're special only in the context of the shell.

Answer 3

In Vi you will need to escape the \ character.
Try using
:%s/'/\\'/g
For me it worked.
Test.txt
\'\'\' \'\'\'

Answer 4

You need to double escape the backelas, so use:
sed -i.bak "s/'/\\\\'/g" cards.json

Answer 5

You can use like this, in vim.
:%s/'/\\\'/g
In sed,
sed "s/'/\\\'/g" filename

Answer 6

Here is an awk version:
cat file
hi'more data here'
awk '{gsub(g,"\\"g)}1' g="'" file
hi\'more data here\'
Or if you need double backslash:
awk '{gsub(g,"\\\\"g)}1' g="'" file
hi\\'more data here\\'

Answer 7

sed "s/'/\\\\&/g" cards.json > cards_cleaned.json
no need of your first escaped in search pattern \'
you should surround by double quote (single if single quote was not the char to change) and escape the escape due to double quote used at shell level in this case