I'm trying to use Google's translate method from its Translation API as documented here, but for some reason the translations I get replace non-Latin characters with underscores.
For instance, with curl on the command-line:
$ curl -X POST 'https://translation.googleapis.com/language/translate/v2/?source=en&target=de&q=Practicing+diligently+each+day+means+inevitable+improvement.&key=MY_API_KEY'
{
"data": {
"translations": [
{
"translatedText": "T_glich flei_ig zu _ben, bedeutet unausweichliche Verbesserung."
}
]
}
}
Compare to the English-to-German result from translate.google.com:
Täglich fleißig zu üben, bedeutet unausweichliche Verbesserung.
It's especially bad when the target is a language like Japanese, which doesn't contain Latin characters:
$ curl -X POST 'https://translation.googleapis.com/language/translate/v2/?source=en&target=ja&q=Practicing+diligently+each+day+means+inevitable+improvement.&key=MY_API_KEY'
{
"data": {
"translations": [
{
"translatedText": "______________________________________________________"
}
]
}
}
Maybe this is a trial account limitation? Nothing I've seen in this docs would indicate this, however.
I believe it's a string-encoding issue.
I assume your HTTP request body is being sent using application/x-www-form-urlencoded - which does not support characters above 0x7F (128) as literal text, see here: application/x-www-form-urlencoded and charset="utf-8"?
I suggest:
POST with an explicit Content-Type: application/json header with the charset=utf-8 field set. (x-www-form-urlencoded does not support the charset field).
Ensure your terminal is using UTF-8
Also take a look using a tool like Wireshark, or create the request in JavaScript using fetch and use Chrome's Developer Tools' Network tab's "Copy as cURL (Bash)" command to get the terminal command to use.
Somewhat embarrassingly, this was actually just an issue with tmux, the terminal multiplexer I was using to read the output of every call I made to the Translation API, both with curl and with the printed output of the code I was writing.
As per this Ask Ubuntu answer to someone else's tmux question, this is fixable by explicitly telling tmux to launch with UTF-8 support, i.e., tmux -u.
Thanks both to Dai and Daniel for pointing to a potential terminal issue.
I just tried with the following request and it worked well:
curl -X POST "https://translation.googleapis.com/language/translate/v2?key=MY_API_KEY" \
-H "Content-Type: application/json" \
--data "{
'q': 'Practicing diligently each day means inevitable improvement.',
'source': 'en',
'target': 'de'
}"
Giving this output:
{
"data": {
"translations": [
{
"translatedText": "Täglich fleißig zu üben, bedeutet unausweichliche Verbesserung."
}
]
}
}
And for the Japanese output:
{
"data": {
"translations": [
{
"translatedText": "毎日熱心に練習することは避けられない改善を意味します。"
}
]
}
}
Hope it helps
When we make a query to Translate API
https://translation.googleapis.com/language/translate/v2?key=$API_KEY&q=hello&source=en&target=e
I only get 1 result in :
{
"data": {
"translations": [
{
"translatedText": "....."
}
]
}
}
Is it possible to get all variations (alternatives) of that word, not only 1 translation?
Microsoft Azure supports one. https://learn.microsoft.com/en-us/azure/cognitive-services/translator/reference/v3-0-dictionary-lookup .
For ex. https://api.cognitive.microsofttranslator.com/dictionary/lookup?api-version=3.0&from=en&to=es
[
{"Text":"hello"}
]
gives you a list of translations like this:
[
{
"normalizedSource": "hello",
"displaySource": "hello",
"translations": [
{
"normalizedTarget": "diga",
"displayTarget": "diga",
"posTag": "OTHER",
"confidence": 0.6909,
"prefixWord": "",
"backTranslations": [
{
"normalizedText": "hello",
"displayText": "hello",
"numExamples": 1,
"frequencyCount": 38
}
]
},
{
"normalizedTarget": "dime",
"displayTarget": "dime",
"posTag": "OTHER",
"confidence": 0.3091,
"prefixWord": "",
"backTranslations": [
{
"normalizedText": "tell me",
"displayText": "tell me",
"numExamples": 1,
"frequencyCount": 5847
},
{
"normalizedText": "hello",
"displayText": "hello",
"numExamples": 0,
"frequencyCount": 17
}
]
}
]
}
]
You can see 2 different translations in this case.
The Translation API service doesn't support the retrieval of multiple translations of a word, as mentioned in the FAQ Documentation:
Is it possible to get multiple translations of a word?
No. This feature is only available via the web interface at
translate.google.com
In case this feature doesn't cover your current needs, you can use the Send Feedback button, located at the lower left and upper right corners of the service public documentation, as well as take a look the Issue Tracker tool in order to raise a Translation API feature request and notify to Google about this desired functionality.
Approach mapping Wiktionary using POS tags, related terms and Google-translated word.
TL;DR
The question is titled 'get-multiple-variations-from-google-translate-api', but in short, you (still) currently can't do this by using Google's service alone (as of Sept. 2022). It seems most companies, such as Google, want to continue charging for this service. This answer provides an approach using a (free) service as a pivot to get the term, related terms, and their POS (Parts of Speech) e.g. noun, verb, etc. before translating those terms and then re-querying the service.
This alternative creates a small pipeline that queries Wiktionary before (on the source language), and after (on the translated terms target language) the translation (using Google).
The small pipeline is written in python and bash.
Rationale
We could get word senses, for each POS (Part of Speech) and corresponding synonyms, then translate for each word sense since Google only translates word to word, and then match word senses for the corresponding target language using a tool such as Wiktionary.
Wiktionary
Fortunately, someone has already created a python library to query Wiktionary for multiple languages.
Script to get definitions / synonyms from Wiktionary (using python):
(requires wiktionaryparser )
e.g. python -m pip install wiktionaryparser
import sys;
import json;
from wiktionaryparser import WiktionaryParser;
parser = WiktionaryParser()
# sys.argv[1] is a language e.g. 'english'
parser.set_default_language(sys.argv[1])
print(
json.dumps(
[
[
{
'pos': d.get('partOfSpeech'),
'text':d.get('text'),
'examples':[e for e in d.get('examples')][0] if d.get('examples') else [],
'related': d.get('relatedWords')
} for d in w.get('definitions')
] for w in parser.fetch(sys.argv[2])
],
indent=2
)
)
Google translate + Wiktionary
The bash script below gets Wiktionary definitions, splits on synonym lists and correlates translations based on POS (Part of Speech).
To be honest this script is a bit convoluted, it uses a lot of utils, but it works. It could be refactored into python like the wiktionary part by anyone wanting to make something a bit more robust.
This github post provided some of the below script that call the free Google translate api.
#!/bin/bash
sl=$1
tl=$2
wiki_sl=$3
wiki_tl=$4
string=$5
ua='Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36'
#echo "$string"
result="{\"${sl}\":[],\"${tl}\":[]}"
#set -x
while IFS= read line; do
# line could be better named 'synonym' here
pos="$(echo ${line} | jq -r ".pos")"
sl_result="$(echo $line | jq . -c)"
tl_result=""
opt_single="single?client=gtx&sl=${sl}&tl=${tl}&dt=t&q=${string//[[:blank:]]/+}"
full_url="http://translate.googleapis.com/translate_a/${opt_single}"
response=$(curl -sA "${ua}" "${full_url}")
tl_word="$(echo ${response} | jq -r '.[[0][0]][] | .[0:1][0]')"
echo "${tl_word}" | grep -q " " && continue 1
tl_result_new="$(python ./get_wiki.py "${wiki_tl}" "${tl_word}" | jq -r -c --arg POS "$pos" '.[][] | select(.pos==$POS)'),"
# making json
tl_result="[${tl_result_new}"
# iterate over synonyms
while IFS= read qry; do
opt_single="single?client=gtx&sl=${sl}&tl=${tl}&dt=t&q=${qry//[[:blank:]]/+}"
full_url="http://translate.googleapis.com/translate_a/${opt_single}"
response=$(curl -sA "${ua}" "${full_url}")
tl_word="$(echo ${response} | jq -r '.[[0][0]][] | .[0:1][0]')"
echo "${tl_word}" | grep -q " " && continue 1
tl_result_new="$(python ./get_wiki.py "${wiki_tl}" "${tl_word}" | jq -r -c --arg POS "$pos" '.[][] | select(.pos==$POS)'),"
# adding to json
tl_result="${tl_result},${tl_result_new}"
done< <(echo "${line}" | jq -c -r ' .related[].words[]' | \
sed -e 's/.*://;s/"//g;s/^ *//g;s/ *$//g' | tr ',' '\n')
tl_result="$(echo "${tl_result_new}" | sed 's/,$//g')"
[ -z "${tl_result}" ] && tl_result=null
[ -z "${sl_result}" ] && sl_result=null
result="{\"${sl}\":${sl_result},\"${tl}\":${tl_result}}"
echo "$result" | jq "."
done< <(python ./get_wiki.py "$wiki_sl" "$string" | \
jq -c -r '.[][]|select(.related[].relationshipType=="synonyms")') 2> /dev/null | jq -c '[.]'
How to use:
The first 2 arguments used are for google (source language, and target language in that order which are two-letter codes.
The second 2 arguments used are for Wiktionary (source language, a full word - e.g. 'English', 'French', etc.)
The final (fifth) argument is the single word to be translated.
./translate.sh en pt english portuguese help
In fact, the python 'wiktionaryparser' lib occasionally breaks and can throw an error, due to the fact that it is a webscraping library, which is why I add 2> /dev/null to silence stderr on output.
./translate.sh en pt english portuguese help 2> /dev/null
This script isn't perfect, but it is a starting point and a proof-of-concept to show you this is possible using a free tool such as wiktionary.
English to Portuguese
$ ./translate.sh en pt english portuguese help 2> /dev/null
Output:
[
{
"en": {
"pos": "noun",
"text": [
"help (usually uncountable, plural helps)",
"(uncountable) Action given to provide assistance; aid.",
"(usually uncountable) Something or someone which provides assistance with a task.",
"Documentation provided with computer software, etc. and accessed using the computer.",
"(usually uncountable) One or more people employed to help in the maintenance of a house or the operation of a farm or enterprise.",
"(uncountable) Correction of deficits, as by psychological counseling or medication or social support or remedial training."
],
"examples": "I need some help with my homework.",
"related": [
{
"relationshipType": "synonyms",
"words": [
"(action given to provide assistance): aid, assistance"
]
}
]
},
"pt": {
"pos": "noun",
"text": [
"assistência f (plural assistências)",
"assistance, aid, help",
"protection"
],
"examples": [],
"related": [
{
"relationshipType": "related terms",
"words": [
"assistir"
]
}
]
}
}
]
[
{
"en": {
"pos": "verb",
"text": [
"help (third-person singular simple present helps, present participle helping, simple past helped or (archaic) holp, past participle helped or (archaic) holpen)",
"(transitive) To provide assistance to (someone or something).",
"(transitive) To assist (a person) in getting something, especially food or drink at table; used with to.",
"(transitive) To contribute in some way to.",
"(intransitive) To provide assistance.",
"(transitive) To avoid; to prevent; to refrain from; to restrain (oneself). Usually used in nonassertive contexts with can."
],
"examples": "Risk is everywhere. […] For each one there is a frighteningly precise measurement of just how likely it is to jump from the shadows and get you. “The Norm Chronicles” […] aims to help data-phobes find their way through this blizzard of risks.",
"related": [
{
"relationshipType": "synonyms",
"words": [
"(provide assistance to): aid, assist, come to the aid of, help out; See also Thesaurus:help",
"(contribute in some way to): contribute to",
"(provide assistance): assist; See also Thesaurus:assist"
]
}
]
},
"pt": {
"pos": "verb",
"text": [
"ajudar (first-person singular present indicative ajudo, past participle ajudado)",
"to help, aid; to assist"
],
"examples": "Ajude-me! ― Help me!",
"related": [
{
"relationshipType": "related terms",
"words": [
"ajuda",
"ajudante"
]
}
]
}
}
]
English to Latin
$ ./translate.sh en la english latin body | jq '.'
[
{
"en": {
"pos": "noun",
"text": [
"body (countable and uncountable, plural bodies)",
"Physical frame.",
"Main section.",
"Coherent group.",
"Material entity.",
"(printing) The shank of a type, or the depth of the shank (by which the size is indicated).",
"(geometry) A three-dimensional object, such as a cube or cone."
],
"examples": "I saw them walking from a distance, their bodies strangely angular in the dawn light.",
"related": [
{
"relationshipType": "synonyms",
"words": [
"See also Thesaurus:body",
"See also Thesaurus:corpse"
]
}
]
},
"la": {
"pos": "noun",
"text": [
"cadāver n (genitive cadāveris); third declension",
"A corpse, cadaver, carcass"
],
"examples": [],
"related": []
}
}
]
When it doesn't work
Sometimes there is no output at all.
Shortcomings of this approach, and going further
Despite a lot of words being on Wiktionary, and a lot of synonyms being present, they are not always inside the 'related' field, sometimes synonyms are in the 'text' field, which gives word senses. I suspect that the partial information wiktionaryparser provides is the same on the Wiktionary site.
One could use any dictionary tool, or online thesaurus, such as wordnet, to first get possible POS tags and a word's synsets, or query a fasttext model to get a word's nearest neighbors, then filter only words that are nearest neighbors from the 'text' field in wiktionary.
I am trying to follow the example on google speech api found here
https://cloud.google.com/speech/docs/getting-started
1) I created the follow json request file
{
'config': {
'encoding':'FLAC',
'sampleRate': 16000,
'languageCode': 'en-US'
},
'audio': {
'uri':'gs://cloud-samples-tests/speech/brooklyn.flac'
}
}
2) Authenticate to my service account
gcloud auth activate-service-account --key-file=service-account-key-file
3) Obtain my authorization token successfully
gcloud auth print-access-token
access_token
4) Then use the following curl command
curl -s -k -H "Content-Type: application/json" \
-H "Authorization: Bearer access_token" \
https://speech.googleapis.com/v1beta1/speech:syncrecognize \
-d #sync-request.json
But I keep getting the following response
{
"error": {
"code": 400,
"message": "Invalid recognition 'config': bad encoding..",
"status": "INVALID_ARGUMENT"
}
}
Do I need access permissions for the uri gs://cloud-samples-tests/speech/brooklyn.flac? Is that what the problem is?
Thanks in advance..
In my opinion, it is a file format issue.
You have to send WAV file instead of FLAC ...
[ FLAC and MP3 format are not supported <=> need a file conversion (representing cost) on the server side ]
Convert your audio file to WAV (using ffmpeg or avconv), then retry.
You may also take a look here (to see a working example)
For me, the solution was to remove the space between "-d #",
so change "-d #sync-request.json" to "-d#sync-request.json".
I got help here: https://groups.google.com/forum/#!topic/cloud-speech-discuss/bL_N5aJDG5A. Apparently the file was being read and processed, but the parms were going to the "curl.exe" instead of being passed to the URL.
I understand this is quite late for an answer. However, it might help others so putting in your error.
The config that you are passing is actually incorrect. The attributes should be like:
{
"config": {
"encoding": "LINEAR16",
"sampleRateHertz": 16000,
"languageCode": "en-US",
"maxAlternatives": 1,
"profanityFilter": true,
"enableWordTimeOffsets": false
},
"uri": {
"content":"<your uri>"
}
}
I build a VirtualBox VM using Packer and I would like to set some VM meta data (e.g. description, version) using the export_opts parameter. The docs say
export_opts (array of strings) - Additional options to pass to the VBoxManage export. This can be useful for passing product information to include in the resulting appliance file.
I am trying to do this in a bash script calling packer:
desc=' ... some ...'
desc+=' ... multiline ...'
desc+=' ... description ...'
# this is actually done using printf, shortened for clarity
export_opts='[ "version", "0.2.0", "description", "${desc}" ]'
# the assembled string looks OK
echo "export_opts: ${export_opts}"
packer build \
... (more options) ...
-var "export_opts=${export_opts}" \
... (more options) ...
<packer configuration file>
I also tried --version instead of version and putting version and the value into the same string, but none of this works; once exported and re-imported, the VM description is empty.
Does anyone have some working sample code or can help me out with what I'm doing wrong ?
Thank you very much.
Update:
Following Anthony Staunton's approach, I figured out that adding
"export_opts": [ "--vsys", "0", "--version", "0.2.0", "--description", "some test description" ],
to the Packer JSON file does work; passing the same string as --var to Packer does not work.
Fixed the problem at long last, updated the packer documentation with the example below, pull requests pending:
Packer JSON configuration file example:
{
"type": "virtualbox-ovf",
"export_opts":
[
"--manifest",
"--vsys", "0",
"--description", "{{user `vm_description`}}",
"--version", "{{user `vm_version`}}"
],
"format": "ova",
}
A VirtualBox VM description may contain arbitrary strings; the GUI interprets HTML formatting. However, the JSON format does not allow arbitrary newlines within a value. Add a multi-line description by preparing the string in the shell before the packer call like this (shell > continuation character snipped for easier copy & paste):
vm_description='some
multiline
description'
vm_version='0.2.0'
packer build \
-var "vm_description=${vm_description}" \
-var "vm_version=${vm_version}" \
"packer_conf.json"
You may have to specify the data as
in your packer json file
"export_opts": [ "--vsys 0 --version \"0.2.0\"", "{{.Name}} --description \"${desc}\" " ],
I am using ExtJS 4.1 and I am deploying my simple HelloExt program on GlassFish V3.1.
I am trying to create a build from Sencha SDK.
I have used the following two commands...
C:\>sencha create jsb -a http://localhost:8080/HelloExt/index.jsp -p appname.jsb
3 -v
C:\>sencha build -p appname.jsb3 -v -d .
As per the documentation, it will create app-all.js file. But where does it create the file?
How can I know IF build are created successfully or not?
Where are the generated JS files?
I made a search but I can not found anything like app-all.js.
For more information:
I am using JDK 1.6.0_12 and GlassFish V3.1 application server.
Here are the edited content of the question ....
And when I am trying to use the sencha SDK, It generates a .dpf file into the class path.
The contents of the .dpf file as as below ...
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-web-app PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN" "http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app error-url="">
<context-root>/HelloExt</context-root>
<class-loader delegate="true"/>
<jsp-config>
<property name="keepgenerated" value="true">
<description>Keep a copy of the generated servlet class' java code.</description>
</property>
</jsp-config>
</glassfish-web-app>
Can anyone tell me Why here it generated .DPF file ? Why its not generating the app-all.js file ?
Try running the command from inside the app root directory and then using a relative path:
0) open cmd window
1) run in cmd window: "cd C:\[webserver_webapp_root]\[app_name]"
In other words change the cmd directory to the app root. Fill in the bracketed text above with the correct paths.
2) run in cmd window: "sencha create jsb -a index.html -p app.jsb3 -v"
The app.jsb3 should be created in your app's root directory (C:\[webserver_webapp_root]\[app_name]). Open it up and make sure it contains all of your app classes, it should look something like this:
{
"projectName": "Project Name",
"licenseText": "Copyright(c) 2012 Company Name",
"builds": [
{
"name": "All Classes",
"target": "all-classes.js",
"options": {
"debug": true
},
"files": [
{
"clsName": "YourApp.view.Viewport",
"name": "Viewport.js",
"path": "app/view/"
},
// plus ALOT more classes...
]
},
{
"name": "Application - Production",
"target": "app-all.js",
"compress": true,
"files": [
{
"path": "",
"name": "all-classes.js"
},
{
"path": "",
"name": "app.js"
}
]
}
],
"resources": []
}
If everything looks fine then you can go onto the next step, if not then there is something wrong with your app directory structure and you need to fix it per Sencha recommended ExtJS application architecture.
You can also use any error messages to help identify the problem.
3) update placeholders ("Project Name", etc) at the top of app.jsb3
4) run in cmd window: "sencha build -p app.jsb3 -d . -v"
The app-all.js file should also be created in the app's root directory. If the cmd window doesn't give any errors before it says "Done Building!" then you are all done. You can now change your index.html script link to point to app-all.js instead of app.js.
If there are errors then you have to fix those and run this again.
Other things you can try:
In response to your last comment, your -p switch parameter should be a jsb3 file not jsb.
Make sure that the web server is running and that your app runs without any errors before you try to use the SDK Tools.
Then try these:
C:\Projects\HelloExt\build\web>sencha create jsb -a index.jsp -p HelloExt.jsb3 -v
C:\Projects\HelloExt>sencha create jsb -a index.jsp -p HelloExt.jsb3 -v
C:\>sencha create jsb -a [actual IP address]:8080/HelloExt/index.jsp -p HelloExt.jsb3 -v
Fill in your actual IP address where the brackets are (not localhost).
This should produce the jsb3 file shown in #2 above then you can move on to step #3 above.