I want to upload dictionary files.
CREATE TEXT SEARCH DICTIONARY ispell (
TEMPLATE = ispell,
DictFile = bulpo,
AffFile = bulpo
);
ERROR: could not open dictionary file "/share/tsearch_data/bulpo.dict": No such file or directory
I try to upload it via cloud shell but result is same.
hey #asapokL you cannot upload files Cloud SQL (rather than database file for migration purposes) as you don't have access to the underlying compute engine instance because Cloud SQL is fully managed solution by Google Cloud. in your case you will need to upload your file (.dict) to the directory tsearch_data directory is in SHAREDIR. I would suggest if you want to accomplish this use case is to create a compute engine then install posgresql on it. but remember you will have to handle all operations on this instance (backups, rollouts, performence, OS ...etc)
Related
My requirement is to upload the PBIX file stored in Azure storage container to Power BI Service without downloading it to local drive as I have to use the PowerShell script in Runbook Automation
Normally we can upload the PBIX file by giving local path like below
$pbixFilePath = "C:\PBIXFileLocation\Test.pbix"
$import = New-PowerBIReport -Path $pbixFilePath -Workspace $workspace -ConflictAction CreateOrOverwrite
$import | Select-Object *
But now which path I have to use if the PBIX file is stored in Azure storage container and how the PowerShell script can be created? Is it possible?
Tried to list the blobs in the container with the Get-AzStorageBlob cmdlet and passed it as a path in above script and ended up with this error:
If possible please help me with a sample PowerShell script to achieve the above requirement
Thanks in Advance!
Issue can be resolved by following my similar post in Azure platform
AnuragSingh-MSFT is a gem explained me clearly and resolved the issue
A basic understanding of Azure Automation runbook execution should help clarify this doubt. When runbooks are designed to authenticate and run against resources in Azure, they run in an Azure sandbox. Azure Automation assigns a worker to run each job during runbook execution in the sandbox. Please see this link for more details - Runbook execution environment These sandboxes are isolated environment with access to only some of the location/path/directories.
The following section should help answer the question - ... which path I have to use if the PBIX file is stored in Azure storage container and how the PowerShell script can be created?
The script snippet provided by Manu above would download the blob content in the same directory inside sandbox from where script is running. You can access this path inside the script using "." --> for example, if the blob that you are downloading is named testBlob, it will be available in location .\testBlob. ("." stands for current directory).
Therefore, the pbixFilePath can be initialized as $pbixFilePath = ".\Test.pbix"
Another option is to use $env:temp as mentioned in the question. It is one of the environments variable available on local machine (on your workstation) which generally resolves to C:\Users<username>\AppData\Local\Temp
In Azure Automation sandbox environment, this variable resolves to C:\Users\Client\Temp
Therefore, you could download the blob content using the following line:
Get-AzStorageBlobContent -Blob $blob -Container $ContainerName -Context $Ctx -Destination $env:temp #Destination parameter sets the target folder. By default it is local directory (.)
In this case, you would initialize pbixFilePath as $pbixFilePath = $env:temp+"\Test.pbix"
Either case is fine as long as the Automation limits are not exceeded.
I'm trying to connect to the HDFS from the ADF. I created a folder and sample file (orc format) and put it in the newly created folder.
Then in ADF I created successfully linked service for HDFS using my Windows credentials (the same user which was used for creating sample file):
But when trying to browse the data through dataset:
I'm getting an error: The response content from the data store is not expected, and cannot be parsed.:
Is there something I'm doing wrongly or it is kind of permissions issue?
Please advise
This appears to be a generic issue, you need to point to a file with appropriate extension rather than a folder itself. Also make sure you are using a supported data store activity.
You can follow this official MS doc to use HDFS server with Azure Data Factory
I need to open a .xlsx file from my bucket on Google Cloud Storage, the problem is I get :FileNotFoundError at /api/ficha-excel
[Errno 2] No such file or directory: 'ficha.xlsx'
These are the settings from my bucket.
UPLOAD_ROOT = 'reportes/'
MEDIA_ROOT = 'reportes'
These are the route bucket/reportes/ficha.xlsx
This is the code of my get function:
directorio = FileSystemStorage("/reportes").base_location
os.makedirs(directorio, exist_ok=True)
# read
print("Directorios: ", directorio)
plantilla_excel = openpyxl.load_workbook(f"{directorio}/ficha.xlsx")
print(plantilla_excel.sheetnames)
currentSheet = plantilla_excel['Hoja1']
print(currentSheet['A5'].value)
What is the problem with the path? I can't figure out.
The below solution doesn’t use Django FileStorage/Storage classes. It opens a .xlsx file from the Cloud Storage bucket on Google Storage using openpyxl.
Summary :
I uploaded the Excel file on GCS, read the Blob data with openpyxl via BytesIO and saved the data in the workbook using the .save() method.
Steps to Follow :
Create a Google Cloud Storage bucket. Choose a globally unique name for it. Keep with the defaults and finally enter Create.
Choose an Excel file from your local system and upload it in the bucket using the “Upload files” option.
Once you have the excel file in your bucket, follow the steps below :
Go to Google Cloud Platform and create a service account (API). Click
Navigation Menu> APIs & Services> Credentials to go to the screen.
Then click Manage Service Accounts.
On the next screen, click Create Service Account.
Enter the details of the service account for each item.
In the next section, you will create a role for Cloud Storage. Choose
Storage Admin (full permission).
Click the service account you created, click Add Key in the Keys
field, and select Create New Key.
Select JSON as the key type and "create" it. Since the JSON file is
downloaded in the local storage, use the JSON file in the next item
and operate Cloud Storage from Python.
We will install the libraries required for this project in Cloud
Shell First, install the Google Cloud Storage library with pip
install to access Cloud Storage:
pip install google-cloud-storage
Install openpyxl using :
pip install openpyxl
Create a folder (excel) with the name of your choice in your Cloud editor.
Create files within it :
main.py
JSON key file (the one that got downloaded in local storage, copy that
file into this folder)
excel
main.py
●●●●●●●●●●.json
Write the below lines of code in main.py file :
from google.cloud import storage
import openpyxl
import io
#Create a client instance for google cloud storage
client = storage.Client.from_service_account_json('●●●●●●●●●●.json') //The path to your JSON key file which is now
#Get an instance of a bucket
bucket = client.bucket(‘bucket_name’) //only the bucketname will do, full path not necessary.
##Get a blob instance of a file
blob = bucket.blob(‘test.xlsx') // test.xlsx is the excel file I uploaded in the bucket already.
buffer = io.BytesIO()
blob.download_to_file(buffer)
wb = openpyxl.load_workbook(buffer)
wb.save('./retest.xlsx')
You will see a file ‘retest.xlsx’ getting created at the same folder in Cloud Editor.
New to GCP. Trying to load a saved model file into an AI Platform notebook. Tried several approaches without success.
Most obvious approach seemed to be to set the value of a variable to the path copied from storage:
model_path = "gs://<my-bucket>/models/3B/export/1600635833/saved_model.pb"
Results: OSError: SavedModel file does not exist at: (the above path)
I know I can connect to the bucket and retrieve contents because I downloaded a csv file from the bucket and printed out the contents.
OSError to me sounds like you are trying to access GCS bucket with a regular file system which do not support looking at GCS. (Example: Python open() function)
To access files in GCS I recommend you use the Client Libraries. https://cloud.google.com/storage/docs/reference/libraries
Another option for testing is to try to connect to SSH and use gsutil command.
Note: I assume <my-bucket> was edited to replace your real GCS bucket name.
According to the GCP documentation enter here , you are able to access Cloud Storage. This page will guide to using Cloud Storage with AI Platform Training.
Hi I am trying out opencv in AWS lambda. I want to save a SVM model in txt file so that I can load it again. Is it possible to save it in tmp directory and load it from there whenever I need it or will I have to use s3?
I am using python and trying to do something like this:
# saving the model
svm.save("/tmp/svm.dat")
# Loading the model
svm = cv2.ml.SVM_load("/tmp/svm.dat")
Its not possible as Lambda execution environment is distributed and therefore the same function might run on several different instances.
The alternative is to save your svm.dat to S3 and then download it every time you start your lambda function.