I was using google cloud platform to host a ckan based website. The website had a file library with about 5 gigs of documents. Our project got put on hold and I removed billing on the website about 8 months ago. Now we are trying to migrate the data, but when I look at the project in Google Cloud Platform, there are no compute instances, buckets or files under the various storage modules. I cannot find the 5gigs of files we uploaded and filed by various categories on the ckan website. What could have happened to them? I'm not very experience with this platform and a bit confused. Is there any way for me to recover my data?
As the GCP public documentation indicates, if your billing account remains invalid for a protracted period, some resources might be removed from the projects associated with your account. For example, if you use Google Cloud, your Compute Engine resources might be removed.
Removed resources are not recoverable, the best option you had was to review your case with tech support and have a more specific answer, I noticed that there isn't any way to recover resources 30 days later it had been removed, this is also described in Data deletion term, including existing copies.
Related
I have several projects in GCP which are publically accessible and will become commercial services. Since I found myself with many projects, I thought that I would try out the Google Cloud organization feature. However, I cannot complete the creation due to odd requirements.
Google puts forward a "Checklist" to set up Google Cloud for the organization. If I go under IAM & Admin -> Identity & Organization, the only option there is to go to the checklist to complete it. However, the checklist includes mandatory steps like creating VPCs and a certain Resource hierarchy, which are things that I don't need/want. I am the only developer and don't want to pay more for services that I don't use. I cannot remove these options. As such, the organization seems to be "unconfigured" from the point of view of Google.
At the same time, I'm wondering if it really changes anything and if there is any benefit for me to use an organization.
Edit: I am not following any documentation per se, as there is a quite detailed interactive "tutorial" which takes you through all the configuration steps directly on GCP. The problem is precisely that I cannot chose an option not in the tutorial for some steps. To give you an idea, see this blog post, specifically Step 5.
I am new to cloud and still learning GCP, I exhausted almost all my free credit for GCP within 2 months while learning different modules.
GCP is great and provides a lot of things to ease the development and maintenance process.
But I realized using different modules cost me a lot.
So I was wondering if I could have a big VM box, install MySQL, Docker, and Java and React required components by myself, I can achieve pretty much what I want without using extra modules.
Am I right?
Can I use the same VM to host multiple sites by changing ports of API, or do I need to have different boxes for that?
Your question is out of GCP domain but about IT architecture. You can create a big VM with all installed on it. But you have to manage it by yourselves and the scalability is hard.
You can also have 1 VM per website, but the management cost is higher (patching and updgrade)! However you can scale with a better granularity (per website).
The standard pattern today is to explode your monolith server into dedicated services. The database on a specific server, the docker and Java in another one, and the react in a static component (like Google Cloud Platform).
If you want to use VM, you can use GKE and you containerize your application. It's far more easier to maintain your VM with an automatic tool like K8S.
The ultimate step, is to use serverless and/or full managed solution. Cloud SQL for your database, GCS for your static content, and App Engine or Cloud Run for your backend. Like this, you pay as you use and if you website is not very used, you won't be charged on it (except for the database).
I am building an OCR based solution to extract information from certain financial documents.
As per the regulation in my country (India), this data cannot leave India.
Is it possible to find the region where Google Cloud Vision servers are located?
Alternately, is it possible to restrict the serving region from the GCP console?
This is what I have tried:
I went through GCP Data Usage FAQ: https://cloud.google.com/vision/docs/data-usage
GCP Terms of Service:
https://cloud.google.com/terms/
(Look at point 1.4 Data Location on this page)
Talking to the GCP Sales rep. He did not know the answer.
I know that I can talk to Google support but that requires $100 to activate, which is a lot for for me.
Any help would be appreciated. I went through the documentation for Rekognition as well but it seems to send some data outside for training so not considering it at the moment.
PS - Edited to make things I have tried clearer.
For anyone looking at this topic recently, Google Vision has introduced multi-region support in December 2019, as can be see in their release notes.
Currently Google Vision supports 2 different processing regions: eu and us, and they say that using a specific endpoint guarantees that processing will only take place in the chosen territory.
The documentation for regionalization mentions that you can simply replace the default API endpoint vision.googleapis.com with either of the regional ones:
eu-vision.googleapis.com
us-vision.googleapis.com
The vision client libraries offer options for selecting the endpoint as well, and the documentation gives code samples.
For example, here is how you would do it in Python:
from google.cloud import vision
client_options = {'api_endpoint': 'eu-vision.googleapis.com'}
client = vision.ImageAnnotatorClient(client_options=client_options)
As pointed out by #Tedinoz in a comment above, the answer can be found here: https://groups.google.com/forum/#!topic/cloud-vision-discuss/at43gnChLNY
To summarise:
1. Google stores images uploaded to Cloud Vision only in memory
2. Data is not restricted to a particular region (as of Dec 6, 2018)
3. They might add data residency features in Q1, 2019.
i am trying to create another agent in my dialogflow account but i can't because my gcp project limit is reached so i deleted the other useless agent but still the same issue now i tried deleting some project from gcp but still same error persist. how can i increase the quota of my dialogflow agent ?
Thank you
regards saif
Google Cloud Projects that are deleted take 30 days to completely delete. In the meantime, they are counted against your quota.
The limit is in place to make sure people don't abuse the resource or that a wayward process doesn't suddenly create tons of projects. They are reasonable about allocating more.
As the orange box indicates, you can request more projects through that link, which probably goes to https://support.google.com/code/contact/project_quota_increase. In it, be reasonable in your request, and mention that you're experimenting, that this is why you've used up all the projects initially allocated, and that you've already deleted some, but need more to continue your work in the meantime.
While use of the Dialogflow Standard Edition is free, there are limits on the amount of requests that you can make. You must bay Google Cloud Platform resources.
I am using the amazon services S3 and CloudFront for a web application and I would like to have various statistics about accessing the data that I am providing through the logs of those services (there is logging activated in both services).
I did a bit of googling and the only thing I could find is how to manage my S3 storage. I also noticed that newrelic offers monitoring for many amazon services but not for those 2.
Is there something that you use? A service that could read my logs periodically and provide me with some nice analytics that would make developers and managers happy?
e.g.
I am trying to avoid writing my own log parsers.
I believe Piwik supports the Amazon S3 log format. Take a look at their demo site to see some example reports.
Well, this may not be what you expect but I use qloudstat for my cloudfront distributions.
The $5 plan covers my needs thats less than a burrito here where I live.
Best regards.
Well, we have a SaaS product Cloudlytics which offers you many reports including, Geo, IP tracking, SPAM, CloudFront cost analysis. You can try it for free for upto 25 MB of logs.
I might be answering this very late. But I have worked on a golang library that can run analysis of CDN and S3 usages and store them in a backend of your choice varying from influxdb, MongoDB or Cassandra for later time series evaluations. The project is hosted at http://github.com/meson10/cdnlysis
See if this fits.
Popular 3rd party analytics packages include S3stat, Cloudlytics and Qloudstat. They all run around $10/month for low traffic sites.
Several stand-alone analytics packages support Amazon's logfile format if you want to download logs each night and feed them in directly. Others might need pre-processing to transform to Combined Logfile Format (CLF) first.
I've written about how to do that here:
https://www.expatsoftware.com/articles/2007/11/roll-your-own-web-stats-for-amazon-s3.html