I've been recently added to a new GCP project which has litterally tons and tons of pods, workloads and bases.
I want to visualize all of it in a schema or model.
Is there any tool or plugin that i can use to modelize the project ?
Probably the best mechanism would be to use Cloud Console and view the project's resources through the various pages built in to the console.
Google provides very many APIs (services) and these may contain multiple resource (types) and, as you've seen, there can be many instances of the resources.
I think anything that enumerate all a project's resources could be somewhat overwhelming whereas Console provides structure.
Choose your project at or append a query string project=... to:
https://console.cloud.google.com
Related
Our organization uses Google Cloud APIs for integrating Maps and other services in a number of websites.
We have often used the same API key, without creating a distinct Google Cloud project (and credentials) for each website/project.
We are trying to better organize our API usage, but we are facing an issue.
While we can consult the reports of our Billing account and see the quota for the unique API project used for every implementation, we cannot see and manage this project (it does not appear in the list) even though it seems to belong to the same organization. (EDIT: I am not sure that the organization id is the same, but the name of the organizazion appears as a prefix to the project name in the billing reports)
This project has been created years ago (and the person that created it appears not to have access to it either), but we need to access it to get a clear understanding of where and how APIs are used.
The connected APIs are still in use and working, so we assume the project exists.
Can someone point out the possible reasons why a project is not shown even though it belongs to an organization for which we have access as administrators?
Thank you in advance
In order to see a project in lists, you need the resourcemanager.projects.list IAM permission on the project and to get it's metadata, the resourcemanager.projects.get permission.
How did you find that it has the same organizationId? If you managed to get the metadata via gcloud projects describe, you are likely missing the list permission.
In any case, if the project is indeed part of the organization, an org admin should be able to use gcloud projects add-iam-policy-binding to add a new owner/editor.
There is a special case with Apps Scripts: Those create a hidden project.
If all fails, reach out to GCP Support. Keep in mind though that they will not be able to help you if the project is not within your organization (eg. created with an unrelated gmail.com account or similar)
I have a project in Google cloud using the following resources
-BigQuery, Google functions (Python), google storage, Cloud Scheduler
is it possible to save the whole project as code and share it, so someone else can just use that code and deploy it using his own tenant ?
the reason, I am asking, I have published all the code and SQL queries in Github, but some users find it very hard to reproduce, they are not necessarily very familiar with Google Cloud, in an ideal situation, they need just to get a file and click deploy ?
When you create a solution for GCP we will commonly find that it consists of code, data and configuration. The code and data you can save in a source repository like GitHub ... but what of the configuration? What if your "solution" expects to have BQ datasets and tables or GCS buckets or Scheduler jobs defined? This is where you can create "Infrastructure As Code" definitions. Google supports its own IaC technology called Deployment Manager but you can also use the popular Terraform as it too has a GCP provider. The definitions for these IaC coordinators are typically text / yaml files that you can also package with your code. Sprinkle in some Make, Chef, Puppet for building apps and pushing code to deployment environments and you have a "build it from source" story. Study also the concepts of CI/CD and you will commonly find that the steps you perform for building CI/CD overlap with the steps for trivial deployment.
There are also projects such as terraformer that can do some kind of a job of reverse engineering an existing configuration to create IaC description that, when run elsewhere, will recreate the configuration.
Is there a way to duplicate an entire project?
The project contains:
2x Cloud SQL: main + backup
1x Cloud Storage
4x Google Compute Engine
We have an exactly the same project already built up and configured, so it would be much easier for us if we could just make a copy of those.
The projects are not under the same account.
There is no such a way to replicate as-is a project.
However, you can use Terraformer starting from your current project: this CLI tool will generate Terraform template files starting from the existing infrastructure (reverse Terraform). Then, you can use these files to re-create the target resources within a second GCP project in a programmatic fashion (see https://cloud.google.com/community/tutorials/getting-started-on-gcp-with-terraform).
Disclaimer: Comments and opinions are my own and not the views of my employer.
If I'm describing and managing the resources in my GCP projects via GDM (Google Deployment Manager), is it possible to propagate a change to multiple Google Cloud projects via GDM (Google Deployment Manager)?
Specifically, I'm looking at defining a BigQuery schema (set of datasets & tables) via GDM, and then syncing this schema across numerous projects (each of which will have their own BigQuery db). So ideally if I make a schema update like adding a column to a table, I only need to update a single "deployment" for the schema change to propagate to all the relevant projects.
Google Cloud Deployment Manager allows for actions to be propagated within the project. That being said, to make changes through multiple projects, you have to make a call per project. Consequently, the schema changes produced on one project will not be reproduced on BigQuery tables found in other projects.
I am thinking about writing my own release storage server and before I do this, I'd like to know what people use to see integration instead of create.
So what do you use to store your builds for internal access?
I'm looking for a web app that allows me to upload artifacts and then reference them by various tags so I can group them together by component or release vehicle. I also want access controls per build by readiness or promotion.
I define staging as placing built artifacts on a server for communities of users to access. The artifacts are usually zip files containing either applications or libraries + documentation. The user communities are developers, QA, and service delivery/operations. Basically, the creators, the checkers and external-users.
We release artifacts individually and as groups in a release vehicle (e.g., release 1.1 contains foo 1.0.1 and bar 1.0.7). Depending on the artifact, we may want to restrict access. Operations shouldn't be able to access pre-released builds and we may want to track who downloads a limited availability release.
So, I'm hoping to find a tool that does most of what I want with a good extensible design so I can add in what I don't have.
Any one know of a good tool for managing the builds post-build?
Examples might be:
quickbuild/lunt build
Team forge
build forge
Jira & confluence as a set
sonatype nexus
home grown
SVN repository using branching to promote builds from dev->Qa->GA
Peter,
Since you're not getting many answers, I'll let you know about AnthillPro whose developer, Urbancode, I work for.
Ok, disclaimers out of the way, AnthillPro is designed to serve exactly the broad audience that you're discussing - dev, checkers, and operations. Compared to the tools you list, AnthillPro is something like a BuildForge (a key competitor of ours) or quick build with a tightly integrated artifact repository (like nexus). So the builds are run, and you can view the results of your builds - and the build artifacts - in a nice web ui. Users with correct permissions can run a secondary process like a deployment or test against prior builds - and the artifacts from the selected build.
The goal is to manage the entire build lifecycle from creation, through various testing tools and deployment environments out through release to production. It's not a big nasty suite, instead we integrate with tools like Subversion and Jira to make sure every release has a manifest of source and problem ticket changes.
Your release packages would map well to AnthillPro's built in dependency system. We often see customers create virtual projects that take little or no source code, but instead either relate or package components into a release bundle.
Where AnthillPro may fall short for you is that generally, we would allow operations to see pre-release builds. However, you could add rules that would immediately fail / block an attempted releaes by operations of any build not marked as "pre-release". AnthillPro's system of statuses allows the team to flag a build with custom markers like, "In QA" or "Approved for Release". Combined with rules about running workflows,that should give you the control you need. If some projects are particularly sensitive, you'd just use the role based security to lock those away.
Hope that gives you something to look into.
-- Eric
My options are
build automation systems like AntHill, QuickBuild, TeamForge, BuildForge
file server
source control server
maven repository manager (nexus, archiva)
My goals are
group build by multiple criteria (artifact type, release vehicle, stage/phase)
promote build from dev -> qa -> released
provide access control for dev builds, qa ready builds, production ready builds
I'm going to focus on either source control as file server (using svn) or maven repos manager as file server using nexus. The rational is as follows:
minimize effort
minimize cost
use something I can easily extend when needed to (because I'm certain my requirements will shift).
maven use is growing and will eventually be the dominant build technology here.
Thanks for the information.