GCP Service Observer
This is an application that displays GCP Service/API endpoint information for a given project. GCP requires API endpoints to be enabled per project before they can be leveraged so knowing which ones are enabled or disabled is an important first step in programmatically using client libraries or RESTFUL APIs.
The GCP Service Observer application is a personal project and not recommended for production or enterprise environments.
Table of Concents
Features
- Project ID permissions checks
- Input sanitation and validation
- Can return all enabled, disabled, or both APIs for a given project
- Table filtering
- Direct-documentation links to enable or disable APIs manually
- Deploy locally for easy debugging
- Low cost - runs via Cloud Run
Configuration
This application is designed to be deployed in a single project but the service account backing Cloud Run would have an IAM binding on the organization node. This configuration allows you to view any project's API/service endpoint status in the GCP organization.
Before configuring for org-wide observability I would highly recommend adding some type of end user authentication. Google provides a nice turorial here.
The below steps are only required if you want to deploy to Cloud Run. If you want to deploy locally jump to Local Deployment.
Steps
Assuming you just want to demo the application in Cloud Run the only configuration needed is to add a value for the project_id variable in the variables.tf file:
variable "project_id" {
description = "The project ID where the service is deployed."
default = "demo-project-id" # add this line
}
If you want to deploy with observability across an entire organization you will need to add the above project_id variable and the below updates:
- Uncomment the
variable "org_id"block in the variables.tf file and add a default value:
# Uncomment if using on org-level
#variable "org_id" {
# description = "The organization ID where the service observer app has API/service view access."
#}
would become:
variable "org_id" {
description = "The organization ID where the service observer app has API/service view access."
default = "my-org-id" # add this line
}
- In the main.tf file uncomment the organization specific resources:
# Enable below for org-level project API / service listing
#resource "google_organization_iam_member" "organization-sa-iam" {
# org_id = var.org_id
# role = google_organization_iam_custom_role.organization-sa-custom-role.id
# member = "serviceAccount:${google_service_account.service-observer-sa.email}"
#}
#resource "google_organization_iam_custom_role" "organization-sa-custom-role" {
# role_id = "ServiceObserverViewerRole"
# org_id = var.org_id
# title = "Service Observer org IAM role. View only."
# description = "Provides permissions to view API/services across the organization."
# permissions = ["resourcemanager.projects.get", "servicemanagement.services.list"]
#}
would become:
resource "google_organization_iam_member" "organization-sa-iam" {
org_id = var.org_id
role = google_organization_iam_custom_role.organization-sa-custom-role.id
member = "serviceAccount:${google_service_account.service-observer-sa.email}"
}
resource "google_organization_iam_custom_role" "organization-sa-custom-role" {
role_id = "ServiceObserverViewerRole"
org_id = var.org_id
title = "Service Observer org IAM role. View only."
description = "Provides permissions to view API/services across the organization."
permissions = ["resourcemanager.projects.get", "servicemanagement.services.list"]
}
- Lastly comment out the project-level IAM resources:
# Disable below if using for org-wide API/service listing
resource "google_project_iam_member" "project-sa-iam" {
project = var.project_id
role = google_project_iam_custom_role.project-sa-custom-role.id
member = "serviceAccount:${google_service_account.service-observer-sa.email}"
}
resource "google_project_iam_custom_role" "project-sa-custom-role" {
project = var.project_id
role_id = "ServiceObserverViewerRole"
title = "Service Observer project IAM role. View only."
description = "Provides permissions to view API/services on a given project."
permissions = ["resourcemanager.projects.get", "servicemanagement.services.list"]
}
would become:
# Disable below if using for org-wide API/service listing
#resource "google_project_iam_member" "project-sa-iam" {
# project = var.project_id
# role = google_project_iam_custom_role.project-sa-custom-role.id
# member = "serviceAccount:${google_service_account.service-observer-sa.email}"
#}
#resource "google_project_iam_custom_role" "project-sa-custom-role" {
# project = var.project_id
# role_id = "ServiceObserverViewerRole"
# title = "Service Observer project IAM role. View only."
# description = "Provides permissions to view API/services on a given project."
# permissions = ["resourcemanager.projects.get", "servicemanagement.services.list"]
#}
Deployment
This application runs via Cloud Run and is deployed via Terraform or can be run locally. Please review the Configuration section before deploying.
In order to deploy to GCP you'll need administrator permissions for Cloud Run and the ability to enable/disable APIs and create IAM resources.
Cloud Deployment
Before deploying authenticate locally via gcloud auth login --update-adc
- To deploy first change directories into the terraform folder and execute
terraform init
# Change directory
cd terraform/
# initialize terraform
terraform init
- You should see a message stating
Terraform has been successfully initialized!
-
Next execute a
terraform plan -out tfplan.jsonand then review the potential updates. -
If you are satisfied with the updates deploy the terraform via
terraform apply tfplan.json -
After the terraform is deployed you will see an output at the very end of the deployment that will look similar to
Outputs:
sa_id = "[email protected]"
- Export the service account ID and the project ID you used for the Terraform
project_idvariable:
export SA_ID="[email protected]"
export PROJ_ID="demo-project-id"
- Now that the infrastructure is mostly deployed we can deploy to Cloud Run via:
# Remember to change directories to the root of the repo
cd ../
# Deploy Cloud Run
gcloud beta run deploy gcpserviceobserver --source "." \
--region="us-east1" \
--service-account=$SA_ID \
--allow-unauthenticated \
--project=$PROJ_ID
- GCP will perform all of the configuration needed for Cloud Run and once completed will provides a URL to access the application. Your output will look similar to below (if nothing went wrong):
Building using Dockerfile and deploying container to Cloud Run service [gcpserviceobserver] in project [gcp-service-observer] region [us-east1]
✓ Building and deploying new service... Done.
✓ Uploading sources...
✓ Building Container... Logs are available at [https://console.cloud.google.com/cloud-build/builds/2730
3291-f10b-4f13-978f-74b941787060?project=285094994588].
✓ Creating Revision...
✓ Routing traffic...
✓ Setting IAM Policy...
Done.
Service [gcpserviceobserver] revision [gcpserviceobserver-00001-mif] has been deployed and is serving 100 percent of traffic.
Service URL: https://gcpserviceobserver-hda7rvdo7a-ue.a.run.app
- Clicking the
Service URLvalue will launch the GCP Service Observer application.
Local Deployment
- To deploy locally you will first need to export the Flask environment url:
export FLASK_APP=main
- Next run Flask:
flask run
- Enter the local URL in your browser window
http://127.0.0.1:5000
Contributing
Contributions are always welcome! This is a personal project but of course feel free to fork and have fun with it!
Comments / concerns / bug info? Message me on Twitter @jasonadyke or LinkedIn.
6 Oct 20, 2022
4 Jul 21, 2022
20 Nov 3, 2022
7 Dec 21, 2022
53 Dec 30, 2022
6 Nov 4, 2022
8 Nov 17, 2022
5 Jun 17, 2022
4 Sep 20, 2022
8 Dec 18, 2022