Suppose we have a kubernetes stack running on AWS and we would like to config our component directly via kubernetes chart files to be able to provision some AWS services (for example a DynamoDB table).
What would be the best practice if we want to achieve this and hopefully also allow our kubernetes component can connect to the provisioned services via IAM way (not just using simple key secret pairs)
So once achieve that, one can let a kubernetes component has a char file such as:
apiVersion: alpha1/v1
Kind: Database
type: dynamodb
table_name: a_test_table
hash_key: some_id
range_key: some_order
Then when deploying this component onto Kubernetes stacks, it will provision a DynamoDB table with name a_test_table for this service to be successfully connected onto.
What you are describing here is more or less what the AWS Service Broker is trying to achieve. Using the Open Service Broker API, the AWS broker would react to the creation of specific kubernetes objects, creating the required resources on AWS. Here you can find an AWS post explaining how this works.
There are different brokers for different cloud providers, not just AWS.
Related
Does anyone know in which case choose Kubernetes secrets instead of google secret manager and the reverse ? Differences between the two ?
With Kubernetes secret (K8S Secret), you use a built in feature of K8S. You load your secrets in config maps, and you mount them on the pods that require them.
PRO
If a day you want to deploy on AWS, Azure or on prem, still on K8S, the behavior will be the same, no update to perform in your code.
CONS
The secrets are only accessible by K8S cluster, impossible to reuse them with another GCP services
Note: With GKE, no problem the ETCD component is automatically encrypted with a key form KMS service to keep the secret encrypted at rest. But, it's not always the same for every K8S installation, especially on premise, where the secrets are kept in plain text. Be aware about this part of the security.
Secret Manager is a vault managed by Google. You have API to read and write them and the IAM service checks the authorization.
PRO
It's a Google Cloud service and you can access it from any GCP services (Compute Engine, Cloud Run, App Engine, Cloud Functions, GKE,....) as long as you are authorized for
CONS
It's Google Cloud specific product, you are locked in.
You can use them together via this sync service: https://external-secrets.io/
I'm using Kubeless on AWS EKS. While installing Kubeless, the installation has a manifest which has some CRDs and a deployment object. The deployment object has already a Service Account attached. I have created another Service Account in the kube-system namespace, which has some AWS IAM roles attached. I also want to attach this newly created Service Account used for IAM roles to the Kubeless deployment object by modifying the Kubeless manifest file.
I want to have 2 Service Accounts attached to the deployment object: One that comes with Kubeless and other for AWS IAM. Any help would appreciated. Thanks
This is not possible. If you look at the API documentation for PodSpec v1 core you can see that serviceAccountName expects a string not an array or object. This is because using a ServiceAccount resource creates a 1:1 relationship between your pod and authentication against the API server.
You will either need to:
Diversify your workload into multiple containers. Which with you can apply different service accounts.
Combine your service account capabilities into a single account and apply it exclusively to this pod.
I recommend #2.
I am developing the application where "form" plays an important role. I use the "form" for data collection from the users which I store it on DynamoDB and then these data should be displayed in my application synchronously or immediately after some trigger functions when the data has been inserted into the dynamoDB.
What is the best way to achieve this? How should I frame my infrastructure on the AWS Cloud? What are the services should I rely on?
For "realtime" applications you should use something like graphQL.
On AWS you can use: AWS App Sync
https://console.aws.amazon.com/appsync/home?region=us-east-1#/
There are multiple ways you can achieve this. However, one of most modernized way of building your applications today is using a Serverless Architecture. You can host your website on S3 and can go serverless with sample architecture as below
(Note: you can just replace the Amazon Aurora with Dynamo DB in the architecture reference)
You can create a server less application on AWS using following AWS services:
AWS Lambda: AWS Lambda is a compute service that lets you run code without provisioning or managing servers.
AWS API Gateway: Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
AWS S3: AWS S3 is Object storage built to store and retrieve any amount of data from anywhere
AWS DynamoDB: Amazon
DynamoDB is a fast and flexible nonrelational database service for
all applications that need consistent, single-digit millisecond latency at any scale.
AWS Route53: For creating and registering a domain name for the web app.
AWS IAM: AWS IAM for creating users,roles and policies.
AWS Cognito: for authentication, access control to your web app.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
Starting with AWS seems to be a pain in the neck. I've already spent countless hours trying to squeeze out some information about what does what in their ocean of products and brand names. But there are no simple answers. First I have to read through countless pages congratulating me on choosing AWS and confirming how easily I'll be able to begin. Then I have to watch a dozen videos in which some deputy chief architect manager of whatever department explains how excited they are to see me. Yeah, thanks, but will you finally tell what does this crap do?! I don't have all the world's time.
Is there a list somewhere a clear and concise lists of AWS services and products without all the inspirational corporate bullshit, something like this one (entirely fictional):
Daffodil: User management service which can be embedded in your codebase.
Trainwreck: Geospatial database API.
Footsmell: Industrial automation API to control robots and drones.
Wristwatch: Thesaurus and grammar checker.
If there was a similar one for Google's services, the better.
This is a slightly old list from March 2017:
Compute
Amazon EC2: Virtual Servers in the Cloud
Amazon EC2 Container Service: Run and Manage Docker Containers
Amazon EC2 Container Registry: Store and Retrieve Docker Containers
Auto Scaling: Automatic Elasticity
AWS Elastic Beanstalk: Run and Manage Web Apps
Amazon LightSail: Launch and Manage Virtual Private Servers
AWS Lambda: Run your code in response to events
AWS Batch: Run Batch Jobs at any Scale
Storage
Amazon S3 (Simple Storage Service): Scalable Storage in the Cloud
Amazon Glacier: Low-Cost Archive Storage in the Cloud
Amazon EBS (Elastic Block Store): Block Storage for EC2
Amazon EFS (Elastic File System): Managed File Storage for EC2
AWS Storage Gateway: Hybrid Storage Integration
Database
Amazon RDS (Relational Database Service): Managed Relational Database Service
Amazon Aurora: High Performance Managed Relational Database
Amazon DynamoDB: Managed NoSQL Database
Amazon Redshift: Fast, Simple, Cost-Effective Data Warehousing
Amazon ElastiCache: In-Memory Caching System
Migration
Snowball: Petabyte-scale Data Transport
AWS Application Discovery Service: Discover On-Premises Apps
AWS Database Migration Service: Migrate Databases with Minimal Downtime
AWS Server Migration Service: Migrate On-Premises Servers to AWS
Networking & Content Delivery
Amazon Virtual Private Cloud (VPC): Isolate Cloud Resources
AWS Direct Connect: Dedicated Network Connection to AWS
Amazon Route 53: Scalable Domain Name Service
Elastic Load Balancing: High Scale Load Balancing
Amazon CloudFront: Global Content Delivery Network
Developer Tools
AWS CodeCommit: Store Code in Private Git Repositories
AWS CodeBuild: Build and Test Code
AWS CodeDeploy: Automate Code Deployment
AWS CodePipeline: Release Software using Continuous Delivery
AWS X-Ray: Analyze and Debug Your Applications
AWS Command-Line Interface: Unified Tool to Manage AWS Services
Management Tools
AWS CloudFormation: Create and Manage Resources with Templates
AWS Service Catalog: Create and Use Standardized Products
Amazon CloudWatch: Monitor Resources and Applications
AWS CloudTrail: Track User Activity and API Usage
AWS Config: Track Resource Inventory and Changes
AWS OpsWorks: Automate Operations with Chef
Amazon EC2 Systems Manager: Configure EC2 Instances and On-Premises Servers
AWS Trusted Advisor: Optimize Performance and Security
AWS Personal Health Dashboard: Personalized View of AWS service health
Security, Identity & Compliance
AWS Identity & Access Management (IAM): Manage User Access and Encryption Keys
AWS Organizations: Policy-Based Management for Multiple AWS Accounts
AWS Directory Service: Host and Manage Active Directory
AWS Cloud Directory: Create flexible cloud-native directories
AWS Key Management Service (KMS): Creation and Control of Encryption Keys
AWS CloudHSM: Hardware-based Key Storage
AWS Certificate Manager: Provision and Deploy SSL/TLS Certificates
Amazon Inspector: Analyze Application Security
AWS Shield: Managed DDoS Protection
AWS Web Application Firewall (WAF): Filter Malicious Web Traffic
Analytics
Amazon Athena: Query Data in S3 using SQL
Amazon EMR: Hosted Hadoop Framework
Amazon CloudSearch: Managed Search Service
Amazon Elasticsearch Service: Run and Scale Elasticsearch Clusters
Amazon Kinesis: Work with Real-Time Streaming Data
Amazon QuickSight: Fast Business Analytics Service
AWS Data Pipeline: Orchestration Service for periodic Data-Driven Workflows
AWS Glue: Prepare and Load Data
Artificial Intelligence
Amazon Machine Learning: Machine Learning for Developers
Amazon Polly: Turn Text into Lifelike Speech
Amazon Rekognition: Search and Analyze Images
Amazon Lex: Build Voice and Text Chatbots
Mobile Services
Amazon Cognito: User Identity and App Data Synchronization
AWS Device Farm: Test Mobile Apps on Real Devices in the Cloud
AWS Mobile Hub & Mobile SDK: Build, Test and Monitor Mobile Apps
Application Services
Amazon API Gateway: Build, Deploy and Manage APIs
AWS Step Functions: Coordinate Distributed Applications
Amazon Elastic Transcoder: Easy-to-Use Scalable Media Transcoding
Messaging
Amazon Simple Queue Service (SQS): Message Queue Service
Amazon Simple Notification Service (SNS): Push Notification Service
Amazon Simple Email Service (SES): Email Sending and Receiving Service
Amazon Pinpoint: Push Notifications for Mobile Apps
Business Productivity
Amazon Chime: Frustration-free meetings, video calls and chats
Amazon WorkDocs: Enterprise Storage and Sharing Service
Amazon WorkMail: Managed Business Email and Calendaring
Desktop & App Streaming
Amazon WorkSpaces: Desktop Computing Service
Amazon AppStream 2.0: Stream desktop applications to a browser
Internet of Things
AWS IoT Platform: Connect Devices to the Cloud
AWS Greengrass: Local Compute, Messaging, Sync for Devices
AWS IoT Button: Cloud Programmable Dash Button
Game Dev
Amazon GameLift: Dedicated Game Server Hosting
Amazon Lumberyard: Free Cross-Platform 3D game engine
There's even more these days!
It's a fair point, and with so many Amazon Web Service (AWS) services, not an easy one to sum up in a few words.
I'd say start here for a summary of the main services: https://d1.awsstatic.com/whitepapers/aws-overview.pdf
Then I think the Tech Essentials training video from acloud.guru (with 7 day free trial) is a good video to get you going: https://acloud.guru/learn/aws-technical-essentials
Google Cloud Platform is a bit more accessible IMO, their main product page gives a brief description of the products:
https://cloud.google.com/products/
Some context to the services: https://cloud.google.com/docs/overview/cloud-platform-services
And again acloud.guru have an introductory video for CDP: https://acloud.guru/learn/gcp-101
and I might as well complete the trifactor…
Microsoft Azure is a very worthy contender,
High level services: https://azure.microsoft.com/en-gb/services/
Intro Video: https://acloud.guru/learn/intro-to-azure
If you want one liners like you mentioned in your questions then click here
On that page click on the category of the service and it will list out services in that category and one line description.
E.g. Click 'Compute' to see the list of provided compute services, click 'Storage' to list if provided storage services and so on
If you want somewhat detailed explanation, click here
Here also services are grouped in categories, and you click on one of categories and you get to see the services (along with some brief explanation about each of the service) within that category
The documentation page of each product gives explanation in a simple way. Moreover, if you read the FAQ that explains things from scratch.
Does AWS Java SDK have an api that could help me to retrieve list of resources (vpc, dynamodb, volumes, ec2 etc...) for a given AWS account number?
I have gone through AWS Java SDK docs at a higher level but everything is related to one specific AWS client for a given resource.
I would like to have an abstract AWS client so that it could provide me just couple attributes of associated AWS resources to an aws account.
Any help is appreciated. Thanks!!
All AWS API calls are related to specific services. For example, you can request a list of Amazon VPCs, a list of Amazon DynamoDB tables, a list of Amazon EBS volumes -- but each would require a different API call.
Another option would be to use AWS Config:
AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.
AWS Config can deliver a Configuration Snapshot into an Amazon S3 bucket at regular intervals (eg daily). This snapshot (example) is a JSON file that contains information about VPCs, Amazon EC2 instances and related resources.
However, the configuration snapshot only contains information related to a limited number of services, such as EC2, VPC, Amazon Redshift, Amazon RDS and Amazon S3. (See Supported AWS Resource Types)