VirtualService Routing for services having similar subset of name - istio

I need to define two VirtualService for same host, .i.e.
host: http://customerA.test.example.com, would be used to serve two service namely "srv" and "srvui"
To Achieve this I respectively, defined Virtual Service as:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: srv
namespace: srv-test01
spec:
gateways:
- http-gateway
hosts:
- customerA.test.example.com
http:
- match:
- uri:
prefix: /srv
route:
- destination:
host: srv.srv-test01.svc.cluster.local
port:
number: 8080
and
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: srvui
namespace: srv-test01
spec:
gateways:
- http-gateway
hosts:
- customerA.test.example.com
http:
- match:
- uri:
prefix: /srvui
route:
- destination:
host: srvui.srv-test01.svc.cluster.local
port:
number: 8080
This has resulted in envoy conf on istio-ingress-gateway as (both virtualService are in same namespace)
"name": "customerA.test.example.com:80",
"domains": [
"customerA.test.example.com",
"customerA.test.example.com:80"
],
"routes": [
{
"match": {
"prefix": "/srv"
},
"route": {
"cluster": "outbound|8080||srv.srv-test01.svc.cluster.local",
"timeout": "0s",
"max_grpc_timeout": "0s"
},
"decorator": {
"operation": "srv.srv-test01.svc.cluster.local:8080/srv*"
},
"per_filter_config": {
"mixer": {
"forward_attributes": {
"attributes": {
"destination.service.host": {
"string_value": "srv.srv-test01.svc.cluster.local"
},
"destination.service.uid": {
"string_value": "istio://srv-test01/services/srv"
},
"destination.service.namespace": {
"string_value": "srv-test01"
},
"destination.service.name": {
"string_value": "srv"
},
"destination.service": {
"string_value": "srv.srv-test01.svc.cluster.local"
}
}
},
"mixer_attributes": {
"attributes": {
"destination.service.host": {
"string_value": "srv.srv-test01.svc.cluster.local"
},
"destination.service.uid": {
"string_value": "istio://srv-test01/services/srv"
},
"destination.service.name": {
"string_value": "srv"
},
"destination.service.namespace": {
"string_value": "srv-test01"
},
"destination.service": {
"string_value": "srv.srv-test01.svc.cluster.local"
}
}
}
}
}
},
{
"match": {
"prefix": "/srvui"
},
"route": {
"cluster": "outbound|8080||srvui.srv-test01.svc.cluster.local",
"timeout": "0s",
"max_grpc_timeout": "0s"
},
"decorator": {
"operation": "srvui.srv-test01.svc.cluster.local:8080/srvui*"
},
"per_filter_config": {
"mixer": {
"forward_attributes": {
"attributes": {
"destination.service.namespace": {
"string_value": "srv-test01"
},
"destination.service.name": {
"string_value": "srvui"
},
"destination.service": {
"string_value": "srvui.srv-test01.svc.cluster.local"
},
"destination.service.host": {
"string_value": "srvui.srv-test01.svc.cluster.local"
},
"destination.service.uid": {
"string_value": "istio://srv-test01/services/srv"
}
}
},
"mixer_attributes": {
"attributes": {
"destination.service.host": {
"string_value": "srvui.srv-test01.svc.cluster.local"
},
"destination.service.uid": {
"string_value": "istio://srv-test01/services/srv"
},
"destination.service.name": {
"string_value": "srvui"
},
"destination.service.namespace": {
"string_value": "srv-test01"
},
"destination.service": {
"string_value": "srvui.srv-test01.svc.cluster.local"
}
}
}
}
}
}
]
},
However the problem here is that even the request coming for /srvui are ending up on /srv virtual-service, logs on istio-ingress-gateway
[2019-02-04T05:46:20.822Z] "**GET /srvHTTP/1.1**" 404 - 0 1077 3 2 "xx.xx.xx.xx, xx.xx.xx.xx" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" "5fa6611c-07ab-954c-b871-09398bd6c2e4" "customerA.test.example.com" "10.192.21.210:8080" outbound|8080||srv.srv-test01.svc.cluster.local - 10.192.11.185:80 10.192.20.101:36536 ---> **end up on correct service on backend as per virtualService defination**
[2019-02-04T05:46:40.864Z] "**GET /srvuiHTTP/1.1**" 404 - 0 1079 3 1 "xx.xx.xx.xx, 10.192.10.101" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" "58e17520-a5df-9c90-9ec4-62dbc2bc1307" "customerA.test.example.com" "10.192.21.210:8080" outbound|8080||srv.srv-test01.svc.cluster.local - 10.192.11.185:80 10.192.10.101:54352 ---> Even the context is srvui it is ending up on srv backend.
it looks like it depends on the numbering of VirtualService in its conf for the routing to take effect,
if we configure srvui virtualService first and then srv routing happens correctly.
however the problem with it, we need to maintain the deployment of these service in numbered order, which I would like to avoid
I cannot use "exact" in place of "prefix", because then I would need to defined more than 100 exact URI path
Other solution is to change the name of the service context so that they do not look like subset of each other like here srv and srvui and thus istio-ingress-gateway can route it properly ,however this also require application change.
kindly let me know if there is any other solution, I could implement here.

That's not surprising as srv is a subset of srvui.
You should be able to use a regex match instead of either exact or prefix. See https://istio.io/docs/reference/config/istio.networking.v1alpha3/#HTTPMatchRequest for more full documentation.
URI to match values are case-sensitive and formatted as follows:
exact: "value" for exact string match
prefix: "value" for prefix-based match
regex: "value" for ECMAscript style regex-based match

Related

Calling AWS Lambda via api gateway on localstack doesn't work as expected

Hi I have downloaded the most basic java lambda example
package helloworld;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;
import java.util.stream.Collectors;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent;
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent;
/**
* Handler for requests to Lambda function.
*/
public class App implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) {
Map<String, String> headers = new HashMap<>();
headers.put("Content-Type", "application/json");
headers.put("X-Custom-Header", "application/json");
APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent()
.withHeaders(headers);
try {
final String pageContents = this.getPageContents("https://checkip.amazonaws.com");
String output = String.format("{ \"message\": \"hello world\", \"location\": \"%s\" }", pageContents);
return response
.withStatusCode(200)
.withBody(output);
} catch (IOException e) {
return response
.withBody("{}")
.withStatusCode(500);
}
}
private String getPageContents(String address) throws IOException{
URL url = new URL(address);
try(BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream()))) {
return br.lines().collect(Collectors.joining(System.lineSeparator()));
}
}
}
I then create a docker image with commands
mvn compile dependency:copy-dependencies -DincludeScope=runtime
docker build -t $DOCKER_TAG .
docker push $DOCKER_TAG
From then I have the following terraforms files
# Create the lambda role (using lambdarole.json file)
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
resource "aws_iam_role" "lambda_role" {
name = "${var.stack_prefix}-role-lambdarole-${var.unique_name}"
assume_role_policy = file("${path.module}/files/lambdarole.json")
}
# Apply the Policy Document we just created
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
resource "aws_iam_role_policy" "lambda_policy" {
name = "${var.stack_prefix}-role-lambdapolicy-${var.unique_name}"
role = aws_iam_role.lambda_role.id
policy = file("${path.module}/files/lambdapolicy.json")
}
# Variables
variable "apiName" { default = "java-lambda" }
# API Gateway
resource "aws_lambda_function" "test_lambda" {
function_name = "${var.apiName}"
image_uri = "localhost:4510/ac.uk-jlambda:1.0.0"
role = aws_iam_role.lambda_role.arn
runtime = "java11"
handler = "example.FunctionConfiguration::Function"
timeout = "60"
publish = true
}
resource "aws_api_gateway_rest_api" "apiLambda" {
name = "${var.apiName}"
}
resource "aws_api_gateway_resource" "proxy" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
parent_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "proxyMethod" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxyMethod.resource_id
http_method = aws_api_gateway_method.proxyMethod.http_method
integration_http_method = "GET"
type = "AWS_PROXY"
uri = aws_lambda_function.test_lambda.invoke_arn
}
resource "aws_api_gateway_method" "proxy_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxy_root.resource_id
http_method = aws_api_gateway_method.proxy_root.http_method
integration_http_method = "GET"
type = "AWS_PROXY"
uri = aws_lambda_function.test_lambda.invoke_arn
}
resource "aws_api_gateway_deployment" "apideploy" {
depends_on = [
aws_api_gateway_integration.lambda,
aws_api_gateway_integration.lambda_root,
]
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
stage_name = "test"
}
resource "aws_lambda_permission" "apigw" {
statement_id = "AllowAPIGatewayInvoke"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.test_lambda.function_name
principal = "apigateway.amazonaws.com"
# The "/*/*" portion grants access from any method on any resource
# within the API Gateway REST API.
source_arn = "${aws_api_gateway_rest_api.apiLambda.execution_arn}/*/*"
}
output "base_url" {
value = aws_api_gateway_deployment.apideploy.invoke_url
}
Then I call terraform init, plan and apply.
Afterwards when I use the command:
awslocal --region=eu-west-2 lambda invoke --function-name java-lambda --payload '{"key1":"hello world!","key2":"hello another world!"}' --cli-binary-format raw-in-base64-out /dev/stdout
I get output
{"body":"{\"message\": \"Go Serverless v2.0! Your function executed successfully!\", \"input\": {\"key1\": \"hello world!\", \"key2\": \"hello another world!\"}}","statusCode":200}
{
"StatusCode": 200,
"LogResult": "",
"ExecutedVersion": "$LATEST"
}
Then I use this command to find the id of the api gateway
awslocal apigateway get-rest-apis --region eu-west-2
and use this id in the following url format
http://localhost:4566/restapis/ngabak60o4/test/_user_request_/
This shows a json response in the browser
{
message: "Go Serverless v2.0! Your function executed successfully!",
input: {
body: "",
headers: {
Authorization: "",
Cache-Control: "max-age=0",
Connection: "keep-alive",
Cookie: "aekt_hostname=ucl-3.ombiel.co.uk; a=96df33c3b846ea04YD9uLTvub3R3%2FxtuaEssxWBymFrbu7Vg5LH%2BpWGOQJvXpE9FywQ4OnoISm8ccMID0yeoi1EnhlDcau66bNXuh1k6PHi317kCOJr3%2B7Qus8vEYkw%2FCs7fwS%2Bmd5%2B7ZvwveqgUgTtYbdlsw1Jn6ozcbDbmw3oqhXt1r3%2Fi%2F9A5I3P9gHHYwyfXjLMHWjqZFmYJ7elQCSgTCPeSkuQW%2BbRGNw3XfqSuCvd8%2BEjQCqApjQ9C3Yb3RX9DcueKaIfjntKCXbxyyyhgdNCI7bYoQuqAOdZpmZs0pygabE%2B1GuDfRTbkJ%2BbLRaSL8eYjUAmrzfKP; __a=471ffc53886568546223239137b5ac60; ombl_web2=yes; ombl_device_grade=med; campusM[7]=8%5B; __utmz=111872281.1650902813.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=111872281.1289308641.1650902813.1652888907.1655293849.25",
Host: "localhost:4566",
Remote-Addr: "172.17.0.1",
Sec-Ch-Ua: "" Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"",
Sec-Ch-Ua-Mobile: "?0",
Sec-Ch-Ua-Platform: ""Windows"",
Sec-Fetch-Dest: "document",
Sec-Fetch-Mode: "navigate",
Sec-Fetch-Site: "none",
Sec-Fetch-User: "?1",
Upgrade-Insecure-Requests: "1",
User-Agent: "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36",
X-Forwarded-For: "172.17.0.1, localhost:4566, 127.0.0.1, localhost:4566",
accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
accept-encoding: "gzip, deflate, br",
accept-language: "en-GB,en-US;q=0.9,en;q=0.8",
x-localstack-edge: "http://localhost:4566",
x-localstack-tgt-api: "apigateway"
},
httpMethod: "GET",
isBase64Encoded: false,
multiValueHeaders: {
Authorization: [
""
],
Cache-Control: [
"max-age=0"
],
Connection: [
"keep-alive"
],
Cookie: [
"aekt_hostname=ucl-3.ombiel.co.uk; a=96df33c3b846ea04YD9uLTvub3R3%2FxtuaEssxWBymFrbu7Vg5LH%2BpWGOQJvXpE9FywQ4OnoISm8ccMID0yeoi1EnhlDcau66bNXuh1k6PHi317kCOJr3%2B7Qus8vEYkw%2FCs7fwS%2Bmd5%2B7ZvwveqgUgTtYbdlsw1Jn6ozcbDbmw3oqhXt1r3%2Fi%2F9A5I3P9gHHYwyfXjLMHWjqZFmYJ7elQCSgTCPeSkuQW%2BbRGNw3XfqSuCvd8%2BEjQCqApjQ9C3Yb3RX9DcueKaIfjntKCXbxyyyhgdNCI7bYoQuqAOdZpmZs0pygabE%2B1GuDfRTbkJ%2BbLRaSL8eYjUAmrzfKP; __a=471ffc53886568546223239137b5ac60; ombl_web2=yes; ombl_device_grade=med; campusM[7]=8%5B; __utmz=111872281.1650902813.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=111872281.1289308641.1650902813.1652888907.1655293849.25"
],
Host: [
"localhost:4566"
],
Remote-Addr: [
"172.17.0.1"
],
Sec-Ch-Ua: [
"" Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102""
],
Sec-Ch-Ua-Mobile: [
"?0"
],
Sec-Ch-Ua-Platform: [
""Windows""
],
Sec-Fetch-Dest: [
"document"
],
Sec-Fetch-Mode: [
"navigate"
],
Sec-Fetch-Site: [
"none"
],
Sec-Fetch-User: [
"?1"
],
Upgrade-Insecure-Requests: [
"1"
],
User-Agent: [
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36"
],
X-Forwarded-For: [
"172.17.0.1, localhost:4566, 127.0.0.1, localhost:4566"
],
accept: [
"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
],
accept-encoding: [
"gzip, deflate, br"
],
accept-language: [
"en-GB,en-US;q=0.9,en;q=0.8"
],
x-localstack-edge: [
"http://localhost:4566"
],
x-localstack-tgt-api: [
"apigateway"
]
},
multiValueQueryStringParameters: null,
path: "/",
pathParameters: { },
queryStringParameters: null,
requestContext: {
accountId: "000000000000",
apiId: "kkjc44n3w7",
authorizer: { },
domainName: "localhost",
domainPrefix: "localhost",
httpMethod: "GET",
identity: {
accountId: "000000000000",
sourceIp: "127.0.0.1",
userAgent: "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36"
},
path: "/test/",
protocol: "HTTP/1.1",
requestId: "81af5979-6221-407e-b1a0-6ea74e09fb96",
requestTime: "23/Jun/2022:13:00:09 +0000",
requestTimeEpoch: 1655989209451,
resourceId: "1gbuakz0ci",
resourcePath: "/",
stage: "test"
},
resource: "/"
}
}
Which claims to be success. But I don't see the output of my handleRequest method.
I follow a similar tutorial for node js and that works correctly.
Can anyone please see what I have done wrong here?
Many thanks

Getting unauthorize message while using #kubernetes/client-node

const { v4: uuidv4 } = require("uuid");
const k8s = require("#kubernetes/client-node");
const kc = new k8s.KubeConfig();
kc.loadFromDefault()
const k8sApi = kc.makeApiClient(k8s.CoreV1Api);
let USER_ID=Date.now(), LEVEL=1, DIFFICULTY="start", DRONE="dr1", RESET=false
const definition = {
apiVersion: "v1",
kind: "Pod",
metadata: {
annotations: {
key1: "value1",
key2: "value2",
},
name: USER_ID,
labels: { app: "simulator-app", name: USER_ID },
},
spec: {
containers: [
{
name: "simulator-app",
image:
".....dkr.ecr.eu-central-1.amazonaws.com/simulator:latest",
ports: [{ containerPort: 8080 }, { containerPort: 9090 }],
tty: true,
stdin: true,
imagePullPolicy: "IfNotPresent",
command: ["/bin/bash", "-c"],
args: [
'echo START....;LEVEL=%s;DIFFICULTY=%s;DRONE=%s;echo "LEVEL=${LEVEL}";echo "DIFFICULTY=${DIFFICULTY}";echo "DRONE=${DRONE}";cd /home/;source ~/.bashrc;echo "Bashrc Sourced";pwd;ls;echo "Starting simulator with ${LEVEL} level, ${DRONE} vehicle and ${DIFFICULTY} difficulty";source "/opt/ros/foxy/setup.bash";source "/home/rossimulator/install/setup.bash";ros2 launch simulator_bringup sim_nodes.launch.py level:=${LEVEL} drone_type:=${DRONE} difficulty:=${DIFFICULTY};echo Done;' %
(LEVEL, DIFFICULTY, DRONE),
],
},
],
},
};
k8sApi.createNamespacedPod("Default", definition)
.then(console.log)
.catch(console.log);
Getting the below error on the console
{
response: {
statusCode: 401,
body: {
kind: "Status",
apiVersion: "v1",
metadata: {},
status: "Failure",
message: "Unauthorized",
reason: "Unauthorized",
code: 401,
},
headers: {
"audit-id": "07eba07d-6121-492f-9993-ece3fa8827c5",
"cache-control": "no-cache, private",
"content-type": "application/json",
date: "Mon, 28 Mar 2022 02:48:59 GMT",
"content-length": "129",
connection: "close",
},
request: {
uri: {
protocol: "https:",
slashes: true,
auth: null,
host: ".....gr7.eu-central-1.eks.amazonaws.com",
port: 443,
hostname: "......gr7.eu-central-1.eks.amazonaws.com",
hash: null,
search: null,
query: null,
pathname: "/api/v1/namespaces/Default/pods",
path: "/api/v1/namespaces/Default/pods",
href: "https://..................eu-central-1.eks.amazonaws.com/api/v1/namespaces/Default/pods",
},
method: "POST",
headers: {
Accept: "application/json",
Authorization: "Bearer k8s-aws-v1.....",
"content-type": "application/json",
"content-length": 506,
},
},
},
body: {
kind: "Status",
apiVersion: "v1",
metadata: {},
status: "Failure",
message: "Unauthorized",
reason: "Unauthorized",
code: 401,
},
statusCode: 401,
name: "HttpError",
};
What could be the possible reason for getting this error?
While the it works in the pythong this way
from flask import Flask, jsonify
import time
from kubernetes import client, config
import uuid
from kubernetes.client.rest import ApiException
app = Flask(__name__)
config.load_kube_config(
context="arn:aws:eks:eu-central-1:9010......:cluster/sim-cluster"
)
v1 = client.CoreV1Api()
# definition - same as above
v1.create_namespaced_pod(body=definition, namespace="default")
I did not found load_kube_config equivalent method in the library

Istio VirtualService not used in k8s Service

Hi I'm very newby in Istio/K8s, and I'm trying to make a service that I have test-service to use a new VirtualService that I've created.
Here the steps that I did
kubectl config set-context --current --namespace my-namespace
I create my VirtualService
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: test-service
namespace: my-namespace
spec:
hosts:
- test-service
http:
- fault:
delay:
fixedDelay: 60s
percentage:
value: 100
route:
- destination:
host: test-service
port:
number: 9100
Then I apply into K8s
kubectl apply -f test-service.yaml
But now when I invoke the test-service using gRPC I can reach the service, but the fault with the delay is not happening.
I dont know in which log I can see of this test-service is using the VirtualService that I created or not
Here my gRPC Service config:
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "test-service",
"namespace": "my-namespace",
"selfLink": "/api/v1/namespaces/my-namespace/services/test-service",
"uid": "8a9bc730-4125-4b52-b373-7958796b5df7",
"resourceVersion": "317889736",
"creationTimestamp": "2021-07-07T10:39:54Z",
"labels": {
"app": "test-service",
"app.kubernetes.io/managed-by": "Helm",
"version": "v1"
},
"annotations": {
"meta.helm.sh/release-name": "test-service",
"meta.helm.sh/release-namespace": "my-namespace"
},
"managedFields": [
{
"manager": "Go-http-client",
"operation": "Update",
"apiVersion": "v1",
"time": "2021-07-07T10:39:54Z",
"fieldsType": "FieldsV1",
"fieldsV1": {
"f:metadata": {
"f:annotations": {
".": {},
"f:meta.helm.sh/release-name": {},
"f:meta.helm.sh/release-namespace": {}
},
"f:labels": {
".": {},
"f:app": {},
"f:app.kubernetes.io/managed-by": {},
"f:version": {}
}
},
"f:spec": {
"f:ports": {
".": {},
"k:{\"port\":9100,\"protocol\":\"TCP\"}": {
".": {},
"f:port": {},
"f:protocol": {},
"f:targetPort": {}
}
},
"f:selector": {
".": {},
"f:app": {}
},
"f:sessionAffinity": {},
"f:type": {}
}
}
},
{
"manager": "dashboard",
"operation": "Update",
"apiVersion": "v1",
"time": "2022-01-14T15:51:28Z",
"fieldsType": "FieldsV1",
"fieldsV1": {
"f:spec": {
"f:ports": {
"k:{\"port\":9100,\"protocol\":\"TCP\"}": {
"f:name": {}
}
}
}
}
}
]
},
"spec": {
"ports": [
{
"name": "test-service",
"protocol": "TCP",
"port": 9100,
"targetPort": 9100
}
],
"selector": {
"app": "test-service"
},
"clusterIP": "****************",
"type": "ClusterIP",
"sessionAffinity": "None"
},
"status": {
"loadBalancer": {}
}
}
According to the Istio documentation, configuring fault only works for HTTP traffic, not for gRPC:
https://istio.io/latest/docs/reference/config/networking/virtual-service/#HTTPFaultInjection

How do I set up a publicly accessible API call for with an API Gateway?

I have an API Gateway Call with an associated Lambda call.
When I click on this link (it's used to confirm an email address) this is what I get:
https://<api-gateway-code>.execute-api.us-east-2.amazonaws.com/default/retrieveEmailConfStringAPI&p1=value1&p2=value2
Results:
{"message":"Missing Authentication Token"}
Complete Headers:
X-Firefox-Spdy: h2
content-length: 42
content-type: application/json
date: Mon, 06 Jan 2020 06:35:32 GMT
x-amz-apigw-id: <amazon-api-gateway-id>
x-amzn-errortype: MissingAuthenticationTokenException
x-amzn-requestid: <amazon-request-id>
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
Host: <api-gateway-id>.execute-api.us-east-2.amazonaws.com
TE: Trailers
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:71.0) Gecko/20100101 Firefox/71.0
However, when I change this up a little to the following web address:
https://<api-gateway-id>.execute-api.us-east-2.amazonaws.com/default&p1=value1&p2=value2
This is the result:
{"message":"Forbidden"}
These are the raw headers results:
X-Firefox-Spdy: h2
content-length: 23
content-type: application/json
date: Mon, 06 Jan 2020 06:45:13 GMT
x-amz-apigw-id: <amazon-api-gateway-id>
x-amzn-errortype: ForbiddenException
x-amzn-requestid: <amazon-request-id>
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
Host: <api-gateway-id>.execute-api.us-east-2.amazonaws.com
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:71.0) Gecko/20100101 Firefox/71.0
This is the set up using the API Gateway in terms of a JSON object.
MacBook-Pro-2:lambdaZipFilesDev chauncey$ aws2 apigateway get-resources --rest-api-id <rest api id> --region us-east-2
{
"items": [
{
"id": "<resource id>",
"parentId": "<parent id>",
"pathPart": "retrieveEmailConfStringAPI",
"path": "/retrieveEmailConfStringAPI",
"resourceMethods": {
"ANY": {},
"GET": {},
"OPTIONS": {}
}
},
{
"id": "<parent resource>",
"path": "/"
}
]
}
Here's the second command I ran in order to build this public API call:
MacBook-Pro-2:lambdaZipFilesDev chauncey$ aws2 apigateway update-method --rest-api-id <rest api id> --resource-id <resource id> --http-method GET --patch-operation op="replace",path="/apiKeyRequired",value="false"
{
"httpMethod": "GET",
"authorizationType": "NONE",
"apiKeyRequired": false,
"requestParameters": {
"method.request.querystring.p1": false,
"method.request.querystring.p2": false
},
"methodResponses": {
"200": {
"statusCode": "200",
"responseParameters": {
"method.response.header.Access-Control-Allow-Origin": false
},
"responseModels": {
"application/json": "Empty"
}
}
},
"methodIntegration": {
"type": "AWS_PROXY",
"httpMethod": "POST",
"uri": "arn:aws:apigateway:us-east-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-2:<internal id code>:function:retrieveEmailConfStringAPI/invocations",
"passthroughBehavior": "WHEN_NO_MATCH",
"contentHandling": "CONVERT_TO_TEXT",
"timeoutInMillis": 29000,
"cacheNamespace": "<resource id>",
"cacheKeyParameters": [],
"integrationResponses": {
"200": {
"statusCode": "200",
"responseParameters": {
"method.response.header.Access-Control-Allow-Origin": "'*'"
},
"responseTemplates": {
"application/json": null
}
}
}
}
}
Finally, I ran this command in order to create a public API call:
MacBook-Pro-2:lambdaZipFilesDev chauncey$ aws2 apigateway update-method --rest-api-id <rest-api-id> --resource-id <resource-id> --http-method GET --patch-operation op="replace",path="/authorizationType",value="NONE"
{
"httpMethod": "GET",
"authorizationType": "NONE",
"apiKeyRequired": false,
"requestParameters": {
"method.request.querystring.p1": false,
"method.request.querystring.p2": false
},
"methodResponses": {
"200": {
"statusCode": "200",
"responseParameters": {
"method.response.header.Access-Control-Allow-Origin": false
},
"responseModels": {
"application/json": "Empty"
}
}
},
"methodIntegration": {
"type": "AWS_PROXY",
"httpMethod": "POST",
"uri": "arn:aws:apigateway:us-east-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-2:<aws-internal-id>:function:retrieveEmailConfStringAPI/invocations",
"passthroughBehavior": "WHEN_NO_MATCH",
"contentHandling": "CONVERT_TO_TEXT",
"timeoutInMillis": 29000,
"cacheNamespace": "<resource-id>",
"cacheKeyParameters": [],
"integrationResponses": {
"200": {
"statusCode": "200",
"responseParameters": {
"method.response.header.Access-Control-Allow-Origin": "'*'"
},
"responseTemplates": {
"application/json": null
}
}
}
}
}
I just need to click the link and then for that to trigger my api link to my lambda function and for it to ahve some action on the backend.
The answer to this question is really simple and stupid.
https://<api-gateway-code>.execute-api.us-east-2.amazonaws.com/default/retrieveEmailConfStringAPI?p1=value1&p2=value2
The change is the ? as opposed to the & in the parameter string.

Kubernetes PetSet - PersistentVolumeClaim "ProvisioningFailed" on AWS

I’m in the process of migrating ElasticSearch to a Kubernete’s PetSet but there’s a problem when provisioning the PersistentVolumeClaim.
I have two volumes in AWS “vol-84094e30” “vol-87e59f33”. Both are 5 GiB, 100/3000 IOPS, us-west-2a Availability Zone and are volume type gp2.
I have a StorageClass definition for the volume type:
{
"kind": "StorageClass",
"apiVersion": "apps/v1alpha1",
"metadata": {
"name": "fast"
},
"provisioner": "kubernetes.io/aws-ebs",
"parameters": {
"type": "gp2",
"zone": "us-west-2a"
}
}
… two PersistentVolume definitions:
{
"kind": "PersistentVolume",
"apiVersion": "v1",
"metadata": {
"name": "es-persistent-vol",
"labels": {
"type": "amazonEBS"
}
},
"spec": {
"capacity": {
"storage": "5Gi"
},
"accessModes": [
"ReadWriteOnce"
],
"awsElasticBlockStore": {
"volumeID": "vol-87e59f33",
"fsType": "ext4"
}
}
}
{
"kind": "PersistentVolume",
"apiVersion": "v1",
"metadata": {
"name": "es-persistent-vol2",
"labels": {
"type": "amazonEBS"
}
},
"spec": {
"capacity": {
"storage": "5Gi"
},
"accessModes": [
"ReadWriteOnce"
],
"awsElasticBlockStore": {
"volumeID": "vol-84094e30",
"fsType": "ext4"
}
}
}
… an Elastic Search service:
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
"name": "elasticsearch-logging",
"namespace": "kube-system",
"labels": {
"k8s-app": "elasticsearch-logging",
"kubernetes.io/name": "Elasticsearch"
}
},
"spec": {
"ports": [
{
"port": 9200,
"protocol": "TCP",
"targetPort": "db"
}
],
"selector": {
"k8s-app": "elasticsearch-logging"
}
}
}
… and an Elastic Search PetSet:
{
"apiVersion": "apps/v1alpha1",
"kind": "PetSet",
"metadata": {
"name": "elasticsearch-logging-v1",
"namespace": "kube-system",
"labels": {
"k8s-app": "elasticsearch-logging",
"version": "v1",
"kubernetes.io/cluster-service": "true"
}
},
"spec": {
"serviceName": "elasticsearch-logging-v1",
"replicas": 2,
"template": {
"metadata": {
"annotations": {
"pod.beta.kubernetes.io/initialized": "true"
},
"labels": {
"app": "elasticsearch-data"
}
},
"spec": {
"containers": [
{
"image": "gcr.io/google_containers/elasticsearch:v2.4.1",
"name": "elasticsearch-logging",
"resources": {
"limits": {
"cpu": "1000m"
},
"requests": {
"cpu": "100m"
}
},
"ports": [
{
"containerPort": 9200,
"name": "db",
"protocol": "TCP"
},
{
"containerPort": 9300,
"name": "transport",
"protocol": "TCP"
}
],
"volumeMounts": [
{
"name": "es-persistent-storage",
"mountPath": "/data"
}
]
}
]
}
},
"volumeClaimTemplates": [
{
"metadata": {
"name": "es-persistent-storage",
"annotations": {
"volume.beta.kubernetes.io/storage-class": "fast"
},
"labels": {
"type": "amazonEBS"
}
},
"spec": {
"accessModes": [
"ReadWriteOnce"
],
"resources": {
"requests": {
"storage": "5Gi"
}
}
}
}
]
}
}
When I create all these the PersistentStorageVolume (defined in PetSet) is unable to provision the volumes (ProvisioningFailed - no volume plugin matched). I think it may be AWS specific, I’ve looked on various forums for (what I thought were) the same issue, but were not applicable to AWS.
Any help is much appreciated.
Here are the kubectl describe outputs:
$ kubectl describe pv
Name: es-persistent-vol
Labels: type=amazonEBS
Status: Available
Claim:
Reclaim Policy: Retain
Access Modes: RWO
Capacity: 5Gi
Message:
Source:
Type: AWSElasticBlockStore (a Persistent Disk resource in AWS)
VolumeID: vol-87e59f33
FSType: ext4
Partition: 0
ReadOnly: false
No events.
Name: es-persistent-vol2
Labels: type=amazonEBS
Status: Available
Claim:
Reclaim Policy: Retain
Access Modes: RWO
Capacity: 5Gi
Message:
Source:
Type: AWSElasticBlockStore (a Persistent Disk resource in AWS)
VolumeID: vol-84094e30
FSType: ext4
Partition: 0
ReadOnly: false
No events.
$ kubectl describe pvc
Name: es-persistent-storage-elasticsearch-logging-v1-0
Namespace: kube-system
Status: Pending
Volume:
Labels: app=elasticsearch-data
Capacity:
Access Modes:
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
18s 1s 3 {persistentvolume-controller } Warning ProvisioningFailed no volume plugin matched
Name: es-persistent-storage-elasticsearch-logging-v1-1
Namespace: kube-system
Status: Pending
Volume:
Labels: app=elasticsearch-data
Capacity:
Access Modes:
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
19s 2s 3 {persistentvolume-controller } Warning ProvisioningFailed no volume plugin matched
$ kubectl describe petset
Name: elasticsearch-logging-v1
Namespace: kube-system
Image(s): gcr.io/google_containers/elasticsearch:v2.4.1
Selector: app=elasticsearch-data
Labels: k8s-app=elasticsearch-logging,kubernetes.io/cluster-service=true,version=v1
Replicas: 2 current / 2 desired
Annotations: <none>
CreationTimestamp: Wed, 02 Nov 2016 13:06:23 +0000
Pods Status: 0 Running / 1 Waiting / 0 Succeeded / 0 Failed
No volumes.
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 1m 1 {petset } Normal SuccessfulCreate pvc: es-persistent-storage-elasticsearch-logging-v1-0
1m 1m 1 {petset } Normal SuccessfulCreate pet: elasticsearch-logging-v1-0
1m 1m 1 {petset } Normal SuccessfulCreate pvc: es-persistent-storage-elasticsearch-logging-v1-1
$ kubectl describe service el
Name: elasticsearch-logging
Namespace: kube-system
Labels: k8s-app=elasticsearch-logging
kubernetes.io/name=Elasticsearch
Selector: k8s-app=elasticsearch-logging
Type: ClusterIP
IP: 192.168.157.15
Port: <unset> 9200/TCP
Endpoints: <none>
Session Affinity: None
No events.