When the user has many roles associated with it, the Authorization bearer token will exceed the max header size provided by istio which blocks all the requests.
The authorization bearer token has more than 35,000 characters with 34K size.
How can I increase the max header size provided by istio?
As far as I know since istio 1.5 you would use Control Headers and Routing
But as mentioned in above link
The mixer policy is deprecated in Istio 1.5 and not recommended for production usage.
Consider using Envoy ext_authz filter, lua filter, or write a filter using the Envoy-wasm sandbox.
As mentioned in envoy documentation, you can use max_request_headers to increase your header size.
max_request_headers_kb
The maximum request headers size for incoming connections. If unconfigured, the default max request headers allowed is 60 KiB. Requests that exceed this limit will receive a 431 response. The max configurable limit is 96 KiB, based on current implementation constraints.
As mentioned above you could use envoy filter to change it.
EnvoyFilter provides a mechanism to customize the Envoy configuration generated by Istio Pilot. Use EnvoyFilter to modify values for certain fields, add specific filters, or even add entirely new listeners, clusters, etc. This feature must be used with care, as incorrect configurations could potentially destabilize the entire mesh. Unlike other Istio networking objects, EnvoyFilters are additively applied. Any number of EnvoyFilters can exist for a given workload in a specific namespace. The order of application of these EnvoyFilters is as follows: all EnvoyFilters in the config root namespace, followed by all matching EnvoyFilters in the workload’s namespace.
Take a look at example with max_request_headers here.
And you can find more examples with envoy filter here and here.
Related
I am using istio 1.9.0.
Only during an issue did I discover that Service Entries are applied at a cluster level even if the namespace is specified in the manifest. This becomes particularly thorny if you have multiple Service Entries in different namespaces for the same hostnames.
I later found the docs that mention the following:
exportTo:
A list of namespaces to which this service is exported. Exporting a
service allows it to be used by sidecars, gateways and virtual
services defined in other namespaces. This feature provides a
mechanism for service owners and mesh administrators to control the
visibility of services across namespace boundaries.
If no namespaces are specified then the service is exported to all
namespaces by default.
The value “.” is reserved and defines an export to the same namespace
that the service is declared in. Similarly the value “*” is reserved
and defines an export to all namespaces.
From here.
Q1.
The docs however explicitly fails to clarify if adding the exportTo=’.’ ensures that a Service Entry in my namespace will always take priority. Is this implied? particularly interested to find some documentation that calls out the expected behaviour.
Q2.
Also how do you go about checking which Service Entry has been applied for the same hostname if multiple exist? How does istio handle this?
The problem is related to ServiceEntry concept design and it's quite complex.
My understanding of the current situation is that one should have an explicitly defined ServiceEntry in the certain namespace to prevent Istio from searching other ServiceEntries in another namespaces for the same host endpoint.
More information about the problem you can find in the issue #13008 :
A ServiceEntry from one namespace can interfere with the traffic of another namespace #13008
The changes was made in the PR #13631 :
Make ServiceEntry follow Sidecar isolation #13631
See the design doc for more details. I'd recommend to read the whole document to see a full picture.
The main point of the change is to implement the following:
Proposed Behavior
Pilot will internally be modified to key Services by a hostname and namespace pair, rather than just hostname. When determining which Services to use for a hostname we will follow this resolution:
If the hostname exists in the client's namespace, use only that namespace's service.
If the hostname exists only in one namespace imported by the Sidecar, use that namespace's service.
Otherwise, if there are multiple namespaces with services, an arbitrary one can be chosen (based on creation timestamp, like other configs).
The end result of this is that while hostnames are not distinct at a global level, any given proxy will have a single unique hostname -> service mapping.
Additionally, a ServiceEntry for an internal service will be rejected if it is created in different namespace. For example, if foo.ns1.svc.cluster.local was defined in namespace ns2, it would be rejected. This prevents the case of a user importing [ns1/, ns2/] and having their requests to foo.ns1.svc.cluster.local hijacked.
Unfortunately I wasn't able to find the final state of the implemented changes for the recent Istio version.
Not having a minimal reproducible example from you doesn't allow me to check if you were facing the implementation flaw or your configuration is a corner case that the change wasn't suppose to solve.
It's sad that current Istio documentation doesn't explain this aspect well enough.
I am trying to take advantage of the built-in Cloud Storage edge caching feature. When a valid Cache-Control header is set, the files can be stored at edge locations. This is without having to set up Cloud Load Balancer & CDN. This built-in behavior is touched on in this Cloud Next '18 video.
What I am seeing though is a hard limit of 10 MB. When I store a file over 10 MB and then download it, it's missing the Age response header. A 9 MB file will have it. The 10 MB limit is mentioned in the CDN docs here, though. What doesn't make sense to me is why files over 10 MB don't get cached to the edge. After all, the Cloud Storage server meets all the requirements, it even says: Cloud Storage supports byte range requests for most objects.
Does anyone know more about the default caching limits? I can't seem to find any limits documented for Cloud Storage.
At the moment, the information of the cache limit managed by the objects in Cloud Storage is documented in the Cloud DN documentation and what you describe is an expected behavior.
Origin server does not support byte range requests: 10 MB (10,485,760
bytes)
Performing tests on my side with files of an exact size of 10,485,760 bytes include the Age field.
However files above that limit such as 10,485,770 no longer include it.
I recommend you create a feature request here in order to improve the Google Cloud Storage documentation.
In this way you will have direct communication with the team responsible for the documentation and your request may be supported by other members of the community.
What are the advatages of the jaeger tracing with istio and without istio?
For example with istio it will reduce the latency for collecting the more traces
If you are already using Istio in the deployment, then enabling tracing in it will provide more complete picture of request processing, such as accounting for the time spent in the network between the proxies.
You also don't need to have full tracing instrumentation in your services as long as they pass through certain headers, then Istio can still provide a pretty accurate picture of the traces (but you cannot capture any business specific data in the traces).
Traces generated by Istio will have standardized span names that you can use to reason about the SLAs across the whole infrastructure, whereas explicit tracing instrumentation inside the services can often use different naming schemes, especially when services are written in different languages and using different frameworks.
For the best of both worlds, I would recommend adding instrumentation inside the services for full fidelity, and also enabling tracing in Istio to capture full picture of request execution (and all network latencies).
I want to start by saying that I'm a total newcomer to AWS.
I'm investigating using AWS WAF for dynamic rate limiting based on a component of the request URL. The AWS website has a tutorial for doing this by IP address, but I have no idea if it can be modified to do what I need.
So, with that in mind, please tell me what, if any, of the following is actually possible:
Rate limit by a component of the URL (an API key in this case)
Determine limit dynamically (different behaviour for different keys)
Perform some non-blocking action in the first instance of exceeding
the limit, then block if the limit is exceeded consistently
Log both of the above actions and do something with the outputted logs (i.e. forward them somewhere)
Again, I'm not looking for detailed how-tos here as they would probably warrant seperate questions - just want to know if this is possible.
API Gateway is probably the right fit for what you are looking to implement. It has throttling implemented out of the box.
Take a look at API Gateway Usage Plans for implementation details for your specific use case.
We are using Google Drive API to upload files and update permissions in our application. Requirement is to update the permissions ~60 users/groups.
There are three ways by which we can update permissions on a file :
Use File's Patch Endpoint
Use File's Update Endpoint
Use Permissions's Insert Endpoint
If we go with #3, we have to make ~60 calls based on the permission change which is not good actually as it has to make that much http calls and it affects the quota usage.
So we tried with #1, we provide the necessary input in "permissions" key. It returns 200 but the file is not shared as per the given input.
Is there anything that I am missing ?
Permissions.Insert is the only way to add permissions to a file; it's not feasible via operations on the Files API.
The Google Drive API does however support batching, which means that instead of sending 60 separate HTTP requests, you can send a single batch that contains 60 requests. This won't help with quota, but will likely perform better. More information here:
https://developers.google.com/drive/v3/web/batch