We are hosting several projects with separate Hosted Origins on a single cloudfront. The cloudfront has a default behavior that points to Origin1 .
We can access this by going to website.com
There is a second behavior for the subdirectory /experience that should point to another origin, Origin2 .
The cloudfront uses a Cloudfront function to handle the trailing / as follows
function handler(event) {
var request = event.request;
var uri = request.uri;
// Check whether the URI is missing a file name.
if (uri.endsWith('/')) {
request.uri += 'index.html';
}
// Check whether the URI is missing a file extension.
else if (!uri.includes('.')) {
request.uri += '/index.html';
}
return request;
}
When going to website.com/experience/ the expected result of Origin2 happens. However, if you leave off the trailing slash with website.com/experience the Origin seems to point to Origin1 . The index.html of /experience is called, but the file paths in that folder expect the root to be /experience, however the origin still points to Origin1. Is there any way to set the behavior of /experience to point to the correct origin?
I looked into the Cloudfront functions to do a redirect, but it looks like you cannot change the origin.
When a function changes the uri value, it doesn’t change the cache behavior for the request or the origin that an origin request is sent to.
From Cloudfront Functrions event Structure | Request Object
We also set up a static hosted s3 Bucket to host a project in, and the same results happen with a subdirectory of /experience in that Bucket. The origin path on the cloudfront had a path pattern of /experience* and the origin path was /experience
AWS Buckets are "linear" so there is no heirarchy, but I would expect if the directory name is there, or the cloudfront behavior points to the correct origin that there wouldn't be an issue.
This is the expected behavior. Like you said, changing the uri doesn't change the origin.
You can do a URL redirect to handle the website.com/experience path:
function handler(event) {
var request = event.request;
var uri = request.uri;
// Check whether the URI is missing a file name.
if (uri.endsWith('/')) {
request.uri += 'index.html';
}
// Check whether the URI is missing a file extension.
else if (!uri.includes('.')) {
request.uri += '/index.html';
}
// Check /experience path
else if (uri === '/experience') {
var response = {
statusCode: 302,
statusDescription: 'Found',
headers:
{ "location": { "value": Origin2-PATH } }
}
return response;
}
return request;
}
I am looking to add Basic User Authentication to a Static Site I will have up on AWS so that only those with the proper username + password which I will supply to those users have access to see the site. I found s3auth and it seems to be exactly what I am looking for, however, I am wondering if I will need to somehow set the authorization for pages besides the index.html. For example, I have 3 pages- index, about and contact.html, without authentication setup for about.html what is stopping an individual for directly accessing the site via www.mywebsite.com/about.html? I am more so looking for clarification or any resources anyone can provide to explain this!
Thank you for your help!
This is the perfect use for Lambda#Edge.
Because you're hosting your static site on S3, you can easily and very economically (pennies) add some really great features to your site by using CloudFront, AWS's content distribution network, to serve your site to your users. You can learn how to host your site on S3 with CloudFront (including 100% free SSL) here.
While your CloudFront distribution is deploying, you'll have some time to go set up your Lambda that you'll be using to do the basic user auth. If this is your first time creating a Lambda or creating a Lambda for use #Edge the process is going to feel really complex, but if you follow my step-by-step instructions below you'll be doing serverless basic-auth that is infinitely scalable in less than 10 minutes. I'm going to use us-east-1 for this and it's important to know that if you're using Lambda#Edge you should author your functions in us-east-1, and when they're associated with your CloudFront distribution they'll automagically be replicated globally. Let's begin...
Head over to Lambda in the AWS console, and click on "Create Function"
Create your Lambda from scratch and give it a name
Set your runtime as Node.js 8.10
Give your Lambda some permissions by selecting "Choose or create an execution role"
Give the role a name
From Policy Templates select "Basic Lambda#Edge permissions (for CloudFront trigger)"
Click "Create function"
Once your Lambda is created take the following code and paste it in to the index.js file of the Function Code section - you can update the username and password you want to use by changing the authUser and authPass variables:
'use strict';
exports.handler = (event, context, callback) => {
// Get request and request headers
const request = event.Records[0].cf.request;
const headers = request.headers;
// Configure authentication
const authUser = 'user';
const authPass = 'pass';
// Construct the Basic Auth string
const authString = 'Basic ' + new Buffer(authUser + ':' + authPass).toString('base64');
// Require Basic authentication
if (typeof headers.authorization == 'undefined' || headers.authorization[0].value != authString) {
const body = 'Unauthorized';
const response = {
status: '401',
statusDescription: 'Unauthorized',
body: body,
headers: {
'www-authenticate': [{key: 'WWW-Authenticate', value:'Basic'}]
},
};
callback(null, response);
}
// Continue request processing if authentication passed
callback(null, request);
};
Click "Save" in the upper right hand corner.
Now that your Lambda is saved it's ready to attach to your CloudFront distribution. In the upper menu, select Actions -> Deploy to Lambda#Edge.
In the modal that appears select the CloudFront distribution you created earlier from the drop down menu, leave the Cache Behavior as *, and for the CloudFront Event change it to "Viewer Request", and finally select/tick "Include Body". Select/tick the Confirm deploy to Lambda#Edge and click "Deploy".
And now you wait. It takes a few minutes (15-20) to replicate your Lambda#Edge across all regions and edge locations. Go to CloudFront to monitor the deployment of your function. When your CloudFront Distribution Status says "Deployed", your Lambda#Edge function is ready to use.
Deploying Lambda#edge is quiet difficult to replicate via console. So I have created CDK Stack that you just add your own credentials and domain name and deploy.
https://github.com/apoorvmote/cdk-examples/tree/master/password-protect-s3-static-site
I have tested the following function with Node12.x
exports.handler = async (event, context, callback) => {
const request = event.Records[0].cf.request
const headers = request.headers
const user = 'my-username'
const password = 'my-password'
const authString = 'Basic ' + Buffer.from(user + ':' + password).toString('base64')
if (typeof headers.authorization === 'undefined' || headers.authorization[0].value !== authString) {
const response = {
status: '401',
statusDescription: 'Unauthorized',
body: 'Unauthorized',
headers: {
'www-authenticate': [{key: 'WWW-Authenticate', value:'Basic'}]
}
}
callback(null, response)
}
callback(null, request)
}
By now, this is also possible with CloudFront functions which I like more because it reduces the complexity even more (from what is already not too complex with Lambda). Here's my writeup on what I just did...
It's basically 3 things that need to be done:
Create a CloudFront function to add Basic Auth into the request.
Configure the Origin of the CloudFront distribution correctly in a few places.
Activate the CloudFront function.
That's it, no particular bells & whistles otherwise. Here's what I've done:
First, go to CloudFront, then click on Functions on the left, create a new function with a name of your choice (no region etc. necessary) and then add the following as the code of the function:
function handler(event) {
var user = "myuser";
var pass = "mypassword";
function encodeToBase64(str) {
var chars =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
for (
// initialize result and counter
var block, charCode, idx = 0, map = chars, output = "";
// if the next str index does not exist:
// change the mapping table to "="
// check if d has no fractional digits
str.charAt(idx | 0) || ((map = "="), idx % 1);
// "8 - idx % 1 * 8" generates the sequence 2, 4, 6, 8
output += map.charAt(63 & (block >> (8 - (idx % 1) * 8)))
) {
charCode = str.charCodeAt((idx += 3 / 4));
if (charCode > 0xff) {
throw new InvalidCharacterError("'btoa' failed: The string to be encoded contains characters outside of the Latin1 range."
);
}
block = (block << 8) | charCode;
}
return output;
}
var requiredBasicAuth = "Basic " + encodeToBase64(`${user}:${pass}`);
var match = false;
if (event.request.headers.authorization) {
if (event.request.headers.authorization.value === requiredBasicAuth) {
match = true;
}
}
if (!match) {
return {
statusCode: 401,
statusDescription: "Unauthorized",
headers: {
"www-authenticate": { value: "Basic" },
},
};
}
return event.request;
}
Then you can test with directly on the UI and assuming it works and assuming you have customized username and password, publish the function.
Please note that I have found individual pieces of the function above on the Internet so this is not my own code (other than piecing it together). I wish I would still find the sources so I can quote them here but I can't find them anymore. Credits to the creators though! :-)
Next, open your CloudFront distribution and do the following:
Make sure your S3 bucket in the origin is configured as a REST endpoint and not a website endpoint, i.e. it must end on .s3.amazonaws.com and not have the word website in the hostname.
Also in the Origin settings, under "S3 bucket access", select "Yes use OAI (bucket can restrict access to only CloudFront)". In the setting below click on "Create OAI" to create a new OAI (unless you have an existing one and know what you're doing). And select "Yes, update the bucket policy" to allow AWS to add the necessary permissions to your OAI.
Finally, open your Behavior of the CloudFront distribution and scroll to the bottom. Under "Function associations", for "Viewer request" select "CloudFront Function" and select your newly created CloudFront function. Save your changes.
And that should be it. With a bit of luck a matter of a couple of minutes (realistically more, I know) and especially not additional complexity once this is all set up.
Thanks for the useful post. An alternative to listing the pain text user name and password in the code, and to having base64 encoding logic, is to pre-generate the base64 encoded string. One such encoder, https://www.debugbear.com/basic-auth-header-generator
From there the script becomes simpler. The following is for 'user' / 'password'
function handler(event) {
var base64UserPassword = "Y3liZXJmbG93c3VyZmVyOnRhbHR4cGNnIzIwMjI="
if (event.request.headers.authorization &&
event.request.headers.authorization.value === ("Basic " + base64UserPassword)) {
return event.request;
}
return {
statusCode: 401,
statusDescription: "Unauthorized ",
headers: {
"www-authenticate": { value: "Basic" },
},
}
}
Here is already exists answer how to use Cloudfront functions, but I want to add improved version of the function:
Hardcoded credentials stored as SHA256 hash instead of plain (or base64 that is the same as plain) text. And that is more secure.
It is possible to allow access by whitelisted global IP addresses:
function handler(event) {
var crypto = require('crypto');
var headers = event.request.headers;
var wlist_ips = [
"1.1.1.1",
"2.2.2.2"
];
var authString = "9c06d532edf0813659ab41d26ab8ba9ca53b985296ee4584a79f34fe9cd743a4";
if (
typeof headers.authorization === "undefined" ||
crypto.createHash(
'sha256'
).update(headers.authorization.value).digest('hex') !== authString
) {
if (
!wlist_ips.includes(event.viewer.ip)
) {
return {
statusCode: 401,
statusDescription: "Unauthorized",
headers: {
"www-authenticate": { value: "Basic" },
"x-source-ip": { value: event.viewer.ip}
}
};
}
}
return event.request;
}
Command below may be used to get correct authString hash value for username user and password password:
printf "Basic $(printf 'user:password' | base64 -w 0)" | sha256sum | awk '{print$1}'
For our AWS API Endpoints we use AWS_IAM authorization and want to make a call from Swagger UI.
To make a successful call there must be 2 headers 'Authorization' and 'x-amz-date'. To form 'Authorization' we use following steps from aws doc.
We must to change 'x-amz-date' with every call to go through authorization.
The question is: How to write script in Swagger to sign request, which run every time before request send to aws?
(We know how to specify both headers one time before loading Swagger page, but this process should be re-run before every call).
Thanks in advance.
There is built-in support in swagger-js to add requestInterceptors to do just this. The swagger-ui project uses swagger-js under the hood.
Simply create a request interceptor like such:
requestInterceptor: {
apply: function (request) {
// modify the request object here
return request;
}
}
and apply it to your swagger instance on creation:
window.swaggerUi = new SwaggerUi({
url: url,
dom_id: "swagger-ui-container",
requestInterceptor: requestInterceptor,
Here you can set headers in the request object (note, this is not the standard javascript http request object, inspect it for details). But you do have access to all headers here, so you can calculate and inject them as needed.
You can pretty easily monkeypatch signing from the AWS SDK into SwaggerJS(and thus SwaggerUI). See here
I have a slightly modified SwaggerUI here. Given some AWS credentials and an API ID, it will pull down the Swagger definition, display it in SwaggerUI, and then you can call the API using sigv4.
The Authorizer implementation looks like this:
var AWSSigv4RequestSigner = function(credentialProvider, aws) {
this.name = "sigv4";
this.aws = aws;
this.credentialProvider = credentialProvider;
};
AWSSigv4RequestSigner.prototype.apply = function(options, authorizations) {
var serviceName = "execute-api";
//If we are loading the definition itself, then we need to sign for apigateway.
if (options && options.url.indexOf("apigateway") >= 0) {
serviceName = "apigateway";
}
if(serviceName == "apigateway" || (options.operation && options.operation.authorizations && options.operation.authorizations[0].sigv4))
{
/**
* All of the below is an adapter to get this thing into the right form for the AWS JS SDK Signer
*/
var parts = options.url.split('?');
var host = parts[0].substr(8, parts[0].indexOf("/", 8) - 8);
var path = parts[0].substr(parts[0].indexOf("/", 8));
var querystring = parts[1];
var now = new Date();
if (!options.headers)
{
options.headers = [];
}
options.headers.host = host;
if(serviceName == "apigateway")
{
//For the swagger endpoint, apigateway is strict about content-type
options.headers.accept = "application/json";
}
options.pathname = function () {
return path;
};
options.methodIndex = options.method;
options.search = function () {
return querystring ? querystring : "";
};
options.region = this.aws.config.region || 'us-east-1';
//AWS uses CAPS for method names, but swagger does not.
options.method = options.methodIndex.toUpperCase();
var signer = new this.aws.Signers.V4(options, serviceName);
//Actually add the Authorization header here
signer.addAuthorization(this.credentialProvider, now);
//SwaggerJS/yourbrowser complains if these are still around
delete options.search;
delete options.pathname;
delete options.headers.host;
return true;
}
return false;
};
I have the following Javascript code to make a XMLHttpRequest to a server:
function createCORSRequest(method, url) {
var xhr = new XMLHttpRequest();
xhr.open(method, url, true);
return xhr;
}
function makeCorsRequest(word) {
var url = "https://localhost:8080/Method/Dictionary/" + word;
var xhr = createCORSRequest('GET', url);
xhr.onload = function() {
var responseText = xhr.responseText;
document.querySelector("#bar").innerHTML = responseText;
};
xhr.onerror = function() {
document.querySelector("#bar").innerHTML = 'Connection not allowed';
};
xhr.send();
}
makeCorsRequest("word");
At the server, I've got a REST structure (written using Jersey) similiar to:
#Path("/Dictionary")
public class Main{
public Definition returnDefinition(String word){
Definition definition = new Definition();
try{
...//play with Definition object
return definition;
}
catch(IOException IOE){
...
return definition;
}
}
#Path("{word}")
#GET
#Produces("text/xml ; charset=UTF-8") //"Definition" is already set as a XMLRoot element
public Definition main (#PathParam("word") String word){
return returnDefinition(word);
}
}
I try to make this request in two environments:
First environment: The JS code is inside a normal web page. In this case, I receive the following error after trying to make the request:
XMLHttpRequest cannot load http://localhost:8080/Method/Dictionary/word. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.
Second environment: The JS code is inside a content script (that is itself inside a chrome extension). In this case, after about 30s of trying to make the request, I receive this:
Failed to load resource: net::ERR_TIMED_OUT
How to proceed?
EDIT: I've added a command to print something at the console in the beginning of the server method. And it is not printed. So, the requests are not reaching the server.
Origin 'null' happens if you are running the page as a file in the browser, using the file:// protocol, rather than set it up in a local webserver and access it using http://.
I want to strip out values from a url in varnish, so I can take different actions based on the url, for example:
URL: /product/123/price/available.
I'd like to convert this to /products?id=123&sort=price&available=true
I would also like to be able to (if this is possible), set values on the request header, so instead of passing all the params on the URL, I could do the following:
/products?id=123&sort=price
with header: x-show-available-only: true
I appreciate the second example appears a bit odd, but this way we could pass new params back to our legacy application, and ensure none of the new params interfere with current params - we would just read new params via the header, until we migrate all our functionality to our new platform.
I'm sure it's a regex thing, but can't work out how to do it.
This is NOT tested, but should be close enough that you can tweak it to get it right.
Based on your URL:
/product/123/price/available
You'll need to specify two backends (change the IPs for your own ones):
backend old_app {
.host = "127.0.0.1";
.port = "80";
}
backend new_app {
.host = "127.0.0.2";
.port = "80";
}
And the code in your vcl_recv:
if (req.url ~ "^/product/") {
set req.url = regsub(req.url, "/product/([0-9]/)/([a-zA-Z])", "/products?id=\1&sort=\2");
set req.backend = old_app;
} else {
set req.backend = new_app.
}
if (req.url ~ "/available") {
set req.http.x-show-available-only = "true";
}
You can add more regex rules as if/else blocks.
Good luck!