Connecting to AWS Kinesis with erlang library kinetic - amazon-web-services

Question

I am trying to connect to kinesis using the erlang library kinetic. https://github.com/AdRoll/kinetic... my development.config has my aws key and secret in it, however I am not sure what the metadata_base_url should be or what else I need in order to make it work...currently i have:
%% -*- erlang -*-
[{kinetic,
[{args, [
% All of these values are optional
% kinetic will get all of the context from the instance
{metadata_base_url, "https://kinesis.us-east-1.amazonaws.com"},
{aws_access_key_id, "mykey"},
{aws_secret_access_key, "mysecret"},
{iam_role, "kinetic"},
{lhttpc_opts, [{max_connections, 5000}]}
]}]
}].
Below is my results when trying to start it...
kinetic (master) $ make
==> lhttpc (get-deps)
==> jiffy (get-deps)
==> meck (get-deps)
==> kinetic (get-deps)
==> lhttpc (compile)
==> jiffy (compile)
==> meck (compile)
==> kinetic (compile)
Compiled src/kinetic.erl
kinetic (master) $ erl -pa ebin -pa deps/*/ebin -s inets -s crypto -s ssl -s lhttpc -config development -s kinetic
Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Eshell V5.10.4 (abort with ^G)
1>
=INFO REPORT==== 2-Dec-2014::11:51:31 ===
application: kinetic
exited: {{shutdown,
{failed_to_start_child,kinetic_config,
{{badmatch,{error,403}},
[{kinetic_config,new_args,1,
[{file,"src/kinetic_config.erl"},{line,127}]},
{kinetic_config,update_data,1,
[{file,"src/kinetic_config.erl"},{line,42}]},
{kinetic_config,init,1,
[{file,"src/kinetic_config.erl"},{line,55}]},
{gen_server,init_it,6,
[{file,"gen_server.erl"},{line,304}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}}},
{kinetic,start,[normal,[]]}}
type: temporary
When removed the base_url config...
=ERROR REPORT==== 2-Dec-2014::12:41:30 ===
{failed_connect,[{to_address,{"169.254.169.254",80}},{inet,[inet],etimedout}]}
=INFO REPORT==== 2-Dec-2014::12:41:30 ===
application: kinetic
exited: {{shutdown,
{failed_to_start_child,kinetic_config,
{{badmatch,
{error,
{failed_connect,
[{to_address,{"169.254.169.254",80}},
{inet,[inet],etimedout}]}}},
[{kinetic_config,new_args,1,
[{file,"src/kinetic_config.erl"},{line,127}]},
{kinetic_config,update_data,1,
[{file,"src/kinetic_config.erl"},{line,42}]},
{kinetic_config,init,1,
[{file,"src/kinetic_config.erl"},{line,55}]},
{gen_server,init_it,6,
[{file,"gen_server.erl"},{line,304}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}}},
{kinetic,start,[normal,[]]}}
type: temporary

Answer 1

It seems that in case of running kinetic application outside of ec2 cluster you need to specify region in config:
[{kinetic,
[{args, [
{region, "us-east-1"}, %% just an example
...
]}]
}].
and use fixed version of kinetic which won't be trying to discover region.
Second solution is to set metadata_base_url option to your http service which on get request for "/latest/meta-data/placement/availability-zone" will respond with your region.
I've never used aws and some of statements might be improper.

Answer 2

The error message says that the access to https://kinesis.us-east-1.amazonaws.com/latest/meta-data/placement/availability-zone return an error 403: Forbidden. Reading http documentation it means that the authentication was correct but that you do not have the right access to this ressource.