NPM Verdaccio - set uplink to own remote server - amazon-web-services

I just built an NPM Verdaccio private registry server within our local network and I would like configure an UPLINK to our remote NPM Verdaccio server which is hosted at AWS (and also keep the original npmjs registry).
snippet from Verdaccio config.yaml
uplinks:
npmjs:
url: https://registry.npmjs.org/
our-NPM-AWS-server:
url: https://our-NPM-AWS-server.com
based on the documentation (Verdaccio_UPLINK), I have to set the Authentication parameters there, anyhow.
I found the usage of the UPLINKS here - uplink authorization & here - getting an Auth Token , but it is pretty confusing for me because I am not sure what to set as an AUTH method:
auth:
token:
type: bearer | basic,
token: "token",
token_env: true | <get name process.env> `NPM_TOKEN`
I was not able to find any tutorial which would guide me, so I would like to ask for some insight & help - what is necessary to set on the Internal NPM server & also on the remote NPM AWS server.
Configuration:
Internal NPM server
ubuntu 16.04, node v8.11.1, npm v5.8, Verdaccio v.2.7.4, access is controlled by .htpasswd, NPM is accessible on port 80 (listens on http://127.0.0.1:4873)
Remote own NPM server at AWS
ubuntu 14.04, node v6.14.1, npm v3.10.10, Verdaccio v.2.7.4, access is controlled by .htpasswd, NPM is accessible only via 443 from the outside (proxy_http listens on http://127.0.0.1:4873 with an url_prefix: https://our-NPM-AWS-server.com)
Both servers are operating normally (you can log there with your NPM account, push the packages, etc).
thank you very much
EDIT 2018-04-26
The AWS NPM server is registered into Application ELB, which listens on port 443. The AWS NPM server listens on port 443 and is located in private subnet.
I tried to place AWS Verdaccio instance into public subnet and to access it directly without ELB, however it didnt have any affect and the behavior was same.
The config.yaml file of AWS NPM
The UPLINKS part was not changed
packages:
'#*/*':
# scoped packages
access: $all
publish: $authenticated
proxy: npmjs
'**':
# allow all users (including non-authenticated users) to read and
# publish all packages
#
# you can specify usernames/groupnames (depending on your auth plugin)
# and three keywords: "$all", "$anonymous", "$authenticated"
access: $authenticated
# allow all known users to publish packages
# (anyone can register by default, remember?)
publish: $authenticated
# if package is not available locally, proxy requests to 'npmjs' registry
proxy: npmjs
I tried to set
'**':
access: $all
However, it didnt have any effect.
The config.yaml of Internal Verdaccio Server
uplinks:
aws:
url: https://our-NPM-AWS-server.com/
#strictUrlMatch: false
headers:
authorization: "Basic <token_which_I_harvested_from_/.npmrc_file>"
packages:
'#*/*':
# scoped packages
access: $all
publish: $authenticated
proxy: aws
'**':
# allow all users (including non-authenticated users) to read and
# publish all packages
#
# you can specify usernames/groupnames (depending on your auth plugin)
# and three keywords: "$all", "$anonymous", "$authenticated"
access: $all
# allow all known users to publish packages
# (anyone can register by default, remember?)
publish: $authenticated
# if package is not available locally, proxy requests to 'npmjs' registry
proxy: aws
On Internal Verdaccio instance, I tried to get some package from AWS Verdaccio instance
npm pack --verbose verdaccio-bitbucket
and this is log from AWS Verdaccio:
{"name":"verdaccio","hostname":"hostname_our-NPM-AWS-server","pid":8494,"sub":"in",
"level":30,"req":{"method":"GET","url":"/verdaccio-bitbucket",
"headers":{"host":"our-NPM-AWS-server.com","x-forwarded-for"
:"Public_IP_of_Internal_Verdaccio, 10.XXX.XX.XXX","x-forwarded-proto"
:"https","x-forwarded-port":"443","x-amzn-trace-id":
"Root=X-XXXXXX-XXXXXXXXXXXXXXXX","accept":"application/json;",
"accept-encoding":"gzip","user-agent":"npm (verdaccio/2.7.4)",
"via":"1.1 f8d74eab3cc6 (Verdaccio)","authorization":"<Classified>",
"x-forwarded-host":"our-NPM-AWS-server.com",
"x-forwarded-server":"our-NPM-AWS-server.com","connection":"Keep-Alive"},
"remoteAddress":"127.0.0.1","remotePort":42608},"ip":"127.0.0.1",
"msg":"#{ip} requested '#{req.method} #{req.url}'",
"time":"2018-04-26T20:12:38.893Z","v":0}
{"name":"verdaccio","hostname":"hostname_our-NPM-AWS-server","pid":8494,"sub":"in",
"level":35,"request":{"method":"GET","url":"/verdaccio-bitbucket"},
"remoteIP":"Public_IP_of_Internal_Verdaccio, 10.XXX.XX.XXX via
127.0.0.1","**status":403,"error":"unregistered users are not allowed
to access package verdaccio-bitbucket"**,"bytes":
"in":0,"out":180},"msg":"#{status}, user: #{user}(#{remoteIP}),
req: '#{request.method} #{request.url}', error: #{!error}",
"time":"2018-04-26T20:12:38.895Z","v":0}
and this is log from Internal Verdaccio, where the command was ran from:
http --> 200, req: 'GET https://our-NPM-AWS-server.com/verdaccio-bitbucket' (streaming)
http --> 200, req: 'GET https://our-NPM-AWS-server.com/verdaccio-bitbucket', bytes: 0/34578
http <-- 200, user: <npm_account>(127.0.0.1), req: 'GET /verdaccio-bitbucket', bytes: 0/5038
http <-- 500, user: <npm_account>(127.0.0.1), req: 'GET /verdaccio-bitbucket/-/verdaccio-bitbucket-1.0.0.tgz', error: bad uplink status code: 403
http <-- 500, user: <npm_account>(127.0.0.1), req: 'GET /verdaccio-bitbucket/-/verdaccio-bitbucket-1.0.0.tgz', error: bad uplink status code: 403
http <-- 500, user: <npm_account>(127.0.0.1), req: 'GET /verdaccio-bitbucket/-/verdaccio-bitbucket-1.0.0.tgz', error: bad uplink status code: 403

Your configuration is correct but slightly wrong. Let me fix it.
uplinks:
aws:
url: https://our-NPM-AWS-server.com/
#strictUrlMatch: false
headers:
authorization: "Bearer <token_which_I_harvested_from_/.npmrc_file>"
Do not use Basic, it is Bearer verdaccio uses JWT, unfortunately, verdaccio middleware does not accept bearer in lowercase (it does since verdaccio#v3.0.0-beta.7).
For clarification about Basis and JWT. Since version verdaccio#2.3.0 all tokes are generated with JWT library, somehow, for legacy/unit testing reasons we still accept Basis authentification headers, but, all new tokens generated since verdaccio#2.3.0 must use Bearer in headers instead Basis.
There are a couple of issues I will report on Github, minor ones, but still, causes issues like this one.
I hope it helps.

Related

Conan fails to upload to Artifactory instance over HTTPS

Currently I have Artifactory set up through a system.yaml file
configVersion: 1
shared:
security:
exposeApplicationHeaders: true
node:
id: "*.example.com"
ip: artifacts.example.com
metrics:
enabled: true
artifactory:
#port: 8081
tomcat:
httpsConnector:
enabled: true
port: 8443
certificateFile: "$JFROG_HOME/artifactory/var/etc/artifactory/security/trusted/server2.crt"
certificateKeyFile: "$JFROG_HOME/artifactory/var/etc/artifactory/security/trusted/server.key"
frontend:
featureToggler:
commonProjects: true
And I'm able to access the webview on port 8082 through https just fine
I created a repo for conan artifacts and generated an api key. Then using the "set me up" prompt I ran the following commands on my dev machine
conan remote add myremote https://artifacts.example.com:8081/artifactory/api/conan/myremote
conan user -p <apikey> -r myremote will
I then get the following error from Conan
ERROR: HTTPSConnectionPool(host='artifacts.example.com', port=8081): Max retries exceeded with url: /artifactory/api/conan/myremote/v1/ping (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)')))
Unable to connect to myremote=https://artifacts.example.com:8081/artifactory/api/conan/myremote
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.
I tried to repeat the same steps but using http instead of http and all worked fine. What am I doing wrong that won't let https access work?

deploy nuxtjs application to GCP, but it gives server error?

I'm new to GCP (google cloud platform) I'm facing this problem, Backend dev created an Apache server on gcp, and then I cloned my repo on the server through VM, build it and then deployed it after following countless tutorials.. but while deploying it gives
Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable.
when i go to the generated link which is https://vetution.oa.r.appspot.com/ it gives server error, I've searched a lot but I failed.
Have I done something wrong or the app.yaml file should have different configurations.
anyway here it is
runtime: nodejs12
instance_class: F2
handlers:
- url: /_nuxt
static_dir: .nuxt/dist/client
secure: always
- url: /(.*\.(gif|png|jpg|ico|txt))$
static_files: static/\1
upload: static/.*\.(gif|png|jpg|ico|txt)$
secure: always
- url: /.*
script: auto
secure: always
env_variables:
HOST: '0.0.0.0'
NODE_ENV: 'production'
And It gives me this error on deploying now
please, Help me I'm very frustrated.
The error states that, your app failed to start because it's not listening to a port as you may not have configure it to your app. This topic is answered on this Stackoverflow thread.
You mentioned that you followed countless tutorials, it is possible that you are using Cloud Run and App Engine interchangeably, which are 2 separate serverless products. Please evaluate on your end which platform is right for your app and use case.
Currently, there is not much debugging info to your question but either way, make sure that you configure your app's port to listen to incoming request/s, it can be port 8080 or can be any of your choice. Also, check your GCP logs, backend issues and server errors can be seen and monitored by simply looking at the logs for App Engine or Cloud Run.
Check Cloud Run configuring the container ports, for port configuration.

docker exec cli peer channel create | failed to create new connection: context deadline exceeded | amazon managed blockchain

I am trying to setup hyperledger fabric blockchain network using amazon managed blockchain following this guide. In the step 6, to create the channel I have executed the following command,
docker exec cli peer channel create -c hrschannel -f /opt/home/hrschannel.pb -o orderer.n-zzzz.managedblockchain.us-east-1.amazonaws.com:30001 --cafile /opt/home/managedblockchain-tls-chain.pem --tls
But I am getting the following error,
Error: failed to create deliver client: orderer client failed to connect to orderer.n-zzzz.managedblockchain.us-east-1.amazonaws.com:30001: failed to create new connection: context deadline exceeded
Help me to fix this issue.
Edited:
I asked the same question in reddit. One user replied that he added listenAddress environment variable in my configtx.yaml file. He did not say clear information about which listenAddress and where to add that address in configtx.yaml. Here is my configtx.yaml file.
################################################################################
#
# Section: Organizations
#
# - This section defines the different organizational identities which will
# be referenced later in the configuration.
#
################################################################################
Organizations:
- &Org1
# DefaultOrg defines the organization which is used in the sampleconfig
# of the fabric.git development environment
Name: m-CUB6HI
# ID to load the MSP definition as
ID: m-B6HI
MSPDir: /opt/home/admin-msp
# AnchorPeers defines the location of peers which can be used
# for cross org gossip communication. Note, this value is only
# encoded in the genesis block in the Application section context
AnchorPeers:
- Host:
Port:
################################################################################
#
# SECTION: Application
#
# - This section defines the values to encode into a config transaction or
# genesis block for application related parameters
#
################################################################################
Application: &ApplicationDefaults
# Organizations is the list of orgs which are defined as participants on
# the application side of the network
Organizations:
################################################################################
#
# Profile
#
# - Different configuration profiles may be encoded here to be specified
# as parameters to the configtxgen tool
#
################################################################################
Profiles:
OneOrgChannel:
Consortium: AWSSystemConsortium
Application:
<<: *ApplicationDefaults
Organizations:
- *Org1
Help me to fix this issue.
One must check if the peer container is able to communicate with the orderer container. curl orderer.endpoint port can be used to check the connection. If the peer is unable to communicate then either the orderer container is down or could be due to different security groups.
Update:
As OP mentioned in the comments, changing the port helped in resolving the issue. One must give it a try.

How to deploy to Cloud Foundry with Docker Image from Private Docker Registry and Self-Signed SSL certificate?

How to add self-signed certificate to Cloud Foundry (PCFDev), so I would be able to deploy with Docker Image from private Docker Registry?
For this example I'm using PCFDev:
user#work:(0):~/Documents/$ cf push app-ui -o nexus-dev/app/app-ui:latest
Creating app app-ui in org pcfdev-org / space pcfdev-space as user...
OK
Creating route app-ui.local.pcfdev.io...
OK
Binding app-ui.local.pcfdev.io to app-ui...
OK
Starting app app-ui in org pcfdev-org / space pcfdev-space as user...
Creating container
Successfully created container
Staging...
Staging process started ...
Failed to talk to docker registry: Get https://nexus-dev/v2/: x509: certificate signed by unknown authority
Failed getting docker image by tag: Error parsing HTTP response: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>\r\n<body bgcolor=\"whit
e\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>The plain HTTP request was sent to HTTPS port</center>\r\n<hr><center>nginx/1.10.0 (Ubuntu)</center>\r\n</body>\r\n</html>\r\n"
Staging process failed: Exit trace for group:
builder exited with error: failed to fetch metadata from [app/app-ui] with tag [latest] and insecure registries [] due to Error parsing HTTP response: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>
400 The plain HTTP request was sent to HTTPS port</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>The plain HTTP request was sent to HTTPS port</center>\r\n<hr><center>nginx/1.10.0
(Ubuntu)</center>\r\n</body>\r\n</html>\r\n"
Exit status 2
Staging Failed: Exited with status 2
Destroying container
Successfully destroyed container
FAILED
Error restarting application: StagingError
TIP: use 'cf logs app-ui --recent' for more information
You can start pcfdev with -r option,
e.g.
cf dev start -r host.pcfdev.io:5000
from Insecure Docker Registries

ssl passenger nginx rails 4 routing combine wrong port with https

I folowed the rails tutorial org tuto and made a web site
Installed passenger, nginx. Works perfectly on prod env on heroku=>
http://klat.herokuapp.com/ (app not finished but still better than nothing)
Here i use the =>
config.force_ssl = true
(only for heroku)
my routes from routes.rb =>
Rails.application.routes.draw do
get 'users/new'
root 'static_pages#home'
get 'help' => 'static_pages#help'
get 'about' => 'static_pages#about'
get 'contact' => 'static_pages#contact'
get 'login' => 'static_pages#login'
get 'signup' => 'users#new'
resources :users
end
~/.bash_profile and ~/.bashrc got
#a generated key
export SECRET_KEY_BASE="my_very_big_generated_key"
export SECRET_TOKEN="my_very_big_generated_key"
I generated with openssl my own certificates
ls ../cert/
server.crt server.csr server.key server.key.org server.pem
And I launch my server with this command in local
passenger start -p 2000 --ssl-certificate ../cert/server.csr --ssl-port 20001 -e production
I passed two days to get to this point but i can't find out why my server starts at
http ://0.0.0.0:2000/
and redirects at
https ://0.0.0.0:2000
and no one page is accessible: error
An error occurred during a connection to 0.0.0.0:2000. SSL received a record that exceeded the maximum permissible length. (Error code: ssl_error_rx_record_too_long)
But in logs we clearly see
App 12832 stdout:
App 12854 stdout:
App 12854 stdout: Started HEAD "/" for 127.0.0.1 at 2015-04-24 18:09:30 +0200
App 12854 stdout: Processing by StaticPagesController#home as HTML
{... rendering homepage wiht _header.erb _footer.erb etc ...}
So i can visit my website non rendered with css at address:
http: //127.0.0.1:2000/
and i still getting those messages from my nginx server
ActionController::RoutingError (No route matches [GET] "/images/rails.png"):
ActionController::RoutingError (No route matches [GET] "/stylesheets/application.css"):
{...and more ...}
I commented in every file in my ruby on rails app all ssl_force
when i uncomment in config/production.rb
config.force_ssl = true
for local use, i get
on
http ://127.0.0.1:2000/ => redirect to https ://127.0.0.1:2000/ and get => (Error code: ssl_error_rx_record_too_long)
http ://127.0.0.1 => unable to locate
https ://127.0.0.1 => unable to locate
http ://127.0.0.1:2000 => unable to locate
http ://127.0.0.1:2001 => unable to locate
https ://127.0.0.1:2000 => unable to locate
https ://127.0.0.1:2001 => unable to locate
Just can't find out why my openssl generated keys don't work, the pages are not rendered with css and other ressources, it's maybe a routing problem...
A little help is much appreciated!
PS: (But please no help such as "use Thin or Puma or another server, i have a constraint to use at choice nginx, lighthttpd or Apache (the obvious answer is nginx for me)")
And to set up my own ssl certification in my own server is a constraint too in my studies
finally
the command i did to start my server without errors is
passenger start --ssl-certificate ../cert/server.crt --ssl-certificate-key ../cert/server.key -e production --ssl
without specifying the ssl port and standart port
i added
gem 'rack-ssl', :require => 'rack/ssl'
to my gem file
and did several bundle install/--deployment etc...
set up in production.rb
config.force_ssl = true
and in application_controller.rb
force_ssl
so my connection is encrypted now as i connect to
https://0.0.0.0:3000/
BUT...
I'm still unhappy with all those configurations because of the presence of the port number in the url
Anyway, it will do for now...