get status code from a gRPC endpoint not using a gRPC client - amazon-web-services

How could I get the status code from a gRPC endpoint without having a gRPC client?
I need to test some gRPC endpoints that are behind an AWS application load balancer (target group) with a health check configured to only accept status 12
UNIMPLEMENTED 12 The operation is not implemented or is not supported/enabled in this service.
I tried grpcurl for example:
grpcurl -plaintext 10.1.2.8:8443 AWS.ALB/healthcheck:
But in many cases I get:
Error invoking method "AWS.ALB/healthcheck": failed to query for service descriptor "AWS.ALB": server does not support the reflection API
Any alternatives or ideas? I am just interested in the status code 12 or description UNIMPLEMENTED

For status code 12, you can change it through
alb.ingress.kubernetes.io/success-codes: '0' // change success code to 0
spec:
rules:
- host: server.xxx.com
http:
paths:
- path: /* // set to * to match all grpc service paths
backend:
serviceName: server
servicePort: 9420
Also enable server reflection
## -40,6 +40,7 ## import (
"google.golang.org/grpc"
pb "google.golang.org/grpc/examples/helloworld/helloworld"
+ "google.golang.org/grpc/reflection"
)
const (
## -61,6 +62,8 ## func main() {
}
s := grpc.NewServer()
pb.RegisterGreeterService(s, &pb.GreeterService{SayHello: sayHello})
+ // Register reflection service on gRPC server.
+ reflection.Register(s)
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}

Related

Message is being sent with "send_message" in ejabberd using postman but not received by client

I have configured and install ejabberd on the ubuntu 22.04 and I have successfully configured and create one user with administrator right and as well as create some users into it.
I am using Version
OS - ubuntu 22.04 LTS
ejabberd 22.10
And also configured ejabberd API by mod_http_api and then I test APIs with POSTMAN, almost every (link) "API reference" working fine with it except send_message.
Here is my ejabberd.yml configuration:-
hosts:
- B660M-D2H-DDR4
- localhost
- XX.XXX.37.XX
loglevel: info
ca_file: /opt/ejabberd/conf/cacert.pem
certfiles:
- /opt/ejabberd/conf/server.pem
## If you already have certificates, list them here
# certfiles:
# - /etc/letsencrypt/live/domain.tld/fullchain.pem
# - /etc/letsencrypt/live/domain.tld/privkey.pem
listen:
-
port: 5222
ip: "::"
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5223
ip: "::"
tls: true
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5269
ip: "::"
module: ejabberd_s2s_in
max_stanza_size: 524288
-
port: 5443
ip: "::"
module: ejabberd_http
tls: true
request_handlers:
/admin: ejabberd_web_admin
/api: mod_http_api
/bosh: mod_bosh
/captcha: ejabberd_captcha
/upload: mod_http_upload
/ws: ejabberd_http_ws
-
port: 5280
ip: "::"
module: ejabberd_http
request_handlers:
/admin: ejabberd_web_admin
/.well-known/acme-challenge: ejabberd_acme
/api: mod_http_api
-
port: 3478
ip: "::"
transport: udp
module: ejabberd_stun
use_turn: true
## The server's public IPv4 address:
# turn_ipv4_address: "203.0.113.3"
## The server's public IPv6 address:
# turn_ipv6_address: "2001:db8::3"
-
port: 1883
ip: "::"
module: mod_mqtt
backlog: 1000
s2s_use_starttls: optional
acl:
admin:
user: "admin#localhost"
local:
user_regexp: ""
loopback:
ip:
- 127.0.0.0/8
- ::1/128
access_rules:
local:
allow: local
allow: XX.XXX.37.XX
c2s:
deny: blocked
allow: all
announce:
allow: admin
configure:
allow: admin
muc_create:
allow: local
pubsub_createnode:
allow: local
trusted_network:
allow: loopback
api_permissions:
"console commands":
from:
- ejabberd_ctl
who: all
what: "*"
"admin access":
who:
access:
allow:
- acl: admin
oauth:
scope: "ejabberd:admin"
access:
allow:
- acl: admin
what:
- "*"
- "!stop"
- "!start"
"public commands":
who:
ip: 127.0.0.1/8
what:
- status
- connected_users_number
shaper:
normal:
rate: 3000
burst_size: 20000
fast: 100000
shaper_rules:
max_user_sessions: 10
max_user_offline_messages:
5000: admin
100: all
c2s_shaper:
none: admin
normal: all
s2s_shaper: fast
modules:
mod_adhoc: {}
mod_admin_extra: {}
mod_announce:
access: announce
mod_avatar: {}
mod_blocking: {}
mod_bosh: {}
mod_caps: {}
mod_carboncopy: {}
mod_client_state: {}
mod_configure: {}
mod_disco: {}
mod_fail2ban: {}
mod_http_api: {}
mod_http_upload:
put_url: https://#HOST#:5443/upload
custom_headers:
"Access-Control-Allow-Origin": "https://#HOST#"
"Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
"Access-Control-Allow-Headers": "Content-Type"
mod_last: {}
mod_mam:
## Mnesia is limited to 2GB, better to use an SQL backend
## For small servers SQLite is a good fit and is very easy
## to configure. Uncomment this when you have SQL configured:
## db_type: sql
assume_mam_usage: true
default: always
mod_mqtt: {}
mod_muc:
access:
- allow
access_admin:
- allow: admin
access_create: muc_create
access_persistent: muc_create
access_mam:
- allow
default_room_options:
mam: true
mod_muc_admin: {}
mod_offline:
access_max_user_messages: max_user_offline_messages
mod_ping: {}
mod_privacy: {}
mod_private: {}
mod_proxy65:
access: local
max_connections: 5
mod_pubsub:
access_createnode: pubsub_createnode
plugins:
- flat
- pep
force_node_config:
## Avoid buggy clients to make their bookmarks public
storage:bookmarks:
access_model: whitelist
mod_push: {}
mod_push_keepalive: {}
mod_register:
## Only accept registration requests from the "trusted"
## network (see access_rules section above).
## Think twice before enabling registration from any
## address. See the Jabber SPAM Manifesto for details:
## https://github.com/ge0rg/jabber-spam-fighting-manifesto
ip_access: trusted_network
mod_roster:
versioning: true
mod_s2s_dialback: {}
mod_shared_roster: {}
mod_stream_mgmt:
resend_on_timeout: if_offline
mod_stun_disco: {}
mod_vcard: {}
mod_vcard_xupdate: {}
mod_version:
show_os: false
I have two observation with messages (send_message):
client to client (PSI)
postman to Client
In first observation I have success to exchange messages between users on psi(PSI) but when we try to send message with postman using "mod_http_api" API to the client, I am getting result 200 ok, but Message is not being delivered. And it is not showing anywhere (logs).
Am I missing something that is important for receiving a message using ejabberd's REST API with postman?
What a strange problem, I cannot reproduce it. You didn't show your command query, and didn't mention what exact client and configuration you are using.
Summary: Check if the command works correctly when using the ejabberdctl command line tool, and use "normal" message type, and send to bare JID, and use another client for example Gajim (just for debugging the problem).
Details:
I installed ejabberd 22.10 from source code, copied your configuration, disabled cert and tls options, started ejabberd, registered account, logged in it and executed this command:
$ ejabberdctl send_message headline uuu#localhost user1#localhost Restart aaa
The client that was logged in user1#localhost received the stanza, and displayed the headline message:
<message to='user1#localhost'
from='uuu#localhost'
type='headline'
id='18154938236359942834'>
<body>aaa</body>
<subject>Restart</subject>
</message>
Please note: in XMPP, "headline" messages are not stored in the offline storage: they are only received by online sessions with positive priority. Maybe you are sending "headline" messages to sessions that are offline, or online with negative priority, or online with no initial presence?
It's preferable to send a "normal" message, which are stored offline:
ejabberdctl send_message normal uuu#localhost user1#localhost ThisisNormal bbb
Also, make sure your client is logged in with positive priority (this is the standard).

ESP32 MQTT with TLS not working with NATS MQTT

Hi i have a project about connect ESP32 to NATS via MQTT
After I try --insecurity it work normally.
But when I add TLS it doesn't work in my ESP32, i also try with Python it work normally with my self-signed SSL certificate.
I already find solution from here but it not work: https://github.com/espressif/arduino-esp32/issues/5021
My idea code get from here: https://github.com/debsahu/ESP-MQTT-AWS-IoT-Core/blob/master/Arduino/PubSubClient/PubSubClient.ino
Is MQTT TLS in ESP32 not work with self-signed cert or I done something wrong ?
Cert TLS:
"-----BEGIN CERTIFICATE-----\n"
"MIID8TCCAtmgAwIBAgIUfceZXKK1JIqHi57rc98EBmJoy1kwDQYJKoZIhvcNAQEL\n"
"BQAwgYcxCzAJBgNVBAYTAlZOMRAwDgYDVQQIDAd2aWV0bmFtMRAwDgYDVQQHDAd2\n"
"aWV0bmFtMQ4wDAYDVQQKDAVwZWNvbTENMAsGA1UECwwEdGVzdDENMAsGA1UEAwwE\n"
"bXF0dDEmMCQGCSqGSIb3DQEJARYXY3B0cHJpY2UxMjNAb3V0bG9vay5jb20wHhcN\n"
"MjIxMTAzMDgxMDEzWhcNMjMxMTAzMDgxMDEzWjCBhzELMAkGA1UEBhMCVk4xEDAO\n"
"BgNVBAgMB3ZpZXRuYW0xEDAOBgNVBAcMB3ZpZXRuYW0xDjAMBgNVBAoMBXBlY29t\n"
"MQ0wCwYDVQQLDAR0ZXN0MQ0wCwYDVQQDDARtcXR0MSYwJAYJKoZIhvcNAQkBFhdj\n"
"cHRwcmljZTEyM0BvdXRsb29rLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC\n"
"AQoCggEBALRTuKn8m1QuFJI3THb2rkEiKPHD/cdRs/E1Vb96GIBSy4D/s8vJ2OWd\n"
"GHlbLK557OpAH7JrRg6tVEVVr3293u8imwDIcNyOHlBYWSO/DBKGXsoCbOL1u6Gd\n"
"zAn/G+96eX3RUIHRbBF/rE6DZS5Y1Piq7FwdaReHSZhMPB+UMB4xUEC3pC6CzqFt\n"
"xjudk9zT5VpR60XiJAls3YtYpUu4zRZUw2Sb1ZsPmT555QFYbOcF4XlC82MVi/o3\n"
"M91LJ8DyiOvNWxuioIT2frEyIXaTleug3Ev0ALiu8ug9/v/zTWZWq3KA98HZJcm+\n"
"Hr8dChlMewpMpabEi1e0twlzTPw9QyMCAwEAAaNTMFEwHQYDVR0OBBYEFE3SQ0F5\n"
"yzsBkHUcFp/KucgyGHpWMB8GA1UdIwQYMBaAFE3SQ0F5yzsBkHUcFp/KucgyGHpW\n"
"MA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAC+pjnAd9c71DfCv\n"
"RUMvYg93mraaqtoEw22ggtT9AfIZfI+o8L5Nxk5us+9k9IBEP4hi6DHtnFxqfFt8\n"
"YkzNNDMTDvLfg/1INUwg8yBYS9Z2+puoPlPTvaxOJiuz2+DkYV/LnUdTolKPqYrO\n"
"IBIbmwMNz0Bqn1XZ7Mjx9U7p+A2N/87NGl3fo0I0tWBRSGXFZB7IYipgCPQG5Eb+\n"
"ZL9vrgFuNJhAVALvDfwKxCX6VdyNpthAMA7cmra+s0/weZsfQLyU2TtnsIg0uoq0\n"
"L4sTpL6Q7Fr4UpOJrezNx/kuHHxBBKhJtlz4Tcaw/OKu/h2g5jjHFT9yN86KUxSY\n"
"PDH47kk=\n"
"-----END CERTIFICATE-----\n";
// You can use x.509 client certificates if you want
const char *test_client_key =
"-----BEGIN RSA PRIVATE KEY-----\n"
"MIIEogIBAAKCAQEAllaf/na5h3yDr2zoYsYGGqt/d93/AAUTculGTXdGGbRwyHue\n"
"b0BaMeX+ht9siZ82iuaZ/5mJ/kq8WVwlrkegOAvU7SQDoALPM7VLMLSMbnn2Wqog\n"
"WE48TkWU0WddtTFHVDGLX8zMC1TQ1VKyVzp2QtCW9RPJNun9CVJSoZ34uM5hBL1f\n"
"7MY7t/QsDYi14UtULDsSnVz+tDLiPrBkZOPEVhopCH1gvljcDTcICfawyK5nlCKc\n"
"AnUWTHEUzf89WCJkPNk1W3LhscGKfx2bV8XVv+izg2zMLec5aYM/LrJg6HpJzgQJ\n"
"IKBt1tWQkxRvO7LO3znSp8A9DXotvr0MkIqcjQIDAQABAoIBAEMAaF3oW9deTvIn\n"
"/4nF54KLXEv3zGYd3QUhogt0VPGv0XQIZBwA+jGy5zUE7kKHiq9tBsU7kJycgkTx\n"
"JHn/whA4dbUaj+MIXYAWFGSoks3J3Vma6L9yXr4jlKefAcx3IesMCamwhF+odUod\n"
"iQ4HKB2vCRhAsTSgI/27isgst2TlJsGMf7ED2N1jae8ZyOITi2g0F1edRYBwgSHq\n"
"MZvccZh/IpuTOPEVxuITYyQT9WF0TCz7cK4wCP5dACQQB6Or8l2xiUf9dx3I7kwR\n"
"7wvivI+jAoxR/peOXx2o0bHPcqh41rbhbE00XOcIReGoyLsRDvicw3hgFe6UxcEm\n"
"PlpFzaECgYEAyBPpzK3x0iXj66iO6erXzciN5cXF8IZhC7xcCgGOpnjgrMV3FUNv\n"
"L0Qu8zUlTJHfWpITCZawPpbNMaNShykLU6NqxUPXGtaH/xVUZm9VbkRwBQoQKg+w\n"
"x2+hAWTGu4rWtSaWMHJuwI0SYyopvJtBgDO8PkmzDG24RQuRVBSE+ycCgYEAwFu6\n"
"QHVHvVm4ri1FCIK313uXTWoYhKDCm8ygDKT608bHzBoqOcXPT5mcr3IZmZitsg3Y\n"
"DyVvPGmmbLp8FmxXcz2c71e1Bupeq9V8HrMiSgMVPEIRuNKVC7WE/Ymuvpvfd+h/\n"
"RyDCu2wTI4GcJRhmAB+SpjPPOH0qaqV2eHZgSysCgYAO5eyy4QDwtQGTuqlpoaMQ\n"
"H67xPRjQIDF5vjzcQeFtY/LW6p1DaBIPYvRcB8kPOo13IQlp3V6iSnhdCdxLVDMT\n"
"t0dsCPErfm4CAISYXBHwdAgjV+x8NU7kittiTy69KEl0k7r7QIoerGKCH9GbybPG\n"
"6BNMUBCVDFZ8TbA0opKEYQKBgEl0/fxNjTbXA3qoWPt2B8SnMtFiWbiUN50NmHUb\n"
"r5meCIB94XAshQ2NyNMLDJGmR3Z+aOrnzcHRSresw2RAvWiJt9uCr+PTLpIKNZr3\n"
"p3mCEeLwDBp7eGV/TSkRIgUyOzVsOOatsQ+nputhPILB/XnAlN0ZXeHhkoglZcd8\n"
"1Sr/AoGAU7nlyAMQNd/tckwPTnM++0ewrFvwrfpS7f2dhcYbIhfqQ3I03Gwzjkkg\n"
"G85uzTg/8iO4oxPRjqPvc7JaoDDmGY/efQvjR+FdwDOuy+XZPImZIgGjl0yvAMFU\n"
"6azU+OxtwV+Yyfad4rGxaXZsqOIs18to94t2kjI0t8ur/4Q7C5w=\n"
"-----END RSA PRIVATE KEY-----\n"; // to verify the client
const char *test_client_cert =
"-----BEGIN CERTIFICATE-----\n"
"MIIDnjCCAoYCFBRQlTP3aMzr8YtFlYoaVtrPIN6xMA0GCSqGSIb3DQEBCwUAMIGH\n"
"MQswCQYDVQQGEwJWTjEQMA4GA1UECAwHdmlldG5hbTEQMA4GA1UEBwwHdmlldG5h\n"
"bTEOMAwGA1UECgwFcGVjb20xDTALBgNVBAsMBHRlc3QxDTALBgNVBAMMBG1xdHQx\n"
"JjAkBgkqhkiG9w0BCQEWF2NwdHByaWNlMTIzQG91dGxvb2suY29tMB4XDTIyMTEw\n"
"MzA4MTMzMVoXDTIzMDIxMTA4MTMzMVowgY4xCzAJBgNVBAYTAlZOMRAwDgYDVQQI\n"
"DAd2aWV0bmFtMRAwDgYDVQQHDAd2aWV0bmFtMQ4wDAYDVQQKDAVwZWNvbTENMAsG\n"
"A1UECwwEdGVzdDETMBEGA1UEAwwKbXF0dGNsaWVudDEnMCUGCSqGSIb3DQEJARYY\n"
"Y3B0cHJpY2VAMTIzQG91dGxvb2suY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A\n"
"MIIBCgKCAQEAllaf/na5h3yDr2zoYsYGGqt/d93/AAUTculGTXdGGbRwyHueb0Ba\n"
"MeX+ht9siZ82iuaZ/5mJ/kq8WVwlrkegOAvU7SQDoALPM7VLMLSMbnn2WqogWE48\n"
"TkWU0WddtTFHVDGLX8zMC1TQ1VKyVzp2QtCW9RPJNun9CVJSoZ34uM5hBL1f7MY7\n"
"t/QsDYi14UtULDsSnVz+tDLiPrBkZOPEVhopCH1gvljcDTcICfawyK5nlCKcAnUW\n"
"THEUzf89WCJkPNk1W3LhscGKfx2bV8XVv+izg2zMLec5aYM/LrJg6HpJzgQJIKBt\n"
"1tWQkxRvO7LO3znSp8A9DXotvr0MkIqcjQIDAQABMA0GCSqGSIb3DQEBCwUAA4IB\n"
"AQCF33dWLyL/QJKDBNtKc6WwmOn97u74jkIYdgRHgQwNvrmLHRgZPb6Bhzy5KAIY\n"
"qJcPA6Cn/m4utUWjAXRPj9zDT5xyeC843R22KQASjmPBnEyfDZuXmUPjjNJUSUx6\n"
"JGk/bwPQDLT2ID+vl3OInm4ypgwbGaqlhn41m0F2smanuZUFgEmN5+tJpkwK/tVP\n"
"IYHJ5HPnFqDFs84Fp12HU2QcqbOUEZ/d77Yw/dfb20cvgW2xkHKEAhz7d9EpD4ov\n"
"S5ZnelKxvqlVzI2v2I6MJkRdeP2IfYofNfRo2s7S5u+h/2SQu1MbbarS/jd32Ldz\n"
"14EVvDj+sCF2g7skdJ3kYCPI\n"
"-----END CERTIFICATE-----\n"; // to verify the client
Server config:
listen:0.0.0.0:4222
jetstream:{
max_memory_store: 1073741824
max_file_store: 1073741824
}
mqtt {
# Specify a host and port to listen for websocket connections
#
listen: "0.0.0.0:8883"
# It can also be configured with individual parameters,
# namely host and port.
#
# host: "hostname"
# port: 1883
# TLS configuration.
tls {
cert_file: /etc/tls/mqtt/broker/broker.crt
key_file: /etc/tls/mqtt/broker/broker.key
ca_file: /etc/tls/mqtt/ca/ca.crt
verify: true
timeout: 2.0
# verify_and_map: true
}
# no_auth_user: "my_username_for_apps_not_providing_credentials"
# authorization {
# # username: "my_user_name"
# # password: "my_password"
# # token: "my_token"
# # timeout: 2.0
# }
ack_wait: "1m"
max_ack_pending: 100
}
tls:{
cert_file: /etc/tls/natsio/server-cert.pem
key_file: /etc/tls/natsio/server-key.pem
ca_file: /etc/tls/natsio/ca-cert.pem
}
http_port: 8222
# system_account: AAOQAS43OSVDMF3ERYSNL3GMGZRD7GILDGDET6R52NFZKEWJOTTVNYZ4
# resolver: {
# type: full
# dir: './jwt'
# allow_delete: false
# interval: "2m"
# limit: 1000
# }
Logs from NATS server:
test-nats-dev-1 | [1] 2022/11/03 10:29:32.794114 [ERR] 192.168.1.14:57479 - mid:699 - TLS handshake error: remote error: tls: bad certificate
test-nats-dev-1 | [1] 2022/11/03 10:29:37.989099 [ERR] 192.168.1.14:57480 - mid:700 - TLS handshake error: remote error: tls: bad certificate
ESP32 logs:
Attempting MQTT connection...[2959556][E][ssl_client.cpp:37] _handle_error(): [start_ssl_client():276]: (-9984) X509 - Certificate verification failed, e.g. CRL, CA or signature check failed
[2959559][E][WiFiClientSecure.cpp:135] connect(): start_ssl_client: -9984
failed, rc=-2 try again in 5 seconds
Attempting MQTT connection...[2964762][E][ssl_client.cpp:37] _handle_error(): [start_ssl_client():276]: (-9984) X509 - Certificate verification failed, e.g. CRL, CA or signature check failed
[2964765][E][WiFiClientSecure.cpp:135] connect(): start_ssl_client: -9984
failed, rc=-2 try again in 5 seconds
Attempting MQTT connection...[2976298][E][ssl_client.cpp:37] _handle_error(): [start_ssl_client():276]: (-9984) X509 - Certificate verification failed, e.g. CRL, CA or signature check failed
[2976301][E][WiFiClientSecure.cpp:135] connect(): start_ssl_client: -9984
failed, rc=-2 try again in 5 seconds
I found the problem is my cert is using domain for certification and I using IP for connection. That is make it not working.
After change to using domain in connection it works normally.
Domain TLS: https://docs.cpanel.net/knowledge-base/general-systems-administration/what-is-domain-tls/
IP TLS: Is it possible to have SSL certificate for IP address, not domain name?

How to connect to Amazon managed blockchain network using hyperledger-fabric-nodesdk 2.2

I want to connect to hyperledger fabric blockchain network on Amazon Managed Blockchain using nodejs sdk.
Fabric client is a cloud9 instance, which already setup and successfully connected to peer node using fabric cli inside a docker container.
But when I try to use nodejs sdk to connect to network with this code:
'use strict';
const FabricCAServices = require('fabric-ca-client');
const { Wallets, Gateway, X509Identity, User } = require('fabric-network');
const fs = require('fs');
const path = require('path');
const yaml = require('js-yaml');
const ccpPath = path.resolve(__dirname, 'connection_profile.yaml');
const ccp = yaml.load(fs.readFileSync(ccpPath, 'utf8'));
async function main() {
try {
const walletPath = path.join(process.cwd(), 'wallet');
const wallet = await Wallets.newFileSystemWallet(walletPath);
const gateway = new Gateway();
const gatewayOptions = {identity: 'admin', wallet: wallet, discovery: {enabled: true, asLocalhost: false }}
await gateway.connect(ccp, gatewayOptions);
const network = await gateway.getNetwork('mychannel');
} catch (error) {
console.error(`Some error is occurred: ${error.stack}`);
process.exit(1);
}
}
main();
With content of "connection_profile.yaml" file is:
name: "ABC"
x-type: hlfv1
version: "1.0"
channels:
mychannel:
orderers:
- ABCOrderer
peers:
peer1:
endorsingPeer: true
chaincodeQuery: true
ledgerQuery: true
eventSource: true
organizations:
abc:
mspid: m-***
peers:
- peer1
certificateAuthorities:
- abc
orderers:
ABCOrderer:
url: grpcs://orderer.n-***.managedblockchain.ap-northeast-1.amazonaws.com:30001
grpcOptions:
ssl-target-name-override: orderer.n-***.managedblockchain.ap-northeast-1.amazonaws.com
tlsCACerts:
# path: /home/ec2-user/managedblockchain-tls-chain.pem
path: /home/ec2-user/admin-msp/admincerts/cert.pem
peers:
peer1:
url: grpcs://nd-***.managedblockchain.ap-northeast-1.amazonaws.com:30003
eventUrl: grpcs://nd-***.managedblockchain.ap-northeast-1.amazonaws.com:30004
grpcOptions:
ssl-target-name-override: nd-***.managedblockchain.ap-northeast-1.amazonaws.com
tlsCACerts:
# path: /home/ec2-user/managedblockchain-tls-chain.pem
path: /home/ec2-user/admin-msp/admincerts/cert.pem
certificateAuthorities:
abc:
url: https://ca.m-***.managedblockchain.ap-northeast-1.amazonaws.com:30002
httpOptions:
verify: true
tlsCACerts:
# path: /home/ec2-user/managedblockchain-tls-chain.pem
path: /home/ec2-user/admin-msp/admincerts/cert.pem
caName: m-***
"/home/ec2-user/admin-msp/admincerts/cert.pem" is file is created by enroll member admin identity (follow this aws guide: https://docs.aws.amazon.com/managed-blockchain/latest/hyperledger-fabric-dev/get-started-enroll-admin.html).
Then after 3s the console show this error:
2022-07-05T13:22:52.812Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer1, url:grpcs://nd-***.managedblockchain.ap-northeast-1.amazonaws.com:30003, connected:false, connectAttempted:true
2022-07-05T13:22:52.814Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer1 url:grpcs://nd-***.managedblockchain.ap-northeast-1.amazonaws.com:30003 timeout:3000
2022-07-05T13:22:52.814Z - info: [NetworkConfig]: buildPeer - Unable to connect to the endorser peer1 due to Error: Failed to connect before the deadline on Endorser- name: peer1, url:grpcs://nd-***.managedblockchain.ap-northeast-1.amazonaws.com:30003, connected:false, connectAttempted:true
2022-07-05T13:22:55.817Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Committer- name: ABCOrderer, url:grpcs://orderer.n-***.managedblockchain.ap-northeast-1.amazonaws.com:30001, connected:false, connectAttempted:true
2022-07-05T13:22:55.817Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server ABCOrderer url:grpcs://orderer.n-***.managedblockchain.ap-northeast-1.amazonaws.com:30001 timeout:3000
2022-07-05T13:22:55.818Z - info: [NetworkConfig]: buildOrderer - Unable to connect to the committer ABCOrderer due to Error: Failed to connect before the deadline on Committer- name: ABCOrderer, url:grpcs://orderer.n-***.managedblockchain.ap-northeast-1.amazonaws.com:30001, connected:false, connectAttempted:true
Some error is occurred: TypeError: Cannot read property 'toArray' of null
at EC.sign (/home/ec2-user/src-test/node_modules/elliptic/lib/elliptic/ec/index.js:104:30)
at CryptoSuite_ECDSA_AES.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/impl/CryptoSuite_ECDSA_AES.js:215:25)
at Signer.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/Signer.js:59:28)
at SigningIdentity.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/SigningIdentity.js:71:23)
at IdentityContext.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/IdentityContext.js:91:40)
at DiscoveryService.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/ServiceAction.js:69:40)
at NetworkImpl._initializeInternalChannel (/home/ec2-user/src-test/node_modules/fabric-network/lib/network.js:298:35)
at NetworkImpl._initialize (/home/ec2-user/src-test/node_modules/fabric-network/lib/network.js:250:20)
at Gateway.getNetwork (/home/ec2-user/src-test/node_modules/fabric-network/lib/gateway.js:350:26)
at main (/home/ec2-user/src-test/enrollUser.js:38:35)
So I think the problem probably is my connection_profile setting, which I mimic connection-profile-template.yaml file from aws blockchain samples code (https://github.com/aws-samples/non-profit-blockchain/tree/master/ngo-lambda).
# Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License.
# A copy of the License is located at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# or in the "license" file accompanying this file. This file is distributed
# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
# express or implied. See the License for the specific language governing
# permissions and limitations under the License.
name: "ngo"
x-type: "hlfv1"
description: "NGO Network"
version: "1.0"
channels:
mychannel:
orderers:
- orderer.com
peers:
peer1:
endorsingPeer: true
chaincodeQuery: true
ledgerQuery: true
eventSource: true
organizations:
Org1:
mspid: %MEMBERID%
peers:
- peer1
certificateAuthorities:
- ca-org1
orderers:
orderer.com:
url: grpcs://%ORDERINGSERVICEENDPOINT%
grpcOptions:
ssl-target-name-override: %ORDERINGSERVICEENDPOINTNOPORT%
tlsCACerts:
path: %CAFILE%
peers:
peer1:
url: grpcs://%PEERSERVICEENDPOINT%
eventUrl: grpcs://%PEEREVENTENDPOINT%
grpcOptions:
ssl-target-name-override: %PEERSERVICEENDPOINTNOPORT%
tlsCACerts:
path: %CAFILE%
certificateAuthorities:
ca-org1:
url: https://%CASERVICEENDPOINT%
httpOptions:
verify: false
tlsCACerts:
path: %CAFILE%
caName: %MEMBERID%
So, any idea or suggestion on how can I fix it? Any help would be appreciated.
Thank You!
Updated_1:
I try both key file from AWS S3(managedblockchain-tls-chain.pem) and key created by CA for admin. But both do not seem to work. Here is the error when I try with key file from S3:
Some error is occurred: TypeError: Cannot read property 'toArray' of null
at EC.sign (/home/ec2-user/src-test/node_modules/elliptic/lib/elliptic/ec/index.js:104:30)
at CryptoSuite_ECDSA_AES.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/impl/CryptoSuite_ECDSA_AES.js:215:25)
at Signer.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/Signer.js:59:28)
at SigningIdentity.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/SigningIdentity.js:71:23)
at IdentityContext.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/IdentityContext.js:91:40)
at DiscoveryService.sign (/home/ec2-user/src-test/node_modules/fabric-common/lib/ServiceAction.js:69:40)
at NetworkImpl._initializeInternalChannel (/home/ec2-user/src-test/node_modules/fabric-network/lib/network.js:298:35)
at NetworkImpl._initialize (/home/ec2-user/src-test/node_modules/fabric-network/lib/network.js:250:20)
at Gateway.getNetwork (/home/ec2-user/src-test/node_modules/fabric-network/lib/gateway.js:350:26)
at main (/home/ec2-user/src-test/enrollUser.js:38:35)
Updated 2:
Maybe the problem is my admin identity inside wallet, so I update the code to save admin identity to the wallet:
const caURL = ccp.certificateAuthorities['abc'].url;
const ca = new FabricCAServices(caURL);
const enrollment = await ca.enroll({ enrollmentID: 'admin', enrollmentSecret: 'Adminpassword' });
const X509Identity = {
credentials: {
certificate: enrollment.certificate,
privateKey: enrollment.rootCertificate,
},
mspId: ccp.organizations['abc'].mspid,
type: 'X.509',
};
// Create a new file system based wallet for managing identities.
const walletPath = path.join(process.cwd(), 'wallet');
const wallet = await Wallets.newFileSystemWallet(walletPath);
await wallet.put('admin', X509Identity);
Updated_3:
As #david_k suggests, the problem is my identity inside the wallet is wrong, and as the result, it is denied by the gateway. So the line privateKey in Update_2 needs to be changed from privateKey: enrollment.rootCertificate, to privateKey: enrollment.key.toBytes(),
Thank you very much #david_k!

Spring Boot Application failed on start when using #SqsListener

I'm trying to implement SQS listener in my Spring Boot app (Kotlin). I'm using spring-cloud-aws-messaging. Here is an article that walked me through implementation.
Problem: application is unable to start.
Logs says:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMessageListenerContainer' defined in class path resource [org/springframework/cloud/aws/messaging/config/annotation/SqsConfiguration.class]: Invocation of init method failed; nested exception is com.amazonaws.services.sqs.model.AmazonSQSException: null (Service: AmazonSQS; Status Code: 500; Error Code: 500 ; Request ID: null)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1803)
...
Kindly asking for help :)
docker-compose.yml
version: "2"
services:
localstack:
image: localstack/localstack:0.11.5
ports:
- "8085:8080"
- "4569-4576:4569-4576"
environment:
- SERVICES=sqs:4573
- DOCKER_HOST=unix:///var/run/docker.sock
- DEFAULT_REGION=eu-west-1
- DATA_DIR=/tmp/localstack/data
volumes:
- ./docker/dev/localstack-init-scripts:/docker-entrypoint-initaws.d
docker-entrypoint-initaws.sh
#!/usr/bin/env bash
set -x
awslocal sqs create-queue --queue-name my-queue-name
set +x
InfrastructureConfiguration.kt
#EnableSqs
class InfrastructureConfiguration(...) {
#Primary
#Bean
fun amazonSQSAsync(): AmazonSQSAsync {
val credentials: AWSCredentials = BasicAWSCredentials("accessKey", "secretKey")
return AmazonSQSAsyncClientBuilder
.standard()
.withEndpointConfiguration(
AwsClientBuilder.EndpointConfiguration(
"http://localstack:4573",
Regions.fromName("eu-west-1").toString()
)
).withCredentials(AWSStaticCredentialsProvider(credentials)).build()
}
#Primary
#Bean
fun simpleMessageListenerContainerFactory(amazonSQSAsync: AmazonSQSAsync):
SimpleMessageListenerContainerFactory {
val factory = SimpleMessageListenerContainerFactory()
factory.setAmazonSqs(amazonSQSAsync)
factory.setAutoStartup(false)
factory.setMaxNumberOfMessages(10)
factory.setWaitTimeOut(20)
return factory
}
}
AmazonSqs.kt
val logger = KotlinLogging.logger {}
#Lazy
#Component
class AmazonSqs {
companion object {
const val QUEUE_NAME = "my-queue-name"
}
#SqsListener(QUEUE_NAME)
fun receiveMessage(message: String?) {
logger.info("Received message: {}")
}
}
The problem lies in localstack. I've run docker-compose up again and noticed that port 4573 is deprecated.
localstack_1 | Starting edge router (https port 4566)...
localstack_1 | Starting mock SQS service on http ports 4566 (recommended) and
4573 (deprecated)...
The solution is to expose port 4566 in docker-compose and use it instead.
ports:
- "8085:8080"
- "4566-4576:4566-4576"

AWS X-Ray, Dotnet Core 3.1, X-Ray Daemon Locally

We are trying to get X-Ray trace data from a local dotnet core 3.1 app sending trace data to a local X-Ray Daemon. As a start, we've created a generic web api and added swagger (just to make testing easier).
Startup.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.HttpsPolicy;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.OpenApi.Models;
using Amazon.XRay.Recorder.Core;
using log4net;
using log4net.Config;
using System.Reflection;
using System.IO;
using Amazon;
using System.Net;
using Amazon.XRay.Recorder.Core.Internal.Utils;
using Amazon.XRay.Recorder.Core.Sampling.Local;
namespace AWS_XRay
{
public class Startup
{
public static ILog log;
static Startup() // create log4j instance
{
var logRepository = LogManager.GetRepository(Assembly.GetEntryAssembly());
XmlConfigurator.Configure(logRepository, new FileInfo("log4net.config"));
log = LogManager.GetLogger(typeof(Startup));
AWSXRayRecorder.RegisterLogger(LoggingOptions.Log4Net);
}
public Startup(IConfiguration configuration)
{
Configuration = configuration;
Environment.SetEnvironmentVariable("AWS_XRAY_DAEMON_ADDRESS", "127.0.0.1:2000");
Environment.SetEnvironmentVariable("AWS_XRAY_CONTEXT_MISSING", "LOG_ERROR");
var recorder = new AWSXRayRecorderBuilder().WithSamplingStrategy(newLocalizedSamplingStrategy("sampling-rules.json")).Build();
AWSXRayRecorder.InitializeInstance(configuration, recorder);
}
public IConfiguration Configuration { get; }
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
// Register the Swagger generator, defining 1 or more Swagger documents
services.AddSwaggerGen(c =>
{
c.SwaggerDoc("v1", new OpenApiInfo { Title = "My API", Version = "v1" });
});
}
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.UseXRay("WeatherForecast");
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UseSwagger();
app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint("/swagger/v1/swagger.json", "My API V1");
c.RoutePrefix = string.Empty;
});
app.UseHttpsRedirection();
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
}
}
}
Then we decorated the controller with the relevant or what we think is relevant
WeatherController
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Amazon.XRay.Recorder.Core;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
namespace AWS_XRay.Controllers
{
[ApiController]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
private static readonly string[] Summaries = new[]
{
"Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};
private readonly ILogger<WeatherForecastController> _logger;
public WeatherForecastController(ILogger<WeatherForecastController> logger)
{
_logger = logger;
}
[HttpGet]
[Route("GetWeather")]
public async Task<IActionResult> WeatherForecast()
{
AWSXRayRecorder.Instance.BeginSegment("weatherget"); // generates `TraceId` for you
try
{
var rng = new Random();
var result = Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Date = DateTime.Now.AddDays(index),
TemperatureC = rng.Next(-20, 55),
Summary = Summaries[rng.Next(Summaries.Length)]
})
.ToArray();
// can create custom subsegments
return Ok(result);
}
catch (Exception e)
{
AWSXRayRecorder.Instance.AddException(e);
return StatusCode(500, e);
}
finally
{
AWSXRayRecorder.Instance.EndSegment();
}
}
}
}
When running the application, looking at the logs. This is what we see...
*sdk-log.txt"
2020-04-14 16:04:21,740 [1] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Initializing with custom sampling configuration : sampling-rules.json
2020-04-14 16:04:22,035 [1] DEBUG Amazon.XRay.Recorder.Core.Internal.Utils.IPEndPointExtension - Determined that 127.0.0.1:2000 is an IP.
2020-04-14 16:04:22,039 [1] INFO Amazon.XRay.Recorder.Core.Internal.Utils.IPEndPointExtension - Using custom daemon address for UDP and TCP: 127.0.0.1:2000
2020-04-14 16:04:22,042 [1] DEBUG Amazon.XRay.Recorder.Core.Strategies.DefaultExceptionSerializationStrategy - Setting max stack frame size : 50
2020-04-14 16:04:22,073 [1] DEBUG Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl - Context missing mode : RUNTIME_ERROR
2020-04-14 16:04:22,073 [1] DEBUG Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl - AWS_XRAY_CONTEXT_MISSING environment variable is set to LOG_ERROR. Override local value.
2020-04-14 16:04:22,078 [1] DEBUG Amazon.XRay.Recorder.Core.Internal.Utils.IPEndPointExtension - Determined that 127.0.0.1:2000 is an IP.
2020-04-14 16:04:22,078 [1] INFO Amazon.XRay.Recorder.Core.Internal.Utils.IPEndPointExtension - Using custom daemon address for UDP and TCP: 127.0.0.1:2000
2020-04-14 16:04:22,078 [1] DEBUG Amazon.XRay.Recorder.Core.Strategies.DefaultExceptionSerializationStrategy - Setting max stack frame size : 50
2020-04-14 16:04:22,078 [1] DEBUG Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl - Context missing mode : RUNTIME_ERROR
2020-04-14 16:04:22,078 [1] DEBUG Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl - AWS_XRAY_CONTEXT_MISSING environment variable is set to LOG_ERROR. Override local value.
2020-04-14 16:04:22,078 [1] DEBUG Amazon.XRay.Recorder.Core.AWSXRayRecorder - Using custom X-Ray recorder.
2020-04-14 16:04:22,079 [1] DEBUG Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl - Context missing mode : RUNTIME_ERROR
2020-04-14 16:04:22,080 [1] DEBUG Amazon.XRay.Recorder.Core.AWSXRayRecorderImpl - AWS_XRAY_CONTEXT_MISSING environment variable is set to LOG_ERROR. Override local value.
2020-04-14 16:04:22,899 [4] DEBUG Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware - Trace header doesn't exist or not valid : (). Injecting a new one.
2020-04-14 16:04:22,911 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = localhost, path = /index.html, method = GET
2020-04-14 16:04:23,393 [4] DEBUG Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware - Trace header doesn't exist or not valid : (). Injecting a new one.
2020-04-14 16:04:23,394 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = localhost, path = /swagger/v1/swagger.json, method = GET
2020-04-14 16:04:27,497 [4] DEBUG Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware - Trace header doesn't exist or not valid : (). Injecting a new one.
2020-04-14 16:04:27,499 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = localhost, path = /WeatherForecast/GetWeather, method = GET
2020-04-14 16:04:27,602 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = , path = , method =
2020-04-14 16:04:29,740 [4] DEBUG Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware - Trace header doesn't exist or not valid : (). Injecting a new one.
2020-04-14 16:04:29,741 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = localhost, path = /WeatherForecast/GetWeather, method = GET
2020-04-14 16:04:29,745 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = , path = , method =
2020-04-14 16:04:30,149 [13] DEBUG Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware - Trace header doesn't exist or not valid : (). Injecting a new one.
2020-04-14 16:04:30,150 [13] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = localhost, path = /WeatherForecast/GetWeather, method = GET
2020-04-14 16:04:30,152 [13] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = , path = , method =
2020-04-14 16:04:30,346 [4] DEBUG Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware - Trace header doesn't exist or not valid : (). Injecting a new one.
2020-04-14 16:04:30,346 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = localhost, path = /WeatherForecast/GetWeather, method = GET
2020-04-14 16:04:30,349 [4] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = , path = , method =
2020-04-14 16:04:30,517 [13] DEBUG Amazon.XRay.Recorder.Handlers.AspNetCore.Internal.AWSXRayMiddleware - Trace header doesn't exist or not valid : (). Injecting a new one.
2020-04-14 16:04:30,518 [13] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = localhost, path = /WeatherForecast/GetWeather, method = GET
2020-04-14 16:04:30,529 [13] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Found a matching rule : (hostToMatch=*, httpMethodToMatch=Get, urlPathToMatch=*, fixedTarget=0, rate=0, description=Weather) for host = , path = , method =
2020-04-14 16:30:02,682 [1] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Initializing with custom sampling configuration : sampling-rules.json
Question 1
Based on the output in the config file, is there any trace data being sent to the daemon? We can't see any errors from the output, log level is set to DEBUG. Can't definitively say it is sending trace data to although no errors.
Daemon Config & Logs
cfg.yaml
# Maximum buffer size in MB (minimum 3). Choose 0 to use 1% of host memory.
TotalBufferSizeMB: 0
# Maximum number of concurrent calls to AWS X-Ray to upload segment documents.
Concurrency: 8
# Send segments to AWS X-Ray service in a specific region
Region: "eu-west-1"
# Change the X-Ray service endpoint to which the daemon sends segment documents.
Endpoint: "xray.eu-west-1.amazonaws.com"
Socket:
# Change the address and port on which the daemon listens for UDP packets containing segment documents.
UDPAddress: "127.0.0.1:2000"
# Change the address and port on which the daemon listens for HTTP requests to proxy to AWS X-Ray.
TCPAddress: "127.0.0.1:2000"
Logging:
LogRotation: true
# Change the log level, from most verbose to least: dev, debug, info, warn, error, prod (default).
LogLevel: "dev"
# Output logs to the specified file path.
LogPath: "xray.log"
# Turn on local mode to skip EC2 instance metadata check.
LocalMode: true
# Amazon Resource Name (ARN) of the AWS resource running the daemon.
ResourceARN: ""
# Assume an IAM role to upload segments to a different account.
RoleARN: "************************"
# Disable TLS certificate verification.
NoVerifySSL: false
# Upload segments to AWS X-Ray through a proxy.
ProxyAddress: ""
# Daemon configuration file format version.
Version: 2
Looking at the log file
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] Skipped telemetry data as no segments found
2020-04-14T16:35:40+02:00 [Debug] telemetry: done!
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] Segment batch: done!
2020-04-14T16:35:40+02:00 [Debug] processor: done!
2020-04-14T16:35:40+02:00 [Debug] Trace segment: received: 0, truncated: 0, processed: 0
2020-04-14T16:35:40+02:00 [Debug] Shutdown finished. Current epoch in nanoseconds: 1586874940496183800
2020-04-14T16:35:42+02:00 [Info] Initializing AWS X-Ray daemon 3.2.0
2020-04-14T16:35:42+02:00 [Debug] Listening on UDP 127.0.0.1:2000
2020-04-14T16:35:42+02:00 [Info] Using buffer memory limit of 80 MB
2020-04-14T16:35:42+02:00 [Info] 1280 segment buffers allocated
2020-04-14T16:35:42+02:00 [Debug] Using Endpoint read from Config file: xray.eu-west-1.amazonaws.com
2020-04-14T16:35:42+02:00 [Debug] Using proxy address:
2020-04-14T16:35:42+02:00 [Debug] Fetch region eu-west-1 from commandline/config file
2020-04-14T16:35:42+02:00 [Info] Using region: eu-west-1
2020-04-14T16:35:42+02:00 [Debug] ARN of the AWS resource running the daemon:
2020-04-14T16:35:42+02:00 [Debug] No Metadata set for telemetry records
2020-04-14T16:35:42+02:00 [Debug] Using Endpoint: https://xray.eu-west-1.amazonaws.com
2020-04-14T16:35:42+02:00 [Debug] Telemetry initiated
2020-04-14T16:35:42+02:00 [Info] HTTP Proxy server using X-Ray Endpoint : xray.eu-west-1.amazonaws.com
2020-04-14T16:35:42+02:00 [Debug] Using Endpoint: https://xray.eu-west-1.amazonaws.com
2020-04-14T16:35:42+02:00 [Debug] Batch size: 50
Question 2
Looking at the log file of the daemon, the line Trace segment: received: 0, truncated: 0, processed: 0 seems to indicate that it never received trace data? Why not, what are we missing? I'm suspecting that we are not instrumenting the application properly, but not sure.
For anyone that's interested. Herewith the solution to the problem (actually multiple problems)
Step 1 - Startup File Code
public Startup(IConfiguration configuration)
{
AWSXRayRecorder.InitializeInstance(configuration: Configuration); // Inititalizing Configuration object with X-Ray recorder
AWSSDKHandler.RegisterXRayForAllServices(); // All AWS SDK requests will be traced
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
//Make sure this is after env.IsDevelopment()
app.UseXRay("WeatherForecast");
.....
}
Make sure appsettings.json and sampling-rules.json mimic's the Sample App
Once the code runs, the log file of the app would look something like this.
I felt that the AWS.SDK package generates a lot of noise even when using the Sample App, which I omitted here. That said, DEBUG logs tend to be that way.
2020-04-15 11:34:04,262 [5] INFO Amazon.XRay.Recorder.Core.Internal.Utils.DaemonConfig - The given daemonAddress () is invalid, using default daemon UDP and TCP address 127.0.0.1:2000.
2020-04-15 11:34:04,368 [5] INFO Amazon.Runtime.Internal.RuntimePipelineCustomizerRegistry - Applying runtime pipeline customization X-Ray Registration Customization
2020-04-15 11:34:04,389 [5] INFO Amazon.XRay.Recorder.Core.Sampling.DefaultSamplingStrategy - No effective centralized sampling rule match. Fallback to local rules.
2020-04-15 11:34:04,390 [5] DEBUG Amazon.XRay.Recorder.Core.Sampling.Local.LocalizedSamplingStrategy - Can't match a rule for host = localhost, path = /index.html, method = GET
2020-04-15 11:34:04,573 [5] DEBUG **Amazon.XRay.Recorder.Core.Internal.Emitters.UdpSegmentEmitter - UDP Segment emitter endpoint: 127.0.0.1:2000.**
Ultimately, you are looking for the last line Amazon.XRay.Recorder.Core.Internal.Emitters.UdpSegmentEmitter - UDP Segment emitter endpoint: 127.0.0.1:2000.
Step 2 - Configure the Daemon
If you install the Daemon as a Windows Service locally. I ran into a couple of additional problems.
A - It doesn't put everything in one place and it doesn't look at the configuration file that it extracted. Unless you put the cfg.yaml file in System32.
B - The service probably won't have access to the .aws folder where the credentials are stored.
I fixed problems A, by doing the following (i'm sure you could achieve the same goal in multiple ways)
Since i'm not a powershell expert, I just moved the extracted content to a folder of my choosing and modified the service path in the registry to point to that folder as well as added the appropriate flags so that it logs to the location you expect as well as use the cfg.yaml file you expect.
regedit -> Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\AWSXRayDaemon
Set image path with flags -f for log file and -c for config file
C:\YOUR USER\.aws\aws-xray-daemon\xray.exe -f C:\YOUR USER\.aws\aws-xray-daemon\xray-daemon.log -c C:\YOUR USER\.aws\aws-xray-daemon\cfg.yaml
The last problem was the Daemon not having the appropriate permissions to access the credentials file inside the .aws folder.
Log file will look something like this
2020-04-15T09:35:54+02:00 [Debug] processor: sending partial batch
2020-04-15T09:35:54+02:00 [Debug] processor: segment batch size: 1. capacity: 50
2020-04-15T09:35:54+02:00 [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2020-04-15T09:35:54+02:00 [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
The NoCredentialProviders line indicates a permission issue.
I then modified the service to run as an administrator, which solved problem B.
daemon.log
2020-04-15T09:41:31+02:00 [Debug] Received request on HTTP Proxy server : /GetSamplingRules
2020-04-15T09:41:32+02:00 [Debug] processor: sending partial batch
2020-04-15T09:41:32+02:00 [Debug] processor: segment batch size: 1. capacity: 50
2020-04-15T09:41:33+02:00 [Debug] Received request on HTTP Proxy server : /GetSamplingRules
2020-04-15T09:41:33+02:00 [Info] Successfully sent batch of 1 segments (0.871 seconds)
2020-04-15T09:41:34+02:00 [Debug] processor: sending partial batch
2020-04-15T09:41:34+02:00 [Debug] processor: segment batch size: 1. capacity: 50
2020-04-15T09:41:34+02:00 [Info] Successfully sent batch of 1 segments (0.197 seconds)
You are looking for the line successfully sent batch as confirmation that the Daemon sent the trace to the X-Ray service.
Hope this helps someone.
Cheers
By looking at the daemon logs looks like trace data is not sent to the service. I think instrumentation could be the issue. I would recommend you to read this documentation for instrumentation (https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-dotnet.html). You might have to instrument outgoing HTTP calls, incoming http request and outgoing AWS SDK calls in order to see trace view of your application. Hope this helps!