I'm trying to scp a jar file to an EC2 instance that hosts a service I'm working on. I'm able to ssh into that EC2 instance without issue. When I attempt to scp the file with the following command:
scp -vvv foo.jar instance-id:/tmp/
I'm prompted for a password and then get the following output after I authenticate
debug1: Authentication succeeded (password).
Authenticated to <instance id> (via proxy).
...
debug1: Entering interactive session.
debug1: pledge: filesystem full
...
debug1: Sending command: scp -v -t /tmp/
debug2: channel 0: request exec confirm 1
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow#openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: chan_shutdown_read: channel 0: (i0 o0 sock -1 wfd 3 efd 7 [write])
debug2: channel 0: input open -> closed
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: chan_shutdown_write: channel 0: (i3 o1 sock -1 wfd 6 efd 7 [write])
debug2: channel 0: output drain -> closed
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
lost connection
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
#0 client-session (t4 r0 i3/0 o3/0 e[write]/0 fd -1/-1/7 sock -1 cc -1)
debug3: send packet: type 1
debug1: fd 0 clearing O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
Transferred: sent 2752, received 2780 bytes, in 0.3 seconds
Bytes per second: sent 9360.6, received 9455.8
debug1: Exit status 1
As far as I can tell I'm successfully connecting to the host. It seems like the file transfer immediately exits afterwards. I've done some digging on similar questions and responders have said that this can indicate shell problems. I'm not echoing anything in my zshrc which was a commonly indicated issue. Any insight or debugging suggestions would be appreciated!
Related
I'm trying to set up a data channel between a server written in C++ and a Python client. The server crashes with SIGSEGV error when it tries to set a local session description created in method "CreateAnswer"
The server and client exchange SDP information via WebSocket and should open the data channel without video and audio streams. Both programs are working under docker-compose in different services. So no audio or video devices are provided. I use WebRTC Native API from m76 branch.
Crashing handler:
static void OnAnswerCreated(WebRTCManagerImpl* impl_, webrtc::SessionDescriptionInterface* desc) {
LOG4CPLUS_INFO_FMT(impl_->logger_, "Answer created session_id %s", desc->session_id().c_str());
std::string offer_string;
desc->ToString(&offer_string);
LOG4CPLUS_DEBUG_FMT(impl_->logger_, "Offer string: %s", offer_string.c_str());
impl_->peer_connection_->SetLocalDescription(&impl_->set_session_description_observer_, desc);
impl_->signaling_->SendSessionDescription(*desc);
};
I create my connection with this factory:
webrtc::PeerConnectionFactoryDependencies CreatePeerConnectionFactoryDependencies() {
webrtc::PeerConnectionFactoryDependencies dependencies;
dependencies.network_thread = nullptr;
dependencies.worker_thread = nullptr;
dependencies.signaling_thread = nullptr;
dependencies.call_factory = webrtc::CreateCallFactory();
dependencies.task_queue_factory = webrtc::CreateDefaultTaskQueueFactory();
dependencies.event_log_factory = absl::make_unique<webrtc::RtcEventLogFactory>(dependencies.task_queue_factory.get());
cricket::MediaEngineDependencies mediaDependencies;
mediaDependencies.task_queue_factory = dependencies.task_queue_factory.get();
mediaDependencies.adm = rtc::scoped_refptr<webrtc::FakeAudioDeviceModule>(new webrtc::FakeAudioDeviceModule);
mediaDependencies.audio_encoder_factory = webrtc::CreateBuiltinAudioEncoderFactory();
mediaDependencies.audio_decoder_factory = webrtc::CreateBuiltinAudioDecoderFactory();
mediaDependencies.audio_processing = webrtc::AudioProcessingBuilder().Create();
mediaDependencies.video_encoder_factory = webrtc::CreateBuiltinVideoEncoderFactory();
mediaDependencies.video_decoder_factory = webrtc::CreateBuiltinVideoDecoderFactory();
dependencies.media_engine = cricket::CreateMediaEngine(std::move(mediaDependencies));
return dependencies;
}
webrtc::PeerConnectionFactoryDependencies deps = CreatePeerConnectionFactoryDependencies();
deps.signaling_thread = signaling_thread_.get();
// deps.network_thread = network_thread.get();
// deps.worker_thread = worker_thread.get();
peer_connection_factory_ = webrtc::CreateModularPeerConnectionFactory(std::move(deps));
The call stack:
<unknown> 0x0000000001e798f7
webrtc::PeerConnection::ValidateSessionDescription(webrtc::SessionDescriptionInterface const*, cricket::ContentSource) 0x00000000005e74dc
webrtc::PeerConnection::SetLocalDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*) 0x00000000005bb677
void webrtc::ReturnType<void>::Invoke<webrtc::PeerConnectionInterface, void (webrtc::PeerConnectionInterface::*)(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*), webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>(webrtc::PeerConnectionInterface*, void (webrtc::PeerConnectionInterface::*)(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*), webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*) 0x000000000059b814
webrtc::MethodCall2<webrtc::PeerConnectionInterface, void, webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>::OnMessage(rtc::Message*) 0x0000000000598f5f
webrtc::internal::SynchronousMethodCall::Invoke(rtc::Location const&, rtc::Thread*) 0x00000000007198fc
webrtc::MethodCall2<webrtc::PeerConnectionInterface, void, webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*>::Marshal(rtc::Location const&, rtc::Thread*) 0x0000000000593706
webrtc::PeerConnectionProxyWithInternal<webrtc::PeerConnectionInterface>::SetLocalDescription(webrtc::SetSessionDescriptionObserver*, webrtc::SessionDescriptionInterface*) 0x000000000058c982
preprocessor::p2p::WebRTCManager::WebRTCManagerImpl::OnAnswerCreated webrtc_manager.cpp:226
std::__invoke_impl<void, void (*&)(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, webrtc::SessionDescriptionInterface*), preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*&, webrtc::SessionDescriptionInterface*> invoke.h:60
std::__invoke<void (*&)(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, webrtc::SessionDescriptionInterface*), preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*&, webrtc::SessionDescriptionInterface*> invoke.h:95
std::_Bind<void (*(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, std::_Placeholder<1>))(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, webrtc::SessionDescriptionInterface*)>::__call<void, webrtc::SessionDescriptionInterface*&&, 0ul, 1ul>(std::tuple<webrtc::SessionDescriptionInterface*&&>&&, std::_Index_tuple<0ul, 1ul>) functional:467
std::_Bind<void (*(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, std::_Placeholder<1>))(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, webrtc::SessionDescriptionInterface*)>::operator()<webrtc::SessionDescriptionInterface*, void>(webrtc::SessionDescriptionInterface*&&) functional:549
std::_Function_handler<void (webrtc::SessionDescriptionInterface*), std::_Bind<void (*(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, std::_Placeholder<1>))(preprocessor::p2p::WebRTCManager::WebRTCManagerImpl*, webrtc::SessionDescriptionInterface*)> >::_M_invoke(std::_Any_data const&, webrtc::SessionDescriptionInterface*&&) std_function.h:316
std::function<void (webrtc::SessionDescriptionInterface*)>::operator()(webrtc::SessionDescriptionInterface*) const std_function.h:706
preprocessor::p2p::CreateSessionDescriptionObserver::OnSuccess webrtc_manager.cpp:79
webrtc::WebRtcSessionDescriptionFactory::OnMessage(rtc::Message*) 0x0000000000b90785
rtc::MessageQueue::Dispatch(rtc::Message*) 0x00000000005712f8
rtc::Thread::ProcessMessages(int) 0x0000000000553398
rtc::Thread::Run() 0x0000000000552993
rtc::Thread::PreRun(void*) 0x0000000000552950
start_thread 0x00007ffff76536db
clone 0x00007ffff608a88f
WebRTC logs:
(audio_processing_impl.cc:435): Capture analyzer activated: 0
Capture post processor activated: 0
Render pre processor activated: 0
(webrtc_voice_engine.cc:196): WebRtcVoiceEngine::WebRtcVoiceEngine
(webrtc_video_engine.cc:479): WebRtcVideoEngine::WebRtcVideoEngine()
(webrtc_voice_engine.cc:219): WebRtcVoiceEngine::Init
(webrtc_voice_engine.cc:227): Supported send codecs in order of preference:
(webrtc_voice_engine.cc:230): opus/48000/2 { minptime=10 useinbandfec=1 } (111)
(webrtc_voice_engine.cc:230): ISAC/16000/1 (103)
(webrtc_voice_engine.cc:230): ISAC/32000/1 (104)
(webrtc_voice_engine.cc:230): G722/8000/1 (9)
(webrtc_voice_engine.cc:230): ILBC/8000/1 (102)
(webrtc_voice_engine.cc:230): PCMU/8000/1 (0)
(webrtc_voice_engine.cc:230): PCMA/8000/1 (8)
(webrtc_voice_engine.cc:230): CN/32000/1 (106)
(webrtc_voice_engine.cc:230): CN/16000/1 (105)
(webrtc_voice_engine.cc:230): CN/8000/1 (13)
(webrtc_voice_engine.cc:230): telephone-event/48000/1 (110)
(webrtc_voice_engine.cc:230): telephone-event/32000/1 (112)
(webrtc_voice_engine.cc:230): telephone-event/16000/1 (113)
(webrtc_voice_engine.cc:230): telephone-event/8000/1 (126)
(webrtc_voice_engine.cc:233): Supported recv codecs in order of preference:
(webrtc_voice_engine.cc:236): opus/48000/2 { minptime=10 useinbandfec=1 } (111)
(webrtc_voice_engine.cc:236): ISAC/16000/1 (103)
(webrtc_voice_engine.cc:236): ISAC/32000/1 (104)
(webrtc_voice_engine.cc:236): G722/8000/1 (9)
(webrtc_voice_engine.cc:236): ILBC/8000/1 (102)
(webrtc_voice_engine.cc:236): PCMU/8000/1 (0)
(webrtc_voice_engine.cc:236): PCMA/8000/1 (8)
(webrtc_voice_engine.cc:236): CN/32000/1 (106)
(webrtc_voice_engine.cc:236): CN/16000/1 (105)
(webrtc_voice_engine.cc:236): CN/8000/1 (13)
(webrtc_voice_engine.cc:236): telephone-event/48000/1 (110)
(webrtc_voice_engine.cc:236): telephone-event/32000/1 (112)
(webrtc_voice_engine.cc:236): telephone-event/16000/1 (113)
(webrtc_voice_engine.cc:236): telephone-event/8000/1 (126)
(apm_helpers.cc:32): Setting AGC mode to 0
(audio_processing_impl.cc:699): Highpass filter activated: 0
(audio_processing_impl.cc:717): Gain Controller 2 activated: 0
(audio_processing_impl.cc:719): Pre-amplifier activated: 0
(webrtc_voice_engine.cc:309): WebRtcVoiceEngine::ApplyOptions: AudioOptions {aec: 1, agc: 1, ns: 1, hf: 1, swap: 0, audio_jitter_buffer_max_packets: 200, audio_jitter_buffer_fast_accelerate: 0, audio_jitter_buffer_min_delay_ms: 0, audio_jitter_buffer_enable_rtx_handling: 0, typing: 1, experimental_agc: 0, extended_filter_aec: 0, delay_agnostic_aec: 0, experimental_ns: 0, residual_echo_detector: 1, }
(render_delay_buffer.cc:341): Applying total delay of 5 blocks.
(matched_filter.cc:450): Filter 0: start: 0 ms, end: 128 ms.
(matched_filter.cc:450): Filter 1: start: 96 ms, end: 224 ms.
(matched_filter.cc:450): Filter 2: start: 192 ms, end: 320 ms.
(matched_filter.cc:450): Filter 3: start: 288 ms, end: 416 ms.
(matched_filter.cc:450): Filter 4: start: 384 ms, end: 512 ms.
(audio_processing_impl.cc:699): Highpass filter activated: 0
(audio_processing_impl.cc:717): Gain Controller 2 activated: 0
(audio_processing_impl.cc:719): Pre-amplifier activated: 0
(apm_helpers.cc:48): Echo control set to 1 with mode 0
(audio_processing_impl.cc:699): Highpass filter activated: 0
(audio_processing_impl.cc:717): Gain Controller 2 activated: 0
(audio_processing_impl.cc:719): Pre-amplifier activated: 0
(audio_processing_impl.cc:699): Highpass filter activated: 0
(audio_processing_impl.cc:717): Gain Controller 2 activated: 0
(audio_processing_impl.cc:719): Pre-amplifier activated: 0
(apm_helpers.cc:62): NS set to 1
(webrtc_voice_engine.cc:447): Stereo swapping enabled? 0
(webrtc_voice_engine.cc:452): NetEq capacity is 200
(webrtc_voice_engine.cc:458): NetEq fast mode? 0
(webrtc_voice_engine.cc:464): NetEq minimum delay is 0
(webrtc_voice_engine.cc:470): NetEq handle reordered packets? 0
(webrtc_voice_engine.cc:481): Delay agnostic aec is enabled? 0
(webrtc_voice_engine.cc:491): Extended filter aec is enabled? 0
(webrtc_voice_engine.cc:501): Experimental ns is enabled? 0
(webrtc_voice_engine.cc:511): Setting AGC to 1
(webrtc_voice_engine.cc:533): Typing detection is enabled? 1
(audio_processing_impl.cc:699): Highpass filter activated: 1
(audio_processing_impl.cc:717): Gain Controller 2 activated: 0
(audio_processing_impl.cc:719): Pre-amplifier activated: 0
(webrtc_sdp.cc:3255): Ignored line: a=sctpmap:5000 webrtc-datachannel 65535
(rtc_event_log_impl.cc:63): Creating legacy encoder for RTC event log.
(peer_connection_factory.cc:361): Using default network controller factory
(bitrate_prober.cc:69): Bandwidth probing enabled, set to inactive
(paced_sender.cc:421): ProcessThreadAttached 0xec072e20
(cpu_info.cc:53): Available number of cores: 8
(aimd_rate_control.cc:105): Using aimd rate control with back off factor 0.85
(remote_bitrate_estimator_single_stream.cc:71): RemoteBitrateEstimatorSingleStream: Instantiating.
(remote_estimator_proxy.cc:44): Maximum interval between transport feedback RTCP messages (ms): 250
(openssl_identity.cc:44): Making key pair
(peer_connection.cc:5531): Local and Remote descriptions must be applied to get the SSL Role of the SCTP transport.
(openssl_identity.cc:92): Returning key pair
(openssl_certificate.cc:58): Making certificate for WebRTC
(openssl_certificate.cc:108): Returning certificate
(p2p_transport_channel.cc:519): Set backup connection ping interval to 25000 milliseconds.
(p2p_transport_channel.cc:528): Set ICE receiving timeout to 2500 milliseconds
(p2p_transport_channel.cc:535): Set ping most likely connection to 0
(p2p_transport_channel.cc:542): Set stable_writable_connection_ping_interval to 2500
(p2p_transport_channel.cc:555): Set presume writable when fully relayed to 0
(p2p_transport_channel.cc:564): Set regather_on_failed_networks_interval to 300000
(p2p_transport_channel.cc:583): Set receiving_switching_delay to 1000
(jsep_transport_controller.cc:1214): Creating DtlsSrtpTransport.
(dtls_srtp_transport.cc:61): Setting RTCP Transport on 0 transport 0
(dtls_srtp_transport.cc:66): Setting RTP Transport on 0 transport dc004830
(p2p_transport_channel.cc:465): Received remote ICE parameters: ufrag=YAvY, renomination disabled
(peer_connection.cc:4185): Session: 7301418690559709073 Old state: kStable New state: kHaveRemoteOffer
(peer_connection.cc:5531): Local and Remote descriptions must be applied to get the SSL Role of the SCTP transport.
(peer_connection.cc:5559): Local and Remote descriptions must be applied to get the SSL Role of the session.
(paced_sender.cc:293): Elapsed time (12680 ms) longer than expected, limiting to 2000 ms
Signal: SIGSEGV (Segmentation fault)
I guess the problem is not in callback but in the connection initialization. But what am I doing wrong?
I've found the error in my code:
peer_connection_->SetRemoteDescription(&set_session_description_observer_, desc.get());
I passed the raw pointer then release the smart one with the memory.
I'm trying to set up a Kafka broker with SSL enabled communication.
I followed the guide as described in the Kafka documentation here under 7.2: https://kafka.apache.org/documentation/#security
When I'm trying to execute the command ./kafka-console-producer.sh --broker-list ec2-ip.eu-central-1.compute.amazonaws.com:9093 -topic test --producer.config ../config/producer.properties
I get:
ERROR [Producer clientId=console-producer] Connection to node -1 (ec2-ip.eu-central-1.compute.amazonaws.com/172.31.42.33:9093) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2019-08-22 13:15:56,728] ERROR [Producer clientId=console-producer] Connection to node -1 (ec2-ip.eu-central-1.compute.amazonaws.com/172.31.42.33:9093) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
Testing the SSl connection in general with
openssl s_client -debug -connect ec2-ip.eu-central-1.compute.amazonaws.com:9093 -tls1
results in
0020 - a0 41 45 81 42 b6 83 d3-2b 94 02 9f ac 42 73 42 .AE.B...+....BsB
---
Certificate chain
0 s:/C=Unknown/ST=Unknown/L=Unknown/O=Unknown/OU=Unknown/CN=Unknown
i:/C=Unknown/ST=Unknown/L=Unknown/O=Unknown/OU=Unknown/CN=Unknown
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDuTCCAqGgAwIBAgIEPsshdTANBgkqhkiG9w0BAQsFADBsMRAwDgYDVQQGEwdV
bmtub3duMRAwDgYDVQQIEwdVbmtub3duMRAwDgYDVQQHEwdVbmtub3duMRAwDgYD
VQQKEwdVbmtub3duMRAwDgYDVQQLEwdVbmtub3duMRAwDgYDVQQDEwdVbmtub3du
MB4XDTE5MDgyMjEyNTIwOFoXDTIwMDgyMTEyNTIwOFowbDEQMA4GA1UEBhMHVW5r
bm93bjEQMA4GA1UECBMHVW5rbm93bjEQMA4GA1UEBxMHVW5rbm93bjEQMA4GA1UE
ChMHVW5rbm93bjEQMA4GA1UECxMHVW5rbm93bjEQMA4GA1UEAxMHVW5rbm93bjCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAI5EKxmSDLQvW/ePu9mB3qdf
E4cG1ok1kI48S6yL4gchx2egIC6Zjr5gBhuek36mhKV82H1nVYfDzJVyxoyaTZ5q
ux8xq+ooNN/chKHy0ExpCZ2HHvrQiTPQit//dyA5zc5yOyRcJo/m2EhYfRMuLfFM
kx1mEg2cs6Qa/hd6jKjCSvbcKDfrOhf4rLagHzI+2GRQfF6LXxYBE3Ti7u6fkqog
o67CTrNR6IER1b4wGIeHga8eXiVhOuP4JiWdpF69i5py0RVfzsesP0W8RB2qElte
+7EwShaFO72Xa7MPiDz1rE4D5pohQ09AWGb/B0YIcI7qh9jyInbj4eWhhlnd1lsC
AwEAAaNjMGEwQAYDVR0RBDkwN4I1ZWMyLTM1LTE1OC0yMzItMTc2LmV1LWNlbnRy
YWwtMS5jb21wdXRlLmFtYXpvbmF3cy5jb20wHQYDVR0OBBYEFNGugrNUmFiVKVyU
Uiepd0OUme1wMA0GCSqGSIb3DQEBCwUAA4IBAQAToYYUsvQ6MlOQ+HHWnfax5L0u
M/IrExjcrSmnxAevnTem5+Hnz+AmlRHSF1/FfCWhi2XMSHBHXU52uhIt+kDLNhFh
D17Je0FydthW8roBsLZ+XsuHjBi8MXo41YAA19+cjW60/se3+kWxOHH1rEfV9ky/
K9kWufQPl3U3fbUzh0rWwriSDaPUTjHg/IeROhVe8gt/QeBfTU2KUqnln6K58PfC
kE2c2zCTQwpsfwi5gqvrdqWZc/A5qriv1l39zKU1icfTZzdeejcjtbCdFqajOqar
lMXwyl1bPuPiOHas7K9J9MSREQYql89gbN5V9gO//IHmM97eLt4npFSpu8Cp
-----END CERTIFICATE-----
subject=/C=Unknown/ST=Unknown/L=Unknown/O=Unknown/OU=Unknown/CN=Unknown
issuer=/C=Unknown/ST=Unknown/L=Unknown/O=Unknown/OU=Unknown/CN=Unknown
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1451 bytes and written 236 bytes
Verification error: self signed certificate
---
New, TLSv1.0, Cipher is ECDHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1
Cipher : ECDHE-RSA-AES256-SHA
Session-ID: 5D5E9882C4E8DD80427108B85C7649F0EC1AA87B9DAA0EA1FCE8C7063C83A61B
Session-ID-ctx:
Master-Key: 861677EDA9E19E3D8926889A9B0DE299593C7FCD49DB8A55EBF4D222800169E16CDB74DCE0EC392A3B491268FCCF5F07
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1566480514
Timeout : 7200 (sec)
Verify return code: 18 (self signed certificate)
Extended master secret: yes
---
which looks ok to me.
Configuration of the Broker:
listeners=SSL://ec2-ip.eu-central-1.compute.amazonaws.com:9093,PLAINTEXT://ec2-ip.eu-central-1.compute.amazonaws.com:9092
group.initial.rebalance.delay.ms=0
advertised.listeners=PLAINTEXT://ec2-ip.eu-central-1.compute.amazonaws.com:9092,SSL://ec2-ip.eu-central-1.compute.amazonaws.com:9093
ssl.endpoint.identification.algorithm=SSL
ssl.keystore.location=/kafka/kafka_2.12-2.2.0/config/certs/server.keystore.jks
ssl.keystore.password=Welcome01
ssl.key.password=Welcome01
ssl.truststore.location=/kafka/kafka_2.12-2.2.0/config/certs/server.truststore.jks
ssl.truststore.password=Welcome01
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.keystore.type=JKS
ssl.truststore.type=JKS
Configuration of the Producer:
bootstrap.servers=localhost:9092
security.protocol=SSL
ssl.truststore.location=/kafka/kafka_2.12-2.2.0/config/certs/client.truststore.jks
ssl.truststore.password=Welcome01
logging.level.org.apache.kafka: DEBUG
log4j.rootLogger=DEBUG
Do you see some misconfiguration? How can I proceed to investigate further to get down to the issue?
Thanks to mazaneiche I was able to get the error that leads to the failed handshake:
kafka-producer-network-thread | console-producer, fatal error: 46: General SSLEngine problem
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
%% Invalidated: [Session-3, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384]
kafka-producer-network-thread | console-producer, SEND TLSv1.2 ALERT: fatal, description = certificate_unknown
Why does this error occur? How can it be fixed?
Apparently not the issue here but I have been struggling with the same error for a while and in my case the kafka-console-producer.sh was failing with the similar errors:
$ kafka-console-producer.sh --broker-list 192.168.1.173:9093 --topic kafka-security-topic --producer.config ~/ssl/client-ssl-auth.properties
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
checking with debug
$ export KAFKA_OPTS="-Djavax.net.debug=ssl"
kafka-producer-network-thread | console-producer, fatal error: 46: General SSLEngine problem
java.security.cert.CertificateException: No subject alternative names present
%% Invalidated: [Session-1, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384]
kafka-producer-network-thread | console-producer, SEND TLSv1.2 ALERT: fatal, description = certificate_unknown
[2021-07-13 16:21:53,909] ERROR [Producer clientId=console-producer] Connection to node -1 (/192.168.1.173:9093) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
In my case turned out the reason is using IP instead of hostname when specifying broker-list parameter.
That worked:
$ kafka-console-producer.sh --broker-list red03:9093 --topic kafka-security-topic --producer.config ~/ssl/client-ssl-auth.properties
I'm setting a local vpn environment, and I want to capture traffic locally through virtual network interface then forwarding them to real destinations through a physical network interface binding socket. However, I cannot even connect real destination after setting up a tun virtual network interface.
My testing machine:
Linux testing-VirtualBox 3.19.0-15-generic #15-Ubuntu SMP Thu Apr 16 23:32:37 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
First I successfully create a virtual network interface named tun0 as below:
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:192.168.2.1 P-t-P:192.168.2.1 Mask:255.0.0.0
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
For brevity, I just add the target server's ip address into route table:
route add -host 45.113.192.102 dev tun0
The route table is as below:
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 xxx.xx.xxx.xxx 0.0.0.0 UG 0 0 0 eth0
45.113.192.102 0.0.0.0 255.255.255.255 UH 0 0 0 tun0
xxx.xx.xxx.xxx 0.0.0.0 255.255.255.128 U 0 0 0 eth0
192.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 tun0
xxx.xx.xxx.xxx is my internal host/gateway ip address.
At last, I create a socket and bind the socket into physical network interface. I use libuv here and should no matter to the issue.
struct sockaddr_in remote_addr;
memset(&remote_addr, 0, sizeof(remote_addr));
remote_addr.sin_family = AF_INET;
remote_addr.sin_port = ntohs(443);
inet_pton(AF_INET, "45.113.192.102", &remote_addr.sin_addr);
uv_os_sock_t sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (sock < 0) {
std::cout << "ERROR--- create socket failed\n";
return -1;
}
int32_t r;
r = setsockopt(sock, SOL_SOCKET, SO_BINDTODEVICE, "eth0", strlen("eth0"));
if (r != 0) {
std::cout << "setsockopt failed: " << errno;
return -1;
}
uv_tcp_init(g_uv_loop, &socket_handle);
r = uv_tcp_open(&socket_handle, sock);
uv_connect_t connect_req;
r = uv_tcp_connect(&connect_req, &socket_handle,
(struct sockaddr *) &remote_addr, _tcp_connect_cb);
I run my code and found I cannot connect to "45.113.192.102".
I capture the traffic through wireshark and found that my program has sent SYN to "45.113.192.102", and "45.113.192.102" also replied SYN,ACK. However, after that seems my program did not send ACK, which causes connecting failed.
In the following, Client continuously sends [TCP Spurious Retransmission] SYN and Server replys [TCP Retransmission] SYN,ACK.
I have written Google cloud function (HTTP trigger). It tests successfully within the google cloud interface. Now I need to call it from Unity Game (C# .Net core).
First I tried to run it from browser. I took URL shown in Google cloud function interface. It does not work. I do not get stackdriver log (it does not even hit the URL?).
Is it because it needs oauth 2.0 token?
I am creating Http request , embedding JSON
request.ContentType = "application/json";
request.Method = "POST";
request.Headers["Authorization"] = "Bearer " + token;
//token obtained from oauth 2.0 JWT call
byte[] JsonContentBytes = Encoding.UTF8.GetBytes(strjsonReqContent);
// my Json object. requestJSON.ToString());
request.ContentLength = JsonContentBytes.Length;
Here I tried with Oauth 2.0 token as well. (It works for storage API. My service account for cloud function has project editor access)
I saw one post about using cloud function from outside the cloud project. It did not solve my problem.
Please let me know all the criteria for using cloud functions from outside the cloud project.
EDIT1:
Thanks for the suggestion. Here is the log from Curl execution. I am not used to web debugging. Please suggest what should I check further. I could make out that it is 'Bad Request'. The URL is the one I picked from cloud function console.
>curl -v -X POST https://us-central1- myproject.cloudfunctions.net/myfunctiontest -H "Content-Type:application/json" -d '{"username":"myemail#mydomain.com","password":"password1"}'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 172.217.163.174...
* TCP_NODELAY set
* Connected to us-central1-myproject.cloudfunctions.net (172.217.163.174) port 443 (#0)
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 212 bytes...
* schannel: sent initial handshake data: sent 212 bytes
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: encrypted data got 4096
* schannel: encrypted data buffer: offset 4096 length 4096
* schannel: encrypted data length: 4028
* schannel: encrypted data buffer: offset 4028 length 4096
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 5052 length 5052
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 6076 length 6076
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 7100 length 7100
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 8124 length 8124
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: encrypted data got 317
* schannel: encrypted data buffer: offset 8441 length 9148
* schannel: sending next handshake data: sending 93 bytes...
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 2/3)
* schannel: encrypted data got 284
* schannel: encrypted data buffer: offset 284 length 9148
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with us-central1-myproject.cloudfunctions.net port 443 (step 3/3)
* schannel: stored credential handle in session cache
> POST /logontest HTTP/1.1
> Host: us-central1-myproject.cloudfunctions.net
> User-Agent: curl/7.55.1
> Accept: */*
> Content-Type:application/json
> Content-Length: 59
>
* upload completely sent off: 59 out of 59 bytes
* schannel: client wants to read 102400 bytes
* schannel: encdata_buffer resized 103424
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: encrypted data got 506
* schannel: encrypted data buffer: offset 506 length 103424
* schannel: decrypted data length: 477
* schannel: decrypted data added: 477
* schannel: decrypted data cached: offset 477 length 102400
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: decrypted data buffer: offset 477 length 102400
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 477
* schannel: decrypted data buffer: offset 0 length 102400
< HTTP/1.1 400 Bad Request
< Content-Type: text/html
< Function-Execution-Id: qyx8g4vv8hn1
< X-Cloud-Trace-Context: 6f8012ea791b4a0f900ac4b84debaf5f;o=1
< Date: Thu, 02 May 2019 13:23:14 GMT
< Server: Google Frontend
< Content-Length: 192
< Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>The browser (or proxy) sent a request that this server could not
understand.</p>
* Connection #0 to host us-central1-myproject.cloudfunctions.net left intact
The 'bad request' may not be due to URL since I took from the cloud function UI. Please suggest how I should proceed. Thanks.
Is it possible to filter debug data function text? I want to display commands at one time and full output at the other (I want for example to filter out Adding handle: send: 0). All the time I get a lot of messages. I want something nice like Filezilla Short messages .
Here is my code for debug function and below it a message. I have verbose enabled
int Uploader::DebugDataCallBack(CURL* handle, curl_infotype infotype, char* msg, size_t size, void* f)
{
int level= 1; //debug info 0-None, 1-necessary 2 - All TODO: Should come from config
switch(level) //error level
{
case 0:
{
break; //do nothing
}
case 1:
{
//only necessary, skip headers
if(infotype==CURLINFO_TEXT)
{
static_cast<Uploader*>(f)->SendMessage(wxString(msg));
}
}
default:
{
//full debug messages
static_cast<Uploader*>(f)->SendMessage(wxString(msg));
}
}
return 0;//must return 0
}
----------Thu Dec 26 14:14:40 2013----------
STATE: INIT => CONNECT handle 0x7fffd0001a08; line 998 (connection #-5000)
[14:14:40]
STATE: INIT => CONNECT handle 0x7fffd0001a08; line 998 (connection #-5000)
[14:14:40]
Rebuilt URL to: ftp://ftp.mysite.com/
[14:14:40]
Rebuilt URL to: ftp://ftp.mysite.com/
[14:14:40]
About to connect() to ftp.mysite.com port 21 (#0)
[14:14:40]
About to connect() to ftp.mysite.com port 21 (#0)
[14:14:40]
Trying 31.170.162.203...
[14:14:40]
Trying 31.170.162.203...
[14:14:40]
Adding handle: conn: 0x7fffd0013b48
[14:14:40]
Adding handle: conn: 0x7fffd0013b48
[14:14:40]
Adding handle: send: 0
[14:14:40]
Adding handle: send: 0
[14:14:40]
Adding handle: recv: 0
[14:14:40]
Adding handle: recv: 0
[14:14:40]
Curl_addHandleToPipeline: length: 1
[14:14:40]
Curl_addHandleToPipeline: length: 1
[14:14:40]
0x7fffd0001a08 is at send pipe head!
[14:14:40]
0x7fffd0001a08 is at send pipe head!
[14:14:40]
- Conn 0 (0x7fffd0013b48) send_pipe: 1, recv_pipe: 0
[14:14:40]
- Conn 0 (0x7fffd0013b48) send_pipe: 1, recv_pipe: 0
[14:14:40]
STATE: CONNECT => WAITCONNECT handle 0x7fffd0001a08; line 1045 (connection #0)
[14:14:40]
STATE: CONNECT => WAITCONNECT handle 0x7fffd0001a08; line 1045 (connection #0)
[14:14:40]
Connected to ftp.mysite.com (31.170.162.203) port 21 (#0)
[14:14:40]
Connected to ftp.mysite.com (31.170.162.203) port 21 (#0)
[14:14:40]
FTP 0x7fffd0013fe0 (line 3174) state change from STOP to WAIT220
[14:14:40]
FTP 0x7fffd0013fe0 (line 3174) state change from STOP to WAIT220
[14:14:40]
STATE: WAITCONNECT => PROTOCONNECT handle 0x7fffd0001a08; line 1158 (connection #0)
[14:14:40]
STATE: WAITCONNECT => PROTOCONNECT handle 0x7fffd0001a08; line 1158 (connection #0)
[14:14:40]
First Enable curl by putting verbose to 1L that is CURLOPT_VERBOSE in curl_easy_setopt. Theb set the debug function, that is CURLOPT_DEBUGFUNCTION to receive the debug messages. Then use codes infotypes to filter out what you want. If you want to get command/response like I wanted just take message from Header in/out. Here is piece of code just to show it!
switch(infotype)
{
case CURLINFO_HEADER_OUT:
{
wxString message = _("COMMAND: ")+wxString(msg);
SendMessage(message);
break;
}
case CURLINFO_HEADER_IN:
{
wxString message = _("RESPONSE: ")+wxString(msg);
SendMessage(message);
break;
}
}