High latency after transferring Lua codebase from custom HTTP server to Nginx - c++

For the past few years we have been running a custom, pre-forking, HTTP server written in C++, libevent and embedded Lua (LuaJit using the exact same LuaJit shared library that openresty, an Nginx package, uses). It's got almost the same architecture as Nginx just by coincidence.
Most of the business functionality of this server is in the Lua script "handlers" and when we saw Nginx supported Lua we thought we'd move our code base over to Nginx and take advantage of all the community support that Nginx has. We got all the code moved over and our testing shows everything working, as far as the functionality being preserved, however....
We have run into a problem. The latency for an HTTP request on the old custom, C++ server was typically 2ms round trip. The new server based on Nginx is up around 60ms. The weird thing is, our internal measurements of the actual Lua code executing (using Lua os.clock() to time code regions) shows the two server versions are virtually identical as far as Lua execution speed.
Using various profilers doesn't show any extremely large area of slowness on the Nginx code. pcre is the highest with about 6% of the time spent.
One more thing. The setup we have has a Proxy Nginx server talking to our Nginx Lua Server, both on the same machine, talking over 127.0.0.1, so it's LAN => Nginx Proxy => Nginx Lua Server.
ldd nginx
linux-vdso.so.1 => (0x00007ffc02ff3000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f88d0b46000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f88d090f000)
libluajit-5.1.so.2 => /d1/apps/mt3/openresty/luajit/lib/libluajit-5.1.so.2 (0x00007f88d069a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f88d0418000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f88d01db000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f88cff79000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f88cfb81000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f88cf97d000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f88cf765000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f88cf3da000)
/lib64/ld-linux-x86-64.so.2 (0x00007f88d0d72000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f88cf1c4000)
Anyone got any ideas?

Related

Gatsby develop throws this error: Error: listen EADDRINUSE: address already in use 127.0.0.1:52179

This error starts when I change the time zone of the Wordpress from where the data is pulled via gatsby-source-wordpress plugin. Before I changed the time zone everything was working fine.
The complete error:
C:\Users\XXXX\Documents\XXXX\XXX\2021\www\XXX-www\node_modules\yoga-layout-prebuilt\yoga-layout\build\Release\nbind.js:53
throw ex;
^
Error: listen EADDRINUSE: address already in use 127.0.0.1:51116
at Server.setupListenHandle [as _listen2] (net.js:1318:16)
at listenInCluster (net.js:1366:12)
at GetAddrInfoReqWrap.doListen (net.js:1503:7)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:69:8)
Emitted 'error' event on Server instance at:
at emitErrorNT (net.js:1345:8)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
code: 'EADDRINUSE',
errno: -4091,
syscall: 'listen',
address: '127.0.0.1',
port: 51116
}
The development server is deployed, but right after that the app starts to update posts, create categories and other Wordpress data stuff:
info gatsby-source-wordpress update mediaItem getz-150lb (#425)⠀
⠀
info gatsby-source-wordpress create mediaItem tolva-getz-450lb (#445)⠀
⠀
info gatsby-source-wordpress create mediaItem bomba-transfer-co2 (#423)⠀
⠀
info gatsby-source-wordpress create tag getz (#417)
To finally crash throwing the mentioned error.
Gatsby info:
System:
OS: Windows 10 10.0.19042
CPU: (4) x64 Intel(R) Core(TM) i5-4200U CPU # 1.60GHz
Binaries:
Node: 14.15.5 - C:\Program Files\nodejs\node.EXE
npm: 7.5.3 - C:\Program Files\nodejs\npm.CMD
Browsers:
Chrome: 90.0.4430.212
Edge: Spartan (44.19041.964.0), Chromium (90.0.818.66)
npmPackages:
gatsby: ^3.4.1 => 3.6.0
gatsby-plugin-google-analytics: ^3.5.0 => 3.6.0
gatsby-plugin-image: ^1.5.0 => 1.6.0
gatsby-plugin-manifest: ^3.5.0 => 3.6.0
gatsby-plugin-react-helmet: ^4.5.0 => 4.6.0
gatsby-plugin-sharp: ^3.5.0 => 3.6.0
gatsby-plugin-sitemap: ^4.1.0 => 4.2.0
gatsby-source-filesystem: ^3.5.0 => 3.6.0
gatsby-source-wordpress: ^5.5.0 => 5.6.0
gatsby-transformer-sharp: ^3.5.0 => 3.6.0
npmGlobalPackages:
gatsby-cli: 3.5.0
Any ideas in how to fix this?
Regards!
According to this GitHub thread the issue is related more to Node dependencies rather than Gatsby.
Try lowering your Node version to 10, remove your node_modules as well as your package-lock.json, and install the dependencies again as the thread suggests.
Try this release gatsby-source-wordpress#5.7.0-alpha-wordpress.0+fe81b6e978 .It will fix the issue, it worked for me and suggested here on github github issue
Stumbled upon this page today when we were having the same issue... at first I completely skimmed over the "WordPress timezone" aspect, but then realized that we had just made the same change on our WordPress.
In my case, I had one post that I had published shortly before the timezone setting change, and that post currently exists with created/modified timestamps in the future. I set that post to "draft" and now gatsby develop is running.

SSL not available in watson-developer-cloud example microphone-speech-to-text.py

Hello i try to get the following example running:
microphone-speech-to-text
I run this python script on Gentoo Linux device with a Python 2.7.6.
The device is connected to the internet via a mobile Hotspot from my mobile phone.
Unfortunately it does not work and get the following output:
Enter CTRL+C to end recording...
Error received: SSL not available.
Connection closed
I ran the script on my windows and it worked fine.
SSL libs are installed:
ldconfig -p | grep ssl
vocon_ssl.so (libc6) => /opt/aldebaran/lib/vocon_ssl.so
libssl.so.1.0.0 (libc6) => /usr/lib/libssl.so.1.0.0
libssl.so (libc6) => /usr/lib/libssl.so
libgnutls-openssl.so.27 (libc6) => /usr/lib/libgnutls-openssl.so.27
libgnutls-openssl.so (libc6) => /usr/lib/libgnutls-openssl.so
libevent_openssl-2.0.so.5 (libc6) => /usr/lib/libevent_openssl-2.0.so.5
If i run another example (speech_to_text_v1)
I get the correct result from watson followed by:
Error received: SSL not available.
Does anybody have an idea what could be the issue here?
Thanks
I could get rid of the "SSL not available" error by
pip install backports.ssl-match-hostname
I got this hint from here.
Also here is mentioned that websocket_client depends on backports.ssl-match-hostname for Python 2.x
After installing the package i got:
Error received: _ssl.c:334: No root certificates specified for verification of other-side certificates.
This could be fixed temporarily by calling disable_SSL_verification() of SpeechToTextV1 before the processing.
To fix it in the long term an approach could be downgrading the websocket-client library to 0.47.0 link1 link2
Another approach to get rid of the "No root certificates specified" Error is setting the Environment variable WEBSOCKET_CLIENT_CA_BUNDLE that is checked by the websocket library.
e.g.
os.environ['WEBSOCKET_CLIENT_CA_BUNDLE'] = '/etc/ssl/certs/ca-certificates.crt'
If I were to hazard a guess it would be that your Gentoo Linux device hasn't been setup with the appropriate TLS / SSL libraries. As a test run a list models request through cURL on your device, as per the API documentation - https://cloud.ibm.com/apidocs/speech-to-text#list-models
curl -X GET -u "apikey:{apikey}" "https://stream.watsonplatform.net/speech-to-text/api/v1/models"
That is going over TLS so, if you can get that to work, you may have better luck with the python application.

App not loading: "localhost didn’t send any data. ERR_EMPTY_RESPONSE"

I just spent a week upgrading rails to 4.2.10, and upgraded a whole bunch of gems and such, and now that I've finally convinced rails to run without errors, my app won't actually load.
This page isn’t working
localhost didn’t send any data.
ERR_EMPTY_RESPONSE
I have no idea where to even start trying to debug this.
The development.log file doesn't have any errors. The entire file says:
^[[1m^[[36mAppConfiguration Load (0.5ms)^[[0m ^[[1mSELECT `configurations`.* FROM `configurations` WHERE `configurations`.`type` IN ('AppConfiguration') AND `configurations`.`name` = 'Default configuration' LIMIT 1^[[0m
Cache read: Spree::Config
Dalli::Server#connect memcached:11211
Cache fetch_hit: Spree::Config
Cache read: Spree::Config
Cache fetch_hit: Spree::Config
Cache read: Spree::Config
Cache fetch_hit: Spree::Config
And nothing new is printed when I hit the page.
docker-compose logs -f abc just says:
=> Booting WEBrick
=> Rails 4.2.10 application starting in development on http://localhost:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
[2018-08-17 14:48:01] INFO WEBrick 1.3.1
[2018-08-17 14:48:01] INFO ruby 2.1.5 (2014-11-13) [x86_64-linux-gnu]
[2018-08-17 14:48:01] INFO WEBrick::HTTPServer#start: pid=18 port=3000
Would my Gemfile be useful? What else would helpful to share?
It's not a local browser issue, because curl responds with (52) Empty reply from server, as well.
In Firefox: The connection to the server was reset while the page was loading.
ETA: Solved by adding -b 0.0.0.0 at the end of bundle exec rails server

ssl passenger nginx rails 4 routing combine wrong port with https

I folowed the rails tutorial org tuto and made a web site
Installed passenger, nginx. Works perfectly on prod env on heroku=>
http://klat.herokuapp.com/ (app not finished but still better than nothing)
Here i use the =>
config.force_ssl = true
(only for heroku)
my routes from routes.rb =>
Rails.application.routes.draw do
get 'users/new'
root 'static_pages#home'
get 'help' => 'static_pages#help'
get 'about' => 'static_pages#about'
get 'contact' => 'static_pages#contact'
get 'login' => 'static_pages#login'
get 'signup' => 'users#new'
resources :users
end
~/.bash_profile and ~/.bashrc got
#a generated key
export SECRET_KEY_BASE="my_very_big_generated_key"
export SECRET_TOKEN="my_very_big_generated_key"
I generated with openssl my own certificates
ls ../cert/
server.crt server.csr server.key server.key.org server.pem
And I launch my server with this command in local
passenger start -p 2000 --ssl-certificate ../cert/server.csr --ssl-port 20001 -e production
I passed two days to get to this point but i can't find out why my server starts at
http ://0.0.0.0:2000/
and redirects at
https ://0.0.0.0:2000
and no one page is accessible: error
An error occurred during a connection to 0.0.0.0:2000. SSL received a record that exceeded the maximum permissible length. (Error code: ssl_error_rx_record_too_long)
But in logs we clearly see
App 12832 stdout:
App 12854 stdout:
App 12854 stdout: Started HEAD "/" for 127.0.0.1 at 2015-04-24 18:09:30 +0200
App 12854 stdout: Processing by StaticPagesController#home as HTML
{... rendering homepage wiht _header.erb _footer.erb etc ...}
So i can visit my website non rendered with css at address:
http: //127.0.0.1:2000/
and i still getting those messages from my nginx server
ActionController::RoutingError (No route matches [GET] "/images/rails.png"):
ActionController::RoutingError (No route matches [GET] "/stylesheets/application.css"):
{...and more ...}
I commented in every file in my ruby on rails app all ssl_force
when i uncomment in config/production.rb
config.force_ssl = true
for local use, i get
on
http ://127.0.0.1:2000/ => redirect to https ://127.0.0.1:2000/ and get => (Error code: ssl_error_rx_record_too_long)
http ://127.0.0.1 => unable to locate
https ://127.0.0.1 => unable to locate
http ://127.0.0.1:2000 => unable to locate
http ://127.0.0.1:2001 => unable to locate
https ://127.0.0.1:2000 => unable to locate
https ://127.0.0.1:2001 => unable to locate
Just can't find out why my openssl generated keys don't work, the pages are not rendered with css and other ressources, it's maybe a routing problem...
A little help is much appreciated!
PS: (But please no help such as "use Thin or Puma or another server, i have a constraint to use at choice nginx, lighthttpd or Apache (the obvious answer is nginx for me)")
And to set up my own ssl certification in my own server is a constraint too in my studies
finally
the command i did to start my server without errors is
passenger start --ssl-certificate ../cert/server.crt --ssl-certificate-key ../cert/server.key -e production --ssl
without specifying the ssl port and standart port
i added
gem 'rack-ssl', :require => 'rack/ssl'
to my gem file
and did several bundle install/--deployment etc...
set up in production.rb
config.force_ssl = true
and in application_controller.rb
force_ssl
so my connection is encrypted now as i connect to
https://0.0.0.0:3000/
BUT...
I'm still unhappy with all those configurations because of the presence of the port number in the url
Anyway, it will do for now...

Install CURL extension in WAMP

I want to install CURL extension in my WAMP server. I tried so many solutions including this
But it's not work for me. Can any one help me.
To turn curl extension on in WAMPServer all you need to do is using the wampmanager icon menus do :-
wampmanager -> PHP -> Extensions -> php_curl
If that is ticked then the extension is enabled, if not then click it and repeat the process to check it is now ticked.
If this fails, edit php.ini (careful as there are more than one of these )
wampmanager -> PHP -> php.ini
And check this line does not have a ; semi colon in front of it
extension=php_curl.dll
Now if you are using WAMPServer 2.4 or earlier version you also may need to copy these 2 dll's from the php folder to the Apache folder.
copy \wamp\bin\php\{phpversion}\libeay32.dll \wamp\bin\apache\{apacheversion}\bin
copy \wamp\bin\php\{phpversion}\ssleay32.dll \wamp\bin\apache\{apacheversion}\bin
If you are using WAMPServer 2.5 it should have already created SYMLINKS for these 2 files in the \wamp\bin\apache{apacheversion}\bin folder, but if not then do this to make it redo the symlink processing
wampmanager -> Apache -> {apacheversion}
And just click the Apache version number and it will rebuild the symlinks.
It's probably also a good idea to activate the php_openssl extension as I belive cUrl can use this in certain situations. Do this just like you did to activate the php_curl extension.
If you want to use cURL from the command line, then you also need to manually edit the php.ini file that the CLI uses.
Edit \wamp\bin\php{phpversion}\php.ini and uncomment php_curl and php_openssl
Added additional simple curl test
To test if PHP has cURL extension loaded run this simple script
<?php
// Get curl version array
$version = curl_version();
echo '<pre>' . print_r($version,true) . '</pre>';
?>
If it is loaded then you will get a result something like this but with different values as you are probably running a different version
Array
(
[version_number] => 466432
[age] => 3
[features] => 3005
[ssl_version_number] => 0
[version] => 7.30.0
[host] => i386-pc-win32
[ssl_version] => OpenSSL/0.9.8y
[libz_version] => 1.2.7
[protocols] => Array
(
[0] => dict
[1] => file
[2] => ftp
[3] => ftps
[4] => gopher
[5] => http
[6] => https
[7] => imap
[8] => imaps
[9] => ldap
[10] => pop3
[11] => pop3s
[12] => rtsp
[13] => scp
[14] => sftp
[15] => smtp
[16] => smtps
[17] => telnet
[18] => tftp
)
)
If it is not loaded you will just see an error on the page, or in the error log if you are not showing errors to the browser.
ADDITIONAL INFO
If you are using 64bit PHP V5.3x or V5.4x, there were some releases that came with a bad version of php_curl.dll. You can get a fixed version from http://www.anindya.com/php-5-4-3-and-php-5-3-13-x64-64-bit-for-windows/
Download the cURL version that corresponds to your PHP version under "Fixed curl extensions:".
You will need the Thread Safe version to run on WAMPServer i.e. the one without -nts- in the name.