Is there a way to logformat Dropwizard access.log - jetty

I have used the following in my configuration file, still I am getting standard server logs in access.log, is there a way to modify it? Dropwizard version 0.7
server:
adminConnectors:
-
port: 8889
type: http
applicationConnectors:
-
acceptorThreads: 7
port: 8888
selectorThreads: 14
type: http
maxQueuedRequests: 1024
maxThreads: 1024
requestLog:
appenders:
-
archive: true
archivedFileCount: 3
archivedLogFilenamePattern: /var/log/access-%i.log
currentLogFilename: /var/log/access.log
logFormat: '[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%-5level]'
maxFileSize: 200MB
threshold: ALL
timeZone: IST
type: file-size-rolled
timeZone: IST

You need to put the section under "server" (like you have), like this:
server:
requestLog:
appenders:
- type: console
threshold: ALL
logFormat: '%h [%date{ISO8601}] "%r" %s %b %D [%i{User-Agent}]'
and you need to use a "newer" version (I think from 0.9.something) to get the logFormat stuff working.
I think it is because they changed from Jetty access log implementation to using logback access (http://logback.qos.ch/access.html) in newer versions.

Related

Message is being sent with "send_message" in ejabberd using postman but not received by client

I have configured and install ejabberd on the ubuntu 22.04 and I have successfully configured and create one user with administrator right and as well as create some users into it.
I am using Version
OS - ubuntu 22.04 LTS
ejabberd 22.10
And also configured ejabberd API by mod_http_api and then I test APIs with POSTMAN, almost every (link) "API reference" working fine with it except send_message.
Here is my ejabberd.yml configuration:-
hosts:
- B660M-D2H-DDR4
- localhost
- XX.XXX.37.XX
loglevel: info
ca_file: /opt/ejabberd/conf/cacert.pem
certfiles:
- /opt/ejabberd/conf/server.pem
## If you already have certificates, list them here
# certfiles:
# - /etc/letsencrypt/live/domain.tld/fullchain.pem
# - /etc/letsencrypt/live/domain.tld/privkey.pem
listen:
-
port: 5222
ip: "::"
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5223
ip: "::"
tls: true
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5269
ip: "::"
module: ejabberd_s2s_in
max_stanza_size: 524288
-
port: 5443
ip: "::"
module: ejabberd_http
tls: true
request_handlers:
/admin: ejabberd_web_admin
/api: mod_http_api
/bosh: mod_bosh
/captcha: ejabberd_captcha
/upload: mod_http_upload
/ws: ejabberd_http_ws
-
port: 5280
ip: "::"
module: ejabberd_http
request_handlers:
/admin: ejabberd_web_admin
/.well-known/acme-challenge: ejabberd_acme
/api: mod_http_api
-
port: 3478
ip: "::"
transport: udp
module: ejabberd_stun
use_turn: true
## The server's public IPv4 address:
# turn_ipv4_address: "203.0.113.3"
## The server's public IPv6 address:
# turn_ipv6_address: "2001:db8::3"
-
port: 1883
ip: "::"
module: mod_mqtt
backlog: 1000
s2s_use_starttls: optional
acl:
admin:
user: "admin#localhost"
local:
user_regexp: ""
loopback:
ip:
- 127.0.0.0/8
- ::1/128
access_rules:
local:
allow: local
allow: XX.XXX.37.XX
c2s:
deny: blocked
allow: all
announce:
allow: admin
configure:
allow: admin
muc_create:
allow: local
pubsub_createnode:
allow: local
trusted_network:
allow: loopback
api_permissions:
"console commands":
from:
- ejabberd_ctl
who: all
what: "*"
"admin access":
who:
access:
allow:
- acl: admin
oauth:
scope: "ejabberd:admin"
access:
allow:
- acl: admin
what:
- "*"
- "!stop"
- "!start"
"public commands":
who:
ip: 127.0.0.1/8
what:
- status
- connected_users_number
shaper:
normal:
rate: 3000
burst_size: 20000
fast: 100000
shaper_rules:
max_user_sessions: 10
max_user_offline_messages:
5000: admin
100: all
c2s_shaper:
none: admin
normal: all
s2s_shaper: fast
modules:
mod_adhoc: {}
mod_admin_extra: {}
mod_announce:
access: announce
mod_avatar: {}
mod_blocking: {}
mod_bosh: {}
mod_caps: {}
mod_carboncopy: {}
mod_client_state: {}
mod_configure: {}
mod_disco: {}
mod_fail2ban: {}
mod_http_api: {}
mod_http_upload:
put_url: https://#HOST#:5443/upload
custom_headers:
"Access-Control-Allow-Origin": "https://#HOST#"
"Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
"Access-Control-Allow-Headers": "Content-Type"
mod_last: {}
mod_mam:
## Mnesia is limited to 2GB, better to use an SQL backend
## For small servers SQLite is a good fit and is very easy
## to configure. Uncomment this when you have SQL configured:
## db_type: sql
assume_mam_usage: true
default: always
mod_mqtt: {}
mod_muc:
access:
- allow
access_admin:
- allow: admin
access_create: muc_create
access_persistent: muc_create
access_mam:
- allow
default_room_options:
mam: true
mod_muc_admin: {}
mod_offline:
access_max_user_messages: max_user_offline_messages
mod_ping: {}
mod_privacy: {}
mod_private: {}
mod_proxy65:
access: local
max_connections: 5
mod_pubsub:
access_createnode: pubsub_createnode
plugins:
- flat
- pep
force_node_config:
## Avoid buggy clients to make their bookmarks public
storage:bookmarks:
access_model: whitelist
mod_push: {}
mod_push_keepalive: {}
mod_register:
## Only accept registration requests from the "trusted"
## network (see access_rules section above).
## Think twice before enabling registration from any
## address. See the Jabber SPAM Manifesto for details:
## https://github.com/ge0rg/jabber-spam-fighting-manifesto
ip_access: trusted_network
mod_roster:
versioning: true
mod_s2s_dialback: {}
mod_shared_roster: {}
mod_stream_mgmt:
resend_on_timeout: if_offline
mod_stun_disco: {}
mod_vcard: {}
mod_vcard_xupdate: {}
mod_version:
show_os: false
I have two observation with messages (send_message):
client to client (PSI)
postman to Client
In first observation I have success to exchange messages between users on psi(PSI) but when we try to send message with postman using "mod_http_api" API to the client, I am getting result 200 ok, but Message is not being delivered. And it is not showing anywhere (logs).
Am I missing something that is important for receiving a message using ejabberd's REST API with postman?
What a strange problem, I cannot reproduce it. You didn't show your command query, and didn't mention what exact client and configuration you are using.
Summary: Check if the command works correctly when using the ejabberdctl command line tool, and use "normal" message type, and send to bare JID, and use another client for example Gajim (just for debugging the problem).
Details:
I installed ejabberd 22.10 from source code, copied your configuration, disabled cert and tls options, started ejabberd, registered account, logged in it and executed this command:
$ ejabberdctl send_message headline uuu#localhost user1#localhost Restart aaa
The client that was logged in user1#localhost received the stanza, and displayed the headline message:
<message to='user1#localhost'
from='uuu#localhost'
type='headline'
id='18154938236359942834'>
<body>aaa</body>
<subject>Restart</subject>
</message>
Please note: in XMPP, "headline" messages are not stored in the offline storage: they are only received by online sessions with positive priority. Maybe you are sending "headline" messages to sessions that are offline, or online with negative priority, or online with no initial presence?
It's preferable to send a "normal" message, which are stored offline:
ejabberdctl send_message normal uuu#localhost user1#localhost ThisisNormal bbb
Also, make sure your client is logged in with positive priority (this is the standard).

ID apache-service-running in SLS apache.service.running is not a dictionary

I'm trying to install and configure apache using the v1.2.2 saltstack apache-formula
salt server-test state.apply apache test=true
But keep getting the following error:
server-test:
Data failed to compile:
ID apache-service-running in SLS apache.service.running is not a dictionary
my master salt path looks like that:
file_roots:
base:
- /srv/salt/states
- /srv/salt/formulas/php
- /srv/salt/formulas/nginx
- /srv/salt/formulas/apache-formula
- /srv/salt/formulas/mysql-formula
- /srv/salt/files
pillar_roots:
base:
- /srv/salt/pillar/base
dev:
- /srv/salt/pillar/dev
prod:
- /srv/salt/pillar/prod
I can't figure out where the problem is! any hints?
pillar/base/apache/server-test.sls
apache:
manage_service_states: False
lookup:
version: '2.4'
# for each site name there must be a config in salt://configs/webserver/apache2/sites/
site_names: [ 'server-test' ]
security:
ServerTokens: Prod
modules:
enabled:
- ssl
- alias
- rewrite
- headers
- shib
- http2
#disabled:
# - php7.3
# others are managed elsewhere
# - status
# - proxy
#- proxy_fcgi
mpm:
module: mpm_event
params:
start_servers: 3
min_spare_threads: 50
max_spare_threads: 100
thread_limit: 64
threads_per_child: 25
max_request_workers: 2000
server_limit: 80
max_connections_per_child: 0
The formula is broken: #383
manage_service_states: False doesn't work.

Connecting cassandra-stress to AWS Keyspaces

I've provisions a Keyspace on AWS and in order to make sure it can achieve our desired performance I'm trying to run the cassandra-stress tool on it and compare it to other architectures we're experimenting with.
I managed to connect to it using the following cqlshrc:
[connection]
port = 9142
factory = cqlshlib.ssl.ssl_transport_factory
[ssl]
validate = true
certfile = /root/.cassandra/AmazonRootCA1.pem
And the following command (hoping that soon enough there will be Python3 support, the development was completed this February according to their Jira ticket):
cqlsh cassandra.eu-central-1.amazonaws.com 9142 -u "myuser-at-722222222222" -p "12/12ZmHmtD1klsDk9cgqt/XXXXXXXXxUz6Sy687z/U=" --ssl --cqlversion="3.4.4"
Surprisingly or not, when using the official AWS guides things tend to work.
So I went on and tried connecting the cassandra-stress tool (I have it inside a Docker container, I'd rather keep my OS Java free) to the same Keyspace.
First I converted the AWS AmazonRootCA1.pem into cassandra_truststore.jks using the following commands (explained here):
openssl x509 -outform der -in AmazonRootCA1.pem -out temp_file.der
keytool -import -alias cassandra -keystore cassandra_truststore.jks -file temp_file.der
Now when I'm trying to run the actual tool like this:
./cassandra-stress write -node cassandra.eu-central-1.amazonaws.com -port native=9142 thrift=9142 jmx=9142 -transport truststore=/root/.cassandra/cassandra_truststore.jks truststore-password=mypassword -mode native cql3 user="myuser-at-722222222222" password="12/12ZmHmtD1klsDk9cgqt/XXXXXXXXxUz6Sy687z/U="
I'm getting the following error:
******************** Stress Settings ********************
Command:
Type: write
Count: -1
No Warmup: false
Consistency Level: LOCAL_ONE
Target Uncertainty: 0.020
Minimum Uncertainty Measurements: 30
Maximum Uncertainty Measurements: 200
Key Size (bytes): 10
Counter Increment Distibution: add=fixed(1)
Rate:
Auto: true
Min Threads: 4
Max Threads: 1000
Population:
Sequence: 1..1000000
Order: ARBITRARY
Wrap: true
Insert:
Revisits: Uniform: min=1,max=1000000
Visits: Fixed: key=1
Row Population Ratio: Ratio: divisor=1.000000;delegate=Fixed: key=1
Batch Type: not batching
Columns:
Max Columns Per Key: 5
Column Names: [C0, C1, C2, C3, C4]
Comparator: AsciiType
Timestamp: null
Variable Column Count: false
Slice: false
Size Distribution: Fixed: key=34
Count Distribution: Fixed: key=5
Errors:
Ignore: false
Tries: 10
Log:
No Summary: false
No Settings: false
File: null
Interval Millis: 1000
Level: NORMAL
Mode:
API: JAVA_DRIVER_NATIVE
Connection Style: CQL_PREPARED
CQL Version: CQL3
Protocol Version: V4
Username: myuser-at-722222222222
Password: *suppressed*
Auth Provide Class: null
Max Pending Per Connection: 128
Connections Per Host: 8
Compression: NONE
Node:
Nodes: [cassandra.eu-central-1.amazonaws.com]
Is White List: false
Datacenter: null
Schema:
Keyspace: keyspace1
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Replication Strategy Pptions: {replication_factor=1}
Table Compression: null
Table Compaction Strategy: null
Table Compaction Strategy Options: {}
Transport:
factory=org.apache.cassandra.thrift.TFramedTransportFactory; truststore=/root/.cassandra/cassandra_truststore.jks; truststore-password=mypassword; keystore=null; keystore-password=null; ssl-protocol=TLS; ssl-alg=SunX509; store-type=JKS; ssl-ciphers=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA;
Port:
Native Port: 9142
Thrift Port: 9142
JMX Port: 9142
Send To Daemon:
*not set*
Graph:
File: null
Revision: unknown
Title: null
Operation: WRITE
TokenRange:
Wrap: false
Split Factor: 1
java.lang.RuntimeException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: cassandra.eu-central-1.amazonaws.com/3.127.48.183:9142 (com.datastax.driver.core.exceptions.TransportException: [cassandra.eu-central-1.amazonaws.com/3.127.48.183] Channel has been closed))
at org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:220)
at org.apache.cassandra.stress.settings.SettingsSchema.createKeySpacesNative(SettingsSchema.java:79)
at org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:69)
at org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:228)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:57)
at org.apache.cassandra.stress.Stress.run(Stress.java:143)
at org.apache.cassandra.stress.Stress.main(Stress.java:62)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: cassandra.eu-central-1.amazonaws.com/3.127.48.183:9142 (com.datastax.driver.core.exceptions.TransportException: [cassandra.eu-central-1.amazonaws.com/3.127.48.183] Channel has been closed))
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:233)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:79)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1424)
at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:403)
at org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:160)
at org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:211)
... 6 more
I've tried changing some parameters such as the jks password etc. (Just in case I was wrong) but I got a different error message so it's probably not the case.
Did I miss something?
Try using TLP Stress instead.
tlp-stress run RandomPartitionAccess -d 10m --host cassandra.us-east-1.amazonaws.com --port 9142 --username alice --password fLyWYFlTCD5J2gzGAZ –ssl --max-requests 4000 --dc us-east-2 --threads 10
https://thelastpickle.com/tlp-stress/

Google Cloud Scheduler - HTTP headers not respected

I'm trying to send push to my app using Google Cloud Scheduler:
gcloud beta scheduler jobs create http PUSH --schedule="0 * * * *" --uri="https://fcm.googleapis.com/fcm/send" --description="desc" --headers="Authorization: key=<AUTHKEY> --http-method="POST" --message-body="{\"to\":\"/topics/allDevices\",\"priority\":\"low\",\"data\":{\"success\":\"ok\"}}"
The result is always 401 Unauthorized. After issuing command:
gcloud beta scheduler jobs describe PUSH
I do not get these headers back:
description: desc
httpTarget:
body: eyJ0byI6Ii90b3BpY3MvYWxsnByaW9yaXR5IjoRGV2aWNlcyIsIiaGlnaCIsImRhdGEiOnsic3VjY2VzcyI6Im9rIn19 <--- THIS IS WEIRD
headers:
Content-Type: application/octet-stream
User-Agent: Google-Cloud-Scheduler
httpMethod: POST
uri: https://fcm.googleapis.com/fcm/send
lastAttemptTime: '2018-11-07T20:32:37.657408Z'
name: projects/..../locations/europe-west1/jobs/PUSH
retryConfig:
maxBackoffDuration: 3600s
maxDoublings: 16
maxRetryDuration: 0s
minBackoffDuration: 5s
schedule: 0 * * * *
scheduleTime: '2018-11-07T21:00:00.681498Z'
state: ENABLED
status:
code: 16
timeZone: Etc/UTC
userUpdateTime: '2018-11-07T20:29:15Z'
The first question about body:
body:eyJ0byI6Ii90b3BpY3MvYWxsnByaW9yaXR5IjoRGV2aWNlcyIsIiaGlnaCIsImRhdGEiOnsic3VjY2VzcyI6Im9rIn19
<--- THIS IS WEIRD
This is the base64 encoding of
{\"to\":\"/topics/allDevices\",\"priority\":\"low\",\"data\":{\"success\":\"ok\"}}
Google is taking your --message-body and encoding it in base64.
Next regarding the header issue. You have a several errors in your '--headers`.
--headers="Authorization: key=AUTHKEY
You are missing a quote mark after AUTHKEY. I will assume that this issue is just editing mistake creating the question. (Note I could not figure out how to include the less-than and greater-than characters in this response).
However, the syntax for --headers is wrong. The --headers expects KEY=VALUE, not KEY:VALUE. In this example the KEY is Authorization and the VALUE is key=AUTHKEY.
--headers="Authorization=key=AUTHKEY"

django & send_mail in docker leads to SMTPServerDisconnected

I setup a google email backend for django and all is working fine until i run the whole stuff in docker-compose containers.
SMTPServerDisconnected at /events/register/4/
please run connect() first
Request Method: GET
Request URL: http://192.168.99.100:80/events/register/4/
Django Version: 1.11.3
Exception Type: SMTPServerDisconnected
Exception Value:
please run connect() first
Exception Location: /usr/local/lib/python3.6/smtplib.py in send, line 359
Python Executable: /usr/local/bin/uwsgi
Python Version: 3.6.2
Python Path:
['.',
'',
'/usr/local/lib/python36.zip',
'/usr/local/lib/python3.6',
'/usr/local/lib/python3.6/lib-dynload',
'/usr/local/lib/python3.6/site-packages',
'/app']
Server time: Tue, 25 Jul 2017 11:07:45 +0200
Any idea what's going wrong here? Why is it disconnecting in a container?
Just figured it out:
In the docker-compose.yml i have something like this:
env_file:
- ".env"
The content of the .env was:
key = value
This runs great on a local machine loading the .env file first.
However docker needs a more strict format:
key=value
No spaces!!
The error was that my EMAIL_* settings wasn't set correctly with spaced key/value pairs