the below observation is not always the case, but after some time accessing the SUT several times with ssh with root user and correct password the python code gets into trouble with:
Apr 25 05:51:56 SUT sshd[31570]: pam_tally2(sshd:auth): user root (0) tally 83, deny 10
Apr 25 05:52:16 SUT sshd[31598]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.10.10.13 user=root
Apr 25 05:52:21 SUT sshd[31568]: error: PAM: Authentication failure for root from 10.10.10.13
Apr 25 05:52:21 SUT sshd[31568]: Connection closed by 10.10.10.13 [preauth]
This is the below python code:
COMMAND_PROMPT = '.*:~ #'
SSH_NEWKEY = '(?i)are you sure you want to continue connecting'
def scp(source, dest, password):
cmd = 'scp ' + source + ' ' + dest
try:
child = pexpect.spawn('/bin/bash', ['-c', cmd], timeout=None)
res = child.expect([pexpect.TIMEOUT, SSH_NEWKEY, COMMAND_PROMPT, '(?i)Password'])
if res == 0:
print('TIMEOUT Occurred.')
if res == 1:
child.sendline('yes')
child.expect('(?i)Password')
child.sendline(password)
child.expect([pexpect.EOF], timeout=60)
if res == 2:
pass
if res == 3:
child.sendline(password)
child.expect([pexpect.EOF], timeout=60)
except:
print('File not copied!!!')
self.logger.error(str(self.child))
When the ssh is unsuccessful, this is the pexpect printout:
version: 2.3 ($Revision: 399 $)
command: /usr/bin/ssh
args: ['/usr/bin/ssh', 'root#100.100.100.100']
searcher: searcher_re:
0: re.compile(".*:~ #")
buffer (last 100 chars): :
Account locked due to 757 failed logins
Password:
before (last 100 chars): :
Account locked due to 757 failed logins
Password:
after: <class 'pexpect.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 2284
child_fd: 5
closed: False
timeout: 30
delimiter: <class 'pexpect.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0
delayafterclose: 0.1
delayafterterminate: 0.1
Any clue maybe what could it be, is it maybe anything missing or wrong configured for pam authentication on my SUT? The problem is that when the SUT starts with this pam failures then python code will always have the problem and only a reboot of the SUT seems to help :(
Manually accessing the SUT via ssh root#... is always working, even if pexpect can't!!! The account seems not to be locked according to:
SUT:~ # passwd -S root
root P 04/24/2017 -1 -1 -1 -1
I have looked into some other questions but no real solution is mentioned or could work with my python code.
Thanks in adv.
My work around is to modify for testing purpose the pam_tally configuration files. It seems that the SUT acknowledge the multiple access as a threat and locks even the root account!
By removing this entry even_deny_root root_unlock_time=5 in the several pam_tally configuration files:
/etc/pam.d/common-account:account required pam_tally2.so deny=10 onerr=fail unlock_time=600 even_deny_root root_unlock_time=5 file=/home/test/faillog
/etc/pam.d/common-auth:auth required pam_tally2.so deny=10 onerr=fail unlock_time=600 even_deny_root root_unlock_time=5 file=/home/test/faillog
Those changes will be activated dynamically no restart of service needed!
Note: after reboot those entries will be most likely back!
Related
I am trying to set a HTCondor batch system, but when I do condor_status it only shows the master in both the master and worker nodes. They both show this:
Name OpSys Arch State Activity LoadAv Mem
[master ip] LINUX X86_64 Unclaimed Idle 0.000 973
Total Owner Claimed Unclaimed Matched Preempting Backfill Drain
X86_64/LINUX 1 0 0 1 0 0 0 0
Total 1 0 0 1 0 0 0 0
Condor_restart on the master node works fine, but on the worker nodes yields this error:
ERROR
SECMAN:2010:Received "DENIED" from server for user unauthenticated#unmapped using no authentication method, which may imply host-based security. Our address was '[ip address of master]', and server's address was '[ip address of worker]'. Check your ALLOW settings and IP protocols.
Here are the config files:
of the master node:
CONDOR_HOST = [private ip of master]
DAEMON_LIST = COLLECTOR, MASTER, NEGOTIATOR, SCHEDD, STARTD
# to avoid user authentication
HOSTALLOW_READ = *
HOSTALLOW_WRITE = *
HOSTALLOW_ADMINISTRATOR = *
of the worker node:
CONDOR_HOST = [private ip of master]
DAEMON_LIST = MASTER, STARTD
# to avoid user authentication
HOSTALLOW_READ = *
HOSTALLOW_WRITE = *
HOSTALLOW_ADMINISTRATOR = *
I am allowing on the same security group:
All TCP TCP 0 - 65535
All ICMP-IPv4 All
SSH on port 22
This is how it looks like (security group ending in '6')
Apparently the issue was running condor_reconfig -full. I just reinstalled it without doing that and using systemctl restart condor instead and it worked. If someone wants to bring some insight on why it was so please do so :)
I have a Flask app running on Heroku with uwsgi server in which each user connects to his own database. I have implemented the solution reported here for a very similar situation. In particular, I have implemented the connection registry as follows:
class DBSessionRegistry():
_registry = {}
def get(self, URI, **kwargs):
if URI not in self._registry:
current_app.logger.info(f'INFO - CREATING A NEW CONNECTION')
try:
engine = create_engine(URI,
echo=False,
pool_size=5,
max_overflow=5)
session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)
a_session = Session()
self._registry[URI] = a_session
except ArgumentError:
raise Exception('Error')
current_app.logger.info(f'SESSION ID: {id(self._registry[URI])}')
current_app.logger.info(f'REGISTRY ID: {id(self._registry)}')
current_app.logger.info(f'REGISTRY SIZE: {len(self._registry.keys())}')
current_app.logger.info(f'APP ID: {id(current_app)}')
return self._registry[URI]
In my create_app() I assign a registry to the app:
app.DBregistry = DBSessionRegistry()
and whenever I need to talk to the DB I call:
current_app.DBregistry.get(URI)
where the URI is dependent on the user. This works nicely if I use uwsgi with one single process. With more processes,
[uwsgi]
processes = 4
threads = 1
sometimes it gets stuck on some requests, returning a 503 error code. I have found that the problem appears when the requests are handled by different processes in uwsgi. This is an excerpt of the log, which I commented to illustrate the issue:
# ... EVERYTHING OK UP TO HERE.
# ALL PREVIOUS REQUESTS HANDLED BY PROCESS pid = 12
INFO in utils: SESSION ID: 139860361716304
INFO in utils: REGISTRY ID: 139860484608480
INFO in utils: REGISTRY SIZE: 1
INFO in utils: APP ID: 139860526857584
# NOTE THE pid IN THE NEXT LINE...
[pid: 12|app: 0|req: 1/1] POST /manager/_save_task =>
generated 154 bytes in 3457 msecs (HTTP/1.1 200) 4 headers in 601
bytes (1 switches on core 0)
# PREVIOUS REQUEST WAS MANAGED BY PROCESS pid = 12
# THE NEXT REQUEST IS FROM THE SAME USER AND TO THE SAME URL.
# SO THERE IS NO NEED FOR CREATING A NEW CONNECTION, BUT INSTEAD...
INFO - CREATING A NEW CONNECTION
# TO THIS POINT, I DON'T UNDERSTAND WHY IT CREATED A NEW CONNECTION.
# THE SESSION ID CHANGES, AS IT IS A NEW SESSION
INFO in utils: SESSION ID: 139860363793168 # <<--- CHANGED
INFO in utils: REGISTRY ID: 139860484608480
INFO in utils: REGISTRY SIZE: 1
# THE APP AND THE REGISTRY ARE UNIQUE
INFO in utils: APP ID: 139860526857584
# uwsgi GIVES UP...
*** HARAKIRI ON WORKER 4 (pid: 11, try: 1) ***
# THE FAILED REQUEST WAS MANAGED BY PROCESS pid = 11
# I ASSUME THIS IS WHY IT CREATED A NEW CONNECTION
HARAKIRI: -- syscall> 7 0x7fff4290c6d8 0x1 0xffffffff 0x4000 0x0 0x0
0x7fff4290c6b8 0x7f33d6e3cbc4
HARAKIRI: -- wchan> poll_schedule_timeout
HARAKIRI !!! worker 4 status !!!
HARAKIRI [core 0] - POST /manager/_save_task since 1587660997
HARAKIRI !!! end of worker 4 status !!!
heroku[router]: at=error code=H13 desc="Connection closed without
response" method=POST path="/manager/_save_task"
DAMN ! worker 4 (pid: 11) died, killed by signal 9 :( trying respawn ...
Respawned uWSGI worker 4 (new pid: 14)
# FROM HERE ON, NOTHINGS WORKS ANYMORE
This behavior is consistent over several attempts: when the pid changes, the request fails. Even with a pool_size = 1 in the create_engine function the issue persists. No issue instead is uwsgi is used with one process.
I am pretty sure it is my fault, there is something I don't know or I don't understand about how uwsgi and/or sqlalchemy work. Could you please help me?
Thanks
What is hapeening is that you are trying to share memory between processes.
There are some exaplanations in these posts.
(is it possible to share memory between uwsgi processes running flask app?).
(https://stackoverflow.com/a/45383617/11542053)
You can use an extra layer to store your sessions outsite of the app.
For that, you can use uWsgi's SharedArea(https://uwsgi-docs.readthedocs.io/en/latest/SharedArea.html) which is very low level or you can user other approaches like uWsgi's caching(https://uwsgi-docs.readthedocs.io/en/latest/Caching.html)
hope it helps.
When I use the plugin for authentication at server.conf, authentication wont work, but without it, non existent users can authenticate also.
I have added the following lines in the server conf and clinet
Commands in the server.conf file
================================
mode server
tls-server
plugin /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so login
key-direction 0
================================
Commands in the client file
=================================
port 1194
proto udp
dev tun
nobind
key-direction 1
redirect-gateway def1
tls-version-min 1.2
auth SHA256
auth-user-pass
tls-client
remote-cert-tls server
resolv-retry infinite
persist-key
persist-tun
verb 3
===============================
Logs:
==============================================================
PLUGIN_CALL: POST /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so/PLUGIN_AUTH_USER_PASS_VERIFY status=1
PLUGIN_CALL: plugin function PLUGIN_AUTH_USER_PASS_VERIFY failed with status 1: /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so
TLS Auth Error: Auth Username/Password verification failed for peer
Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #7 / time = (1559124952) Wed May 29 10:15:52 2019 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
TLS Error: incoming packet authentication failed from [AF_INET6]::ffff:
openvpn[10420]: pam_unix(login:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= user=*****```
==============================================================
I have used differen approached, although in production plugin /usr/lib64/openvpn/plugin/lib/openvpn-auth-pam.so login is recommended way, but I have taken one shell script and got authentication, but remember it is dangerous.
add following lines in your /etc/openvpn/server.conf file
--verify-cline-cert none
script-security 2
auth-user-pass-verify /etc/openvpn/example.sh via-file
Now create a file in /etc/openvpn/example.sh with following content
!/bin/bash
echo "started"
username=`head -1 $1`
password=`tail -1 $1`
if grep "$username:$password" $0.passwd > /dev/null 2>&1
then
exit 0
else
if grep "$username" $0.passwd > /dev/null 2>&1
then
echo "auth-user-pass-verify: Wrong password entered for user '$username'"
else
echo "auth-user-pass-verify: Unknown user '$username'"
fi
exit 1
fi
Now create username and password in /etc/openvpn/example.sh.passwd with following content
userone:securepassworduserone
usertwo:securepasswordusertwo
Now create a client file and import and connect using your password, but this where I am stack as I don't want to provide client file.
Below is what happened to one mail send from a drupal client.
$ grep 'B6693C0977' /var/log/maillog
Jan 19 14:12:30 instance-1 postfix/pickup[19329]: B6693C0977: uid=0 from=<admin#mailgun.domainA.com>
Jan 19 14:12:30 instance-1 postfix/cleanup[20035]: B6693C0977: message-id=<20170119141230.B6693C0977#mail.instance-1.c.tw-pilot.internal>
Jan 19 14:12:30 instance-1 postfix/qmgr[19330]: B6693C0977: from=<admin#mailgun.domainA.com>, size=5681, nrcpt=1 (queue active)
Jan 19 14:12:33 instance-1 postfix/smtp[20039]: B6693C0977:
to=<username#hotmail.com>, relay=smtp.mailgun.org[52.41.19.62]:2525, delay=2.4,
delays=0.02/0.05/1.8/0.53, dsn=5.7.1, status=bounced (host smtp.mailgun.org
[52.41.19.62] said: 550 5.7.1 **Relaying denied** (in reply to RCPT TO command))
Jan 19 14:12:33 instance-1 postfix/bounce[20050]: B6693C0977: sender non-delivery notification: ABB94C0976
Jan 19 14:12:33 instance-1 postfix/qmgr[19330]: B6693C0977: removed
Relevant excerpts from my /etc/postfix/main.cf are below
# RELAYHOST SETTINGS
smtp_tls_security_level = encrypt
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost_map
and from /etc/postfix/sasl_passwd is follows
#mailgun.domainA.com postmaster#mailgun.domainA.com:password
and from /etc/postfix/relayhost_map is follows
#mailgun.domainA.com [smtp.mailgun.org]:2525
The permissions of the db files are as follows
# ls -lZ /etc/postfix/relayhost_map.db
-rw-r-----. root postfix unconfined_u:object_r:postfix_etc_t:s0 /etc/postfix/relayhost_map.db
# ls -lZ /etc/postfix/sasl_passwd.db
-rw-r-----. root postfix unconfined_u:object_r:postfix_etc_t:s0 /etc/postfix/sasl_passwd.db
The problem is
Outbound mails are not going.
No logs are shown in mailgun console.
Any insight is appreciated
I know that this is an old question now but I've just had the same issue and wanted to post a response for anyone who comes across this article in future.
I believe your issue is in /etc/postfix/relayhost_map where you should have the following, note that there are no brackets, for me it was the inclusion of brackets that was causing the issue:
#mailgun.domainA.com smtp.mailgun.org:2525
For anyone who is not using /etc/postfix/relayhost_map and is doing it all in /etc/postfix/sasl_passwd directly the same applies:
smtp.mailgun.org:2525 postmaster#mailgun.domainA.com:password
Don't forget to regenerate the postfix sasl_passwd.db file and restart the service afterwards
sudo postmap /etc/postfix/sasl_passwd
sudo systemctl restart postfix
Or sudo service postfix restart if you're on an older system / not running systemd.
Usually this is realted to problems on their platform if everything was working ok previously just open a ticket and usually they fix it in a few hours (yes that its kind of hard a few hours)
I am trying to use the puppet file function (not the type) in the following way
class iop_users {
include 's3file::curl'
include 'stdlib'
$secretpath=file('/etc/secret','dev/null')
notify { 'show secretpath':
message =>"secretpath is $secretpath"
}
s3file { '/opt/utab.yaml':
source => "mybucket/$secretpath/utab.yaml",
ensure => 'latest',
}
exec { 'fix perms':
command => '/bin/chmod 600 /opt/utab.yaml',
require => S3file['/opt/utab.yaml']
}
if ( $::virtual == 'xenhvm' and defined(S3file['/opt/utab.yaml']) ) {
$uhash=loadyaml('/opt/utab.yaml')
create_resources(iop_users::usercreate, $uhash)
}
}
If I run this then here is some typical output. The manifest fails as the initial "secret" used to find the path is not loaded
https_proxy=https://puppet:3128 puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for ip-10-40-1-68.eu-west-1.compute.internal
Info: Applying configuration version '1431531382'
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: % Total % Received % Xferd Average Speed Time Time Time Current
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: Dload Upload Total Spent Left Speed
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml] 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: curl: (56) Received HTTP code 404 from proxy after CONNECT
Error: curl -L -o /opt/utab.yaml https://s3-eu-west.amazonaws.com/mybucket//utab.yaml returned 56 instead of one of [0]
Error: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: change from notrun to 0 failed: curl -L -o /opt/utab.yaml https://s3-eu-west.amazonaws.com/mybucket//utab.yaml returned 56 instead of one of [0]
Notice: /Stage[main]/Iop_users/Exec[fix perms]: Dependency Exec[fetch /opt/utab.yaml] has failures: true
Warning: /Stage[main]/Iop_users/Exec[fix perms]: Skipping because of failed dependencies
Notice: secretpath is
Notice: /Stage[main]/Iop_users/Notify[show secretpath]/message: defined 'message' as 'secretpath is '
Notice: Finished catalog run in 1.28 seconds
However on the same host that the above puppet agent run fails on, if I use "apply" to try it outside of the context of a manifest, it works fine
puppet apply -e '$z=file("/etc/secret") notify { "z": message => $z}'
Notice: Compiled catalog for ip-x.x.x.x.eu-west-1.compute.internal in environment production in 0.02 seconds
Notice: wombat
Notice: /Stage[main]/Main/Notify[z]/message: defined 'message' as 'wombat
'
Notice: Finished catalog run in 0.03 seconds
What am I doing wrong? Are there any better alternative approaches I could make?
As usual I was confused about the way puppet works
Apparently, functions are always executed on the master
So any files being loaded in this way must be on the master
As soon as I added a "/etc/secret" file to the puppetmaster it all worked