iptables with regex url matching - regex

What I need to achieve
I need count how many users open correct websites and sub websites
e.g. `http://stackoverflow.com'
'https://stackoverflow.com/questions'
What I have
router set on Centos 7
Iptables
https://github.com/xnsystems/kpcre/wiki/iptables-string-regex
I can't use squid.
this count system can't change IP headers (src and dst IP address)
that is why I need count use iptables with log option
or maybe there is software which will count urls without any packet modification
Problem
I can't find a way in which I will see how many IP adress (how many users) hit correct url
My iptables row looks like this and doesn't work
I mean match all urls (with all sub urls) not only http://stackoverflow.com
sudo iptables -I OUTPUT -p tcp --dport 80 -m string --string "^http://stackoverflow.com$" --algo regex -j ACCEPT
Please help me with this case.
UPDATE:
Below three examples please try make them works
sudo iptables -I OUTPUT -p tcp --dport 80 -m string --string stackoverflow.com --algo regex -j ACCEPT
sudo iptables -I OUTPUT -p tcp --dport 80 -m string --string stackoverflow.com/documentation --algo regex -j ACCEPT
sudo iptables -I OUTPUT -p tcp --dport 80 -m string --string stackoverflow.com/questions'; --algo regex -j ACCEPT
As you see I used --algo regex.
If you know another solution without it please suggest.

I checked the specification link, it is not very complete (e.g. \bseems know, but is not explained), but I think the regex you need is:
^(http:\/\/){0,1}[a-zA-Z0-9_-]+(\.[a-zA-Z0-9_-]+)+(\/[a-zA-Z0-9_-]+){0,}$
UPDATE: Allow http://prefix and /documentation suffix.
Please try.
The use of capture groups () is not specified, but that usually works.
If you give examples which match and should not or do not match and should, I will adapt this answer.
The easiest adaptation is to insert more characters (to be allowed for the URL) between the [a-zA-Z0-9_and the -], in two places.

The line that includes the URL in the packet has not only the URL but also some extra keywords around it.
If you want to block "http://stackoverflow.com" only, then you should specify the URL like this:
iptables -I OUTPUT -p tcp --dport 80 -m string --string "/\/ .+Host: stackoverflow.com/si" --algo regex -j DROP
or if you want to block "http://stackoverflow.com/questions" only, then you should specify the URL like this:
iptables -I OUTPUT -p tcp --dport 80 -m string --string "/\/questions .+Host: stackoverflow.com/si" --algo regex -j DROP

Related

Something is flusing iptables after some hours of fail2ban reload

Several days ago I asked this question Confuse about fail2ban behavior with firewallD in Centos 7
It is a large text with several comments.
It seems something starts flushing iptables after some hours of fail2ban restart I don't get what it is.
A couple of months ago I moved a few Virtual Hosts from a dedicated server I used for more than 10 years to a Contabo VPS.
All goes fine but fail2ban jail. Prisoners escape. :)
My move was from Centos 6 to Centos 7 Webmin/Virtualmin LAMP fail2ban; leaving /etc/sysconfig/iptables, now using firewalld.
As said, after some hours of fail2ban restart, and after some successfully banned IPs, as #sebres suggested, something is flushing iptables because of the symptom "after effects" like
2019-12-05 16:55:20,856 fail2ban.action [1514]: ERROR iptables -w -n -L INPUT | grep -q 'f2b-proftpd[ \t]' -- stdout: ''
and "already banned" notices.
None of the changes I tried in default configurations changed that.
At the end I deleted the Webmin module to manage fail2ban and reinstalled the service.
Renamed /etc/fail2ban to keep backup configurations.
rpm -qa | grep -i fail2ban
then
yum remove fail2ban-server
yum remove fail2ban-firewalld
yum install fail2ban-firewalld (also installs -server)
yum install fail2ban-systemd
then copied old jail.local to new /etc/fail2ban directory
[DEFAULT]
banaction = iptables-multiport
banaction_allports = iptables-allports
[sshd]
enabled = true
port = ssh
maxretry = 4
bantime = 7200
[ssh-ddos]
enabled = true
port = ssh,sftp
filter = sshd-ddos
[webmin-auth]
enabled = true
port = 10000
[proftpd]
enabled = true
bantime = -1
[postfix]
enabled = true
bantime = -1
[dovecot]
enabled = true
bantime = -1
[postfix-sasl]
enabled = true
bantime = -1
I also checked cron jobs to see if something can be flushing iptables in any way.
At this time I have running, periodically, a script to manually reject those "already banned" IPs once.
firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='xxx.xxx.xxx.xxx' reject"
So my question is how to know what is flusing iptables.
UPADATE 1
After update to stable V 0.10 fail2ban release it seemed problems gone but after 5 days they started again.
Previously, after v0.9 restart, problems started after few hours.
UPADATE 2
Running fail2ban-client -d I got "Found no accessible config files for 'filter.d/sshd-ddos'". That's because I kept the old ssh-ddos config in jail.conf.
So, a subquestion is if I'm right simply making this change (at least no errors in fail2ban-client -d
#filter = sshd-ddos
filter = sshd
mode = agressive (as suggested by #sebres)
Here's the output of fail2ban-client -d
"No, the after effect is there because something is flushing rules, not vice versa"
I understand that, I'm not that fluent in English speaking, I meant that that was a symptom that something happens so the effect.
"So which banning action do you use really?"
Sorry my poor knowledge on this matter. Is that what is included in [Default] part of jail.local?
"(for example can you exclude some service implemented by Contabo installed or integrated in your VPS, that doing that?"
I asked them some time ago but their answer was "...we are providing our customers with the basic installations..." nothing that technical. They have several VPS services and I don't see other people complaining about that.
UPDATE 3
The first jail.local (from fresh Webmin/Virtualmin install) actions were
action = firewallcmd-ipset[]
action_ = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
I changed by
banaction = iptables-multiport
banaction_allports = iptables-allports
some time ago.
Now I went back with firewallcmd-ipset as [DEFAULT] and this is the fail2ban-client -d output.
I'll check fail2ban.log. .... After few hours, problems again.
About firewallD Webmin has a section with defined zones/rules and tools to manage them instead of having to write commands in shell. Nothing more.
I cannot imagine fail2ban blame here, but to exclude it (or some action of fail2ban is broken on your side), we should take a look into your configuration...
So provide your whole (unmodified) configuration dump:
fail2ban-client -d
or at least dump of all actions of all jails:
fail2ban-client -d | grep 'action'
now using firewalld
I see pretty sure iptables in your config and the log excerpt (error message).
So which banning action do you use really?
something is flushing iptables because of "after effects" like ...
No, the after effect is there because something is flushing rules, not vice versa.
This can be for example some "firewall" script which you apply to configure iptables (ports, default reject rules, etc), restart of some service due to dependency (restarting or reloading iptables), some script mistakenly implementing knocking, and many others.
I don't think it'd be simple to find it, if you don't have it under your control (for example can you exclude some service implemented by Contabo installed or integrated in your VPS, that doing that?).
UPDATE 1:
I don't see any error in your excerpt... to remove f2b-proftpd (or flush input chain) it should contain:
either <iptables> -D INPUT ... -j f2b-proftpd
or even <iptables> -F INPUT somewhere in actions parameters, excepting actionstop (which is intended stop by shutdown or restart case only).
But it is only available in actionstop as expected:
all entries containing removal from INPUT chain (actionstop only):
<iptables> -D INPUT -p tcp -m multiport --dports ssh -j f2b-sshd
<iptables> -D INPUT -p tcp -m multiport --dports 10000 -j f2b-webmin-auth
<iptables> -D INPUT -p tcp -m multiport --dports ftp,ftp-data,ftps,ftps-data -j f2b-proftpd
<iptables> -D INPUT -p tcp -m multiport --dports smtp,465,submission -j f2b-postfix
<iptables> -D INPUT -p tcp -m multiport --dports pop3,pop3s,imap,imaps,submission,465,sieve -j f2b-dovecot
<iptables> -D INPUT -p tcp -m multiport --dports smtp,465,submission,imap,imaps,pop3,pop3s -j f2b-postfix-sasl
<iptables> -D INPUT -p tcp -m multiport --dports ssh,sftp -j f2b-ssh-ddos
Thus we could exclude the blame of fail2ban.
Just you said "now using firewalld" - where? In your config dump I see iptables-multiport only:
['set', 'sshd', 'addaction', 'iptables-multiport']
['set', 'webmin-auth', 'addaction', 'iptables-multiport']
['set', 'proftpd', 'addaction', 'iptables-multiport']
['set', 'postfix', 'addaction', 'iptables-multiport']
['set', 'dovecot', 'addaction', 'iptables-multiport']
['set', 'postfix-sasl', 'addaction', 'iptables-multiport']
['set', 'ssh-ddos', 'addaction', 'iptables-multiport'],
So for example if firewald would rewrite INPUT chain of iptables (removes fail2ban entries), you would catch exactly this issue.
Thus better find out which major netfilter your system use and use it as banaction (or avoid rewriting of fail2ban rules e. g. by adding it to fail2ban dependencies, etc).

tshark - filtering by webservice's name

I'm trying to use tshark to record each request sent to a WebService called myservice.
When I use below command, I can see in output file every request sent on port 8280 (Tomcat) :
tshark -n -i eth1 port 8280 -w output.pcap
Considering I have a lot of WebServices in that Tomcat instance, I would like to filter by service name, something like that :
tshark -n -i eth1 port 8280 -w output.pcap -R 'http.request.uri contains "myservice"'
According to man, it looks like I should rather use -f (capture filter) than -R (display filter) since you can't use -R with -w :
tshark: Read filters aren't supported when capturing and saving the captured packets.
I took a look at documentation about capture filters but I can't see a way to do that. I also tried with tcpdump without success.
You can use ngrep for this
sudo ngrep -O output.pcap -i -d eth0 'myservice'
Although you might get some false positive depending on whether or not 'myservice' is found on an irrelevant packet that was not intended for your application. To avoid this, you might want to apply a bpf filter to grep only traffic that was directed to your service/app
sudo ngrep -O output.pcap -i -d eth0 'myservice' 'tcp dst port 8280'

Jetty JMX open port 1099

I use CentOS 6.5 and Jetty 9.1.0.v20131115. I use Jetty's JMX capabilities.
I want to have JMX accessible only from within the running computer (localhost, or 127.0.0.0/8), but not from outside (e.g. JMX shall not be accessible from public.example.com).
Therefore, I configured Jetty's JMX RMI host to use jetty.jmxrmihost=localhost instead of a wildcard jetty.jmxrmihost=0.0.0.0.
Yet still, my Jetty server instance is accessible from "outside", allowing anyone to connect to my Jetty server via JMX.
What do I have to configure to make Jetty listen to only those JMX connections which originate from localhost?
Here are my Jetty configuration files that are relevant to this topic:
file ${jetty.base}/start.d/jmx.ini:
--module=jmx
#jetty.jmxrmihost=localhost # I tried this one, but it didn't work either
jetty.jmxrmihost=127.0.0.1
jetty.jmxrmiport=1099
file ${jetty.base}/start.d/jmx-remote.ini:
--module=jmx-remote
Just from the way the question is asked, it seems like it is less of a Jetty/JMX issue and more of a firewall issue - what you want is to block unwanted outside traffic to the JMX port on this server.
If you have permissions and are willing to do so, you will want to remove any rule from /etc/sysconfig/iptables that is opening the JMX port (in this example, 1099). Such a rule will look like the following:
[0:0] -A INPUT -s SOME_IP_SUBNET -p tcp -m tcp --dport 1099 -j ACCEPT
Or, on the flip side, you may want to enable JMX monitoring only for a specific subnet (such as for a company's subnet), at which point, you'd want to add the following:
[0:0] -A INPUT -s MY_IP_SUBNET_HERE -p tcp -m tcp --dport JMX_PORT -j ACCEPT
, replacing MY_IP_SUBNET_HERE and JMX_PORT with your own IP subnet and JMX port, respectively.
I haven't written a lot of rules for iptables myself, so please consider the above as an example and not necessarily the exact syntax you need. *nixCraft provides a basic guide to handling iptables/sysctl, which also covers how to modify rules without editing the file (I usually just modify the file).
Two notes, if you go the route of modifying the iptables file:
Be sure to restart iptables (/etc/init.d/iptables restart or service iptables restart)
Call /sbin/sysctl -p after restarting iptables. Restarting iptables wipes out any custom rules from sysctl.conf, calling sysctl -p will restore those rules.

I only can connect to my nginx server from local computer

First of all, please excuse my low-level English.
I'm not native English speaker..
but I'll try to explain well as far as possible.
I really have no idea about this situation.
I thought that it's iptables problem.. but it seems not.
I'm getting a server hosting(CentOS).
I installed Nginx + Django and nginx uses 8080 port.
A domain is connected to the server.
When I executed "wget [domain]:8080/[app name]/" in the server,
it works.
Of course, "wget 127.0.0.1:8080/[app name]/" has no problem.
(wget [server ip]:8080/[app name]/, either)
However, from other computers, connecting failed.
I checked my firewall setting.
I executed these commands.
iptables -I INPUT -p tcp --dport 8080 -j ACCEPT
iptables -I OUTPUT -p tcp --sport 8080 -j ACCEPT
iptables -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8080 -j ACCEPT
/etc/init.d/iptables restart
I don't really understand all options of commands and I think there were useless commands, but I just tried all googled iptables settings.
But still I cannot connect to my server.
What should I check, first?
I don't know if this is important, but I am adding to this post.
On 80 port, an apache server is running.
It works fine, I can connect to apache from other computers.
There is DB connecting issue, (PHP to MySQL) but I think that it is just PHP coding bug.
Thank you for reading this question.
I tried to stop my firewall, and It worked.
So the problem is on my iptables setting.
I had allowed 8080 port, but I think there was a mistake on the settings. I regret that I didn't read and study settings carefully.
I flushed all setting, and restart server. All looks fine.

C++ weird RAW sockets and iptables issue

With reference to C++ iptables redirection forming separate packets, I am facing an extremely peculiar problem now. I am trying to redirect all incoming traffic on UDP port 5060 to port 56790, and all outgoing traffic from 5060 to the port 56789. I used these iptables rules:
iptables -t nat -I PREROUTING -p udp ! -s localhost --dport 5060 -j REDIRECT --to-port 56790
iptables -t nat -I OUTPUT -p udp ! -s localhost --sport 5060 -j REDIRECT --to-port 56789
I listen on both ports using RAW SOCKETS after setting the interface to PROMISCUOUS mode using ioctl.
I see packets ONLY on 56789 i.e.SENDING side, and I do not see any packets on 56790, while wireshark shows that many packets are delivered to port 5060.
Why would this happen? Any ideas? Do you think it's a problem with iptables rules or something to do with raw sockets?
raw sockets get a copy of the original packet before modification (incoming). On outgoing it's reversed.