I'm new to zabbix. I need to monitor the log file and create a warning if its content doesn't meet the following:
{ "10.0.0.1": "0" }
I tried to use vfs.file.regmatch for this and with following regexes:
vfs.file.regmatch[/tmp/test72.log,'\{ \"10\.0\.0\.1\"\: \"0\" \}']
vfs.file.regmatch[/tmp/test72.log,‘{ "10.0.0.1": "0"}’]
But all of them returns 0 (False), when I check in zabbix web interface, but trying to run in shell I get the following (expected result):
zabbix_agentd -t vfs.file.regmatch[/tmp/test72.log,'{ "10.0.0.1": "0" }']
vfs.file.regmatch[/tmp/test72.log,{ "10.0.0.1": "0" }] [u|1]
Have I missed some escape characters?
ALso, my Zabbix server and agents versions are 3.4.15.
Thanks in advance!
Related
I am seeking to extract timestamps and ip addresses out of log entries containing a varying amount of information. The basic structure of a log entry is:
<timestamp>, <token_1>, <token_2>, ... ,<token_n>, <ip_address> <token_n+2>, <token_n+3>, ... ,<token_n+m>,-
The number of tokens n between the timestamp and ip address varies considerably.
I have been studying regular expressions and am able to grep timestamps as follows:
grep -o "[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}T[0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\}"
And ip addresses:
grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
But I have not been able to grep both patterns out of log entries which contain both. Every log entry contains a timestamp, but not every entry contains an ip address.
Input:
2021-04-02T09:06:44.248878+00:00,Creation Time,EVT,WinEVTX,[4624 / 0x1210] Source Name: Microsoft-Windows-Security-Auditing Message string: An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tREDACTED$\n\tAccount Domain:\t\tREDACTED\n\tLogon ID:\t\tREDACTED\n\nLogon Type:\t\t\t10\n\nNew Logon:\n\tSecurity ID:\t\tREDACTED\n\tAccount Name:\t\tREDACTED\n\tAccount Domain:\t\tREDACTED\n\tLogon ID:\t\REDACTED\n\tLogon GUID:\t\tREDACTED\n\nProcess Information:\n\tProcess ID:\t\tREDACTED\n\tProcess Name:\t\tC:\Windows\System32\winlogon.exe\n\nNetwork Information:\n\tWorkstation:\tREDACTED\n\tSource Network Address:\t255.255.255.255\n\tSource Port:\t\t0\n\nDetailed Authentication Information:\n\tLogon Process:\t\tUser32 \n\tAuthentication Package:\tNegotiate\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\t-\n\tKey Length:\t\t0\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested. Strings: ['S-1-5-18' 'DEVICE_NAME$' 'NETWORK' 'REDACTED' 'REDACTED' 'USERNAME' 'WORKSTATION' 'REDACTED' '10' 'User32 ' 'Negotiate' 'REDACTED' '{REDACTED}' '-' '-' '0' 'REDACTED' 'C:\\Windows\\System32\\winlogon.exe' '255.255.255.255' '0' '%%1833'] Computer Name: REDACTED Record Number: 1068355 Event Level: 0,winevtx,OS:REDACTED,-
Desired Output:
2021-04-02T09:06:44, 255.255.255.255
$ sed -En 's/.*([0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}).*[^0-9]([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1, \2/p' file
2021-04-02T09:06:44, 255.255.255.255
Your regexps can be reduced by removing some of the explicit repetition though:
$ sed -En 's/.*([0-9]{4}(-[0-9]{2}){2}T([0-9]{2}:){2}[0-9]{2}).*[^0-9](([0-9]{1,3}\.){3}[0-9]{1,3}).*/\1, \4/p' file
2021-04-02T09:06:44, 255.255.255.255
It could be simpler still if all of the lines in your log file start with a timestamp:
$ sed -En 's/([^,.]+).*[^0-9](([0-9]{1,3}\.){3}[0-9]{1,3}).*/\1, \2/p' file
2021-04-02T09:06:44, 255.255.255.255
If you are looking for lines that contain both patterns, it may be easiest to do it two separate searches.
If you're searching your log file for lines that contain both "dog" and "cat", it's usually easiest to do this:
grep dog filename.txt | grep cat
The grep dog will find all lines in the file that match "dog", and then the grep cat will search all those lines for "cat".
You seem not to know the meaning of the "-o" switch.
Regular "grep" (without "-o") means: give the entire line where the pattern can be found. Adding "-o" means: only show the pattern.
Combining two "grep" in a logical AND-clause can be done using a pipe "|", so you can do this:
grep <pattern1> <filename> | grep <pattern2>
I have a complex problem. Below is the ndxconfig.ini file I want to Edit
# /etc/ndxconfig.ini will override this file
# if APP_ID is added in service propery, service discovery will be using marathon;
# HOST/PORT specified will override values retrieved from marathon
[MARATHON]
HOSTS = {{ ','.join(groups['marathon'])}}
PORT = 8080
PROTOCOL = http
SECRET = SGpQIcjK2P7RYnrdimhhhGg7i8MdmUqwvA2JlzbyujFS4mR8M88svI7RfNWt5rnKy4WHnAihEZmmIUb940bnlYmnu47HdUHE
[MYSQL]
; APP_ID = /neon/infra/mysql
HOST = {{keepalived_mysql_virtual_ip}}
PORT = 3306
SECRET = tIUFN1rjjDBdEXUsOJjPEtdieg8KhwTzierD48JsgDeYc84DD6Uy5a6kzHfKolq1MNS1DKwlSqxENk33UulJd9DPHPzYCxFm
I want to change specifically marathon protocol conf from http to https. Not other's protocol conf. I have to match PROTOCOL = http 3 lines below the [MARATHON] line. I researched and couldn't find any solution. There's only 1 line below sed solutions.
One idea stuck mine was somehow specially grep [MARATHON] and 3 lines below and tail 1 line. I don't know.
How can fix this? Please Help.
Solution found here
sed '/\[MARATHON\]/{N;N;N;s/http/https/;}' <file>
If you have python available, you can use crudini:
crudini --set ndxconfig.ini MARATHON protocol https
This screams for ed, which treats the file as a whole, not processing it a line at a time like sed (And also doesn't depend on the availability of the non-posix -i extension to sed for in-place editing of files):
ed -s ndxconfig.ini <<'EOF'
/MARATHON/;/^$/ s/^PROTOCOL = http$/PROTOCOL = https/
w
EOF
This will replace the PROTOCOL line in the block that starts with a line matching MARATHON and ending with a blank line. The protocol entry can be the second, third, fourth, etc. line; doesn't matter. If the protocol is already https, it won't do anything (Except print a question mark)
We have recently setup AWS logs agent on one of our test servers. Our log files usually contain multi-line events. e.g one of our log event is:
[10-Jun-2016 07:30:16 UTC] SQS Post Response: Array
(
[Status] => 200
[ResponseBody] => <?xml version="1.0"?><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2009-02-01/"><SendMessageResult><MessageId>053c7sdf5-1e23-wa9d-99d8-2a0cf9eewe7a</MessageId><MD5OfMessageBody>8e542d2c2a1325a85eeb9sdfwersd58f</MD5OfMessageBody></SendMessageResult><ResponseMetadata><RequestId>4esdfr30-c39b-526b-bds2-14e4gju18af</RequestId></ResponseMetadata></SendMessageResponse>
)
The log agent reference documentation says to use 'multi_line_start_pattern' option for such logs. Our AWS Log agent config is as follows:
[httpd_info.log]
file = /var/log/httpd/info.log*
log_stream_name = info.log
initial_position = start_of_file
log_group_name = test.server.name
multi_line_start_pattern = '(\[)+\d{2}-[a-zA-Z]{3}+-\d{4}'
However, the logs agent reporting breaks on aforementioned and similar events. The way it is being reported to CloudWatch Logs is as follows:
Event 1:
[10-Jun-2016 11:21:26 UTC] SQS Post Response: Array
Event 2:
( [Status] => 200 [ResponseBody] => <?xml version="1.0"?><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2009-02-01/"><SendMessageResult><MessageId>053c7sdf5-1e23-wa9d-99d8-2a0cf9eewe7a</MessageId><MD5OfMessageBody>8e542d2c2a1325a85eeb9sdfwersd58f</MD5OfMessageBody></SendMessageResult><ResponseMetadata><RequestId>4esdfr30-c39b-526b-bds2-14e4gju18af</RequestId></ResponseMetadata></SendMessageResponse>
Event 3:
)
Despite of the fact that its only a single event. Any clue whats going on here?
I think all you need to add is the following to your awslogs.conf
datetime_format = %d-%b-%Y %H:%M:%S UTC
time_zone = UTC
multi_line_start_pattern = {datetime_format}
http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html
multi_line_start_pattern
Specifies the pattern for identifying the start of a log message. A log message is made of a line that matches the pattern and any following lines that don't match the pattern. The valid values are regular expression or {datetime_format}. When using {datetime_format}, the datetime_format option should be specified. The default value is ‘^[^\s]' so any line that begins with non-whitespace character closes the previous log message and starts a new log message.
If that datetime format didn't work, you would need to update your regex to actually match your specific datetime. I don't think the one you have listed above actually works for your given format.
You could try this for instance:
[\d{2}-[\w]{3}-\d{4}\s{1}\d{2}:\d{2}:\d{2}\s{1}\w+]
does match
[10-Jun-2016 11:21:26 UTC]
See here: http://www.regexpal.com/?fam=96811
Once completed, issue a restart of the service and check to see if its parsing correctly.
$ sudo service awslogs restart
I am actually using wamp 2.5 with PHP 5.5.12 and when I try to create a phar file it returns me the following message :
Uncaught exception 'UnexpectedValueException' with message 'creating archive "..." disabled by the php.ini setting phar.readonly'
even if I turn to off the phar.readonly option in php.ini.
So how can I enable the creation of phar files ?
I had this same problem and pieced together from info on this thread, here's what I did in over-simplified explanation:
in my PHP code that's generating this error, I added echo phpinfo(); (which displays a large table with all sort of PHP info) and in the first few rows verify the path of the php.ini file to make sure you're editing the correct php.ini.
locate on the phpinfo() table where it says phar.readonly and note that it is On.
open the php.ini file from step 1 and search for phar.readonly. Mine is on line 995 and reads ;phar.readonly = On
Change this line to phar.readonly = Off. Be sure that there is no semi-colon at the beginning of the line.
Restart your server
Confirm that you're phar project is now working as expected, and/or search on the phpinfo()table again to see that the phar.readonly setting has changed.
phar.readonly can only be disabled in php.ini due to security reasons.
If you want to check that it's is really not done using other method than php.ini then in terminal type this:-
$ php -r "ini_set('phar.readonly',0);print(ini_get('phar.readonly'));"
If it will give you 1 means phar.readonly is On.
More on phar.configuration
Need to disable in php.ini file
Type which php
Gives a different output depending on machine e.g.
/c/Apps/php/php-7.2.11/php
Then open the path given not the php file.
E.g. /c/Apps/php/php-7.2.11
Edit the php.ini file
could do
vi C:\Apps\php\php-7.2.11\php.ini
code C:\Apps\php\php-7.2.11\php.ini
[Phar]
; http://php.net/phar.readonly
phar.readonly = Off
; http://php.net/phar.require-hash
phar.require_hash = Off
Save
Using php-cli and a hashbang, we can set it on the fly without messing with the ini file.
testphar.php
#!/usr/bin/php -d phar.readonly=0
<?php
print(ini_get('phar.readonly')); // Must return 0
// make sure it doesn't exist
#unlink('brandnewphar.phar');
try {
$p = new Phar(dirname(__FILE__) . '/brandnewphar.phar', 0, 'brandnewphar.phar');
} catch (Exception $e) {
echo 'Could not create phar:', $e;
}
echo 'The new phar has ' . $p->count() . " entries\n";
$p->startBuffering();
$p['file.txt'] = 'hi';
$p['file2.txt'] = 'there';
$p['file2.txt']->compress(Phar::GZ);
$p['file3.txt'] = 'babyface';
$p['file3.txt']->setMetadata(42);
$p->setStub('<?php
function __autoload($class)
{
include "phar://myphar.phar/" . str_replace("_", "/", $class) . ".php";
}
Phar::mapPhar("myphar.phar");
include "phar://myphar.phar/startup.php";
__HALT_COMPILER();');
$p->stopBuffering();
// Test
$m = file_get_contents("phar://brandnewphar.phar/file2.txt");
$m = explode("\n",$m);
var_dump($m);
/* Output:
* there
**/
✓ Must be set executable:
chmod +x testphar.php
✓ Must be called like this:
./testphar.php
// OUTPUT there
⚠️ Must not be called like this:
php testphar.php
// Exception, phar is read only...
⚠️ Won't work called from a CGI web server
php -S localhost:8785 testphar.php
// Exception, phar is read only...
For anyone who has changed the php.ini file, but just doesn't see any changes. Try to use the CLI version of the file. For me, it was in /etc/php/7.4/cli/php.ini
Quick Solution!
Check:
cat /etc/php/7.4/apache2/php.ini | grep phar.readonly
Fix:
sed -i 's/;phar.readonly = On/;phar.readonly = Off/g' /etc/php/7.4/apache2/php.ini
Has anyone ran into this issue? When the script gets to the banner text the script just hangs.
I am using Net::Appliance::Session
Here is the error I get in debug. The rest of the script inserts code perfectly. I did test what I read about adding a # to the banner for each line. Same result.
banner login +
[ 4.092880] tr nope, doesn't (yet) match (?-xism:[\/a-zA-Z0-9._\[\]-]+ ?(?:\(config[^)]*\))? ?[#>] ?$)
[ 4.093124] du SEEN:
banner login +
[ 4.093304] tr nope, doesn't (yet) match (?-xism:[\/a-zA-Z0-9._\[\]-]+ ?(?:\(config[^)]*\))? ?[#>] ?$)
[ 4.305872] du SEEN:
Enter TEXT message. End with the character '+'
[ 4.306121] tr nope, doesn't (yet) match (?-xism:[\/a-zA-Z0-9._\[\]-]+ ?(?:\(config[^)]*\))? ?[#>] ?$)
We had an issue when accessing the device : 10.49.216.74
The reported error was : read timed-out at /usr/lib/perl5/site_perl/5.10.0/Net/CLI/Interact/Transport/Wrapper/Net_Telnet.pm line 35
Here is a snip of code.
my $session_obj = Net::Appliance::Session->new(
host => $ios_device_ip,
transport => 'Telnet',
personality => 'ios',
timeout => 60,
);
#interace
$session_obj->set_global_log_at('debug');
eval {
# try to login to the ios device, ignoring host check
$session_obj->connect(
username => $ios_username,
password => $ios_password,
#SHKC => 0
);
# get our running config
$version_info = $session_obj->begin_privileged;
$session_obj->cmd('conf t');
$session_obj->cmd('line con 0');
$session_obj->cmd('exec-character-bits 8');
$session_obj->cmd('international');
$session_obj->cmd('line vty 0 4');
$session_obj->cmd('exec-character-bits 8');
$session_obj->cmd('international');
$session_obj->cmd('line vty 5 15');
$session_obj->cmd('exec-character-bits 8');
$session_obj->cmd('international');
$session_obj->cmd('exit');
$session_obj->cmd('no banner login');
$session_obj->cmd('banner login +');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('* test *');
$session_obj->cmd('* *');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('+');
$session_obj->cmd('no banner MOTD');
$session_obj->cmd('banner motd +');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('* test *');
$session_obj->cmd('* *');
$session_obj->cmd('*************************************************************************');
$session_obj->cmd('+');
$session_obj->cmd('exit');
$session_obj->cmd('write memory');
$session_obj->end_privileged;
# close down our session
$session_obj->close;
};
If you look at the regexp that matches the prompt before sending a new command you'll see that it requires a specific string that closely matches user, privileged or config mode of a router.
When you send the banner login + command you get the Enter TEXT message. End with the character '+' followed by blank line from a router (instead of Router(config)# that your script expects. After a while it just times out since there is no match for the regexp.
The easiest solution is to try to send the whole banner in one command. Try concatenating your banner with a \r in one string and sending it as a one command that looks like (note the double quotes):
$session_obj->cmd("banner login + line1 \r line2 \r line3\r +");
Took way too long to figure this out... spaces are not your friend.
$session_obj->cmd("banner login + \rline1\rline2\rline3\r+");
Example with my orginal problem:
$session_obj->cmd('*************************************************************************\r* test *\r* *\r*************************************************************************');