Logstash grok filter apache pattern - regex

Question

This is a sample Apache Tomcat log:
portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2
I would like to capture the following variables
09/Sep/2019:00:08:02 +0200
/en/search-results?p_p_id=portal2....
35S64857F6860FDFC0F60B5B47A97E18
Can you help me with that? I want to index only those and drop the others, is it possible? Thank you

Answer 1

Use this grok pattern:
%{GREEDYDATA:field1} %{IP:ip1} - - \[%{GREEDYDATA:date}] \"%{WORD:method} %{GREEDYDATA:request}" %{WORD:numbers} %{WORD:numbers} %{WORD:numbers} - - %{WORD:last_parameter}
input:
portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2
output:
{
"field1": [
[
"portal.portal.some.thing.int:8443"
]
],
"ip1": [
[
"13.233.220.113"
]
],
"IPV6": [
[
null
]
],
"IPV4": [
[
"13.233.220.113"
]
],
"date": [
[
"09/Sep/2019:00:08:02 +0200"
]
],
"method": [
[
"GET"
]
],
"request": [
[
"/en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1"
]
],
"numbers": [
[
"230",
"334734",
"6261"
]
],
"last_parameter": [
[
"35S64857F6860FDFC0F60B5B47A97E18"
]
]
}
fields you want are:
date
request
last_parameter
You can remove other fields using remove field in mutate filter.