I tried out the basic example for HTTP streaming of HTTP-Kit (http://www.http-kit.org/server.html#async), but if I curl to the endpoint I get all the messages at once, after the connection is closed (instead of getting them every 200ms, as it works in the browser). What's going on?
I also tried it like this:
(schedule-task (* id 200) ;; send a message every 200ms
(send! channel
{:status 200
:headers {"Content-Type" "text/event-stream"
"Connection" "keep-alive"}
:body (str "message from server #" id)}
false)) ; false => don't close after send
The behaviour is the same.
Seems to be an issue with curl and Streaming http. According to this post, the order of options is important
Also, try using the -N, --no-buffer option
Related
I'm using https://curl.haxx.se/libcurl/c/http2-download.html to send mulitple http2 requests to a demo http server. This server is based on spring webflux. To verify if libcurl can send http2 requests concurrently, the server will delay 10 seconds before return response. In this way, I hope to observe that the server will receive multiple http2 requests at almost the same time over the same connection, after 10 seconds, the client will receive responses.
However,I noticed that the server received the requests sequentially. It seems that the client doesn't send the next request before geting the response of previous request.
Here is the log of server, the requests arrived every 10 seconds.
2021-05-07 17:14:57.514 INFO 31352 --- [ctor-http-nio-2] i.g.h.mongo.controller.PostController : Call get 609343a24b79c21c4431a2b1
2021-05-07 17:15:07.532 INFO 31352 --- [ctor-http-nio-2] i.g.h.mongo.controller.PostController : Call get 609343a24b79c21c4431a2b1
2021-05-07 17:15:17.541 INFO 31352 --- [ctor-http-nio-2] i.g.h.mongo.controller.PostController : Call get 609343a24b79c21c4431a2b1
Any guys can help figure out my mistakes? Thank you
For me,
curl -v --http2 --parallel --config urls.txt
did exactly what you need, where urls.txt was like
url = "localhost:8080/health"
url = "localhost:8080/health"
the result was that at first, curl sent first request via HTTP/1.1, received 101 upgrade to http/2, immediately sent the second request without waiting for response, and then received two times 200 response in succession.
Note: -v is added for verbosity to validate it works as expected. You don't need it other than for printing the underlying protocol conversation.
I am using clj-slack start method for getting the web-socket url and then passing it to the connect method of gniazdo for listening to the web-socket.
(require '[clj-slack.rtm :as slack])
(require '[gniazdo.core :as ws])
(def connection {:api-url "https://slack.com/api" :token "your token"})
(defn reset-conn [options]
(def socket
(ws/connect
(:url (slack/start options))
:on-receive #(prn 'received %))))
(reset-conn connection)
This code is working fine but it only provides information about who is active or away for slack:
received"{\"type\":\"presence_change\",\"presence\":\"active\",\"user\":\"U7BCQHWHY\"}"
But I also want to listen to the messages.
I don't know why messages are not printed though it was stated in slack-api documentation that rtm(real time messaging) is for messaging session.
While running my tests with lein doo phantom, I receive a -1 status response and an empty string as the body. However, when I run the test in the repl, I am able to retrieve the request data with a 200 status response and the appropriate data in the body. Is this because a manytomany channel is being returned first as mentioned below, thus giving me the inappropriate response? If so, how could I account for this?
https://github.com/r0man/cljs-http#async-response-handling
I also thought maybe I need to use a timeout to wait for the request to complete. If so, how would I apply that appropriately with my existing code? It looks like cljs-http has :timeout as a parameter but I haven't been ably to get it to work appropriately (assuming this is the cause of the issue).
(deftest test-async
(async done
(go (let [response (<! (http/get "http://localhost:3000/api/user/1"
{:with-credentials? false
:query-params {"id" 1}}))]
(is (= {:status 200}
(select-keys response [:status]))))
(done))))
Since you are running your test under phantomjs. Phantomjs default disable cross domain XHR access and your tests js are running on localhost,all external ajax calls are denied.
you can set the --web-security=false to allow your test to do cross domain ajax.
In your project.clj add this
:doo {:paths {:phantom "phantomjs --web-security=false"}}
more info about phantomjs
http://phantomjs.org/api/command-line.html
I am stuck with creating downchannel stream which is designed to remain open in a half-closed state from the client and open from AVS for prolonged periods of time.
It receives JSON response from ALEXA API without making a request since it is open all the time in a half-closed state.
Currently I am making a GET request using libcurl that shuts down as a 1 way process
Sample Request
:method = GET
:scheme = https
:path = /{{API version}}/directives
authorization = Bearer {{YOUR_ACCESS_TOKEN}}
I want to know how to keep connection on in a half-closed state using libcurl so that it gives response without making requests.
I am not expecting any code just a way to do so will do.
I`m getting lots of false positives [??]after just setting up mod_security. I'm running it in detection only so no issues yet but these filters will start blocking requests once I need it to go live.
Afraid I don't 100% understand what the significance of these filters are, I get 100s of them on nearly every domain & all the requests look legitimate.
Request Missing a User Agent Header
Request Missing an Accept Header
What is the best thing to do here? Should I disable these filters? Can I set the severity lower so that requests won't be blocked?
Here is a complete entry
[22/Nov/2011:21:32:37 --0500] u6t6IX8AAAEAAHSiwYMAAAAG 72.47.232.216 38543 72.47.232.216 80
--5fcb9215-B--
GET /Assets/XHTML/mainMenu.html HTTP/1.0
Host: www.domain.com
Content-type: text/html
Cookie: pdgcomm-babble=413300:451807c5d49b8f61024afdd94e57bdc3; __utma=100306584.1343043347.1321115981.1321478968.1321851203.4; __utmz=100306584.1321115981.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=XXXXXXXX%20clip%20ons
--5fcb9215-F--
HTTP/1.1 200 OK
Last-Modified: Wed, 23 Nov 2011 02:01:02 GMT
ETag: "21e2a7a-816d"
Accept-Ranges: bytes
Content-Length: 33133
Vary: Accept-Encoding
Connection: close
Content-Type: text/html
--5fcb9215-H--
Message: Operator EQ matched 0 at REQUEST_HEADERS. [file "/etc/httpd/modsecurity_crs/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "47"] [id "960015"] [rev "2.2.1"] [msg "Request Missing an Accept Header"] [severity "CRITICAL"] [tag "PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"]
Message: Operator EQ matched 0 at REQUEST_HEADERS. [file "/etc/httpd/modsecurity_crs/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "66"] [id "960009"] [rev "2.2.1"] [msg "Request Missing a User Agent Header"] [severity "NOTICE"] [tag "PROTOCOL_VIOLATION/MISSING_HEADER_UA"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"]
Message: Warning. Operator LT matched 5 at TX:inbound_anomaly_score. [file "/etc/httpd/modsecurity_crs/base_rules/modsecurity_crs_60_correlation.conf"] [line "33"] [id "981203"] [msg "Inbound Anomaly Score (Total Inbound Score: 4, SQLi=5, XSS=): Request Missing a User Agent Header"]
Stopwatch: 1322015557122593 24656 (- - -)
Stopwatch2: 1322015557122593 24656; combined=23703, p1=214, p2=23251, p3=2, p4=67, p5=168, sr=88, sw=1, l=0, gc=0
Producer: ModSecurity for Apache/2.6.1 (http://www.modsecurity.org/); core ruleset/2.2.1.
Server: Apache/2.2.3 (CentOS)
If you look under Section H of the audit log entry you showed at the Producer line, you will see that you are using the OWASP ModSecurity Core Rule Set (CRS) v2.2.1. In this case, I suggest you review the documentation information on the project page -
https://www.owasp.org/index.php/Category:OWASP_ModSecurity_Core_Rule_Set_Project#tab=Documentation
Specifically, you should review these two blog posts that I did -
http://blog.spiderlabs.com/2010/11/advanced-topic-of-the-week-traditional-vs-anomaly-scoring-detection-modes.html
http://blog.spiderlabs.com/2011/08/modsecurity-advanced-topic-of-the-week-exception-handling.html
Blog post #1 is useful so that you understand which "mode of operation" you are using for the CRS. By looking at your audit log, it appears you are running in anomaly scoring mode. This is where the rules are doing detection but the blocking decision is being done separately by inspecting the overall anomaly score in the modsecurity_crs_49_inbound_blocking.conf file.
Blog post #2 is useful so that you can decided exactly how you want to handle these two rules. If you feel that these are not important to you - then I would suggest that you use the SecRuleRemoveById directive to disable these rules from your own modsecurity_crs_60_exceptions.conf file. The way that it stands now, these two alert are only generating an inbound anomaly score of 4 - which is below the default threshold of 5 set in the modsecurity_crs_10_config.conf file so it is not blocked.
Looking at your audit log example, while this request did generate alerts, the transaction was not blocked. If it was, the message data under Section H would have stated "Access denied...".
As for the purposed of these rules - they are meant to flag requests that are not generated from standard web browsers (IE, Chrome, Firefox, etc...) as all of these browsers will send both User-Agent and Accept requests headers per the HTTP RFC spec.
One last comment - I would suggest that you use the official OWASP ModSecurity CRS mail-list for these types of questions -
https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
You can also search the archives that for answers.
Cheers,
Ryan Barnett
ModSecurity Project Lead
OWASP ModSecurity CRS Project Lead
This aren't false positives. Your request headers lack User-Agent and Accept headers. Usually these are sent from scanner- or hack-tools.