Suggetions on Opencart robots.txt - opencart

I'm a bit confused about how my robots.txt file should be. My website is www.partspatibd.com
Can anyone check my current file? See if it's OK or not...
User-agent: *
Disallow: /*&filter
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=account/
Disallow: /*?route=affiliate/
Disallow: /*?route=checkout/
Allow: /
sitemap: https://partspatibd.com/sitemap.xml

According to my experience in OpenCart 2 and 3 I suggest you to use following configuration for robots.txt
User-agent: *
Disallow: /*route=account/
Disallow: /*route=affiliate/
Disallow: /*route=checkout/
Disallow: /*route=product/search
Disallow: /index.php?route=product/product*&manufacturer_id=
Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /system
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter=
Disallow: /*&filter=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=
Disallow: /*?tracking=
Disallow: /*&tracking=
Sitemap: https://partspatibd.com/sitemap.xml
Host: https://partspatibd.com/
This will protect system directories and queries from unnecessary indexation. Your Sitemap file is approachable and totally fine. Host statement is recommended.

Related

Facebook Graph API: How To Publish Text To Page, Including Image Attachment?

The documentation gives an example of how to upload an image, not via url, but as an attachment:
https://developers.facebook.com/docs/messenger-platform/reference/attachment-upload-api
curl -F "message={'attachment':{'type':'image', 'payload':{'is_reusable':true}}}" -F "filedata=#square.png;type=image/png" "https://graph.facebook.com/v2.12/1843037429354999/photos?access_token=<ACCESS_TOKEN>"
This leads to the following request:
POST https://graph.facebook.com/v2.12/<MY_PAGE_ID>/photos?access_token=<MY_ACCESS_TOKEN> HTTP/1.1
Host: graph.facebook.com
User-Agent: curl/7.52.1
Accept: */*
Content-Length: 2833
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------c71046aadf1a4e33
--------------------------c71046aadf1a4e33
Content-Disposition: form-data; name="message"
{'attachment':{'type':'image', 'payload':{'is_reusable':true}}}
--------------------------c71046aadf1a4e33
Content-Disposition: form-data; name="filedata"; filename="square.png"
Content-Type: image/png
<IMAGE_DATA>
--------------------------c71046aadf1a4e33--
However, this example does not work as advertised. The json object in the message parameter is literally posted as text in the post.
I'd like to be able to send along json post data that includes a 'message' and a 'access_token', so that I don't have to include the access token in the url.
How can I ensure that the type/image/payload json object is applied as a configuration, rather than used as the literal post message?
And how can I send along 'message' and 'access_token' in the postdata?
Found the solution.
This command...
curl -F "access_token=<MY_ACCESS_TOKEN>" -F "message=This is the message I wanna post." -F "filedata=#square.png;type=image/png" "https://graph.facebook.com/v2.12/<MY_PAGE_ID>/photos"
Gives this working request:
POST https://graph.facebook.com/v2.12/<MY_PAGE_ID>/photos HTTP/1.1
Host: graph.facebook.com
User-Agent: curl/7.52.1
Accept: */*
Content-Length: 3077
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------e94f377a15b0500f
--------------------------e94f377a15b0500f
Content-Disposition: form-data; name="access_token"
<MY_ACCESS_TOKEN>
--------------------------e94f377a15b0500f
Content-Disposition: form-data; name="message"
This is the message I wanna post.
--------------------------e94f377a15b0500f
Content-Disposition: form-data; name="filedata"; filename="square.png"
Content-Type: image/png
<IMAGE_DATA>
--------------------------e94f377a15b0500f--
All of a sudden it dawned on me. If the 'message' parameter posts a message, then why not simply put the message I want to post in there. Might as well include 'access_token' then, too. And it worked.
Instead of the deprecated 'message', you can also use 'caption' and it will also work.
Anyway, I can now attempt to reproduce this http request from C#. Glad I found this, because it saves me the effort from having to install the Facebook PHP SDK just to see how such a request should be formed.
(Why is there so little Facebook Graph API C# support, anyway?)
I still have no idea how to properly send along the attachment/type/image/payload json object from Facebook's documentation, though.
Oh, well.

Regex for creating a new line

i would like your help to help me create a regex that will replace each "\r\n" syntax to a new line, like this example:
POST / HTTP/1.0\r\nHost: mywebsite.net\r\nConnection: close\r\nContent-Length:
400\r\nContent-Type: text/xml; charset=utf-8\r\nUser-Agent: Apache-
HttpClient/4.5.1 (Java/1.8.0_74)\r\nAccept-Encoding: gzip,def1ate
To this:
POST / HTTP/1.0
Host: mywebsite.net
Connection: close
Content-Length: 400
Content-Type: text/xml; charset=utf-8
User-Agent: Apache-HttpClient/4.5.1 (Java/1.8.0_74)
Accept-Encoding: gzip,deflate
many thanks!
Sorry for that.
I am using java pattern class to evaluate regular expressions.
It is on a graylog system that collect the access log (request and response) from the server.
extractor configuration
I was trying to make \\r\\n and replace with \r\n but it didnt help.

Read Request Payload in ColdFusion

Trying out Content Security Policy and using the Report Only method and the data is sending a POST (See edits, supposed to be POST, but Request Headers say GET) to my page which I then use to email myself. In the browser developer tools I can see the data in the Request Payload, but I don't know how to access this scope. The FORM scope is empty and the GetHttpRequestData() doesn't show a Payload key.
How do I access the data in the Request Payload in ColdFusion?
EDIT: In the dev tools I can see in the network tab a POST to the page with 4 sections: General, Response Headers, Request Headers and Request Payload. I need to access the Request Payload data.
Request Headers:
GET /campuses/content-report/ HTTP/1.1
Host: domain.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36
Accept: */*
Referer: http://domain.com/page/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: **[Removed]**
Request Payload:
{
"csp-report":{
"document-uri":"http://domain.com/page/",
"referrer":"http://domain.com/",
"violated-directive":"default-src 'self' 'sha256-L2Tc50iUaBz2udc-dnkwO-FKzrsl5cLnzkFu5LgX5ao=' https://stats.g.doubleclick.net http://netdna.bootstrapcdn.com http://www.google-analytics.com https://www.google.com https://fonts.gstatic.com https://ajax.googleapis.com https://cdn.jsdelivr.net https://maxcdn.bootstrapcdn.com http://www.etutoring.org",
"effective-directive":"img-src",
"original-policy":"default-src 'self' 'sha256-L2Tc50iUaBz2udc-dnkwO-FKzrsl5cLnzkFu5LgX5ao=' https://stats.g.doubleclick.net http://netdna.bootstrapcdn.com http://www.google-analytics.com https://www.google.com https://fonts.gstatic.com https://ajax.googleapis.com https://cdn.jsdelivr.net https://maxcdn.bootstrapcdn.com;style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://maxcdn.bootstrapcdn.com;script-src 'self' https://www.google-analytics.com https://ajax.googleapis.com https://cdn.jsdelivr.net 'nonce-3E89EBDB49F712C7D90D1B39E348BBBF';report-uri /campuses/content-report;",
"blocked-uri":"https://www.google-analytics.com",
"status-code":200
}
}
GetHttpRequestData contains Request Headers but not Request Payload. I did notice that the website I read said the Security policy would send the data via POST but the Headers say GET. Would that cause the issue? Should they be in the URL scope? Not at my machine at the moment.
So I was looking in the right spot, GetHttpRequestData but what I didn't realize was I setup the url to not have a trailing slash and my IIS Rewrite rules was redirecting to the / version of the page and the header data was being lost in the process.

Analyzing HTTP headers in Firebug

I was just trying to analyse all the HTTP header fields in Firefox plugin - Firebug. First I logged out from Stack Overflow and then cleared all the cookies from my browser.
Then I went to the Stack Overflow's home page. I mean while saw the HTTP request and response header fields. This is what I saw:
Response Headers
Via 1.0 proxy_server
Content-Length 135
Date Mon, 05 Mar 2012 06:01:33 GMT
Content-Type application/json
Cache-Control private
X-Cache MISS from sampark.ncb.ernet.in
Request Headers
Host stackoverflow.com
User-Agent Mozilla/5.0 (Windows NT 6.0; rv:12.0a2) Gecko/20120303 Firefox/12.0a2
Accept application/json, text/javascript, */*; q=0.01
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip, deflate
Proxy-Connection keep-alive
X-Requested-With XMLHttpRequest
Referer http://stackoverflow.com/
Cookie __qca=P0-383120279-1330927291125; __utma=140029553.974890682.1330927291.1330927291.1330927291.1; __utmb=140029553.1.10.1330927291; __utmc=140029553; __utmz=140029553.1330927291.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); gauthed=1
There is a cookie included in the request header. But as I said I have removed all the cookies from my browser. How is the cookie included in the request? What is actually happening here?
I did as Andy Davies told. I first cleared all the cookies, restarted Firefox and then went to www.stackoverflow.com. Firebug shows this:
GET http://stackoverflow.com/tags/ios/subscriber-info?_=1331946084371
The headers for the above request contained:
Cache-Control private
Content-Encoding gzip
Content-Length 390
Content-Type text/html; charset=utf-8
Date Sat, 17 Mar 2012 01:01:19 GMT
Vary Accept-Encoding<Br>
Request Headers
Accept text/html, */*; q=0.01
Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection keep-alive
Cookie __utma=140029553.1336172974.1331946082.1331946082.1331946082.1; __utmb=140029553.1.10.1331946082; __utmc=140029553; __utmz=140029553.1331946082.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __qca=P0-115511794-1331946081644; gauthed=1
Host stackoverflow.com
Referer http://stackoverflow.com/
User-Agent Mozilla/5.0 (Windows NT 6.0; rv:12.0a2) Gecko/20120303 Firefox/12.0a2
X-Requested-With XMLHttpRequest
If this is not the first request, then why is it not showing the first request?
Did you restart the browser after clearing the cookies as, from my memory, some browsers don't clear the cookies for any sites currently open?
The snippet you've posted looks like a JSON response part way through loading the page, so it is not the initial request for the HTML page.
When the HTML page would have been requested again, the Google Analytics cookie (which is what you've got above) would have been resent, so any subsequent components on the page will also get the cookie too.

Django CSRF failure on ajax post requests on Opera only

Ok so AJAX POST requests work fine in Mozilla and Chromium, but fail in Opera. I get the standard CSRF error (403). I tried different versions of Opera and they failed in every one I tried. Btw, I'm using the jquery/django snippet that sets X-CSRFToken in the header, as verified in the "bad Opera request" below.
I made a view in a different project that was very simple and ajax post requests worked fine in Opera. I looked at the request details and see differences. The good request doesn't set any weird X-Opera-Info and other opera params even thought I'm using the same browser. If this is the issue, is there a way to remove those extra params? Or does anyone have any other advice or ideas on what the issue may be? I know it's not my view function because I tried just returning an HttpResponse immediately and even that gets 403'd. Thanks a million guys.
####################
OPERA GOOD REQUEST
##############
Request details
POST /test HTTP/1.1
User-Agent: Opera/9.80 (X11; Linux i686; U; en) Presto/2.7.62 Version/11.00
Host: 127.0.0.1:8000
Accept-Language: en-US,en;q=0.9
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Referer: http://127.0.0.1:8000/test
Cookie: csrftoken=1c6441404c991f7ae3b6d7d49f91f280
Cookie2: $Version=1
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
Content-Length: 6
Content-Type: application/x-www-form-urlencoded
Accept: */*
X-CSRFToken: 1c6441404c991f7ae3b6d7d49f91f280
X-Requested-With: XMLHttpRequest
Content-Transfer-Encoding: binary
###################
OPERA BAD REQUEST
####################
Request details
POST http://facebook.example.com/remove-person HTTP/1.1
User-Agent: Opera/9.80 (X11; Linux i686; U; en) Presto/2.7.62 Version/11.00
Host: facebook.example.com
Accept-Language: en-US,en;q=0.9
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Referer: http://facebook.example.com/
Cookie: signed_request=5-f0_7pZLILrp6MLocsdMoNYAaZr-wCnU2cPbLC1bZg.eyJhbGdvcml0aG0iOiJITUFDLVNIQTI1NiIsImV4cGlyZXMiOjEzMDg4MTYwMDAsImlzc3VlZF9hdCI6MTMwODgxMTQyNywib2F1dGhfdG9rZW4iOiIyMjMyNDY5NDEwMjc3MTR8Mi5BUURVRGM2ZFFLSElnN1h3LjM2MDAuMTMwODgxNjAwMC4xLTU0MDIwMjZ8b2QtX1diNTh3aG1wTnNHYUh4cTNtOVBpWkswIiwidXNlciI6eyJjb3VudHJ5IjoidXMiLCJsb2NhbGUiOiJlbl9VUyIsImFnZSI6eyJtaW4iOjIxfX0sInVzZXJfaWQiOiI1NDAyMDI2In0; csrftoken=d4cdc6a75ed264d295a410dd98982c42; fbs_223246941027714="access_token=223246941027714%7C2.AQBlhzavZjzd8c7J.3600.1308819600.1-5402026%7CdsD6VESpGJb3m0EdD1mhFZtDI24&base_domain=example.com&expires=1308819600&secret=QaTNS988wl0FU6A0LG9qDQ__&session_key=2.AQBlhzavZjzd8c7J.3600.1308819600.1-5402026&sig=61e7e13091501f35793d3cda8c20835b&uid=5402026"
Cookie2: $Version=1
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
Content-Length: 14
X-Opera-Info: ID=448, p=4, f=15, sw=1440, sh=900
X-Opera-ID: e79c37b56a58510d26b56882453bddb6d2c2dae858129139113f6346ea23ca6b
X-Opera-Host: r18-02:12420
X-OA: 1322 b5834cb13259fbd50b87b576b5e8b9a8bcc1384478c2ea79cc65614dc1b67c27
X-OB: evenes
Content-Type: application/x-www-form-urlencoded
Accept: */*
X-CSRFToken: d4cdc6a75ed264d295a410dd98982c42
X-Requested-With: XMLHttpRequest
Content-Transfer-Encoding: binary
I recently ran into a similar problem. I was trying to do, for the first time, a post from AJAX on a view where Django wasn't sending a CSRF cookie. For reasons I can't explain, this was working on all browsers I tried except Opera.
This scenario is described in the Django docs, and they suggest using the ensure_csrf_cookie decorator.
Another thing you can do if it is too burdensome to wrap all the potential views with that decorator is to add something like this to your base template:
<script>
var csrf_token = "{{ csrf_token }}";
</script>
And then whenever you do an AJAX post, always include the key pair csrfmiddlewaretoken: csrf_token with your POST data.
Once I did the above, my posts with Opera started working.
Does it work if you disable Opera Turbo? These extra headers seem to be Turbo-related and perhaps this makes some kind of difference.
X-CSRFToken: {{ csrf_token }}}
value should be send from client in AJAX request HTTP headers to be recognized by Django