I am building a app using Ocsigen, this app will not be connected to a database and the goal is to copy the content from the main website to here. (i am using curl to do Get requests)
So my problem here is, I am trying to do a "log in" with a user, and I want to build a service that saves to my code the username and the password that is introduced by the user. Then I could use the username and the password to do the GET request to the main website.
My problem it's only one, I have tried a lot of ways to build a service that saves this two strings but I can't figure out a way.
I did a lot of diferent tries and my last one I did something like the following:
let save_data =
Eliom_registration.Action.create
~meth:
(Eliom_service.Post
(Eliom_parameter.unit,
Eliom_parameter.string "username"))
~path:Eliom_service.No_path
(fun () username -> username)
Based on an example that Ocsigen has on their website but I doesn't work...
I created a form that is successfully executing the services I created, but the service is never doing what I need.
Sorry if I explained badly the situation... Is any idea how to solve this problem? I am going in the wrong way?
Thnx!
PS: I have also tried to edit the Os_services.connect_service form the Ocsigen-Start src folder, and I guess that I have to do a Post Service to this case, but again I failed to do what I want.
I think what you are looking for is Eliom_reference, that is if you wish to keep the information stored for later.
https://ocsigen.org/eliom/api/server/Eliom_reference
I'm a little bit rusty when it comes to Eliom but it would look like this:
let current_username : string option Eliom_reference.eref =
Eliom_reference.eref ~scope:Eliom_common.default_process_scope None
let save_data = Eliom_registration.Action.create
~meth:(Eliom_service.Post
(Eliom_parameter.unit,
Eliom_parameter.string "username"))
~path:(Eliom_service.No_path)
(fun () username -> Eliom_reference.set current_username (Some username))
Related
My django app uses update_or_create() to update a bunch of records. In some cases, updates are really few within a ton of records, and it would be nice to know what got updated within those records. Is it possible to know what got updated (i.e fields whose values got changed)? If not, does any one has ideas of workarounds to achieve that?
This will be invoked from the shell, so ideally it would be nice to be prompted for confirmation just before a value is being changed within update_or_create(), but if not that, knowing what got changed will also help.
Update (more context): Thought I'd give more context here. The data in this Django app gets updated through various means (through users coming on the web site, through the admin page, through scripts (run from the shell) that populate data from a csv etc.). The above question is important mostly for the shell scripts that update data from csvs, hence a solution at the database/trigger/signal level may not be helpful here (I guess).
This is what I ended up doing:
for row in reader:
school_obj0, created = Org.objects.get_or_create(school_id = row[0])
if (school_obj0.name != row[1]):
print (school_obj0.name, '==>', row[1])
confirmation = input('proceed? [y/n]: ')
if (confirmation == 'y'):
school_obj1, created = Org.objects.update_or_create(
school_id = row[0], defaults={"name": row[1],})
Happy to know about improvements to this approach (please see the update in the question with more context)
This will be invoked from the shell, so ideally it would be nice to be
prompted for confirmation just before a value is being changed
Unfortunately, databases don't work like that. It's the responsibility of applications to provide this functionality. And django isn't an application. You can however use django to write an application that provides this functionality.
As for finding out whether an object was updated or created, that's what the return value gives you. A tuple where the second value is a flag for update or create
In my application , I am going to use connectwise API , but I can't figure out how to call their API, like
How to connect
How to pass header (json data)
How to authentication (I have company id , public and private key)
How to make call and take response
I am Python guy
Thanks in advance
This question is quite broad, and you'll need to get familiar with something such as the Requests module if you aren't already.
Also, ConnectWise has a lot of documents at their developer site and registration is free and easy.
However I went through this journey myself over the last few weeks and learnt a lot so I'll share it with you.
Authentication
Authentication is done by creating an API user. In CW Manage you can create a member user with the API license class.
In ConnectWise Manage:
Go to System => Members
Go to the API Members tab.
Create a new API member that gives the API access to the areas that you need it, such as corporate/territorial levels. Note the username, amd that you cannot specify a password. Complete all the mandatory fields (owner ID, system default, group, approvers, etc).
When you have made this user go to the API keys tab. Enter a description for your new set of keys, and save it. When you do this you'll see the public and private keys once (and once only). Note them down.
Authorisation Header
In Python:
>>> import base64
>>> base64.b64encode("a+b:c")
...where a is your ConnectWise company name (what you type in in the login box), b is your public key, and c is your private key. This will return something which will be what you use to access the system. Copy it.
Making your first request
Now in Python make a new file and put this in it (this is quick and dirty but it should get you started):
import requests
cwToken = ""
# This is the ConnectWise access code generated earlier
cwUrl = "https://api-eu.myconnectwise.net/v4_6_release/apis/3.0/"
# check the URL matches your region, look at your CW Manage login box if not
cwHeaders = {"Authorization":"Basic " + cwToken,
"clientID":"<insertyoursecretClientIDhere>",
"Content-Type":"application/json"}
# this is your authorisation payload
try:
r = requests.get(cwUrl + "company/companies?pageSize=1000&conditions=type/id==1", headers=cwHeaders)
# request has been made
r.raise_for_status()
except:
print(r.text)
raise
companies = r.json()
The companies object now contains a list of your first 1,000 clients (type is ID 1 -- client). I've included an example of the conditions string because it took me a while to work out what it is and how to use it. But just take it off if you don't want the server to do your filtering for you.
You'll then be able to modify the above, or turn it into a module (which is what I did) to then make quick and portable calls, something like
cw.getCompanies()
which would simply return you a JSON object with all your companies in.
Going further
Study the documentation!! It is comprehensive enough to get you started. And the forums are actively monitored by staff (although they help mostly with C# queries and not scripting).
Just an update for 2020 since Daniel's response was incredibly helpful, but is missing a change Connectwise made in 2019 that now requires ClientIDs in order to auth.
Make a client ID here (scroll to bottom, although it's quick and useful to read all of it):
https://developer.connectwise.com/ClientID#What_is_a_clientId.3F
Daniel's code should then look like:
cwHeaders = {"Authorization":"Basic " + cwToken,
"clientID":"<insertyoursecretClientIDhere>",
"Content-Type":"application/json"}
I have built some RESTful api's with REstlet 2.3.4. I've been using HTTP_BASIC which let the browser prompt for credentials but it's time for a proper login form. I figure the easiest way to implement this is CookieAuthenticator. I can't find full working examples on github/google. I am sure i'm over looking them can someone provide a working example implementing CookieAuthenticator in Restlet?
I did get this to work after all. I have a longer answer here with some code examples. First, i was missing the fact that CookieAuthenticator is a filter and HAS the logic to handle login and logout. You need to create EMPTY ServerResources with a method annotated with #Post that has nothing in the body. Second, extend CookieAuthenticator and overwrite isLoggingIn(..) and isLoggingOut(..) with the code found in the link.
Cheers,
-ray
I have 500 or so spambots and about 5 actual registered users on my wiki. I have used nuke to delete their pages but they just keep reposting. I have spambot registration under control using reCaptcha. Now, I just need a way to delete/block/merge about 500 users at once.
You could just delete the accounts from the user table manually, or at least disable their authentication info with a query such as:
UPDATE /*_*/user SET
user_password = '',
user_newpassword = '',
user_email = '',
user_token = ''
WHERE
/* condition to select the users you want to nuke */
(Replace /*_*/ with your $wgDBprefix, if any. Oh, and do make a backup first.)
Wiping out the user_password and user_newpassword fields prevents the user from logging in. Also wiping out user_email prevents them from requesting a new password via email, and wiping out user_token drops any active sessions they may have.
Update: Since I first posted this, I've had further experience of cleaning up large numbers of spam users and content from a MediaWiki installation. I've documented the method I used (which basically involves first deleting the users from the database, then wiping out up all the now-orphaned revisions, and finally running rebuildall.php to fix the link tables) in this answer on Webmasters Stack Exchange.
Alternatively, you might also find Extension:RegexBlock useful:
"RegexBlock is an extension that adds special page with the interface for blocking, viewing and unblocking user names and IP addresses using regular expressions."
There are risks involved in applying the solution in the accepted answer. The approach may damage your database! It incompletely removes users, doing nothing to preserve referential integrity, and will almost certainly cause display errors.
Here a much better solution is presented (a prerequisite is that you have installed the User merge extension):
I have a little awkward way to accomplish the bulk merge through a
work-around. Hope someone would find it useful! (Must have a little
string concatenation skills in spreadsheets; or one may use a python
or similar script; or use a text editor with bulk replacement
features)
Prepare a list of all SPAMuserIDs, store them in a spreadsheet or textfile. The list may be
prepared from the user creation logs. If you do have the
dB access, the Wiki_user table can be imported into a local list.
The post method used for submitting the Merge & Delete User form (by clicking the button) should be converted to a get method. This
will get us a long URL. See the second comment (by Matthew Simoneau)
dated 13/Jan/2009) at
http://www.mathworks.com/matlabcentral/newsreader/view_thread/242300
for the method.
The resulting URL string should be something like below:
http: //(Your Wiki domain)/Special:UserMerge?olduser=(OldUserNameHere)&newuser=(NewUserNameHere)&deleteuser=1&token=0d30d8b4033a9a523b9574ccf73abad8%2B\
Now, divide this URL into four sections:
A: http: //(Your Wiki domain)/Special:UserMerge?olduser=
B: (OldUserNameHere)
C: &newuser=(NewUserNameHere)&deleteuser=1
D: &token=0d30d8b4033a9a523b9574ccf73abad8%2B\
Now using a text editor or spreadsheet, prefix each spam userIDs with part A and Suffix each with Part C and D. Part C will include the
NewUser(which is a specially created single dummy userID). The Part D,
the Token string is a session-dependent token that will be changed per
user per session. So you will need to get a new token every time a new
session/batch of work is required.
With the above step, you should get a long list of URLs, each good to do a Merge&Delete operation for one user. We can now create a
simple HTML file, view it and use a batch downloader like DownThemAll
in Firefox.
Add two more pieces " Linktext" to each line at
beginning and end. Also add at top and at
bottom and save the file as (for eg:) userlist.html
Open the file in Firefox, use DownThemAll add-on and download all the files! Effectively, you are visiting the Merge&Delete page for
each user and clicking the button!
Although this might look a lengthy and tricky job at first, once you
follow this method, you can remove tens of thousands of users without
much manual efforts.
You can verify if the operation is going well by opening some of the
downloaded html files (or by looking through the recent changes in
another window).
One advantage is that it does not directly edit the
MySQL pages. Nor does it require direct database access.
I did a bit of rewriting to the quoted text, since the original text contains some flaws.
For a project I'm trying to get data from a website only acessible when you're logged in from the site Goodreads.com. I'm new to Jsoup, since I'm using it only for this particular project. Getting the relevant data from the website is not a problem, but I can't seem to get to the particular page I need. The page I'm trying to acces is viewable only when logged in, when not logged in it rederects to the log-in page.
I've looked through the answers here, but the answers given so far have not helped.
What I have now:
String url = "http://www.goodreads.com/friend/user/7493379-judith";
Connection.Response res = Jsoup.connect("http://www.goodreads.com/user/sign_in")
.data("email", "MYEMAIL", "user_password", "MYPASSWORD")
.method(Connection.Method.POST)
.execute();
Document doc2 = res.parse();
String sessionId = res.cookie("_session_id");
Document doc = Jsoup.connect(url)
.cookie("_session_id", sessionId)
.get();
I got this far with help of the answers here, but it doesn't work, I'm still only getting the data from the log-in page it rederects to.
I have several questions:
Most importantly of course; How can I make it work?
The given answers here heve used method.(Method.POST) instead of method.(Connection.Method.POST) . When I use the first one however, I get an error that Method cannot be resolved. Anyone know why this is?
The examples I've seen have used "username" and "password" in .data() . What exactly do these refer to? I've now used the name of the input box. Is it the name, the type, the id, what exactly? Since Goodreads does not refer to the log in as the username, but as the e-mail, I assume I have to change them. (username & password doesn't work either)
Examples also use http://example.com/login.php as example url. Goodreads doesn't have a /login.php page though. Am I correct to assume I have to use the url with the log-in screen?
_session_id is the name of the relevant cookie on Goodreads.
I'd be very grateful if anyone can point me in the right direction!
See carefully what data is posted on login:
user[email]:email#email
remember_me:on
user[password]:plain_pasword
n:667387
So your post must execute exact same keys.
2.Make sure, you make right import: import org.jsoup.Connection.Method;
but Connection.Method.POST is still good.
3.See p1
4.Yes, you are correct
5.what is the question?
Goodreads requires two things when logging in: first, that you have a session ID stored in a cookie, and second, that you have a random generated number. You can get these when first visiting the login page without logging in: it will set a cookie with a session ID, and the form will contain a hidden input form (i.e. ) with the name "n" and value a number. Save these and pass them along as respectively a cookie and a form value when logging in.
Some remarks about the way I found this out:
The first thing you need to realise is that you're trying to recreate the exact same requests your browser does with Jsoup. So, in order to check whether what you have right now will work, you can try to recreate the exact same situation with your browser.
To recreate your code, I went to the login page, then I deleted all my Goodreads cookies (as you don't send along any cookies when you send the login request as well), and attempted to sign in with only passing the username and password form values. It gave an error that my session had timd out. When I first loaded the login page and then deleted all cookies except the session ID and did not remove the "n" form value, I could log in successfully. Therefore, you want to make a general GET request to the sign in page first, retrieve the session ID cookie you get there and the hidden form value, and pass it along with the POST request.
It could be that the API changed or that there just are several ways. Using Connection.Method.POST will do fine, in any case.
Yes, they refer to the names of the input boxes. This should be id, however, since name was used in the past and not all versions of all browsers supported passing the ids as data, most websites are just adding both. Either should be fine.
If you look at the source code of the sign in form, you can see that the "method" attribute of the form element is indeed the sign in page itself, so that's where it sends the request to.
PS. As a general tip, you can use the Firefox extension "Tamper Data" to remove form data or even cookies (though there are easier extensions for that).
You can log in with this code:
public static void main(String[] args) throws Exception {
Connection.Response execute = Jsoup
.connect("https://www.goodreads.com/")
.method(Connection.Method.GET).execute();
Element sign_in = execute.parse().getElementById("sign_in");
String authenticityToken = sign_in.select("input[name=authenticity_token]").first().val();
String n = sign_in.select("input[name=n]").first().val();
Document document = Jsoup.connect("https://www.goodreads.com/user/sign_in")
.data("cookieexists", "✓")
.data("authenticity_token", authenticityToken)
.data("user[email]", "user#email.com")
.data("user[password]", "password")
.data("remember_me", "on")
.data("n", n)
.cookies(execute.cookies())
.post();
}