Regex and POST in same connection - regex

Question

please bear with me, i'm brand new to Python!
I'm trying to login to a website which uses PHP. The form contains two hidden fields, the value of one and the name of another are generated on page load.
My code below successfuly accesses the page and using regex manages to return the values - great!
The problem I am having is that I then generate my querystring that will be used for the POST (this contains the two values obtained earlier) and opens the url again. This generates brand new tokens/values and my originals are of no use.
Can someone shed some light on how I can connect to a site, use regex to get the values and then POST all in the same connection.
I hope i've made myself clear, if not please let me know.
Thanks in advance for your help.
import urllib2,urllib,re,cookielib
url='http://www.example.com/index.php'
req = urllib2.Request(url)
req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3 Gecko/2008092417 Firefox/3.0.3')
response = urllib2.urlopen(req)
link=response.read()
response.close()
token1=re.compile('<input type="hidden" name="return" value="(.+?)" />').findall(link)
token2=re.compile('<input type="hidden" name="(.+?)" value="1" />').findall(link)
print token1[0]
print token2[0]
username = 'username'
password = 'password'
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
login_data = urllib.urlencode({'username' : username, 'password' : password, 'return' : token1[0], token2[0] : '1', 'Submit' : 'Log in', 'option' : 'com_users', 'task' : 'user.login'})
opener.open('http://www.example.com/index.php', login_data)
resp = opener.open('http://www.example.com/index.php')
FORM:
<form action="/index.php/welcome2" method="post" id="login-form" >
<fieldset class="userdata">
<p id="form-login-username">
<label for="modlgn-username">User Name</label>
<input id="modlgn-username" type="text" name="username" class="inputbox" size="18" />
</p>
<p id="form-login-password">
<label for="modlgn-passwd">Password</label>
<input id="modlgn-passwd" type="password" name="password" class="inputbox" size="18" />
</p>
<p id="form-login-remember">
<label for="modlgn-remember">Remember Me</label>
<input id="modlgn-remember" type="checkbox" name="remember" class="inputbox" value="yes"/>
</p>
<input type="submit" name="Submit" class="button" value="Log in" />
<input type="hidden" name="option" value="com_users" />
<input type="hidden" name="task" value="user.login" />
<input type="hidden" name="return" value="aW5kZXgucGhwP0l0ZW1pZD0xMjc=" />
<input type="hidden" name="c813c34837e4e48e8e3268c0a42912a2" value="1" />
</fieldset>
<ul>
<li>
<a href="/index.php/my-account/my-details?view=reset">
Forgot your password?</a>
</li>
<li>
<a href="/index.php/my-account/my-details?view=remind">
Forgot your username?</a>
</li>
<li>
<a href="/index.php/register">
Create an account</a>
</li>
</ul>
</form>

Answer 1

When you write...
opener.open('http://www.example.com/index.php', login_data)
resp = opener.open('http://www.example.com/index.php')
Why not just this?
resp = opener.open('http://www.example.com/index.php', login_data)
I've never used this Python library, but my first reaction is that this would give you the response text all in one request, with which you can get the new token, wouldn't it?
Update based on form: It looks like your problem is you're POSTing the login info to index.php rather than index.php/welcome.