I've been told that I should post about solutions to problems that I come across, so as I always try to please my fans (you know it's true
Brian
!), here is a short description of a problem I came across this week and my solution.
Basically I had to find a way to automatically fill in webforms on Amazon. The Amazon API is not very helpful in this department so I went about filling the forms in manually first in order to get used to Amazon's way of doing things. Here's some of the things I learnt:
(By the way, the Firefox browser with the Web Developer Toolbar was a great help. I recommend it for anyone doing this kind of work.
Available here
)
One of the problems with filling in webforms automatically is the host of hidden fields that may be present. As a human filling in these forms is not a problem as that's why they're hidden, they don't need human interaction. But trying to get them to run automatically is a real pain, especially since some of the forms had up to 24 hidden fields. That's why I came up with the following piece of code
MatchCollection matches = Regex.Matches(strResponse,
"(?nsi-x)[^\"]*)\" value=\"(?[^\"]*)\">"
);
for
(i=0; i"name"
].Value); objBuilder.Append(
"="
); objBuilder.Append(matches[i].Groups[
"value"
].Value); objBuilder.Append(
"&"
);}
The Regex will find any parts of
strResponse
(which is the HTML code of the page your currently on) which are in the form:
<
INPUT
TYPE
="hidden"
NAME
="example_input_name"
VALUE
="example_input_value"
>
The
Regex.Matches()
method then stores all the values of
Name
and the values of
Value
in the
matches
object. Be warned this will only work for HTML pages with quotes around tag attributes. Another short cut you can use when filling in forms is to use regular expressions to extract the action attribute of the form. e.g.
Match match = Regex.Match(strResponse,
"(?nsi-x)action=\"(?[^\"]*)\""
);
This will extract the URL of the form action script to
formUrl
.
Of course you must know what page you are currently on so that you can fill in the fields that aren't hidden!! In order to upload these to the form action script simply add
&example_form_input_name=example_form_input_value
to the end of the URL for each name-value pair, e.g.
&username=my_username
You can then reconstruct the URL of the completed form by doing the following:
Match match = Regex.Match(strResponse,
"(?nsi-x)action=\"(?[^\"]*)\""
);MatchCollection matches = Regex.Matches(strResponse,
"(?nsi-x)[^\"]*)\" value=\"(?>[^\"]*)\">"
);StringBuilder objBuilder =
new
StringBuilder();objBuilder.Append(match.Groups[
"formUrl"
].Value);objBuilder.Append(
"?"
);
for
(
int
i=0; i"name"
].Value); objBuilder.Append(
"="
); objBuilder.Append(matches[i].Groups[
"value"
].Value); objBuilder.Append(
"&"
);}objBuilder.Append(
"&username=my_username&password=my_password"
);
string
strFilledFormUrl = objBuilder.ToString();
The fully filled in form URL is now stored in
strFilledFormUrl
. You can now navigate to this URL using
HttpRequest
,
HttpResponse
and
Cookies
;), but that's a story for another day.