SecureBlackbox 16: Form-Based Authentication and the HTTP Client

Form-based authentication has little to do with the HTTP protocol itself: HTTP acts as a transport, but authentication happens at a higher level. In form-based authentication, you enter data into an HTML form and post this data to the site. You then get some information in the response, then you use that information further -- to accomplish this, you need to parse HTML, possibly execute JavaScript, etc. This all is beyond HTTP.

This KB outlines how to automate this operation to log into a site.

First of all, you need to look at the source code of the web page with the login form. You need to find the <form> tag and analyze what fields (form elements are identified with the id and name parameters) are present in the form, and depending on their type, create your POST request.

The page to submit the data to is specified in the <form> tag, field names are taken from the form element tags (such as <input> or <textarea>), and your application provides field values. Next, you use the POST method of TElHTTPSClient to post the data.

The next step is to capture the cookies that are returned by the server in response to the POST operation. This is implemented by handling the OnCookie event and saving the provided cookie information. Note that the cookie contains not just the cookie data itself, but also supplementary information (metadata) that tells you how to apply the cookie and the period during which this cookie is valid.

The following step is to send this cookie when you request the target resource (for which you performed authorization). During this step, you need to add the obtained cookie to the RequestCookies list.

The major problem that presents itself is that there exist many sites that explicitly make automated authentication hard or impossible. This usually includes a CAPTCHA or JavaScript that decrypts some form data on-the-fly (using JavaScript mechanisms) or encrypts the parameters passed to the server. There's no uniform solution to this problem available.