SecureBlackbox 16: HTTP and HTTPS proxies

Introduction

Modern proxy servers can be used as gateways for requests that access both HTTP and HTTPS resources. This causes confusion that leads to misconfiguration and sometimes security breaches. Below we will discuss how a proxy should be done properly for each type of request.

HTTP Proxy

In this flow, the HTTP client sends a request to the HTTP proxy, asking the proxy to retrieve the remote resource and forward it to the client. The resource can be accessed using a protocol different from HTTP; i.e., if the HTTP proxy supports this, the client can pass an FTP or other URL. This includes HTTPS resources as well. The HTTP client sends a request using common HTTP verbs such as GET, POST, HEAD, etc.

The HTTP proxy accepts the request from the client, analyzes it, and acts accordingly. If the remote resource needs to be retrieved (and cannot be taken from the cache, for example), the HTTP proxy establishes the connection to the remote server and acts as a client for that remote server. The resource is downloaded and passed to the client.

If the remote resource is accessed using the HTTPS protocol, the HTTP proxy validates the X.509 certificate presented by the remote server.

End-to-end security cannot be achieved merely using this HTTP connection. It is possible to ensure security by protecting the resource beforehand, but even when both the client and the proxy use HTTPS, the proxy has access to the original data not protected by HTTPS. Moreover, the unprotected data possibly stays in the cache of the proxy (if the proxy uses caching).

HTTPS Proxy

HTTPS proxies were invented to ensure communication with end-to-end security. In this flow, the client sends a special request to the proxy with the CONNECT verb. The proxy builds an opaque tunnel by connecting to the requested server using TCP and nothing else. After the socket connection is established, the HTTPS proxy sends a 200 OK response to the client and starts forwarding data from the client to the server and back. Such a design means that the client and the server are not limited to HTTPS traffic. In fact, any protocol can be tunneled using an HTTPS proxy and the CONNECT verb.

End-to-end security is achieved by establishing a secure channel between the client and the server after the proxy has connected to the server and confirmed the operation to the client.

An HTTP proxy should not be used for HTTPS resources for purposes other than debugging or espionage.

The SecureBlackbox components support both HTTP proxies (in the HTTP/HTTPS client and server components) and HTTPS proxies (in the socket class and all socket-based components and classes). The HTTPS proxy is called WebTunneling in SecureBlackbox.