1. Home
  2. Header Modification, Routing, URL-Rewrite, Redirects, and General URI Substitution

Header Modification, Routing, URL-Rewrite, Redirects, and General URI Substitution

The Snapt Aria Load Balancer allows for advanced header modification, rewriting requests, client redirection and routing in an explicit or conditional manner.

The Load Balancer uses some built-in variables to capture or fetch samples of traffic information like headers, client source address, hostname, request path and queries and so on. This information can be transformed using converter functions, or substituted (optionally using regular expressions) for new information in part or in full.

This allows for scenarios like client location redirection, on-the-fly rewriting of requests URLs and for manipulating request and response headers. It is also possible to specify conditions under which any of these actions are performed.

Keywords

These are directives used to specify the load balancer configuration. They are typically followed by variables and parameters that describe the detail of logic to be carried out on traffic, with or without condition. Some of the most commonly used keywords are http-request, http-response, use_backend, use-server, timeout, option, acl, and others include tcp-request, tcp-response, stick, stick-table, and so on. This list is by no means exhaustive.

The combination of keywords, and associated actions, traffic sample variables, converter functions, and conditional ACLs, explained in further detail below, can be configured for a group, frontend, or backend in the header modification configuration section under the HTTP options tab. Multiple action lines can be configured. This section of the load balancer configuration is generally used to apply some desirable logic to traffic for routing or header manipulation purposes.

 

Traffic sample variables

These are built-in and custom variables that are automatically populated with data sampled from traffic by the load balancer during a request or response transaction. They can be accessed and used for URL and header manipulation purposes. Built-in variables include, among others:

  • Request path, URL and method information – path, url, method
  • Client IPv4 address – src
  • Response status – status
  • Header information – hdr(<header>)
  • Header information with regular expression matching applied – hdr_reg(<header>)
  • Path information with regular expression matching – path_reg
  • URL with substring matching applied – url_sub
  • Request destination and source ports – dst_port, src_port, etc. 

 

Some variables also refer to some aspects of the load balancer’s state including:

  • The number of currently established connections – be_conn(<backend>)
  • Average queue across servers in a backend – avg_queue(<backend>)
  • Number of usable servers in backends – nbserv(<backend>)
  • Number of connection slots available – connslots(<backend>)
  • Session rates on both frontends and backends – fe_sess_rate(<frontend>), be_sess_rate(<backend>) and so on.

 

HTTP Request and Response keywords

These are probably the most common keywords used which specify what operations to perform during the request and response phase of an HTTP transaction. These actions include redirect, set-path, set-uri, add-header, replace-header, allow, deny, set-status, and many more. Some actions such as set-status can only be performed during the response phase of communication. These actions can be configured on request and response traffic on the load balancer group, frontend, and backend configurations.

Header Modification

Using the HTTP request and response keywords, we can perform multiple and complex header modifications to request or response traffic. The following actions can be specified for use in header manipulation:

  • add-header – Adds a new header line with the name and value specified.
  • replace-header – Replaces all occurrences of the header specified which match a regular expression pattern with new content or content modified from the original header value.
  • replace-value – This matches against every comma-delimited value in the specified header. Useful for headers that are allowed to have more than one value per line such as the X-Forwarded-For header.
  • set-header – Similar to add-header except the header specified is removed if it exists. Useful for passing information to backend servers using headers that should not be manipulated by clients.
  • del-header – This removes all occurrences of the header specified.

 

Here are some examples:

http-request set-header host “snapt.test”

http-request add-header X-Forwarded-For src (Utilizes the src traffic sample; described below)

http-response del-header content-length

 

Another useful scenario is when rewriting set-cookie headers in responses from backend servers. Such as:

http-response replace-header set-cookie (cvar=[^;]*);(.*) \1;ip=%bi;\2

When the configuration line above is applied to response traffic with the header – set-cookie: cvar1=test; cvar2=test, the header is modified to – set-cookie: cvar1=test;ip=192.168.0.111

; cvar2=test. This assumes the backend IP, retrieved with the built-in load balancer state variable – %b, is 192.168.0.111.

 

A common mechanism used for server-instantiated client redirection is the use of status codes such as 301 and 302 sent in a response to clients. This causes each client to resend its request to the location specified in the location header. A redirect response sent by backends server can be intercepted and modified or even initiated by the load balancer by setting the required status and location header in the response set to the client. For instance:

http-response set-header location https://snapt.net/redirected

http-response set-status 302 reason “My temporary redirect”

For this particular scenario there are better ways of achieving the same results, some of which are explored in the ‘redirects’ section below. The recommended method employs the use of a one-liner that allows specifying both the new locations and the status code. An example will look like:

http-request redirect location https://snapt.net/redirected code 302

 

Variable converter functions

These allow for the inline conversion or transformation of variables. Multiple inline transformations can be applied to built-in or user-defined variables by appending a comma-separated list of functions to a variable name. These functions can be used to carry out arithmetic, bitwise and comparator operations as well as string transformation (uppercase, lowercase, etc), string searches and matching, input format transformations (e.g binary to base64), setting custom variables based on built-in variables or their transformations and much more. They include among others:

  • Regular expression based substitution – regsub
  • Input matching against files that contain input-output mappings – map
  • Transform string to upper or lower case – upper, lower
  • Length of a string variable – length
  • Arithmetic operations with variables – add, sub, div, sub
  • Boolean operations – not, odd, even
  • Bitwise operations – and, or, xor

 

ACLs

These are used to define conditions upon which header manipulation, URL rewriting, client redirections, routing and more take place. For instance, it is possible to take certain actions if the domain, path, query, request method, etc matches specified values. By default, the action instructions specified are always executed. Examples of ACL declarations include:

  • acl ecppath path_beg -i /ecp
  • acl acl2 hdr_beg(host) -i www
  • acl acl3 hdr(host) -i test.com
  • acl acl4 path -i /owa/foldera
  • acl sourceip src -i 192.168.0.2

A typical ACL line includes the ‘acl’ directive or action keyword, the ACL name, a match criterion which is usually a built-in variable optionally transformed using converter functions, some match-related flags and match values.

These and more can be specified under Balancer -> ACL Management.

ACL names are then referenced in other config lines whereby the conditions they describe are checked/evaluated before the config lines are executed by the load balancer. The result of the evaluation decides whether the config lines are executed or not.

 

Routing

The load balancer can be used to selectively route client traffic to different backend server pools or specific servers with backend server pools. ACLs are used to set conditions upon which the desired routing is done.

 

 

The following routing paths are possible:

  • From load balancer groups to different backends (server groups) and/or individual servers within the group.
  • From a load balancer frontend to different backends (server groups).
  • From a load balancer backend to individual servers within the backend’s server pool.

 

Redirects

Snapt Aria load balancer also supports client redirection to a static or dynamically modified URL (scheme, hostname, path and queries). 

Some examples are provided below. These can be configured under the header modification options section of a load balancer group, frontend or backend.

 

 

Client redirection with a modified hostname

The path and queries in the original request are preserved.

Http-request redirect location http://www.%[hdr(host)]%[capture.req.uri]

The redirect action is used with the redirection location specified with the location parameter.

We append www to the hostname specified – fetched by hdr(host) – and append the original path and query string specified. %[capture.req.uri] is another way of capturing the path and query string in a URL.

For example, given https://snapt.test/test/another-page/index.html?q=1, this will cause a 302 redirection response to be sent to the client with the new location – https://www.snapt.test/test/another-page/index.html?q=1

Typically, this type of redirection will be accompanied by an ACL that is used to check whether www is already part of the original request. For example, to check whether the original host name specified begins with www, we declare an ACL as follows:

acl checkwww hdr_beg(host) -i www

The redirection instruction with the ACL will then look like:

Http-request redirect location http://www.%[hdr(host)]%[capture.req.uri] unless checkwww

This essentially means perform the redirection unless the hostname specified already starts with www.

 

Client redirection with regex-based path substitution

To have clients redirected to the new path using regular expression based substitution, we can specify:

Http-request redirect location http://%[hdr(host)]%[url,regsub(/test,/sample,i)]

%[hdr(host)] fills in the original hostname concatenated with the amended path. So for https://snapt.test/test/another-page/index.html?qa=1&qb=2, clients are redirected to – https://snapt.test/sample/another-page/index.html?qa=1&qb=2

The url built-in variable refers to everything after the first / in the request URL including the query string. This is passed through the regsub converter function which substitutes the first occurrence of “/test” with “/sample”. As the url variable includes the query string, it can be used to make substitutions across the path, and query, strings specified by connecting users. 

Adding ‘g’ to the third parameter of the regsub converter function would cause the load balancer to carry out a global search and perform substitutions wherever there is a match in the content of the built-in variable – url. So with: 

Http-request redirect location http://%[hdr(host)]%[url,regsub(test,sample,ig)] 

https://snapt.test/test/another-page/index.html?test=1&qb=test will be transformed to https://snapt.test/sample/another-page/index.html?sample=1&qb=sample

Typically an ACL is used to avoid redirecting on every request unless the specific conditions are met. In this case, we can declare and use an ACL that checks using path_beg whether the path in the original request begins with /sample:

acl pathtest path_beg -i /sample

Updating the location redirection instructions, we get:

Http-request redirect location http://%[hdr(host)]%[url,regsub(test,sample,ig)] unless pathtest

 

Basic URL Rewrites

It is also possible to rewrite the URL presented to backend servers without client redirection.

The following examples and more can be configured under the header modification options section of a load balancer group, frontend, or backend.

 

 

Rewrite path without redirecting clients

We can partially rewrite the path in request URLs without redirecting clients. This changes the request path seen by the backend server(s). For instance, https://snapt.test/test/another-page/index.html?qa=1&qb=2 can be changed to https://snapt.test/sample/another-page/index.html?qa=1&qb=2 in-flight so that the backend server sees the second request URL using:

Http-request set-path %[path,regsub(/test,/sample,i)]

The path variable contains everything from the first / and before ? in the original URL specified by clients.

Here we use the built-in regsub converted function to substitute ‘sample’ in the place of the first occurrence of ‘test’ in the request path sent to backend server(s).

This leaves the query string (part of URL after the first ‘?’) in place.

The third parameter in the regsub converter function, ‘i’ in this case, instructs the load balancer to carry out a case-insensitive match test of ‘/test’. Adding ‘g’ would cause the load balancer to carry out a global input search and substitute for all the occurrences of /test in the URL path. For instance, https://snapt.test/test1/test2/another-page/index.html?qa=1&qb=2 will produce https://snapt.test/sample1/sample2/another-page/index.html?qa=1&qb=2.

Full path rewrite

This involves a full path rewrite without client redirection. For example, to rewrite the full URL path, we can use:

Http-request set-path /newpath/newsubpath/index.html

With this, https://snapt.test/test/another-page/index.html becomes https://snapt.test/newpath/newsubpath/index.html when the request reaches our backend server(s).

Injecting a new base path

To inject a new base path into the path specified by clients (path prefixing), we can also use the set-path directive in addition to captured path data in the original client request.

http-request set-path /newbasepath%[path]

%[path] substitutes in the original path specified by the client so https://snapt.test/test/another-page/index.html becomes https://snapt.test/newbasepath/test/another-page/index.html

URL query rewrite

The query section is the part of the URL after the ?. It is possible to search through the query and perform a regular expression based replacement such as:

http-request set-query %[query,regsub(%3D,=,g)]

The built-in query variable contains the URL query which the regsub converter function operates on by replacing any occurrences of “%3D” with “=”.

The third parameter in the regsub converter function, ‘g’, instructs the load balancer to carry out a global search for “%38” and replacement with “&” in the query input as opposed to stopping the search and replace after the first hit.

Given https://www.snapt.test/test/another-page/index.html?qa%38queryval1&qb%38queryval2 the URL sent to the backend server(s) will be https://www.snapt.test/test/another-page/index.html?qa=queryval1&qb=queryval2

It is also possible to refer to query variables using their associated parameter name. For instance,

http-request set-query first=%[urlp(qa)]&new=addedquery

With a client input of https://www.snapt.test/test/another-page/index.html?qa=queryval1&qb=queryval2, this will produce https://www.snapt.test/test/another-page/index.html?first=queryval1&new=addedquery

 

Advanced URL Rewrite

Full URL rewrite

The set-uri function can be used in certain use cases where it is useful to re-compose the entire URL, including the scheme, host, path and query, that is sent onwards to backend server(s). Some parts of the URL can be statically specified and others dynamically fetched from client requests or server responses, as needed. For example:

  • To rewrite the scheme, host, path and query all at once, we can use:

Http-request set-uri “http://new.snapt.net/testing?query=1”

  • To change both host and path, leaving the original request query in place:

Http-request set-uri “http://new.snapt.net/testing?%[query]”

Updated on July 5, 2021


Was this article helpful?