# HTTP Headers ## General Headers [General headers](https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html) are used in both HTTP requests and responses. They are contextual and are used to `describe the message rather than its contents`. ## Entity Headers Similar to general headers, [Entity Headers](https://www.w3.org/Protocols/rfc2616/rfc2616-sec7.html) can be `common to both the request and response`. These headers are used to `describe the content` (entity) transferred by a message. They are usually found in responses and POST or PUT requests. | **Header** | **Example** | **Description** | | ------------------ | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Content-Type` | `Content-Type: text/html` | Used to describe the type of resource being transferred. The value is automatically added by the browsers on the client-side and returned in the server response. The `charset` field denotes the encoding standard, such as [UTF-8](https://en.wikipedia.org/wiki/UTF-8). | | `Media-Type` | `Media-Type: application/pdf` | The `media-type` is similar to `Content-Type`, and describes the data being transferred. This header can play a crucial role in making the server interpret our input. The `charset` field may also be used with this header. | | `Boundary` | `boundary="b4e4fbd93540"` | Acts as a marker to separate content when there is more than one in the same message. For example, within a form data, this boundary gets used as `--b4e4fbd93540` to separate different parts of the form. | | `Content-Length` | `Content-Length: 385` | Holds the size of the entity being passed. This header is necessary as the server uses it to read data from the message body, and is automatically generated by the browser and tools like cURL. | | `Content-Encoding` | `Content-Encoding: gzip` | Data can undergo multiple transformations before being passed. For example, large amounts of data can be compressed to reduce the message size. The type of encoding being used should be specified using the `Content-Encoding` header. | ## Request Headers | **Header** | **Example** | **Description** | | --------------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Host` | `Host: www.inlanefreight.com` | Used to specify the host being queried for the resource. This can be a domain name or an IP address. HTTP servers can be configured to host different websites, which are revealed based on the hostname. This makes the host header an important enumeration target, as it can indicate the existence of other hosts on the target server. | | `User-Agent` | `User-Agent: curl/7.77.0` | The `User-Agent` header is used to describe the client requesting resources. This header can reveal a lot about the client, such as the browser, its version, and the operating system. | | `Referer` | `Referer: http://www.inlanefreight.com/` | Denotes where the current request is coming from. For example, clicking a link from Google search results would make `https://google.com` the referer. Trusting this header can be dangerous as it can be easily manipulated, leading to unintended consequences. | | `Accept` | `Accept: */*` | The `Accept` header describes which media types the client can understand. It can contain multiple media types separated by commas. The `*/*` value signifies that all media types are accepted. | | `Cookie` | `Cookie: PHPSESSID=b4e4fbd93540` | Contains cookie-value pairs in the format `name=value`. A [cookie](https://en.wikipedia.org/wiki/HTTP_cookie) is a piece of data stored on the client-side and on the server, which acts as an identifier. These are passed to the server per request, thus maintaining the client's access. Cookies can also serve other purposes, such as saving user preferences or session tracking. There can be multiple cookies in a single header separated by a semi-colon. | | `Authorization` | `Authorization: BASIC cGFzc3dvcmQK` | Another method for the server to identify clients. After successful authentication, the server returns a token unique to the client. Unlike cookies, tokens are stored only on the client-side and retrieved by the server per request. There are multiple types of authentication types based on the webserver and application type used. | ## Response Headers | **Header** | **Example** | **Description** | | --------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Content-Security-Policy` | `Content-Security-Policy: script-src 'self'` | Dictates the website's policy towards externally injected resources. This could be JavaScript code as well as script resources. This header instructs the browser to accept resources only from certain trusted domains, hence preventing attacks such as [Cross-site scripting (XSS)](https://en.wikipedia.org/wiki/Cross-site_scripting). | | `Strict-Transport-Security` | `Strict-Transport-Security: max-age=31536000` | Prevents the browser from accessing the website over the plaintext HTTP protocol, and forces all communication to be carried over the secure HTTPS protocol. This prevents attackers from sniffing web traffic and accessing protected information such as passwords or other sensitive data. | | `Referrer-Policy` | `Referrer-Policy: origin` | Dictates whether the browser should include the value specified via the `Referer` header or not. It can help in avoiding disclosing sensitive URLs and information while browsing the website. | ## cURL If we were only interested in seeing the response headers, then we can use the `-I` flag to send a `HEAD` request and only display the response headers. Furthermore, we can use the `-i` flag to display both the headers and the response body (e.g. HTML code). ```bash eldeim@htb[/htb]$ curl -I https://www.inlanefreight.com Host: www.inlanefreight.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Cookie: cookie1=298zf09hf012fh2; cookie2=u32t4o3tb3gg4 Accept: text/plain Referer: https://www.inlanefreight.com/ Authorization: BASIC cGFzc3dvcmQK Date: Sun, 06 Aug 2020 08:49:37 GMT Connection: keep-alive Content-Length: 26012 Content-Type: text/html; charset=ISO-8859-4 Content-Encoding: gzip Server: Apache/2.2.14 (Win32) Set-Cookie: name1=value1,name2=value2; Expires=Wed, 09 Jun 2021 10:18:14 GMT WWW-Authenticate: BASIC realm="localhost" Content-Security-Policy: script-src 'self' Strict-Transport-Security: max-age=31536000 Referrer-Policy: origin ``` ``` eldeim@htb[/htb]$ curl https://www.inlanefreight.com -A 'Mozilla/5.0' ...SNIP... ``` > Exercise: Try to use the `-I` or the `-v` flags with the above example, to ensure that we did change our User-Agent with the `-A` flag. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://eldeim.gitbook.io/brain_fuck/notes/certifications/eastereggs/htb-cbbh/web-requests-fundamentals/http-fundamentals/http-headers.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.