Xathrya Sabertooth

Linux Kernel, Components, and Integration
Install Squid as Local Proxy Cache for Slackware64
List of HTTP Status Code
Revealing HTTP Request and Response
QEMU on Windows

Linux Kernel, Components, and Integration

Posted: 13 Mar 2013 06:01 AM PDT

Kernel and Linux Kernel

In term of Computer Science, Kernel is the core of an Operating System. A machine (for example: personal computer) can use various hardware produced by different vendors. All can be assembled into a single machine. A hardware like processor, RAM, Hard Disk, etc is the component to build a computer. But once the computer is built, we need an operating system to make all of these hardware can be operated. The kernel does the job.

Operating System receives the request from user and processes it on user's behalf. Requests are received by command shell or some other kind of user interface and are processed by the kernel. So, kernel acts like an engine of the operating system which enables a user to use a computer system. Shell is the outer part of the operating system that provides an interface to the user for communicating with kernel.

Kernel is bridging Applications and Hardware

Linux is one of Kernel. It is a UNIX-like kernel created by Linus Torvalds on 1991. Linux is Open Source means everyone can contribute, develop, and make their own kernel using Linus’ kernel. Nowadays, every smart system use kernel to operate and some of those system using (or maybe subset of) Linux.

Components

If we observe more, kernel can be divide into some components. The major components forming a kernel are:

Low Level Drivers : Architecture specific drivers and are responsible for CPU, MMU and on-board devices initialization .
Process Scheduler : Scheduler is responsible for fair cpu time slice allocation to different processes. Imagine you have some resource and must ensure every application can use fair amount of resource.
Memory Manager : Memory management system is responsible for allocating and sharing memory to different processes.
File System : Linux supports many file system types, e.g. - fat, ntfs, jffs and lot more. User doesn't have to worry about the complexities of underlying file system type. For this linux provides a single interface, named as virtual file system . Using a single Virtual File System interface users can use the services of different underlying file systems. The complexities of different file systems are abstracted from the user.
Network Interface : This component of linux kernel provides access and control to different networking devices.
Device Drivers : These are high level drivers .
IPC : Inter Process Communication , IPC subsystem allows different processes to share data among themselves.

Integrations

We have seen that a kernel consists of different components. Integration design tells how these different components are integrated to create kernel's binary image.

There are mainly two integration designs used for operating system kernels , monolithic and micro. Although there are more than two but we will limit our discussion into two most used integration design.

In monolithic design all the kernel components are built together into a single static binary image . At boot up time , entire kernel gets loaded and then runs as a single process in a single address space. All the kernel components/services exist in that static kernel image . All the kernel services are running and available all the time.

Since inside the kernel everything resides in a single address space, no IPC kind of mechanism is needed for communicating between kernel services. For all these reasons monolithic kernels are high performance. Most of the unix kernels are monolithic kernels.

The disadvantage of this static kernel is lack of modularity and hotswap ability. Once the static kernel image is loaded, we can’t add/remove any component or service from the kernel. Our option is only change the hardcoded kernel. Another reason is that kernel use much memory. So, resource consumption is higher in case of monolithic kernels.

The second kind of kernel is microkernel. In microkernel a single static kernel image is not built, instead kernel image is broken down into different small services.

At boot up time , core kernel services are loaded , they run in privileged mode . Whenever some service is required , it has to get loaded for running.

Unlike monolithic kernel all services are not up and running all the time. They run as and when requested. Also, unlike monolithic kernels , services in microkernels run in separate address spaces. Therefore communication between two different services requires IPC mechanism . For all these reasons microkernels are not high performance kernels but they require less resources to run .

Linux kernel takes best of both these designs. Fundamentally it is a monolithic kernel. Entire linux kernel and all its services run as a single process , in a single address space , achieving very high performance . But it also has the capability to load / unload services at run time in the form of kernel modules .

Install Squid as Local Proxy Cache for Slackware64

Posted: 13 Mar 2013 04:59 AM PDT

Squid is a caching proxy for Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-request web pages. Squid has extensive access controls and makes a great server accelerator. Run on most available operating system and licensed under GNU GPL.

Squid is used by hundreds of Internet Providers world-wide to provide their users with the best possible web access. Squid optimises the data flow between client and server to improve performance and caches frequently-used content to save bandwidth. Squid can also route content requests to servers in a wide variety of ways to build cache server hierarchies which optimise network throughput.

Thousands of web-sites around the Internet use Squid to drastically increase their content delivery. Squid can reduce server load and improve delivery speeds to clients. Squid can also be used to deliver content from around the world – copying only the content being used, rather than inefficiently copying everything. Finally, Squid’s advanced content routing configuration allows you to build content clusters to route and load balance requests via a variety of web servers.

In this article we will discuss about how to install Squid, gives a simple configuration, and then use it as a local cache server. Our goals is to improves response times and minimizing bandwidth on Slackware64 machine.

I use following:

Slackware64 14.0 with multilib support.
Squid Cache 3.3.3 source code

Obtain the Materials

The only material we need is squid’s source code which can be downloaded from their official site. At the time of writing this article, the latest stable version available is version 3.3.3. The direct download can be made on here.

As stated on site, we need Perl installed on our system. On Slackware64 it, is already installed by default, unless you have uninstalled it before. Make sure Perl is available.

Installation

Create a working directory. You can use any directory you want but in this case I will use my home directory /home/xathrya/squid. The archive we got is squid-3.3.3.tar.xz.

Now extract and configure the makefile.In this article I use /usr/local/squid directory as root of installation which is the default path for installing squid. If you want to install squid on another directory, on ./configure use –prefix=/path/to/new/squid where /path/to/new/squid is a path like /usr After compilation finished, install Squid using root privilege. A complete command to do so is given below:

  tar -Jxf squid-3.3.3.tar.xz  cd squid-3.3.3  ./configure  make  make install clean

The compilation might took some times, depending on your machine.

Setup

Squid is officially installed on this stage, but we need to do some setup to make it work properly.

Before we proceed we need to specify what are resource allocated to squid and what configuration must we set to meet our need. In my case, the squid can be activated on demand, the directory for caching is using a dedicated partition on /cache (you can also use other directory, and a dedicated partition is not a must) which is 48.0 GiB allocated, squid can use some peer that can be configured dynamically without need for me to change the configuration file directly.

Your need might be different from me, so adjust it yourself.

Create Basic Configuration File

In this example, the configuration file is located at /usr/local/squid/etc/squid.conf, but might be vary if you install squid on different directory than /usr/local/squid. On general, squid configuration file is located on <root directory>/etc/squid.conf

Now adjust your configuration file. Below is the configuration I use:

  ###############################################################  ##  ## BlueWyvern Proxy Service  ## XGN-Z30A : SquidProxy  ##  ###############################################################    ##  #      Proxy Manager Information  ##  cache_mgr xathrya@celestial-being.net  visible_hostname proxy.bluewyvern.celestial-being.net    ###############################################################    ##  #    Basic Configuration  ##  cache_effective_user squid  cache_effective_group squid    # DNS server (not required)  # Use this if you want to specify a list of DNS servers to use instead  # of those given in /etc/resolv.conf  #dns_nameservers 127.0.0.1 8.8.8.8    # Set Squid to listens port 1351 (normally listens to port 3128)  http_port 1351    # Timeouts  dead_peer_timeout 30 seconds  peer_connect_timeout 30 seconds    # Load the peer  include /usr/local/squid/peers.conf    ###############################################################    ##  #    Access Control List  #  #    My machine allow client from self, so IP other than self will be rejected  #    Also define some safe ports  ##  acl localnet src 10.0.0.0/8        # RFC1918 possible internal network  acl localnet src 172.16.0.0/12    # RFC1918 possible internal network  acl localnet src 192.168.0.0/16    # RFC1918 possible internal network  acl localnet src fc00::/7       # RFC 4193 local private network range  acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines    acl SSL_ports port 443  acl Safe_ports port 80        # http  acl Safe_ports port 21        # ftp  acl Safe_ports port 443        # https  acl Safe_ports port 70        # gopher  acl Safe_ports port 210        # wais  acl Safe_ports port 1025-65535    # unregistered ports  acl Safe_ports port 280        # http-mgmt  acl Safe_ports port 488        # gss-http  acl Safe_ports port 591        # filemaker  acl Safe_ports port 777        # multiling http  acl CONNECT method CONNECT    #  # Recommended minimum Access Permission configuration:  #  # Only allow cachemgr access from localhost  http_access allow localhost manager  http_access deny manager    # Deny requests to certain unsafe ports  http_access deny !Safe_ports    # Deny CONNECT to other than secure SSL ports  http_access deny CONNECT !SSL_ports    # We strongly recommend the following be uncommented to protect innocent  # web applications running on the proxy server who think the only  # one who can access services on "localhost" is a local user  #http_access deny to_localhost    #  # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS  #    # Example rule allowing access from your local networks.  # Adapt localnet in the ACL section to list your (internal) IP networks  # from where browsing should be allowed  http_access allow localnet  http_access allow localhost    # And finally deny all other access to this proxy  http_access deny all    ###############################################################    ##  #    Directory & Logs  #  #    We use /cache for directory  #    I have 48.0 GiB = 51 GB available  #        64 directories, 256 subdirectories for each directory  #  ##    # Cache directory 48GiB = 51500MB  cache_dir ufs /cache 51500 64 256    # Coredumps is specified on /cache too  coredump_dir /cache    # Squid logs  cache_access_log /var/log/squid/access.log  cache_log /var/log/squid/cache.log  cache_store_log /var/log/squid/store.log    # Defines an access log format  logformat custom %{%Y-%m-%d %H:%M:%S}tl %03tu %>a %tr %ul %ui %Hs %mt %rm %ru %rv %st %Sh %Ss    ###############################################################    ##  #    Other  ##  refresh_pattern ^ftp:        1440    20%    10080  refresh_pattern ^gopher:    1440    0%    1440  refresh_pattern -i (/cgi-bin/|\?) 0    0%    0  refresh_pattern .        0    20%    4320

Make user squid and group squid if you don’t have it yet. Then create cache directory if you don’t have any and change ownership to user squid and group squid (or any user and group you assign to squid, see squid.conf). Also I use /var/log/squid directory to log things squid need to. After all the preparations are ready, we need to do initial setup. Below is the snippet I use to do:

  ln -s /usr/local/squid/sbin/squid /usr/bin/squid    /bin/egrep -i "^squid" /etc/group  if [ $? -ne 0]; then  groupadd squid  fi    /bin/egrep -i "^squid" /etc/passwd  if [ $? -ne 0 ]; then  useradd -g squid -s /bin/false -M  squid  fi    if [ ! -d /cache ]; then  mkdir /cache  fi    chown squid.squid /cache    if [ ! -d /var/log/squid ]; then  mkdir /var/log/squid  fi    chown squid.squid /var/log/squid    /usr/local/squid/sbin/squid -z

Now create a file /usr/local/squid/etc/peers.conf and write all peer you want to use.

Creating Scripts

All the system are ready. Now we need to create a control panel script which can execute system. Using this script, I can start and stop squid, and also purge content from cache. The script I use is:

  #! /bin/bash  ROOTFOLDER=/usr/local/squid  SQUID=${ROOTFOLDER}/sbin/squid  SQUIDCLIENT=${ROOTFOLDER}/bin/squidclient    case $1 in  "start")  $SQUID start  ifconfig | grep inet  ;;  "purge")  $SQUIDCLIENT -h 127.0.0.1 -p 8080 -m purge $2  ;;  "stop")  $SQUID stop  ;;  esac

List of HTTP Status Code

Posted: 13 Mar 2013 01:47 AM PDT

HTTP Status codes is one of data sent from server to client as part of response. This response indicate what is the status of client request on server side. All of the statuses are defined by Internet Engineering Task Force (IETF) using some related Request For Comments (RFC) documents.

Currently, the official registry of HTTP status code are maintained by Internet Assigned Numbers Authority (IANA).

Note that, some server also extend the codes with their status codes. This is not universally implemented but on this article we will list all of the known and reported status codes used as response.

The Taxonomy

HTTP status code is consists of three digit number range from 0 to 9 for each digit. The first digit indicated what category of message is and the rest of digit indicates what specific information it has. Globally, the status code is divided into five categories with first digit range from 1 to 5.

1xx Informational

This category indicates that request has been received and process is continuing. This class of status is provisional response, consisting of the status-line and optional headers and terminated by an empty line. The status code is not defined on HTTP/1.0

100 Continue: This means that the server has received the request headers, and that the client should proceed to send the request body (in the case of a request for which a body needs to be sent; for example, a POST request). If the request body is large, sending it to a server when a request has already been rejected based upon inappropriate headers is inefficient. To have a server check if the request could be accepted based on the request’s headers alone, a client must send Expect: 100-continue as a header in its initial request and check if a 100 Continue status code is received in response before continuing (or receive 417 Expectation Failed and not continue).
101 Switching Protocols: This means the requester has asked the server to switch protocols and the server is acknowledging that it will do so.^[2]
102 Processing (WebDAV; RFC 2518): As a WebDAV request may contain many sub-requests involving file operations, it may take a long time to complete the request. This code indicates that the server has received and is processing the request, but no response is available yet.^[3] This prevents the client from timing out and assuming the request was lost.

2xx Success

The information is received and the request is processed successfully. In specifically the server has received, understood, accepted, and process the request.

200 OK: Most common response given by server. This is the standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request the response will contain an entity describing or containing the result of the action.
201 Created: The request has been fulfilled and resulted in a new resource being created.
202 Accepted: The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place.
203 Non-Authoritative Information (since HTTP/1.1): The server successfully processed the request, but is returning information that may be from another source.
204 No Content: The server successfully processed the request, but is not returning any content.
205 Reset Content: The server successfully processed the request, but is not returning any content. Unlike a 204 response, this response requires that the requester reset the document view.
206 Partial Content: The server is delivering only part of the resource due to a range header sent by the client. The range header is used by tools like wget to enable resuming of interrupted downloads, or split a download into multiple simultaneous streams.
207 Multi-Status: The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made.
208 Already Reported (WebDAV; RFC 5842): The members of a DAV binding have already been enumerated in a previous reply to this request, and are not being included again.
250 Low on Storage Space (RTSP; RFC 2326): The server returns this warning after receiving a RECORD request that it may not be able to fulfill completely due to insufficient storage space. If possible, the server should use the Range header to indicate what time period it may still be able to record. Since other processes on the server may be consuming storage space simultaneously, a client should take this only as an estimate.
226 IM Used (RFC 3229): The server has fulfilled a GET request for the resource, and the response is a representation of the result of one or more instance-manipulations applied to the current instance.

3xx Redirection

The request is received, but client must take additional action to complete the request. In this class, the further action must be performed by clients.The action required may be carried out by the user agent without user interaction and only if the method used in the second request is GET or HEAD. A user agent should not automatically redirect a request more than five times, since such redirections usually indicate an infinite loop.

300 Multiple Choices: Indicates multiple options for the resource that the client may follow. It, for instance, could be used to present different format options for video, list files with different extensions, or word sense disambiguation.
301 Moved Permanently: This and all future requests should be directed to the given URI.
302 Found: This is an example of industry practice contradicting the standard. The HTTP/1.0 specification (RFC 1945) required the client to perform a temporary redirect (the original describing phrase was “Moved Temporarily”), but popular browsers implemented 302 with the functionality of a 303 See Other. Therefore, HTTP/1.1 added status codes 303 and 307 to distinguish between the two behaviours. However, some Web applications and frameworks use the 302 status code as if it were the 303.
303 See Other (since HTTP/1.1): The response to the request can be found under another URI using a GET method. When received in response to a POST (or PUT/DELETE), it should be assumed that the server has received the data and the redirect should be issued with a separate GET message.
304 Not Modified: Indicates that the resource has not been modified since the version specified by the request headers If-Modified-Since or If-Match. This means that there is no need to retransmit the resource, since the client still has a previously-downloaded copy.
305 Use Proxy (since HTTP/1.1): The requested resource is only available through a proxy, whose address is provided in the response. Many HTTP clients do not correctly handle responses with this status code, primarily for security reasons.
306 Switch Proxy: No longer used.Originally meant “Subsequent requests should use the specified proxy.”
307 Temporary Redirect (since HTTP/1.1): In this case, the request should be repeated with another URI; however, future requests should still use the original URI.In contrast to how 302 was historically implemented, the request method is not allowed to be changed when reissuing the original request. For instance, a POST request should be repeated using another POST request.
308 Permanent Redirect (experimental RFC): The request, and all future requests should be repeated using another URI. 307 and 308 (as proposed) parallel the behaviours of 302 and 301, but do not allow the HTTP method to change. So, for example, submitting a form to a permanently redirected resource may continue smoothly.

4xx Client Error

Request is received, but server indicate that the request is error on client side. In this case, server should include an entity containing an explanation of the error situation, and explain whether it is a temporary or permanent condition. From RFC, the user agent should display any included entity to the user.

400 Bad Request: The request cannot be fulfilled due to bad syntax.
401 Unauthorized: Similar to 403 Forbidden, but specifically for use when authentication is required and has failed or has not yet been provided. The response must include a WWW-Authenticate header field containing a challenge applicable to the requested resource.
402 Payment Required: Reserved for future use. The original intention was that this code might be used as part of some form of digital cash or micropayment scheme, but that has not happened, and this code is not usually used.
403 Forbidden: The request was a valid request, but the server is refusing to respond to it. Unlike a 401 Unauthorized response, authenticating will make no difference. On servers where authentication is required, this commonly means that the provided credentials were successfully authenticated but that the credentials still do not grant the client permission to access the resource (e.g. a recognized user attempting to access restricted content).
404 Not Found: One of the most response found. The requested resource could not be found but may be available again in the future. Subsequent requests by the client are permissible.
405 Method Not Allowed: A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource.
406 Not Acceptable: The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request.
407 Proxy Authentication Required: The client must first authenticate itself with the proxy.
408 Request Timeout: The server timed out waiting for the request.According to W3 HTTP specifications: “The client did not produce a request within the time that the server was prepared to wait. The client MAY repeat the request without modifications at any later time.”
409 Conflict: Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.
410 Gone: Indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed and the resource should be purged. Upon receiving a 410 status code, the client should not request the resource again in the future. Clients such as search engines should remove the resource from their indices. Most use cases do not require clients and search engines to purge the resource, and a “404 Not Found” may be used instead.
411 Length Required: The request did not specify the length of its content, which is required by the requested resource.
412 Precondition Failed: The server does not meet one of the preconditions that the requester put on the request.
413 Request Entity Too Large: The request is larger than the server is willing or able to process.
414 Request-URI Too Long: The URI provided was too long for the server to process.
415 Unsupported Media Type: The request entity has a media type which the server or resource does not support. For example, the client uploads an image as image/svg+xml, but the server requires that images use a different format.
416 Requested Range Not Satisfiable: The client has asked for a portion of the file, but the server cannot supply that portion. For example, if the client asked for a part of the file that lies beyond the end of the file.
417 Expectation Failed: The server cannot meet the requirements of the Expect request-header field.
418 I’m a teapot (RFC 2324): Well, this is not an actual code used by HTTP server. Originally on 1998, it is used by IETF as Appril Fools’ jokes.
420 Enhance Your Calm (Twitter): Also not HTTP standard. This code is returned by the Twitter Search and Trends API when the client is being rate limited. Other services may wish to implement the 429 Too Many Requests response code instead.
422 Unprocessable Entity (WebDAV; RFC 4918): The request was well-formed but was unable to be followed due to semantic errors.
423 Locked (WebDAV; RFC 4918): The resource that is being accessed is locked.
424 Failed Dependency (WebDAV; RFC 4918): The request failed due to failure of a previous request (e.g. a PROPPATCH).
424 Method Failure (WebDAV): Indicates the method was not executed on a particular resource within its scope because some part of the method’s execution failed causing the entire method to be aborted.

425 Unordered Collection (Internet draft): Defined in drafts of “WebDAV Advanced Collections Protocol”, but not present in “Web Distributed Authoring and Versioning (WebDAV) Ordered Collections Protocol”.
426 Upgrade Required (RFC 2817): The client should switch to a different protocol such as TLS/1.0.
428 Precondition Required (RFC 6585): The origin server requires the request to be conditional. Intended to prevent “the ‘lost update’ problem, where a client GETs a resource’s state, modifies it, and PUTs it back to the server, when meanwhile a third party has modified the state on the server, leading to a conflict.”
429 Too Many Requests (RFC 6585): The user has sent too many requests in a given amount of time. Intended for use with rate limiting schemes.
431 Request Header Fields Too Large (RFC 6585): The server is unwilling to process the request because either an individual header field, or all the header fields collectively, are too large.
444 No Response (Nginx): Used in Nginx logs to indicate that the server has returned no information to the client and closed the connection (useful as a deterrent for malware).
449 Retry With (Microsoft): A Microsoft extension. The request should be retried after performing the appropriate action.; Often search-engines or custom applications will ignore required parameters. Where no default action is appropriate, the Aviongoo website sends a “HTTP/1.1 449 Retry with valid parameters: param1, param2, . . .” response. The applications may choose to learn, or not.
450 Blocked by Windows Parental Controls (Microsoft): A Microsoft extension. This error is given when Windows Parental Controls are turned on and are blocking access to the given webpage.
451 Parameter Not Understood (RTSP): The recipient of the request does not support one or more parameters contained in the request.
451 Unavailable For Legal Reasons (Internet draft): Defined in the internet draft “A New HTTP Status Code for Legally-restricted Resources”. Intended to be used when resource access is denied for legal reasons, e.g. censorship or government-mandated blocked access. A reference to the 1953 dystopian novel Fahrenheit 451, where books are outlawed.
451 Redirect (Microsoft): Used in Exchange ActiveSync if there either is a more efficient server to use or the server can’t access the users’ mailbox.; The client is supposed to re-run the HTTP Autodiscovery protocol to find a better suited server.
452 Conference Not Found (RTSP): The conference indicated by a Conference header field is unknown to the media server.
453 Not Enough Bandwidth (RTSP): The request was refused because there was insufficient bandwidth. This may, for example, be the result of a resource reservation failure.
454 Session Not Found (RTSP): The RTSP session identifier in the Session header is missing, invalid, or has timed out.
455 Method Not Valid in This State (RTSP): The client or server cannot process this request in its current state. The response SHOULD contain an Allow header to make error recovery easier.
456 Header Field Not Valid for Resource (RTSP): The server could not act on a required request header. For example, if PLAY contains the Range header field but the stream does not allow seeking.
457 Invalid Range (RTSP): The Range value given is out of bounds, e.g., beyond the end of the presentation.
458 Parameter Is Read-Only (RTSP): The parameter to be set by SET_PARAMETER can be read but not modified.
459 Aggregate Operation Not Allowed (RTSP): The requested method may not be applied on the URL in question since it is an aggregate (presentation) URL. The method may be applied on a stream URL.
460 Only Aggregate Operation Allowed (RTSP): The requested method may not be applied on the URL in question since it is not an aggregate (presentation) URL. The method may be applied on the presentation URL.
461 Unsupported Transport (RTSP): The Transport field did not contain a supported transport specification.
462 Destination Unreachable (RTSP): The data transmission channel could not be established because the client address could not be reached. This error will most likely be the result of a client attempt to place an invalid Destination parameter in the Transport field.
494 Request Header Too Large (Nginx): Nginx internal code similar to 431 but it was introduced earlier.
495 Cert Error (Nginx): Nginx internal code used when SSL client certificate error occurred to distinguish it from 4XX in a log and an error page redirection.
496 No Cert (Nginx): Nginx internal code used when client didn’t provide certificate to distinguish it from 4XX in a log and an error page redirection.
497 HTTP to HTTPS (Nginx): Nginx internal code used for the plain HTTP requests that are sent to HTTPS port to distinguish it from 4XX in a log and an error page redirection.
499 Client Closed Request (Nginx): Used in Nginx logs to indicate when the connection has been closed by client while the server is still processing its request, making server unable to send a status code back.

5xx Server Error

Request is received but server failed to fulfill an apparently valid request. In this case, the server cannot give the appropriate response due to some error occurred on server side.

500 Internal Server Error: A generic error message, given when no more specific message is suitable.
501 Not Implemented: The server either does not recognize the request method, or it lacks the ability to fulfill the request.
502 Bad Gateway: The server was acting as a gateway or proxy and received an invalid response from the upstream server.
503 Service Unavailable: The server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state.
504 Gateway Timeout: The server was acting as a gateway or proxy and did not receive a timely response from the upstream server.
505 HTTP Version Not Supported: The server does not support the HTTP protocol version used in the request.
506 Variant Also Negotiates (RFC 2295): Transparent content negotiation for the request results in a circular reference.
507 Insufficient Storage (WebDAV; RFC 4918): The server is unable to store the representation needed to complete the request.
508 Loop Detected (WebDAV; RFC 5842): The server detected an infinite loop while processing the request (sent in lieu of 208).
509 Bandwidth Limit Exceeded (Apache bw/limited extension): This status code, while used by many servers, is not specified in any RFCs.
510 Not Extended (RFC 2774): Further extensions to the request are required for the server to fulfill it.
511 Network Authentication Required (RFC 6585): The client needs to authenticate to gain network access. Intended for use by intercepting proxies used to control access to the network (e.g. “captive portals” used to require agreement to Terms of Service before granting full Internet access via a Wi-Fi hotspot).
551 Option not supported (RTSP): An option given in the Require or the Proxy-Require fields was not supported. The Unsupported header should be returned stating the option for which there is no support.
598 Network read timeout error (Unknown): This status code is not specified in any RFCs, but is used by Microsoft HTTP proxies to signal a network read timeout behind the proxy to a client in front of the proxy.
599 Network connect timeout error (Unknown): This status code is not specified in any RFCs, but is used by Microsoft HTTP proxies to signal a network connect timeout behind the proxy to a client in front of the proxy.

Revealing HTTP Request and Response

Posted: 13 Mar 2013 01:08 AM PDT

In world wide web service, Hypertext Transfer Protocol (HTTP) is main application protocol used for communication and distributing information. Hypertext is a multi-linear set of objects, building a network by using logical links (thus called as hyperlinks) between the nodes (can be text or words).

A session on HTTP is actually a sequence of network request and response transactions. But what is these two things actually? In this article we will discuss about what request and response in HTTP is.

In this article I will also demonstrate the concept of HTTP request and response using two machine: my Slackware64 as a client and FreeBSD 8.3 with Apache as web server. The IP of client will be 192.168.1.5 and IP Of server will be 192.168.1.3. All scenario use an isolated network to ensure no noise occurred. Thus we only have 2 nodes connected peer to peer.

The Key Concept

HTTP is fall in as Application layer protocol (layer 7 in OSI reference model, or layer 4 in TCP/IP model). The default port is 80, except defined otherwise.

In network, at least there are two nodes communicate. One as the server and other as client. A client ask a request to server and a server must give a response.

HTTP is a stateless protocol, which means each and every connection is independent of each other. The atomic transaction is called as session which consists of one request and replied by one response.

HTTP is built on top of TCP (Transmission Control Protocol) in which established a connection between the server and a client. Despite of built on TCP, the stateless property of HTTP will not persist the connection after response is done given to client.

Peeking to the Network Level

To understand what data goes in and out when transaction is done, we will dive in to lower level.

In this scenario we will initiate connection from client to the server and capture all the traffic on the server that’s coming from the client and see what’s happening. Specifically we will use wget to download a file from web server. Therefore, we need at least two terminal on client: one for requesting using wget, one for tcpdump the network interface.

First, fire up tcpdump so we can get any data in and out from interface can be sniffed.

  root@BlueWyvern:/# tcpdump -i eth0 -s0 -n -A host 192.168.1.3  tcpdump: verbose output suppressed, use -v or -vv for full protocol decode  listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes

Next we use wget to send and receive data.

  root@BlueWyvern:/# wget http://192.168.1.3  --2013-03-13 14:16:54--  http://192.168.1.3/  Connecting to 192.168.1.3:80... connected.  HTTP request sent, awaiting response... 200 OK  Length: 45 1  Saving to: 'index.html'    100%[======================================>] 45          --.-K/s   in 0s    2013-03-13 14:16:54 (8.92 MB/s) - 'index.html' saved [45/45]

Now, let see what happende during the http request. Following is the result when we did connection:

  14:16:54.185814 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [S], seq 3142401191, win 43690, options [mss 65495,sackOK,TS val 2109181 ecr 0,nop,wscale 7], length 0  E..<.Z@.@.*`...........P.M<..........0.........  . ..........  14:16:54.185834 IP 192.168.1.3.80 > 192.168.1.5.48262: Flags [S.], seq 618379695, ack 3142401192, win 43690, options [mss 65495,sackOK,TS val 2109181 ecr 2109181,nop,wscale 7], length 0  E..<..@.@.<..........P..$....M<......0.........  . ... ......  14:16:54.185856 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [.], ack 1, win 342, options [nop,nop,TS val 2109181 ecr 2109181], length 0  E..4.[@.@.*g...........P.M<.$......V.(.....  . ... ..  14:16:54.185913 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [P.], seq 1:108, ack 1, win 342, options [nop,nop,TS val 2109181 ecr 2109181], length 107  E....\@.@.)............P.M<.$......V.......  . ... ..GET / HTTP/1.1  User-Agent: Wget/1.14 (linux-gnu)  Accept: */*  Host: 127.0.0.1  Connection: Keep-Alive    14:16:54.185948 IP 192.168.1.3.80 > 192.168.1.5.48262: Flags [.], ack 108, win 342, options [nop,nop,TS val 2109181 ecr 2109181], length 0  E..4.M@.@..t.........P..$....M=....V.(.....  . ... ..  14:16:54.225066 IP 192.168.1.3.80 > 192.168.1.5.48262: Flags [P.], seq 1:336, ack 108, win 342, options [nop,nop,TS val 2109220 ecr 2109181], length 335  E....N@.@..$.........P..$....M=....V.w.....  . /$. ..HTTP/1.1 200 OK  Date: Wed, 13 Mar 2013 07:16:54 GMT  Server: Apache/2.4.3 (Unix) PHP/5.4.7  Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT  ETag: "2d-432a5e4a73a80"  Accept-Ranges: bytes  Content-Length: 45  Keep-Alive: timeout=5, max=100  Connection: Keep-Alive  Content-Type: text/html    <html><body><h1>It works!</h1></body></html>    14:16:54.225093 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [.], ack 336, win 350, options [nop,nop,TS val 2109220 ecr 2109220], length 0  E..4.]@.@.*e...........P.M=.$......^.(.....  . /$. /$  14:16:54.225897 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [F.], seq 108, ack 336, win 350, options [nop,nop,TS val 2109221 ecr 2109220], length 0  E..4.^@.@.*d...........P.M=.$......^.(.....  . /%. /$  14:16:54.252701 IP 192.168.1.3.80 > 192.168.1.5.48262: Flags [F.], seq 336, ack 109, win 342, options [nop,nop,TS val 2109247 ecr 2109221], length 0  E..4.O@.@..r.........P..$....M=....V.(.....  . /?. /%  14:16:54.252738 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [.], ack 337, win 350, options [nop,nop,TS val 2109247 ecr 2109247], length 0  E..4._@.@.*c...........P.M=.$......^.(.....  . /?. /?

Well, lot of text we’ve got for a simple request and response. Let’s examine it in detail. On that scenario, we will divide to three stage for understand it easier.

Stage 1: Establishing TCP Connection

Like all of use know that on TCP based connection, we must establish a connection before communicate. This stage is called as three-way handshake. This “handshake” is done from client to server with three phase. Now if we examine closely, the first three connection is the handshake done by client and server. As seen here:

14:16:54.185814 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [S], seq 3142401191, win 43690, options [mss 65495,sackOK,TS val 2109181 ecr 0,nop,wscale 7], length 0
E..<.Z@.@.*`………..P.M<……….0………
. ……….
14:16:54.185834 IP 192.168.1.3.80 > 192.168.1.5.48262: Flags [S.], seq 618379695, ack 3142401192, win 43690, options [mss 65495,sackOK,TS val 2109181 ecr 2109181,nop,wscale 7], length 0
E..<..@.@.<……….P..$….M<……0………
. … ……
14:16:54.185856 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [.], ack 1, win 342, options [nop,nop,TS val 2109181 ecr 2109181], length 0
E..4.[@.@.*g...........P.M<.$......V.(.....

This article won't cover three way handshake in detail. I assume you have known what and how three way handshake done.

At this stage we have established a connection between client and server. At the same point, client can now send a request to server. Therefore, let's go to second stage.

Stage 2: Client Initiating HTTP GET Request

Next on our example, client will give a request to server. Client initiate a HTTP "GET" request.

GET is one of request defined for HTTP and used mostly on web. This request are used by clients for retrieving data from the server. If we look closely, it reveals some informations as below:

What resource is requested by client. In our scenario, it's a simple "/". GET "/" , which tell the server to retrieve the root directory (the default page of the website).
What version of http is used. In our case its HTTP/1.1
What agent used by client. In our case, we got information that the client use wget. More specific: Wget/1.14 (linux-gnu)
What type of data client ready to accept. HTTP allows all MIME media types. In this scenario, the client is ready to accept any kind of data hence it is */*
HOST field tells the hostname of the server, from which client is requesting the data.
Most of clients request a keep-alive type of connection from the server. Keep alive is used to keep the tcp connection made by the client, so that the overhead of creating a tcp connection is reduced for subsequent requests. Despite of that, it is server who made the decision to keep TCP connection active or not. Note that although a keep alive connection made, HTTP is still a stateless protocol.

All can be seen here:

  14:16:54.185913 IP 192.168.1.5.48262 > 192.168.1.3.80: Flags [P.], seq 1:108, ack 1, win 342, options [nop,nop,TS val 2109181 ecr 2109181], length 107  E….\@.@.)…………P.M<.$……V…….  . … ..GET / HTTP/1.1  User-Agent: Wget/1.14 (linux-gnu)  Accept: */*  Host: 127.0.0.1  Connection: Keep-Alive

Stage 3: Server Reply with HTTP Response

On getting the GET request from client, the server must responds, by revealing some information about itself and metadata about the data asked by the client along with the data.

The server sends a status code (this informs client software about the result of the request). There are some common status code. Discussing all the status code is beyond the scope of this article thus we will limit ourselves to key point and discuss the rest on another article. The server replies with status code 200 and the http version it is using. In our case it is HTTP /1.1.
We also got the date and time which response was originated.
Server field tells us what server application is used. In this case is Apache.
Last modified field tells us the modified date and time of the data requested.
Etag is a string identifies the modification of the data requested. Mostly used for caching and improves the quality of web caching.
Another important factor that makes http a wonderful protocol is that we can request your required bytes of a resource. For example we can download a file of 100M and in between the connection dropped we can later resume the download by specifying the range of bytes from where to start the download in the GET request. This method also used as foundation of download accelerator / manager like IDM does.
The Content-Length field specifies the size of the resource requested in bytes. In our case it's familiar Apache word: It works! and some formatting which takes 45 bytes.
Connection: keep-alive. This conclude that our keep-alive request is granted by server. On some situation, where our keep-alive wish is not granted, we will get Connection: close.
Content type specifies the type of the content. This file is simply a text with HTML format, which is Text/HTML.
Last is the data after these headers.

We can see the actual packet from tcpdump output:

  14:16:54.225066 IP 192.168.1.3.80 > 192.168.1.5.48262: Flags [P.], seq 1:336, ack 108, win 342, options [nop,nop,TS val 2109220 ecr 2109181], length 335  E....N@.@..$.........P..$....M=....V.w.....  . /$. ..HTTP/1.1 200 OK  Date: Wed, 13 Mar 2013 07:16:54 GMT  Server: Apache/2.4.3 (Unix) PHP/5.4.7  Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT  ETag: "2d-432a5e4a73a80"  Accept-Ranges: bytes  Content-Length: 45  Keep-Alive: timeout=5, max=100  Connection: Keep-Alive  Content-Type: text/html    <html><body><h1>It works!</h1></body></html>

Client will give ACK to server and later after no more data transmitted, the connection will be terminated.

The HTTP Request Types

On above scenario, we only discuss about GET request type. In this section, we will get in touch with other requests:

HTTP Head

Similar to HTTP GET request. It is the easiest method to know the complete details of the resource available on a particular URL, without downloading the entire data.

In our scenario, if we use HEAD request we wil get all the header's in the response without getting "<html><body><h1>It works!</h1></body></html>" message.

Mostly used to retrieve attributes / metada regardless of data and can save much of bandwidth if the data is big.

HTTP Post

Mostly used to send data from client to server. Following is the example of HTTP post request from client to server.

  POST /receiver.php HTTP/1.1  Host: 192.168.1.3  User-Agent: ELinks/0.11.1 (textmode; Linux; 80x25-2)  Referer: http://192.168.1.3/  Accept: */*  Accept-Encoding: gzip  Accept-Language: en  Connection: Keep-Alive  Content-Type: application/x-www-form-urlencoded  Content-Length: 62  name=sarath&last=pillai&email=&telephone=&comments=

As seen on above snippet, the request is being send to "receiver.php". This data being send to the server will ocntain other header's as well that we saw during our GET request example.

The last line sends the exact data to the server.

HTTP Put

Similar to the post request. PUT request sends or creates a resource in the specified URI. If the resource is already present in that specified URI, server then update the URI, otherwise the resource will be created.

HTTP Delete

Request server to delete a specific resource on a specified URI.

Well, it's not advisable to configure a webserver for HTTP delete operation. However, if such functionality is desired we can use a HTTP Post operation using a web form which intern will delete a resource.

HTTP Trace

Used to troubleshoot HTTP webpages.

On simple case, if suppose a web page is not getting loaded the way we want in browser. We can then trace request is used to retrieve the complete request that the server got from the client back to the client itself. It is much like see what command you have send to server.

This request is mostly disabled in most of the web server's. The reason is simple, this operation is similar to viewing web server log of the request we send.

Famous Response

Now, let's discuss about some widely known HTTP status code. The list below is some status code often occurred on transaction. For a complete list, you can read it on this article.

400: Bad Request Type. The respond occured when the server recieves malformed request of any kind.
403: Forbidden Access. This response means we can get the resource as the resource has limited access and we don't have the access.
404: The most famous error code. Not Found. This error indicates that the resource is not found on server side.
500: An unknown error occurred on web server. This error inform the client that some internal error happen on server side but the error is not specified or not a specific problem is shown by web server.

QEMU on Windows

Posted: 12 Mar 2013 08:59 AM PDT

QEMU is an emulator for various CPUs. QEMU can emulate Operating System like Linux, Windows, FreeBSD, etc such as Virtual Box and VMWare do. But unlike other emulator, QEMU can also emulate machine. Hence you can get an ARM “run” on top of your machine.

When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (our machine) by using dynamic translation with very good performance. When used as a virtualizer, QEMU achieves near native performance by executing the guest code directly on the host CPU.

On previous article we have discuss many things on QEMU such as: Installing Slackware on QEMU and also Installing QEMU on Slackware64. In this article we will discuss something different. We will discuss about how to install QEMU on Windows using Windows 8 64-bit. Although I use Windows 8 64 bit as example, you can also build QEMU on your 32-bit Windows machine.

Obtain the Materials

QEMU binaries for Windows is officially not supported and not maintained by QEMU. To obtain QEMU for Windows we can get the unofficial binaries from some contributors. Here we have two provider: Eric Lassauge and Takeda Toshiya. Another QEMU provider can be found here. Thanks to you for providing the binaries!

In this article I will use QEMU 1.3.1 which is provided by Eric Lassauge. You can do direct download here.

Installation

Well, there is no special treatment for QEMU installation. First, extract the archive (in this case: Qemu-1.3.1-windows.zip). You will get a directory Qemu-1.3.1-windows and some binaries inside. Rename the folder to qemu.

Move qemu folder to C: thus you will get C:\qemu now.

Now open Control Panel. Now choose Edit the system environment variables. If you can’t find it, enter Control Panel\System and Security\System to address bar and choose Advanced system settings. Click on Environment Variables button and edit path. Add following: “;C:\qemu” without quote “.

At this point you already have qemu on your system (easy, right?).

Proyek Teknologi

Rabu, 13 Maret 2013