Squid Web Proxy Cache Server

Proxy server’s web caching function

Proxy location in a network

Froxy different functionsProxy cachingNetwork Address TranslationFiltering

Benefits of proxy caching

NAT Function

Packet filter and Proxy Server

System RequirementsDisk random seek timeFor a proxy cache, make sure this number is as low as possible. The problem is that operating systems try to speed up disk access times using various methods that usually slow the system’s performance Amount of system memoryRAM is also extremely important when using a proxy cache. Squid keeps an in-memory table of its objects in RAM, which should always remain in RAM. If part of the table goes to swap, the performance of Squid is greatly degraded.

Download and Install The Squid PackageDownload the latest stable version of Squid (www.squid-cache.org) Install the RPM by using the rpm –i command.

Lưu ý khi Cài đặt SquidSau khi cài đặt lại Squid là một chương trình thay vì là một dịch vụ.Trước khai cài đặt tạo phân vùng /cacheChạy dòng lệnh trong terminal (phải có quyền root) # useradd -d /cache/ -r -s /dev/null squidGiải nén gói cài đặt squid-2.4.STABLE1-src.tar.gz# tar xzpf squid-2.4.STABLE1-src.tar.gz

Lưu ý khi Cài đặt SquidDi chuyển đến thư mục vừa giải nén của Squid và cấu hình squid bật chức năng delay pools trước khi cài đặt./configure --prefix=/opt/squid --exec-prefix=/opt/squid --enable-delay-pools --enable-cache-digests --enable-poll --disable-ident-lookups --enable-truncate --enable-removal-policies# make all# make install

Squid: Starting and stopping# /etc/init.d/squid stop# /etc/init.d/squid start# /etc/init.d/squid restart# /etc/init.d/squid reload

Squid: LogFiles/var/log/squid/cache.logContains run-time status messages, warnings, and errors./var/log/squid/access.logOne line for each client request, including URL, bytes trans-ferred, status code, and more./var/log/squid/store.logTransaction log for objects that enter and leave the cache.Open a new terminal window and run:$ tail -f /var/log/squid/cache.logOpen another new terminal window and run:$ tail -f /var/log/squid/access.log

Configuring: Cache DisksThe cache dir directive(s) tell Squid how and where to store cached objects.cache_dir type path megabytes L1 L2cache_dirufs /var/spool/squid 100 16 256The default typeis ufs, but aufs has better performance on Linux.pathcan be anywhere on the filesystem, but is usually a dedicated disk or partition.megabytesis an upper limit on how much space Squid should use for this cachedir. It should be less than 90% of the actual capacity.L1and L2specify the number of first- and second-level directories to use. Use 16 and 256 by default. These should not be changed after Squid has placed objects on the disk.

The cache directory structure for ufs-based storage schemes

Squid: Create Swap DirectoriesAfter adding a cache dir , you need to initialize it with this command:# squid -z2006/10/12 09:48:24| Creating Swap DirectoriesOwnership and permissions are a common problem at this stage. Squid runs under a certain user ID, specified with cache_effective_user in squid.conf. This user ID must have read and write permission under each cache_dir directory. If not, you'll see a message like this:Creating Swap Directories FATAL: Failed to make swap directory /usr/local/squid/var/cache/00: (13) Permission denied In this case, you should make sure that all components of /usr/local/squid/var/cache are accessible to the user ID given in squid.conf. The final component—the cache directory—must be writable by this user ID as well.

Check Your Configuration File for Errors Before trying to start Squid, you should verify that your squid.conf file makes sense. This is easy to do. Just run the following command:# squid -k parseIf you see no output, the configuration file is valid, and you can proceed to the next step.However, if your configuration file contains an error, Squid tells you about it:squid.conf line 62: http_access allow okay2aclParseAccessLine: ACL name 'okay2' not found.Here you can see that the http_access directive on line 62 references an ACL that doesn't exist. Sometimes the error messages are less informative:FATAL: Bungled squid.conf line 76: memory_poolsIn this case, we forgot to put either on or off after the memory_pools directive on line 76.

Configuring: User IDUnfortunately, running Squid isn't always so simple. In some cases, you may need to start Squid as root, depending on your configuration. For example, only root can bind a TCP socket to privileged ports like port 80. If you need to start Squid as root, you must set the cache_effective_user directive. It tells Squid which user to become after performing the tasks that require special privileges. For example:cache_effective_user squid If you start Squid as root without setting cache_effective_user, Squid uses nobody as the default value. Whatever user ID you choose for Squid, make sure it has read access to the files installed in $prefix/etc, $prefix/libexec, and $prefix/share. The user ID must also have write access to the log files and cache directory.

Configuring: Port Numbers The http_port directive tells Squid which port number to listen on for HTTP requests. The default is port 3128:http_port 3128Youcan instruct Squid to listen on multiple ports with additional http_port lines.For example, the browsers from one department may be sending requests to port 3128, while another department uses port 8080. Simply list both port numbers as follows:http_port 3128http_port 8080You can also use the http_port directive to make Squid listen on specific interface addresses, simply put the IP address in front of the port number:http_port 192.168.1.1:3128

Configuring: Visible Hostname Squid wants to be sure about its hostname for a number of reasons:The hostname appears in Squid's error messages. This helps users identify the source of potential problems.The hostname appears in the HTTP Via header of cache misses that Squid forwards. When the request arrives at the origin server, the Via header contains a list of all proxies involved in the transaction. Squid also uses the Via header to detect forwarding loops. Squid uses internal URLs for certain things, such as the icons for FTP directory listings. When Squid generates an HTML page for an FTP directory, it inserts embedded images for little icons that indicate the type of each file in the directory. The icon URLs contain the cache's hostname so that web browsers request them directly from Squid.Each HTTP reply from Squid includes an X-Cache header. Syntax: visible_hostname squid.hcmuaf.edu.vn

Quid: ACLsACL elements are the building blocks of Squid's access control implementation. These are how you specify things such as IP addresses, port numbers, hostnames, and URL patterns. Each ACL element has a name, which you refer to when writing the access list rules. acl name type value1 value2 ...For example:acl Workstations src 10.0.0.0/16 In most cases, you can list multiple values for one ACL element. You can also have multiple acl lines with the same name. For example, the following two configurations are equivalent:acl Http_ports port 80 8000 8080 acl Http_ports port 80 acl Http_ports port 8000 acl Http_ports port 8080

ACL type: IP AddressUsed by: src, dstSquid has a powerful syntax for specifying IP addresses in ACLs. You can write addresses as subnets, address ranges, and domain names. Squid supports both "dotted quad" and CIDR prefix subnet specifications. In addition, if you omit a netmask, Squid calculates the appropriate netmask for you. For example, each group in the next example are equivalent:acl Foo src 172.16.44.21/255.255.255.255 acl Foo src 172.16.44.21/32 acl Foo src 172.16.44.21 acl Xyz src 172.16.55.32/255.255.255.248 acl Xyz src 172.16.55.32/28 acl Bar src 172.16.66.0/255.255.255.0 acl Bar src 172.16.66.0/24 acl Bar src 172.16.66.0You can also specify hostnames in IP ACLs. acl Squid dst www.squid-cache.org

ACL type: domain nameUsed by: srcdomain, dstdomain, and the cache_host_domain directiveA domain name is simply a DNS name or zone. For example, the following are all valid domain names:www.squid-cache.org, squid-cache.org, orgDomain name matching can be confusing, so let's look at another example so that you really understand it. Here are two slightly different ACLs:acl A dstdomain foo.com acl B dstdomain .foo.com A user's request to get http://www.foo.com/ matches ACL B, but not A. ACL A requires an exact string match, but the leading dot in ACL B is like a wildcard.On the other hand, a user's request to get http://foo.com/ matches both ACLs A and B. Even though there is no word before foo.com in the URL hostname, the leading dot in ACL B still causes a match.

ACL type: Regular expressions Used by: srcdom_regex, dstdom_regex, url_regex, urlpath_regex, browser, referer_regex, ident_regex, proxy_auth_regex, req_mime_type, …A number of ACLs use regular expressions (regex) to match character strings. For Squid, the most commonly used regex features match the beginning and/or end of a string. For example, the ^ character is special because it matches the beginning of a line or string:^http://This regex matches any URL that begins with http://. The $ character is also special because it matches the end of a line or string:.jpg$With all of Squid's regex types, you have the option to use case-insensitive comparison. Matching is case-sensitive by default. To make it case-insensitive, use the -i option after the ACL type. For example:acl Foo url_regex -i ^http://www

ACL types: TCP port numbers Used by: port, myportThis type is relatively straightforward. The values are individual port numbers or port number ranges. Recall that TCP port numbers are 16-bit values and, therefore, must be greater than 0 and less than 65,536. Here are some examples:acl Foo port 123 acl Bar port 1-1024 acl Safe_ports port 443 563

ACL type: time The time ACL allows you to control access based on the time of day and the day of the week. The syntax is somewhat cryptic:acl name [days] [h1:m1-h2:m2] You can specify days of the week, starting and stopping times, or both. Days are specified by the single-letter codes:S:Sunday; M:Monday; T: Tuesday; W: Wednesday; H: Thursday; F: Friday; A: Saturday; D: All weekdays (M-F)Times are specified in 24-hour format. The starting time must be less than the ending time, which makes it awkward to write time ACLs that span "midnights."acl Working_hours MTWHF 08:00-17:00 or: acl Working_hours D 08:00-17:00acl Offpeak1 20:00-23:59 acl Offpeak2 00:00-04:00

Access Control Rules: http_access TagThe http_access tag permits or denies access to Squid. You can allow or deny all requests. You can also allow or deny requests based on a defined access list. If you remove all of the http_access entries, all requests are allowed by default. NOTE: Squid should never be used without some type of authentication system or access control list. You must restrict Internet users from relaying requests through your Web proxy cache. Syntax: http_accessallow|deny[!]aclname [aclname] ...http_access allow Net1 WorkingHours http_access allow Net2 WorkingHours http_access allow Net4 http_access deny All

Squid authentication 1) Create the password file. The name of the password file should be /etc/squid/squid_passwd, and you need to make sure that it's universally readable. # touch /etc/squid/squid_passwd# chmod o+r /etc/squid/squid_passwd2) Use the htpasswd program to add users to the password file. You can add users at anytime without having to restart Squid. In this case, you add a username called www: # htpasswd /etc/squid/squid_passwd wwwNew password:Re-type new password:Adding password for user www3) Find your ncsa_auth file using the locate command. # locate ncsa_auth/usr/lib/squid/ncsa_auth

Squid authentication4) Edit squid.conf; specifically, you need to define the authentication program in squid.conf, which is in this case ncsa_auth. Next, create an ACL named ncsa_users with the REQUIRED keyword that forces Squid to use the NCSA auth_param method you defined previously. Finally, create an http_access entry that allows traffic that matches the ncsa_users ACL entry. Here's a simple user authentication example; the order of the statements is important: #Add this to the auth_param section of squid.confauth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd# Add this to the bottom of the ACL section of squid.confacl ncsa_users proxy_auth REQUIRED# Add this at the top of the http_access section of squid.confhttp_access allow ncsa_users

Squid authentication5) This requires password authentication and allows access only during business hours. Once again, the order of the statements is important: # Add this to the auth_param section of squid.confauth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd# Add this to the bottom of the ACL section of squid.confacl ncsa_users proxy_auth REQUIREDacl business_hours time M T W H F 9:00-17:00# Add this at the top of the http_access section of squid.confhttp_access allow ncsa_users business_hours

Scenarios: Restricting Web Access By Time # Add this to the bottom of the ACL section of squid.confacl home_network src 192.168.1.0/24acl business_hours time M T W H F 9:00-17:00acl RestrictedHost src 192.168.1.23# Add this at the top of the http_access section of squid.confhttp_access deny RestrictedHosthttp_access allow home_network business_hours# Or, you can allow morning access only: # Add this to the bottom of the ACL section of squid.confacl mornings time 08:00-12:00# Add this at the top of the http_access section of squid.confhttp_access allow mornings

Scenarios: Restricting Access to specific Web sites Squid is also capable of reading files containing lists of web sites and/or domains for use in ACLs. In this example we create to lists in files named /usr/local/etc/allowed-sites.squid and /usr/local/etc/restricted-sites.squid. #File:/usr/local/etc/allowed-sites.squid www.openfree.org Linuxhomenetworking.com# File: /usr/local/etc/restricted-sites.squid www.porn.com illegal.com

Scenarios: Restricting Access to specific Web sitesThese can then be used to always block the restricted sites and permit the allowed sites during working hours. This can be illustrated by expanding our previous example slightly. # Add this to the bottom of the ACL section of squid.confacl home_network src 192.168.1.0/24acl business_hours time M T W H F 9:00-17:00acl GoodSites dstdomain "/usr/local/etc/allowed-sites.squid"acl BadSites dstdomain "/usr/local/etc/restricted-sites.squid"# Add this at the top of the http_access section of squid.confhttp_access deny BadSiteshttp_access allow home_network business_hours GoodSites

Configuring SquidThe visible_hostname TagSquid will fail to start if you don't give your server a hostname. You can set this with the "visible_hostname" parameter. visible_hostname bigboyThe http_port TagThe http_port tag configures the HTTP port on which Squid listens for proxy clients. Default port is 3128. We can configure Squid to listen on ports 3128 and 8080 for proxy clients.http_port 3128 8080The Cache_dir TagThe cache_dir tag specifies where the cached data is stored. By default, the following cache_dir tag value is presented:cache_dirufs /var/spool/squid 100 16 256

Defining the Default cache_dir tag

Configuring the acl Tagaclaclnamesrcip-address/netmask ... (clients IP address)aclaclnamesrcaddr1-addr2/netmask ... (range of addresses)aclaclnamedstip-address/netmask ... (URL host's IP address)aclaclnamesrcdomain .foo.com ... reverse lookup, client IPaclaclnamedstdomain .foo.com ... Destination server from URLaclaclnameurl_regex [-i] ^http://… regex matching on whole URLaclaclnameurlpath_regex [-i] .gif$... regex matching on URL path

Configuring the acl Tagaclaclnameport807021aclaclnameport0-1024...ranges allowedaclaclnameprotoHTTPFTP ...aclaclnamemethodGETPOST ...aclaclnametime [day] [h1:m1-h2:m2]day:S - SundayM - MondayT - TuesdayW - WednesdayH - ThursdayF - FridayA - Saturdayh1:m1 must be less than h2:m2aclhome_networksrc192.168.1.0/24aclbusiness_hourstimeM T W H F 9:00-17:00

Recommended minimum configurationacl all src 0.0.0.0/0.0.0.0acl manager proto cache_objectacl localhost src 127.0.0.1/255.255.255.255acl to_localhost dst 127.0.0.0/8acl SSL_ports port 443 563acl Safe_ports port 80 # httpacl Safe_ports port 21 # ftpacl Safe_ports port 443 563 # https, snewsacl Safe_ports port 70 # gopheracl Safe_ports port 210 # waisacl Safe_ports port 1025-65535 # unregistered portsacl Safe_ports port 280 # http-mgmtacl Safe_ports port 488 # gss-httpacl Safe_ports port 591 # filemakeracl Safe_ports port 777 # multiling httpacl CONNECT method CONNECT

The http_access TagThe http_access tag permits or denies access to Squid. You can allow or deny all requests. You can also allow or deny requests based on a defined access list. If you remove all of the http_access entries, all requests are allowed by default. Proxy clients will be unable to use the Squid proxy-caching server until you modify the http_access tags. Please note that some level of access control is recommended, so do not remove all of the http_access tags. NOTE: Squid should never be used without some type of authentication system or access control list. You must restrict Internet users from relaying requests through your Web proxy cache. Syntax:http_accessallow|deny[!]aclname [aclname] ...

Recommended minimum configurationhttp_accessallowmanagerlocalhosthttp_accessdenymanagerhttp_accessdeny!Safe_portshttp_accessdenyCONNECT!SSL_ports# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTSFor example:http_accessallowhome_networkbusiness_hourshttp_accessallowlocalhosthttp_accessdenyall

The icp_port tagThe icp_port tag: Internet Cache Protocol (ICP) : Queries other caches for a specific objecticp_port: The port number where Squid sends and receives ICP queries to and from neighbor caches. Default is 3130. To disable use "0".icp_port 8082The cache_peer tag: To specify other caches in a hierarchy, use the format: cache_peer hostname type http_port icp_port For exampleproxy icp hostname type port port options -------------------- -------- ----- ----- -----------cache_peerproxy2.hcmuaf.edu.vnparent80808082cache_peerproxy.kcntt.hcmuaf.edu.vnsibling80808082Type: ‘parent’ : parent proxy in higher level ‘sibling’: peer proxy

Configuring Proxy Clients (IE)Open Internet Explorer.Click the Tools menu and choose Internet Options. Select the Connections tab, and click LAN Settings. Deselect Automatically Detect Setting. In the Proxy server section, click the Use a proxy server check box. In the Address field, enter the IP address of your Squid Web Proxy Cache server. In the Port field, enter port 8080 Click OK twice to return to the browser. In Internet Explorer, enter the following URL: www.squid-cache.org.The Squid home page will appear. If not, your browser proxy settings are incorrectly configured.

Configuring Proxy Clients (IE)

Forcing Users To Use Your Squid ServerThis is called a "transparent proxy" configuration. It is usually achieved by configuring a firewall between the client PCs and the Internet to redirect all HTTP (TCP port 80) traffic to the Squid server on TCP port 3128 (which is Squid server default TCP port).In both cases below:The firewall is connected to the internet on interface eth0 and to the home network on interface eth1.The firewall is the default gateway for the home network which uses NAT to access the Internet.Only the squid server has access to the internet on port 80 (HTTP). This happens because all HTTP traffic, except that coming from the squid server, is redirected.

Firewall configurationSquid Server And Firewall Are The Same ServerHere all HTTP traffic from the home network is redirected to the firewall itself on the squid port of 3128. iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j REDIRECT --to-ports 3128 iptables -A OUTPUT -j ACCEPT -m state --state NEW -o eth0 -p tcp --dport 80Squid Server And Firewall Are Different ServersHere all HTTP traffic from the home network except from the squid server at IP address 192.168.1.100 is redirected to the Squid server on the squid port of 3128. iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT --to 192.168.1.100:8080 -s ! 192.168.1.100/32 iptables -A OUTPUT -j ACCEPT -m state --state NEW -o eth0 -p tcp --dport 80

SummaryBenefits of Proxy Server Implementation A Web proxy cache server can cache Web pages and FTP files for proxy clients. They can also cache Web sites for load balancing.  Caching increases the performance of the network by decreasing the amount of data transferred from outside of the local network. Web proxy caching reduces bandwidth costs, increases network performance during normal traffic and spikes, performs load balancing, caches aborted requests, and functions even when a network’s Internet connection fails. Differentiating between a Packet Filter and a Proxy Server Packet filters analyze traffic at the Network (Layer 3) and Transport layers (Layer 4) of the OSI model. A packet filter can determine whether it will allow a certain IP address or IP address range to pass through, or filter traffic by service, or port number. A proxy server analyzes packets at the Application layer (Layer 7) of the OSI model. This feature provides flexibility because the traffic within one service, such as port 80 (HTTP) traffic, can be filtered.

SummaryImplementing the Squid Web Proxy Cache Server The Squid Web Proxy Cache server allows administrators to set up a Web proxy caching service, add access controls (rules), and cache DNS lookups.  Client protocols supported by Squid must be sent as a proxy request in HTTP format, and include FTP, HTTP, SSL, WAIS, and Gopher.  Squid is configured using the /etc/squid/squid.conf file, which defines configurations such as the HTTP port number on which Squid listens for HTTP requests, incoming and outgoing requests, timeout information, and firewall access data.  Each configuration option in squid.conf is identified as a tag. The http_port tag configures the HTTP port on which Squid listens for proxy clients. The cache_dir tag specifies where the cached data is stored. The acl tag allows you to define an access list. The http_access tag permits or denies access to Squid. Squid will not function until you make changes to the squid.conf file.