Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
FreedomCenter_-_LAMP_System The Free Network Foundation operates a substantial number of web properties in our MCI facility. We've deployed a very robust/scalable system to support a large scale, read heavy HTTP workload. Contents • 1 Overall web flow • 2 Load balancing and content/code caching ♦ 2.1 Varnish ♦ 2.2 APC ♦ 2.3 Memcached • 3 Apache server ♦ 3.1 Modules in use ♦ 3.2 vhosts • 4 MySQL ♦ 4.1 Caching ♦ 4.2 Databases ♦ 4.3 Replication Overall web flow The overall web flow, depicted in the diagram below, begins when web traffic hits our network edge. It flows from our edge router (wan01), through layer two switching (sw01) to the machine that hosts our virtualized computer environment (vm01). At this point, it flows into a virtual machine called lamplb (for load balancer), which caches and loadbalances. Details of this configuration, are described in the section 'Varnish', below. Varnish provides a cache for all static HTTP objects (html/css/js) from those virtual machines that host the applications themselves. The majority of applications are able to share a VM called 'lamppublic', which does some further caching, described below. Applications that require a specialized environment (Ruby on Rails for Chili, for instance) are hosted on their own machine. A unified, optimized, replicated and backed up sql server, described in the 'MySQL' section, backs all services. Storage for all VMs is served from the storage array (stor01), which is documented in FreedomCenter_-_Storage. Load balancing and content/code caching Varnish We use varnish for caching all of our HTTP content and routing traffic to back end servers. It lives on lamplb.corp.thefnf.net (10.250.4.38) and it's configuration can be found in /etc/varnish/default.vcl Here is a configuration snippet: #Define various real servers... backend default { .host = "10.250.4.40"; .port = "80"; } Contents 1 FreedomCenter_-_LAMP_System backend chili { .host = "10.250.4.32"; .port = "80"; } <snipped> sub vcl_recv { if (req.http.host == "chili.freenetworkfoundation.org") { set req.backend = chili; return (lookup); } if (req.http.host == "learn.freenetworkfoundation.org") { set req.backend = canvas; return (lookup); } <snipped more of the same> then the default else { set req.backend = default; } } If you need to add a new web property that isn't on the lamppubic system, just create a new backend and update the vcl_recv stanza. At this time, it appears that a web front end for Varnish may be a proprietary add on. One can use varnishtop / varnishstat if they need information on varnish performance. APC We use APC on lamppublic. APC is a PHP object cache/compiler. Provided that you are on the VPN, you can view cache statistics at http://lamppublic.corp.thefnf.net/apc Memcached Wordpress/Mediawiki are configured to use memcached, which lives on the lamplb box. You can hit that via telnet and gather stats: [1] root@lamplb> echo "stats" | nc 10.250.4.38 11211 ~ [1] root@lamplb> echo "stats" | nc 10.250.4.38 11211 | grep bytes_ STAT bytes_read 6911958 STAT bytes_written 5470422 ^C [130] root@lamplb> Varnish 2 FreedomCenter_-_LAMP_System Apache server We use Apache for all our production sites. It's a stock Debian package configuration, except for enabling a few modules (listed below). Modules in use • mod_rewrite • mod_pagespeed gpl module from Google which implements the same things server wide that w3tc/minify does for just wordpress vhosts Here is a list of our current virtual hosts: root@lamppublic> date && ls /etc/apache2/sites-enabled|grep -v legacy Wed Sep 26 16:32:52 CDT 2012 apc bookmarks commons credman crm dispatch documents fnm forum frontpage hr incbits incbits-test kb legwatch mblog meetings owa pastebin photos prodmanuals racktable sharing snetinsight social staticbits support ushadi video webstats xibo root@lamppublic> ~ (grep -v legacy is to remove our vhosts for redirecting from freenetworkfoundation.org to thefnf.org). MySQL Apache server 3 FreedomCenter_-_LAMP_System Caching Databases Replication FreedomCenter Documentation (edit) High-level: Landing Page - Overview - System - Migration Sites: MCI - ATX - DFW Subsystems: Network - Compute - Storage - Out of Band Functions: LAMP - DNS - Monitoring - SSL - LDAP Catalogs: Public Services - Administrative Services - Walkthroughs Caching 4