my digital life by gregory tomlinson
Found an awesome post by Tim Linden on backing up MySQL to Amazon s3. It took me awhile to find what I needed, but once I found Tim’s post it took me 10 minutes to get it running. The only gotcha was it should be:
# /usr/sbin/backup-manager --verboseThanks Tim!
Need to run both php4 and php5 on your Mac? I did – so I used Mamp. It’s awesome! There is even a great little tutorial on installing Wordpress on Mamp here. There is a very simple toggle for running php4 or running php5 built into Mamp. I am also using the native php5 build that ships with Mac OSX 10.5.
Getting started with Apache Mod Deflate was fairly simple. Like mod concat from the first piece in the Blog Optimization series, it’s just a matter of compiling and installing mod deflate. Mod deflate comes with Apache 2.2, the first step is locating it.
prompt:$ find / -name mod_deflate.c # change directory (cd) to the location of mod_deflate.c prompt:$ apxs -i -a -c mod_deflate.c prompt:$ apachectl configtest # make sure the syntax is okay prompt:$ apachectl graceful # restart the server
Now that mod_deflate is enabled, it’s time to configure it. I couldn’t find much information during the research phase on configuring the module. I knew from previous experience that I wanted to gzip, or deflate, all static files. Eventually, I found this post on configuring mod_deflate with cPanel. It was helpful, I modified it slightly to include a few other file types
<Location /> AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css application/x-javascript application/x-shockwave-flash </Location>
The Apache 2.2 docs indicate the AddOutputFilterByType directive is deprecated and mod_filter should be used instead, I haven’t had time to play around with mod filter and the Apache 2.2 docs urge caution since it’s not fully tested. If anyone has experience with mod filter, please share in the comments section below.
Mod deflate has done wonders to reduce my bandwidth costs and site performance. On average each request uses 25% less bandwidth than without mod_deflate. Obviously images don’t benefit from gzip compression, but all JavaScript, CSS, HTML and flash files are benefiting by being about 50% smaller in size. For instance, JQuery is about 30.3kb on the server. When it’s delivered using mod_deflate it’s 16kb according to Firebug.
Have a look at some before and after numbers from my server logs.
| Day | Number of visits |
Pages | Hits | Bandwidth |
|---|---|---|---|---|
| 26 Dec | 761 | 3966 | 12131 | 242.29 MB |
| 31 Dec | 761 | 4161 | 12809 | 196.94 MB |
Do you have any blog optimization techniques you are using on your site? Check out my related posts on Blog Bloat and using Mod concat if you interested in optimizing your blog or website.
Following my post on Blog Bloat, I have finally gotten around to installing mod_concat. I asked my hosting provider ServInt to handle the upgrade to Apache 2.2.x, they did it in about 20 mins – thanks guys, you rock!
The actual installation of mod_concat was fairly simple thanks to the included instructions, though it lacked a few of the finer points, like needing to install the module into the Apache modules directory and modifying httpd.conf to include mod_concat, this is easily handled by adding the -i and -a flag. First grab the project out of SVN, then you can compile it.
prompt:$ svn checkout http://modconcat.googlecode.com/svn/trunk/ modconcat-read-only prompt:$ cd modconcat-read-only/mod_concat prompt:$ apxs -i -a -c mod_concat.c #Make sure the config file is working okay prompt:$ apachectl configtest #Then it's safe to restart and use the concat module prompt:$ apachectl -k graceful
That’s all it takes to get the module working with Apache 2.2.x. I setup a super simple test here to ensure mod_concat is working.
Optimizing WordPress K2 using mod_concat
Getting this Apache module working with WordPress and K2 wasn’t quite as simple as I was hoping. I ended up having to do more customization work than I desired. The issue with K2 is that many of the included JavaScript files, for handling the rolling archives and live search, have a .php extension that includes a gzip directive header. Additionally, the K2 files are nested so deep that mod_concat can’t handle it.
To deal with these short comings, I sourced the K2 JavaScript files and moved them to a root directory. Currently, the files to be concatenated are hard coded into the plugin. Perhaps a future revision will resolve this, however, I am more interested in testing out mod_jsmin and mod_deflate before I get around to this enhancement.
Here is the plugin code I wrote in order to use mod_concat with K2.
<?php /* Plugin Name: WP Mod Concat Description: This plugin leverage the Apache 2.2.x module mod_concat to combine a list of files into a single file Version: 0.1 Author: Gregory Tomlinson Author URI: http://gregorytomlinson.com/encoded/ */ remove_action('wp_head', 'wp_print_scripts'); add_action( 'wp_head', 'wp_mod_concat_optimzie' ); function wp_mod_concat_optimzie( $handles = false ) { $base_url = "http://gregorytomlinson.com/js/??"; $script_files = array("jquery_1_2_6.js", "k2.functions.js", "k2.livesearch.js", "k2.rollingarchives.js", "k2.slider.js", "k2.trimmer.js", "swfobject.js" ); echo '<script type="text/javascript" src="'. $base_url . join(",", $script_files) . '"></script>'; } ?>
By using remove_action(’wp_head’, ‘wp_print_scripts’);, I can stop WordPress from loading JavaScript files. I then replace the head script files with the concatenated version leveraging mod_concat, which produces a script tag that looks like the following.
<script type="text/javascript" src="http://gregorytomlinson.com/js/??jquery_1_2_6.js,k2.functions.js,k2.livesearch.js,k2.rollingarchives.js,k2.slider.js,k2.trimmer.js,swfobject.js"></script>You can learn more about mod_concat here.
The other drawback to mod_concat is it’s incapable of handling query parameters. I would like to be able to append a ?ver=1.1 in order to handle busting the browser cache, but the current version isn’t able to do this, maybe Ian can fix this 
That’s all it takes to use mod_concat. It’s probably not the ideal method to use with WordPress and K2, you may consider using PHP Speedy. Dave Artz runs down a list of other concatenation methods in this blog post.
A little while back, I posted a basic install of ffmpeg for CentOS 5. After working with that build over the last month, I found I needed to expand it to include many different video codecs. Because of licensing restrictions, binaries of the build cannot be distributed. However, instructions for a similar build can! I spent several days (or more) researching the packages I would need and I must have combed dozens of blogs to find this information. The build I made strips sound from videos [See update at the end of this post for more information on this subject], this decreases the final file size. If you need sound, you will not want to follow these steps exactly. I also stripped ffserver and ffplay from my build.
These packages were executed in the following order. I cannot say for certain this exact order must be followed, it was the order in which ffmpeg threw the errors. This tutorial assumes you have wget, bzip2, tar and subversion installed on your system. If you do not, please find and install the lastest versions of these utilities for CentOS 5. Additionally, I have allowed all packages to install into their default directories, typically this is /usr/local/[bin | lib]. If at any point during the process of a ‘make’ you run into errors, be sure and run ‘make clean’ before running ‘make’ again. You will either need root access of su access to install ffmpeg. The marker ‘codec:$’ is my prompt, it is merely to indicate separate commands.
Getting Started
codec:$ codec:$ mkdir -p ./tmp/ffmpeg-packages codec:$ cd ./tmp/ffmpeg-packages
Installing a52
codec:$ wget http://liba52.sourceforge.net/files/a52dec-0.7.4.tar.gz codec:$ tar -zxf a52dec-0.7.4.tar.gz codec:$ cd a52dec-0.7.4 codec:$ ./configure --enable-shared=PKGS codec:$ make && make install codec:$ cd ..
Installing FAAD2
codec:$ wget http://downloads.sourceforge.net/faac/faad2-2.6.1.tar.gz codec:$ tar zxf faad2-2.6.1.tar.gz codec:$ cd faad2 codec:$ autoreconf -vif codec:$ ./configure --disable-drm --disable-mpeg4ip codec:$ make && make install codec:$ cd ..
Installing FAAC
codec:$ wget http://downloads.sourceforge.net/faac/faac-1.26.tar.gz codec:$ tar zxfv faac-1.26.tar.gz codec:$ cd faac codec:$ ./bootstrap codec:$ ./configure --disable-mp4v2 codec:$ make && make install codec:$ cd ..
Installing LAME
codec:$ wget \ http://superb-east.dl.sourceforge.net/sourceforge/lame/\ lame-3.98b8.tar.gz codec:$ tar zxfv lame-3.98b8.tar.gz codec:$ cd lame-3.98b8 codec:$ ./configure codec:$ make && make install codec:$ cd ..
Installing yasm
YASM is a modular assembler, it is required by the x264 package.
codec:$ wget \ http://www.tortall.net/projects/yasm/releases/yasm-0.7.0.tar.gz codec:$ tar zfvx yasm-0.7.0.tar.gz codec:$ cd yasm-0.7.0 codec:$ ./configure codec:$ make && make install codec:$ cd ..
Installing x264
The x264 package is under git revision control, which is much like CVS or SVN. Thankfully, they provide daily tarballs. I grabbed this one:
codec:$ wget \ ftp://ftp.videolan.org:21//pub/videolan/x264/snapshots/\ x264-snapshot-20080513-2245.tar.bz2
I just did a little exploring via an FTP program to find the snapshot I was after. If you are feeling adventurous, download git and try checking out the latest version from their repository.
codec:$ bzip2 -d x264-snapshot-20080513-2245.tar.bz2 codec:$ tar xfv x264-snapshot-20080513-2245.tar codec:$ cd x264-snapshot-20080513-2245 codec:$ ./configure --enable-mp4-output --enable-shared --enable-pthread codec:$ make && make install codec:$ cd ..
Installing Xvid
codec:$ wget http://downloads.xvid.org/downloads/xvidcore-1.1.3.tar.gz codec:$ tar zxfv xvidcore-1.1.3.tar.gz codec:$ cd xvidcore-1.1.3/build/generic codec:$ ./configure codec:$ make && make install codec:$ cd ../../..
Installing libraw1394
codec:$ wget http://www.linux1394.org/dl/libraw1394-1.3.0.tar.gz codec:$ tar zxfv libraw1394-1.3.0.tar.gz codec:$ cd libraw1394-1.3.0 codec:$ ./configure codec:$ make dev codec:$ make && make install codec:$ cd ..
Installing libdc1394
This project requires libraw1394, you must build it first. This project is a little confusing there is both a libdc1394 and a libdc1394-2. I have only installed the former. Visit Sourceforge here. I grabbed the tarball here:
codec:$ wget \ http://superb-west.dl.sourceforge.net/sourceforge/libdc1394/\ libdc1394-1.2.2.tar.gz codec:$ tar zxfv libdc1394-1.2.2.tar.gz codec:$ cd libdc1394-1.2.2 codec:$ ./configure codec:$ make && make install codec:$ cd ..
Installing ffmpeg
For FFMPEG, you will need to get the latest out of SVN. FFMPEG doesn’t ever make releases. To do so, run:
codec:$ svn checkout svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg codec:$ cd ffmpeg
codec:$ ./configure --enable-gpl --enable-postproc --enable-nonfree --enable-postproc --enable-libfaad --enable-swscale --enable-avfilter --enable-pthreads --enable-libxvid --enable-libx264 --enable-libmp3lame --enable-libdc1394 --enable-liba52 --enable-libfaac --disable-ffserver --disable-ffplay codec:$ make codec:$ make install
It usually takes at least 5 mins for ‘make’ to run, be sure you have a good file before attempting to install. You will need to be sure and run the ldconfig setting if you have to re-compile, otherwise ffmpeg will throw an error that it cannot find library files. Here is a good resource for using ffmpeg.
Post-Installation Actions
After the install in complete, you may need to add the /usr/local/lib directory to your ld.so.config file. Do the following:
codec:$ cd /etc/ codec:$ cd ld.so.conf.d codec:$ vi ffmpeg.conf
You need to add ‘/usr/local/lib’ [sans-quotes] to this file and save it. Press Esc, :, x to do so in vi. Afterwords you need to run the following from the commandline
codec:$ ldconfig
That’s it, you now have a fairly robust build of ffmpeg that can take almost anything you can throw at it. Sadly, licensing restrictions prevent binaries from being made of the above steps. Happy (video) encoding!
Update: This build doesn’t actually strip sound from videos, I am doing that with a flag when I run ffmpeg, however, it should be noted that all the audio codecs you may find in many common video formats may not be supported in this build of ffmpeg, which is why I am choosing to strip audio entirely from videos.
I recently had the need to install FfMpeg for a project I am working on. After Googling for some time I learned a little bit about the ffmpeg project. The most interesting thing I learned was their are no formal releases of the project. So I just downloaded it from SVN using the following command:
svn export svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
Then I just ran
./configure
In the directory I downloaded ffmpeg too, and finally
make && make install
That’s it, it was up and running in a few minutes. I have added this here because so many of the blogs with information regarding Ffmpeg and CentOS 5 were out of date or wrong. It also helps that my version of CentOS already had gcc and gcc-c++ installed, both of which are needed to compile these from their sources.
Update:
I have written a tutorial for a much more robust install of ffmpeg. Read: A Robust ffmpeg Install for CentOS 5.
Just wanted to make mention of Servint, a dedicated server and VPS provider that I have been working with since January 2007. I continue to be impressed by their customer service and professionalism. So far I am happy to recommend them to others.
