Old dog, new tricks

High performance websites

Google guru Matt Cutts has recently reviewed the year at Google and focused quite a bit time on performance, noting that the speed of a web site may become a factor in search results ranking in the future.

So I took a look at our busiest website using Yahoo's YSlow and was dismayed - but not terribly surprised - to get a 'D' rating for one of the most complex pages.  The result for the overall speed has probably got a lot to do with the number of complex SQL queries but that wasn't what was bothering YSlow.  So I spent a fun afternoon making these fairly simple changes.

Where to start?  Yahoo's "Best Practices for Speeding Up Your Web Site" starts with Minimize HTTP Requests so let's go with that.

1. Combining and Minifying CSS

Google has a great tool called Minify which sits on your server and gathers up your CSS files and serves then minified, compressed and with cache control headers.  It's a PHP app which you just download and slap on your server.  Minimal setup is required.

This site is ten years old and now has six CSS files, including two that were only for print.  They aren't big but that's not the problem.  The problem is that they make multiple HTTP calls and that slows everything down.  A compliant web browser only asks for two things at a time so it will wait while each pair arrive.  In tests, using www.webpagetest.org, I trimmed off about 0.3s (about 25%) of the time taken to start rendering the page.  Why was I worried about the time taken to start rendering, rather than completion?  Because that's what user's notice.  Once the page has started to arrive, they feel rewarded.  As long as the rest of the page arrives with out undue delay, yhey'll be happy.

The first thing I did was remove the need for the Print CSS files by moving their content into their Screen equivalents and wrapping it with @media print { }.  The @media instruction didn't work in Netscape 4 and it looks like parts of this site date back to when Netscape 4 mattered.  So then there were four.

The next step was to let Minify loose on the remaining four.  By editing the groupsConfig.php file in Minify, you can define groups of CSS files.  Version control is useful here as it will allow me to override the cache control I'm about to impose.  So I set up groups:

    'west-one-v01' => array( '//_css/westwing.css', 

    '//_css/firstfloor.css', 

    '//_css/nifty/niftyCorners.css',

    '//_css/main.css' ),

Note my old friend Nifty is still in use (but probably not for much longer).  I have called my group 'west-one' but have added 'v01' to the group name.  If I need to override the cache control, I can create an identical group with a new name 'west-one-v02' and link to that instead.

Then I wrote a PHP function to generate the link:

    <link type="text/css" rel="stylesheet" href="/min/g=west-one-v01" />

It's easy to change that function if I need to in the future - easier than changing a Dreamweaver page template.

2. Combining and Minifying Javascript

Minify will do the same job on Javascript.  I don't really remember what half this Javascript does or I would feed it through Google's Closure compiler.  That would hugely improve it but there's not much of it so I'm going to skip ahead.

As with the CSS files, I can create a Minify group that covers most of them.  However, I also use Mint and I can't do anything about the link to its Javascript.  Mint is/was a great package and well worth every cent of its $30 price but Google Analytics must be very hard for the excellent Shaun Inman to compete with. I'll keep using it because it's one-page summary of a website is still better than paging through GA.

So I've left Mint where it is.  Maybe it could be moved from the header to the footer which would improve the time the page takes to start rendering but I'm not certain the gain is worth it.  I'll ponder on that.  I hope Shaun will to.

3. Cache Control

So far, so good.  YSlow likes what I've done and now gives me Grade A on "Make fewer HTTP requests" but is still giving me an 'E' for Add Expires headers.  

I'll do that it two ways.  Firstly I can edit Minify's config.php and tell it to put 30 days on the CSS and Javascript.  Actually yahoo suggests a month so I've made it 32 days to Yahoo gets off my back.  

    $min_serveOptions['maxAge'] = 2764800;

I also need to cache my images.  For that I've gone to my .htaccess file and added:

    <FilesMatch "\.(ico|jpg|jpeg|png|gif)$">

Header set Cache-Control "max-age=2764800, public, must-revalidate"

    </FilesMatch>

Again, if I need to change an image I'll have to give it a new name.  

Again, YSlow is impressed and gives me Grade C.  Remember that I haven't touched Mint yet.

4. Compression

Why isn't all website content compressed?  It's so easy to do and all modern browsers accept it but still the majority of websites don't compress.  I have no idea why.  Maybe they'd just never thought of it.  Like me!

I'm using Apache 1.3 and I should really be using Apache 2.  If I was using Apache 2 I could use mod_deflate and just add these lines to my .htaccess:

    <IfModule mod_deflate.c>

    # compress all text & html:

AddOutputFilterByType DEFLATE text/html text/plain text/xml

    </IfModule>

But that doesn't do anything because I don't have mod_deflate.  But I do have mod_gzip so this should work:

    <IfModule mod_gzip.c>

        mod_gzip_on       Yes

        mod_gzip_dechunk  Yes

        mod_gzip_item_include file      \.(html?|txt|css|js|php|pl)$

        mod_gzip_item_include handler   ^cgi-script$

        mod_gzip_item_include mime      ^text/.*

        mod_gzip_item_include mime      ^application/x-javascript.*

        mod_gzip_item_exclude mime      ^image/.*

        mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*

    </IfModule>

But it doesn't work because I'm hosting this at Fasthosts.  I really like Fasthosts - I've used them for ten years and they are completely reliable.  They aren't going to be the most up-to-date on technology but at £50 a month for a reseller account, you can't expect them to be.  Anyway, GZIP didn't work and the engineer couldn't help.  

But there is a third solution - to do it in PHP.  Amazingly, all you have to do is add one line to the top of your PHP files:

    <?php ob_start("ob_gzhandler"); ?>

 But you do need to ask yourself whether this is worth it.  Out of the 1.2 seconds it originally took my page to arrive, less than 0.05 seconds was taken up receiving the content.  On pages with lots of content, this may be worthwhile but not everywhere.  Also, if your page is designed to lay out one piece at a time, you may not want to buffer the output, which is what this command will do.  

And that's where I stopped.  A page that started out taking 1.2 seconds, now takes less than 0.5 seconds.  If I move Mint into the footer, I can trim that further.  YSlow now gives me a Grade B for my smaller pages, where Mint is more significant, and a Grade A for my larger pages, where I've added compression.  Not bad for two hours' work.

Word's "magic quotes" in HTML

Sometimes it is necessary to take content in Word and put it into your CMS.  Some tools will replace Word's "magic quotes" with the correct HTML codes but some won't.  If you find you need to do it yourself, you might find this Word macros useful:

'

' WordReplaceSpecialChars Macro

'

    Selection.Find.ClearFormatting

    Selection.Find.Replacement.ClearFormatting

    With Selection.Find

        .Text = ChrW(8220)

        .Replacement.Text = "&#8220;"

        .Forward = True

        .Wrap = wdFindContinue

        .Format = False

        .MatchCase = True

        .MatchWholeWord = False

        .MatchWildcards = False

        .MatchSoundsLike = False

        .MatchAllWordForms = False

    End With

    Selection.Find.Execute Replace:=wdReplaceAll

    

    Selection.Find.ClearFormatting

    Selection.Find.Replacement.ClearFormatting

    With Selection.Find

        .Text = ChrW(8221)

        .Replacement.Text = "&#8221;"

        .Forward = True

        .Wrap = wdFindContinue

        .Format = False

        .MatchCase = True

        .MatchWholeWord = False

        .MatchWildcards = False

        .MatchSoundsLike = False

        .MatchAllWordForms = False

    End With

    Selection.Find.Execute Replace:=wdReplaceAll

    Selection.Find.ClearFormatting

    Selection.Find.Replacement.ClearFormatting

    With Selection.Find

        .Text = ChrW(8216)

        .Replacement.Text = "&#8216;"

        .Forward = True

        .Wrap = wdFindContinue

        .Format = False

        .MatchCase = True

        .MatchWholeWord = False

        .MatchWildcards = False

        .MatchSoundsLike = False

        .MatchAllWordForms = False

    End With

    Selection.Find.Execute Replace:=wdReplaceAll

    Selection.Find.ClearFormatting

    Selection.Find.Replacement.ClearFormatting

    With Selection.Find

        .Text = ChrW(8217)

        .Replacement.Text = "&#8217;"

        .Forward = True

        .Wrap = wdFindContinue

        .Format = False

        .MatchCase = True

        .MatchWholeWord = False

        .MatchWildcards = False

        .MatchSoundsLike = False

        .MatchAllWordForms = False

    End With

    Selection.Find.Execute Replace:=wdReplaceAll

End Sub

A bug in Nifty Corners

I'm have been using the latest version of Nifty Corners (NiftyCube) in two sites for some time but one of them, my main Rails site, has been exhibiting a strange bug which I couldn't quite isolate.  Sometimes the page would finish downloading but wouldn't display until either (a) I touched a key or (b) some considerable time had passed.  I decided to ignore it.

But now I'm completing a third site to use Nifty Corners and I'm guessing that the client won't let me get away with ignoring it this time.  I noticed that the problem was at its worst when using the Back or Forward keys as most of the pages on this third site were straight HTML with only a dash of PHP and should display almost instantly.  But there was that five or ten second delay again.  Maybe it was trying to download one last component of the HTML but I couldn't see why that would stop the page downloading.  I decided it had to be Javascript in the window.onload function.

After some digging through Alessandro Fulciniti's very elegant Javascript, I spotted a function AddCss() where he creates a link tag to load in his CSS file niftyCorners.css.  The problem was that the code assumed that the CSS file was in the current folder.  I changed this to specify the folder:

function AddCss(){
niftyCss=true;
var l=CreateEl("link");
l.setAttribute("type","text/css");
l.setAttribute("rel","stylesheet");
l.setAttribute("href","/_css/niftyCorners.css");
l.setAttribute("media","screen");
document.getElementsByTagName("head")[0].appendChild(l);
}

Now I can get on with what I was supposed to be doing this afternoon.  Hope this post helps someone.

Converting ASP sites to PHP

OK, here's a quick tip of you're converting some ancient ASP site to PHP.  Did I mention I used to create sites in ASP.  Sorry, it's a bit embarrassing you see...

When changing a site's technology you don't want old URLs to stop working so you'll want each of the old URLs with .asp in them to safely find the new URLs with .php in them. 

So you'll have a URL such as:

http:⁄⁄www.example.com/foo/bar.asp?someid=14&someotherid=apple

and you want it to be:

http:⁄⁄www.example.com/foo/bar.php?someid=14&someotherid=apple

Apache makes this easy.  Just get hold of the .htaccess file in the root folder of your site and add a line to it:

RewriteRule  ^(.*)\.asp(.*)$ $1.php$2 [R=301]

Care what it means?  Didn't think so.

I'll tell you anyway.  It tells Apache to look for a URL that has ".asp" in it somewhere and tell the browser to redirect to a URL that is made up of the bit before the ".asp" plus ".php" plus the bit after the "'asp".  In regular expressions, anything enclosed in brackets, e.g. (.*), can be accessed in the replacement string using $1, $2 etc.

The bit on the end - [R=301] - tells Apache to send a status code of 301 to the browser telling it is a permanent redirect.

Told you it was quick!

PS There are other ways of doing this of course.  You could keep all the ASP files and fill them with individual instructions to redirect to the new URL or you could put the PHP code into the old ASP pages and keep the URLs the same.  If you go one of those routes, you'll need to tell Apache to execute the .asp page as if it's a PHP page.  The line to go into your .htaccess page would be something like this:

AddType application/x-httpd-php .php .phtml .php3 .asp

Contribution Manager - Gift Aid software on Rails

Perhaps unfairly, Rails is mainly associated with social networking sites so it's nice to be able to show that it's being used for mainstream business applications.

Contribution Manager (www.contributionmanager.co.uk) is intended for small UK charities to help them with the administration and claim process for Gift Aid.  It has been designed for use by churches and other charities that use contribution envelopes but can also be used by scout groups, music societies, parent-teacher groups and any other small charity.  Pricing is remarkably low as small charities have enough trouble raising funds and won't want to spend their hard-earned contributions on software. 

As it's designed for UK tax laws, it is unlikely to be much help to non-UK charities/not-for-profit organisations but if it could be modified for use in other countries, I'm sure the authors would appreciate suggestions.

gift aid

Domain mapping and Typepad revisited

Last year, I published a how-to on setting up domain mapping with Typepad.  Since then, I have noticed that some sites link to emilyrose.org instead of linking to www.emilyrose.org.  Because of the way I'd set this up, emilyrose.org still links to my ISP's parking page so Technorati, for example, was showing a picture of the parking page instead of a picture of my home page

So, as I'm using godaddy, I've switched on domain forwarding as well but only to forward emilyrose.org to http://www.emilyrose.org.  The CNAME record to then forward www.emilyrose.org on to olddog.typepad.com does the rest of the job. 

This may seem a roundabout solution but it does work. 

Social networking is not exactly new

I think I know how this started.  The Guardian G2 editor reads a story that William Wales (aka 'Prince William', an English 'it boy') has joined Facebook and thinks to himself "Facebook - have I heard of that?".  Then Lily Allen (English singer) joins MySpace and he think "I need to get someone to explain this to me".  So some young people are duly rounded up by the technology correspondent and an article appears on the "social networking revolution".  The young people are asked to talk about Facebook, MySpace, Bebo, Twitter and SecondLife.  No LinkedIn.  No Ecademy.  No realisation that 'old' people do it too. 

Social networking has been around a while.  How do you suppose the second and third people to master fire found out about it?  Read a book?  Looked it up on Wikipedia?  I'd guess one of their trusted network told them about it.  Social networking was thriving in pubs when I was a nipper and going pretty strong at the IoD when I was a member there.  Just because there's a way to do it online doesn't make it new.   

MySpace started in September 2003, Facebook in February 2004, Bebo in January 2005 and Twitter in March 2006.  But Linkedin has been going since before any of them and Ecademy since June 1998.  Even a mention of Friendster (March 2002) or Friends Reunited (July 2000) would have given the article a little more perspective.

So I did a test.  I logged into Facebook, set up a profile and then looked for university friends.  Not one.  So I pretended I was five years younger and got a few.  Ten years; more.  But I had to act like a 25-year-old before I found significant quantities.  The same test in LinkedIn showed a much broader cross section, right up my former MD who must by 58 or 59 by now. 

So if you think social networking is leaving you behind, don't just pile into Facebook and start making friends with teenages.  That might not actually be very good for your reputation.  Find out what your friends use and give that one a try instead.  And may I give you a tip; don't just form an online connection with anyone and everyone in an attempt to have more connections than your friends and to prove to them that you're not Billy No-Mates after all.  I'd strongly advise that you stick with people you know and people who are personally recommended to you.  Go for quality over quantity.

Mephisto and cyclothymic development

It's always nice to learn a new word and Nikolay Kolev has taught me 'cyclothymic', a personality disorder characterised by bouts of intense activity followed by periods of the complete opposite.  Nikolay associates this disorder with OSS Rails development and it's easy to see his point.  Many Rails developments have set off in a burst of enthusiasm and code marathons followed by a rapid descent into inactivity.  I was involved in one myself - I still don't know where half the development team disappeared to.

Mephisto is one such sufferer.  Rick Olsen has confessed that it's been comatose since the end of last year and the prognosis isn't good.  The 'community' around Mephisto hasn't produced any meaningful documentation yet - and yes, I am one of that community so I am part of the problem - so the product isn't getting the following it deserves.  To pick up on a point made by Justin Palmer, it is still seen as a great Rails publishing tool, not a great publishing tool.

This blog uses Typepad and I have other sites that use Wordpress.  Neither tool is leading edge or even particularly impressive but they both do exactly what they need to do and they do it effectively.  I have no need for more than what they do.  So no, I don't use Mephisto and I doubt that I will for a good while yet.  I hope it pulls through as it's a great piece of software ... but there's more to a successful software project than good software.

It is clear that two distinct and damaging problems are afflicting the Rails world.  Firstly, there is a desire to rework existing applications as Rails applications even if no specific advantage is offered to the end user - for example blogging tools, forum tools, to-do lists and so on.  Secondly, the documentation is then neglected so only fellow Rails enthusiasts can be confident of using the software successfully.  I remember the RTFM responses I got from early Unix vendors but at least there was a manual to read.  RTFSVN is no answer at all.  If these issues aren't addressed by the core team, I'm worried that much of the buzz that surrounded Rails in 2005 will be wasted.

Rails, email and multipart/alternative

All of a sudden, I need to produce an HTML email for a client.  Is that the only way it ever happens - without warning?

Having recently bought Chad Fowler's Rails Recipes, a book I would recommend, and having spotted its recipe for Gracefully Degrading Rich-Content Emails, I thought I'd use Rails for this exercise.  The recipe uses the multipart/alternative MIME type which allows you to specify an HTML version and a plain text version within the same email.  If the mail client can deal with HTML it displays that version, otherwise it displays the plain text version.  With more emails being picked up on phones and Blackberries, having a plain text fall back option is essential.

The recipe is #67, if you have the book, but I've found one or two problems.  The code for the controller in the book is roughly as follows:

def multipart_alternative(recipient, name, sent_at = Time.now)
  subject      "Test multipart HTML email"
  recipients   recipient
  from         'Old Dog <olddog@examplel.com>'
  sent_on      sent_at
  content_type "multipart/alternative" 

  part :content_type => "text/html", 
    :body => render_message("multipart_alternative", :name => name)

  part :content_type => "text/plain", 
    :body => render_message("multipart_alternative_plain", :name => name)

end

The problem with this code is that multipart_alternative behaves like any Rails method and, by default, renders the view multipart_alternative.rhtml if it finds it.  So the email generated has the content of multipart_alternative appearing twice and thus messing up the MIME syntax.

There are many possible fixes but I preferred to simply rename multipart_alternative.rhtml to multipart_alternative_html.rhtml and alter the render_message accordingly.

def multipart_alternative(recipient, name, sent_at = Time.now)
  subject      "Test multipart HTML email"
  recipients   recipient
  from         'Old Dog <olddog@example.com>'
  sent_on      sent_at
  content_type "multipart/alternative" 

  part :content_type => "text/html", 
    :body => render_message("multipart_alternative_html", :name => name)

  part :content_type => "text/plain", 
    :body => render_message("multipart_alternative_plain", :name => name)

end

The second problem I discovered is when I tested this on a variety of email tools.  Outlook picks up the HTML version and displays it fine but Google Mail chooses the plain text version instead.  Testing is, as ever, essential.

Rails and Windows, a natural partnership?

I happened to be looking at the stats on a website I recently created for a large well-known IT company (whose name is a three-letter acronym) and was surprised how few of the hits were from Firefox.  It was 96% IE and just 4% Firefox.  When I looked at OS, it was virtually 100% Windows.  Curious, I thought, that's not what I would have expected from an IT company.

So that got me thinking.  I took a look at Feedburner's stats for this blog, which could be categorised a Rails Blog as I rarely talk about anything else, and found Firefox 54%, IE 26%, Safari 11% and Opera 9%.  So I scrolled down expecting to see masses of Macs and yet ... 67% Windows, 25% Macs and 8% Linux. 

So does this mean more people develop Rails applications on Windows than anywhere else?  I strongly suspect it does.