Improving Geekzone page load time breaks ad blockers... Ask the ad blocker developers to fix their software
Warning: long-ish, geek-ish post ahead...
For the last few months I have spent at least a few hours every week trying to make Geekzone the fast as possible. I really believe the web needs faster websites, and the answer is not only on laying bigger pipes, but smartly using the technology available.
There are many reasons to have a fast website. For the publishers I can think of higher position on Google search results, reduced bounce rates, improving time on site, and many more. For readers the most obvious is that instant gratification when they can try loading a page and start using it almost instantly.
Our site is hosted in Auckland, mainly because 65% of our New Zealand traffic actually comes from that single city in the country. Also because it's closest to the point where most data traffic leaves New Zealand, which is one less hop for all our overseas readers (and we have a large number of those, almost 55% of the total visitors number).
We implemented domain sharding, by creating domains for our css, images, avatars, flags, etc. This was later replaced with a CDN using a base domain name different from our main Geekzone one, so to create a cookieless domain, avoiding unnecessary data traffic when browsers requested those resources.
I also spent some time changing our database schema, indexes and creating a session management that didn't rely on the standard ASP IIS session. This allowed us to increase the number of workers processing web requests.
One of the best changes we made was implementing Aptimize, a server-side software that transparently optimizes our pages by automatically creating image sprites, CSS inline graphics, and other special tricks. Running Aptimize pratically reduces the number of requests by half when loading the Geekzone front page, and cut the number of requests in our forum pages down by a third. Don't believe? Check this test with Aptimize off and a test with Aptimize on.
Aptimize also helped me create the CDN bits by automatically rewriting references to static resources to point to a domain of my choice.
Those and other changes (including code changes) proved well worth the work so far. Average page load times were down from ten seconds to about six seconds (this includes overseas visitors), and New Zealand visitors see an average of two to three seconds load time. This includes the time to load advertising in our pages.
But we still had some other tricks, but let's talk about something else before going there.
Online advertising. Since 27th October 1994 we have lived with online ads. That's when HotWired ran the first banner ad on a webpage. Since then many content publishers, including myself, can only afford to continue providing their service for free because of banner ads.
It's a common misconception that publishers make money only if you click an ad, and since you won't be clicking an ad then blocking banner ads won't impact publishers. This is wrong. Online advertising comes in many "models", including CPM (Cost per thousand), and CPC (Cost per click). In CPM models publishers get paid by the number of impressions of an ad, regardless of how many clicks it gets. So this clearly means that blocking ads prevents publishers from realising important revenue - money that goes into paying hosting, traffic, CDN, software, development, salaries, etc. Some time ago Ars Technica wrote an important article on why Ad Blocking is devastating to the sites you love.
Back when the only way to access the Internet was on dial up people started using "ad blockers" because, seriously, any extra bit to download impacted on the time a browser spent loading a page. This is not true on bradband connections anymore. We are talking about seconds, down from minutes. And we are talking about seriously breaking the web when only large media companies could be able to run websites (and they still run ads on them anyway).
I once received a tweet from someone saying "hey, the Geekzone layout is broken when I use ad block, any chance to fix the layout?"... I know that person. As in I have met him before. And I seriously think he wouldn't work for free. But he was asking me to help him with a problem created by his own action while blocking ads on my site. Sorry but I couldn't spend time on that problem, seeing I still have the "Speed Project", the "API Project", and many other things to work on the site. Things that actually cost money to create. Money he was blocking me from making.
It's good for you isn't it? You can see our content in less than a second! I could be getting less ad impressions if you close the page before the ads are loaded, but at least I thought of you and presented the content first, even if it means a little less for me, right?
But there's the law of unintended consequences... Ad blockers actually block the script that loads all other scripts in our pages, the "mother script".
We use Google search embedded on Geekzone and being the search a script it will only show if it can execute inside your browser. And since the mother script is being blocked by ad blockers, our embedded Google search doesn't work anymore if you are running an ad blocker.
Also logged in users have a "Geekzone Live" feature in our home page, where they can see forum updates as they happen, instead of the static front page visitors see. That script is also not working, because the mother script is being blocked by ad blockers.
Since implementing this change I received many comments in our forums and in private. People are angry. They don't care the pages are loading in less than a second now. What they care is that search and live updates are not working anymore when they run ad blocking software. A few of these people actually think I made this change intentionally, to punish people using ad blocking software.
No, I did not do it to punish people using ad blocking software. What they don't realise though is the only broken thing here is the ad blocking software itself. If they still want to run that type of software they should ask the developers to make it work with new web technologies, and keep up with times.
Better still, people should just not use it. Just read why Ad Blocking is devastating to the sites you love again...
Just received this release from Alcatel-Lucent and decided to post here:
Alcatel-Lucent has announced the publication of a user-friendly guide to understanding the potential of ultra-fast broadband (UFB) for New Zealand.
Titled Ultra-Fast: Understanding the Next Generation of Broadband in New Zealand, the guide helps to answer some of the common questions many New Zealanders have about UFB. Like, "What is it?" and, "What will it mean for me at home and at work?"
Shifting the focus away from contracts, layers and architecture Ultra-Fast: Understanding the Next Generation of Broadband in New Zealand looks at the potential of UFB for New Zealanders, in particular within the health, education and business sectors.
"People have said to me, 'what's all the fuss about UFB, it's just fast internet right'?" said Jyoti Mahurkar-Thombre, CEO Alcatel-Lucent New Zealand and Pacific Islands.
"UFB is about so much more than just fast internet. I'm excited not only because it will allow us to do all of the things we can imagine with higher bandwidth like super-high-quality video calling - but also because it will allow us to do things we can't yet imagine."
Students at Pt England School in Auckland, are already leading the way with ICT-enabled learning. Podcasting, blogging, television and cloud computing are well utilised e-learning techniques.
"Today, Pt England School uses a mix of technologies to provide connectivity for students, however with a UFB connection delivered over fibre, the school will be able to take its pioneering learning approaches even further. There will be no limits," said Jyoti.
The Ultra-Fast: Understanding the Next Generation of Broadband in New Zealand guide has been endorsed by the Minister of Communications and Information Technology, Hon. Steven Joyce, Crown Fibre Holdings and the Telecommunications Users Association of New Zealand (TUANZ).
To download a copy of Ultra-Fast: Understanding the Next Generation of Broadband in New Zealand, or view the video please visit: www.ultra-fast.co.nz.
I am lucky to have been selected as one of the first people testing TelstraClear's new 100 Mbps-capable cable network (available in Wellington, Christchurch, Kapiti). The telco has upgraded its cable network to support the DOCSIS 3 standard, giving users access to 100 Mbps download speeds, and 10 Mbps upload speeds.
One of the first things I've done was to start a discussion on Geekzone about it. But I will put a summary in this post - installation, infrastructure, performance, expectations, etc.
Installation in itself is easy, since the entire network is now DOCSIS 3 capable. A simple modem swap and we are ready to go. My first modem was a Motorola, which is being replaced today with a Cisco model. All the screenshots are therefore based on the Motorola modem. Of course (like in any other network situation) you have to put a bit of investment here. My current router is a Cisco Small Business Pro SRP 521W. This router supports two VoIP lines, and WiFi 802.11n. I am noting this because when you get access to a faster network your router will be an important piece of the puzzle. How many packets can it process? How reliable is it?
The 802.11n is important here. If you have a 802.11g router and use wireless as your main connection then you would already be limiting your bandwidth to at least half (nominal) or about one fifth (actual). More considerations on speed later (this is going to be a long post).
Also we have two gigabit switches in our home network, and all our PCs are either ethernet gigabit capable or 802.11n capable.
With no further ado, here are a few Speedtest captured when I got the modem installed:
Interesting, right? A speedtest to Christchurch gives me better results than one to Wellington, where I live. That's because of peering. It's important to note (as in any other broadband service), what you get is a line speed. Final results will depend on how packets are routed, the capacity on the target server providing the content you want, etc. Obviously this is limiting but only uses portion of the bandwidth available.
Let's get another test going on. Let's stress this connection by initiating an large number of downloads. Nothing best to test this than P2P software, so I decided to download lots of documentaries. All those were sourced from Vodo, a legal torrent site, distributing copyright free content. I removed any limits on number of connections, so this obviously tested not only my line speed, but also the Cisco SRP router. Here are some screenshots and comments:
Twenty five downloads, with different number of seeds available, automatic prioritisation by uTorrent. Note the download speeds - the peak was 6,428 KB/s (kilobytes per seconds), equivalent to 51 MB/s (megabits per second). As the software finished downloading the files you can see the ETA for download being drastically reduced as more priority was given to the remaining files:
And the last screenshot:
My next test involved the YouTube MySpeed Test page, and here are my results. You can see the big jump once we had this connection installed here, and below the instant results on a test video, at 26,788 Kb/s:
As part of my testing I agreed to have a TrueNet device installed here. TrueNet tests your broadband connection on random intervals, reporting individual and aggregate information (TueNet is alway looking for volunteers). This is my speeds on the first few days:
Good, but still not great, right? After meeting with TrueNet and discussing the results, they came to the conclusion some work was needed. For example small files didn't have the expected performance because very fast connections don't have time to ramp up to faster speeds. They changed the methodology and last week I received a new chart (below), comparing my 100 Mbps cable service with a 100 Mbps fibre service. Notice how my speed go consistently higher since TrueNet decided on a different approach for testing and statistical analysis:
Right, so what do I use all this speed for? We already used a large plan (25 Mbps, 120 GB) with TelstraClear before. We have two adults working from home. We consume a large amount of online content, including at least two or more video rentals from iTunes. I have an online backup account, using about 130 GB and about 1GB uploads daily, mainly documents updates, Outlook PST files updates, photos and home videos taken with Flip MinoHD. We also have two VoIP lines here (VFX with WorldxChange and an Australian number with Mynetfone).
On average we use about 100 GB a month. Count those 300 MB Apple mac updates, 100 MB Windows updates, MSDN downloads, a few LiveMeeting events every month, a few Skype video calls and you have a good idea of how much we use.
The big question for me now is how TelstraClear can make this a more compelling proposition. Price of course is going to be of impact. In broadband world "you get what you pay for" is very true and while the existing 25 Mbps cable service is not cheap, it's of the highest quality. Other things are value added services. I'd really like the option of moving my online backup (provided by a US vendor) to a local service. Obvious reason would be possible upload speed increase. Even better if TelstraClear provided a bundled service with non-metered traffic for this service. Or perhaps TelstraClear could join forces with Microsoft and offer a hosted communications services (which I am actually going to be testing very soon, with another provider).
I will post about this trial later with an update after changing the modem to the Cisco model.
Full disclosure: I have been a paying TelstraClear customer for about ten years now. We are so happy with the service itself that when we decided to buy a house one of the requisites was to have a cable-modem service available at the new address. As part of this trial my current plan is not being charged, but I pay for any traffic over my original plan. This happened once in the first month because all of the downloading and testing I wanted to do.
A $80 million investment, the IBM Highbrook Datacentre is the latest addition to IBM's network of green datacentres around the world.
So here is the video on that DVD:
PS: Bella watched it with me. Her comments were "This is a slow movie. Can we watch Hi-5 instead?". She's four.
Sometimes faults are reported to ISPs and telcos but no job is logged. This can happen because simply power cycling the router restores the service - but it may not actually fix the problem, and the companies don't know something is going on.
We also see many new discussions here on Geekzone with topics such as "Is [service] down in Christchurch or is just me?"
To help us get a better view of these small problems - and their real impact - I wanted to start collecting information from our users - the "crowdsource" part of the project.
I put together a Geekzone ISP/Telco Fault Report page where you can record faults as they happen.
We will make the collected data publicly available in the coming days with views by ISP and region, including some charts and hopefully a Google Map plugin.
Big thanks to Brett Roberts for starting the process...
A few weeks ago I received a suggestion to award one of our Geekzone users for his contributions. It was a great suggestion and after talking to our moderator team we agreed Stephen Abbot (cyril7 on Geekzone) has been of great help to our entire community and deserved some recognition. On his own words:
"I am deeply honoured and to be honest, quite humbled to be offered such a gift. As you probably are well aware my contribution is provided without hooks and in an effort to demystify stuff for the average person that they normally would not have easy access to."
Stephen lurks around Geekzone and helps people on lots of different subjects, from broadband performance to Freeview and Sky configurations.
I reached out to Dell AU/NZ and they offered a ST2320L full HD monitor - 23" widescreen HD (1920 by 1080) monitor, for use with desktops, laptops, gaming consoles, media centres, digital video recorders. It's here with me now and we will arrange for Stephen to get it this coming week.
After thinking about the suggestion for a little bit we also agreed to give a prize to the user who created the most epic discussion this year. This honour goes to Adam Jobbins (ajobbins on Geekzone, @ajobbins on Twitter) who created the topic "iPhone 4 NZ Launch Date".
This topic got really hot during the days - and nights - preceding the iPhone 4 release in New Zealand, and all the problems around the lack of external communications from both Apple and Vodafone New Zealand during that period. The discussion gathered so much momentum that on the launch night it got more than 70 pages of replies. In total there are 160 pages with 2,399 replies, making it the most epic discussion of the year (just a few replies ahead of another huge discussion, the Samsung Galaxy S I9000 by lokhor).
Adam will receive an Amazon Kindle (which I bought specially for this award).
Congratulations to both Stephen and Adam.
I am thinking we will establish a more formal award for 2011, with additional categories to be announced during the year.
For the third year I am a judge at IT Rockstar, a crazy competition to find the top New Zealand IT professional.
Brett (NZICT CEO)
Josh (IT Rockstar 2010)
Mauricio (Geekzone BDFL)
Rhoda (Greenstone CEO)
This year the team at 920 are accepting individual and team nomination.
Below is a video of the IT Rockstar 2010 final - skip to the 33 second mark to see me and fellow judge Brett Roberts (another Brett):
I am reproducing here Juha's blog post "New unjust copyright law alert: you are guilty until you prove your innocence" because this issue affects all New Zealand Internet users in general and I want it to get even more exposure.
Despite Labour and various lobby groups patting themselves on the back, saying a compromise has been reached that makes the amended copyright law workable and fair, it looks like things have in fact taken a turn for the worse.
Please make sure that you read this: Internet law guru Rick Shera is ringing the alarm bell about a completely new provision being introduced into the Copyright (Infringing File Sharing) Amendment Bill at the very last moment by the Commerce Select Committee.
Called Section 122MA, the provision essentially presumes an account holder is guilty if accused and must prove his/her innocence. Rights holders infringement notices are considered as conclusive evidence however. There is no sanction against rights holders who present erroneous or false evidence in the proposed new law either.
This is really bad from every perspective, and I fully concur with Rick that Section 122MA must be deleted. It's not necessary, not for rights holders, and not for copyright protection either.
As it stands, S122MA can be used maliciously
, and applied to not just file sharers.
Update: Rick sent a correction - "The new regime can ONLY be used to target file sharing (you suggest that it could be used for something else). Section 92C of course has exactly the same problem with guilt on accusation. Irony is many submitted that the new regime and section 92C should be made consistent. Little did we know that they would retrofit the new one!"
The irony here is that I remember David Farrar, wearing his InternetNZ hat, saying during the S92A debacle that it was necessary to engage and help shape the new law. If not, something much worse than S92A et al could appear.
Clearly, there have been many, many submissions on the new copyright bill and plenty of public opinion expressed too - and correct me if I'm wrong, but I don't think anyone with the possible exception of rights holders have advocated that we remove the principle of innocent until found guilty from our laws.
So why did the Select Committee insert Section 122MA? This needs to be explained.
An interesting sequence of events started here earlier this week. I received an unexpected box, posted from Dunedin, apparently from one "Olga Fiedo". Inside the box a small clay ornament, with a numbered card.
I don't know Olga Fiedo, and search attempts didn't return anything meaningful.
But this morning I got a phone call. Someone asked for "Olga Fiedo". It seems this person also got a box from this mysterious "Olga Fiedo". With my phone number in the sender details.
Now curious, I looked at the box I received earlier this week and noticed that the phone number in the recipient details was not from Dunedin, but a Wellington area number. So I called that number and reached a family voice mail. No "Olga Fiedo" in the welcome message though, only mum, dad and two kids.
It's obvious now that whoever sent this wanted people to call each other, and spread the name - or get curious about it. As I posted to my Twitter, ok Olga, I'm listening...
Do you know Olga Fiedo?
Thanks to Fake Steve Jobs for refreshing our memory...