For web based businesses, being able to accurately report the number of unique visitors to your site and the amount of time they spend on your site is essential you need to be armed with this information and the evidence to back it up in order to attract good rates from advertisers.

Back in the old days, you’d just parse your weblogs and count “hits”, every access was a hit, no matter what it was. That was fine until pages linked in Javascript, CSS, Images etc. Then we had to move to “Page Views”. We just looked at the specific “root” file for a page, and discounted the other objects that were loaded as a result.

Of course, then that doesn’t allow for pages like the request-o-meter on snakenet, which auto-refreshes to show people when they can next request. Does this really count as another page view? Not to advertisers mostly. Advertisers don’t want to see a buy ads on a site with 10,000 page views per hour, if that page view count comes from 2 users.

And so, sites and advertisers started to track with cookies, but that’s no good any more:

BBC NEWS | Technology | Web counting tools ‘need change’:

The way web audiences are measured could be ripe for an overhaul, according to two reports out this week….

In comScore’s study, an analysis of 400,000 home PCs in the US found that a hardcore minority of web users are clearing their cookies from their computers on a regular basis.

This causes servers to deposit new cookies which in turn could lead to an over-estimate of unique users to a particular website.

It found that 7% of computers accounted for 35% of all cookies, which extrapolated could mean the size of a site’s audience is being overstated by as much as 150%, said comScore.

Justly, many users don’t like cookies. And these are the most web literate users. It’s an invasion of your privacy they argue. People trying to profile you and track your habits. So they can better target their advertising to the right sites and the right people.

Perhaps unsurprisingly, comScore offers a very different approach to audience measurement - using the panel-based system favoured by the TV and radio industries which relies on using a representative sample of net users to gauge behaviour.

This works for TV, because TV is pretty limited. It takes a big chunk of capital to set up a TV channel, to make a TV show, to get someone to air it. It’s free to set up a web site. And thus, there are a lot of them. A lot of competition, lots of sudden start-ups. Things are niche. Take social networking, there’s MySpace, Facebook, LiveJournal, LinkedIn. Loads of them. People gravitate towards different ones on all sorts of random basis.

You can’t generalise traffic to one of those sites. There are new services getting tons of traffic from markets you can’t generalise. There are massively trafficked sites I’ve not heard of.

Perhaps the complexity is just in finding out how to come up with sensible representative survey groups. With the social networking it’s a bit easier. People who’ve been online a long time are more likely to use LiveJournal, which isn’t a true social networking site, but has aspects of it and has been around a long time. School kids and students will gravitate to facebook. Professionals to LinkedIn. That’s an easy one. But to music sites? You can’t judge the internet radio sites people visit that easily.

I know of several sites with no reason to choose between them that can be generalised. It’s down to the community. Some people fit in better elsewhere. So how are you going to be able to calculate their sites on that basis?

But, I guess this approach isn’t really about figuring out the traffic for every site, but just for the big sites, that everyone knows about.

So, they’re playing round with ways to get an accurate idea to estimate the number of unique people visiting bigger sites so they can sell advertising.

People expect the web to be free. They don’t want to pay to use facebook. Facebook need to make money. Even if it’s a cool idea that someone sets up because they think that would be a cool service running a big site is not free. The internet is only free to the consumer. The middle-men, the people who run the sites have to pay money to the bandwidth people, the power people. They don’t get that free. So you have to find a way to get money that doesn’t directly take it out of the pockets of your users.

And that’s advertising.

And your users hate it.

There are many things around that exist purely to take the advertising out of the web. I use AdBlock Plus in firefox to strip the vast majority of adverts out of every site I visit. I combine this with Greasemonkey and some user scripts to take other adverts out of sites I use a lot. And so do a lot of other people. Plus people use tools to block cookies that are used to track their use of sites and their browsing habits. Some people just purge all, but there are tools more like AdBlock Plus that do extend and enhance Firefox’s user permissions to blacklist and whitelist cookie providers. Spybot, Adaware and other tools rip out these tracking cookies.

So on the one hand, you’ve got the problem accurately reporting your usage and getting that user base up to get revenue from advertisers. On the other hand you have a hard-core minority of web users trying to eliminate that revenue stream because they don’t want your crappy invasive adverts and they don’t want to be profiled and tracked.

You have to find another revenue stream for your online business. Adverts alienate users and dilute your brand image. But people don’t want to pay for things. I guess that’s a bit of a problem for me isn’t it? On the one hand, I won’t attract users and keep users if I make Multiblog a pay service. I will have problems attracting advertisers to start with and producing evidence of my true user base to get them later on. And if I do get adverts, many of my users will be blocking adverts or alienated by them. Blogging to multiple-blogs in one hit is a very web literate thing to want to do. My core demographic is the advert avoiders.

So what do I do?

The answer isn’t great. I’m not going to waste my time building an advertising funded service. My users will ad-block. I’m not going to get anywhere charging for everything. I have to follow the web equivalent of the shareware model. I have to start free, then I have to introduce charges for advanced features later.

It’s essential that I’ve identified this model early, because I’m going to have to design user accounts and features around the fact that some features or sub-features will one day switch to paid user only, and I may even have tiers of paid users. Critical thing. if you miss that at the start, then you’re really not going to manage to deliver the payment extensions later on.

Popularity: 39% [?]

Typically for me, I’ve got sidetracked for the last few articles with technical stuff. What platform? Where to develop? Get the development environment set up. All that kind of stuff. Of course, that just hand waves away the fundamentally most important aspect of writing software:

Clearly identifying what you are going to produce, what it is going to do and how it is going to do it.

That is where both bespoke and product development most typically comes unstuck. Not agreeing up-front what you’re going to write and what it’s going to do. Now, for bespoke work, where you’re creating a custom solution tailored to a customers exacting requirements, that’s a massive issue. There are many problems en-route. Such as simple mis-understandings and, even worse, the things the customer thinks go without saying.

These are the things that cause scope creep and late delivery. Much bespoke work involves fixed price and/or timescales, so this is a major issue.

Of course, in the product world it’s not such a serious issue as much of the time. Obviously, there are issues making sure that your software does something that customers want, and does it in a way that will make them want to pay for it. Also, there are issues with making that first release in a timely fashion to start getting a revenue stream. Or some users and feedback etc.

But for product software, the biggest failing at this requirements capture phase is short-termism. You might decide that the software is going to do so and so. All very well and good, and that might lead down a certain path. That path might turn out to be a cal-de-sac when you realise later on that it was obvious that so and so was right, but, implementing it in such a way means you can’t do a natural extension of so and so that could have been an obvious thing to want to do later right at the start.

So, before I get any more carried away with the platform and the development environment, I think it’s time to sit down and figure out what the basic features of Multiblog are, and where the road map will take it in the future.

Popularity: 33% [?]

This morning I was reading a post by Jeff Atwood about Mike Gunderloy switching away from Microsoft. Jeff says:

Still, his [Mike Grunderloy's] attitude frustrates me, because it falls so egregiously into the stereotypical, religious love/hate dichotomy that I’ve observed again and again in software developers. You either love Microsoft and use exclusively Microsoft products, or you hate Microsoft, and you vow never to use any of their products ever again. There’s nothing in between. No middle ground.

This is something I too see all the time. People take their choice of development platform incredibly seriously. Personally, I’ve not seen too much positive Microsoft love, in my life I’ve been more exposed people with a blind and all consuming love elsewhere. More specifically, I’ve been exposed to Java Zealot’s who will not, under any circumstances, accept that anything other than Java is the solution to their problem. I’ve been exposed to Java developers who are Microsoft haters, and Microsoft haters who are not Java developers also. But overridingly, the zeolotry I’ve seen has been for Java.

My professional work has had a predominately Microsoft bias. I’ve ended up for whatever reason falling into teams using Microsoft tools and products to build solutions. My non-professional work on the other hand has had the opposite bias. I’ve tended towards Open Source languages and tools such as PHP and other scripting languages. I’ve used Delphi heavily. And when I have been using Microsoft languages (mostly scripting), I’ve been using non-Microsoft products to write the code (namely Ultraedit).

I’ve even written Java.

I’ve found this to be a huge advantage to me in my career.

Each language I learn, each tool I use, comes with further knowledge and appreciation of the art of programming. Each new language that appears is designed to solve a perceived problem with a pre-existing language, and so comes with a new development ethos.

As language and platform are a religion for so many developers, the communities round these platforms become isolated and insular. There is little cross-pollination of ideas and techniques between these communities. In biological terms, this limits the evolution and growth of development. It’s gene pool is kept artificially shallow.

I’m driven by a deep interest in programming. I love to pick up new tools and new ideas. I find that every time I learn a new language, I find new approaches to common problems, that could work in a number of other languages that I am fluent in. And then I try and take these ideas and use them outside of their spawning ground.

Looking at some of the endorsements I’ve had provided on my LinkedIn profile from ex-colleagues, it’s not just my perception that this has been good for me. Other people have seen the benefits I’ve given previous companies from this breadth of knowledge. I can think outside the box in any language, because I know so many languages. For example, this quote was from one of the senior technical architects in the Java division of my company, a division that I only directly worked with once however, provided input to many times:

“Mike is a very skilled and highly professional developer. Perhaps the most impressive aspect of Mike’s work is the sheer number of languages and technologies Mike knows to a professional standard - no doubt a consequence of Mike’s eagerness to take on, and complete, new challenges which others may shy away from. Mike’s breadth of knowledge makes him a great asset to have around as he is very approachable and will find time to share his expertise with his colleagues.”

As Jeff says:

As a software developer, you’re doing yourself a disservice by pledging allegiance to anything other than yourself and your craft– whether it’s Microsoft or the principle of free software. Stop with the us vs. them mentality. Let go of the partisanship. We’re all in this thing together.

I’m a pragmatist. For now, I choose to live in the Microsoft universe. But that doesn’t mean I’m ignorant of how the other half lives. There’s always more than one way to do it, and just because I chose one particular way doesn’t make it the right way– or even a particularly good way. Choosing to be provincial and insular is a sure-fire path to ignorance. Learn how the other half lives. Get to know some developers who don’t live in the exact same world you do. Find out what tools they’re using, and why.

It is your duty to yourself, your company, your colleagues and most of all your career to keep your eyes on the alternatives. Don’t start to feel safe that you’ve made the best choice. Pick up other languages as they become of interest, use them in anger, understand why the Ruby way of doing something is different to the Java way, the .NET way and the Visual Basic way. Take that knowledge, absorb it, and re-use it in other languages, on other projects.

Unless you are happy to be a day coder, you need to do this. If you are serious about being a real programmer, and being all that you can be as a developer. You must have breadth. You must get away from the us-and-them, the religion, the I’ve chosen the right tool, yours sucks attitude.

You must understand programming as an art, not just the particular paintbrush that was shoved in your hand. If you don’t have the time or the passion to pick up and code in other languages, at least read blogs about them. Follow the design and issues that surround using that language. Keep yourself open to the new ideas.

It will make a huge difference

Popularity: 47% [?]

I’m fresh home from work, I’ve got a bit of time to myself as the kids have just gone to bed. I’ve got a number of choices as to what I do. I could watch TV. Watch a movie. Read a novel. Or something else. The top contenders on my list right now are to play a PC game (Ghost Recon: Advanced Warfighter) that I was hooked on before Christmas and haven’t had the time to play since then, or to write some software.

Today I found sux0r, it’s an RSS aggregator with a Baysean filter built in so it learns to categorise the posts I read. It’s pretty neat and with a bit of work I could have it trained to read all the BBC news feeds for me and only pass on the things I am interested in, so I don’t have to keep skipping the sports or international politics and can concentrate on UK news, tech, science and health.

It doesn’t quite work how I would like, so I’ve been glancing at the source and considering making some enhancements and submitting a patch back to the core project.

When I left for work this morning however, I faithfully promised that this evening would be dedicated to slacking off and doing something purely to relax. But all I want to do now is sit and hack away at an open source piece of software. To code. Which is my normal work.

I’m not alone, Eric Sink, the founder and a core developer at SourceGear posted this in his blog the other day:

Poker or Citizen Kane?:

A few weeks ago I saw an enticing little flash game on Reddit. It’s called Virus. After playing it just a couple of times, I wished that it had a solver. It took me 25 turns to win that round? How fast could a computer do it with 3 levels of look-ahead?

That night, the question was still nagging me. The kids are in bed. My wife turned in early. Should I pop in a DVD and watch X-Men 3 again? Or, I wonder how quickly I could clone that Flash game?

Eric choose to sit and re-invent a flash game to prove he could. And a few nights later, he ditched Citizen Kane in favour of writing a Poker hand comparison algorithm.

On the surface of it, this may seem like strange behaviour to non-programmers. We’re at home. We could be doing something for relaxation, instead we’re  working. But that’s the point. For a lot of real programmers, we do it because that’s what we love to do. Footballers play football because they love the sport. We code because we love to.

At the end of the day, working on your own problems for your own interest is a real buzz. It’s the problem solving. The challenge.

If you don’t get this, then you’re in the wrong job. People who aren’t enjoying it, aren’t relishing the challenge and wanting to stretch themselves can be programmers, but they won’t reach the same heights that someone who is driven to program can.

Popularity: 26% [?]

I’ve been doing a lot of DIY lately, after we had some building work done. Painting walls, laying new flooring, putting on new skirting boards, putting up new blinds and curtain rails. I’m not very practical, so my Father-in-Law is usually drafted in to help. One thing has become abundantly clear.

It’s all about having the right tools.

About 6 years ago when we moved into this house we turned the old owner’s kids bedroom into a grown up guest room. In the process we put coving up. This involves cutting stuff at 45 degree angles to make corners. When we did the coving, we used a Coving Mitre:

This is an odd shaped bit of plastic which can be layed at various positions on the coving to guide a hand-saw into cutting the right angle. It’s fiddly, and a pain in the ass to be honest. And because you are hand-sawing it’s hard work, and you can’t just nibble a couple of millimetres off. It was a long hard job.

A year or so later, when we put up some Dado Rail in another room my Father-in-Law had bought a Mitre Saw:

This makes it a lot easier to keep the saw in the right position and cut the angles. It clamps the wood in place. It made the job much faster. But it’s still impossible to cut a couple of Millimetres off a slightly too long piece of dado.

Last week, we put up skirting boards. Now my Father-in-Law has bought a Power Mitre Saw:

With this bitch you can do anything. It’s trivial to cut any angle in anything. You can trim a millimetre off the end of a bit of skirting in a second. It makes the job faster and easier.

When we were laying the flooring. My Father-in-Law cracked out his jigsaw and a Square (thing for getting right angles dead straight). Tools I don’t have, I’d have struggled with a big handsaw, a hacksaw and a steel ruler. And it would have taken ages and been crap.

It’s all about having the right tools.

Since this experience, I’ve decided. There is no point in doing a DIY chore without the right tools. And it’s no use going for the cheap-o option (the Coving Mitre) I need to get something decent. The Manual Mitre Saw would have been the best option, balance of cost and utility.

I’ve bought a cordless drill with a set of drill bits and screwdriver bits. I’ve bought a spirit level. I’ve bought a hacksaw and numerous other tools I’ve been doing without. The DIY I’ve been doing since has been much much more productive. I had to assemble some flatpack furniture yesterday. Normally a chore that takes hours and leaves me with hands sore from the screwing in of a million screws.

Yesterday, I used my cordless power drill and a couple of screw driver packs and had everything together in no time at all. Today, I put the curtains and blinds back up. Screwing them in took seconds, and I used the big spirit level with the built in ruler to make sure they’re level and centred on the window. And it took much less time than the blind I put up in our bedroom a couple of years ago, which wasn’t straight.

I still have to get my Father-in-Law round to do something if I’ve not done it before, and to show me what the right tools for the job are. But, my DIY is light years ahead of where it was 6 years ago.

It’s all about the having the right tools and knowing when to use which tool.

This isn’t just true of DIY. Take photography. When it comes to taking pictures of small children, you need the right camera. I’ve got an old 4 megapixel Nikon Coolpix 4300 at the moment. I bought it before my kids were born, and used to take pictures of landscapes on holiday, or us by the pool on holiday. Great camera, lots of control over what it does.

But it’s useless for taking pictures of kids. They don’t stay still looking cute for more than 2 seconds at a time, and the shutterlag on the Coolpix is about 3-4 seconds with a 10 second minimum delay between taking shots. It’s a useless tool for taking pictures of little kids.

It’s not just true of photography and DIY, it’s true of software engineering. Sure, you could write a full blown replacement for Microsoft Office using a command line compiler and notepad. But it would take orders of magnitude more effort than doing so using a rich IDE, powerful debugging tools and suites of standard components.

I know that I’ve said that coding is not like building a house. But, there are things they do share in common, such as the impact of trying to do a quality job in a decent time with the wrong tools.

It’s important that as a programmer, you have the right tools and you use them for the right job. Make sure that you are using the best programming editor for your language, if you aren’t, get the best or as near it as your budget will go. Make sure you have a debugger, profiler, lint tool, test tool. Make sure you have all the tools, and make sure you know how to use them. If you do, you’ll soon be producing better software faster.

If you aren’t, it’s not the right tool, or you’re not using it at the right stage.

Popularity: 34% [?]

I’ve already posted an article about choosing a platform for your development. I missed a very important point, when choosing a language to develop in you also buy into the platform available to you, in this context, this means the third party libraries and supporting frameworks available to you. And as Jeff Atwood points out:

When you choose a language, like it or not, you’ve chosen a platform. And as Steve so patiently and calmly explained to all the Lisp enthusiasts, the platform around the language, more than the language itself, sets the tone for your development experience. The availability of common, popular libraries and the maturity of the development environment end up trumping any particular significance the language holds.

This was something I was planning to move on to talk about in more detail later. When you pick a language, you do need to look around at the choices of libraries available to you. PHP is lucky. It’s mature. There are a lot of libraries out there.

But, you must be very careful when confronted by such choice. Take a look at the options available for templating. Pear (a major source of libraries) have several implementations. There’s Smarty, and numerous other “just templating” libraries. The new Zend Framework also has a templating implementation.

There is a lot to choose from. Making the choice to go Pear::Flexy because that fits in with your use of Pear::DB, or to make use of the entire Zend Framework might be completely the wrong thing to do. You might want to create your own lightweight abstraction. Or go for something like SimpleT.

Picking a language with a small platform footprint can actually be better than picking a language with a large platform footprint.

But I’ll be talking about the costs associated with blindly using a library from the “PHP platform” for every project. When all you have is a hammer, everything looks like a nail. If you only have one type of hammer, that’s not worse than having a whole basket of different hammers.

Popularity: 58% [?]

So I’ve decided on a platform for development. PHP/Apache/MySQL. Next we need to think a little bit about how we’re going to build this thing.

Right now, downstairs, I have a builder working on a new extension to my living room and a carpenter fitting a new kitchen. I don’t want to trivialise the problems and issues they face, but, the building industry is thousands of years old. Materials and techniques do change. But not that much. Lighter stronger materials. But, a brick is a brick. And it takes a certain amount of time to build a wall out of them. And the calculations required to ensure it meets requirements are well known and understood.

There is always the uncertainty of weather, and suprises like (random attempt to appear to know something about building…) dryrot to deal with. But generally, there aren’t that many suprises and new challenges in building. You do have some aspects of interpretting the customer’s desires, but, blueprints, 3d modelling of what’s being planned in the kitchen sort all that stuff out with a pretty exact level of detail.

Software specification, design and development on the other hand is another thing entirely.

Software is not a science, it’s not even really engineering. Software has been likened to a collaborative sport. I quite like this article I read recently where one parallel drawn is that software development is like rock climbing. This rang very true for me, as I’m a software developer and ex-rock climber.

Techniques and methods are changing all the time, we still haven’t managed to reach the point where we have such a set of tools, techniques and materials that we can get a bunch of people to go through an apprenticship in purely practical things and follow a basic recipie to turn out a repeatable delivery on time of a finished product, like we can with building (troublesome builders who run over, turn up late etc asside).

Software development (commercial, enterprise grade) takes highly skilled and educated people to deliver the mess we currently deliver. We’re far behind the building industry, which is delivered by (sorry, but it is) the less bright, less able, less educated people in society. Becuase they can! Becuase those of us with degrees in software engineering are crap at it and are too busy being crap at writing software. But crap in that certain special way.

Anyway, opinionated rant asside, things are getting better. We’re starting to establish some common, re-useable techniques to get software delivered better. There are a whole bunch of generic approaches to development, each one with a myriad of different interpretations.

I’m going to talk about Object Oriented software development, because, in my opinion that is the best approach.

The basic concepts of Object Oriented software design and development are covered well in a number of places. My general preference as the font of all knowledge is usually Wikipedia, and as it’s a software topic, they give Object Orientation a pretty good coverage.

Basically though, in object oriented software, everything is should be an “object”. And by that we mean the literal term for an object, in the sense of the fact a car is an object. That object should wrap up all the attributes of that object, such as it’s colour, and all the things you can do to it, such as start the engine.

In procedural programming, you would have a whole bunch of unrelated variables, such as $carColour and routines such as function startCarEngine() that lived somewhere in that spagetti of code you call a system. Object Orientation keeps things clean and tidy. You have your Car class, and you have a variable $myCar that represents a specific car. You can find out what colour that car is ($myCar->Colour) and you can start it ($myCar->StartEngine()) without worrying about the mess of variables.

But that’s just the start. What if you have different types of car. Old fashioned cars needed a crank handle to start them. But they still had wheels, tyres and so forth. In the procedural world, we’d have effectively duplicated code for old fashioned car starting and new modern car starting. The difference to the user is turning the key, or cranking the handle. But internally, much of the implementation may be the same (pump fuel, fire spark plugs etc).

That’s where object orientation gets useful. Objects can be built of other objects. This is called inheritance. Basically, you define a basic car class that defines all the features common to all cars. Then from that you inherit all that code, and extend it or replace it to provide an implementation of a type of car. Such as a Crank Handle Car or a Modern Car. From that you can inherit down and down until you have a class that defines a Ford Mondeo 1.8 Deisel, because it has details that are different to the Ford Mondeo 1.8i Petrol.

Indeed, your base car class, could have inherited common properties from the Vehical class. In some implementations of object orientation you can inherit multiple things. So it may have inherited “Internal Combustion”, “Wheeled Transport” and several other base classes.

You code once, and use or extend that code in many other places.

This is important, because it means you fix a root bug once and fix it everywhere.

OO is important for Multiblog, because Multiblog needs to post to many different web services. But, each web service will have certain things in common. At the very least, I’ll need the ability to make an HTTP POST operation to a URL. If I write my blog service implementations as classes, I can derive common functionality from a BlogService base class.

Furthermore, with Object Orientation there is a rich collection of Design Patterns. These are as close as software engineering gets to fixed, repeatable recipies for development. They give us ways to do things.

The typical object oriented patterns make it really simple to layout and engineer your software in such a way it will be easy to change, expand and manage your development. We’ll see that in action later on (I hope!)

Popularity: 26% [?]