O Labour, Where Art Thou?

After a landslide victory for the SNP, the loss of much of Labour's heartland and the near defeat of Labour leader Iain Gray, the red flag is at half mast in Scotland today. Where now for Scottish Labour? Can they ever really hope to recover from this? If so, over what timescale, and what will they have to accomplish in the meantime?

Labour's problem is a complex one, and the first thing they need to realise is that it is their problem. Their habit of putting the blame on other people has reached ridiculous proportions in the course of this election and the problems it causes are manifold. For one thing, it directly reduces people's sympathy for them – they are seen as bitchy rivals and sore losers. For another, it often puts them at odds with the popular perception of events to the extent that they seem deluded, which hardly inspires confidence in their policies (a shame as they have some good stuff there). Beyond this, it simply stops then from devising and implementing solutions.

Iain Gray's conciliatory speech the morning after went some way toward tackling this problem, albeit a little late. It suggests that there is room, even among senior Labour politicians, for a new line to be taken; but, of course, it has to be about more than talk. And the message has to be communicated throughout the party. Today there are still numerous labour activists insisting that the people of Scotland have made a terrible mistake and don't realise what they've done. They don't seem to realise that patronising the electorate like this played a significant part in costing them the election.

Alongside a shift in how they treat their opponents and the electorate, Labour need to change the way they treat themselves. Some of their people are complaining, now, about a press campaign against Iain Gray. Is that true? I'm not sure. When I wrote my own piece about Iain Gray it wasn't intended as an attack, it was intended as a warning. Seeing what was happening to the party I felt it necessary to speak out, like a small boy warning an emperor about the scantiness of his attire. It should have been obvious to Labour that Gray was a liability. Did they really fail to notice? Or is it simply that the pattern of cliques and alliances within the party made it impossible for them to do anything about it?

One of the difficulties for the Labour party is that its internal structure makes it easy for established cliques to dominate at both the local and national level. Despite the self-titled 'reformers' who have been at the helm over the past twenty five years, little has really been done about this. With the exception of those able to take advantage of positive discrimination lists, anybody wanting to move up in the party hierarchy has to win the support of a committee at every stage. It's at the grassroots stage that this is most problematic, because success in that context requires popularity within a small group of people; it requires being able to fit in socially in a very particular social context. This ensures that local parties keep on putting forward the same sorts of candidates generationally. A little further up in the hierarchy, where it's important to win wider approval, the party effectively selects for candidates skilled at flattery. Some of those who are merely talented inevitably fall by the wayside, but the bigger problem inherent in this system is that it breeds politicians who, for all their internal squabbles, are strongly inclined to flatter those in power and to root out dissent. There is a shortage of serious internal debate. It's too easy for those who can establish themselves socially to cling on even when they are obviously untalented or well past their sell by date.

This is not to pretend that power struggles within the party cannot be vicious, nor that there is a complete stifling of discussion (the Conservative party has far more problems of the latter sort), but it does mean that Labour tends to be slow to let go of poorly thought out policy. And policy development also suffers within this system, as it is formed through a series of committee decisions in which social priorities lead to overcooked, bland results.

Take the party's 2011 election manifesto. It promises a consultation on same sex marriage. On the face of it this may not seem like a big deal, but now let's put it in context. Any intent to consider same sex marriage is going to put off a certain set of voters. This needn't be a problem for a party if it wins over enough pro-lgbt voters to compensate. But by suggesting that equality should be up for debate, this stated policy puts off those voters too. It's a masterpiece of self-harm. Now consider that the actual plan behind the party is to consult not on whether or not same sex marriage should happen, but on how it should be structured, and the circle of fail is complete. The committee process has ensured that a potentially useful policy appeals to nobody.

This isn't to say that parties shouldn't operate internal democracy. Policies, however, need to be drawn up clearly by unbiased people who know what they're doing. They can then be submitted to a vote as finished products not to be compromised further. The same thing needs to happen with manifesto commitments and, more generally, with the party's image. Decisions need to be clear and they need to be informed. In other words, the party needs management. And that management must be free from the fear of dismissal for treading on people's toes.

In the immediate term, Labour can benefit from making stronger decisions about candidate selection that focus more on the preferences of the electorate and less on popularity within the party. But in the longer term, only structural changes will help it to produce the policy, the materials and the candidates necessary to ensure electoral success. And policy is an area where it needs to do still more. In this case it needs to understand that following the preferences of the electorate is not a solution. I met one Labour candidate, elected today, who told me “Sometimes we have to do things just because that's what the public want.” Um, no. That's never a good reason to do anything. Did your teacher never ask you at school, if your friends told you to jump out of a window, would you do it? (Many children's honest answer to this would be 'yes', but adults should know better.)

The policy that particular candidate was referring to was Labour's approach to knife crime, and it's another excellent example of how to do these things wrong. Knife crime in certain parts of Scotland is a really nasty problem, and it's understandable that Labour thought a) action was needed and b) action would be popular. Unfortunately, their hasty attempt to devise a solution was fraught with problems and was substantiated with really dodgy, misinterpreted statistics. As a consequence what ought to have been a vote winner became a liability.

Populism is by nature fickle and can easily lead an over-eager party into a trap. Pursuing it also makes the party look fickle. Yes, it is important to be seen to listen, but if you give the impression you'll do anything you're told, nobody will have any confidence in your ability to govern. You shouldn't be chasing other people's agendas – you should be setting your own. Take the initiative. Show strength, show confidence, frame the debate in your own terms. Just doing what you think other people want, like sniping at the policies of others, does nothing to substantiate your own identity. People will not vote for you if they can no longer believe in who you are.

As I write, John McTernan is on BBC Newsnight continuing to get it wrong. Labour needed to challenge the SNP on law and order, he says, missing the fact that a move in that direction inevitably placed Labour to the right of a party they were calling the Tartan Tories. His reasoning is that “The SNP will start to look weak” in this area. Um, sorry mate, but they've been in power for four years; if they were going to look weak on that, in the eyes of the electorate, they'd do so already. Ergo this was the wrong area in which to tackle them. Ought to be obvious...

So much about this mess ought to be obvious that it's really hard to figure out just how Labour managed to miss it. What matters now, though, is that the party's more clueful activists take control and get it sorted out. It's going to take a while to do the job and it won't be pleasant – 'cleansed with fire' is a phrase I've heard a lot from the activists themselves – but hard work and sacrifice now will be worth it in the long term. Labour still has some talented people. It still has a core identity worth fighting for. And it still has a connection with the people of Scotland that could, in time, see the red rose flower again.