THIS article is based on David’s A Point of View broadcast on BBC Radio 4 on June 28/30 which can be heard here.

It is hard to speak with any confidence about the mentalities, or levels of contentment, of people in earlier eras. But it may be that industrial society, especially in its most recent phase in the 20th century, including democratic equality and welfare states, was better at distributing self-respect and status than today’s post-industrial society, especially for men. We are richer and freer but often more adrift.

The movement from the country to the town in the early 19th century was often physically disruptive and involved, at least for a short period, an increase in suffering and reduced longevity. But life quite soon become significantly better than it had been in the countryside. Indeed, 1845, the year that Friedrich Engels published his description of industrial misery, The Condition of the Working Class in England, was just about the point when things started to improve.

Improved public hygiene and the arrival of cheap food from Australia and the Americas were just two ways in which the lives of most people were transformed for the better in the second half of the 19th century.

This was not just about incomes and living conditions; new forms of meaning and self-respect soon established themselves too. The urban identity itself quickly came to be seen as superior to the rural one, indeed the term peasant was soon established as a term of abuse, and there was no discernible movement back from the town to the country. Moreover, new forms of skilled and semi-skilled work in the factories and workshops conferred status and respect on both men and women.

Urbanisation was associated with education and betterment, as opposed to what Marx called the idiocy of rural life – it is no surprise that when the franchise was extended to the ordinary man (at this stage not the ordinary woman) it was the urban householder who was entrusted with the vote almost two decades before his rural cousin.

Moreover, for all of the external upheaval in the early decades of industrialisation, people’s inner life and moral universes often remained surprisingly unchanged. There was a significant amount of continuity in traditional understandings of the world and in social and gender roles.

Christianity remained the central organising belief system for most people, although it took new forms in the new urban centres in low-church Methodism and similar radical forms of Protestantism. Women began working outside the home in larger numbers, but family life appears to have been strengthened not weakened with the movement into towns, as rates of illegitimacy fell throughout the 19th century, at least in Britain.

It is in the nature of class stratified societies for status, meaning your place in the pecking order of social respect, to be a powerful given that few people can escape from. This restricts human opportunity but also gives people an explanation of their position in the hierarchy that has little bearing on their abilities or intelligence. In a relatively immobile class society, if you fail to rise from the working class into more genteel society it is no reflection on your own aptitudes, it is just the way things are.

It can also consolidate a sense of group solidarity and make a project of collective advance – such as trade unionism or socialism – seem more relevant than individual aspiration. Trapped in steerage together, the passengers will probably develop a sense of common interest. Given a ladder to the upper decks and people are more likely to elbow each other out of the way.

And this is what has happened in the post-industrial society. The main distributor of status and income is individual educational achievement and the ethos, if not always the reality, of the society is open and meritocratic. Those who do not enter into the cognitive class by taking A-levels at school (or graduating from high school in the US, taking the Bac in France or the Abitur in Germany) and going into higher education have, in the main, more restricted opportunities and lower social status than those who do. And this lower status is partly the result of their own cognitive limitations, or at least is widely perceived to be so by society and by many of the individuals themselves.

Some people have the resilience to avoid feelings of failure and may achieve a degree of success despite being in society’s slow lane but many others internalise a feeling of low self-respect which creates a pervasive sense of anxiety and uselessness.

There are several compounding problems that contribute to post-industrial disenchantment.

Post-industrial society has eroded the potentially compensating belief systems provided by religion, which recognises you for your character, not your ability, and sees you as equally valued in the eyes of God. It has done the same to the traditional roles in the private realm – such as male breadwinner or nurturing mother and wife – which may be restrictive but also confer meaning and purpose. The institutions that have historically accepted you as a member unconditionally – such as family, church, nation – are all weakened in a freer and more individualistic society. Achieved identities based on educational and career success have eclipsed ascribed identities based on attachment to place and group.

And post-industrial society elevates the values and priorities of the highly educated to a more dominant position and those views tend to be secular, individualistic, anti-tradition and anti-authority. Post-industrial modernity is often fluid and disorientating – more wealth, but less meaning, as the historian Yuval Noah Harari puts it – and is easier for some people to adapt to than others.

There is another problem. We now have, broadly speaking, a single route into a single dominant cognitive class. And although the graduate cognitive class is now much bigger and more democratic than it was 50 years ago it passes through a narrower funnel: go to a decent university, a Russell Group one if possible, and then into a professional middle-class career. Until quite recently there were many more and varied routes to a degree of achievement and respect. There was a working-class elite, for example, in the trade unions, friendly societies and Labour party, different regional elites in provincial cities and, at a work-place level, meaningful progression from below was possible without qualifications.

And, in any case, it looks increasingly likely in today’s economy that the cognitive class is going to shrink not grow. We will continue to need a thin layer of highly talented people who write the relevant software and provide the top professionals in everything from law to medicine, engineering, design, and finance. But the next round of AI is going to cut a swathe through the middle ranks of professional life and potentially create an even bigger status gulf between the most able and the rest.

Finally, the frame of comparison that people use to evaluate their lives has become much wider. People used to compare themselves in terms of wealth or talents of various kinds to people in their own village. Even after urbanisation people in a stratified class society would mainly compare themselves to people a few rungs above or below them on the ladder. For many, the shop steward, the supervisor or lay preacher was a figure of respect and an achievable model for emulation. With mass media, television and radio, and now social media it is harder for people to avoid comparing themselves unfavourably with the brightest, most beautiful and most talented in the world, which is thought to be one of the causes of rising mental stress especially among young women.

People seek meaning in their lives and a sense of being useful to others as well as material comfort, freedom and justice. Post-industrial society provides far more comfort and opportunity than industrial society but maybe the latter did better on meaning and self-respect. We cannot and do not want to go back to a past when social horizons and life chances were far more limited, but a recognition of some of the merits of earlier eras might help us to see more clearly the pathologies of today’s post-industrial achievement society.

If you appreciated this article, perhaps you might consider making a donation to The Conservative Woman. Our contributors and editors are unpaid but there are inevitable costs associated with running a website. We receive no independent funding and depend on our readers to help us, either with regular or one-off payments. You can donate here. Thank you.