The Alternative Comedy Experience, yet another stand-up showcase for television, begins on Comedy Central tonight at 11pm. Featuring the kind of comics that other comedians stay out late to see, and an uncharacteristically high proportion of female acts, the show is intended as an Alternative to the increasingly formulaic content and presentation of stand-up on television. But, Comedy Central would like to point out, it can be enjoyed in addition to the usual fare, without spoiling your appetite for stand-up on TV generally.

I am a stand-up comedian, and I helped curate the choice of acts for the show, and inform its look and style. Before making The Alternative Comedy Experience, all of us involved in the programme thought long and hard about the history of the presentation of live comedy on television. Learning from the successes and mistakes of all that has gone before, we firmly believe that The Alternative Comedy Experience represents an apogee of TV stand-up. Here’s a summary of the lineage that led up to it. Warning – contains spoilers.

In the early days of ITV, back in the 1950s, most of its light entertainment was essentially outsourced to the showbiz agent and impresario Jack Hylton. Hylton provided the new channel, at a price, with screen adaptations of his own hit West End comedy variety shows and, in his new capacity moonlighting as adviser of light entertainment to the channel itself, he set up Jack Hylton Television Productions Ltd to fill Associated-Rediffusion’s schedules with dozens of poor-quality vehicles for comedians he already represented as an agent –  Arthur Askey, Flanagan and Allen, the Crazy Gang and Alfred Marks.

Their profiles were raised by his ability, as A-R’s light entertainment adviser, to place his turns on television. Hylton then put his agent/promoter hat back on and reaped further financial rewards from increased attendance at his own acts’ live shows, which he also promoted.  Hylton was relieved of his adviser to light entertainment role in 1959, and it has taken 50 years for the production of live comedy on television to come full circle.

The free market doctrine-driven insistence that broadcasters subcontract the actual business of making television to independent production companies – allowing agent-producers with their own commercial interests at heart to select which comedians get exposure on panel shows and stand-up showcases paid for by advertisers or licence-payers – has increased the amount, but narrowed the range, of stand-ups on television. The ideologically inspired land-grabs of 1980s ”alternative” comedy are now distant folk memory.

I’m 44. I don’t remember any of Jack Hylton’s shows, though I do remember one made by his nemesis Val Parnell, namely Val Parnell’s Sunday Night At The London Palladium. Parnell left his role as managing director of the Moss Theatres variety circuit in 1956 to meet the threat of the new technology of television head on, become MD of the fledgling Midland independent broadcaster ATV, and channel all his comedy and variety contacts into the weekly smash hit which dominated live comedy on TV from 1955 to 1974.

It’s astonishing to think that the promoter-producer of a variety show was once considered so significant that his name was in the title. The modern equivalent would be if one of today’s top TV promoter-producers had given his name to a TV show, despite not appearing in it himself, and we found ourselves watching a smash stand-up TV hit called Addison Creswell’s Comedy Roadshow, or Jon Thoday’s Good News. But unlike Hylton’s, Val Parnell’s programmes boasted professional production values and were held in high esteem, and the Sunday Night At The London Palladium format actually outlived the live variety circuit that detractors maintain it helped to kill, just as today’s comedy club promoters maintain that the prevalence of live stand-up on television is destroying their livelihoods now.

I think I remember watching Sunday Night At The London Palladium in its final, mid-70s, incarnation. But I’m not sure. All modern memories seem contaminated by YouTube. Do I remember watching Sunday Night At The London Palladium, with its mod-haired Jimmy Tarbuck and already cadaverous Bruce Forsyth appearing front-curtain to vast clapping crowds, or have I mentally misfiled the act of looking up clips on YouTube to confirm a childhood memory? I can’t be sure.

I am certain, however, that I do remember watching two other 70s stand-up shows that were to be a huge influence on the presentation of The Alternative Comedy Experience: The Comedians (Granada 1971-79) and The Wheeltappers’ and Shunters’ Social Club (Granada 1974-77).  Both brought to the screen a specifically working-class experience of live comedy that was already in its dying days.

It’s difficult for people brought up in the entirely class-free world of Blair’s Britain or Cameron’s Big Society to understand how socially segregated we used to be. I know I wouldn’t have watched The Comedians and Wheeltappers’ with my father’s more middle-class family, where both shows would have been viewed as “common”, so I must have seen them at my mother’s parents’ home, where ITV, a channel initially differentiated from the BBC by reflecting working-class tastes, as opposed to the Reithian notion of pushing improving viewing, was embraced. BBC2, in contrast, was forbidden, seen suspiciously by my grandmother as a hot-bed of liberalism and decadence. Explaining this childhood distinction to the right-wing commentator and BBC political analyst Andrew Neil last year he told me, confidentially and with a twinkle in his eye, that my grandmother had been right all along, and probably still was, and that he would very much like to have met her.

Wheeltappers’  saw some surprisingly imaginative, if often politically dubious, variety acts and club comics, introduced by liberal bête noir Bernard Manning, in a convincingly recreated studio set of the sort of working men’s club already disappearing in Britain, before a studio audience served trays of sloppy Watney’s by barmaids, wreathed in dense clouds of acrid fag smoke. It obviously made a deep impression on me and was exactly the kind of feel I wanted for my own BBC2 series Stewart Lee’s Comedy Vehicle, which ended up being filmed in one of the last remaining real working men’s clubs in London. Comics and crowd are in it together, the acts living and dying in front of a real audience, albeit one placed in a bizarre virtual recreation of their usual environment. In choosing to film our own Alternative Comedy Experience in the close confines of the 140-seater Stand in Edinburgh, we amplified Wheeltappers’ comedy vérité vibe quite self-consciously.

The content of The Comedians is largely best consigned to the toilet bowl of history. Each week a dozen or so bowtie-sporting comics of the mother-in-law-baiting school came out and told interchangeable jokes, all drawn from the same public-domain pub gag stockpile. A production assistant stood in the wings, literally crossing off the ones that had been done for the benefit of acts further up the bill. They were differentiated only by the extreme, and often brilliantly compelling, stylistic approaches of the respective performers, and weirdly interspersed with tepid white-guy trad jazz. The Comedians’ genius move was its editing, crosscutting small snatches of each act across each episode in fast-moving fillets that made the show a decade-long hit. Was there anything we could learn from this, now abandoned and discredited, approach?

For me, the series Dave Allen At Large and Dave Allen (BBC 1971-90) are the best stand-up comedy ever seen on television, and socio-studies students who would consign ’70s stand-up in its entirety to a black bin bag of racism and homophobia need to realise that Allen’s Beckettian monologues were unfolding in the same time-stream as Jim Davidson’s Chalky character. The languid Irish raconteur was allowed to stretch out on his stool with no more support than a Scotch and a cigarette, his slow-burning tall tales unspoiled by snappy edits or needless cutaways. The show respects its audience’s intelligence and attention span like no other stand-up show before or since. When I pitched Stewart Lee’s Comedy Vehicle to BBC2 in 2005 I said I wanted to copy it, and I did. But it’s not an approach that works for everyone, and the performers on The Alternative Comedy Experience were too disparate, stylistically, to benefit across the board from the imposition of Dave Allen At Large’s meditative model.

The existence of Dave Allen disturbs the convenient truth that we needed the punk-era alternative comedy movement to sweep into the late 70s and save us from unremitting awfulness, but it certainly felt that way to me as a barely teenage suburban comedy fan. The mere two editions of Boom Boom… Out Go The Lights (BBC2 1980-81), a low-tech showcase of the emerging alternative comedy scene made back in the days when a producer with a bright idea could follow it up immediately, were life-changing, and contained within them the DNA of the next 30 years of British stand-up. Brightly lit social workers drinking BBC orange squash on plastic chairs in a storage room were filmed failing to process the innovations of Rik and Ade’s Dangerous Brothers double act, Pauline Melville’s radical housewife, and the masked machine-gunman Keith Allen, all inexplicably interspersed with feelgood R&B from pre-punk blues rocker Paul Jones.

Had it been better supported, Boom Boom… Out Go The Lights could have solidified the sea change of alternative comedy ahead of schedule, but ground was lost to Chris Tarrant’s weirdly unresolved O.T.T. (Central 1982). In a small zoo-format studio, Tarrant let rip his inner rugby-club social sec, hosting both anti-Thatcher pop bands and communist Alexei Sayle, alongside dodgy race jokes from Bernard Manning and lots of already dated 1970s-style female nudity. During these sections the young black British comic Lenny Henry, and the Oxford fine art graduate Helen Atkinson Wood, wilted like whipped dogs.

Saturday Live (Channel 4 1985-88) was the live TV stand-up show that defined the 80s, and it remains trapped in amber, like a laughing prehistoric fly. Ben Elton bellowed from a disco nightclub stage over a floodlit sheep-pen of caned football casuals and gluey psychobillies, but his routines are among the most dated bits of the shows if you revisit them. Some of the guests are fantastic and we’re lucky they were documented, the finest being the snail-paced Irish surrealist Kevin McAleer, whose presentation of his minimalist slideshow of inexpressive owls is lent extra drama by Elton’s obvious dislike of it, and by the philistine studio floor manager visibly trying to force McAleer to speed up his perfectly paced performance. The bright lights and shouting of Saturday Live seem quaint now, and, foretelling the mass modern comedy phenomenon of failed irony, Harry Enfield’s Loadsamoney scored heavily with the cash-rich Thatcherites it set out to ridicule.

By the late 1990s, a mini stand-up comedy boom was evident in the form of a bottleneck of cheaply produced shows featuring box-fresh young acts like Noel Fielding, Lee Mack, Julian Barratt, Jason Byrne and Peter Kay, on the sort of brightly lit studio floors that rarely support an especially subtle performance. Gas (C4 1997-98) and Comedy Network (Paramount 1997-98) were the twin last gasps of straight stand-up on TV in the 20th century. It was soon presumed by executives to “not really work”. Those of us who participated in them found ourselves endlessly repeated for years on late-night cable, a gaggle of reverse Dorian Grays, condemned to a form of immortality under the terms of our one-payment buy-out deals, our virgin stand-up sets eventually chopped into differently titled shows, and sliced into 79p audio-only iTunes clips, for time immemorial. Comedy as commodity had arrived, and these shows typify the time when TV stand-up started to feel like worthless one-size-fits-all filler, soon to disappear from the schedules.

It fell to the mighty Live at the Apollo and Michael McIntyre’s Comedy Roadshow, both made by Open Mike Productions for BBC1, to revive the fortunes of stand-up on television, and to convince executives it was viable. All comedians owe them a huge debt of thanks for this. Live at the Apollo consolidated its audience in October 2008 after being drafted in to replace one of Open Mike’s other prime-time hits, Friday Night With Jonathan Ross, when it was sacrificed on the temporary altar of public outrage in the wake of Sachsgate. The apparently bulletproof format of both shows presents tight, short sets by club-hardened professionals, with little to spook the horses, on a status-conferring big stage, in front of a massive audience, peppered with celebrities who can be cut to, creating a diverting secondary narrative of famous face-spotting that plays alongside the comedians’ actual routines.

All major BBC1 stand-up shows, along with Channel 4’s Stand Up For The Week (C4, 2010-), are made by a production company called Open Mike, a sister company to the Off The Kerb comedians’ agency. Most BBC3 stand-up shows are made by Avalon productions, a subsidiary of the Avalon comedy management group, who launched my career in the early 90s. These shows have often leaned heavily, but by no means exclusively, on featuring acts their parent companies manage, ensuring their promotion wing a healthy return on subsequent live shows, with ticket sales boosted by TV appearances.

Eighty per cent of the acts on the last two series of Stand Up For The Week, 60 per cent of the comics on BBC3’s Live At The Electric, 39 per cent of acts on BBC3’s Russell Howard’s Good News, and 34 per cent of acts on Live At The Apollo were clients of the companies that produce the shows, in a situation weirdly reminiscent of that original 1950s Jack Hylton/Associated-Rediffusion model, but broadcasters are legally obliged to outsource programmes, and both agencies boast impressive rosters of TV-friendly acts. And later series of Live At The Apollo and both series of Comedy Roadshow varied the sources of their star turns so much as for accusations of nepotism to be all but negligible. But the comedy industry website Chortle used pie charts to show that production companies were 20 times more likely to book comics their parent companies represented, and that the two major stand-up comedy show producers, Off The Kerb’s Open Mike productions and Avalon, very rarely feature each other’s clients in their shows.

This lack of diversity is not especially sinister. Open Mike’s ideal stand-up is a 30-year-old man who dresses like a 40-year-old man, whereas Avalon’s is a 40-year-old man who dresses like a 20-old-man, and they have dominated their twin target markets of grown-up BBC1 and youth-orientated BBC3 respectively by deploying these tested models.  Sadly, Avalon are unable to find the regular high-profile outlets to expose the varied range of their comedy clients – from the stream-of-consciousness storyteller stand-up Maeve Higgins to the business, sports and morning television humorist Adrian Chiles – that Open Mike have. But other comedy agents, or comedians without any representation, can only look on enviously at their dominance of the spectrum. Inevitably, with two gatekeeper companies funneling in most of the stand-up that we see on our sets, the public’s perception of the art form is bound to be skewed towards certain stylistic norms.

And that’s where The Alternative Comedy Experience comes in. The show is not about consensus. In front of a small audience, the polar opposite from the vulgar displays of scale seen on most television stand-up shows, we see a selection of smart, strange, stupid and above all utterly original acts, from stand-up’s past and its present, dividing and delighting the room. There are no supportive cutaways to laughing audience members to tell you what you’re seeing is funny. The onus is on you, the viewer, to switch on, engage and stop expecting to be spoon-fed. There are few faces famous enough for you to feel you are watching a string of celebrity personal appearances. The Alternative Comedy Experience showcases a variety of styles of stand-up that explore, expand and experiment with the form. These are the comedians who do the research and development that has already, and will continue to, inform the stadium-fillers of tomorrow.

The Alternative Comedy Experience starts tonight at 11pm on Comedy Central (Sky 112, Virgin 132).