Stewart Lee . . . offering a peep behind the curtains. Photograph: Jo Hale/Getty Images

No one under 30 will believe this, but stand-up comedy was once exciting and unpredictable. You could actually enjoy it sober. Like the punks before them, the alternative comedians who sprang up in the 1980s saw it as their duty and their pleasure to challenge the old farts who dominated both showbiz and the country. They were going to smash the system. Yes, even Ben Elton! True story!

Stewart Lee started on the circuit in 1989, just as the movement was losing its way. Five years before, as a Solihull schoolboy, he’d had his head turned by Ted Chippington, an obscure West Midlander with a scowl and a monotone delivery. Watching Chippington, Lee says, he realised that “Stand-up could be anything you wanted it to be. You didn’t even have to look as if you were enjoying it.”

Lee rarely did: he’s one of those few people who can look both smug and miserable at the same time. But for most of the 90s it seemed to outsiders that he was doing very nicely. His double act with Richard Herring spawned 10 radio and television series for the BBC. Alone, he played thousands of live dates. Yet, he says now, he was barely scraping by. He kept finding himself in front of audiences who wanted undemanding gags about soccer or Space Hoppers, not a man whose idea of a good time was to alienate his public and then, with luck, win it back. He stopped enjoying himself. In 2001, during a particularly painful gig in Liverpool, he realised he was “a mumbling relic from another age . . . Alternative comedy was dead. I was spent . . . So, quietly and without any fuss, I decided, then and there, to stop.” And, just like that, he did.

Nine years on, however, Lee is once more standing up and loving it. His latest BBC TV show, Stewart Lee’s Comedy Vehicle, has been commissioned for a second series, and next month sees him performing in 18 cities. How I Escaped My Certain Fate is the story of how he regained his taste for performance, and a paying audience. It’s a reminder that sometimes you need to work out what you do best and chuck out everything that gets in the way. In Lee’s case that included his former management company, Avalon.

The book is also a masterclass in how to appreciate comedy – at least Lee’s intellectual, sometimes wilfully inaccessible kind of comedy. “Within a few years,” he threatens, “‘jokes’, as we comedians call them, will have been entirely purged from my work in favour, exclusively, of grinding repetition, embarrassing silences and passive-agressive monotony.”

Although there are bits of introduction and conventional exposition – notably covering Lee’s involvement with the allegedly blasphemous Jerry Springer – The Opera, which he directed during his break from stand-up, the heart of the book is three live shows: Stand-Up Comedian, which began his comeback in 2004; the following year’s 90s Comedian, “a single hour-long shaggy-dog story about anal investigation, religious persecution and a blasphemous physical encounter with Jesus”; and 2007’s 41st Best Stand-up Ever, about the disparity between Lee’s supposed high status and his inability to earn a decent crust.

For each of these, Lee provides not only a full transcript but a parallel narrative in the form of exhaustive footnotes, the longest of which stretches to three and a half pages. Here he explains the workings of the jokes he hasn’t yet managed to eradicate, or revels in their failure. His basic philosophy is that the customer is always wrong. “It is a constant source of frustration to comics,” he writes, “that you, the public, are often inordinately thrilled by things that we do which are quite easy, and baffled or bored by the stuff we are proud of, or else assume that our finest moments are errors or accidents.” It’s no surprise to learn that he’s a fan of free jazz, with its “sheer bloody-mindedness” and its “take-it-or-leave-it attitude to critical and public approval”.

You don’t often see the inner workings of a comedian’s act, and in most cases that’s no loss. This time, however, it feels like a privilege to peep behind the curtains.