Dermot O’Leary skids in 20 minutes late, plants a kiss on both my cheeks and considers a fry-up. “I’m not sure I can handle a big breakfast,” he mutters, scouring the menu, scratching his head and ordering a cappuccino instead.

We are sitting in a restaurant in Highbury, around the corner from his North London home, a week before the opening of his Brighton restaurant Fishy Fishy in a handsome three-storey, Grade II listed building on the edge of the Lanes. According to O’Leary it was once home to Martha Gunn, the 19th-century “dipper” who used to assist female bathers in and out of the water from their modesty-preserving bathing machines. O’Leary’s girlfriend of seven years, Dee, a production assistant, and her best friend Alice have worked their magic on the interiors — forget-me-not-blue walls, retro mirrors, fish illustrations in chipped gilt frames and reclaimed chandeliers from the area’s junk shops. The diving helmet was O’Leary’s find. “I suppose you’d call the restaurant shabby chic,” he says.

For a man with such a busy working schedule, the X Factor host and Radio 2 presenter plans to be refreshingly hands-on with his new restaurant. “I’m looking forward to working front of house from time to time. This isn’t a vanity project. I’m not an idiot; I know there are so many other things I could put my money into,” he says, explaining that when he is not filming he will stay in Brighton with his Fishy Fishy business partners and friends, Paul Shovlin and James Ginzler.

Surely this is a first — a celebrity-owned restaurant where the celebrity intends to play a part in the business rather than just lording it on the best table. It turns out that O’Leary, 36, is a frustrated chef who has wanted to open a restaurant since he worked in one as a student. “It ticked all the boxes for me,” he says enthusiastically. “It has a similar adrenalin rush to live TV: you have to put on a show and you don’t get a second chance, everything is so now. And I love that comedown afterwards — you know, did those last five hours really happen?”

He nearly studied as a chef, too, after flunking his GCSEs. “I was never any good at school. I loved it but just didn’t apply myself and left with two GCSEs,” says the Essex boy, born to Irish parents, who was brought up in Colchester. “I had to make a decision to either train as a chef or resit my A levels.” He resat the exams and went on to do a degree in media and politics but the desire to be a chef never left him, so he continued part-time bar and catering work, even while he was a runner, until he got his first break on Channel 4’s T4.

O’Leary still doesn’t rule out becoming a chef one of these days. I can’t quite picture him practising turning vegetables with 16-year-olds at catering college, so maybe a course at Le Cordon Bleu is more his thing? “Possibly,” he agrees, revealing that he completed a butchery course last month at the meat supplier Ginger Pig, as well as a fish course at Billingsgate in London. “Retraining as a chef is still in my mind. I’m going to do it one day, maybe much later in life, though, hopefully, I still have a long career ahead of me in TV.”

As the interior suggests, Fishy Fishy is a modestly priced, casual seafood brasserie where the most expensive dish on the menu is the lemon sole at £16.50. The fish is normally landed at nearby Shoreham-by-Sea or sourced along the coast, from Looe to the Lizard. Smoked salmon is from “our new friends at The Springs, a family-run smokehouse just over the Downs”; oysters are organic Dorset Rocks, farmed in Poole Harbour, and there is Sussex-smoked mackerel pâté plus “Brightonbaisse” — a play on the classic French fish soup bouillabaisse, made with local fish. Everything on the menu is sourced as locally as possible, even the bubbly, which comes from the Ridgeview Estate in Sussex. Bream is O’Leary’s fish of the moment, while clam linguini is one of his top dishes. “I’ve also just bought a smoker, so I thought I’d play around with that,” he says enthusiastically. “I want to conquer how to make ceviche for my next project.”

Fishy Fishy intends to promote as much sustainable fish as possible. “We are committed to the promotion and improvement of responsible practices and ethical management of the sea. Through this we can contribute to the responsible management of fish stocks by demanding the fish we use cause minimum damage to the marine environment,” reads its website. So you won’t find any bluefin tuna on the menu, à la Nobu, but you will find beerbattered sustainable fish and chips at £9.95. Other dishes include herrings grilled with black pepper and lemon, and boiled Channel crab for £12.50.

The nearest competition is English’s, a few doors along (though rather staid by comparison), or Brighton’s old favourite the Regency, plus Riddle & Finns, which is pricier and where O’Leary found his head chef, Loz Tallant. “Brighton has pitifully few fish restaurants so there is plenty of room for more,” he says.

It turns out that when O’Leary is not hosting The X Factor or doing his Radio 2 show on a Saturday afternoon, he can often be found fishing off Brighton beach, which was the inspiration for the restaurant’s location — a project that has been two years in the making, begun well before any rumbles of recession. O’Leary is passionate about the state of our fishing industry — particularly fishing quotas. “Do you know that our fishermen have to throw 10 per cent back into the water, dead, because of these quotas? At least the EU now agrees that this is ridiculous,” he says. “Fishermen are always being portrayed as evil thugs but they are not the mercenaries they are made out to be — they are honest, hard-working folk,” he says earnestly. But he is soon grinning again when he returns to the topic of cooking fish and chefs such as Rick Stein and Valentine Warner. He says of the latter: “He’s gutsy and earthy and that’s what I like best about cooking — he’s a classic, old-fashioned greedy guts.”

O’Leary’s favourite restaurants are, predictably, fishfocused and include Scott’s and J Sheekey in London. He doesn’t do Michelin stars, it transpires. “Every time I go to that kind of restaurant I feel, ugh. I don’t like reduced sauces and I never finish the food — it’s too much. Though I do understand and appreciate the craft,” he concedes quickly. “Yes, it would be easier for me to do this in London — but where’s the challenge in that?” Indeed.

Fishy Fishy, 36 East Street, Brighton (01273 723750) www.fishyfishy.co.uk

Explore Food & Drink

• Gordon Ramsay

• Heston Blumenthal

• Recipes

• Wine

• Real Food

• Drinks

• Anjum Anand

• Eating Out

• Video Recipes

Times Recommends

• Top Ten most searched for recipes
• Chalet Chef 2010: the winning menus
• Gin and pomegranate jellies