Jim Gaffigan Jokes Being a Dad Is Like Being a ‘Backup Quarterback’

Jim Gaffigan is getting real about his role as a father.

The comedian, 57, appeared on The Kelly Clarkson Show on Wednesday, May 1, and spoke with Kelly Clarkson, 42, about his family of seven. Gaffigan, who shares his five kids Marre, 19, Jack, 18, Katie, 14, Michael 12, and Patrick, 11, with wife Jeannie, chatted with Clarkson about his upcoming Mother’s Day plans.

“For some reason, Mother’s Day is held as this like hero moment,” Clarkson says.

“Yeah, I mean the mother is the hero,” Gaffigan says, as Clarkson quips, “Good answer.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories. 

“Well, some of it is, you know, I have this joke where I say being a dad is like being a backup quarterback in the NFL. You’re on the team but you’re not the star,” Gaffigan jokes. “And on those rare occasions that you’re brought into the game, people are nervous, you know?”

“But maybe you’re Purdy and you’re just sitting there waiting, and then it’s like a star moment,” Clarkson adds, referring to NFL quarterback Brock Purdy. “I think you’re the Purdy of fathers.”

“Oh well, I do feel like it’s the most important thing I will fail at, you know what I mean?” Gaffigan teases.

In March, Gaffigan spoke with PEOPLE about how he’s raising his five kids in New York City. “It was a huge mistake,” Gaffigan joked.

But the proud dad said that he thinks his kids enjoyed growing up in a big city while adding that a lot has changed. “I mean, also, it’s different, right? Because now there’s — I’d love to walk a block in New York City and not smell weed. You know what I mean?” Gaffigan said.

“When you’re in your twenties, you’re like, ‘Yeah, there should be no rules.’ And then when you’re a parent, you’re like, ‘Hey, maybe not every block.’ And of course, I live in the East Village, so I’m kind of asking for it,” he continued.