In the Story Grid, Shawn Coyne talks at length about the conventions and obligatory scenes in different genres. Writing Excuses Season 11 goes into this as well.

Every genre has conventions. “Specific requirements in terms of the story’s cast or methods in moving the plot forward” (Story Grid, 47). The crime thriller is going to have a crime committed, a detective to solve it, and a criminal to commit the crime. A romance is going to have two characters fall in love with each other. Those facts are the conventions.

Obligatory scenes are the specific way those conventions are carried out. For instance, in a romance novel, there’s a first kiss scene. In a hero’s journey there’s the darkest night scene.

The fact that genres and conventions have obligatory scenes doesn’t mean that every single darkest night is the same or every conventional character is the same. It’s the way authors take what’s expected, what’s required for a genre and change it to fit their story that makes the conventions and obligatory scenes work. That moment in Inside Out where Joy is stuck down in the pit sobbing over Riley’s memories works even though a darkest night has been done in literally every movie and story ever written before. But you couldn’t take that moment and put it in something even similar. It wouldn’t have worked in Wreck it Ralph for instance because his darkest night had to feature him wrecking something.