100% true. Muppets are capable of doing this, as revealed in the recently-discovered pitch pilot for “Inner Tube” (a precursor to The Jim Henson Hour), which featured a Muppet named Crasher who jumps in and out of people’s TVs at random.
PUPPET COMEDY FACT:
During Walter's nightmare in "THE MUPPETS" (2011), various Muppets are shown exiting and entering Walter and Gary's TV set. Contrary to popular belief, no special effects were utilized. The Muppets can actually just do that to enter various homes.