You heard it in dozens of movies in the 1930s and '40s. It stuck around longer than that, as I recall people saying in the 1960s. Jezebel looked into the history of the catchphrase here.
The saying emerged around 1828, when property ownership was removed as a prerequisite for suffrage, and voters needed only be free, white, and 21 (and also, it needn’t be said, male). It should have died with the passing of the 15th amendment in 1870, but of course racism is stronger than the law, and by the end of the century, legislators were working to bring the two back into harmony. In 1898, when Louisiana put forward its version of the grandfather clause, a judge asserted that the new legislation was simply a way of maintaining the “right of manhood,” deserved of all men “free, white, and twenty-one.”The movies, strangely, were mostly about how unmarried adult women would come to a bad end by asserting their freedom; they rarely even acknowledged the white privilege inherent in the phrase -because it was just understood. (via Nag on the Lake)
Yet it took women to popularize the phrase—or fictional women at least.