The New York University Stern School of Business professor, entrepreneur, bestselling author (he just recently published his latest book, Notes on Being a Man), and podcaster said he was a gifted student in third grade—sent to take math and English with fifth-graders. 

“[I was] supposedly smarter than everyone else at our level,” Galloway wrote in a memoir excerpt published by the Wall Street Journal on Tuesday. “I also was an all-star pitcher. There was promise.”

But following his parents’ divorce, his grades started slipping, and he gave up on sports. In high school, he earned mostly B’s and C’s and didn’t study for the SAT, he wrote, although he still applied to UCLA. He got an 1130 on the SAT, far below the range for top-tier universities, and was denied admission from UCLA (part of only 24% of applicants at the time who didn’t get in). After that, he took a job installing shelving, making $18 an hour. 

“I came home one day, and I was just so upset,” he said during a talk at the University of Chicago. “My whole life people had told me I was creative and I was smart and I was funny and the rest of it.”

That was until he and his mom discovered an appeals process to get into UCLA. He had an interview at UCLA with an admissions officer who told him he wasn’t qualified to go there, but accepted him anyway because he was a “native son of California.” Galloway was born in New York City, but was raised primarily in the L.A. area.

“Because they had a lot of seats relative to the number of applicants, they had the bandwidth to let in unremarkable kids,” Galloway said. “And this is a flex, and I’m going to make it.”

Now, Galloway is worth $150 million, and in the past five years has given back $16 million to UCLA, he said. 

“It’s worked out for all of us,” Galloway said of his unique admissions process. “No institution or individual can recognize or be the arbiter of greatness in an 18-year-old.” 

That experience has also influenced Galloway’s belief that higher education should be giving “as many kids a shot as possible.” 

At the same time, though, he’s said higher education has morphed from a public good into a luxury, scarcity-driven business that desperately needs a fundamental overhaul. Take UCLA, for example, which had a 70% acceptance rate when Galloway applied, but today has just a 9% admissions rate, he told the Brief But Spectacular interview series. 

“Unless you are in one of two cohorts—those being the children of rich people who can afford test prep, who have friends that can nudge the admissions committee or wealthy donors to give a second or third look to the application; or two, you’re part of a cohort that I would affectionately refer to as freakishly remarkable—you’re not getting the same opportunities as my generation had,” he said. 

And he has a point: Data shows it’s much more difficult to get into college today than it was even 20 years ago. For example, the acceptance rate for Harvard’s 1995 freshman class was 11.8%, according to KD College Prep. For this year’s incoming freshmen, the acceptance rate was just 4.2%.

“Higher education represents a dangerous trend in America where we’ve fallen out of love with the remarkables,” Galloway said. “We’ve decided our economy is a platform to turn the top 1% into billionaires. And that’s not what America is about.”