Tony's originally opened in a small shack of a building. In 1961, he expanded the dining room with a lower patio area with sliding glass windows. Customers immediately fell in love with sliding windows, which they frequently opened to feed the birds and take in the fresh ocean air. Speaking of 1961 and falling in love, it was also in 1961 that a beautiful lady showed up at Tony's and applied for a waitress job. Although the restaurant was well-staffed and not currently hiring, Tony felt compelled to hire her on the spot anyway. This turned out to be a good thing, since they got married three years later.

In 1963 he expanded yet again to include the Top o' Tony's Bar above the restaurant, which has a view from Palos Verdes to Santa Monica. Many famous people who have visited Tony's grace the walls of the spiral staircase leading up to the bar. Tony's is now a landmark.