The son of a Latvian cobbler, Rabin studied at the Guildhall School of Music and formed his first band while in his teens. He struck up a partnership with the guitarist Harry Davis and, in 1924, became co-leader of the Oscar Rabin and Harry Davis Orchestra. The band remained popular, playing major venues in London until the 1950's, Davis being the front man and Rabin providing the business acumen.