Anthony was born in Marylebone, London, England, to actors Michael Anthony (né Michael Adam Anthony Chodzko, born in Jersey, Channel Islands, of Polish and English descent) and Bernadette Milnes. The couple later divorced. Lysette was an only child. Her Paris-born paternal grandfather, Alexander Victor Chodzko, was a mariner and journalist of Polishdescent.