Italian, Spanish and Portuguese derivative of rufus, a Latin nickname meaning "redhaired". The nickname became a family name and gave rise to Rufus as an English given name in the 19th century.
Thanks to Donald Whyte who provided much of the initial information for this site.