A US Court of Appeals for the Fifth Circuit panel questioned whether former members of a Louisiana library board had the ability to sue over their removal.