Immigration judges and nuclear safety workers have been caught up in the push to slash the federal workforce, leading to fears over its impact.